Perform an Auto-Submit Form Post from an Angular page

Keywords:
angular angular2 form post auto submit

Problem:
You need to send the user to another application (in the current view or new window) via a form-post that includes a payload of inputs.

How do you:

1. avoid the form being handled by the angular framework; and (for bonus points)
2. make the form dynamically auto-submit (to avoid the user having to hit a submit button)?

Solution:
For (a) the key to ensuring the form element in the template is not handled by the angular framework - and therefore allowing it to submit to the defined action is adding the ngNoForm attribute. There's notes in a discussion How to submit form to server in Angular2 ... at the time of writing, it doesn't appear to be well documented.

<form ngNoForm 
    #myFormPost name="myFormPost" 
    action="https://httpbin.org/post" 
    method="POST"
    target="_blank" >
    ...
  </form>

For (b) the dynamic auto-submit, you can obtain the reference to the form and submit it from the component code:

@ViewChild('myFormPost') myFormPost: ElementRef;
  ...
  // in some place where the form has been set
  this.myFormPost.nativeElement.submit();

A full working example is here: angular-form-post-auto-submit (on StackBlitz)
(this includes demo handling of hidden multi-line inputs)

Database schema naming for Vendor Neutrality

Keywords:
database vendor-agnostic identifier limitations Oracle SQL Server PostgreSQL HSQL MySQL table column

Problem:
It's surprisingly not a common topic of dicussion, but if you want to author database scripts in a way that is vendor-agnostic the consensus seems to be to conform to ANSI SQL. Are there any guides for how to name the schema identifiers to comply with the various vendor's (character limit) limitations?

Solution:
I couldn't find a good cross-vendor summary, so here's an attempt at looking at the important identifier limitations across some vendors.

Vendor	Identifier Limit (characters)			Notes
	TABLE	COLUMN	CONSTRAINT
Oracle	30	30	30	https://docs.oracle.com/database/121/SQLRF/sql_elements008.htm#SQLRF51129
PostgreSQL	63	63	63	http://www.postgresql.org/docs/current/interactive/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS
MS SQL Server	128	128	128	https://msdn.microsoft.com/en-us/library/ms175874.aspx
MySQL	64	64	64	https://dev.mysql.com/doc/refman/5.0/en/identifiers.html
HSQL	128	128	128	http://hsqldb.org/doc/2.0/guide/databaseobjects-chapt.html#dbc_table_creation
DB2	30 *	30 *	18 *	*Limit increased to 128 from v9.5 https://www-01.ibm.com/support/knowledgecenter/SSEPGG_9.5.0/com.ibm.db2.luw.wn.doc/doc/c0051391.html

If you are concerned with complying with all the above vendors, then the lowest common denominator here is Oracle at 30 characters for table/column & constraints.

Avoid using java.io.File.lastModified() for sorting Files

Keywords:
java File lastModified timestamp ext3 ext4 linux windows sort order milliseconds

Problem:
It seems like a basic use-case - order a list of files based on the order they were modified. File.lastModified() seems a reasonable choice for basing a Comparator on, based on the documentation:

A long value representing the time the file was last modified, measured in milliseconds since the epoch ...

So, assuming the files are not made any quicker than 1-per millisecond this should work fine right? No. For some reason, on linux based systems File.lastModified() is always rounded to the second. This appears to be a (legacy) limitation with 'ext3' (and earlier?) file-systems that persists to this day - even if the file-system is ext4 (with nanosecond precision). On a windows based system (with NTFS, 100ns precision) the File.lastModified() values are to the millisecond (I'm not the first person to notice the difference - File.lastModified() on windows vs linux).

Problems with sorting files on linux don't seem to be isolated to java "Order files by creation time to the millisecond in Bash" - though I suspect that thread is actually based on using an ext3 file-system, as on an ext4 file-system you can get this working with ls -latr --full-time.

Q: How do you know what type of file-system you're running?
A: df -T

Q: What precision will the file-system support?
A: See Comparison of file systems article and the 'Max Timestamp Resolution' column.

Q: If I'm still unsure, is there a test case I can try?
A: Here's a basic one:

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;

public class FileModified {
    public static void main(String[] args) throws IOException {
        File file = File.createTempFile("timestamp-test-", ".txt");
        FileWriter writer = new FileWriter(file);
        writer.append("updated");
        writer.close();
        
        System.out.println("via " + file.getClass().getName() 
                    + "\n\t" + file.getAbsolutePath() + "\n\t\tmodified: " + file.lastModified() + "ms");        
    }
}

If you establish that the target environment(s) for your java application are on file-systems that do support a higher than 1 second precision how do you get access to this from java?


Solution:
The closest bug report I can find is JDK-6939260. But it appears that the bug title has been changed to propose exposing higher than millisecond precision for files - the side note made in the comments states "it's not possible to support this with java.io.File because it specifies that the last modified time is returned in milliseconds". This missed the point in the original bug description stating "The end of the number is always 000" - ie parking micro/nano second precision, shouldn't the method provide at least millisecond precision if the underlying filesystem supports this (or higher)?

It appears for new features (and bug fixes?) related to the capabilites of the file-system, this functionality is going to be implemented in java.nio.file. So if you're using java 1.8+ you may be able access more accurate timestamps via Files.getLastModifiedTime(Path,...). The updated test case becomes:

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.FileTime;
import java.util.concurrent.TimeUnit;

public class FileModified {
    public static void main(String[] args) throws IOException {
        File file = File.createTempFile("timestamp-test-", ".txt");
        FileWriter writer = new FileWriter(file);
        writer.append("updated");
        writer.close();
        
        System.out.println("via " + file.getClass().getName() 
                           + "\n\t" + file.getAbsolutePath() + "\n\t\tmodified: " + file.lastModified() + "ms");
        
        Path filePath = FileSystems.getDefault().getPath(file.getAbsolutePath());
        FileTime modified = Files.getLastModifiedTime(filePath);
        System.out.println("via " + modified.getClass().getName() + "\n\t" + filePath 
                           + "\n\t\tmodified: " + modified.toMillis() + "ms (" + modified.to(TimeUnit.NANOSECONDS) + "ns)");
    }
}

Note that although java.nio.file.attribute.FileTime exists in java 1.7, JDK-6939260 claims that it's only going to be fixed in 8 onwards (I can't tell which update - I tried the latest available and it still fails).

If you need to support java 1.7 or below on linux there is no staightforward option (short of parsing ls --full-time command output). If your system is in control of the files being written, my suggestion would be to maintain a separate record of creation/modified time - assuming order by time is important and you need higher than per-second precision.

Quick Start to tracing JDBC SQL operations using log4jdbc

Keywords:
database logging show SQL log4jdbc tomcat JNDI log4j trace performance timing

Problem:
Given a complex web application (with many moving parts), there's performance issues that first impressions seem to point at environment-specific database latency. Is there any way to trace/profile/monitor all the SQL statements being made to the database (preferably with timings)?

Solution:
log4jdbc seamlessly lets you configure a logger between your application and the JDBC connection. Documentation on the log4jdbc site is very good and has just about all you need to know, below is a "quick start" for the impatient.

Step 1: Jar File(s)

Drop the log4jdbc4-[version].jar file into [webapp]/WEB-INF/lib.
You'll also need:

• a supported logging framework (log4j: log4j-[version].jar)
• the SLF4J API (slf4j-api-[version].jar - already had it)
• the SLF4J jar to use this logging framework (slf4j-log4j12-[version].jar - already had it)

Step 2: Driver Class and URL

1. Change the database resource definition to use the driver class net.sf.log4jdbc.DriverSpy ('spy' as it will work out the correct 'real' driver class to use).
   
2. Simply prepend jdbc:log4 to the existing URL.

For a PostgreSQL database connection configured as a JNDI/JDBC Resource in Tomcat, this will look like:

    <Resource
            name="jdbc/example"
            type="javax.sql.DataSource"
            factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"
            driverClassName="net.sf.log4jdbc.DriverSpy"
            url="jdbc:log4jdbc:postgresql://localhost:5432/example"
            username="postgres" password="***" 
            maxActive="20" maxIdle="10" maxWait="-1" 
            removeAbandoned="true" removeAbandonedTimeout="120" logAbandoned="true"
            auth="Container"
            charset="UTF-8" />

Step 3: Logging Config

There's detailed notes on the five key loggers and an example log4j.properties is available. But if you just want to see SQL + timings, the following will log this all to console:

log4j.logger.jdbc.audit=FATAL,Log4JDBC
log4j.additivity.jdbc.audit=false

log4j.logger.jdbc.resultset=FATAL,Log4JDBC
log4j.additivity.jdbc.resultset=false

log4j.logger.jdbc.sqlonly=FATAL,Log4JDBC
log4j.additivity.jdbc.sqlonly=false

log4j.logger.jdbc.sqltiming=INFO,Log4JDBC
log4j.additivity.jdbc.sqltiming=false

log4j.logger.jdbc.connection=FATAL,Log4JDBC
log4j.additivity.jdbc.connection=false

log4j.appender.Log4JDBC=org.apache.log4j.ConsoleAppender
log4j.appender.Log4JDBC.layout=org.apache.log4j.PatternLayout
log4j.appender.Log4JDBC.layout.ConversionPattern=%-5p [%d{DATE} %c]: %m%n

That's it. Start up the application and it should function exactly as it did before, but all SQL operations (made via the JNDI connection pool resource in my example) will be logged in the form:

INFO  [01 Nov 2013 12:00:00,000 jdbc.sqltiming]: SELECT NAME FROM PERSON WHERE ID=54321
{executed in 1 msec}

Notes:

Bonus Step: Trace back to the Application Code

If your issue is working what part of the application code triggered the execution of a given SQL statement, a neat feature of the log4jdbc loggers is including the class & method-name of the code that invoked the JDBC driver. As there may be a few layers between your code and the JDBC driver (hibernate > application server connection pool, etc) you can tell it what level you what to capture from.

1. change log level to DEBUG
log4j.logger.jdbc.sqltiming=DEBUG,Log4JDBC
   
2. set the system property log4jdbc.debug.stack.prefix
-Dlog4jdbc.debug.stack.prefix=com.example
   

Log statements will then be in the form:

DEBUG [01 Nov 2013 12:30:00,000 jdbc.sqltiming]:  com.example.PersonDAO.findPerson(PersonDAO.java:111)
2. SELECT NAME FROM PERSON WHERE ID=54321
{executed in 53 msec}

NB: this extra logging adds a noticeable overhead to the database usage (and timings).

Querying JSON using JSONPath

Keywords:
JSON query filter evaluate JSONPath expression

Problem:
Dealing with code that's doing a fair bit of JSON processing (traversing object structures and collating the values) it leads to the question - is there a standard way to lookup or filter data from the JSON structure using a query or expression?

The same question has been already been asked on StackOverflow: Is there a query language for JSON? and the answer is essentially that there are many approaches. There's another list of approaches with examples in 8 ways to query json structures.

So if there's no 'standard' approach (for example, backed by an RFC as is the case with JSON), which approach should I pick?


Solution:
JSONPath proposed by Stefan Goessner allows for XPath-like expressions to be evaluated against JSON. What's possible with the expression syntax seems to cover most of the traversal and collating scenarios I was interested in and the JavaScript implementation is remarkably lightweight (implemented as just one function jsonPath(<object>, <expression>, <optional arguments object>)).

Here some extra expression examples (beyond those shown in the JSONPath articles and jsonpath on Google Code):

Expression	Result
$	The root element (the same JSON back out - i.e. an identity transform).
$.*	All child elements of the root.
$.store.book	All book elements.
$.store.book[1]	The second book element.
$..price	The price of every element - result will be Numbers.
$.store.book[?(@.isbn=='0-553-21311-3')]	The book element where isbn is '0-553-21311-3'.
$.store.book[?(@.category=='fiction' && @.price > 10)]	The book element where category is 'fiction' and price is more than 10.

It seems like there's still some discussion around what's to be supported and there isn't exactly high activity on the project. So if using it, treat is as beta and evaluate if it does what you need. It's likely to be one of those things that once it gets past v0 there will be less (if any) changes to expect.

Some things to look out for:

• Where's the latest version?
  Although the downloads page says the latest version is 0.8.0, this few years old (2007). You can access later versions (of the JavaScript source) either via the SVN path to the trunk or via the Google Code Browser. The javascript implementation lists the version as 0.8.5 - I'm not sure what the cycle is for getting this to 'release'.
• No results returned as false
  I'd expect this to be simply an empty array [].
• Results as Arrays of Arrays
  If querying a nested JSON structure of objects and the matching results match different parts of the tree, what you get back is a nested array structure indicating the relative location of the matches. I'd prefer to get this back as a flattened array. I guess the logic is that have the context of where the items were found and if you want, you can post-process (and flatten) the Arrays of Arrays - some useful approaches discussed on StackOverflow:Merge/flatten an Array of Arrays in JavaScript?.
• the Object and it's properties in the results?
  In using the latest code mentioned above, it seems that - in some scenarios - where an object matches the expression the result includes the object as well as the properties as additional items in the result. For example:
  

  Instead of:	You get:
  [ { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99 } ]	[ { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99 }, "fiction", "Herman Melville", "Moby Dick", "0-553-21311-3", 8.99 ]

  This wasn't the case with 0.8.0 and may change in future releases?

Stop Chrome from Prerending URLs

Keywords:
Google Chrome prefetch POST prerender random timeout speculative sockets preconnections browser learning predict network actions

Problem:
Chrome has become my browser of choice - it seems faster and is less reliant on the disk which is critical if you're on machine that's continually doing disk-intensive work (such as builds).

That speed can come at a cost. Perhaps it was an update or perhaps because of the repetition of certain actions in a web-wizard style application in development but all of a sudden using Chrome was giving random unexpected results that weren't reproducible in other browsers.

Tracing the server-side, it seems that URLs are being called - with GET requests - independently of my actions in the browser. These phantom requests include my credentials (HTTP BASIC auth) and session information (as cookies), but on the client-side (tracking Network in Developer Mode) there's no explanation for how/why/when these URLs are being hit.

Is my browser possessed?

Solution:
Short answer? YES, yes it is (kind of) possessed! What I'm seeing is Chrome predicting my next action in the web-application and hitting those URLs (on my behalf) in the interests of speed. It was initially hard to find good references for this because it seems to be referenced with various terms. Officially, it's "Prerender" and the Google Whitepaper "Web Developer's Guide to Prerendering in Chrome" describes it as:

Prerendering extends the concept of prefetching. Instead of just downloading the top-level resource, it does all of the work necessary to show the page to the user—without actually showing it until the user clicks. Prerendering behaves similarly to if a user middle-clicked on a link on a page (opening it in a background tab) and then later switched to that tab. However, in prerendering, that "background tab" is totally hidden from the user, and when the user clicks, its contents are seamlessly swapped into the same tab the user was viewing. From the user’s perspective, the page simply loads much faster than before.

As mentioned in the quote, it's separate to "Prefetching" - which is the mechanism where a page can explicitly flag a URL as safe/recommended to begin fetching.

If this was a chess program, it's like the browser is thinking a few moves ahead but instead of just anticipating moves, it's actually making them.

As raised (and debated) in Chromium Issue 85229 - this time under the term "speculative sockets" - there's no way for the server/application to opt-out or ask it not to do this.

You can view diagnostics for what URLs are being Prerendered by opening the following in a tab (if you're experiencing erroneous/random behaviour it can make for interesting reading):

chrome://net-internals/#prerender

Shouldn't GET requests be safe? As discussed in On Chrome and URL Pre-fetching, yes but surely not always 100% safe. If you have an app that has URLs that only ever get POST requests there's nothing stopping Chrome from hitting these with Prerender GETs - and if the URLs (for convenience) handle the GET requests you could be trouble. Which leads to some inconsistencies in the above mentioned whitepaper for "Situations in which prerendering is aborted" ("...Chrome may run into a situation that could potentially lead to user-visible behavior that is incorrect"):

• POST, PUT, and DELETE XMLHTTPRequests
  False: although in your usage of an application/site a URL may only ever be invoked via POST, you can't stop it attempting the same URL with a GET, just minus the parameters.
• Developer Tools is open
  False: In my usage anyway, using the Developer tools (trying to track Network requests) will not stop the Prerender requests being made.

Which (finally) leads to the solution - turn off Prerending. The whitepaper has a section titled "Ensuring you have prerender turned on" (despite it being enabled by default). So disabling it involves following the reverse:

1. Click the 'three bars' icon in the top right of Chrome.
2. Click 'Settings'
3. Click 'Show advanced settings...'
4. Under 'Privacy'
   • un-check the option 'Predict network actions to improve page load performance'
   • un-check 'Use a prediction service to help complete searches and URLs typed in the address bar'
     (this specifically relates to prefetch/prerender of URLs typed into the 'Omnibox' address bar - Chrome will actually be calling up the suggested sites in the background if they happen to be URLs you commonly select)
5. Open a new tab and paste in "chrome://net-internals/#prerender"
   Confirm that it lists:
   • Prerender Enabled: false
   • Prerender Omnibox Enabled: false

Notes:
There is some confusion about the requests being made for Prerender and reports of Chrome making multiple (and unnecessary) background requests for the favicon - e.g. Issue 39402. At the server-side it will be clear that Prerender requests are for URLs that you tend to access, before you actually access them rather than URLs ending in /favicon.ico. There doesn't seem to be a way to disable favicon 'polling' but the above mentioned issue seems to indicate improvements were being considered to reduce the server load.

Capture console output in ant - for any command or task - using Record

Keywords:
ant capture task console output stdout stderr javac jspc "Problems opening file using a recorder entry" absolute path

Problem:
Many tasks include options to capture output. Each of them different, such as exec - with the redirector option; java - with the @output attribute. What if you want to capture the output (stderr and stdout) from a task that doesn't support output capture (such as javac) or you'd like more control - start|stop|start appending output from many commands to shared file(s)?

Solution:
Record is the solution. Just BEWARE the @name attribute is treated as plain file path rather than a File attribute as in many other ant tasks. This means if supplied with a relative path it will be relative to where you're running ant from - a problem if you have ant scripts invoking other ant scripts. When path issues occur, you'll get this error:

Problems opening file using a recorder entry

Run ant with the -v option to get the full stack trace of the IOException. You'll see what path it's attempted - and failed - to use.

ALWAYS use absolute paths and you'll (hopefully) be fine.

To start recording:

<!-- (1) guarantee that you have an absolute path to the log file by setting a property with @location as the relative reference -->
<property name="build.logpath" location="${a.build.location}/sub_folder/log_file.log"/>
<!-- (2) start recording -->
<record action="start" name="${build.logpath}" loglevel="verbose"/>

All tasks that follow will have the console output (and logging if you set the loglevel attribute) go to the capture file:

<jasper2 ... />
<javac ... />
etc.

Then to stop:

<record action="stop" name="${build.logpath}"/>

Easy as that.

Using CXF in Containers with JAX-WS handling - WebSphere/WebLogic

Keywords:
cxf jax-ws websphere weblogic annotation DisableIBMJAXWSEngine Ignore-Scanning-Packages Ignore-Scanning-Archives prefer-web-inf-classes prefer-application-packages prefer-application-resources

Problem:
So CXF is your chosen JAX-WS framework for your application - perhaps because you want your appplication to work the same way in every servlet container - tomcat included - or because you can't avoid the need to reference the implementation rather than just the pure JAX-WS spec (access to the http-session for example). While these reasons seem valid they seem to have been considered as an afterthought in containers with built in JAX-WS handling - such as WebSphere and WebLogic.

So you follow the CXF notes and perhaps blog/mail-list posts and either are in the state where: (a) the application is not working or (b) is working but you're not sure how or why. What steps can you perform to guarantee successful deployment, and (if possible) can you understand the context for them - so you can decide if they're needed for example.

Solution:
The following are useful references:

• CXF Notes - Application Server Specific Configuration Guide
• WebSphere - Using a third-party JAX-WS Web services engine
• WebSphere - Reducing annotation searches during application deployment
• WebLogic - Deployment Descriptor Reference

Normally for these container/class-loader issues you can get the desired behaviour by simply getting the container to load the application's libraries first (parent last). The complication in this case is the annotation processing, which seems (in my testing at least) can happen independently of the annotation processing implementation - particularly for the association of @Resource references.

To get past this, there's essentially three areas to cover:

1. Supply JAX-WS annotation processing libraries (geronimo) that will override the container defaults - this includes all libraries that the CXF framework and annotation processing require - because nothing can be used from the container (parent)
2. Tell the container you're handling annotations - explicitly
3. Setup Parent-Last Class-loading - get the container to use your applications libraries before its own

Step 1: Supply JAX-WS annotation processing libraries and dependencies

There's a longer list of libraries in the CXF Notes but many of these are not specifically essential to the JAX-WS issue (the latest JAXB libraries for example will be required by CXF in a tomcat deployment). These are the libraries that appear to be need in addition to those that would have otherwise been included with the application:

• geronimo-annotation_1.0_spec-1.1.1.jar - Annotation Processing
• geronimo-jaxws_2.2_spec-1.1.jar - Runtime Override
• geronimo-stax-api_1.0_spec-1.0.1.jar - Runtime Override
• geronimo-ws-metadata_2.0_spec-1.1.3.jar - Annotation Processing
• stax2-api-3.1.1.jar - Runtime Override
• woodstox-core-asl-4.1.1.jar - Library Requirement

geronimo and woodstox/stax2 are the same Apache license as CXF and these libraries are supplied as part of the CXF distribution. In WebSphere you'll know it's taken effect as in deployment you'll see:

[19/10/12 8:34:46:641 EST] 00000044 AbstractInjec W   CWNEN0070W: The javax.annotation.Resource annotation class will not be recognized because it was loaded from the 
    file:/E:/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/SERVERNode01Cell/example-app.ear/lib/geronimo-annotation_1.0_spec-1.1.1.jar location rather than from a product class loader.
[19/10/12 8:34:46:645 EST] 00000044 AbstractInjec W   CWNEN0070W: The javax.xml.ws.WebServiceRef annotation class will not be recognized because it was loaded from the 
    file:/E:/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/SERVERNode01Cell/example-app.ear/lib/geronimo-jaxws_2.2_spec-1.1.jar location rather than from a product class loader.

The "Runtime Override" libraries listed above are essential as the overridden annotation processing code can not load certain classes from the parent - this may include classes from javax.xml.*. These 'parent prevention' issues are most likely going to be reported as java.lang.VerifyError. In WebSphere for example you'll encounter 'parent prevention' issues as:

Caused by: java.lang.VerifyError: JVMVRFY013 class loading constraint violated; 
    class=org/apache/cxf/jaxws/context/WebServiceContextImpl,
    method=getEndpointReference([Lorg/w3c/dom/Element;)Ljavax/xml/ws/EndpointReference;,
    pc=0
    at java.lang.J9VMInternals.verifyImpl(Native Method)
    at java.lang.J9VMInternals.verify(J9VMInternals.java:85)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:162)
    at org.apache.cxf.jaxws.context.WebServiceContextResourceResolver.resolve(WebServiceContextResourceResolver.java:61)

Step 2: Tell the container you're handling annotations

This seems the strangest part but both WebSphere and WebLogic will continue to report errors - in particular about the @Resource annotation. For example, on WebSphere:

[19/10/12 17:11:03:261 EST] 00000053 webapp        E com.ibm.ws.webcontainer.webapp.WebAppImpl populateJavaNameSpace SRVE8084E: An unexpected internal server error occurred while populating the namespace.
    com.ibm.wsspi.injectionengine.InjectionException: CWNEN0044E: A resource reference binding could not be found for the following resource references [org.example.hello.HelloSoapService/context], defined for the example-app component.
    at com.ibm.wsspi.injectionengine.InjectionProcessor.resolveInjectionBindings(InjectionProcessor.java:1208)

For example, on WebLogic:

<24/10/2012 2:17:27 PM EST> <Error> <J2EE> <BEA-160223> 
    <The resource-env-ref 'org.example.hello.HelloSoapService/context' declared in the standard descriptor or annotation has no JNDI name mapped to it. 
    The resource-env-ref must be mapped to a JNDI name using the resource-env-description element of the weblogic proprietary descriptor or corresponding annotation.>

For WebSphere - Disable and Ignore Annotation Scanning

There are ways to configure this globally - in the application server settings and configuration. I'll only note here the application-specific approach. This involves setting attributes in the META-INF/MANIFEST.MF of the war file.

1. DisableIBMJAXWSEngine: true
2. Ignore-Scanning-Packages: comma-separated list of packages where there are service implementations - and use of the @Resource and @WebService annotations
3. Ignore-Scanning-Archives: comma-separated list of jar-file libraries where there are service implementations - and use of the @Resource and @WebService annotations

You don't necessarily have to do both 2 & 3 - use 2 if implementations are in (war-app)/WEB-INF/classes for example. It does seem setting the Ignore-Scanning-Archives setting can significantly speed application deployment - and is harmless if there is no annotation processing required. Hand-coding MANIFEST files is risky - I'd recommend using an ant task (and define all three attributes):

<target name="dist" description="make the war and ear">
        <!-- list of packages with WS implementations - which should be ignored by container annotation processing -->
        <property name="service.packages">
            org.example.hello,
            org.example.goodbye
        </property>
        <loadresource property="service.packages.delim">
            <propertyresource name="service.packages"/>
            <filterchain>
                <tabstospaces/>
                <deletecharacters chars=" "/>
                <striplinebreaks/>
            </filterchain>
        </loadresource>    
        <path id="webapp.archives">
            <fileset dir="./example-app/WEB-INF/lib">
                <include name="**/*.jar"/>
            </fileset>
        </path>
        <pathconvert property="webapp.archives.delim" refid="webapp.archives" pathsep="," dirsep="/">
            <map from="${basedir}/example-app/WEB-INF/lib/" to=''/>
        </pathconvert>
        
     <!-- Define META-INF attributes -->  
        <manifest file="./example-app/META-INF/MANIFEST.MF" mode="update" flattenAttributes="true">
            <attribute name="DisableIBMJAXWSEngine" value="true"/>
            <attribute name="Ignore-Scanning-Packages" value="${service.packages.delim}"/>
            <attribute name="Ignore-Scanning-Archives" value="${webapp.archives.delim}"/>
        </manifest>
        <copy todir="./example-app/WEB-INF/lib">
            <fileset dir="./runtime">
                <include name="**/*.*"/>
            </fileset>
        </copy>
            
        <jar destfile="example-app.war" basedir="./example-app/"
            manifest="./example-app/META-INF/MANIFEST.MF"/>       
        <ear destfile="example-app.ear" appxml="metadata/application.xml">
            <fileset dir="." includes="example-app.war"/>
            <metainf dir="metadata" includes="*.*" excludes="application.xml"/>
        </ear>
    </target>

For WebLogic - Prefer Packages and Resources

In the (ear-app)/META-INF/weblogic-application.xml you must explicitly preference the packages supplied as part of the CXF solution and the service resources these libraries include:

<?xml version="1.0" encoding="UTF-8"?>
<weblogic-application xmlns="http://www.bea.com/ns/weblogic/90">
    <xml>
        <parser-factory>
            <saxparser-factory>org.apache.xerces.jaxp.SAXParserFactoryImpl</saxparser-factory>
            <document-builder-factory>org.apache.xerces.jaxp.DocumentBuilderFactoryImpl</document-builder-factory>
            <transformer-factory>org.apache.xalan.processor.TransformerFactoryImpl</transformer-factory>
        </parser-factory>
    </xml>
    <application-param>
        <param-name>webapp.encoding.default</param-name>
        <param-value>UTF-8</param-value>
    </application-param>
    <prefer-application-packages>
        <!-- // for logging  --> 
        <package-name>org.apache.log4j.*</package-name> 
        <!-- // for jaxb  --> 
        <package-name>com.sun.xml.*</package-name> 
        <!-- // for apache commons lang/io  --> 
        <package-name>org.apache.commons.*</package-name> 
  <!-- // for spring/hibernate --> 
        <package-name>antlr.*</package-name> 
        <package-name>org.springframework.*</package-name>
        <!-- // for jstl -->
        <package-name>javax.servlet.jsp.jstl.*</package-name>
        <!-- // for jax-ws -->
        <package-name>javax.jws.*</package-name>
        <package-name>javax.ws.*</package-name>
  <!-- // xml processing -->
        <package-name>javax.xml.*</package-name>
        <package-name>javax.xml.stream.*</package-name>
        <package-name>org.xml.sax.*</package-name>
        <package-name>org.w3c.*</package-name>
        <package-name>org.apache.xmlcommons.*</package-name>
        <package-name>org.apache.xml.serializer.*</package-name>
        <package-name>org.apache.xerces.*</package-name>
        <package-name>org.apache.xalan.*</package-name>
        <package-name>com.ctc.wstx.*</package-name>
        <package-name>org.codehaus.*</package-name>        
    </prefer-application-packages>
    <prefer-application-resources> 
        <resource-name>META-INF/services/javax.ws.rs.ext.RuntimeDelegate</resource-name> 
        <resource-name>META-INF/services/javax.xml.bind.JAXBContext</resource-name> 
        <resource-name>META-INF/services/javax.xml.datatype.DatatypeFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.parsers.DocumentBuilderFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.parsers.SAXParserFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.stream.XMLEventFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.stream.XMLInputFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.stream.XMLOutputFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.transform.TransformerFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.validation.SchemaFactory</resource-name> 
        <resource-name>META-INF/services/javax.xml.ws.spi.Provider</resource-name> 
        <resource-name>META-INF/services/javax.xml.xpath.XPathFactory</resource-name> 
        <resource-name>META-INF/services/org.apache.cxf.bus.factory</resource-name> 
        <resource-name>META-INF/services/org.apache.xalan.extensions.bsf.BSFManager</resource-name> 
        <resource-name>META-INF/services/org.apache.xml.dtm.DTMManager</resource-name>
        <resource-name>META-INF/services/org.codehaus.stax2.validation.XMLValidationSchemaFactory.dtd</resource-name>
        <resource-name>META-INF/services/org.codehaus.stax2.validation.XMLValidationSchemaFactory.relaxng</resource-name>
        <resource-name>META-INF/services/org.codehaus.stax2.validation.XMLValidationSchemaFactory.w3c</resource-name>
        <resource-name>META-INF/services/org.osgi.framework.launch.FrameworkFactory</resource-name> 
        <resource-name>META-INF/services/org.relaxng.datatype.DatatypeLibraryFactory</resource-name> 
        <resource-name>META-INF/services/org.w3c.dom.DOMImplementationSourceList</resource-name> 
        <resource-name>META-INF/services/org.xml.sax.driver</resource-name> 
        <!-- // geronimo (at present) has no service such declaration (glassfish and others do) - include for future reference -->
        <resource-name>META-INF/services/com.sun.xml.ws.spi.db.BindingContextFactory</resource-name>
    </prefer-application-resources>    
</weblogic-application>

Step 3: Setup Parent-Last Class-loading

This is probably the easiest step.

For WebSphere - set flags during deployment

Note that this should be set at the ear and war (web module) level.

For WebLogic - set prefer-web-inf-classes

This doesn't seem to be a comprehensive setting based on what is required in the weblogic-application.xml file, but in the (war-app)/WEB-INF/weblogic.xml file, ensure prefer-web-inf-classes is set to true:

<?xml version="1.0" encoding="UTF-8"?>
<weblogic-web-app 
 xmlns="http://www.bea.com/ns/weblogic/90" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.bea.com/ns/weblogic/90 http://www.bea.com/ns/weblogic/90/weblogic-web-app.xsd"> 
    <jsp-descriptor>
        <keepgenerated>false</keepgenerated>
        <page-check-seconds>-1</page-check-seconds>
        <precompile>true</precompile>
        <precompile-continue>true</precompile-continue>
        <verbose>false</verbose>
    </jsp-descriptor>
    <container-descriptor>
        <servlet-reload-check-secs>-1</servlet-reload-check-secs>
        <prefer-web-inf-classes>true</prefer-web-inf-classes>
    </container-descriptor>
</weblogic-web-app>

Happy CXF servicing.

Stop Hibernate info level logging to Console in WebSphere

Keywords:
websphere hibernate logging info commons logging log4j info SystemOut.log Console stdout WAS7 WAS8.5

Problem:
The application is packaged with hibernate and configured with log4j logging (to a rolling, application-specific file - hibernate set to WARN level logging) and deployed to WebSphere with parent last class loading.

Despite all this info-level messages are being output to the WebSphere system-out/console log (in WAS_HOME\profiles\[profile_name]\logs\[server_name]\SystemOut.log).

Why is the application log4j configuration (in log4j.properties) being ignored and how can the application's hibernate logging be diverted away from the console?

Solution:
Various resources on the web describe configuring WebSphere logging or hibernate logging but the bottom of this post on Hibernate and Logging specifically describes this 'leaking' of log configuration/output and how to resolve it.

Create a commons-logging.properties with the content:

org.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger

... and put this file in the application's classpath (WEB-INF/classes for example).



Notes:
By default, WebSphere Application Server is installed to use info level tracing (for all loggers that it knowns about). In WAS 8.5 you'll see this in the SystemOut.log:

[22/10/12 12:00:00:000 EST] 00000001 ManagerAdmin  I   TRAS0017I: The startup trace state is *=info.

You can edit this by going to (in WAS 8.5) Application Servers > [server_name] > Troubleshooting | Change log detail levels. You can specify the logging definition in plain text. For example, turning default logging to audit-level and disabling logging for the verbose com.ibm.ws.webcontainer.annotation module (this is a ':' delimited list):

*=audit: com.ibm.ws.webcontainer.annotation=off

Alternatively, use the "Components and Groups" section of the interface to expand items of the "tree" and right-click to select custom logging levels for a given "node" (NB for log4j users, the log levels may not match log4j - eg "audit" and "warning" instead of "warn").

Also interesting to note is that without the commons-logging.properties file defined in your web-app you will see every package from your application(s) and their libraries (where classes include logging definitions) included in this "Components and Groups" interface. Therefore if you see org.hibernate.* disappear from this interface you can be certain the configuration has worked.

Managing Oracle JDBC Memory Usage - Executive Summary

Keywords:
Oracle JDBC memory leak usage java.lang.OutOfMemoryError fetchSize white paper summary 10g 11.2

Problem:
So there appears to be memory issues with your system. Oracle JDBC drivers seem like the prime candidate (based on the heap dumps). The Oracle JDBC Memory Management white paper says as much:

The Oracle JDBC drivers can use large amounts of memory. This is a conscious design choice, to trade off large memory use for improved performance. For the most part and for most users this has proved to be a good choice. Some users have experienced problems with the amount of memory the JDBC drivers use.

The white paper is only a dozen or so pages, but it seems each JDBC version introduces new connection properties or changes the meaning of previously used properties. Is there an executive-summary/cheat-sheet/quick-reference that would essentially say what you can do for a given version?

Solution:
There doesn't seem to be, but I'll have a go ...

First, some summary points:

• From 9i to 10g performance of the JDBC drivers has been improved "... on average about 30% faster".
• This was achieved by a greater use a caches. The justification being "memory is relatively cheap".
• In large scale applications with complex (and batch) data usage, there seem to be two key caches that may cause memory usage issues - the "Implicit Statement Cache" and the "Internal Buffer Cache".
• Tuning the caches can be done via connection properties (all of which can also be defined as system "-D" properties).
  Tuning should be done in consideration of:
  
  • the table design (column types and sizes)
  • the query design (columns needed / batches)
  • the fetch size. Setting this incorrectly will definitely cause OutOfMemoryErrors - as seen in JCR-2892.

Now, for the connection properties:

Property	Default	Notes	Applicable for Version
			10.0.2.4	11.1.0.6.0	11.1.0.7.0	11.2
oracle.jdbc.freeMemoryOnEnterImplicitCache	false	When true row-data buffers are cleared when a PreparedStatement is cached (in the "Implicit Statement Cache").	Yes (new)	Ignored (from 11.1.0.6.0 onwards, buffers are put in a new "Internal Buffer Cache")
oracle.jdbc.maxCachedBufferSize	Integer.MAX_VALUE (2,147,483,647 ~ 2Gb)	Sets an upper limit on the "Internal Buffer Cache". Look out for the change in meaning from 11.1.0.7 to 11.2 - though if you use a value >30 in 11.2 it will revert to treating it as an integer. Oracle recommends: "... start with 18. If you have to set the value to less than 16, you probably need more memory." Each connection will (may?) have its own buffer cache. So in a connection pool setup multiply the pool-size by the maxCachedBufferSize.	N/A (no "Internal Buffer Cache")	No (no way to set the size)	Yes Set as an integer value. 102400 ~ 100Kb	Yes Set as a log2 value. 18 = 2^18 = 262,144 ~ 256Kb
oracle.jdbc.useThreadLocalBufferCache	false	By storing the buffer cache as a TreadLocal instead of on the Connection you'll save memory if (and only if) there are less Threads than Connections. Avoid if using code that uses Connections across Threads.	N/A	N/A	Yes (new)	Yes
oracle.jdbc.implicitStatementCacheSize	0	An initial size for the "Implicit Statement Cache". But it doesn't seem that setting it to 0 or -1 will disable it, so it can perhaps be ignored for memory management issues - it may improve performance.	N/A	N/A	N/A	Yes (new)

Running Native SQL deletes or updates via Hibernate

Keywords:
hibernate manual native SQL update delete bulkUpdate child table spring

Problem:
Probably not the best idea but for performance, limitations in hibernate mappings or errors in the objects you find it'd be much easier to just run some manual SQL. Can you do this via hibernate?

Solution:
Yes you can, hibernate call this "Native SQL". This is fairly well document, but I noticed its missing an example of doing a manual update/delete - where the result is not going to give you a list() of anything.

For Query you make via createSQLQuery(...), use executeUpdate() which returns the number of rows effected.

Here an example (using the CATS/DOGS table names from the hibernate document):

Query deleteQuery = session.createSQLQuery(
    "delete from CATS "
    + "where DOG_ID in ( "
        + "select d.ID from DOGS d "
        + "where d.STATUS = ?)");
deleteQuery.setString(0, "VICIOUS");
int updated = deleteQuery.executeUpdate();

If you're integrating with hibernate via spring, the same can be done via the template-callback mechanism (where your DAO extends org.springframework.orm.hibernate3.support.HibernateDaoSupport):

Integer deletedData = (Integer)getHibernateTemplate().execute(new HibernateCallback () {
    public Object doInHibernate(Session session) throws HibernateException, SQLException {
        // delete the data
        Query deleteQuery = session.createSQLQuery(
            "delete from CATS "
            + "where DOG_ID in ( "
                + "select d.ID from DOGS d "
                + "where d.STATUS = ?)");
        deleteQuery.setString(0, "VICIOUS");
        int updated = deleteQuery.executeUpdate();
    }
});
if (log.isDebugEnabled()) {
    log.debug("rows deleted from CATS: " + deletedData);
}

Notes:
If you're wondering if you can avoid hard coding the table references by asking hibernate for the table-name mapping, this does seem possible - Get the table name from the model in Hibernate - but this approach doesn't seem to be "public" so may disappear.

Get the generated source code for JSPs in WebLogic

Keywords:
generated JSP java class source Oracle WebLogic line numbers

Problem:
Given a stack trace such as the following:

java.lang.NullPointerException
 at jsp_servlet._web_45_inf._jsp._demo.__example._jspService(__example.java:117)
 at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)
 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:300)
 at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:416)
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:326)
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:183)
 at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:526)
 at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:253)

It won't always be obvious what the issue in example.jsp corresponds to in the generated __example.java. How do you get at the generated source code for the JSPs?

Solution:
The solution involves configuring a [app_name]/WEB-INF/weblogic.xml file in your web-app. The documentation is in the WebLogic 10.3 docs - see weblogic.xml Deployment Descriptor Elements but keep in mind the file will be validated against the schema so the elements must be in the correct spot.

Below is an example - defining the jsp-descriptor with keepgenerated and working-dir elements:

<?xml version="1.0" encoding="UTF-8"?>
<weblogic-web-app
    xmlns="http://www.bea.com/ns/weblogic/90"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.bea.com/ns/weblogic/90 http://www.bea.com/ns/weblogic/90/weblogic-web-app.xsd">
    <jsp-descriptor>
        <keepgenerated>true</keepgenerated>
        <working-dir>c:/my_folder/temp</working-dir>
    </jsp-descriptor>
    <container-descriptor>
        <prefer-web-inf-classes>true</prefer-web-inf-classes>
    </container-descriptor>
</weblogic-web-app>

NTLM from an Axis (SOAP) service client - in 3 steps

Keywords:
NTLM authentication Negotiate Apache axis SOAP IIS Windows Integrated Authentication CommonsHTTPSender NTCredentials

Problem:
Authenticating a service request with BASIC authentication is (relatively) straightforward:

        import java.net.URL;

        import org.apache.axis.client.Stub;

        import com.example.service.Example;

        import com.example.service.ExampleServiceLocator;

        import com.example.service.ExampleRequest;

        import com.example.service.ExampleResponse;



        // get access to the web service

        ExampleServiceLocator locator = new ExampleServiceLocator();

        String serviceURL = "http://server/application/services/example";

        Example example = locator.getexample(new URL(serviceURL));

        // set credentials 

        ((Stub)example).setUsername("myusername");

        ((Stub)example).setPassword("mypassword");

        

        // setup request

        ExampleRequest request = new ExampleRequest();

        request.setProperty("SomeProperty");

        

        ExampleResponse response = example.example(request);

What if the (SOAP) service being called required NTLM authentication (e.g. the service is running in IIS and security is set as "Windows Integrated Authentication")?

Solution:
The following three steps are assuming Axis 1.x. The Apache Axis Client Tips and Tricks is a good reference, in particular for step 2, but also for other "tips".

Step 1: Add Apache commons-httpclient (3.1) and commons-codec libraries

Note you must add the commons httpclient jar file and not the (latest/refactored) apache httpclient to the project - or you will get ClassNotFound exceptions.

Step 2: Define custom client-config with CommonsHTTPSender

It's mentioned in the "Tips and Tricks" article mentioned above, but you can either: (a) define a custom client-config.wsdd file in the classpath before axis.jar; (b) edit the generated ...ServiceLocator.java generated class and make it override getEngine...; or (c) at runtime simply feed the customised config XML to your ...ServiceLocator object.

I prefer the latter - for example, define a static method with the config XML as a string:

    protected static org.apache.axis.EngineConfiguration getEngineConfiguration() {

        java.lang.StringBuffer sb = new java.lang.StringBuffer();

        sb.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n");

        sb.append("<deployment name=\"defaultClientConfig\"\r\n");

        sb.append("xmlns=\"http://xml.apache.org/axis/wsdd/\"\r\n");

        sb.append("xmlns:java=\"http://xml.apache.org/axis/wsdd/providers/java\">\r\n");

        // sb.append("<transport name=\"http\" pivot=\"java:org.apache.axis.transport.http.HTTPSender\" />\r\n");

        sb.append("<transport name=\"http\" pivot=\"java:org.apache.axis.transport.http.CommonsHTTPSender\" />\r\n");

        sb.append("<transport name=\"local\" pivot=\"java:org.apache.axis.transport.local.LocalSender\" />\r\n");

        sb.append("<transport name=\"java\" pivot=\"java:org.apache.axis.transport.java.JavaSender\" />\r\n");

        sb.append("</deployment>\r\n");

        org.apache.axis.configuration.XMLStringProvider config = 

            new org.apache.axis.configuration.XMLStringProvider(sb.toString());

        return config;

    }  

Then the call to the locator would become:

        // get access to the web service

        ExampleServiceLocator locator = new ExampleServiceLocator(getEngineConfiguration());

Step 3: Set the username as DOMAIN\username

Set the username as you did with BASIC authentication but you must ensure is set in the form DOMAIN\username (keeping in mind that if expressing this in java code - as a string - or as a property value in a properties file this would be set as "DOMAIN\\username" - \\ being the escape sequence for \):

        ((Stub)example).setUsername("MY_NT_DOMAIN\\myusername");

        ((Stub)example).setPassword("mypassword");

With the above 3 steps covered you're using NTLM.

Notes:
Avoid setting the system property -Djava.ext.dirs as the above relies on the sunjce_provider.jar library which is in JRE_HOME\lib\ext by default. Ext-path problems may give you errors such as:
"Cannot find any provider supporting DES/ECB/NoPadding"


Failing to set the username in the form DOMAIN\username will result in the error:

org.apache.commons.httpclient.auth.InvalidCredentialsException: 

    Credentials cannot be used for NTLM authentication: 

        org.apache.commons.httpclient.UsernamePasswordCredentials

 at org.apache.commons.httpclient.auth.NTLMScheme.authenticate(NTLMScheme.java:332)

 at org.apache.commons.httpclient.HttpMethodDirector.authenticateHost(HttpMethodDirector.java:282)

 at org.apache.commons.httpclient.HttpMethodDirector.authenticate(HttpMethodDirector.java:234)

 at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)

 at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)

 at org.apache.axis.transport.http.CommonsHTTPSender.invoke(CommonsHTTPSender.java:186)

This is because the format of the username determines the Credentials instance created. With the DOMAIN\... prefix on the username you get an instance of org.apache.commons.httpclient.NTCredentials rather than org.apache.commons.httpclient.UsernamePasswordCredentials - which as the message explains can't be used for NTLM.

Enable debug/trace level logging for Tomcat 6 Realms

Keywords:
tomcat 6 realm logging juli logging.properties debug="9" debug="99" debug="true" JNDIRealm trace verbose jndi realm JNDIRealm

Problem:
Apparently Tomcat 6 Logging is greatly improved ... more granularity, flexibility in choosing java.util.logging or log4j, etc. This is great, but I'm happy with the default logging - ie if something goes wrong let me get the detail from a log file.

This attitude hits a snag where things go wrong and there's nothing in the logs - in my case, setup of a org.apache.catalina.realm.JNDIRealm is not letting me in but there's no details why. It used to be a matter of simply setting debug="9" on the Realm definition and you have verbose logging - the examples in the Tomcat 6 Realm documentation still use this:

<Realm className="org.apache.catalina.realm.JNDIRealm" debug="99"
    connectionURL="ldap://localhost:389"
    userPattern="uid={0},ou=people,dc=mycompany,dc=com"
    roleBase="ou=groups,dc=mycompany,dc=com"
    roleName="cn"
    roleSearch="(uniqueMember={0})"
/>

But this has no effect on logging. You'll get a warning telling you as much:

03/03/2010 10:56:08 AM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Realm} Setting property 'debug'
 to '99' did not find a matching property.

What's the minimum I have to do to enable debug?

Solution:
You have to edit the $CATALINA_HOME/conf/logging.properties file.

1. Configure debug logging for Realms and Authentication

Insert the following lines (in blue):

############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# This would turn on trace-level for everything
# the possible levels are: SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST or ALL
#org.apache.catalina.level = ALL
#org.apache.catalina.handlers = 2localhost.org.apache.juli.FileHandler
org.apache.catalina.realm.level = ALL
org.apache.catalina.realm.useParentHandlers = true
org.apache.catalina.authenticator.level = ALL
org.apache.catalina.authenticator.useParentHandlers = true

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = 2localhost.org.apache.juli.FileHandler

This will give you debug/trace level logging to console and the file assuming you've kept the default config. But you only see debug in the console, not the catalina.[date yyyy-MM-dd].log file - in fact, the log file empty? The buffering means the file-logging is only written when the buffer is full.

2. Disable buffering for FileHandler logging

(until the issue is resolved of course) Insert the line (in blue):

1catalina.org.apache.juli.FileHandler.level = FINE
1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.
1catalina.org.apache.juli.FileHandler.bufferSize = -1

These two inserts give you pretty much the equivalent of the old debug="9" and you'll (hopefully) get the verbose information required - happy debugging ...

Configure endorsed libraries in Tomcat 6

Keywords:
Tomcat 6 endorsed java.endorsed.dirs Endorsed Standards Override Mechanism XML libraries xerces

Problem:
If your webapp needs its own XML libraries (xerces in particular) how do you get Tomcat 6 to use this and not the JAXP APIs packaged into the JSE? This used to be as simple as dropping them into ${CATALINA_BASE}/common/endorsed but there's only a ${CATALINA_BASE}/lib folder ...

Solution:
Thankfully found the solution in this blog (and comments).

Simply, create a ${CATALINA_BASE}/endorsed folder and drop the jar files in there. Tomcat will be setup to use this if it exists.


Notes:
No explicit mention of this in Tomcat 6 Class Loader notes

It does note the -Djava.endorsed.dirs system property is set but you need to check setclasspath.[bat|sh] for when it's set and what it's set to by default - ie ${CATALINA_BASE}/endorsed.

HSQL random function - calling scalar functions via JDBC

Keywords:
HSQL JDBC select random scalar built-in function rand fn

Problem:
I want to demstrate database interaction but the data doesn't really matter in this case. HSQL is ideal because you can set a connection to in-memory database .. but there's no data. Random would do, is there a random function in HSQL? Where is the function reference? Is it called rand(), random() or other?

Solution:
Short answer is it's called "rand". So the SQL would be:

CALL rand()

With JDBC this is:

PreparedStatement stmt = connection.prepareStatement("CALL rand()");
stmt.execute();

... and I can't find a complete function reference for HSQL.


Digging a bit further there is JDBC convention for calling scalar/built-in function in a vendor independent way - See JDBC 2.0: Escape Syntax and Scalar Functions. This involves the use of the "fn" keyword and braces - so the syntax is:

{fn <function()>}

Whether the JDBC driver recognises the syntax and maps it correctly to the underlying function ("rand" is called "random" in postgres for example) is up to the vendor. I can confirm that HSQL seems to handle most numeric, string and date functions (as mentioned above, can't find doco for this. The derby JDBC function reference is a reasonably good overview). So the vendor agnostic random call becomes:

PreparedStatement stmt = connection.prepareStatement("CALL {fn rand()}");

Notes:
What I could find was documentation of HSQL "Java Stored Procedures". This effectively lets you call any public static java method (from a class in the classpath) right there in the SQL statement. So, yet another (non-JDBC standard) way of making the random call would be:

CALL "java.lang.Math.random"()

... or doing more work in the SQL:

CALL "java.lang.Math.floor"("java.lang.Math.random"() * 100)

Pretty cool, I must think of a use for this :)

IE8 Compatibility Issues and working around them for HTML, Java and GWT

Keywords:
IE8 compatibility X-UA-Compatible java filter http-equiv GWT "links not working" broken EmulateIE7 compliance

Problem:
You're either certain your website/webapp is broken in IE8 or suspect it might be. What can you do?

For browsers such as firefox and IE7, you used the DOCTYPE switch to indicate if it should treat your page as being in "standards" mode or "quirks" mode. As discussed on A List Apart "standards" is open to interpretation and even if the page didn't fully comply with the standard it referenced, it could expect the browser to have a reasonable crack at letting it work.

Not any more. IE8 renders all pages in standards mode by default and it's interpretation is as far as I can tell, stricter than any other browser. In this mode, many HTML, CSS and javascript/DOM elements (that were introduced by older versions IE! - and in some cases understood by other browsers such as firefox) are not just deprecated, they're completely not there - meaning any reference to them will in most cases stop the page/application working in the browser. When this happens, the page layout may be askew, links may stop working and miscellaneous javascript errors will be flagged in the status bar.

If you're not sure if your webpage/webapp is strictly standards compliant, it most probably isn't.

GWT is effected - see Dan Moore's post and GWT Issue #3329.

Solution:
IE8 users can make broken pages work again by opting-out of the "most strict" standards treatment on a site-by-site basis using the "Compatibility View" button but you can save them the trouble by using a new http-header tag X-UA-Compatible.

The A List Apart article above describes the merits of being able to target specific versions (and the comments seem to mostly disagree :) but in the majority of cases I imagine you just want the page to be treated the way IE7 did - "standards" mode (with "flexibility") for pages that require it and "quirks" mode for all others. So use the "IE=EmulateIE7" value by either:

1. making the server side code add this http header to every response
   

   X-UA-Compatible: IE=EmulateIE7



2. or add the following meta tag to the <head> element of the page (which is essentially the equivalent to setting values on the http response)

   <meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />
   

You can cover the entire java web application by defining a filter as documented by Mark McLaren:

    public void doFilter(ServletRequest request, 
                         ServletResponse response, 
                         FilterChain chain)
            throws IOException, ServletException {
        if (response instanceof HttpServletResponse) {
            ((HttpServletResponse) response).setHeader("X-UA-Compatible", "IE=EmulateIE7");
        }
        chain.doFilter(request, response);
    }

Apache hosted sites can be covered by using mod_headers. For example, assuming you have the module loaded, add the following to the httpd.conf file:

Header add X-UA-Compatible "IE=EmulateIE7" 

Notes:
Having trouble installing the release version of IE8? (for goodness sake, just get firefox :) But, if you really do want to get it working for yourself see Internet Explorer 8 is not supported on this operating system for a discussion of common issues.

Redirect from a JSP page (to another server)

Keywords:
best practice tips JSP redirect XML

Problem:
<jsp:forward> is not for redirecting to an external server (and the address you forward to will have access the request attributes and parameters, which may be desirable, see discussion here: when to use response redirect and jsp forward.

Is using response.sendRedirect(...) ok/best-practice?
Solution:
The short answer, is it's fine. In summary, there are four approaches I can think of:

Approach #1: Scriptlet to sendRedirect

<%
    response.sendRedirect("http://www.example.com");
%>

This has worked since the introduction of JSPs (see: Implementing a Redirect in a JSP Page) ... sometimes you need the servlet API in your JSP page either because there's other content (eg HTML) that makes it reasonable not to be a pure servlet. Other times it's just convenient to have a text file compiled on the fly.

Though there are many that would disagree (Sciptlet snobs? see How to redirect a page in JSP - I don't see why you'd get wound up about scriptlets. Why would you write a servlet when one line of code in a text file gives you the same result?).

Approach #2: Use Apache JSTL and the c:redirect tag

<c:redirect url="http://www.example.com"/>

or scriptlets again:

<c:redirect url="<%=scriptlet logic%>"/>

Sure, it's 'pure' XML but you'll need the taglib definition in the JSP file and include the JSTL jars in your web application - as discussed in the notes for a previous post.

Approach #3: Refresh meta tag

You could get the JSP to produce a HTML page that contains the Refresh meta tag.

<html>
<head>
  <meta http-equiv="Refresh" content="0; url=http://www.example.com/">
</head>
<body></body>
</html>

Where 0 is the delay in seconds. If more than zero, you'd probably want to put some text on the page to explain what's happening.

Approach #4: Javascript

As above, you could get the JSP to produce a HTML page that contains javascript to perform the redirect.

<html>
<head>
<script type="text/javascript">
window.location = "http://www.example.com/";
</script>
</head>
<body></body>
</html>

You could use setTimeout to get the redirect to happen after a period of milliseconds.


Notes:
Out of curiosity I checked TCP monitor for what the browser is actually receiving when you use Approach #1 or #2 (sendRedirect or c:redirect)

HTTP/1.1 302 Moved Temporarily
Location: http://www.example.com

So it saves you from 2 lines of servlet (or scriptlet) code:

response.setStatus(302);
response.setHeader("Location", "http://www.example.com");

So the difference to Approaches #3 & #4 isn't great or much more inefficient really, in all cases it's up to the browser to go to the specified address.

Comments in JSON

Keywords:
comments JSON javascript

Problem:
Can you have comments in JSON?

Solution:
A discussion leading to the answer is on the Bytes IT forum.

Basically, though it's not defined in the JSON Grammar - on json.org - you can use slash-star /* ... */ comments to get most(?) javascript engines to ignore the comment text.

For example:

<script type="text/javascript">
    var nested_arrays = {
        cities: [
            ["Sydney", "Melbourne", "Canberra"]  /* Australia */
          , ["London", "Birmingham"]             /* UK */
          , ["Los Angeles", "New York"]          /* US */
        ]
    };
</script>

How do you remove a windows service?

Keywords:
manually remove windows service registry sc

Problem:
OK, I've uninstalled some software but it's left a service definition. It can't do much harm as most (but not all) of the resources it needs have been removed ... but I can't ignore it - how do you get rid of it?

Note to self: before uninstalling software with services it may help to stop them all and possibly disable them all too? I think this is why this particular thing could not be removed.

Note to developers: stop and remove services as part of uninstall programs.

Solution:
Some solutions talk about using regedit and modifying:

HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services

A better approach is to use sc ".. a command line program used for communicating with the Service Control Manager and services".

It should be as simple as:

sc delete [service name]

But the tricky thing is getting the name of the service right. What you see in the Service window is the display name not the service name. Use:

sc query state= all

To get a listing like

...
SERVICE_NAME: ExampleService
DISPLAY_NAME: Example Service That I Want To Delete
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN))
        WIN32_EXIT_CODE    : 1077  (0x435)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

...

SERVICE_NAME: helpsvc
DISPLAY_NAME: Help and Support
        TYPE               : 20  WIN32_SHARE_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN))
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

...

So the above example listing:

sc delete ExampleService

Notes:
Just type sc at the command prompt for usage and more example commands (hit y to get help for the sc query commands).