Configure endorsed libraries in Tomcat 6

Keywords:
Tomcat 6 endorsed java.endorsed.dirs Endorsed Standards Override Mechanism XML libraries xerces

Problem:
If your webapp needs its own XML libraries (xerces in particular) how do you get Tomcat 6 to use this and not the JAXP APIs packaged into the JSE? This used to be as simple as dropping them into ${CATALINA_BASE}/common/endorsed but there's only a ${CATALINA_BASE}/lib folder ...

Solution:
Thankfully found the solution in this blog (and comments).

Simply, create a ${CATALINA_BASE}/endorsed folder and drop the jar files in there. Tomcat will be setup to use this if it exists.


Notes:
No explicit mention of this in Tomcat 6 Class Loader notes

It does note the -Djava.endorsed.dirs system property is set but you need to check setclasspath.[bat|sh] for when it's set and what it's set to by default - ie ${CATALINA_BASE}/endorsed.

XSLT output is missing the doctype

Keywords:
xslt identity transform missing doctype Document Type java DOM

Problem:
Running an XSLT 'identity transform' I would expect the XML data out to be identical to the data in ... it almost is, but it's missing the doctype declaration.
Can you tell the XSTL to keep the doctype in?

Solution:
Apparently not ... I can't find a good reference for this, but it seems that the spec (or at least xalan) is a bit vague on how this should be handled. Proposed work-arounds seem to be hard-coding the doctype in the <xsl:output> element (if outputting to xml) or hiding the doctype in the XSLT via <xsl:text disable-output-escaping="yes">.

If you want to make the inclusion of the doctype generic there's no pure XSTL solution - you can add it after the transform, but even that is not straightforward using pure java DOM elements ... here is the code:

    // assuming there's an inputDocument
    TransformerFactory factory = TransformerFactory.newInstance();
    StreamSource stylesheet = new StreamSource(stylesheetData);

    DOMSource source = new DOMSource(inputDocument);
    DOMResult result = new DOMResult();

    Transformer transformer = factory.newTransformer(stylesheet);
    transformer.transform(source, result);

    Document outputDocument = (Document)result.getNode();
    if (inputDocument.getDoctype() != null) {
        DocumentType inputDocType = inputDocument.getDoctype();
        
        // you can't importNode for DocumentType nodes but you can create 
        // them via DOMImplementation from there insert is easy
        DOMImplementation domImpl = outputDocument.getImplementation();
        DocumentType outputDocType = domImpl.createDocumentType(
              inputDocType.getName()
            , inputDocType.getPublicId()
            , inputDocType.getSystemId());
        outputDocument.insertBefore(outputDocType, outputDocument.getDocumentElement());
    } 

Notes:
It's interesting that this code is not allowed:

    DocumentType outputDocType = (DocumentType) outputDocument.importNode(inputDocType, true /*deep*/);

It seems DocumentType is a special element that can't be imported.

It's also interesting there's no method straight on the document:

    DocumentType outputDocType = outputDocument.createDocumentType(...);

The only way to get this kind of element is to obtain the special DOMImplementation object via the Document.getImplementation() method.

Quirks with the implementation? I've looked at this issue too long already to bother to find out why :)

JAXB xs:any xml content missing from marshalled output

Keywords:
JAXB xs:any xml content missing from marshalled output processContents="lax"

Problem:
I'm defining an XML schema that needs to be able to include any XML - "xs:any" seems perfect for the job. Also need to generate JAXB objects from this schema ... everything seems fine on compilation and in constructing the JAXB objects. The problem is at runtime, it appears that the "any" content is missing from the JAXB objects.

example schema

<xs:schema
    targetNamespace="http://example.com/xml"
    xmlns="http://example.com/xml"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    elementFormDefault="qualified"
    attributeFormDefault="unqualified">
    <xs:element name="MyXmlObject">
        <xs:complexType mixed="true">
            <xs:sequence minOccurs="0">
                <xs:any namespace="##any" processContents="lax" minOccurs="0"/>
            </xs:sequence>
            <xs:attribute name="firstName" type="xs:string" use="required"/>
            <xs:attribute name="lastName" type="xs:string" use="required"/>
            <xs:attribute name="happy" type="xs:boolean" use="optional"/>
        </xs:complexType>
    </xs:element>
</xs:schema>

example JAXB object usage

String xmlContent = "<foo>misc. text</foo>";

InputSource source = new InputSource(new StringReader(xmlContent));
Element domElem = (Element)NodeUtils.parseStream(source, true);
myXmlObject.getContent().add(domElem);

JAXBContext jc = JAXBContext.newInstance("com.example.xml");
// marshal to System.out
Marshaller m = jc.createMarshaller();
m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
m.marshal( response, System.out);

example XML output (missing my any content)

<MyXmlObject xmlns="http://example.com/xml" firstName="bar" lastName="bar" happy="true">
</MyXmlObject>

Solution:
The JAXB 1.0.x FAQ had the answer - Can I access <xs:any> as a DOM node?. There's detailed notes here with links to example schema and the Bug report, but basically "the spec doesn't support the mapping to DOM, so the RI is not allowed to do this" ... but it will if you put it in "extension mode".

This involves adding some extra elements to your schema and compiling with the "-extension" switch (or extension="true" attribute if using the XJC ant task).

new example schema (with JAXB customisation)

<xs:schema
    targetNamespace="http://example.com/xml"
    xmlns="http://example.com/xml"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
    xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
    elementFormDefault="qualified"
    attributeFormDefault="unqualified"
    jaxb:extensionBindingPrefixes="xjc"
    jaxb:version="1.0">
    <xs:element name="MyXmlObject">
        <xs:complexType mixed="true">
            <xs:sequence minOccurs="0">
                <xs:any namespace="##any" processContents="lax" minOccurs="0">
                    <xs:annotation>
                        <xs:appinfo>
                            <xjc:dom/>
                        </xs:appinfo>
                    </xs:annotation>
                </xs:any>
            </xs:sequence>
            <xs:attribute name="firstName" type="xs:string" use="required"/>
            <xs:attribute name="lastName" type="xs:string" use="required"/>
            <xs:attribute name="happy" type="xs:boolean" use="optional"/>
        </xs:complexType>
    </xs:element>
</xs:schema>

What version of xerces have I got?

Keywords:
xerces version print classpath parser

Problem:
This is kind of related to the previous two posts. With XML parsing + classpath issues it's often useful to know not only what version of Xerces you're using but also where it's been loaded from.

Solution:
This is based on the equivalently useful site of Java Almanac - Determining from Where a Class Was Loaded. The point they make about the system classloader is crucial: "It is not possible to determine the location of classes loaded by the system class loader in the same way since the class' code source is null."

If your Xerces is the one loaded by the system, then this may explain some unusual behaviour if your web application expects to ship with it's own version.

  public static String getXercesDetails() {
       // no hard-writed references to the version class because it might be unavailable
       Class c;
       Method m;
       Object obj;

       try {
           c = Class.forName("org.apache.xerces.impl.Version");
           m = c.getDeclaredMethod("getVersion", new Class[] {});
           if (Modifier.isStatic(m.getModifiers())) {
               // Xerces 2.x
               obj = null;
           } else {
               // Xerces 2.1.x
               obj = c.newInstance();
           }
          
           ProtectionDomain pDomain = c.getProtectionDomain();
           CodeSource cSource = pDomain.getCodeSource();
           String loc = "?";
           if (cSource != null) {
               loc = cSource.getLocation().toString();
           } else {
               // CodeSource is null, Xerces loaded by system class loaded.
               loc = "(system classloader? sun.boot.class.path = " + System.getProperty("sun.boot.class.path") + ")";
           }
          
           return ((String) m.invoke(obj, new Object[] {})) + " from " + loc;
       } catch (Exception e) {
           return "unkown (" + e.getMessage() + ")";
       }
   }

Notes:
Xerces (and Xalan) in the endorsed directory can cause some nasty issues for applications that expect to use their own version. See the JBoss Issue: JBAS-2073

An invalid XML character (Unicode: 0x0) was found in the comment

Keywords:
xerces "An invalid XML character (Unicode: 0x0) was found in the comment"

Problem:
This is a very specific problem I admit, but what are you meant to do? From the error message list for the Xerces parser, it's clear that this "0x0" character is a bad thing, but it's also clear that I haven't put this character in the document.

Something's wrong with the reading/processing of the byte stream containing the XML content.

Solution:
A very specific solution too. In this case it was that file was being stored using a Slide implementation that came with JBoss. I don't know anything about this implementation, but it doesn't seem to work when you're accessing it (via WebDAV) with jakarta-slide-client libraries. Switching the storage to using the latest version from Jakarta fixed this issue.