Mark Wolfe's Blog

I discovered a couple of interesting issues when using Apache CXF and Maven behind a proxy this week. It started when I sent out a package of stubs in a Maven project I had built to enable developers of integration systems to regenerate there own stubs from the live WSDL. This project uses the wsdl2java tool from Apache CXF to generate some JAX-WS based SOAP stubs from the WSDL hosted on a staging server on the internet. When run on one of the developers sites it became apparent that the Maven cxf-codegen-plugin doesn’t pass through the already configured Maven proxy settings to wsdl2java, this was a bit annoying.

So being a happy consumer of open source software, I had a browse over the sources to Apache CXF tools and discovered the method for retrieval of the WSDL files was using java.net.URL. To enable a proxy server for use by this class is as simple as passing some extra switches to the mvn command as in the following example.

$ mvn -Dhttp.proxyHost=proxy -Dhttp.proxyPort=8080 package

Once we had overcome this issue we hit another interesting hurdle. My integration tests in this Maven project were using the spring configuration method, these were also failing. Turns out we also needed to set the proxy in the Apache CXF configuration as well. This was done using a conduit as follows.

<http-conf:conduit name="*.http-conduit">
    <http-conf:client ProxyServer="squid.wolfe.id.au" ProxyServerPort="3128"/>
</http-conf:conduit>

So in summary if your working behind a proxy server building web services projects using Maven and Apache CXF you will need to do the following.

Configure a proxy in your Maven configuration so that assets can be retrieved, this is done as follows in your settings.xml.

     <proxy>
       <id>optional</id>
       <active>true</active>
       <protocol>http</protocol>
       <host>squid.wolfe.id.au</host>
       <port>3128</port>
       <nonProxyHosts></nonProxyHosts>
     </proxy>

Whenever you invoke any tests or calls which invoke wsdl2java you will need to pass the proxy settings in as previously described.

$ mvn -Dhttp.proxyHost=proxy -Dhttp.proxyPort=8080 package

When running any tests or routines which use Apache CXF driven web services you will need the conduit configured, in this example it is global and applies to all http connections.

<http-conf:conduit name="*.http-conduit">
    <http-conf:client ProxyServer="squid.wolfe.id.au" ProxyServerPort="3128"/>
</http-conf:conduit>

So there is no confusion about which libraries I am they are:

• Maven 2.2.1
• Apache CXF 2.4.0
• Sun\Oracle Java 1.6_24

So overall a frustrating day, but I won in the end, now all i need to do is incorporate all this information in my project so the onsite developer can work. Hopefully this post helps someone people that run into the same issues.

Recently I have been looking into setting up my own build server, having used Cruisecontrol, Hudson, Teamcity in the passed I wanted to give something new a try. With the recent release of Bamboo 3.1.1 I thought I would see what all the fuss is about. I logged onto Atlassian’s site and payed my 10 bucks, which much to my delight and respect, goes to charity.

I then spun up a clean Ubuntu server running 10.04.02, yes old trusty Long Term Support release, and ran through the following steps.

First install the Sun Java development kit, I use this in preference over OpenJDK at the moment for QA as it is what I normally deploy to for customers. To install this package we need to modify the /etc/apt/sources.list and remove the comments at the start of the lines which include the partner repositories. These look as follows

deb http://archive.canonical.com/ubuntu lucid partner
deb-src http://archive.canonical.com/ubuntu lucid partner

Run apt-get to update the cache of packages.

$ sudo apt-get update

Then install the dpkg for Sun\Oracle JDK.

$ sudo apt-get install sun-java-jdk

I will be using PostgreSQL for my database follow the configuration process described in Ubuntu PostgreSQL Installation Howto.

Now create a PostgreSQL user and database, note you will be prompted to enter a password for the bamboo user.

$ sudo -u postgres createuser -R -D -S -P -e bamboo
$ sudo -u postgres createdb -O bamboo bamboo_db

To test this is login will work fine I use the psql command and pass it a hostname to connect to using TCP/IP, this is to ensure our JDBC driver will connect fine.

$ psql -h localhost -U bamboo bamboo_db
Password for user bamboo: 
psql (9.0.4)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.

bamboo_db=> \q

Install tomcat6 using apt, the reason I am using this package rather than the all in from Atlassian is I like my tomcat maintained. Dumping a tomcat in your OS with no one watching your ass is unwise in my opinion.

$ sudo apt-get install tomcat6

Rather than just popping the war file into /var/lib/tomcat6/webapps, I prefer to externalise the application by extracting the archive into it’s own directory under /var/lib. I then configure tomcat to load the application from this location. This is done so that I can guarantee tomcat upgrade won’t nuke or otherwise disturb my Bamboo installation.

$ sudo mkdir /usr/share/atlassian-bamboo-3.1.1
$ cd /usr/share/atlassian-bamboo-3.1.1
$ sudo jar xvf ~/atlassian-bamboo-3.1.1.war

Before we start ensure tomcat is shut down, otherwise it mite go and deploy bamboo before we are ready!

$ sudo /etc/init.d/tomcat6 stop

Next we need to configure tomcat to use load this web application, to do this we create navigate to the tomcat configuration directory.

$ cd /var/lib/tomcat6/conf/Catalina/localhost

Backup the original ROOT.xml, being careful to preserve the permissions.

$ sudo cp -ipv ROOT.xml ROOT.bak

Edit file named ROOT.xml as root, I will be making my bamboo the ROOT application in this tomcat, in other words served at http://myserver.com/. Put the following content in this file. Note you also need to generate a password for your DB login and enter it where the XXXXXXXX is.

<Context path="/" docBase="/usr/share/atlassian-bamboo-3.1.1">

  <Resource name="jdbc/BambooDS" auth="Container" type="javax.sql.DataSource"
            username="bamboo"
            password="XXXXXXXX"
            driverClassName="org.postgresql.Driver"
            url="jdbc:postgresql://localhost:5432/bamboo_db"
            />

</Context>

Make the ROOT.xml file only readable by the tomcat user to protect the plain text password located within.

$ sudo chmod 600 ROOT.xml

Remove the current root web application from tomcats /var/lib/tomcat6/webapps directory.

$ sudo rm -r /var/lib/tomcat6/webapps/ROOT

Create a data location for bamboo, and change the file permissions so that tomcat6 can write to this location.

$ sudo mkdir /var/lib/atlassian-bamboo
$ sudo chown tomcat6:tomcat6 /var/lib/atlassian-bamboo

Now install the PostgreSQL driver in tomcat, to do this first download it to your home directory then copy it to /usr/share/tomcat6/lib as follows.

$ cd ~/
$ wget http://jdbc.postgresql.org/download/postgresql-9.0-801.jdbc4.jar
$ sudo cp postgresql-9.0-801.jdbc4.jar /usr/share/tomcat6/lib

Modify the /usr/share/atlassian-bamboo-3.1.1/WEB-INF/classes/log4j.properties as root, to correct the location of the log file. This in my opinion needs to be a MUST configure item during there installation process otherwise the log file could end up anywhere, for example the PWD of the executing start up script.

Navigate to the line that looks like this.

log4j.appender.filelog.File=atlassian-bamboo.log

And change it to this.

log4j.appender.filelog.File=${catalina.base}/logs/atlassian-bamboo.log

Also you will need to configure the bamboo.home in /usr/share/atlassian-bamboo-3.1.1/WEB-INF/classes/bamboo-init.properties. This should be changed to the following value. Again this will need to be done as root.

bamboo.home=/var/lib/atlassian-bamboo

Before we start tomcat we need to increase the amount of memory available to tomcat as well as some other params. This is done by editing /etc/default/tomcat6.

Open the file as root and edit to the following line.

JAVA_OPTS="-Djava.awt.headless=true -Xmx128m"

Change it to the following value.

JAVA_OPTS="-server -XX:MaxPermSize=256m -Djava.awt.headless=true -Xmx512m"

Now start tomcat and then tail the server log file.

$ sudo /etc/init.d/tomcat6 start
$ tail -f /var/log/tomcat6/atlassian-bamboo.log

Now open your browser and to http://servername:8080/ and follow the prompts.

When prompted for data base, select use datasource and specify the following value.

java:comp/env/jdbc/BambooDS

Now I have a nice new Atlassian Bamboo server ready to build my software, more on how that goes in future posts.

Whilst working on my Javascript, ExtJS 4 and Apache CXF skills I came across a solution to a problem I encountered getting ExtJS to talk to Apache CXF using JSON.

The basis for this issue revolves around “wrapping” in JSON and two different schools on what is correct way to encode it. As far as I can see there is the more verbose version which Jettison, the default JSON serialiser in Apache CXF produces, then there is the “unwrapped” version which the alternate serialiser Jackson produces.

In my case I chose Jackson the more terse version, this is good for a couple of reasons:

• It is compatible with ExtJS without any modifications
• It is smaller and therefore produces less data on the wire.

Also I like the annotations that Jackson comes with, and find it a bit easier to work with than Jettison.

So to enable Jackson I modify my projects Maven pom file I add the following dependency.

<dependency>
    <groupId>org.codehaus.jackson</groupId>
    <artifactId>jackson-jaxrs</artifactId>
    <version>1.5.7</version>
</dependency>

In addition to this some changes are required in the spring configuration which houses our RESTful services. In the following excerpt from my spring configuration, I have declared the jsonProvider then set it as one of the providers jaxrs:server.

<bean id="jsonProvider" class="org.codehaus.jackson.jaxrs.JacksonJsonProvider"/>

<jaxrs:server id="restServices" address="/">
        <jaxrs:serviceBeans>
            <ref bean="projectService"/>
        </jaxrs:serviceBeans>
        <jaxrs:providers>
            <ref bean="jsonProvider"/>
        </jaxrs:providers>
        <jaxrs:features>
            <cxf:logging/>
        </jaxrs:features>
</jaxrs:server>

Once Jackson was enabled my ExtJS JSON driven data stores were functioning perfectly, aside from dates. Jackson’s default behaviour for serialisation of a java.util.Date is to convert it to milliseconds since EPOC. To do this I used a feature in spring known as compound property names, this enabled me to instantiate an instance of the mapper, then override the serializationConfig.dateFormat to configure the mapper to produce ISO 8601 dates. This shown in the following excerpt which illustrates the updated jsonProvider using the reconfigured jacksonMapper.

<bean id="jacksonMapper" class="org.codehaus.jackson.map.ObjectMapper">
  <property name="serializationConfig.dateFormat">
    <bean class="java.text.SimpleDateFormat">
      <constructor-arg value="yyyy-MM-dd'T'HH:mm:ss.SZ"/>
    </bean>
  </property>
</bean>

<bean id="jsonProvider" class="org.codehaus.jackson.jaxrs.JacksonJsonProvider"
    p:mapper-ref="jacksonMapper"/>

The result of this is shown in the following JSON sample which illustrates how a project object containing some dates is encoded.

{
    success: true
    message: "Project found."
    data: {
        artifactId: "bobtheapp"
        groupId: "au.id.wolfe.bta"
        inceptionYear: "2011"
        organization: null
        developers: null
        dateAdded: "2011-05-21T20:34:15.862+1000"
        dateUpdated: "2011-05-21T20:34:15.862+1000"
        version: "1.0.0"
    }
}

So after another journey off track back to hacking on my project.

Configuring logging in Apache CXF can be confusing at first, in my view this is further compounded by Apache Tomcat’s logging architecture.

In my cxf based projects the first step is to use the logging configuration available in 2.2.8 or later. This enables you to configure your logger of choice for the entire CXF stack. Needless to say I really like this feature as I am not a big fan of java.util.logging as it’s configuration is not very intuitive.

First step is to change all logging in CXF to my logging stack of choice which is sl4j and logback. As per the CXF documentation I added a file named org.apache.cxf.Logger in my maven web application project located at src/main/resources/META-INF/cxf. This file contained just the following string.

org.apache.cxf.common.logging.Slf4jLogger 

I also add the following dependencies to my maven projects pom.xml.

<!-- This is to override spring's dependence on apache commons logging -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jcl-over-slf4j</artifactId>
    <version>1.5.11</version>
</dependency>

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.5.11</version>
</dependency>

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-core</artifactId>
    <version>0.9.19</version>
</dependency>

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>0.9.19</version>
</dependency>

And I configure a basic logback configuration within in logback.xml located in my web projects src/main/resources directory.

The main aims of this are:

• Summary log file with only warnings and errors.
• Trace file containing all web service messages.

The main reason for separating WS messages into a trace file is to keep the application log file down to an easy to handle size. This is especially helpful for systems which handle large web service messages. Also you may want to roll the trace file at a different interval than the application log.

<configuration>

    <!-- Just used while running in process while developing -->
    <appender name="STDOUT"
              class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>
                %d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <!-- This is just for contents of web service operations and can get quite large -->
    <appender name="WSLOGFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${catalina.home}/logs/mpw-message-trace.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- daily rollover -->
            <fileNamePattern>${catalina.home}/logs/mpw-message-trace.%d{yyyy-MM-dd}.log</fileNamePattern>
        </rollingPolicy>
        <encoder>
            <pattern>
                %d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <!-- Used for application logging to which when deployed is quite terse and restricted to warnings typically -->
    <appender name="APPLOGFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${catalina.home}/logs/mpw.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- daily rollover -->
            <fileNamePattern>${catalina.home}/logs/mpw.%d{yyyy-MM-dd}.log</fileNamePattern>
        </rollingPolicy>
        <encoder>
            <pattern>
                %d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n
            </pattern>
        </encoder>
    </appender>

    <logger name="org.apache.cxf" level="INFO"/>
    <logger name="org.springframework" level="INFO"/>

    <!-- Set additivity to false when deployed -->
    <logger name="org.apache.cxf.interceptor" additivity="true">
        <appender-ref ref="WSLOGFILE"/>
      </logger>

    <root level="INFO">
        <!-- STDOUT is normally this is removed when deployed as it ends up in tomcat server logs -->
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="APPLOGFILE"/>
    </root>

</configuration>

For a more complete sample you can look over the sources to maven-project-wizard on github.

In addition to this is required configuration at the operating system level to compress and archive these log file. I will cover this in a later post.

Recently I have been doing quite a bit of work around Apache CXF, both on the server side and the client side. Over the next few weeks I will post some of the tricks I have learnt.

In this post I will summarise one of the cxf-codegen-plugin options I used to assist me while consuming WCF web services using Apache CXF. When dealing with WCF based services, and indeed any WSDL which uses more than one namespace it is handy knowing how to control name spaces and how these translate into packages in your java stubs.

When consuming WCF services you may encounter either the default namespace for services which is tempura.org, or more commonly, one more custom name spaces configured by the developer.

Below is an example of using selective namespace mappings by specifying the -p option in the cxf-codegen-plugin. This switch instructs wsdl2java to map all xml objects in a given namespace into the java package supplied.

So if we had a WCF based registration service which included the following namespaces:

• http://schemas.datacontract.org/2004/07/System
• http://schemas.datacontract.org/2004/07/Wolfeidau.Model
• http://wolfe.id.au/services/

Only the http://wolfe.id.au/services/ would be mapped into au.id.wolfe.services.registration java package.

The reason this is done is typically to avoid name clashes and issues with overlapping data objects used by more than one service in the one namespace.

<!-- Generate client using WSDL -->
<plugin>
  <groupId>org.apache.cxf</groupId>
  <artifactId>cxf-codegen-plugin</artifactId>
  <version>2.4.0</version>
  <executions>
    <execution>
      <id>generate-sources</id>
      <phase>generate-sources</phase>
        <configuration>
        <sourceRoot>${basedir}/target/generated/src/main/java</sourceRoot>
          <wsdlOptions>
            <wsdlOption>
              <wsdl>http://wolfe.id.au/services/registration?WSDL</wsdl>
              <serviceName>RegistrationService</serviceName>
              <extraargs>
                <extraarg>-client</extraarg>
                <extraarg>-verbose</extraarg>
                <extraarg>-p</extraarg>
                <extraarg>http://wolfe.id.au/services/=au.id.wolfe.services.registration</extraarg>
              </extraargs>
            </wsdlOption>
          </wsdlOptions>
        </configuration>
        <goals>
          <goal>wsdl2java</goal>
        </goals>
    </execution>
  </executions>
</plugin>

As most of the samples on the Apache CXF website are in opinion way to simplistic, I am putting together some more extensive client and server samples which I will post up on wolfeidau Github soon.