Stephane's thoughts corner…

Stephane Bailliez's thoughts on everything

The first reaction when I read this was:

Another good piece of software that will lose thousand of hours in build problems.

Why ? because it will go through the same hurdles that large projects such as Apache Directory Server or Apache Geronimo go through just about every day.

Sylvain points at the Cocoon instructions and the most revealing part is actually:

To build Cocoon, use the following command:

$ mvn -Dmaven.test.skip=true install

In case of any failures, repeat command as necessary till you see the message:

BUILD SUCCESSFUL

which is not a joke and one of the most dramatic aspect of Maven. It is unreliable and unreproductible.

Sylvain already blogged about it extensively and revealed the fact that we are using Ivy to manage our dependencies (which at this time are in the 3-digit figure). A project is indeed only a matter of importing a one-liner build.xml, declaring a name, a version, a package information (for manifest purposes) and ivy.xml.

Interestingly the most annoying elements at this time are due to the extra layer of compatibility with Maven that I needed to insert to offer a painless transition a few months ago. This is something I plan to get rid of very soon now that we are mostly in cruise control mode for this build and I start to see patterns emerging and needs showing up.

Early on, when Maven was initially taken (for no reason at all actually) I spent a lot of time actually getting the dependencies from a reliable source (ie: not ibiblio or whatever but direct from the website). Figuring out the dependencies, rewriting the POM. Anything you get from an online repository is inherently badly written – should it be if it is written by a Maven user that just wanted it on the repo or by the developpers of the module themselves who don’t really know how to write a POM either (there’s no valuable documentation on it, it is extra-verbose as XML attributes seem not to exist in Maven XML land so people tend to get lost and not really update valuable information and/or totally useless information get into the repository)

To describe partly the current build system and without saying too much. I have roughly 2 build files, one being from a normal application that produces a jar, and another one for a web application that imports the ‘normal application’ one and that overrides one target (and define a few others webapp related), a web application producing both a jar and a war.

• A project has therefore just to import either the normal or the web application one. A one-liner. I provide typical templates to use for build.xml, build.properties and ivy.xml, the later having predefined configuration and a set of dependencies to use de-facto (for example servlet-2.4 in compile configuration for webapps, etc…)
• As of now each project is also publishing its sources and javadocs.
• The whole set of application is described using a ‘virtual ivy’ file (no artifacts) which allow to have a graph/report of dependencies or build them (I have another build for that which basically does subant calls)
• It is built using an Ivy build list that determines the order of dependencies.
• I have a few niceties, like for example I have added a target ‘whoami’ which just print out the name of the project using an <echo> this way you can know easily what is the order of the whole build without actually building it. Seems stupid but it is pretty helpful sometimes.
• It is of course doing the whole unit test she-bang on request and gives coverage information.
• For webapp, it launch a pluggable servlet container against your war in remote debug mode as a default.

I’m still not satisfied (and people who know me also know that I’m never satisfied) of the current state of affairs and things could be even better and simpler in many many ways.

I just want to make that build simple and reliable. I don’t want it to be a whole kitchen sink that does everything (and nothing), therefore I’m being very careful not to add too many features that people would want (most of the time they just prove themselves unneeded). I first want to make it build in a very predictable way, if something fails, I want to know where it fails and why…and as early as possible (without again writing a truckload of code, properties, conditions, …). This is basically THE goal. It should not make coffee, we have a couple of coffee machines for that. )

And for any large build, believe me, you don’t want to have something too smart.

The most difficult thing is to actually cater to different styles and anticipate also the way it evolves and/or control it. A codebase normally goes through a lot of changes in the initial phase, the versioning is also much simpler while this tends to evolve with time to components having their own versioning, branch, tags and stabilize. And it stabilizes only if you are able to control the growth phase.

That said, I’m looking forward to the next release of Ivy 1.4 and Ant 1.7.0 to simplify things even further.

There are a couple of things that can however be improved on Ivy but I will keep that to another post.

As for Spring Framework, let’s just hope that Spring does not turn into Fall.
But good luck guys because you will need it.

My other area where I would like to focus afterwards will be integration and deployment and I think my dear friend Steve has a couple of things I want

Oh and btw, Steve is also finishing the 2nd edition of Java Development with Ant (aka jdwa) published at Manning. Believe me you don’t want to miss that one when it will go out and I expect it to have as much success as the very first version. It is satisfying sometimes to know that quantity is synomous of quality.

Kevin Jackson has published an article about Antlibs. For people going to ApacheCon Asia, he will also do the session Antlibs – specifying custom tasks for Apache Ant:

Antlibs a new way to develop custom tasks for Ant 1.7. The presentation covers, how to write a basic antlib, how to test antlibs (using BuildFileTests, or the new antunit facility), and how to use the antlib in a build file.

Thoughworks released Buildix. It’s a ready to deploy VMWare image (or ISO) based on Knoppix Linux distribution that provides out of the box:

• Trac (Project management + Wiki + Issue tracker)
• CruiseControl (continuous integration)
• Subversion (version control)

So in the end, you have everything ready to create a new project. Considering the number of companies out there that still do developpement without a version control system or a wiki…there is probably no more excuse to not use Buildix. And it cost nothing.

I missed AC EU 2006 in Dublin due to very bad planning on my side and unfortunate sum of events and because everybody was leaving, someone had to stay there and pretend to work, right ?

Too bad as it would have been refreshing to stand there and talk to people. In conferences the social aspect is usually more important than the sessions themselves as you always learn a lot of things by talking to your peers.

Emmanuel Lecharny did a session about Apache Directory Server and its existing and upcoming performance and features. Sylvain Wallez is there hanging around all sessions and will probably come back with another bunch of good ideas for us (I guess he was sneaking around Peter Royal’s MINA session , Mads Toftum was also speaking about Apache HTTPd and Gianugo Rabellino about REST.

I’ll recommend Sylvain to go check Steve Loughran and Julio Guijarro session about SmartFrog.

I’ll be apparently in New York from July 5th for about 2 weeks, maybe more, maybe less. That was supposed to be last week/this week (which is also a reason why I missed AC) but eh…things change all the time.

I hacked those last 2 days an Ant task to help launch any servlet container easily against our different webapps module in a snap. This is something I wanted to do for quite some times, but ultimately never actually needed to do as I usually always was usually checking under source control the targeted production container and using some configuration template files with token filtering or variable expansion.

The task works with Tomcat 5.x, Jetty 5.x and Jetty 6.x. For Jetty, it also supports development mode (aka context reloading) which is originally missing from Jetty. It also works in either embedded mode in its own classloader and fork mode (for convenient debugging).

I just have to switch container.type to either tomcat5, jetty5 or jetty6 and it starts the container. To give an idea of what it looks like in a common build file:

NB: this is normally declared in a namespace but the syntax highlighting plugin completly mangles the xml.
There is hardly anything revolutionnary in there and in some way I reinvented the wheel.

• Cargo: I wanted to use it, but actually spent more time finding my way through either documentation or code with virtually no decent result in what I considered a reasonable amount of time. Cargo does normally much more than the above and is much more abstracted than the design I hacked but I was targeting simplicity and efficiency.
• Jetty 6 Maven plugin: provides support for Jetty 6 and development mode (context reloading) if you're using Maven, but that of course was not an option for me.

Everything fits into a single common build file which uses Ivy behind the hood to handle dependencies in different configurations such compile, test, runtime. The module Ant build files uses only a single import:

The next step will probably be able to add minimal logging configuration and configuration file support to be able to run some complex setup (and thus too proprietary) which is not worth spending too much time on.

Nothing revolutionary in there, but I like it as it's quite neat so I just needed auto-congratulation