Selecting Development Infrastructure

This week, I was tasked to create a development infrastructure with the following components:

• Subversion for SCM
• Trac for bugtracking
• Hudson for CI
• Sonar for Quality Reporting
• Nexus, a Maven repository

I added 2 more components:

• A database in order to store data from each application in the same datastore
• A LDAP in order to authentify oneself in each application with the same credentials

Since we had no server to begin from, I had to install the infrastructure on my development machine, meaning Windows and 1 Gb RAM. The following is a personal feedback I have from this week.

Subversion means Apache HTTP Server. Since I already installed Apache a few times, installing (and configuring it) one more time went like a breeze. I really love this software.

Likewise, once one’s got the Subversion installer, installing it and configuring the underlying Apache instance can be done in less than 1 hour.

Now comes the fun part. Trac was a choice not made by myself (I even didn’t know this product until this week). This choice was made from Trac’s features. Little did we know that Trac runs on Python.

I (and my company) have no Python skills, so please, don’t make fun of me because of what follows.

I installed Python and then configured Apache to use it. That was the easy part.

First lesson, Trac doesn’t run on the latest Python (v3.2). I learnt that the hard way.

Second lesson, Trac needs a whole stack of component connectors:

• since Trac can display you Subversion repository, it needs to communicate with Subversion. It is done through Python so here come Python svn bindings. And if you installed Python v2.6 which is compatible with Trac, that’s too bad because bindings are only avalaible for Python 2.5
• Trac uses SQLLite (first time I heard of this, I’m beginning to feel like a complete idiot) out-of-the-box to store its data. Since all the other applications can run on MySQL and I happen to know a little about MySQL, I wanted Trac to use MySQL (a logical step). Not surprisingly, Python needs drivers to use MySQL, like in Java. This is done through MySQL Python. Completely unlike in Java, drivers are not put in the classpath of the running application, but installed in the Python distribution. Python 2.6+ is not supported though Trac supports it. Life is fun.
• Trac needs a installation component called setuptools that manages Python eggs (a specific format for Python extensions).
• Trac uses a templating engine. Version 0.10 of Trac needed ClearSilver, version 0.11 needs Genshi. Since I used Trac v0.11 and completely forgot to install Genshi, it complained about… not finding ClearSilver. Thanks for the misleading error message, I lost about 1 hour.

Third lesson: in order to use Eclipse Mylyn (an Eclipse plugin that manages to connect to a bugtracker and displays issues in Eclipse), Trac needs the XML-RPC plugin to be installed. Using the previously installed setuptools, it only needs a command-line instruction.

Fourth lesson, a project is linked to an environment. An environment is created using a command-line tool and then configured in Apache.

Fifth lesson, it is impossible to create sub-projects though a patch exists to modify the code in order to do so.

Hudson comes into two flavors:

• Hudson standalone as a Java Web Start installer
• a war file

The standalone runs nicely even though I can only guess at the underlying complexity. It can be configured to run as a Windows service. I am amazed: this is too good to be true. My deepest respect to the developement team. Finally, I uninstall Hudson (equally as easy) because…

I will have to run 3 Java applications: Hudson, Sonar and Nexus. If each runs in standalone mode, this means 3 JVM running concurrently and I’m bound to my crappy 1Gb RAM. Since each of these applications is delivered in a WAR archive, better have a lightweight server such as Tomcat (or Jetty). Since I’v used Tomcat since the dawn of time and since I see Jetty more like an embedded container, I installed Tomcat (this is a no-op, even for first time users).

Deploying Hudson in Tomcat is easy enough. Just don’t forget to configure Tomcat’s JVM with the -DHUDSON_HOME Java property. This tells Hudson to use this folder for storing its files (runs, builds, etc.).

Sonar was deployed easily in Tomcat. I configured it to use MySQL and everything went fine. The only comment I have about Sonar is that it comes in a whooping 45+ Mb archive. I mentioned this to the development team: it will likely be curbed in future versions.

Nearly finished I thought and went on to deploy Nexus. Deploying Nexus war in Tomcat went fine. Configuring Nexus went not. I wasn’t able to find a single line of documentation regarding using Nexus not run in standalone mode, even though it is available for download in war format. Having lost too much time with Track, I decided to use Artifactory instead. I already installed this Maven repository on a customer’s site and it gave me complete satisfaction.

Just don’t forget to configure Tomcat’s JVM with the -artifactory.home Java property. This tells Artifactory to use this folder for storing its internal files.

Yet, I ran into an error trying to use MySQL instead of JavaDB. Whatever the configuration I used, Artifactory was not able to use the password I provided to acces MySQL: I kept having Access denied for user 'artifactory'@'localhost' (using password: NO) messages. Having spent a considerable time trying to correct the thing, I went back to using JavaDB. Since Artifactory provides an repository backup functionality out-of-the-box, I can live with it.

Finally, having Artifactory up and running, I kept running into OutOfMemoryError: PermGenSpace errors. Tomcat was using 512Mb (which is a really low value for 3 important applications) but it has no effect on permanent generation space. I tried different -XX:MaxPermSize configurations without any luck. So in a bold move, I switched from a Sun JVM to a JRockit. Since then, I got no such errors anymore, though I have very high Garbage Collection time. Nothing a little more RAM and correct tuning can cure.

I wanted all my applications to use the same authentication component. I used OpenLDAP a couple of times before but it was always ankward to install and use (I’m no LDAP guru) . I browsed the web a little to find OpenSource and free (as a beer) LDAP servers and stumbled upon OpenDS. Easy to install, easy to configure and last but not least, easy to manage LDAP entities. Everything done through its GUI.

Configuring Apache and Tomcat to use LDAP is done through XML editing, for Artifactory and Hudson, it is done through the webapp. Artifactory even provides a test button to check whether the connection is correct.

All in all, I’m pretty happy with the components installed, save one. Trac does provide many functionalities but it has not its place in our infrastructure since Python is not my friend and it needs to many components I won’t be able to debug if we run across a problem. JIRA would be the bugtracker of choice, but since it is not free, it has no chance. I’ve already used MantisBT, and though it uses a PHP stack (and not Java), I feel more at ease with it. In its latest version, it can even integrate third party PHP Wikis. Any thought about it ?