Continuous Integration with Maven on a Git project

How to create a continuous integration pipe for any maven+git application ?

!! Updated !!  

Added SCM-Manager and some Android building dependencies to improve CI platform.

My last goal was to create and install a full working pipe of Continue Integration for my Java « mavenized » and « gitified » projects.

Using our good old pals Jenkins and Sonar, with my older pal ubuntu in a VM, we were able to create a « Ready to use Integration machine ».

So… action !

First Step : VM and Ubuntu 12.04

1. Get VirtualBox

$ sudo apt-get install virtualbox

2. Download ISO image of Ubuntu Server

download the 32bits or 64bits ISO image from http://www.ubuntu.com/download/server (depending on your own local machine supposed to run this VM !)

3. Create a new VM to install the downloaded ISO

set 1GBytes of memory, to start a small project this must met just right.

create a new user with jenkins/jenkins as user/password with an administrator profile.

Console

afeter installation, you may have some issue about console display and encoding. Default is UTF-8, but maybe you want something else.

to perform some changes on console configuration, you have to run the special command:

$ sudo dpkg-reconfigure console-setup

then , just follow dialog’s screen to perform the right choices.

Keyboard

Each time I’ve installed from the ubuntu server distro, I had the wrong keyboard setting !

So to configure your own keyboard, accorfding to your country, and your language, just execute the following line :

$ sudo dpkg-reconfigure keyboard-config

And choose your country, your keyboard disposition, your keyboard variant.

Remote with SSH ?

If you plan to access your server through telnet, you must install the openssh-server package. Nothing easyest than apt-get command:

$ sudo apt-get install openssh-server

Just wait and try to connect from your prefered telnet/ssh client to your brand new server (don’t forget to retrieve IP address of the VM ; ifconfig is your pal, to be able to connect 😉

to connect with your just created user (the one created during the VM installation)

1. from a Windows worstation :

putty.exe" -ssh [username]@[ip or server name]

1. from a linux workstation :

$ ssh [username]@[ip or server name]

example:

$ ssh [username]@[ip or server name]
The authenticity of host '[ip or server name] ([ip or server name])' can't be established.
ECDSA key fingerprint is 72:99:a7:5c:dd:0d:52:84:48:75:d0:43:0c:ac:8c:e1.Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ip or server name]' (ECDSA) to the list of known hosts.[username]@[ip or server name]'s password: 
Welcome to Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-29-generic x86_64) 
* Documentation:  https://help.ubuntu.com/
Last login: Wed Dec 12 05:33:42 2012
[username]@[hostname of server]:~$

You’ll now have to acces to your server, certificate now added to the authentified ones list.

Second Step : Java plateform

Login to your brand new virtual server and follow the next lines.

1. Java 6

Install Java6 SDK from ubuntu repositories:

$ sudo apt-get install openjdk-6-jdk

Verify installation with the following command line

$ java -version

You must get the log bellow :

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.4) (6b24-1.11.4-1ubuntu0.12.04.1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Now we need to set JAVA_HOME. As our server is a java dedicated one, we can add the JAVA_HOME environement variable defintion to the /etc/environment file

just edit the /etc/environment file and add at bottom the line bellow :

JAVA_HOME="/usr/lib/jvm/default-java"

2. Maven 3.0.4

Download from http://maven.apache.org/download.html and unzip it in the /opt/tools/maven/3.0.4 path

Note:
You must install the unzip package to perform unarchiving maven tool: sudo apt-get install unzip.

Set full access to folder :

$ sudo mkdir -p /opt/tools/java/maven/3.0.4
$ unzip apache-maven-3.0.4.zip
$ mv apache-maven-3.0.4/* /opt/tools/java/maven/3.0.4
$ chown -R root:root /opt/tools/maven
$ chmod -R 755 /opt/tools/maven/

add maven to yout system path by adding the 2 lines bellow to the /etc/environement file

MVN_HOME='/opt/tools/maven/3.0.4/'
MVN_REPO='/opt/tools/maven/3.0.4/repository/'

And now, add a symbolic link to mvn command:

$ sudo ln -s /opt/tools/maven/3.0.4/bin/mvn /usr/bin/mvn

then a vm reboot :

sudo reboot

and verify installation of java and mvn

$ mvn -v

You must obtain :

Apache Maven 3.0.4 (r1232337; 2012-01-17 09:44:56+0100)
Maven home: /opt/tools/maven
Java version: 1.6.0_24, vendor: Sun Microsystems Inc.
Java home: /usr/lib/jvm/java-6-openjdk-amd64/jre
Default locale: fr_FR, platform encoding: UTF-8
OS name: "linux", version: "3.2.0-31-generic", arch: "amd64", family: "unix"

Third Step : Jenkins and Sonar

1. Install Jenkins

Add the APT repository from http://pkg.jenkins-ci.org/debian/

$ wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -

Add the following line to /etc/apt/sources.list

$ deb http://pkg.jenkins-ci.org/debian binary/

and the execute the following lines into a terminal:

$ sudo apt-get update
$ sudo apt-get install jenkins

Verify installation with your favorite browser and open the url “http://[server ip or name]:8080/ »

OK, now to perform last step , Tomcat7 installation, you must change the default jenkins listening port. To do such ting, you must edit the file /etc/default/jenkins and then modify lines HTTP_PORT=8080 to HTTP_PORT=8081, and add the --prefix=$PREFIX on the JENKINS_ARGS definition:

...# port for HTTP connector (default 8080; disable with -1)
HTTP_PORT=8081
# port for AJP connector (disabled by default)
AJP_PORT=-1
...
JENKINS_ARGS="--webroot=/var/cache/jenkins/war --prefix=$PREFIX --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT"

then stop and start the service :

$ sudo service jenkins stop
$ sudo service jenkins start

Verify that port change with opening the url “http://[server ip or name]:8081/jenkins”

1.b. Update jenkins war manually

To be able to install an updated version, just download the jenkins.war from the official download page: http://mirrors.jenkins-ci.org/war/latest/jenkins.war to your prefered location (e.g. : ~/).

Then, stop the jenkins service (if it’s already started), and move the last verion of the war file to /usr/share/jenkins/jenkins.war

$ sudo service jenkins stop
$ sudo mv jenkins.war /usr/share/jenkins/jenkins.war
$ sudo service jenkins start

Ok, Jenkins is now updated !

2. Download and setting of Sonar

Download Sonar from http://www.sonarsource.org/downloads/, get the last ZIP version (here is the 3.3.2) unzip it into /opt/tools/sonar/3.3.2/, then change rights on the folder:

$ wget http://dist.sonar.codehaus.org/sonar-3.3.2.zip
$ unzip sonar-3.3.2.zip
$ sudo mkdir -p /opt/tools/sonar/3.3.2
$ mv sonar-3.3.2/* /opt/tools/sonar/3.3.2/
$ sudo chown -R root:root /opt/tools/
$ sudo chmod -R 755 /opt/tools/sonar/3.3.2/

and start Sonar with a :

$ /opt/tools/sonar/3.2.1/bin/sonar.sh start

Verify Sonar web access crowling the http://server_ip_or_name:9000/ url.

According to the last step, installing Tomcat7 with a SCM manager and a Nexus, we need to change the default listening port. go to /opt/tools/sonar/3.3.2./conf/sonar.properties file and edit the following lines, and set sonar.web.port=8082. We willa lso need to change database configruation to switch to MySQL.

...
#---------------------------------------------------------
# WEB SETTINGS - STANDALONE MODE ONLY
# These settings are ignored when the war file is deployed to a JEE server.
#---------------------------------------------------------
# Listen host/port and context path (for example / or /sonar). Default values a
sonar.web.host:                           [server-name]
sonar.web.port:                           8082
sonar.web.context:                        /sonar

Stop and restart the Sonar service with a sudo service sonar restart, and browse the http://server_ip_or_name:8082/sonar, you might discover the Sonar welcome page.

(Optional) Configure Sonar as a service

make a symbolic link on the just installed sonar.sh file:

$ sudo ln -s /opt/tools/sonar/3.3.2/bin/linux-x86-32/sonar.sh /usr/bin/sonar

Create the following file /etc/init.d/sonar and insert lines :

#!/bin/sh
/usr/bin/sonar $*

Then set unix execution right :

$ sudo chown root:root /etc/init.d/sonar
$ sudo chmod 755 /etc/init.d/sonar

and configure this file as a service :

$ sudo update-rc.d sonar defaults 98 02

Now to start or stop service, just try a

1. To start Sonar service:

$ sudo service sonar start

2. To restart Sonar service:

$ sudo service sonar restart

3. To stop Sonar service:

$ sudo service sonar stop

4. To get status for Sonar service:

$ sudo service sonar status

MySQL server (optional)

Install the MySQL server

$ sudo apt-get install mysql-server

and create a MySQL new database sonar :

$ mysql -u root -p
CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER 'sonar' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'%' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'localhost' IDENTIFIED BY 'sonar';
FLUSH PRIVILEGES;

Set the default password with your prefered value 🙂

Edit the sonar configuration file again (/opt/tools/sonar/3.3.2./conf/sonar.properties ) to change database: comment type H2 section and uncomment the MySQL section.

$ sudo gedit /opt/tools/sonar/3.3.2./conf/sonar.properties

...
# Comment the following line to deactivate the default embedded database.
#sonar.jdbc.url:                            jdbc:h2:tcp://localhost:9092/sonar
#sonar.jdbc.driverClassName:                org.h2.Driver
...
#----- MySQL 5.x/6.x
# Comment the embedded database and uncomment the following line to use MySQL
sonar.jdbc.url:   jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewri
# Optional properties
sonar.jdbc.driverClassName:      com.mysql.jdbc.Driver

And now restart sonar service with a sudo service sonar restart and try to browser http://%5Bserver ip or name]:8082/sonar.

Fourth Step: configuration

OK, then configure Jenkins to accept JDK, Maven and Sonar with the right bundle of plugins; choose the following list of plugins:

• git plugin, don’t forget to set the git command path to /usr/bin/git
• sonar plugin, set the sonart path to /opt/tools/sonar/3.2.1
• set the maven path to /opt/tools/maven/3.0.4

Optionaly you can add :

• github plugin to connect github to one of your projects
• *gith OAuth security*login (optional) to authenticate your Continuous Integration Server users with Github users.

Start : Create a Job !

1. Named your project

2. Select your maven version (the just installed 3.0.4 version)

3. Set the git repository (e.g. http://bitbucket.org/McGivrer/easywebapp.git 🙂

4. Activate Sonar in the optional post-build actions (just add Sonar as post action)

go back to the Jenkins home page and then press the magic Build button !

Wait…

Done !

Ready to read build state and tests results ?

^{figure 1 – Jenkins displaying build results}

^{figure 2 – Sonar displaying project analyses with the « Time Machine » view}

Very useful tricks

To perform good build and better results, it’s better to add some additional plugins to Jenkins.

Here is the list of Beautiful things:

• OpenId is an authentication plugin to perform user authentication with an OpenID system. Easily you will be able to connect to Google Account with the specific openId option in the Jenkins administration/configuration panel, using the specific url like http://www.google.com/accounts/o8/id/.

• GoogleCalendarPlugin will publish build results to a specific Google Calendar. Very useful to analyze build stability over the time. Configuration is really simple, just create a Google Calendar or get your already existing one, and from Parameters/general setting, get the XML private url and past it onto your Job configuration. Set authentication stuff and activate « All builds » options, that’s it !

Add a SCM Manager

To set a full services stack to our build server, we are going to add 2 new components:

1. A Source Configuration Manager with the SCM-Manager tool,
2. An Artificat Repository Manager with Nexus from Sonatype.

Tomcat 7 ?

Just install the ubuntu default tomcat server with the following commands:

$ sudo apt-get install tomcat7

Optionaly you can install the tomcat7-admin module to manage java application on the tomcat7 server.

SCM-Manager

First, download the SCM manager web app from scm-manager/wiki/download/scm-webapp-1.22.war section on the bitbucket repository wiki.

and the copy this .war file to /var/lib/tomcat7/webapps/scm.war

$ sudo cp ./scm-webapp-1.22.war /var/lib/tomcat7/webapps/scm.war

Just wait some minutes (depends of your machine processor/ram/hd performances), and try http://localhost:8080/scm/. you must discover the welcome page of SCM Manager.

Note:
The default scm user account is scmadmin/scmadmin.

Add Artifact Repository

What would be an integration server without an artifact repository ?

Nexus OSS

So please, try and install the Nexus appklciation OSS edition from sonatype download page : http://www.sonatype.org/nexus/go, or by shell script

$ wget http://www.sonatype.org/downloads/nexus-2.2-01.war
$ sudo cp nexus-2.2-01.war /var/lib/tomcat7/webapps/nexus.war

Ok, just try to browse http://localhost:8080/nexus

Android build ?

We need to build Android application in our continuous integration pipe ? No problem, we just have to add the android-sdk-linux kit.

First download the kit :

$ wget http://dl.google.com/android/android-sdk_r21.0.1-linux.tgz
$ tar -zxvf android-sdk_r21.0.1-linux.tgz
$ sudo mkdir -p /opt/tools/android-sdk-linux/r21.0.1
$ sudo mv android-sdk_r21.0.1-linux/*  /opt/tools/android-sdk-linux/r21.0.1/
$ sudo chown -R root:root /opt/tools/android-sdk-linux
$ sudo chmod -R 755 /opt/tools/android-sdk-linux

Go and edit again the /etc/environment file to add the ANDROID_HOME variable:

sudo nano /etc/environment

and add the following line at end of file :

ANDROID_HOME='/opt/tools/android-sdk-linux/r21.0.1'

Last step for android is to install the platform tools :

/tools/android update sdk --no-ui --filter platform-tool

and the Jenkins Andoid plugin and configure some things to make it run (see http://code.google.com/p/maven-android-plugin/wiki/GettingStarted for more information).

Conclusion

Ok, now you are ready to perform great Java and Android application continuous integration.

Have fun !

McG.