Autobuild
- Autobuild
- So I heard we have this Autobuild server, where is it?
- What does it do?
- Wow it's all automatic, do I have to do anything?
- Oh! I've heard of that! Is it really as unpleasant as people say?
- Would you tell me more about pre-commit?
- Okay, If I do this I won't break the build. What else?
- How does it work?
- When things go wrong
So I heard we have this Autobuild server, where is it?
- The main page is Open Anzo Hudson
What does it do?
The Autobuild server runs two somewhat similar builds:
"openanzo" - the "continuous integration" build
- polls subversion, kicks off a build when it detects 1 (or more) commits
- make's sure the commit is sane
- compiles the code
- runs all the unit tests
- packages and assembles the code into a build
- runs the regressions tests against the build (using an in memory database called h2 as the SQL RDB)
"openanzo-nightly" - the slow build
- set to run overnight
- does everything the "openanzo" build does plus:
- tags the current revision (tags/openanzo-nightly/SNAPSHOT-<date-time>)
- deploys updated snapshots to Downloads
- deploys updated snapshots to MavenRepositories
There builds like that for trunk and also for stable branches.
Wow it's all automatic, do I have to do anything?
Yes! Don't break the build.
Oh! I've heard of that! Is it really as unpleasant as people say?
Yup, breaking the build is no fun.
Fortunately, it's completely avoidable. Because the build process is not bound to the autobuild server. A developer can run the same build their own machine that the autobuild server will run.
First, install Maven 2.
Once you've installed Maven 2 you can call 'pre-commit':
cd openanzo-maven mvn clean mvn verify
Would you tell me more about pre-commit?
Sure, pre-commit is part of a series of steps a develop should take before checking in code:
- check that the autobuild is not broken (ideally, you should also avoid committing while a build is in progress on the server)
- update from svn
- run pre-commit
- follow commit etiquette for entering messages, linking to tickets, etc..
Okay, If I do this I won't break the build. What else?
Next, help make the build better.
The build is only as good as the tests it runs. Write tests. Found a bug? Write a test so it cannot ever get back into the build undetected. Working on some tricky code? Write a test while you develop so the code continues to work the way you intended.
How does it work?
The Autobuild server is running Hudson. But most of the work is done by Maven 2. Here's the task breakdown:
Hudson
- decides when to run a build
- polls subversion
- runs timed builds at correct time
- runs svn update before building
- kicks of Maven
- keeps track of maven log and unit test reports
- emails any failures
Maven
- runs code generators (jastor, jdbc-utils opgen, javacc)
- compiles code
- runs unit tests
- packages the code into jars
- assembles the jars into deploy-able builds
- runs regressions against the assembled builds
- Nightly build only:
- generates reports
- assembles and deploys builds
When things go wrong
Etiquette
If the build is broken:
- Don't commit new code.
- If you are one committers whose changes went into the broken build, stop everything and look into the problem. Fix the build.
- Communicate early and often. Send email letting everyone know that it's being investigated. If other peoples' commits are involved in the broken build, communicate with them to coordinate your efforts to fix the build.
Debugging broken builds
The Maven build places the output from failed unit tests inside the relevant project's target/surefire-reports directory. For example, openanzo-test/target/surefire-reports. Look there for help first. Also, the server log from the server that runs the regression tests is found at openanzo-maven/target/assembly/var/. Some tests use Selenium to run remote control a web browser instance. Selenium will write log information to openanzo-test/target/selenium. The regression tests in the openanzo-test project write all of their log output to openanzo-test/target/tests.log.
How do I run just a single unit/regression/integration test
If the tests are in the openanzo-test project, they are regression/integration tests, otherwise they are unit tests, the method for running these is different:
Running a single regression/integration test
Running tests from eclipse during development is recommended. However, there are cases where a test passes in eclipse but fails in maven, this can be due to differences in build trees or build directories that need to be fixed before committing. When fixing issues like this, it may be very helpful to only run a single test from maven, heres how:
First, run these commands only if you have changed the server code:
cd openanzo-maven mvn clean mvn -Dmaven.test.skip package
And then you can run the test repeatedly (only changes to the client code are recompiled):
cd openanzo-test mvn -f project.xml -Dtest=YourTestClass verify
Running a single unit test
This is pretty straight-forward, just run:
cd openanzo-projectcontainingtest mvn -f project.xml -Dtest=YourTestClass test
Common Problems
Anzo Regression Tests Fail with AbstractMethodError
This has been seen in both Windows and Linux using Maven 2.0.7 and 2.0.8 running on the Sun JDK 1.6.0_03 and 1.6.0_04. The typical exception in the test's surefire report looks something like:
java.lang.AbstractMethodError: org.slf4j.impl.SimpleLogger.isTraceEnabled()
at org.apache.commons.logging.impl.SLF4JLog.isTraceEnabled(SLF4JLog.java:59)
...
We believe this is related to using older versions of Maven on JDK 6.
The solution is to upgrade to the latest JDK 6 and Maven version. Check what your Maven installation is using by running mvn -version.
We have had consistent success with Maven 2.0.9 and Sun JDK 1.6.0_07. Presumably later versions continue to work. Using JDK 1.5 also works.
Anzo Regression Tests Fail with JMSException: Unknown primitive type
This appears to be caused by a problematic state of the Maven local repository. It typically appears as exceptions in the failing tests that look like this:
org.openanzo.common.exceptions.AnzoRuntimeException: ErrorCode[48:16407] CorrelationId[fcab9201-4bd1-43b3-a7b9-ba2616251c99] [COMBUS_ERROR] [SERVICE_CONTAINER] Unknown server exception executing request javax.jms.JMSException: Unknown primitive type: 0
at org.openanzo.client.ServerQuadStore.find(ServerQuadStore.java:80)
...
The fix for this is to clear you Maven local repository. In Windows, your local repository is found in C:\Documents and Settings\USER\.m2 where USER is your Windows account id. In Unix-like systems, the local repository is found in ~/.m2. Move the current repository out of the way (like rename it). Then run the build again. Note that this may take a long time (like an hour) as it downloads all of the dependencies during the build.
Random Tip: If you would rather keep your local repo somewhere else, create a file called settings.xml in your .m2 directory and fill it with this:
<settings> <localRepository>/absolute/path/to/my/desired/local/repo/location</localRepository> </settings>
Compilation Errors for Missing SLF4JLog.trace Methods
This has been seen in OS X using Maven 2.0.7 Sun JDK 1.6.0_04. We believe this is related to using older versions of Maven on JDK 6.
The solution is to upgrade to the latest JDK 6 and Maven version. Check what your Maven installation is using by running mvn -version.
We have had consistent success with Maven 2.0.9 and Sun JDK 1.6.0_07. Presumably later versions continue to work. Using JDK 1.5 also works.
Compilation Errors related to @Override
This is probably related to a Java 5 versus Java 6 issue. Java 6 allows @Override to be used in more situations than Java 5 did. Open Anzo must support Java 5 so such compilations failures must be repaired in the source. Typically, the fix is to simply remove the offending @Override annotations.
SQLite Version Error (Mac only)
SQLite version errors may be encountered when running versions of Firefox 3.5 that prevent the Selenium recorder from launching properly. You may run into the following error:
dyld: Library not loaded: /usr/lib/libsqlite3.dylib Referenced from: /System/Library/Frameworks/Security.framework/Versions/A/Security Reason: Incompatible library version: Security requires version 9.0.0 or later, but libsqlite3.dylib provides version 1.0.0
To work around this:
- Install (or revert to) Firefox 3.5.3
- Install the following version of sqlite: http://www.sqlite.org/sqlite-amalgamation-3.6.20.tar.gz
- Replace libsql3.dylib from within the Firefox application bundle: sudo cp /usr/lib/libsqlite3.dylib /Applications/Firefox.app/Contents/MacOS/
