Hi, over the last three weeks we've been working hard on making Crunch ready for CI. It turned out to be much more work than initially expected and I cursed myself more than once for starting this. In the end it took four people and many patches to pull this off (thanks a lot for your help, Josh, Rahul, and Gabriel!). In case you're interested, these were the issues that lead us to CI readiness:
* CRUNCH-8: Refactor Crunch's build into a multi-module Maven project * CRUNCH-17: Split unit tests from integration tests * CRUNCH-18: Improve build stability * CRUNCH-21: Make temporary directory configurable * CRUNCH-24: Make test suite suiteable for continuous integration * CRUNCH-26: Make Scrunch tests suiteable for continuous integration We can now start with basic jobs to detect broken commits by running the unit and integration test suites. I suggest starting with an on-commit build using "mvn package" and a nightly build using "mvn verify" (which runs the integration test suite). Running the integration tests on-commit would be great but may be too expensive. To get started we need access to Apache's Jenkins instance [1] which apparently only PMC chairs can grant [2]. I'm new to Apache processes, but perhaps a mentor can help us out here? Next steps: CI gives us the ability to regularly run reports to check code quality. We already started preparations for this (Maven site setup, checkstyle config and basic reports), but I think the best thing would be to use Apache's Sonar instance [3]. I use Sonar a lot at work, it bundles all kinds of code metrics in a nice interface and points out where refactoring time is best spent. It's the most convenient way of tracking code quality over time and catches many issues human reviewers usually overlook. Additional to reports, it's also quite common to deploy nightly SNAPSHOT builds to a Maven repository and to publish up-to-date Maven sites. I'm not sure if this is possible at Apache for legal reasons though. Please let me know where you think we should go next. I know this whole CI episode has been a nuisance, so thanks again for your help. Regards, Matthias [1] https://builds.apache.org [2] http://wiki.apache.org/general/Jenkins [3] https://analysis.apache.org
