Hi folks, It's been about a year since Streamlio joined Splunk and since then we've had a bit of forking with our BK branch. It has gotten to a stage where it's starting to be a problem for us, so we'd like to start to get things back in sync.
There are a couple of big chunks of work to come back. We've added a data integrity checker that replaces a lot of the functionality of autorecovery and allows us to run without a journal. We refactored the bookie to allow dependency injection. We've rewritten the entry logger to use direct I/O (allowing 2GBps writes per bookie). One other thing we've done is to change the build system to use gradle. The major driver for this was that maven is just slow, even before you start running tests. "mvn clean package -DskipTests" takes 4m30 on my laptop. "./gradlew clean jar" takes 40s. Subsequent builds on gradle are much much faster, as it does incremental building. Incremental building exists in maven, but it doesn't work. Gradle also handle multimodule projects better. If I make a change in bookkeeper-common, "./gradlew :bookkeeper-server:test" will pick up the change without having to explicitly "mvn install" the bookkeeper-common. In my opinion it's just a much nicer build system to work with. Even the poms it generates are better as they avoid dependency pollution. What are peoples opinions on moving BookKeeper to gradle (assuming I/splunk do the legwork)? If people are open to it, I'll submit a BP. Another thing that BK (and the whole ecosystem) is missing is structured logging. We also plan to add structured logging to BK in soon. This is a major motivator for converging the branches, as it touches a lot of places. Anyhow, any feedback appreciated. -Ivan
