Hi folks. I was considering sharing some ideas / scripts about how we use bigtop to test stuff either as code (i.e shell script which invokes various maven tasks and copies files) to an examples/ folder in the main repo, or else, write a bigtop blog post (i.e. that walks through the script and explains the tasks).
Not sure if its worth an "official" code commit, or an official blog post or wether I should just dump it on my personal blog and move on ... But let me know ! :)... Here is the basic contents of what the blog post or example script will walk through. 0) Exports environmental variables 1) builds test-artifacts using standard maven online snapshot mode to get dependencies and stuff. 2) then rebuilds the artifacts in offline mode to overlay our test-artifact customizations on top of the work done in the previous step. 3) cp's some pom.xml files into the test-execution/test-artifacts folders, including a custom iTest groovy snippet to run some pig shell tests. 4) starts flume 5) runs the test-execution task Overall, it took a while for me to get it fine tuned to be a good cluster test, but now I'm quite happy with it. I think it could be a useful artifact to anyone that wants to run bigtop smoke tests against a non standard hadoop deployment (i.e. anyone like me ... :) ) I'm leaning towards sharing this as a blog post since the script is built to run against a heavily customized bigtop fork of mine ( github.com/jayunit100/bigtop) which is quite raw and needs some cleanup.... but if someone in the bigtop community wants to collaborate, maybe we can streamline the example script so that people can use it as a template for customizing bigtop smokes to their own needs. Another option, possibly, is we could have a google hangout in the bigtop community where we share the way we use bigtop to do different tasks and provide each other with feedback. -- Jay Vyas http://jayunit100.blogspot.co <http://jayunit100.blogspot.com>
