All sounds cool and doable. But didn't we delete contrib/deploy in favour of https://issues.apache.org/jira/browse/KAFKA-10 ?
I agree that something like https://github.com/pcmanus/ccm would be useful for bringing up local clusters for local testing. (And yeah, we should create a pid file instead of killing every running kafka) On 08/04/2011 09:17 PM, Jay Kreps wrote: > One thing I think would be really helpful would be to make it so you could > easily spawn up a cluster. Hadoop does this and it is nice. For companies > that don't have standardized deployment infrastructure this is really nice, > and also for testing of all kinds. Basically it would be nice if you could > make a list containing one host:port per line and there were scripts like > bin/kafka-cluster-deploy.sh machine-list.txt # rsync the kafka > directory and config around to the given list of machines > bin/kafka-cluster-start.sh machine-list.txt # ssh to each machine > in the list and start it > bin/kafka-cluster-stop.sh machine-list.txt # ssh to each machine > and kill the kafka process > bin/kafka-cluster-delete.sh machine-list.txt # ssh around to each > machine in the list and delete the code and log directory > > To do this I think you would need a way to override properties on the > command line. This would make it so you could rsync out the kafka code to a > bunch of machines but give a different node id and (if needed) port. I think > something like "--property-name value" to give properties that override what > is in the properties file would work. > > One other minor and unrelated issue we have is that our stop script actually > kills all the kafka processes on the machine, which means if you are trying > to run multiple nodes on the same machine it is a little dangerous. > > -Jay > > On Thu, Aug 4, 2011 at 4:59 PM, Chris Burroughs > <chris.burrou...@gmail.com>wrote: > >> I've been looking at how kafka is packaged and the provided scripts for >> running it. There are a few things I want to improve: it should be >> easier to run in the foreground or background, convincing java to keep >> the executable bits set, and reasonable log4j defaults. There are also >> a few plain old bugs detailed in KAFKA-81 that need to be fixed. >> >> If there are other things that would make the results of ./sbt package >> or release-zip more useful, or the bash scripts easier to work with, >> please let me know. >> >> >> Thanks, >> Chris Burroughs >> >
