[
https://issues.apache.org/jira/browse/KAFKA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14202396#comment-14202396
]
Ewen Cheslack-Postava commented on KAFKA-1173:
----------------------------------------------
I took [~joestein]'s patch and built it out a bit. Here's what I added:
* Add EC2 support if you install the vagrant-aws plugin.
* Uses vagrant-hostmanager to manage VM hostnames and configure /etc/hosts.
This gives the machines names like zk1, broker2. It can optionally configure
the host machines /etc/hosts as well, which is particularly nice when you're
dealing with EC2 or running a producer/consumer process on the host.
* Optionally uses vagrant-cachier to avoid duplicated downloads across VMs.
Works for the normal things vagrant-cachier normally supports (apt-get,
rubygems, etc.) as well as specifically for the JDK setup, which requires some
customized code because it's downloaded from Oracle.
* Local config support via Vagrantfile.local (which is just interpreted as Ruby
code). Lets you easily override settings without accidentally checking in those
changes.
* Provide 3 types of machines: zookeeper, broker, and "worker", where workers
just have the code synced but don't start any services. These are useful for
running other processes (producers, consumers, admin operations) or you can use
only worker nodes to run system tests.
* Supports zookeeper ensembles instead of just a single Zookeepr node.
Limitations:
* Scripts are still only setup for Debian/Ubuntu systems. They could easily be
generalized.
* I haven't figured out the best way to handle memory settings. They're just
fixed at a bit above 1G right now because of the settings in the scripts that
start services.
* vagrant-hostmanager is now a hard dependency. Unfortunate, but it seems like
the best solution if the Vagrantfile is going to work with any Vagrant
providers besides local VMs.
* Some default values need ongoing maintenance (default EC2 AMI setting, for
example)
Having a Vagrantfile is nice for testing purposes and a patch like this is low
risk since it's not touching any of the main code.
> Using Vagrant to get up and running with Apache Kafka
> -----------------------------------------------------
>
> Key: KAFKA-1173
> URL: https://issues.apache.org/jira/browse/KAFKA-1173
> Project: Kafka
> Issue Type: Improvement
> Reporter: Joe Stein
> Attachments: KAFKA-1173.patch, KAFKA-1173_2013-12-07_12:07:55.patch
>
>
> Vagrant has been getting a lot of pickup in the tech communities. I have
> found it very useful for development and testing and working with a few
> clients now using it to help virtualize their environments in repeatable ways.
> Using Vagrant to get up and running.
> For 0.8.0 I have a patch on github https://github.com/stealthly/kafka
> 1) Install Vagrant [http://www.vagrantup.com/](http://www.vagrantup.com/)
> 2) Install Virtual Box
> [https://www.virtualbox.org/](https://www.virtualbox.org/)
> In the main kafka folder
> 1) ./sbt update
> 2) ./sbt package
> 3) ./sbt assembly-package-dependency
> 4) vagrant up
> once this is done
> * Zookeeper will be running 192.168.50.5
> * Broker 1 on 192.168.50.10
> * Broker 2 on 192.168.50.20
> * Broker 3 on 192.168.50.30
> When you are all up and running you will be back at a command brompt.
> If you want you can login to the machines using vagrant shh <machineName> but
> you don't need to.
> You can access the brokers and zookeeper by their IP
> e.g.
> bin/kafka-console-producer.sh --broker-list
> 192.168.50.10:9092,192.168.50.20:9092,192.168.50.30:9092 --topic sandbox
> bin/kafka-console-consumer.sh --zookeeper 192.168.50.5:2181 --topic sandbox
> --from-beginning
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
