Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-04 Thread Colin
Hello, We use docker for kafka on vm's with both nas and local disk. We mount the volumes externally. We havent had many problems at all, and a restart has cleared any issue. We are on .8.1 We are also started to deploy to aws. -- Colin +1 612 859 6129 Skype colin.p.clark On Mar 4,

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-04 Thread Otis Gospodnetic
Hi, On Fri, Feb 27, 2015 at 1:36 AM, James Cheng jch...@tivo.com wrote: Hi, I know that Netflix might be talking about Kafka on AWS at the March meetup, but I wanted to bring up the topic anyway. I'm sure that some people are running Kafka in AWS. I'd say most, not some :) Is anyone

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-01 Thread Ewen Cheslack-Postava
On Fri, Feb 27, 2015 at 8:09 PM, Jeff Schroeder jeffschroe...@computer.org wrote: Kafka on dedicated hosts running in docker under marathon under Mesos. It was a real bear to get working, but is really beautiful once I did manage to get it working. I simply run with a unique hostname

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-01 Thread Joseph Lawson
Side question: why run kafka on docker for aws? Is the docker config being used for configuration management? Are there more systems running on the instance other than kafka? Sent by Outlookhttp://taps.io/outlookmobile for Android On Sun, Mar 1, 2015 at 1:10 PM -0800, Ewen Cheslack-Postava

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-02-27 Thread Jeff Schroeder
Kafka on dedicated hosts running in docker under marathon under Mesos. It was a real bear to get working, but is really beautiful once I did manage to get it working. I simply run with a unique hostname constraint and number of instances = replication factor. If a broker dies and it isn't a