Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-04 Thread Colin
Hello, We use docker for kafka on vm's with both nas and local disk. We mount the volumes externally. We havent had many problems at all, and a restart has cleared any issue. We are on .8.1 We are also started to deploy to aws. -- Colin +1 612 859 6129 Skype colin.p.clark On Mar 4,

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-04 Thread Otis Gospodnetic
Hi, On Fri, Feb 27, 2015 at 1:36 AM, James Cheng jch...@tivo.com wrote: Hi, I know that Netflix might be talking about Kafka on AWS at the March meetup, but I wanted to bring up the topic anyway. I'm sure that some people are running Kafka in AWS. I'd say most, not some :) Is anyone

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-01 Thread Ewen Cheslack-Postava
On Fri, Feb 27, 2015 at 8:09 PM, Jeff Schroeder jeffschroe...@computer.org wrote: Kafka on dedicated hosts running in docker under marathon under Mesos. It was a real bear to get working, but is really beautiful once I did manage to get it working. I simply run with a unique hostname

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-03-01 Thread Joseph Lawson
Side question: why run kafka on docker for aws? Is the docker config being used for configuration management? Are there more systems running on the instance other than kafka? Sent by Outlookhttp://taps.io/outlookmobile for Android On Sun, Mar 1, 2015 at 1:10 PM -0800, Ewen Cheslack-Postava

Re: If you run Kafka in AWS or Docker, how do you persist data?

2015-02-27 Thread Jeff Schroeder
Kafka on dedicated hosts running in docker under marathon under Mesos. It was a real bear to get working, but is really beautiful once I did manage to get it working. I simply run with a unique hostname constraint and number of instances = replication factor. If a broker dies and it isn't a

If you run Kafka in AWS or Docker, how do you persist data?

2015-02-26 Thread James Cheng
Hi, I know that Netflix might be talking about Kafka on AWS at the March meetup, but I wanted to bring up the topic anyway. I'm sure that some people are running Kafka in AWS. Is anyone running Kafka within docker in production? How does that work? For both of these, how do you persist data?