Yeah, that is the part I am hoping someone will contribute :) I know I can write that myself. I also know it will be buggy and that I will have lots of trouble.
If you contribute this code, it would be a huge boon to Kafka. It is imo the primary use case for Kafka atm... if only the code gets into git. On Tue, Mar 20, 2012 at 3:04 PM, Niek Sanders <niek.sand...@gmail.com>wrote: > Russell, > > I'm actually in the process of writing a Java code to go from Kafka > messages to S3. I might be able to rip-out my application-specific > parts and share something later tonight. > > The biggest hassle is that you can't append to existing S3 files. So > unless you're planning on uploading each message as a separate S3 > object, this means you need message aggregation smarts on the Kafka > consumer / S3 uploader side of things. > > Best, > Niek > > > > > > > On Tue, Mar 20, 2012 at 12:56 PM, Russell Jurney > <russell.jur...@gmail.com> wrote: > > I wish someone would publish some source that writes events to S3. > > > > Russell Jurney > > twitter.com/rjurney > > russell.jur...@gmail.com > > datasyndrome.com > > > > On Mar 20, 2012, at 11:20 AM, Dave Fayram <dfay...@gmail.com> wrote: > > > >> We've been successfully using Kafka on AWS as well, and JMX wise we > >> just use an SSH tunnel. > >> > >> In general, we've been very happy with the performance on AWS, which > >> some people have reservations about due to the I/O situation on most > >> Amazon boxes. > >> > >> On Tue, Mar 20, 2012 at 9:07 AM, Gautam Singaraju > >> <gautam.singar...@gmail.com> wrote: > >>> We are have been considering Kafka for a new Data Platform. Has someone > >>> used Kafka in AWS? If so, could you please share your experiences with > us? > >>> > >>> Thank you! > >>> --- > >>> Gautam > >> > >> > >> > >> -- > >> -- > >> Dave Fayram > >> dfay...@gmail.com > -- Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
