Russell, I'm actually in the process of writing a Java code to go from Kafka messages to S3. I might be able to rip-out my application-specific parts and share something later tonight.
The biggest hassle is that you can't append to existing S3 files. So unless you're planning on uploading each message as a separate S3 object, this means you need message aggregation smarts on the Kafka consumer / S3 uploader side of things. Best, Niek On Tue, Mar 20, 2012 at 12:56 PM, Russell Jurney <russell.jur...@gmail.com> wrote: > I wish someone would publish some source that writes events to S3. > > Russell Jurney > twitter.com/rjurney > russell.jur...@gmail.com > datasyndrome.com > > On Mar 20, 2012, at 11:20 AM, Dave Fayram <dfay...@gmail.com> wrote: > >> We've been successfully using Kafka on AWS as well, and JMX wise we >> just use an SSH tunnel. >> >> In general, we've been very happy with the performance on AWS, which >> some people have reservations about due to the I/O situation on most >> Amazon boxes. >> >> On Tue, Mar 20, 2012 at 9:07 AM, Gautam Singaraju >> <gautam.singar...@gmail.com> wrote: >>> We are have been considering Kafka for a new Data Platform. Has someone >>> used Kafka in AWS? If so, could you please share your experiences with us? >>> >>> Thank you! >>> --- >>> Gautam >> >> >> >> -- >> -- >> Dave Fayram >> dfay...@gmail.com
