@Matthew - that's an interesting idea, we haven't explored it yet. I'm sure it would be better, but then some backup option is needed or you have to be OK with losing the data, right? Or do you rely on replication or something?
On Thu, Aug 9, 2012 at 4:47 PM, Matthew Rathbone <matt...@foursquare.com> wrote: > @Evan -- how does performance vary if you use the ephemeral drives instead > of EBS? > > On Thu, Aug 9, 2012 at 6:41 PM, Evan Chan <e...@ooyala.com> wrote: > >> We run Kafka instances on AWS and I had posted performance #'s before. >> We achieve around 20MB/sec IO throughput, with the Kafka messages >> stored in EBS, using reiserfs. I wonder how much faster we could get >> though, using 1) ext4, and 2) SSDs. >> >> -Evan >> >> >> On Thu, Aug 9, 2012 at 11:25 AM, Erich Nachbar <er...@nachbar.biz> wrote: >> > We are running 3 Kafka instances on Proxmox (container based >> > virtualization, with typical 1-5% overhead) in production. >> > >> > While I have not benchmarked Kafka on it, other systems I performance >> > tested before on Proxmox showed no measurable performance difference. It >> > runs OpenVZ under the hood for its containers, which is loved by all the >> > cheap $15/mo hosters for its efficiency. >> > >> > On Thu, Aug 9, 2012 at 8:16 AM, Jay Kreps <jay.kr...@gmail.com> wrote: >> > >> >> In the past I had used AWS for some performance testing. I didn't see >> >> anything too unexpected other than the usual overhead on first write and >> >> general mediocre AWS network performance. >> >> >> >> -jay >> >> >> >> On Wed, Aug 8, 2012 at 11:22 AM, Casey Sybrandy < >> >> casey.sybra...@six3systems.com> wrote: >> >> >> >> > Hello, >> >> > >> >> > Has anyone deployed their kafka brokers within a VM? Are their any >> >> > performance issues? I'd expect that once we start hitting it hard, >> the >> >> > disk writes will be an issue, however I wanted to get some additional >> >> > information. >> >> > >> >> > Thanks. >> >> > >> >> > Casey >> >> > >> >> >> >> >> >> -- >> -- >> Evan Chan >> Senior Software Engineer | >> e...@ooyala.com | (650) 996-4600 >> www.ooyala.com | blog | @ooyala >> > > > > -- > Matthew Rathbone > Foursquare | Software Engineer | Server Engineering Team > matt...@foursquare.com | @rathboma <http://twitter.com/rathboma> | > 4sq<http://foursquare.com/rathboma> -- -- Evan Chan Senior Software Engineer | e...@ooyala.com | (650) 996-4600 www.ooyala.com | blog | @ooyala
