It's a question we're asking ourselves at the moment too :-), was hoping you might have had some insight.
Out of curiosity - How are your EBS drives set up? Are they on a raid-0? On Thu, Aug 9, 2012 at 6:55 PM, Evan Chan <e...@ooyala.com> wrote: > @Matthew - that's an interesting idea, we haven't explored it yet. > I'm sure it would be better, but then some backup option is needed or > you have to be OK with losing the data, right? Or do you rely on > replication or something? > > On Thu, Aug 9, 2012 at 4:47 PM, Matthew Rathbone <matt...@foursquare.com> > wrote: > > @Evan -- how does performance vary if you use the ephemeral drives > instead > > of EBS? > > > > On Thu, Aug 9, 2012 at 6:41 PM, Evan Chan <e...@ooyala.com> wrote: > > > >> We run Kafka instances on AWS and I had posted performance #'s before. > >> We achieve around 20MB/sec IO throughput, with the Kafka messages > >> stored in EBS, using reiserfs. I wonder how much faster we could get > >> though, using 1) ext4, and 2) SSDs. > >> > >> -Evan > >> > >> > >> On Thu, Aug 9, 2012 at 11:25 AM, Erich Nachbar <er...@nachbar.biz> > wrote: > >> > We are running 3 Kafka instances on Proxmox (container based > >> > virtualization, with typical 1-5% overhead) in production. > >> > > >> > While I have not benchmarked Kafka on it, other systems I performance > >> > tested before on Proxmox showed no measurable performance difference. > It > >> > runs OpenVZ under the hood for its containers, which is loved by all > the > >> > cheap $15/mo hosters for its efficiency. > >> > > >> > On Thu, Aug 9, 2012 at 8:16 AM, Jay Kreps <jay.kr...@gmail.com> > wrote: > >> > > >> >> In the past I had used AWS for some performance testing. I didn't see > >> >> anything too unexpected other than the usual overhead on first write > and > >> >> general mediocre AWS network performance. > >> >> > >> >> -jay > >> >> > >> >> On Wed, Aug 8, 2012 at 11:22 AM, Casey Sybrandy < > >> >> casey.sybra...@six3systems.com> wrote: > >> >> > >> >> > Hello, > >> >> > > >> >> > Has anyone deployed their kafka brokers within a VM? Are their any > >> >> > performance issues? I'd expect that once we start hitting it hard, > >> the > >> >> > disk writes will be an issue, however I wanted to get some > additional > >> >> > information. > >> >> > > >> >> > Thanks. > >> >> > > >> >> > Casey > >> >> > > >> >> > >> > >> > >> > >> -- > >> -- > >> Evan Chan > >> Senior Software Engineer | > >> e...@ooyala.com | (650) 996-4600 > >> www.ooyala.com | blog | @ooyala > >> > > > > > > > > -- > > Matthew Rathbone > > Foursquare | Software Engineer | Server Engineering Team > > matt...@foursquare.com | @rathboma <http://twitter.com/rathboma> | > > 4sq<http://foursquare.com/rathboma> > > > > -- > -- > Evan Chan > Senior Software Engineer | > e...@ooyala.com | (650) 996-4600 > www.ooyala.com | blog | @ooyala > -- Matthew Rathbone Foursquare | Software Engineer | Server Engineering Team matt...@foursquare.com | @rathboma <http://twitter.com/rathboma> | 4sq<http://foursquare.com/rathboma>
