On Sat, Dec 4, 2010 at 2:57 PM, Vladimir Rodionov <[email protected]>wrote:
> From my own experiments performance difference is huge even on > sequential R/W operations (up to 300%) when you do local File I/O vs HDFS > File I/O > > Overhead of HDFS I/O is substantial to say the least. > > Much of this is from checksumming, though - turn off checksums and you should see about a 2x improvement at least. -Todd > Best regards, > Vladimir Rodionov > Principal Platform Engineer > Carrier IQ, www.carrieriq.com > e-mail: [email protected] > > ________________________________________ > From: Todd Lipcon [[email protected]] > Sent: Saturday, December 04, 2010 12:30 PM > To: [email protected] > Subject: Re: Local sockets > > Hi Leen, > > Check out HDFS-347 for more info on this. I hope to pick this back up in > 2011 - in 2010 we mostly focused on stability above performance in HBase's > interactions with HDFS. > > Thanks > -Todd > > On Sat, Dec 4, 2010 at 12:28 PM, Leen Toelen <[email protected]> wrote: > > > Hi, > > > > has anyone tested the performance impact (when there is a hdfs > > datanode and a hbase node on the same machine) of using unix domain > > sockets communication or shared memory ipc using nio? I guess this > > should make a difference on reads? > > > > Regards, > > Leen > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
