>From my own experiments performance difference is huge even on sequential R/W operations (up to 300%) when you do local File I/O vs HDFS File I/O
Overhead of HDFS I/O is substantial to say the least. Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: [email protected] ________________________________________ From: Todd Lipcon [[email protected]] Sent: Saturday, December 04, 2010 12:30 PM To: [email protected] Subject: Re: Local sockets Hi Leen, Check out HDFS-347 for more info on this. I hope to pick this back up in 2011 - in 2010 we mostly focused on stability above performance in HBase's interactions with HDFS. Thanks -Todd On Sat, Dec 4, 2010 at 12:28 PM, Leen Toelen <[email protected]> wrote: > Hi, > > has anyone tested the performance impact (when there is a hdfs > datanode and a hbase node on the same machine) of using unix domain > sockets communication or shared memory ipc using nio? I guess this > should make a difference on reads? > > Regards, > Leen > -- Todd Lipcon Software Engineer, Cloudera
