On Thu, Apr 15, 2010 at 10:51 AM, Kannan Muthukkaruppan <kan...@facebook.com > wrote:
> Todd: Good stuff! Have you tried this out on a HBase test too? > > Not yet - I basically had a burst of inspiration last night around 11pm, coded til I couldn't stay awake, and threw up a patch :) I hope to give it a try today (and also update patch to trunk so I can get some reviews there) > > I think with it on, we can practically turn off group commit and get > quite a bit speedup. > > Will batching now happen underneath the DFSClient automatically now that > syncs/appends are not serialized. > > It's not quite batching, but essentially I think we get the same benefit. While one thread is waiting for *its* packet/write to be synced, another thread can still append to the stream and call sync for its own data. -Todd -----Original Message----- > From: Todd Lipcon [mailto:t...@cloudera.com] > Sent: Thursday, April 15, 2010 1:59 AM > To: hbase-dev@hadoop.apache.org > Subject: HDFS-895 hflush speedup > > Any of you out there who are running the 0.20+200+etc hadoop with 0.20 > branch hbase want to try this patch? I think with it on, we can practically > turn off group commit and get quite a bit speedup. > > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
