I'm running whats on here: http://github.com/ryanobjc/hadoop-cloudera
Which is basically the patches you sent me previously. I have a cluster I would be willing to test with... -ryan On Thu, Apr 15, 2010 at 11:54 AM, Todd Lipcon <t...@cloudera.com> wrote: > On Thu, Apr 15, 2010 at 10:51 AM, Kannan Muthukkaruppan <kan...@facebook.com >> wrote: > >> Todd: Good stuff! Have you tried this out on a HBase test too? >> >> > Not yet - I basically had a burst of inspiration last night around 11pm, > coded til I couldn't stay awake, and threw up a patch :) I hope to give it a > try today (and also update patch to trunk so I can get some reviews there) > > >> > I think with it on, we can practically turn off group commit and get >> quite a bit speedup. >> >> Will batching now happen underneath the DFSClient automatically now that >> syncs/appends are not serialized. >> >> > It's not quite batching, but essentially I think we get the same benefit. > While one thread is waiting for *its* packet/write to be synced, another > thread can still append to the stream and call sync for its own data. > > -Todd > > -----Original Message----- >> From: Todd Lipcon [mailto:t...@cloudera.com] >> Sent: Thursday, April 15, 2010 1:59 AM >> To: hbase-dev@hadoop.apache.org >> Subject: HDFS-895 hflush speedup >> >> Any of you out there who are running the 0.20+200+etc hadoop with 0.20 >> branch hbase want to try this patch? I think with it on, we can practically >> turn off group commit and get quite a bit speedup. >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
