Re: HDFS-895 hflush speedup

2010-04-15 Thread Ryan Rawson
I'm running whats on here: http://github.com/ryanobjc/hadoop-cloudera Which is basically the patches you sent me previously. I have a cluster I would be willing to test with... -ryan On Thu, Apr 15, 2010 at 11:54 AM, Todd Lipcon wrote: > On Thu, Apr 15, 2010 at 10:51 AM, Kannan Muthukkaruppan

Re: HDFS-895 hflush speedup

2010-04-15 Thread Todd Lipcon
On Thu, Apr 15, 2010 at 10:51 AM, Kannan Muthukkaruppan wrote: > Todd: Good stuff! Have you tried this out on a HBase test too? > > Not yet - I basically had a burst of inspiration last night around 11pm, coded til I couldn't stay awake, and threw up a patch :) I hope to give it a try today (and

RE: HDFS-895 hflush speedup

2010-04-15 Thread Andrew Purtell
I'll try this out when I get a chance. Probably not until the weekend. This looks great. - Andy > From: Kannan Muthukkaruppan > Subject: RE: HDFS-895 hflush speedup > To: "hbase-dev@hadoop.apache.org" > Date: Thursday, April 15, 2010, 10:51 AM > Todd: Good st

RE: HDFS-895 hflush speedup

2010-04-15 Thread Kannan Muthukkaruppan
Todd: Good stuff! Have you tried this out on a HBase test too? > I think with it on, we can practically turn off group commit and get quite a > bit speedup. Will batching now happen underneath the DFSClient automatically now that syncs/appends are not serialized. regards, Kannan -Original