Re: HDFS on trunk is now quite slow

2011-07-06 Thread Todd Lipcon
everting that patch as well to see if it's the cause. -Todd > > -Original Message- > > From: Todd Lipcon [mailto:t...@cloudera.com] > > Sent: Wednesday, July 06, 2011 11:12 AM > > To: hdfs-dev@hadoop.apache.org > > Subject: Re: HDFS on trunk is now quite slow

RE: HDFS on trunk is now quite slow

2011-07-06 Thread Eric Payne
@cloudera.com] > Sent: Wednesday, July 06, 2011 11:12 AM > To: hdfs-dev@hadoop.apache.org > Subject: Re: HDFS on trunk is now quite slow > > On Wed, Jul 6, 2011 at 9:00 AM, Eric Payne wrote: > > > I will attempt to recreate the tests on 20.203. > > > > Current

Re: HDFS on trunk is now quite slow

2011-07-06 Thread Todd Lipcon
others to reproduce? How many concurrent threads access the NN? If you jstack the NN do you see some particular lock causing lots of contention? -Todd > > -Original Message- > > From: Todd Lipcon [mailto:t...@cloudera.com] > > Sent: Wednesday, July 06, 2011 10:26 AM

RE: HDFS on trunk is now quite slow

2011-07-06 Thread Eric Payne
time: 47ms 9ms Average write close time: 658ms 100ms Thanks, -Eric > -Original Message- > From: Todd Lipcon [mailto:t...@cloudera.com] > Sent: Wednesday, July 06, 2011 10:26 AM > To: hdfs-dev@hadoop.apache.org > Subject: Re: HDFS on trunk is now quite slo

Re: HDFS on trunk is now quite slow

2011-07-06 Thread Todd Lipcon
e -- ie overcompensated for the bug. -Todd > > Thanks, > -Eric > > > -Original Message- > > From: Todd Lipcon [mailto:t...@cloudera.com] > > Sent: Friday, July 01, 2011 7:49 PM > > To: hdfs-dev@hadoop.apache.org > > Subject: Re: HDFS on trunk is

RE: HDFS on trunk is now quite slow

2011-07-06 Thread Eric Payne
o the benefit of correct locking has resulted in an acceptable slowdown on the namenode. Is that correct? Thanks, -Eric > -Original Message- > From: Todd Lipcon [mailto:t...@cloudera.com] > Sent: Friday, July 01, 2011 7:49 PM > To: hdfs-dev@hadoop.apache.org > Subject:

Re: HDFS on trunk is now quite slow

2011-07-01 Thread Todd Lipcon
My guess is HDFS-988 caused the slowdown by coarsening some locking that was previously incorrect. Your stress test is NN-only (metadata ops), not an I/O benchmark, right? I/O should be faster in trunk than ever before. -Todd On Fri, Jul 1, 2011 at 8:23 AM, Eric Payne wrote: > Hi gang, > > I ra