Re: HDFS on trunk is now quite slow

2011-07-06 Thread Todd Lipcon
for the bug. -Todd Thanks, -Eric -Original Message- From: Todd Lipcon [mailto:t...@cloudera.com] Sent: Friday, July 01, 2011 7:49 PM To: hdfs-dev@hadoop.apache.org Subject: Re: HDFS on trunk is now quite slow My guess is HDFS-988 caused the slowdown by coarsening some

RE: HDFS on trunk is now quite slow

2011-07-06 Thread Eric Payne
: 47ms 9ms Average write close time: 658ms 100ms Thanks, -Eric -Original Message- From: Todd Lipcon [mailto:t...@cloudera.com] Sent: Wednesday, July 06, 2011 10:26 AM To: hdfs-dev@hadoop.apache.org Subject: Re: HDFS on trunk is now quite slow On Wed, Jul 6, 2011 at 6

Re: HDFS on trunk is now quite slow

2011-07-06 Thread Todd Lipcon
? If you jstack the NN do you see some particular lock causing lots of contention? -Todd -Original Message- From: Todd Lipcon [mailto:t...@cloudera.com] Sent: Wednesday, July 06, 2011 10:26 AM To: hdfs-dev@hadoop.apache.org Subject: Re: HDFS on trunk is now quite slow On Wed, Jul

RE: HDFS on trunk is now quite slow

2011-07-06 Thread Eric Payne
: Wednesday, July 06, 2011 11:12 AM To: hdfs-dev@hadoop.apache.org Subject: Re: HDFS on trunk is now quite slow On Wed, Jul 6, 2011 at 9:00 AM, Eric Payne er...@yahoo-inc.com wrote: I will attempt to recreate the tests on 20.203. Currently, I'm comparing trunk against branches/MR-279

HDFS on trunk is now quite slow

2011-07-01 Thread Eric Payne
Hi all, I ran some stress tests on the latest HDFS trunk yesterday, and the performance is a lot slower (sometimes 10 times slower) when compared with the HDFS in MR-279. The HDFS in MR-279 is slightly behind trunk. The stability of the namenode in trunk seems to be better than in MR-279

Re: HDFS on trunk is now quite slow

2011-07-01 Thread Todd Lipcon
My guess is HDFS-988 caused the slowdown by coarsening some locking that was previously incorrect. Your stress test is NN-only (metadata ops), not an I/O benchmark, right? I/O should be faster in trunk than ever before. -Todd On Fri, Jul 1, 2011 at 8:23 AM, Eric Payne eric.payne1...@yahoo.com