What is the clock rate of your CPUs (desktop vs blade)? -Jack
On Mon, May 16, 2011 at 1:24 PM, Himanish Kushary <[email protected]> wrote: > Yes, it is only the HW that was changed . All the configurations are kept at > default from the cloudera installer. > > The regionserver logs semms ok. > > On Mon, May 16, 2011 at 3:20 PM, Jean-Daniel Cryans > <[email protected]>wrote: > >> Ok I see... so the only thing that changed is the HW right? No >> upgrades to a new version? Also could it be possible that you changed >> some configs (or missed them)? BTW counting has a parameter for >> scanner caching, like you would write: count "myTable", CACHE = 1000 >> >> and it should stream through your data. >> >> Anything weird in the region server logs? >> >> J-D >> >> On Mon, May 16, 2011 at 12:13 PM, Himanish Kushary <[email protected]> >> wrote: >> > Thanks for the reply. We ran the TestDFSIO benchmark on both the >> development >> > and production and found the production to be better.The statistics are >> > shown below. >> > >> > But once we bring HBase into the picture things gets reversed :-( >> > >> > The count operation,map-reduces etc becomes less performing on the >> > production box.We are using Pseudo Distribution mode in both the >> development >> > and production servers for both hadoop and hbase. >> > >> > *DEVELOPMENT SERVER* >> > >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Date & time: Sun May 15 >> > 21:26:26 EDT 2011 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Number of files: 10 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Total MBytes processed: 10000 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Throughput mb/sec: >> > 58.09495038691237 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Average IO rate mb/sec: >> > 59.699485778808594 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: IO rate std deviation: >> > 10.54547265175703 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Test exec time sec: 163.354 >> > 11/05/15 21:26:26 INFO fs.TestDFSIO: >> > >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Date & time: Sun May 15 >> > 21:28:44 EDT 2011 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Number of files: 10 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Total MBytes processed: 10000 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Throughput mb/sec: >> > 682.4075337791729 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Average IO rate mb/sec: >> > 755.5845947265625 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: IO rate std deviation: >> > 229.60029445080488 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Test exec time sec: 63.896 >> > 11/05/15 21:28:44 INFO fs.TestDFSIO: >> > >> > >> > >> > >> > *PRODUCTION SERVER* >> > >> > 5/16 01:00:43 INFO fs.TestDFSIO: ----- TestDFSIO ----- : *WRITE >> PERFORMANCE* >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Date & time: Mon May 16 01:00:43 >> > GMT+00:00 2011 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Number of files: 10 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Total MBytes processed: 10000 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Throughput mb/sec: 69.25447557048375 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Average IO rate mb/sec: >> > 70.06581115722656 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: IO rate std deviation: >> > 7.243961483443693 >> > >> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Test exec time sec: 126.896 >> > >> > >> > 5/16 01:25:01 INFO fs.TestDFSIO: ----- TestDFSIO ----- : *READ >> PERFORMANCE* >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Date & time: Mon May 16 01:25:01 >> > GMT+00:00 2011 >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Number of files: 10 >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Total MBytes processed: 10000 >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Throughput mb/sec: 1487.20999405116 >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Average IO rate mb/sec: >> > 1525.230712890625 >> > >> > 11/05/16 01:25:01 INFO fs.TestDFSIO: IO rate std deviation: >> > 239.54492784268226 >> > >> > > > > -- > Thanks & Regards > Himanish >
