We had issues of moving into 32 core AMD box also. The issue was revolving around datanode getting slow after about 12 hours. What you need to do is check fsreadlatency_ave_time graph, if it appears spiky then you have a problem with IO, next get a graph of "Runnable Threads" they should be flatlining, if they are spiking you might have IO/Memory contention. Run RAM tests head to head, we found our 32 Core ECC RAM would always underperform while using sysbench, by a large margin.
-Jack On Mon, May 16, 2011 at 10:33 AM, Himanish Kushary <[email protected]> wrote: > Hi, > > We are in the process of moving a small Hbase/Hadoop cluster from our > development to production environment.Our development environment were few > intel desktops (8 cores CPU/8 Gigs RAM/7200 rpm disks) running centOS while > the production environment are blades with (24 cores AMD CPU/32 gigs > RAM/15000 rpm disks) AMD architecture running centOS. > > Strangely the hbase performance seems to degrade after moving stuffs to the > production enviroment (suppoesed to have more horse power).We are using the > latest and default installation for cloudera version of hadoop and hbase.No > changes to memory or other parameter were done on both the environment. > > Any idea what could cause this.Could the AMD architecture be the > cause.Pointers to things to look for to improve performance in the > production cluster would be really appreciated. > > Note: We ran "count" from hbase shell on a huge table and found the desktops > to be performing much better. We are in the process of comparing Map-Reduces > presently. > > --------------------------- > Thanks & Regards > Himanish >
