Also, do you think if I query using rowkey instead of hbase time stamp..it would not kick off that many tasks.. since region server knows the exact locations?
thanks venkatesh -----Original Message----- From: Venkatesh <[email protected]> To: [email protected] Sent: Wed, Oct 6, 2010 8:53 am Subject: Re: HBase map reduce job timing Ahh ..ok..That makes sense I've a 10 node cluster each with 36 gig..I've allocated 4gig for HBase Region Servers..master.jsp reports used heap is less than half on each region server. I've close to 800 regions total..Guess it needs to kick off a jvm to see if data exists in all regions.. -----Original Message----- From: Jean-Daniel Cryans <[email protected]> To: [email protected] Sent: Tue, Oct 5, 2010 11:52 pm Subject: Re: HBase map reduce job timing > Regarding number of map tasks 500+, 490 of them processing nothing, do you have an explanation > for that?..Wondering if its kicking off too many JVMs most doing nothing.. This would mean that throughout your regions, only a few have data in the timestamp range you're looking for. > > 'top' reports less free memory (couple of gig.) though box has 36 gig total.. I don't quite trust > top since cached blocks don't show up under free column even if no process is running.. > You only have 1 machine? BTW how much RAM did you give to HBase? J-D =
