Also to add , from the map-reduce we have started seeing org.apache.hadoop.hbase.client.ScannerTimeoutException: 360388ms passed since the last invocation, timeout is currently set to 300000
due to the extremely high time spent on firing the batch Gets - Himanish On Mon, Feb 20, 2012 at 2:50 PM, Himanish Kushary <[email protected]>wrote: > Hi, > > We are on CDH3u3 . > > - Himanish > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Hi, > > We have a business scenario wherein we need to perform lot of gets for > each individual row from a Hbase table. To improve the performance we have > used the batch facilities using HTable.batch(...) > > From unit test case(run locally) ,the time taken for approx 120k gets was > between 5-7 secs . But when we run the same piece of code through > Map-Reduce (the batch get calls being made from the Mapper) > the time take is between 20 - 30 secs for far less amount of gets ( > between 1200 - 2000). > > Could somebody please explain why this could be happening from Map-Reduce > ? Any suggestion on how to improve this scenario is really appreciated. > > -------------------------- > Thanks & Regards > Himanish > > > > -- > Thanks & Regards > Himanish > -- Thanks & Regards Himanish
