Lars, Cool stuff! Thanks a lot! I'm not sure I can apply the patch, cause we're using CDH-4.1.1, but increasing size of internal scanner does the trick - decreased number of scanners. At least temporarily it's good enough.
Thanks! On Wed, Dec 19, 2012 at 6:23 AM, lars hofhansl <[email protected]> wrote: > You might have run into HBASE-7336. > (Not available in any official release, yet) > > If you're using 0.94 (and probably 0.92) you can just apply this patch > (it's save and simple). > > > > ________________________________ > From: Eugeny Morozov <[email protected]> > To: [email protected] > Sent: Tuesday, December 18, 2012 12:01 AM > Subject: Many scanner opening > > Hello! > > We faced an issue recently that the more map tasks are completed, the > longer it takes to complete one more map task. > > In our architecture we have two scanners to read the table. The first one, > which is called 'outer' scanner is reading table and filter some rowkeys. > These rowkeys are used as a filter for second scanner - 'internal'. Thus we > constantly open 'internal' scanner with different filters. > > As an additional symptoms we see that our cluster practically does nothing > - there is no CPU loading, no disk loading, no network, etc. Most of the > time it means we are waiting on some locks, but I'm not sure. > > I would appreciate any ideas or suggestions to understand the case. > Thank you in advance. > -- > Evgeny Morozov > Developer Grid Dynamics > Skype: morozov.evgeny > www.griddynamics.com > [email protected] > -- Evgeny Morozov Developer Grid Dynamics Skype: morozov.evgeny www.griddynamics.com [email protected]
