Is your MR job over the whole table or a subset? If whole table then its odd that the 7 reducers are hitting only 2 regions. What happens if 70 reducers? St.Ack
On Thu, Jun 11, 2009 at 10:42 PM, Bradford Stephens < [email protected]> wrote: > About 70. > > On Thu, Jun 11, 2009 at 10:24 PM, stack<[email protected]> wrote: > > Hey, how many regions? (smile) > > St.Ack > > > > On Thu, Jun 11, 2009 at 10:19 PM, Bradford Stephens < > > [email protected]> wrote: > > > >> I meant, here 'till Midnight :) thanks! > >> > >> On Thu, Jun 11, 2009 at 10:19 PM, Bradford > >> Stephens<[email protected]> wrote: > >> > That actually make a lot of sense. Thanks, awesome people! Me and the > >> > dev team are here to get Katta + HBase to play together, and it's > >> > looking pretty nice. > >> > > >> > On Thu, Jun 11, 2009 at 9:47 PM, stack<[email protected]> wrote: > >> >> On Thu, Jun 11, 2009 at 6:10 PM, Bradford Stephens < > >> >> [email protected]> wrote: > >> >> > >> >>> > >> >>> What I'm noticing is that it's writing to mostly one or two regions > on > >> >>> one box at a time, even though I have 7 reducers running. Monitoring > >> >>> everything with dstat -v, I notice that only 2 of my servers are > doing > >> >>> much. These boxes have very low CPU idling, and high disk output (a > >> >>> few GB a minute). > >> >>> > >> >> > >> >> > >> >> How many regions in your table? > >> >> > >> >> At first, there is one. All reducers will go against it. When it > >> splits, > >> >> then two regions field the 7 reducers and so on. > >> >> > >> >> You can manually split regions from the command-line. See if that > >> helps: > >> >> > >> >> hbase> split_region 'REGIONNAME' > >> >> > >> >> (IIRC -- type 'tools' in shell for help on the admin facilities). > >> >> > >> >> St.Ack > >> >> > >> > > >> > > >
