Oh, I misspoke. The MR job is over tab-delimited text files. I have 14 mappers and 7 reducers -- loading into an empty table. The total amount of regions generated after the job is done and some splits happen is 70.
On Thu, Jun 11, 2009 at 10:47 PM, stack<[email protected]> wrote: > Is your MR job over the whole table or a subset? If whole table then its > odd that the 7 reducers are hitting only 2 regions. What happens if 70 > reducers? > St.Ack > > On Thu, Jun 11, 2009 at 10:42 PM, Bradford Stephens < > [email protected]> wrote: > >> About 70. >> >> On Thu, Jun 11, 2009 at 10:24 PM, stack<[email protected]> wrote: >> > Hey, how many regions? (smile) >> > St.Ack >> > >> > On Thu, Jun 11, 2009 at 10:19 PM, Bradford Stephens < >> > [email protected]> wrote: >> > >> >> I meant, here 'till Midnight :) thanks! >> >> >> >> On Thu, Jun 11, 2009 at 10:19 PM, Bradford >> >> Stephens<[email protected]> wrote: >> >> > That actually make a lot of sense. Thanks, awesome people! Me and the >> >> > dev team are here to get Katta + HBase to play together, and it's >> >> > looking pretty nice. >> >> > >> >> > On Thu, Jun 11, 2009 at 9:47 PM, stack<[email protected]> wrote: >> >> >> On Thu, Jun 11, 2009 at 6:10 PM, Bradford Stephens < >> >> >> [email protected]> wrote: >> >> >> >> >> >>> >> >> >>> What I'm noticing is that it's writing to mostly one or two regions >> on >> >> >>> one box at a time, even though I have 7 reducers running. Monitoring >> >> >>> everything with dstat -v, I notice that only 2 of my servers are >> doing >> >> >>> much. These boxes have very low CPU idling, and high disk output (a >> >> >>> few GB a minute). >> >> >>> >> >> >> >> >> >> >> >> >> How many regions in your table? >> >> >> >> >> >> At first, there is one. All reducers will go against it. When it >> >> splits, >> >> >> then two regions field the 7 reducers and so on. >> >> >> >> >> >> You can manually split regions from the command-line. See if that >> >> helps: >> >> >> >> >> >> hbase> split_region 'REGIONNAME' >> >> >> >> >> >> (IIRC -- type 'tools' in shell for help on the admin facilities). >> >> >> >> >> >> St.Ack >> >> >> >> >> > >> >> >> > >> >
