Thanks a lot for the quick response. It worked like a charm. On Thu, Sep 15, 2011 at 10:44 AM, Stack <[email protected]> wrote:
> Do you know your keyspace roughly? Try creating a pre-split table > with as many regions as you want reducers. > St.Ack > > On Wed, Sep 14, 2011 at 8:25 PM, rajesh balamohan > <[email protected]> wrote: > > ImportTSV internally uses HFileOutputFormat.configureIncrementalLoad(job, > > table); > > > > However, for newly created tables there would not be any keys available. > > Hence, it launches 1 reducer by default. > > > > Is there a way to increase the number of reducers for high volume imports > > like 500+ GB. > > > > ~Rajesh.B > > > > On Thu, Sep 15, 2011 at 8:51 AM, rajesh balamohan < > [email protected]>wrote: > > > >> Hi All, > >> > >> ImportTSV is a great tool for bulk loading the data into HBASE. > >> > >> I have close to 500+GB of raw data which I would like to import into a > >> newly created HTABLE. If I go ahead with ImportTSV, it creates only one > >> reducer which is a bottleneck in terms of sorting and shuffling. > >> > >> Are there any other way, I can increase the number of reducers while > doing > >> bulk loads for new table?. > >> > >> ~Rajesh.B > >> > > >
