Re: Bulkload into empty table with configureIncrementalLoad()

2013-09-19 Thread Jean-Daniel Cryans
You need to create the table with pre-splits, see http://hbase.apache.org/book.html#perf.writing J-D On Thu, Sep 19, 2013 at 9:52 AM, Dolan Antenucci antenucc...@gmail.comwrote: I have about 1 billion values I am trying to load into a new HBase table (with just one column and column family),

Re: Bulkload into empty table with configureIncrementalLoad()

2013-09-19 Thread Dolan Antenucci
Thanks J-D. Any recommendations on how to determine what splits to use? For the keys I'm using strings, so wasn't sure what to put for my startKey and endKey. For number of regions, I have a table pre-populated with the same data (not using bulk load), so I can see that it has 68 regions. On

Re: Bulkload into empty table with configureIncrementalLoad()

2013-09-19 Thread Dolan Antenucci
To follow up on my previous question about how best to do the pre-splits, I ended up using to following when creating my table: admin.createTable(desc, Bytes.toBytes(0), Bytes.toBytes(2147483647), 100); This was somewhat of a stab in the dark, but I based it on