Hello, I have been following this thread, and got a question. I am new to Hbase coding, and I have within the past few days written a standalone (not MapReduce based) Java program to do a bulk upload into one Hbase table. I believe that I got the same error that you folks have been talking about. The program works fine on small uploads, fails with the error msg you mention when moving to import of ten of thousands of rows. So - I wanted to ask: has this import error been reported for only MapReduce-based programs, or is it indeed more general (which I could then assume may be something that affects by current import program, and I should try using the doCommit() code shown below as a fix)? Cheers, Ron Taylor ___________________________________________ Ronald Taylor, Ph.D. Computational Biology & Bioinformatics Group Pacific Northwest National Laboratory 902 Battelle Boulevard P.O. Box 999, MSIN K7-90 Richland, WA 99352 USA Office: 509-372-6568 Email: [email protected] www.pnl.gov
-----Original Message----- From: Stuart White [mailto:[email protected]] Sent: Thursday, April 02, 2009 1:37 PM To: [email protected] Subject: Re: Bulk import - does sort order of input data affect success rate? On Thu, Apr 2, 2009 at 3:30 PM, Ryan Rawson <[email protected]> wrote: > The last thing - success should not be a function of sort order. > > However, speed will be related. How? Sorted = faster, or Sorted = slower? > > One thing I found I had to do was: > private void doCommit(HTable t, BatchUpdate update) throws > IOException { > boolean commited = false; > while (!commited) { > try { > t.commit(update); > commited = true; > } catch (RetriesExhaustedException e) { > // DAMN, ignore > } > } > } > I'm running a mapred job, using TableOutputFormat to write the results to HBase. For the code you've provided, was that for a custom output format? Or a standalone (non-mapred) application? I see the point you're making, I just don't understand where I'd put that code. Thanks!
