I don't know about idle java consuming a load of 0.2... Maybe all that load goes into figuring out a way to store an int in a few hundred bytes. ;-)
Clearly, you were right about the hadoop-site.xml fix. I don't know where I read that this was something to do with TableReduce. I kept looking through the source code and couldn't find where it was being set. This makes a lot more sense now. Yep. Dev is all one machine and production (being built as we speak) will be a more traditional deployment. Trying the job again now and will let you know what kind of speed improvement I get. Thanks, Danny On Tue, Apr 29, 2008 at 2:03 PM, stack <[EMAIL PROTECTED]> wrote: > Daniel Leffel wrote: > >> The Reduce phase is a simple reduce using the TableReduceOutputFormat. >> >> > Try upping the number of reducers -- mapred.reduce.tasks in your > hadoop-site.xml. Default is 1. Try with 4/8/16? > > Is your hbase cluster all running on the same node too? > > In this configuration, it writes 50-100 or so inserts per second (which >> doesn't strike me as terrible), but given the low load factor of 0.20 on >> the >> machine, I don't understand what's keeping it from performing better. >> >> > > Yeah... doesn't java just idling consume a load of 0.2 (smile). Is this a > multicore box? > > > St.Ack >
