Hmm. It often occurs to me. I'll check the logs. On Fri, Nov 21, 2008 at 9:46 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote: > I think a 2 node cluster is simply too small for the full > load of everything. > > When I go that small I leave DFS out of the picture and run > HBase (in "local" mode) on top of a local file system on one > node and the jobtracker and tasktrackers on the other. > Even then I upped RAM on the HBase node to 3GB and run HBase > with 2GB heap for satisfactory results. > > - Andy > > >> From: stack <[EMAIL PROTECTED]> >> Subject: Re: Bulk import question. >> To: [email protected] >> Date: Thursday, November 20, 2008, 9:40 AM >> Edward J. Yoon wrote: >> > When I tried to bulk import, I received below error. >> (code is same >> > with hbase wiki) >> > >> > 08/11/20 15:23:50 INFO mapred.JobClient: map 62% >> reduce 0% >> > 08/11/20 15:27:10 INFO mapred.JobClient: map 30% >> reduce 0% >> > >> > Is it possible? And, hadoop/hbase daemons are crashed. >> > >> Percentage done can go in reverse if framework loses a >> bunch of maps (e.g. if crash). >> >> > - hadoop-0.18.2 & hbase-0.18.1 >> > - 4 CPU, SATA hard disk, Physical Memory 16,626,844 KB >> > - 2 node cluster >> > >> >> So, on each node you have datanode, tasktracker, and >> regionserver running and then on one of the nodes you also >> have namenode plus jobtracker? How many tasks per server? >> Two, the default? >> >> Check out your regionserver logs. My guess is one likely >> crashed, perhaps because it was starved of time or because >> its datanode was not responding nicely because it was >> loaded. >> >> You've enabled DEBUG in hbase so you can get detail, >> upped your file descriptors and your xceiverCount count? >> (See FAQ for how). >> >> St.Ack >> >> >> > >> > ---- >> > 08/11/20 15:23:36 INFO mapred.JobClient: map 57% >> reduce 0% >> > 08/11/20 15:23:40 INFO mapred.JobClient: map 59% >> reduce 0% >> > 08/11/20 15:23:45 INFO mapred.JobClient: map 60% >> reduce 0% >> > 08/11/20 15:23:50 INFO mapred.JobClient: map 62% >> reduce 0% >> > 08/11/20 15:27:10 INFO mapred.JobClient: map 30% >> reduce 0% >> > 08/11/20 15:27:10 INFO mapred.JobClient: Task Id : >> > attempt_200811131622_0019_m_000000_0, Status : FAILED >> > >> org.apache.hadoop.hbase.client.RetriesExhaustedException: >> Trying to >> > contact region server 61.247.201.164:60020 for region >> > mail,,1227162121175, row '?:', but failed >> after 10 attempts. >> > Exceptions: >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > java.io.IOException: Call failed on local exception >> > >> > at >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:863) >> > at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:964) >> > at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:950) >> > at >> com.nhn.mail.Runner$InnerMap.map(Runner.java:59) >> > at >> com.nhn.mail.Runner$InnerMap.map(Runner.java:38) >> > at >> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) >> > at >> org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) >> > at >> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207) >> > >> > > > > >
-- Best Regards, Edward J. Yoon @ NHN, corp. [EMAIL PROTECTED] http://blog.udanax.org
