Edward J. Yoon wrote:
When I tried to bulk import, I received below error. (code is same
with hbase wiki)
08/11/20 15:23:50 INFO mapred.JobClient: map 62% reduce 0%
08/11/20 15:27:10 INFO mapred.JobClient: map 30% reduce 0%
Is it possible? And, hadoop/hbase daemons are crashed.
Percentage done can go in reverse if framework loses a bunch of maps
(e.g. if crash).
- hadoop-0.18.2 & hbase-0.18.1
- 4 CPU, SATA hard disk, Physical Memory 16,626,844 KB
- 2 node cluster
So, on each node you have datanode, tasktracker, and regionserver
running and then on one of the nodes you also have namenode plus
jobtracker? How many tasks per server? Two, the default?
Check out your regionserver logs. My guess is one likely crashed,
perhaps because it was starved of time or because its datanode was not
responding nicely because it was loaded.
You've enabled DEBUG in hbase so you can get detail, upped your file
descriptors and your xceiverCount count? (See FAQ for how).
St.Ack
----
08/11/20 15:23:36 INFO mapred.JobClient: map 57% reduce 0%
08/11/20 15:23:40 INFO mapred.JobClient: map 59% reduce 0%
08/11/20 15:23:45 INFO mapred.JobClient: map 60% reduce 0%
08/11/20 15:23:50 INFO mapred.JobClient: map 62% reduce 0%
08/11/20 15:27:10 INFO mapred.JobClient: map 30% reduce 0%
08/11/20 15:27:10 INFO mapred.JobClient: Task Id :
attempt_200811131622_0019_m_000000_0, Status : FAILED
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
contact region server 61.247.201.164:60020 for region
mail,,1227162121175, row '?:', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
java.io.IOException: Call failed on local exception
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:863)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:964)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:950)
at com.nhn.mail.Runner$InnerMap.map(Runner.java:59)
at com.nhn.mail.Runner$InnerMap.map(Runner.java:38)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
