Dear,
When I run two MR Jobs which will read same Hbase table and write to another
same Hbase table at the same time. The result is one job successful finished.
And another job crashed. And The following shows the error log.
Please help me to find out why ?
<2013-03-25 15:50:34,026> <INFO > <org.apache.hadoop.mapred.JobClient> - map
0% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
<2013-03-25 15:50:36,096> <WARN > <org.apache.hadoop.mapred.Task> - Could not
find output size (Task.java:calculateOutputSize:948)
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
output/file.out in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
at
org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:944)
at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:924)
at org.apache.hadoop.mapred.Task.done(Task.java:875)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:374)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
<2013-03-25 15:50:36,100> <INFO > <org.apache.hadoop.mapred.LocalJobRunner> -
(LocalJobRunner.java:statusUpdate:321)
<2013-03-25 15:50:36,102> <INFO > <org.apache.hadoop.mapred.Task> - Task
'attempt_local_0001_m_000000_0' done.(Task.java:sendDone:959)
<2013-03-25 15:50:36,111> <WARN >
<org.apache.hadoop.mapred.FileOutputCommitter> - Output path is null in
cleanup(FileOutputCommitter.java:cleanupJob:100)
<2013-03-25 15:50:36,111> <WARN > <org.apache.hadoop.mapred.LocalJobRunner> -
job_local_0001(LocalJobRunner.java:run:298)
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
output/file.out in any of the configured local directories
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
at
org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:236)
<2013-03-25 15:50:37,029> <INFO > <org.apache.hadoop.mapred.JobClient> - map
100% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
<2013-03-25 15:50:37,030> <INFO > <org.apache.hadoop.mapred.JobClient> - Job
complete: job_local_0001(JobClient.java:monitorAndPrintJob:1356)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> -
Counters: 15(Counters.java:log:585)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> - File
Input Format Counters (Counters.java:log:587)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> -
Bytes Read=0(Counters.java:log:589)
<2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> -
FileSystemCounters(Counters.java:log:587)
<2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> -
FILE_BYTES_READ=10294950(Counters.java:log:589)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
FILE_BYTES_WRITTEN=10432139(Counters.java:log:589)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
Map-Reduce Framework(Counters.java:log:587)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
Map output materialized bytes=4006(Counters.java:log:589)
<2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> -
Combine output records=0(Counters.java:log:589)
<2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> -
Map input records=500(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
Physical memory (bytes) snapshot=0(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
Spilled Records=500(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
Map output bytes=3000(Counters.java:log:589)
<2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> -
Total committed heap usage (bytes)=202702848(Counters.java:log:589)
<2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> -
CPU time spent (ms)=0(Counters.java:log:589)
<2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> -
Virtual memory (bytes) snapshot=0(Counters.java:log:589)
<2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> -
SPLIT_RAW_BYTES=105(Counters.java:log:589)
<2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> -
Map output records=500(Counters.java:log:589)
<2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> -
Combine input
Thanks a lot.
Best Regards
Weibo: http://weibo.com/guowee
Web: http://www.wbkit.com
-----------------------------------------------------
WesternBridge Tech: Professional software service provider. Professional is
MANNER as well CAPABILITY.
