Hi, So basically, you have one job which is reading from A and writing to B, and one wich is reading from A and writing to C, and the 2 jobs are running at the same time. Is that correct? Are you able to reproduce that each time you are running the job? Which HBased and Hadoop versions are you running?
JM 2013/3/26 GuoWei <[email protected]>: > Dear, > > When I run two MR Jobs which will read same Hbase table and write to another > same Hbase table at the same time. The result is one job successful finished. > And another job crashed. And The following shows the error log. > > Please help me to find out why ? > > > <2013-03-25 15:50:34,026> <INFO > <org.apache.hadoop.mapred.JobClient> - map > 0% reduce 0%(JobClient.java:monitorAndPrintJob:1301) > <2013-03-25 15:50:36,096> <WARN > <org.apache.hadoop.mapred.Task> - Could not > find output size (Task.java:calculateOutputSize:948) > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > output/file.out in any of the configured local directories > at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429) > at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160) > at > org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56) > at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:944) > at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:924) > at org.apache.hadoop.mapred.Task.done(Task.java:875) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:374) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) > <2013-03-25 15:50:36,100> <INFO > <org.apache.hadoop.mapred.LocalJobRunner> - > (LocalJobRunner.java:statusUpdate:321) > <2013-03-25 15:50:36,102> <INFO > <org.apache.hadoop.mapred.Task> - Task > 'attempt_local_0001_m_000000_0' done.(Task.java:sendDone:959) > <2013-03-25 15:50:36,111> <WARN > > <org.apache.hadoop.mapred.FileOutputCommitter> - Output path is null in > cleanup(FileOutputCommitter.java:cleanupJob:100) > <2013-03-25 15:50:36,111> <WARN > <org.apache.hadoop.mapred.LocalJobRunner> - > job_local_0001(LocalJobRunner.java:run:298) > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > output/file.out in any of the configured local directories > at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429) > at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160) > at > org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:236) > <2013-03-25 15:50:37,029> <INFO > <org.apache.hadoop.mapred.JobClient> - map > 100% reduce 0%(JobClient.java:monitorAndPrintJob:1301) > <2013-03-25 15:50:37,030> <INFO > <org.apache.hadoop.mapred.JobClient> - Job > complete: job_local_0001(JobClient.java:monitorAndPrintJob:1356) > <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> - > Counters: 15(Counters.java:log:585) > <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> - > File Input Format Counters (Counters.java:log:587) > <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> - > Bytes Read=0(Counters.java:log:589) > <2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> - > FileSystemCounters(Counters.java:log:587) > <2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> - > FILE_BYTES_READ=10294950(Counters.java:log:589) > <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> - > FILE_BYTES_WRITTEN=10432139(Counters.java:log:589) > <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> - > Map-Reduce Framework(Counters.java:log:587) > <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> - > Map output materialized bytes=4006(Counters.java:log:589) > <2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> - > Combine output records=0(Counters.java:log:589) > <2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> - > Map input records=500(Counters.java:log:589) > <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> - > Physical memory (bytes) snapshot=0(Counters.java:log:589) > <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> - > Spilled Records=500(Counters.java:log:589) > <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> - > Map output bytes=3000(Counters.java:log:589) > <2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> - > Total committed heap usage (bytes)=202702848(Counters.java:log:589) > <2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> - > CPU time spent (ms)=0(Counters.java:log:589) > <2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> - > Virtual memory (bytes) snapshot=0(Counters.java:log:589) > <2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> - > SPLIT_RAW_BYTES=105(Counters.java:log:589) > <2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> - > Map output records=500(Counters.java:log:589) > <2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> - > Combine input > > Thanks a lot. > > > > Best Regards > > Weibo: http://weibo.com/guowee > Web: http://www.wbkit.com > ----------------------------------------------------- > WesternBridge Tech: Professional software service provider. Professional is > MANNER as well CAPABILITY. >
