Sergey Shelukhin created HIVE-17774:
---------------------------------------
Summary: compaction may start with 0 splits and fail
Key: HIVE-17774
URL: https://issues.apache.org/jira/browse/HIVE-17774
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Eugene Koifman
{noformat}
2017-09-26 10:36:01,979 INFO [...]: compactor.CompactorMR
(CompactorMR.java:launchCompactionJob(295)) -
Submitting MINOR compaction job ....
(current delta dirs count=0, obsolete delta dirs count=0.
TxnIdRange[9223372036854775807,-9223372036854775808]
...
2017-09-26 10:36:02,350 INFO [...]: mapreduce.JobSubmitter
(JobSubmitter.java:submitJobInternal(198)) - number of splits:0
...
2017-09-26 10:36:08,637 INFO [...]: mapreduce.Job
(Job.java:monitorAndPrintJob(1380)) -
Job job_1503950256860_15982 failed with state FAILED due to: No of maps and
reduces are 0 job_1503950256860_15982
Job commit failed: java.io.FileNotFoundException: File
.../hello_acid/load_date=2016-03-03/_tmp_a95346ad-bd89-4e66-9b05-e60fdfa11858
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
at
org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
at
org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
at
org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitter.commitJob(CompactorMR.java:776)
at
org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:291)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:285)
at
org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
Looks like the MR job should not have been attempted in this case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
