[ https://issues.apache.org/jira/browse/SQOOP-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415010#comment-16415010 ]
Xiaomin Zhang commented on SQOOP-3171: -------------------------------------- [~sanysand...@gmail.com] KITE-1155 is fixed upstream, however I don't see the fix version for KITE-1155. Can we just update the dependency for Sqoop to include the Kite fix? Thanks. > Import as parquet jobs failed randomly while multiple jobs concurrently > importing into targets with same parent > --------------------------------------------------------------------------------------------------------------- > > Key: SQOOP-3171 > URL: https://issues.apache.org/jira/browse/SQOOP-3171 > Project: Sqoop > Issue Type: Bug > Reporter: Xiaomin Zhang > Assignee: Sandish Kumar HN > Priority: Major > > Running multiple parquet import jobs concurrently into below target > directories: > hdfs://ns/path/dataset1 > hdfs://ns/path/dataset2 > In some cases, one of the sqoop job will be failed with below error: > 17/03/19 08:21:21 INFO mapreduce.Job: Job job_1488289274600_188649 failed > with state FAILED due to: Job commit failed: > org.kitesdk.data.DatasetIOException: Could not cleanly delete > path:hdfs://ns/path/.temp/job_1488289274600_188649 > at > org.kitesdk.data.spi.filesystem.FileSystemUtil.cleanlyDelete(FileSystemUtil.java:239) > at > org.kitesdk.data.spi.filesystem.TemporaryFileSystemDatasetRepository.delete(TemporaryFileSystemDatasetRepository.java:61) > at > org.kitesdk.data.mapreduce.DatasetKeyOutputFormat$MergeOutputCommitter.commitJob(DatasetKeyOutputFormat.java:395) > at > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:274) > at > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.FileNotFoundException: File hdfs://ns/path/.temp does not > exist. > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:705) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:106) > at > org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:763) > at > org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:759) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:759) > at > org.kitesdk.data.spi.filesystem.FileSystemUtil.cleanlyDelete(FileSystemUtil.java:226) > This is due to: > https://issues.cloudera.org/browse/KITE-1155 -- This message was sent by Atlassian JIRA (v7.6.3#76005)