[ https://issues.apache.org/jira/browse/HIVE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290076#comment-16290076 ]
Jason Dere commented on HIVE-18271: ----------------------------------- Dug into some of the details with [~ashutoshc] and [~bslim]. The FileSinkOperator should not even be trying to call Utilities.mvToFinalPath(), because it is a non-native table - there is actually logic in place for that. The problem is that FileSinkOperator.isNativeTable is dependent on being set during initializeOp() .. and it appears that initializeOp() is never being called for the operators in the TezTask (client side)! Making a patch to fix the immediate problem, by making isNativeTable into a method, that does not depend on initializeOp(). We may want to look into whether the operators within TezTask should have initialize() called on them as a future item. > Druid Insert into fails with exception when committing files > ------------------------------------------------------------ > > Key: HIVE-18271 > URL: https://issues.apache.org/jira/browse/HIVE-18271 > Project: Hive > Issue Type: Bug > Reporter: Nishant Bangarwa > Assignee: Nishant Bangarwa > Fix For: 3.0.0 > > Attachments: HIVE-18271.patch > > > Exception - > {code} > 000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty to: > hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved)' > org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move: > hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty > to: > hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved > at org.apache.hadoop.hive.ql.exec.Utilities.rename(Utilities.java:1129) > at > org.apache.hadoop.hive.ql.exec.Utilities.mvFileToFinalPath(Utilities.java:1460) > at > org.apache.hadoop.hive.ql.exec.FileSinkOperator.jobCloseOp(FileSinkOperator.java:1135) > at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:765) > at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:770) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:588) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:286) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242) > at > org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)