[ 
https://issues.apache.org/jira/browse/HIVE-18271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290076#comment-16290076
 ] 

Jason Dere commented on HIVE-18271:
-----------------------------------

Dug into some of the details with [~ashutoshc] and [~bslim]. The 
FileSinkOperator should not even be trying to call Utilities.mvToFinalPath(), 
because it is a non-native table - there is actually logic in place for that. 
The problem is that FileSinkOperator.isNativeTable is dependent on being set 
during initializeOp() .. and it appears that initializeOp() is never being 
called for the operators in the TezTask (client side)! Making a patch to fix 
the immediate problem, by making isNativeTable into a method, that does not 
depend on initializeOp().

We may want to look into whether the operators within TezTask should have 
initialize() called on them as a future item.

> Druid Insert into fails with exception when committing files
> ------------------------------------------------------------
>
>                 Key: HIVE-18271
>                 URL: https://issues.apache.org/jira/browse/HIVE-18271
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Nishant Bangarwa
>            Assignee: Nishant Bangarwa
>             Fix For: 3.0.0
>
>         Attachments: HIVE-18271.patch
>
>
> Exception - 
> {code}
> 000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty to: 
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved)'
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move: 
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty
>  to: 
> hdfs://ctr-e136-1513029738776-2163-01-000003.hwx.site:8020/apps/hive/warehouse/_tmp.all100k_druid_initial_empty.moved
>       at org.apache.hadoop.hive.ql.exec.Utilities.rename(Utilities.java:1129)
>       at 
> org.apache.hadoop.hive.ql.exec.Utilities.mvFileToFinalPath(Utilities.java:1460)
>       at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.jobCloseOp(FileSinkOperator.java:1135)
>       at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:765)
>       at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:770)
>       at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:588)
>       at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:286)
>       at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
>       at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
>       at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
>       at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
>       at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>       at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to