[
https://issues.apache.org/jira/browse/HIVE-25452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17498957#comment-17498957
]
Cedric Baril-Leroux commented on HIVE-25452:
--------------------------------------------
[~gupta.nikhil0007] , I too experience this with the ABFS.
In the context of my cluster setup... e.g. : Spark with the Hive Warehouse
connector connected to the llap(Interactive-hive) cluster, should I also set
this in the spark cluster to ensure no backgroup process on the spark would
delete that staging directory ?
> Hive job fails while closing reducer output - Unable to rename
> --------------------------------------------------------------
>
> Key: HIVE-25452
> URL: https://issues.apache.org/jira/browse/HIVE-25452
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 3.1.0
> Reporter: Nikhil Gupta
> Assignee: Nikhil Gupta
> Priority: Major
>
> I have observed similar issue for Hive on LLAP:
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename
> output from:
> abfs://<redacted>/tmp/hive/root/277a283f-9d32-4525-b35f-03551c6a364e/hive_2021-08-15_20-22-24_848_2416200416962826887-27/-mr-10001/.hive-staging_hive_2021-08-15_20-22-24_848_2416200416962826887-27/_task_tmp.-ext-10002/_tmp.000000_3
> to:
> abfs://<redacted>/tmp/hive/root/277a283f-9d32-4525-b35f-03551c6a364e/hive_2021-08-15_20-22-24_848_2416200416962826887-27/-mr-10001/.hive-staging_hive_2021-08-15_20-22-24_848_2416200416962826887-27/_tmp.-ext-10002/000000_3
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commitOneOutPath(FileSinkOperator.java:272)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:236)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$400(FileSinkOperator.java:168)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1331)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:732)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:756)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:756)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
> ... 16 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
