[
https://issues.apache.org/jira/browse/HIVE-25787?focusedWorklogId=756345&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-756345
]
ASF GitHub Bot logged work on HIVE-25787:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Apr/22 12:05
Start Date: 13/Apr/22 12:05
Worklog Time Spent: 10m
Work Description: hmangla98 commented on code in PR #3170:
URL: https://github.com/apache/hive/pull/3170#discussion_r849404436
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -5215,7 +5215,7 @@ private static void moveAcidFiles(String deltaFileType,
PathFilter pathFilter, F
bucketDest.toUri().toString());
try {
fs.rename(bucketSrc, bucketDest);
- if (newFiles != null) {
+ if (newFiles != null && !newFiles.contains(bucketDest)) {
Review Comment:
Actually, Task reattempt generates more than one temporary files and at the
time of actual copying of data from temporary to table loc, the destination
address is fed into this "newFiles" list multiple times. As a result, there
were multiple files in fileList in TXN_WRITE_NOTIFICAION_LOG. And this is
problematic when we initiate distcp from src to tgt and it fails with
duplicationFilesException.
Issue Time Tracking
-------------------
Worklog Id: (was: 756345)
Time Spent: 20m (was: 10m)
> Prevent duplicate paths in the fileList while adding an entry to
> NotifcationLog
> -------------------------------------------------------------------------------
>
> Key: HIVE-25787
> URL: https://issues.apache.org/jira/browse/HIVE-25787
> Project: Hive
> Issue Type: Bug
> Reporter: Ayush Saxena
> Assignee: Haymant Mangla
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As of now, while adding entries to notification logs, in case of retries,
> sometimes the same path gets added to the notification log entry, which
> during replication leads to failures during copy.
> Avoid having same path more than once for single transaction.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
