yigress commented on code in PR #4937:
URL: https://github.com/apache/hive/pull/4937#discussion_r1430539424
##########
hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java:
##########
@@ -538,17 +538,17 @@ private void moveTaskOutputs(final Configuration conf,
Path srcf, Path srcDir,
final Path finalOutputPath = getFinalPath(destFs, srcF, srcDir,
destDir, immutable);
if (immutable && destFs.exists(finalOutputPath) &&
!org.apache.hadoop.hive.metastore.utils.FileUtils.isDirEmpty(destFs,
finalOutputPath)) {
- throw new HCatException(ErrorType.ERROR_DUPLICATE_PARTITION,
- "Data already exists in " + finalOutputPath
- + ", duplicate publish not possible.");
- }
- if (srcStatus.isDirectory()) {
+ if (partitionsDiscoveredByPath.containsKey(srcF.toString())) {
Review Comment:
updated the PR description with bug conditions.
the move files is called within registerPartitions so
partitionsDiscoveredByPath by now are populated with all the partition paths ,
this condition here is to check if here is at the full part of the partition
(for multi-partitioned, (part1=x, part2=y, part3=z) so there is no need to
drill down the path anymore. otherwise the check is still at parent level and
can continue check down the path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
