deniskuzZ commented on code in PR #6642:
URL: https://github.com/apache/hive/pull/6642#discussion_r3665141069
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -5170,6 +5170,96 @@ private static String getPathName(int taskId) {
return Utilities.replaceTaskId("000000", taskId) + "_0";
}
+ /**
+ * Compute a compact per-query uniqueness tag (8 lowercase hex chars) used
by the non-ACID
+ * rename branch of {@link #mvFile} to make each concurrent writer's
destination key unique
+ * on filesystems whose {@code rename} is not atomic-if-absent. The tag
becomes the copy
+ * suffix ({@code basename_copy_<tag>}) in place of the numeric {@code
_copy_N} counter.
+ * Reads {@code hive.query.id} from the passed {@link HiveConf}; returns the
empty string
+ * when the id is missing. The 8 hex chars come from {@code
queryId.hashCode()}; that is
+ * short enough to keep S3 listings readable and collision-free for
realistic per-partition
+ * concurrency (birthday-collides only at ~65k concurrent writers to the
same partition).
+ * <p>
+ * The shape matches {@link ParsedOutputFileName}'s copy-index group so
downstream filename
+ * parsing (taskId, attemptId, copyIndex) keeps working.
+ */
+ static String computeUniquenessTag(HiveConf conf) {
+ String qid = HiveConf.getVar(conf, ConfVars.HIVE_QUERY_ID);
+ if (qid == null || qid.isEmpty()) {
Review Comment:
````
if (StringUtils.isEmpty(qid)) {
throw new IllegalStateException("hive.query.id is required to derive a
unique destination name");
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
