deniskuzZ commented on code in PR #6642:
URL: https://github.com/apache/hive/pull/6642#discussion_r3764548109
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -5170,6 +5189,110 @@ private static String getPathName(int taskId) {
return Utilities.replaceTaskId("000000", taskId) + "_0";
}
+ /**
+ * Compute a compact per-query uniqueness tag used by the non-ACID rename
branch of
+ * {@link #mvFile} to make each concurrent writer's destination key unique
on filesystems
+ * whose {@code rename} is not atomic-if-absent. The tag becomes the copy
suffix
+ * ({@code basename_copy_<tag>}) in place of the numeric {@code _copy_N}
counter.
+ * <p>
+ * Reads {@code hive.query.id} from the passed {@link HiveConf} and
delegates to
+ * {@link QueryPlan#extractUniquenessTag(String)} for the actual UUID → hex
derivation.
+ * The shape matches {@link ParsedOutputFileName}'s copy-index group so
downstream filename
+ * parsing (taskId, attemptId, copyIndex) keeps working.
+ */
+ static String computeUniquenessTag(HiveConf conf) {
+ String qid = HiveConf.getVar(conf, ConfVars.HIVE_QUERY_ID);
+ if (Strings.isNullOrEmpty(qid)) {
+ throw new IllegalStateException("hive.query.id is required to derive a
unique destination name");
+ }
+ return QueryPlan.extractUniquenessTag(qid);
+ }
+
+ /**
+ * @return {@code true} when the filesystem's URI scheme is one of the known
non-atomic-rename
+ * schemes ({@link #NON_ATOMIC_RENAME_SCHEMES}); {@code false}
otherwise (including a
+ * {@code null} fs or missing scheme).
+ */
+ static boolean isNonAtomicRenameFs(FileSystem fs) {
Review Comment:
should we extract this to `FileUtils`/`BlobStorageUtils `,`FilenameUtils` or
something? Hive class is already massive
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -5170,6 +5189,110 @@ private static String getPathName(int taskId) {
return Utilities.replaceTaskId("000000", taskId) + "_0";
}
+ /**
+ * Compute a compact per-query uniqueness tag used by the non-ACID rename
branch of
+ * {@link #mvFile} to make each concurrent writer's destination key unique
on filesystems
+ * whose {@code rename} is not atomic-if-absent. The tag becomes the copy
suffix
+ * ({@code basename_copy_<tag>}) in place of the numeric {@code _copy_N}
counter.
+ * <p>
+ * Reads {@code hive.query.id} from the passed {@link HiveConf} and
delegates to
+ * {@link QueryPlan#extractUniquenessTag(String)} for the actual UUID → hex
derivation.
+ * The shape matches {@link ParsedOutputFileName}'s copy-index group so
downstream filename
+ * parsing (taskId, attemptId, copyIndex) keeps working.
+ */
+ static String computeUniquenessTag(HiveConf conf) {
+ String qid = HiveConf.getVar(conf, ConfVars.HIVE_QUERY_ID);
+ if (Strings.isNullOrEmpty(qid)) {
+ throw new IllegalStateException("hive.query.id is required to derive a
unique destination name");
+ }
+ return QueryPlan.extractUniquenessTag(qid);
+ }
+
+ /**
+ * @return {@code true} when the filesystem's URI scheme is one of the known
non-atomic-rename
+ * schemes ({@link #NON_ATOMIC_RENAME_SCHEMES}); {@code false}
otherwise (including a
+ * {@code null} fs or missing scheme).
+ */
+ static boolean isNonAtomicRenameFs(FileSystem fs) {
Review Comment:
should we extract this to `FileUtils`/`BlobStorageUtils `/`FilenameUtils` or
something? Hive class is already massive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
