danny0405 commented on code in PR #11052:
URL: https://github.com/apache/hudi/pull/11052#discussion_r1571748491
##########
hudi-io/src/main/java/org/apache/hudi/storage/HoodieStorage.java:
##########
@@ -267,7 +270,7 @@ public final void createImmutableFileInPath(StoragePath
path,
if (content.isPresent() && needTempFile) {
StoragePath parent = path.getParent();
- tmpPath = new StoragePath(parent, path.getName() + TMP_PATH_POSTFIX);
+ tmpPath = new StoragePath(parent, path.getName() + "." +
UUID.randomUUID());
fsout = create(tmpPath, false);
Review Comment:
Here is the Hadoop filesystem atomicity guarantees:
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html#Core_Expectations_of_a_Hadoop_Compatible_FileSystem
For file creation, if the overwrite parameter is false, the check and
creation MUST be atomic. Here we do not hold exclusive access lock as the
invoker, a random suffix would eliminate the requirement for lock because the
tmp file creation would never conflict. And the renaming is itself atomic, so
we can somehow ensure the atomocity of the file creation on HDFS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
