danny0405 commented on code in PR #17472:
URL: https://github.com/apache/hudi/pull/17472#discussion_r2587461531
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/timeline/versioning/v2/LSMTimelineWriter.java:
##########
@@ -208,16 +209,19 @@ private void createManifestFile(HoodieLSMTimelineManifest
manifest, int currentV
int newVersion = currentVersion < 0 ? 1 : currentVersion + 1;
// create manifest file
final StoragePath manifestFilePath =
LSMTimeline.getManifestFilePath(newVersion, archivePath);
- metaClient.getStorage().createImmutableFileInPath(manifestFilePath,
Option.of(HoodieInstantWriter.convertByteArrayToWriter(content)));
+ // create the manifest file with overwrite semantics, to handle the case
like that:
+ // writer_1 creates the manifest file successfully, but fails before
updating the version file,
+ // so we need to allow writer_2 to overwrite the manifest file wth the
same version number.
+ FileIOUtils.createFileInPath(metaClient.getStorage(), manifestFilePath,
Option.of(HoodieInstantWriter.convertByteArrayToWriter(content)));
Review Comment:
the manifest file is introduced to have atomicity semantics here, that's why
we use `createImmutableFileInPath`.
for l0 file atomicity, we have fix from L127 ~ L137.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
