Rajeev-01 commented on code in PR #13473:
URL: https://github.com/apache/hudi/pull/13473#discussion_r2160690240
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -504,6 +505,7 @@ public static void recover(HoodieStorage storage,
StoragePath metadataFolder) th
private static void modify(HoodieStorage storage, StoragePath
metadataFolder, Properties modifyProps, BiConsumer<Properties, Properties>
modifyFn) {
StoragePath cfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE);
StoragePath backupCfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE_BACKUP);
+ StoragePath tempCfgPath = new StoragePath(metadataFolder,
HOODIE_PROPERTIES_FILE + HOODIE_TEMP_FILE_SUFFIX);
Review Comment:
1. `createImmutableFileInPath` will create the temp files with UUID which
can leave the stray files if we use it for properties modification. If we use
the `.temp` suffix even if the file is present we will overwrite the existing
file.
2. Right now during modification the properties file is being generated in
this way create -> write
But if the create call is being failed due to TCP connection failure (in
HDFS due to connection failure between NN and client) and we will end hitting
exception and it is ending up in NN lease recovery process will end up in
closing the File with size 0.
3. So this temp file write and rename will ensure that properties file will
be created with valid properties or it won't be present.
4. The backup logic needs to be present to serve readers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
