prashantwason commented on code in PR #9106:
URL: https://github.com/apache/hudi/pull/9106#discussion_r1260360731
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##########
@@ -116,11 +134,10 @@ public static HoodieWriteConfig createMetadataWriteConfig(
// Below config is only used if isLogCompactionEnabled is set.
.withLogCompactionBlocksThreshold(writeConfig.getMetadataLogCompactBlocksThreshold())
.build())
- .withParallelism(parallelism, parallelism)
- .withDeleteParallelism(parallelism)
- .withRollbackParallelism(parallelism)
- .withFinalizeWriteParallelism(parallelism)
- .withAllowMultiWriteOnSameInstant(true)
+
.withStorageConfig(HoodieStorageConfig.newBuilder().hfileMaxFileSize(maxHFileSizeBytes)
+
.logFileMaxSize(maxLogFileSizeBytes).logFileDataBlockMaxSize(maxLogBlockSizeBytes).build())
+ .withRollbackParallelism(defaultParallelism)
+ .withFinalizeWriteParallelism(defaultParallelism)
Review Comment:
Yes. This was required because the previous code for commits would overwrite
the same instant if it already exists. With the already commited rollback PR,
if we get a commit on MDT with same timestamp as previous applied deltacommit
then we will first rollback the previously applied deltacommit on MDT and then
commit the new change. Hence, multi-write-on-same-instant is never possible in
MDT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
