the-other-tim-brown commented on code in PR #13340:
URL: https://github.com/apache/hudi/pull/13340#discussion_r2113034009
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1583,6 +1583,7 @@ static HoodieActiveTimeline
runPendingTableServicesOperationsAndRefreshTimeline(
* deltacommit.
*/
void compactIfNecessary(BaseHoodieWriteClient<?,I,?,O> writeClient,
Option<String> latestDeltaCommitTimeOpt) {
+ // TODO how to handle this case where compaction needs to be written in
the past
Review Comment:
Chatted with @nsivabalan about this. The metadata commit and the data table
commit happen in the same lock so when the code acquires the lock to initialize
the metadata table (which will run `performTableServices`) it will imply that
if there is a successful commit in the metadata table and an incomplete commit
in the data table, that commit failed and will be rolled back. Therefore we may
be able to exclude the log file from the compaction and schedule the compaction
at the current time without risking issues since that metadata table commit
will be rolled back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
