nsivabalan commented on a change in pull request #4037:
URL: https://github.com/apache/hudi/pull/4037#discussion_r756442934
##########
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##########
@@ -362,11 +362,19 @@ public void completeCompaction(
String compactionCommitTime) {
this.context.setJobStatus(this.getClass().getSimpleName(), "Collect
compaction write status and commit compaction");
List<HoodieWriteStat> writeStats =
writeStatuses.stream().map(WriteStatus::getStat).collect(Collectors.toList());
- writeTableMetadata(table, metadata, new
HoodieInstant(HoodieInstant.State.INFLIGHT, HoodieTimeline.COMPACTION_ACTION,
compactionCommitTime));
- // commit to data table after committing to metadata table.
- finalizeWrite(table, compactionCommitTime, writeStats);
- LOG.info("Committing Compaction {} finished with result {}.",
compactionCommitTime, metadata);
- CompactHelpers.getInstance().completeInflightCompaction(table,
compactionCommitTime, metadata);
+ try {
+ HoodieInstant compactionInstant = new
HoodieInstant(HoodieInstant.State.INFLIGHT, HoodieTimeline.COMPACTION_ACTION,
compactionCommitTime);
+ this.txnManager.beginTransaction(Option.of(compactionInstant),
Option.empty());
+ // Do not do any conflict resolution here as we do with regular writes.
We take the lock here to ensure all writes to metadata table happens within a
+ // single lock (single writer). Because more than one write to metadata
table will result in conflicts since all of them updates the same partition.
+ table.getMetadataWriter().ifPresent(w -> w.update(metadata,
compactionInstant.getTimestamp(),
table.isTableServiceAction(compactionInstant.getAction())));
Review comment:
@danny0405 : fixed it. Can you check please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
