nsivabalan commented on code in PR #11580:
URL: https://github.com/apache/hudi/pull/11580#discussion_r1771701812
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestHoodieSparkMergeOnReadTableCompaction.java:
##########
@@ -163,6 +173,85 @@ public void testWriteDuringCompaction(String payloadClass)
throws IOException {
assertEquals(300, readTableTotalRecordsNum());
}
+ @Test
+ public void testOutOfOrderCompactionSchedules() throws IOException,
ExecutionException, InterruptedException {
Review Comment:
I have added this in PR desc
```
This patch is introducing an additional step to validate the the instant
time used to generate the plan is the highest and there is no other instant
time in the timeline whose time > current instant time of interest.
Ideally, we might need to take table lock before doing this timestamp
validation. But it expands the scope of the patch to take locks for every
operation. With this patch, we are locking while generating instants for
ingestion writes, compaction and clustering. For cleaning and rollbacks, we
plan to take it up later.
```
So, clean and rollbacks are not taking locks here. and atleast for clean,
strictly speaking it does not matter. no other table service depends on cleans.
only subsequent cleans depends on previous clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
