vinothchandar commented on code in PR #13451:
URL: https://github.com/apache/hudi/pull/13451#discussion_r2163784138
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/WriteClientTestUtils.java:
##########
@@ -40,4 +43,22 @@ public static void
startCommitWithTime(BaseHoodieWriteClient<?, ?, ?, ?> writeCl
public static Option<String> scheduleTableService(BaseHoodieWriteClient<?,
?, ?, ?> writeClient, String instantTime, Option<Map<String, String>>
extraMetadata, TableServiceType tableServiceType) {
return writeClient.scheduleTableService(Option.of(instantTime),
extraMetadata, tableServiceType);
}
+
+ public static String createNewInstantTime() {
+ return HoodieInstantTimeGenerator.createNewInstantTime(false,
TestTimeGenerator.INSTANCE, 0);
+ }
+
+ private static class TestTimeGenerator implements TimeGenerator {
+ private static final TimeGenerator INSTANCE = new TestTimeGenerator();
+
+ @Override
+ public long generateTime(boolean skipLocking) {
+ return System.currentTimeMillis();
Review Comment:
do we need a `Thread.sleep(1)` to ensure clock progresses between calls?
(will add time to CI?) or implement something stateful - like having a
`private static final baseInstantTime = System.currentTimeMillis()`
and then
`return ++baseInstantTime;`
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/client/WriteClientTestUtils.java:
##########
@@ -40,4 +43,22 @@ public static void
startCommitWithTime(BaseHoodieWriteClient<?, ?, ?, ?> writeCl
public static Option<String> scheduleTableService(BaseHoodieWriteClient<?,
?, ?, ?> writeClient, String instantTime, Option<Map<String, String>>
extraMetadata, TableServiceType tableServiceType) {
return writeClient.scheduleTableService(Option.of(instantTime),
extraMetadata, tableServiceType);
}
+
+ public static String createNewInstantTime() {
+ return HoodieInstantTimeGenerator.createNewInstantTime(false,
TestTimeGenerator.INSTANCE, 0);
Review Comment:
makes sense to me, to have a class that controls time generation in the
tests..
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1789,12 +1789,10 @@ void compactIfNecessary(BaseHoodieWriteClient<?,I,?,O>
writeClient, Option<Strin
writeClient.compact(compactionInstantTime, true);
} else if (metadataWriteConfig.isLogCompactionEnabled()) {
// Schedule and execute log compaction with new instant time.
- final String logCompactionInstantTime =
metadataMetaClient.createNewInstantTime(false);
- if
(metadataMetaClient.getActiveTimeline().filterCompletedInstants().containsInstant(logCompactionInstantTime))
{
- LOG.info("Log compaction with same {} time is already present in the
timeline.", logCompactionInstantTime);
Review Comment:
```
if
(metadataMetaClient.getActiveTimeline().filterCompletedInstants().containsInstant(logCompactionInstantTime))
```
I was mulling about this check.. this one here, seems to no-op the log
compaction. I think we can ignore.
however, wonder if its good to add a validation inside the time generation
parts (Txn Manager?) to ensure no other action with that time is requested,
while holding the lock? and fail things if so. (can be in a diff PR as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
