RocMarshal commented on pull request #3813:
URL: https://github.com/apache/hudi/pull/3813#issuecomment-968271847
@xushiyan @yanghua @leesf .
Excuse me.I looked into this CI failure case in the backgroud of this pr.
There are some interesting appearance in test cases , such as
`TestHoodieSparkMergeOnReadTableInsertUpdateDelete`, etc.
`
String newCommitTime = "001"; // ---------------(1)
client.startCommitWithTime(newCommitTime);
List<HoodieRecord> records = dataGen.generateInserts(newCommitTime,
200);
insertRecords(metaClient, records, client, cfg, newCommitTime);
`
The new `CompactionTriggerStrategy` will use commit instant time parameters
and use delta commit info parameters to judge the state.
When the variable "001" tagged with `(1)` was transported into
`HoodieInstantTimeGenerator#parseInstantTime(String timestamp)` , the
`newCommitTime` value was not compatible with the parse pattern
`yyyyMMddHHmmss` declared in
`HoodieInstantTimeGenerator#INSTANT_TIME_FORMATTER`. At the same time, the
compact configuration item was set as 'NUM_COMMITS' . As a result of the
setting, the old `CompactionTriggerStrategy` would judge the state without
instantTime parameters. The new `CompactionTriggerStrategy` will use both
instantTime and commitNumber parameters to judge the state on the contrary,
which is the reason why the old `CompactionTriggerStrategy` could be
successful but new `CompactionTriggerStrategy` would be unsuccessful. With my
limited read, we could enhance the validation of commitTime `(1)` in the target
format `yyyyMMddHHmmss` or stop the refactor in the pr.
Please let me know what's your opinion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
