ArnavBalyan opened a new pull request, #7825:
URL: https://github.com/apache/paimon/pull/7825
### Purpose
- Today `compaction.optimization-interval` is not respected. When this
interval set, the full compaction is supposed to run at max once per time
interval. But this actually runs on every writer recreate much more frequently
than the time interval, which can silently inflate i/o and CPU
- The last compaction ts is an inmemory field that starts as `null`.
However since `MergeTreeWriter` can be recreated many times, a new trigger
causes it to be constructed with `null` value and the compaction fires more
frequently.
- Timestamp of the prior full compaction is today lost across writer
recreations of MergeTreeWriter, but can be obtained using restoreFiles.
- Fix this by computing the most recent creationTimeEpochMillis of files at
the max LSM level from restoreFiles and passing it to full compaction create.
- Closes user reported bug #6547.
### Tests
- UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
