jacklong319 commented on PR #10133: URL: https://github.com/apache/paimon/pull/10133#issuecomment-6007431277
> Requirement fit: SUPPORTED. Cross-bucket compaction parallelism addresses the documented DV freshness/backlog bottleneck while preserving per-bucket serialization. Implementation: FINDINGS. The normal Maven build and 93 targeted tests pass (including 48 clustering-table cases), and reader isolation now creates fresh nested cast state. The remaining external-executor metric regression below is reproduced against the actual TableWrite API, including the CDC caller order (withMetricRegistry before withCompactExecutor). Current CI is green apart from the intentionally skipped Python job. A source-free actual rewrite/read probe also passed 20 runs / 480 nested values across modes 2 and -1. @JingsongLi Thanks for the P2 feedback on timer retirement vs executor ownership. We updated the policy as follows: unregister() no longer retires compact timers for any mode, so shared workers (SINGLE / FIXED_POOL / external executor) keep the 60s busy window. Timers are retired only when an internally owned per-bucket executor is released (releaseCompactionExecutor with PER_BUCKET && !externalCompactExecutor), via retireCompactTimersForBucket. In prepareCommit, we call releaseCompactionExecutor before writer.close() so the bucket reporter is still present when timers are retired. withCompactExecutor() therefore keeps shared-worker timer semantics on -1 as well. Tests: CompactionTaskExecutorRoutingTest (internal PER_BUCKET release vs external executor) and CompactionMetricsTest (shared pool unregister keeps timers). Could you take another look when CI is green? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
