beryllw opened a new issue, #3259: URL: https://github.com/apache/fluss/issues/3259
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Description ## Problem The TieringCommitOperator currently runs with parallelism=1 and maxParallelism=1. All table commits are processed sequentially in a single thread. When a table's lake commit triggers manifest compaction (e.g., in Paimon), it can take a significant amount of time, during which all other tables' commits are blocked.This causes data freshness degradation for unrelated tables, potentially triggering alerts even though those tables have data ready to commit. ## Current Behavior The commit flow in TieringCommitOperator.processElement() is synchronous: 1. Collect all bucket write results for a table 2. Call LakeCommitter.toCommittable() 3. Call LakeCommitter.commit() — this is the bottleneck, as it may trigger manifest compaction 4. Call FlussTableLakeSnapshotCommitter.commit() to notify Fluss Since the operator is single-threaded with parallelism=1, step 3 for one table blocks steps 1-4 for all other tables. ### Willingness to contribute - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
