DanielLeens opened a new issue, #12122: URL: https://github.com/apache/seatunnel/issues/12122
## Description This is a scalability task for checkpoint trigger scheduling on active pipelines. It is not a regression report. Verified at `dev` commit `97d461bc0773399d632fd078735736ecd44f5f0b`: - `CheckpointCoordinator` constructs `Executors.newScheduledThreadPool(2, ...)` per pipeline (`CheckpointCoordinator.java:228`); `cleanPendingCheckpoint()` (`:1163`) shuts it down (`:1203`) and constructs a replacement (`:1205`). - Core workers are created on demand, not prestarted (there is no `prestartAllCoreThreads` anywhere in `seatunnel-engine`); periodic triggering starts from `allTaskReady` (`:438`) after tasks report ready, and `restoreCoordinator` (`:673`) handles restored pipelines. A queued, never-started pipeline therefore holds the executor object but no threads. - The single-pending-checkpoint guard (`:802`) is correct and must be kept. Accurate bound today: up to two worker threads per pipeline whose scheduler has been used. ## Expected outcome - A node-level shared, bounded scheduler for active pipelines, keyed by `(jobId, pipelineId)` so cancellation stays per pipeline, with blocking checkpoint/RPC work isolated so one pipeline cannot delay another pipeline's timers. - Keep the single-pending-checkpoint invariant. - Benchmark active checkpoint-enabled pipelines separately from initialized/queued jobs; 500 active pipelines should show a flat timer-thread count. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
