Vivek1106-04 commented on issue #12121: URL: https://github.com/apache/seatunnel/issues/12121#issuecomment-5645331281
Opened #12277 for this. Scope of that first PR, following the three budgets suggested above: - `CooperativeWorkerBudget` carries a global limit and a per job limit, counts promotions and denials, and releases budget when the promoted worker finishes its task rather than when a timer fires. - The decision is wired into `TaskCallTimer` and `RunBusWorkSupplier` rather than left as a standalone policy object, because the denial path is what needs review: a denied promotion keeps the tracker where it is and retries with a bounded backoff (reason `BUDGET_EXHAUSTED` in the log), and a replacement worker is still started when the denied worker is the last one serving the shared queue, so an exhausted budget cannot block readiness. - Both limits default to 0 (unlimited), so the shipped behavior is unchanged until an operator opts in. Documented in the English and Chinese hybrid and separated deployment guides. - Tests: a deterministic unit test for the policy (limits, per job rollback, release and reuse, concurrent acquire), and a service level test that deploys 8 slow cooperative tasks against a node with a global limit of 2 and a per job limit of 1, asserting a bounded worker count, denied promotions, continued progress of every task, normal completion, and budget release. Metrics are currently the worker thread pool status log (`sharedCooperativeWorkers`, `promotedCooperativeWorkers`, `totalCooperativePromotions`, `deniedCooperativePromotions`); richer telemetry can follow once the policy contract is accepted. Happy to adjust the option names or the default if maintainers prefer a finite default instead of opt-in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
