SEZ9 commented on issue #12339: URL: https://github.com/apache/seatunnel/issues/12339#issuecomment-5770443714
@Vivek1106-04 thanks for running both arms in the same JVM invocation - a paired comparison, with the per-pipeline arm reproducing `dev`'s `newScheduledThreadPool(2, ...)` per `(jobId, pipelineId)` construction, makes these numbers straightforward to trust. The sweep supports the design's claims: the shared p50 stays at roughly 10 us from P=1 to P=500, while the per-pipeline curve has its knee between P=100 and P=500 and reaches 45.7 us p50 / 81 us p99. The 509 live scheduler threads at P=500 against a bound of at most 10 for the shared model confirms the thread-count claim with data rather than by assertion. The ~3 us cost below the P=100 crossover from the extra timer-to-dispatcher hop is acceptable to me. Please carry it into the STIP text as a stated trade-off rather than rounding it away - the framing you used here is what I want in the doc. Nothing changes from my earlier reply: no scheduler-thread-count option in this STIP, scheduling-delay observability ahead of any tuning knob, and the remaining gate before the shared-scheduler change is accepted is making the dispatcher blocking contract explicit and proving it with the regression described there (dispatch capacity filled with savepoint-contended coordinators, with an unrelated pipeline still receiving its trigger/watchdog work on time). Ping me once that is pushed and I will take another pass. <!-- streview-comment:1224 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
