davidzollo opened a new issue, #11352: URL: https://github.com/apache/seatunnel/issues/11352
## Background SeaTunnel already exposes some backpressure-related realtime signals, and the current UI can color edges with blocked ratio and queue fill ratio. That is a good starting point, but it is still not enough for real backpressure diagnosis. In practice, when a pipeline is slow, users do not just need to know that "an edge looks red". They need to know: - which edge is the worst bottleneck right now - whether the bottleneck is stable or transient - whether it is caused by source production speed, transform CPU cost, sink write/commit pressure, or downstream queue saturation - whether the pressure is getting worse or recovering ## Problem to solve Backpressure information currently exists mostly as low-level samples. SeaTunnel still lacks a dedicated user-facing view that turns those samples into an actionable diagnosis workflow. This leads to slow troubleshooting and makes it hard to compare multiple blocked paths in the same job. ## Proposed scope Add a dedicated backpressure diagnostics view for running jobs. Suggested first-delivery scope: - rank the most blocked edges and downstream bottleneck vertices in one place - show queue size, queue capacity, fill ratio, blocked ratio, and blocked duration trend together - distinguish between chronic pressure and short spikes - provide a direct path from the backpressure panel into the related vertex/edge details and logs - make the view work together with the runtime graph, but keep it independently useful as a focused bottleneck page ## Why this is not just a UI cleanup A useful backpressure view may require new or clarified contracts for: - how blocked time is sampled and aggregated - whether queue metrics are instantaneous, rolling-window, or max/min summaries - how long samples are retained - how high-frequency metrics are cleaned up after job finish/failure - how to avoid high write amplification for observability-only state ## STIP requirement before implementation Because this feature may extend realtime metric contracts and has non-trivial runtime-cost implications, **the claimant should submit a STIP design first and get maintainer agreement before starting implementation**. The STIP should clarify: - the exact metric model used by the backpressure view - whether existing metrics are sufficient or new metrics are required - how the view handles large jobs and fast-refresh scenarios - how backpressure diagnosis should interact with checkpoint and sink-commit signals - which data is ephemeral observability state versus stable user-facing API output ## Acceptance criteria - Users can quickly identify the top blocked edges and likely bottleneck vertices for a running job. - The view shows both current state and short-term trend, not just a single latest value. - Data definitions are documented clearly enough that users do not have to infer meaning from implementation details. - English and Chinese docs are updated. ## Non-goals for the first version - full root-cause automation for every slowdown - storage of long-term historical backpressure data - connector-specific tuning recommendations in the first delivery ## Related work - PR #11244 - Issue #11351 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
