Re: [DISCUSS] Split-level Watermark Metrics

2025-03-11 Thread Efrat Levitan
Thanks for the feedback Piotr. If there's no other feedback, voting will start in a couple of days. Efrat On Mon, 3 Mar 2025 at 11:15, Piotr Nowojski wrote: > Hi Efrat, > > Thanks for the proposal and +1 from my side for this FLIP. > > Flink currently has a huge observability gap when it comes t

Re: [DISCUSS] Split-level Watermark Metrics

2025-03-03 Thread Piotr Nowojski
Hi Efrat, Thanks for the proposal and +1 from my side for this FLIP. Flink currently has a huge observability gap when it comes to the state of the per-split watermarks, which makes it very difficult for users to understand why any given job generated any particular operator-level watermark. Prov

[DISCUSS] Split-level Watermark Metrics

2025-03-02 Thread אפרת לויטן
Hey everyone! I'd like to propose adding a few watermark related metrics for better visibility on split level watermark alignment and idleness states In addition to per-split watermark, I want to export the split state (active, idle and paused) timers, same as taskIO busy/idle/backpressured time re