Hi, clarification about the 2nd part. Required memory is one single exclusive buffer per channel, so if you are running low on memory, floating buffers are one of the first to go, hence you could observe some `0` for floating buffers.
For analysing backpressure I would recommend to use a different metric: `isBackPressured`. It was added in Flink 1.10, after this blog post had been written. Since 1.11 we also have `idleTimeMsPerSecond` that might be helpful. In 1.13 all of this will become even more simple, with changes to Web UI and an addition of `backPressuredTimeMsPerSecond` and `busyTimeMsPerSecond`. We are planning to release a new updated blog post about analysing backpressure in the following weeks. Best, Piotrek pt., 19 mar 2021 o 11:57 Piotr Nowojski <[email protected]> napisał(a): > Hi Alexey, > > Have you looked at the documentation [1]? > > > inPoolUsage An estimate of the input buffers usage. (ignores > LocalInputChannels) Gauge > > inputFloatingBuffersUsage An estimate of the floating input buffers > usage. (ignores LocalInputChannels) Gauge > > inputExclusiveBuffersUsage An estimate of the exclusive input buffers > usage. (ignores LocalInputChannels) Gauge > > Second reason might be simply that your job/task didn't manage to > acquire all of the recommended memory. Recommended memory is > `floatingBuffers + exclusiveBuffers * numberOfChannels`. Required memory is > just `1 * numberOfChannels`. If there is not enough memory to satisfy > recommended values, Flink will work with anything between recommended and > required amounts. > > Best, > Piotrek > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#default-shuffle-service > > czw., 18 mar 2021 o 08:30 Alexey Trenikhun <[email protected]> napisał(a): > >> Hello, >> While trying to investigate back pressure using hints from [1], I've >> noticed that flink_taskmanager_job_task_buffers_inPoolUsage >> and flink_taskmanager_job_task_buffers_inputFloatingBuffersUsage are always >> 0, which looks suspicious, are these metrics still populated ? >> >> Thanks, >> Alexey >> >> [1] https://flink.apache.org/2019/07/23/flink-network-stack-2.html >> Apache Flink: Flink Network Stack Vol. 2: Monitoring, Metrics, and that >> Backpressure Thing >> <https://flink.apache.org/2019/07/23/flink-network-stack-2.html> >> Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure >> Thing. 23 Jul 2019 Nico Kruber & Piotr Nowojski . In a previous blog post, >> we presented how Flink’s network stack works from the high-level >> abstractions to the low-level details.This second blog post in the series >> of network stack posts extends on this knowledge and discusses monitoring >> network-related metrics to identify ... >> flink.apache.org >> >>
