Not an exact match, but I am guessing it is related to FLINK-7286 <https://issues.apache.org/jira/projects/FLINK/issues/FLINK-7286>, which I reported. Feel free to modify that issue to cover the root cause.
On Wed, Aug 30, 2017 at 8:32 AM, Martin Eden <martineden...@gmail.com> wrote: > Thanks Chesnay, > > Just for completeness, are there any relevant tickets for the discussion > that one can follow, upvote, contribute to? > > M > > On Tue, Aug 29, 2017 at 8:57 PM, Chesnay Schepler <ches...@apache.org> > wrote: > >> Hello, >> >> 1. Because no one found time to fix it. In contrast to the remaining >> byte/record metrics, input metrics for sources / output metrics for sinks >> have to be implemented for every single implementation with their >> respective semantics. In contrast, the output metrics are gathered in the >> intersection between operators, independent of the actual operator >> implementation. Furthermore, this requires system metrics (i.e. metrics >> that Flink itself creates) to be exposed (and be mutable!) to user-defined >> functions, which is something i *generally *wanted to avoid, but it >> appears to be a big enough pain point to make an exception here. >> >> 2. Due to the above it is currently not possible without modifications of >> the code to know how many reads/writes were made. >> >> 3. Do you mean aggregated metrics? The web UI allows the aggregation of >> record/byte metrics on the task level. Beyond that we defer aggregation to >> actual time-series databases that specialize in these things. >> >> >> On 28.08.2017 19:08, Martin Eden wrote: >> >> Hi all, >> >> Just 3 quick questions both related to Flink metrics, especially around >> sinks: >> >> 1. In the Flink UI Sources always have 0 input records / bytes and Sinks >> always have 0 output records / bytes? Why is it like that? >> >> 2. What is the best practice for instrumenting off the shelf Flink sinks? >> >> Currently the only metrics available are num records/bytes in and out at >> the operator and task scope. For the task scope there are extra buffer >> metrics. However the output metrics are always zero (see question 1). How >> can one know the actual number of successful writes done by an off the >> shelf Flink sink? Or the latency of the write operation? >> >> 3. Is it possible to configure Flink to get global job metrics for all >> subtasks of an operator? Or are there any best practices around that? >> >> Thanks, >> M >> >> >> >
