So, I am playing with flink `MetricReporter` (and also with UI) and observe a bit strange thing. Job reading from kafka should report `elements_read` beam counter. I can see it in UI if I select source task. But, there are many many instances. I would expect to see the counter only for one `operator` and its `subtasks` (I am guessing it relates to parallelism). But, what I observe is like `elements_read` counter being defined for each operator within that task. Can this be some sort of double counting or mirror or what is going on? I am not sure how to deal with this in `MetricReporter` so I do not double count in time-series database.
P.S.: I should probably be asking this somewhere else, right? :) [ Full content available at: https://github.com/apache/beam/pull/4548 ] This message was relayed via gitbox.apache.org for [email protected]
