Piotr Nowojski created FLINK-19537:
--------------------------------------
Summary: Processed in-flight bytes metric is not accurate
Key: FLINK-19537
URL: https://issues.apache.org/jira/browse/FLINK-19537
Project: Flink
Issue Type: Improvement
Components: Runtime / Metrics, Runtime / Network
Affects Versions: 1.12.0
Reporter: Piotr Nowojski
Processed in-flight bytes as introduced in FLINK-18662 is not entirely
accurate, as it's ignoring the buffer/bytes accumulated in the record
deserializers. If buffer is processed here, it doesn't mean it was fully
processed (so we can over estimate the amount of processed bytes). On the other
hand some records/bytes might be processed without polling anything from this
{{CheckpointedInputGate}} (underestimating the amount of processed bytes). All
in all this should have been calculated on the {{StreamTaskNetworkInput}}
level, where we have an access to the records deserializers. However the
current is on average accurate and it might be just good enough (at least for
the time being).
Also this metric is currently ignoring chained source inputs to the multiple
input stream task.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
