[
https://issues.apache.org/jira/browse/FLINK-31852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gyula Fora updated FLINK-31852:
-------------------------------
Description:
The numRecordsOut task metrics seem to be incorrect when there are multiple
outputs in an operator chain. Consider a simple pipeline where you filter half
of the stream for one sink and you get all the output for the other sink (and
the sinks are not chained):
In Flink 1.16 we get the following incorrect metrics:
!image-2023-04-19-12-19-40-344.png!
The records sent/received metrics simply dont add up. The records sent seem to
correspond to the output sent to the first downstream task (which contains the
filtered stream, so about half of the events).
was:
The numRecordsOut task metrics seem to be incorrect when there are multiple
outputs in an operator chain. Consider a simple pipeline where you filter half
of the stream for one sink and you get all the output for the other sink (and
the sinks are not chained):
In Flink 1.16 we get the following incorrect metrics:
!image-2023-04-19-12-19-40-344.png!
The records sent/received metrics simply dont add up. The records sent seem to
correspond to the output sent to the first downstream task (which contains the
filtered stream, so about half of the events).
In Flink 1.17 the metrics are even more broken, because there the source output
metrics are also broken, for the same pipeline, we get:
!image-2023-04-19-12-21-35-385.png!
> NumRecordsOut metrics broken
> ----------------------------
>
> Key: FLINK-31852
> URL: https://issues.apache.org/jira/browse/FLINK-31852
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
> Affects Versions: 1.17.0, 1.16.1
> Reporter: Gyula Fora
> Priority: Critical
> Attachments: image-2023-04-19-12-19-40-344.png,
> image-2023-04-19-12-21-35-385.png
>
>
> The numRecordsOut task metrics seem to be incorrect when there are multiple
> outputs in an operator chain. Consider a simple pipeline where you filter
> half of the stream for one sink and you get all the output for the other sink
> (and the sinks are not chained):
> In Flink 1.16 we get the following incorrect metrics:
> !image-2023-04-19-12-19-40-344.png!
> The records sent/received metrics simply dont add up. The records sent seem
> to correspond to the output sent to the first downstream task (which contains
> the filtered stream, so about half of the events).
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
