[
https://issues.apache.org/jira/browse/FLINK-24759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caizhi Weng updated FLINK-24759:
--------------------------------
Description:
I'm running a yarn-per-job streaming job which reads a large csv file
(flights.csv in [Kaggle flight delay
data|https://www.kaggle.com/usdot/flight-delays], ~500MB) from HDFS and writes
to another location (csv format) on the same HDFS. The resulting file seems OK,
but the metrics on web UI seems strange.
I've tried other formats like reading from csv and writing to avro and the
phenomenon is the same.
!metrics.png!
was:
I'm running a yarn-per-job streaming job which reads a large csv file
(flights.csv in [Kaggle flight delay
data|https://www.kaggle.com/usdot/flight-delays], ~500MB) from HDFS and writes
to another location (csv format) on the same HDFS. The resulting file seems OK,
but the metrics on the web UI seems strage.
I've tried other formats like reading from csv and writing to avro and the
phenomenon is the same.
!metrics.png!
> Metrics on web UI is not correct in a filesystem source to filesystem sink
> streaming job
> ----------------------------------------------------------------------------------------
>
> Key: FLINK-24759
> URL: https://issues.apache.org/jira/browse/FLINK-24759
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics, Runtime / Web Frontend
> Affects Versions: 1.14.0
> Reporter: Caizhi Weng
> Priority: Major
> Attachments: metrics.png
>
>
> I'm running a yarn-per-job streaming job which reads a large csv file
> (flights.csv in [Kaggle flight delay
> data|https://www.kaggle.com/usdot/flight-delays], ~500MB) from HDFS and
> writes to another location (csv format) on the same HDFS. The resulting file
> seems OK, but the metrics on web UI seems strange.
> I've tried other formats like reading from csv and writing to avro and the
> phenomenon is the same.
> !metrics.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
