Apoorva Sharma created BEAM-11015:
-------------------------------------
Summary: fileIO issue on latest version of apache-beam to 2.23.0
Key: BEAM-11015
URL: https://issues.apache.org/jira/browse/BEAM-11015
Project: Beam
Issue Type: Bug
Components: beam-community
Affects Versions: 2.23.0
Reporter: Apoorva Sharma
Assignee: Aizhamal Nurmamat kyzy
Fix For: Not applicable
In Apache Beam , while using below P transformation -*fileio.MatchAll() and
fileio.ReadMatches()* dataflow job on load performance .( processing 48
million resources)
For 2-3 minute Throughput(element/sec) goes to zero and no element was
processed during that time , it comes back again back to average normal
(0.25/sec) of my job
In between these three minutes. There is no log from apache beam , if some IO
error came or any warning.
Please check for any fileIO known issue on latest version of *apache-beam to
2.23.0*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
