Dylan Hercher created BEAM-12729:
------------------------------------
Summary: Suppress Avro Runtime Exceptions for Streaming
Key: BEAM-12729
URL: https://issues.apache.org/jira/browse/BEAM-12729
Project: Beam
Issue Type: Improvement
Components: io-java-avro
Reporter: Dylan Hercher
The current design of ReadFileRangesFn continually throws any un-recoverable
errors until the pipeline is turned down for streaming pipelines.
These invalid file errors do not have a resolvable solution and should be
logged as errors for the files in question to allow the pipeline to continue
progressing.
As a file which cannot be read will never recover via a dead letter queue
design. Since no recovery is possible we can simply log the errored file and
continue processing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
