Sergei Lilichenko created BEAM-10693:
----------------------------------------
Summary: Improve error handling in
org.apache.beam.sdk.extensions.ml.CloudVision
Key: BEAM-10693
URL: https://issues.apache.org/jira/browse/BEAM-10693
Project: Beam
Issue Type: New Feature
Components: extensions-java-gcp
Affects Versions: 2.22.0
Reporter: Sergei Lilichenko
CloudVision invokes batchAnnotateImages API ((at AnnotateImages class). Any
exception thrown will propagate to the pipeline. Because of the Vision API's
quota processing large number of images with a large number of workers will
often result in ResourceExhaustedException and subsequent reprocessing of the
complete bundle and pipeline errors.
By catching that exception in the transform and providing an exponential
backoff reprocessing and excessive error reporting can be significantly
minimized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
