[
https://issues.apache.org/jira/browse/BEAM-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252605#comment-17252605
]
Yimin Zhu commented on BEAM-4244:
---------------------------------
I would like to emphasize the *damages* (without this bug fix) had been caused
in the history and now: our entire Data Plane was brought down due to no way to
intervene from the application code - anytime it happened, it was a severely
impacting to our customers.
This is *not* a "nice to have" feature, it is a *must* for a data pipeline to
process unbounded data.
> Provide a better way for programmatically handling errors raised while
> encoding/decoding data
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-4244
> URL: https://issues.apache.org/jira/browse/BEAM-4244
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, runner-core
> Reporter: Chamikara Madhusanka Jayalath
> Priority: P3
>
> Beam runners use coders in various stages of a pipeline to encode/decode
> data. Coders are executed directly by the runner of a pipeline and user do
> not have control over exceptions raised during encoding/decoding (could be
> either due to malformed/corrupted data provided by users or intermediate
> malformed/corrupted data generated during the system execution).
> Currently users can rely on runner-specific worker logging to detect the
> error and update the pipeline but it would be better if we can provide a way
> to programmatically handle these errors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
