[
https://issues.apache.org/jira/browse/BEAM-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241259#comment-17241259
]
Chamikara Madhusanka Jayalath edited comment on BEAM-4244 at 12/1/20, 6:06 AM:
-------------------------------------------------------------------------------
Unfortunately this probably will end up being a significant change for runners
(for example, Dataflow) hence unlikely to be resolved in the near term.
In the mean time I recommend using existing runner specific UX/API features to
easily detect/handle such errors. For example, you can use Dataflow Cloud
Console Stack Traces tab to easily detect errors and/or use StackDriver API to
try to process error logs programatically.
was (Author: chamikara):
Unfortunately this probably will end up being a significant change for runners
(for example, Dataflow) hence unlikely to be resolved in the near term.
In the mean time I recommend runner specific UX/API improvements to easily
detect/handle such errors. For example, you can use Dataflow Cloud Console
Stack Traces tab to easily detect errors and/or use StackDriver API to try to
process error logs programatically.
> Provide a better way for programmatically handling errors raised while
> encoding/decoding data
> ---------------------------------------------------------------------------------------------
>
> Key: BEAM-4244
> URL: https://issues.apache.org/jira/browse/BEAM-4244
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, runner-core
> Reporter: Chamikara Madhusanka Jayalath
> Priority: P3
>
> Beam runners use coders in various stages of a pipeline to encode/decode
> data. Coders are executed directly by the runner of a pipeline and user do
> not have control over exceptions raised during encoding/decoding (could be
> either due to malformed/corrupted data provided by users or intermediate
> malformed/corrupted data generated during the system execution).
> Currently users can rely on runner-specific worker logging to detect the
> error and update the pipeline but it would be better if we can provide a way
> to programmatically handle these errors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
