[
https://issues.apache.org/jira/browse/BEAM-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767254#comment-16767254
]
Alexander Hoem Rosbach commented on BEAM-6304:
----------------------------------------------
Thank you for taking the time to respond.
1) I probably should have mentioned that we're not using Flink but Google Cloud
Dataflow. See _Error and exception handling_ in the
[documentation|https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline].
2) Good suggestion and we are concidering implementing something like that, but
we wanted to how this feature request progressed before pursuing it. Though we
would probably need to fetch the mapping for each element since we don't know
when a mapping changes, as that is external to the dataflow job.
> can ElasticsearchIO add a ExceptionHandlerFn
> ---------------------------------------------
>
> Key: BEAM-6304
> URL: https://issues.apache.org/jira/browse/BEAM-6304
> Project: Beam
> Issue Type: New Feature
> Components: io-java-elasticsearch
> Affects Versions: Not applicable
> Reporter: big
> Assignee: Etienne Chauchot
> Priority: Major
> Labels: triaged
>
> I use ElasticsearchIO to write my data to elasticSearch. However, the data is
> from other platform and not easy to check its validity. If we get the invalid
> data, we can ignore it( even though use batch insert, we can ignore all of
> them). So, I wish has a registered exception catch function to process it.
> From now on, I read the source code about write function in ProcessElement,
> it just throw the exception and cause my job to stop.
> I can catch pipeline.run().waitUntilFinish() on direct runner and force it
> run again use while statement ungracefully. However, when it deploy to Flink,
> it will fail because Flink report exception that it cannot optimize the job.
> If there is a method let user to decide how to process exception is required.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
