[
https://issues.apache.org/jira/browse/FLINK-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai resolved FLINK-5583.
----------------------------------------
Resolution: Fixed
Fix Version/s: 1.3.0
Resolved for {{master}} with
http://git-wip-us.apache.org/repos/asf/flink/commit/c39ad31.
Thanks a lot for your contribution [~wheat9]!
> Support flexible error handling in the Kafka consumer
> -----------------------------------------------------
>
> Key: FLINK-5583
> URL: https://issues.apache.org/jira/browse/FLINK-5583
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Fix For: 1.3.0
>
>
> We found that it is valuable to allow the applications to handle errors and
> exceptions in the Kafka consumer in order to build a robust application in
> production.
> The context is the following:
> (1) We have schematized, Avro records flowing through Kafka.
> (2) The decoder implements the DeserializationSchema to decode the records.
> (3) Occasionally there are corrupted records (e.g., schema issues). The
> streaming pipeline might want to bail out (which is the current behavior) or
> to skip the corrupted records depending on the applications.
> Two options are available:
> (1) Have a variant of DeserializationSchema to return a FlatMap like
> structure as suggested in FLINK-3679.
> (2) Allow the applications to catch and handle the exception by exposing some
> APIs that are similar to the {{ExceptionProxy}}.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
