I think actually most of Flink was not designed to handle NULL values and as
far as I remember, some people think that Flink shouldn’t handle NULL values.
The fact that some parts support NULL values is more by accident than by
conscious planning.
Aljoscha
> On 24. Jun 2019, at 10:07, Becket
Hi Aljoscha,
Thanks for raising the issue. It seems there are two issues here: 1) The
null value handling, and 2) The error handling.
For null value handling, my understanding is the following:
- Null values could have a realistic meaning in some systems. So Flink
needs to support them.
- By
Error handling policy for streaming jobs goes beyond potential corrupted
messages in the source. Users may have subtle bugs while processing some
messages which may cause the streaming jobs to fail. Even though this can
be considered as a bug in user's code, users may prefer skip such messages
(or
Hi Aljoscha,
Sorry for the late reply, I think the solution makes sense. Using the NULL
return value to mark a message is corrupted is not a valid way since NULL
value has semantic meaning in not just Kafka but also in a lot of other
contexts.
I was wondering if we can have a more meaningful
Hi All,
Thanks to Gary, I recently came upon an interesting cluster of issues:
- https://issues.apache.org/jira/browse/FLINK-3679: Allow Kafka consumer to
skip corrupted messages
- https://issues.apache.org/jira/browse/FLINK-5583: Support flexible error
handling in the Kafka consumer
-
