[
https://issues.apache.org/jira/browse/FLINK-35555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910071#comment-17910071
]
Zhanghao Chen commented on FLINK-35555:
---------------------------------------
It turned out that there're more issues with null handling:
# Map/List serializer cannot handle the case where the collection itself is
null
# Mapserializer does not support null key
# List serializer does not support null element
After offline discussion with @reswqa, we think a better way is to create a set
of new nullable TypeInformations whose serializer is a simple wrap of the
existing serializers with NullableSerializer to support values.
Rationale: the current serializers are already adopted by Flink internal state
API where null collection and null elements are explicitly not supported.
Extending them to support null values will add extra null masks, leading to
extra serialization and storage cost of Flink state. Extra state migration code
must be added as well.
> Serializing List with null values throws NPE
> --------------------------------------------
>
> Key: FLINK-35555
> URL: https://issues.apache.org/jira/browse/FLINK-35555
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
> Affects Versions: 1.20.0
> Reporter: Zhanghao Chen
> Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> FLINK-34123 introduced built-in serialization support for java.util.List,
> which relies on the existing {{ListSerializer}} impl. However,
> {{ListSerializer}} does not allow null values, as it is originally designed
> for serializing {{ListState}} only where null value is explicitly forbidden
> in the contract.
> FLINK-23420 is similar to our case here. We can extend ListSerializer to
> allow null values via prefixing each value by a null marker, and rely on
> TypeSerializerSnapshot to deal with state-compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
