[
https://issues.apache.org/jira/browse/FLINK-35555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhanghao Chen updated FLINK-35555:
----------------------------------
Description:
FLINK-34123 introduced built-in serialization support for java.util.List, which
relies on the existing {{ListSerializer}} impl. However, {{ListSerializer}}
does not allow null values, as it is originally designed for serializing
{{ListState}} only where null value is explicitly forbidden in the contract.
FLINK-23420 is similar to our case here. We can extend ListSerializer to allow
null values via prefixing each value by a null marker, and rely on
TypeSerializerSnapshot to deal with state-compatibility.
was:
FLINK-34123 introduced built-in serialization support for java.util.List, which
relies on the existing {{ListSerializer}} impl. However, {{ListSerializer}}
does not allow null values, as it is originally designed for serializing
{{ListState}} only where null value is explicitly forbidden in the contract.
-Directly adding null marker to allow null values will break backwards state
compatibility, so we'll need to introduce a new List serializer called
{{NullableElementListSerializer}} and the corrsponding TypeInformation called
{{NullableElementListTypeInfo}} that allows null values for serializing user
objects, and leaves the existing {{ListSerializer}} and {{ListTypeInfo}} for
Flink's internal state use.-
FLINK-23420 is similar to our case here. We can extend ListSerializer to allow
null values via a binary mask for marking null values and rely on
TypeSerializerSnapshot to deal with state-compatibility.
> Serializing List with null values throws NPE
> --------------------------------------------
>
> Key: FLINK-35555
> URL: https://issues.apache.org/jira/browse/FLINK-35555
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System
> Affects Versions: 1.20.0
> Reporter: Zhanghao Chen
> Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> FLINK-34123 introduced built-in serialization support for java.util.List,
> which relies on the existing {{ListSerializer}} impl. However,
> {{ListSerializer}} does not allow null values, as it is originally designed
> for serializing {{ListState}} only where null value is explicitly forbidden
> in the contract.
> FLINK-23420 is similar to our case here. We can extend ListSerializer to
> allow null values via prefixing each value by a null marker, and rely on
> TypeSerializerSnapshot to deal with state-compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
