[
https://issues.apache.org/jira/browse/FLINK-41104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu updated FLINK-41104:
----------------------------
Description:
In PyFlink process mode, MapState.contains(key) incorrectly treats an existing
mapping with a None value as an absent entry.
Reproduction:
{code:python}
state.put(1, None)
state.contains(1) # Returns False
{code}
Expected:
contains(1) returns True while the mapping exists, and returns False after the
key is removed.
Actual:
It returns False for the existing mapping, both before commit and after commit.
The behavior also occurs with an uncached backend response.
Root cause:
InternalSynchronousMapRuntimeState.contains() determines membership using
self.get(map_key) is not None. This loses the distinction between a missing
entry and an existing null-valued entry, although the write cache and map-state
protocol preserve an explicit existence flag.
The affected implementation is in
flink-python/pyflink/fn_execution/state_impl.py. Membership checks should use
entry existence rather than value nullness.
Regression tests should cover null-valued, non-null-valued, missing, and
removed entries across buffered, cached, and uncached reads, including the "key
in state" syntax.
was:
In PyFlink process mode, MapState.contains(key) incorrectly treats an existing
mapping with a None value as an absent entry.
Reproduction without TTL:
{code:python}
state.put(1, None)
state.contains(1) # Returns False
{code}
Expected:
contains(1) returns True while the mapping exists, and returns False after the
key is removed.
Actual:
It returns False for the existing mapping, both before commit and after commit.
The behavior also occurs with an uncached backend response.
Root cause:
InternalSynchronousMapRuntimeState.contains() determines membership using
self.get(map_key) is not None. This loses the distinction between a missing
entry and an existing null-valued entry, although the write cache and map-state
protocol preserve an explicit existence flag.
The affected implementation is in
flink-python/pyflink/fn_execution/state_impl.py. Membership checks should use
entry existence rather than value nullness.
Regression tests should cover null-valued, non-null-valued, missing, and
removed entries across buffered, cached, and uncached reads, including the "key
in state" syntax.
> MapState.contains() returns false for an existing key whose value is None
> -------------------------------------------------------------------------
>
> Key: FLINK-41104
> URL: https://issues.apache.org/jira/browse/FLINK-41104
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Dian Fu
> Priority: Major
>
> In PyFlink process mode, MapState.contains(key) incorrectly treats an
> existing mapping with a None value as an absent entry.
> Reproduction:
> {code:python}
> state.put(1, None)
> state.contains(1) # Returns False
> {code}
> Expected:
> contains(1) returns True while the mapping exists, and returns False after
> the key is removed.
> Actual:
> It returns False for the existing mapping, both before commit and after
> commit. The behavior also occurs with an uncached backend response.
> Root cause:
> InternalSynchronousMapRuntimeState.contains() determines membership using
> self.get(map_key) is not None. This loses the distinction between a missing
> entry and an existing null-valued entry, although the write cache and
> map-state protocol preserve an explicit existence flag.
> The affected implementation is in
> flink-python/pyflink/fn_execution/state_impl.py. Membership checks should use
> entry existence rather than value nullness.
> Regression tests should cover null-valued, non-null-valued, missing, and
> removed entries across buffered, cached, and uncached reads, including the
> "key in state" syntax.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
