[
https://issues.apache.org/jira/browse/FLINK-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11141:
-----------------------------------
Labels: auto-deprioritized-critical stale-major (was:
auto-deprioritized-critical)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Major but is unassigned and neither itself nor its Sub-Tasks have been updated
for 30 days. I have gone ahead and added a "stale-major" to the issue". If this
ticket is a Major, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Key generation for RocksDBMapState can theoretically be ambiguous
> -----------------------------------------------------------------
>
> Key: FLINK-11141
> URL: https://issues.apache.org/jira/browse/FLINK-11141
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.5.5, 1.6.2, 1.7.0
> Reporter: Stefan Richter
> Priority: Major
> Labels: auto-deprioritized-critical, stale-major
>
> RocksDBMap state stores values in RocksDB under a composite key from the
> serialized bytes of {{key-group-id|key|namespace|user-key}}. In this
> composition, key, namespace, and user-key can either have fixed sized or
> variable sized serialization formats. In cases of at least 2 variable
> formats, ambiguity can be possible, e.g.:
> abcd <-> efg
> abc <-> defg
> Our code takes care of this for all other states, where composite keys only
> consist of key and namespace by checking for 2x variable size and appending
> the serialized length to each byte sequence.
> However, for map state there is no inclusion of the user-key in the check for
> potential ambiguity, as well as for appending the size. This means that, in
> theory, some combinations can produce colliding composite keys in RocksDB.
> What is required is to include the user-key serializer in the check and
> append the length there as well.
> Please notice that this cannot be simply changed because it has implications
> for backwards compatibility and requires some form of migration for the state
> keys on restore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
