[
https://issues.apache.org/jira/browse/FLINK-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuan Mei updated FLINK-11141:
-----------------------------
Priority: Minor (was: Not a Priority)
> Key generation for RocksDBMapState can theoretically be ambiguous
> -----------------------------------------------------------------
>
> Key: FLINK-11141
> URL: https://issues.apache.org/jira/browse/FLINK-11141
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.5.5, 1.6.2, 1.7.0
> Reporter: Stefan Richter
> Priority: Minor
> Labels: auto-deprioritized-critical, auto-deprioritized-major,
> auto-deprioritized-minor
>
> RocksDBMap state stores values in RocksDB under a composite key from the
> serialized bytes of {{key-group-id|key|namespace|user-key}}. In this
> composition, key, namespace, and user-key can either have fixed sized or
> variable sized serialization formats. In cases of at least 2 variable
> formats, ambiguity can be possible, e.g.:
> abcd <-> efg
> abc <-> defg
> Our code takes care of this for all other states, where composite keys only
> consist of key and namespace by checking for 2x variable size and appending
> the serialized length to each byte sequence.
> However, for map state there is no inclusion of the user-key in the check for
> potential ambiguity, as well as for appending the size. This means that, in
> theory, some combinations can produce colliding composite keys in RocksDB.
> What is required is to include the user-key serializer in the check and
> append the length there as well.
> Please notice that this cannot be simply changed because it has implications
> for backwards compatibility and requires some form of migration for the state
> keys on restore.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
