[
https://issues.apache.org/jira/browse/FLINK-40796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gyula Fora closed FLINK-40796.
------------------------------
Fix Version/s: 2.4.0
Assignee: sepuri sai krishna
Resolution: Fixed
merged to master c8453bb52e7f46e4453f93ca7ee0f7a34a075215
> State processor API no longer rejects key types that cannot be hashed reliably
> ------------------------------------------------------------------------------
>
> Key: FLINK-40796
> URL: https://issues.apache.org/jira/browse/FLINK-40796
> Project: Flink
> Issue Type: Bug
> Components: API / State Processor
> Affects Versions: 2.4.0
> Reporter: sepuri sai krishna
> Assignee: sepuri sai krishna
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> FLINK-40751 replaced `stream.keyBy(keySelector)` in
> `StateBootstrapTransformation` with a `PartitionTransformation` built
> directly, so the key type validation that `KeyedStream` performs is no longer
> reached. `OneInputStateTransformation#keyBy` only extracts the key type, so
> nothing validates it now.
> Before, bootstrapping keyed state with an array key failed at build time:
> {code}
> InvalidProgramException: Type byte[] cannot be used as key. Contained
> UNSUPPORTED key types: byte[]. Look at the keyBy() documentation for the
> conditions a type has to satisfy in order to be eligible for a key.
> {code}
> On master the same job builds, runs and writes a savepoint. The same applies
> to an enum key and to a POJO that does not override `hashCode`.
> The check comes from FLINK-5874, which restricted key types in the DataStream
> API because such a key cannot be hashed reliably. Without it the state is not
> partitioned consistently: two equal `byte[] {1, 2, 3}` keys can be assigned
> to different key groups (48 and 39 in a local run with `maxParallelism` 128).
> A job restoring the savepoint would reject the key type in its own `keyBy`,
> so the effect is that an up-front error is replaced by a savepoint that
> cannot be used.
> Reported by Gyula Fora on https://github.com/apache/flink/pull/29268 as worth
> fixing with a test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
