raminqaf opened a new pull request, #29654: URL: https://github.com/apache/flink/pull/29654
## What is the purpose of the change Backport of #29636 to release-2.4. - [FLINK-41243](https://issues.apache.org/jira/browse/FLINK-41243) is a bug since 2.3.0. VARIANT field access with a key that is not a literal fails in code generation: ```sql -- v VARIANT, k STRING SELECT v[k] FROM t; -- java.util.NoSuchElementException: None.get SELECT v[CAST(NULL AS STRING)] FROM t; -- NullPointerException ``` - [FLINK-41241](https://issues.apache.org/jira/browse/FLINK-41241) gives each `v['key']`, and each field of a cast to ROW, a `VariantFieldGetter` that checks the field index of the previous row first. With 4 lookups per row on objects that share the same 13 keys, a row takes 44 ns instead of 214 ns (JMH, JDK 17). Its table commit builds on the FLINK-41243 change in `ScalarOperatorGens`. 2.4.0 is not released yet, so both can ship in 2.4.0. `VariantFieldGetter` and `BinaryVariant` are `@Internal`, and neither commit changes a serializer or the VARIANT binary format. ## Brief change log - Cherry-picks of the three #29636 commits, applied without conflicts: - 3cf57c4d262 [FLINK-41243][table] Support VARIANT field access with a non-literal key - e36950aff2f [FLINK-41241][core] Add VariantFieldGetter to read an object field by a fixed key - 43a9d4bc319 [FLINK-41241][table] Use VariantFieldGetter for v['key'] and CAST to ROW ## Verifying this change This change is already covered by the tests that #29636 added and changed. They pass on this branch: - flink-core: `BinaryVariantTest`, `BinaryVariantInternalBuilderTest`, `JsonVariantFormatterTest` and the VARIANT serializer tests (175 tests) - flink-table-common: `FromVariantConverterTest`, `ToVariantConverterTest`, `VariantCastUtilsTest` (83 tests) - flink-table-planner: `VariantSemanticTest`, `CastFunctionITCase`, `CastRulesTest`, `JsonFunctionsITCase`, `XmlFunctionsITCase` (2936 tests) ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): yes, VARIANT field access by a literal key and the cast to ROW. A miss runs the same search as `getField`. - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code 2.1.312 (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
