raminqaf opened a new pull request, #29804:
URL: https://github.com/apache/flink/pull/29804
## What is the purpose of the change
A cast from `VARIANT` to a binary or `BOOLEAN` target read the value with a
strict `Variant` getter. When the stored kind didn't match, the getter's
`VariantTypeException` escaped, while every other failed cast from `VARIANT`
throws a `TableRuntimeException` built by `unsupportedKind`. A string variant
also couldn't reach a binary target at all, although a regular `STRING` to
`BINARY` cast works and `PARSE_JSON` never produces `BYTES`.
| Query | Before
| After
|
|------------------------------------------|--------------------------------------------------------------|-----------------------------------------------------------------------------|
| `CAST(PARSE_JSON('"abc"') AS BYTES)` | `VariantTypeException: Expected
type BYTES but got STRING` | `x'616263'`
|
| `TRY_CAST(PARSE_JSON('"abc"') AS BYTES)` | `NULL`
| `x'616263'`
|
| `CAST(PARSE_JSON('true') AS BYTES)` | `VariantTypeException: Expected
type BYTES but got BOOLEAN` | `TableRuntimeException: Cannot cast a VARIANT
BOOLEAN value to BYTES. ...` |
| `CAST(PARSE_JSON('"true"') AS BOOLEAN)` | `VariantTypeException: Expected
type BOOLEAN but got STRING` | `TableRuntimeException: Cannot cast a VARIANT
STRING value to BOOLEAN. ...` |
The two commits address the two tickets in order. They share one PR because
the second relies on the first: its catch in `FromVariantConverter#convert`
would otherwise relabel a binary kind mismatch as a malformed variant.
## Brief change log
- **FLINK-40319:** `VariantCastUtils#toBytes` reads a stored `STRING` as its
UTF-8 bytes, then pads or truncates it like a `BYTES` value. Every other kind
fails with `unsupportedKind`. The `VARIANT` cast table in `data-types.md` lists
the new pair.
- **FLINK-41301:** a new `VariantCastUtils#toBoolean` checks the kind the
same way. `FromVariantConverter#convert` wraps a `VariantTypeException` from a
malformed variant in a `TableRuntimeException` and keeps it as the cause. The
`DATE` reader already checks the kind since FLINK-41281.
FLINK-40319 first proposed a raw-bytes accessor for string values. This PR
reads the string with `getString()` instead, so no class in
`org.apache.flink.types.variant` changes. The trade-off is that a stored string
with an invalid UTF-8 payload comes out as `EF BF BD` instead of its stored
bytes. That is the same U+FFFD substitution `CAST(v AS STRING)` makes. Only
bytes wrapped without validation, such as the binary raw encoding from
FLINK-41323, can hold such a string.
## Verifying this change
This change added tests, and each new case fails without its fix:
- `FromVariantConverterTest`: a non-ASCII string reaches `BYTES`,
`VARBINARY(3)` and `BINARY(8)` as its UTF-8 bytes. A boolean, an integer and an
object fail a binary target, and a string fails `BOOLEAN`, each with a
`TableRuntimeException`. A truncated `INT` fails with a `TableRuntimeException`
caused by the `VariantTypeException`.
- `CastRulesTest`: the string case of the `BYTES` cast now passes, where it
was asserted to fail. This test can't show the exception type, because
`CastExecutor` wraps any `Throwable` in a `TableRuntimeException`.
- `CastFunctionITCase`: `CAST` and `TRY_CAST` to `BYTES` and `BOOLEAN` end
to end, asserting the `TableRuntimeException` message.
`CastFunctionMiscITCase`, `VariantFunctionsITCase`, `VariantSemanticTest`,
`ToVariantConverterTest` and `VariantCastUtilsTest` pass unchanged.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): yes. A cast
from `VARIANT` to a binary or `BOOLEAN` target checks the kind once per value,
which the strict getters already did.
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes, a cast from a
`VARIANT` string to `BINARY` and `VARBINARY`
- If yes, how is the feature documented? docs
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code 2.1.319 (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
