raminqaf opened a new pull request, #29087:
URL: https://github.com/apache/flink/pull/29087
## What is the purpose of the change
Second subtask of FLIP-604, building directly on FLINK-40486 (merged as
#29041), which introduced the `UUID` logical type at the type-system layer
only. After that change a `UUID` value still could not be materialized,
serialized, or returned at runtime. This change wires `UUID` through the table
runtime, code generation, and the DataStream type stack so `UUID`
columns work end to end.
Internally a `UUID` is stored as its canonical 16-byte big-endian encoding,
reusing them `byte[]` representation and `getBinary`/`writeBinary` access
already used by `BINARY`/`VARBINARY`. The external/default conversion class is
`java.util.UUID`.
## Brief change log
- Internal representation and serialization:
`LogicalTypeUtils.toInternalConversionClass` returns `byte[]`;
`InternalSerializers`, `BinaryWriter`, `BinaryArrayWriter`, the interpreted
`RowData`/`ArrayData` field getters, and `BinaryArrayData` sizing reuse the
`BINARY`/`VARBINARY` path.
- Code generation: `CodeGenUtils` (type term, hashing, row read/write) and
`GenerateUtils` literal handling emit the `byte[]` path for `UUID`;
`RexLiteralUtil` converts a `java.util.UUID` literal to 16 big-endian bytes.
- External conversion: new `UuidUuidConverter` (`byte[]` <->
`java.util.UUID`), registered in `DataStructureConverters`.
- DataStream API: new `UuidSerializer` and `UuidComparator`, plus
`Types.UUID`. The comparator uses unsigned big-endian byte order (the type's
defined order), not the signed `java.util.UUID.compareTo`.
- Reflective extraction: `UUID` is auto-extracted for Table/SQL via
`ClassDataTypeConverter` but intentionally not auto-registered for the
DataStream API (`BasicTypeInfo.TYPES`); see below.
- Planner plumbing: `FlinkRelMdSize` (16-byte estimate) and
`ExpressionReducer` (skip constant folding, as for `VARIANT`).
## Verifying this change
This change added tests and can be verified as follows:
- `UuidSerializerTest` / `UuidComparatorTest` (`flink-core`):
serialization round-trip and unsigned ordering, including normalized-key
consistency; the sorted test data spans the signed-long boundary (`0x7FFF...`
vs `0x8000...`).
- `DataStructureConvertersTest`: `byte[]` <-> `java.util.UUID` round-trip.
- `ClassDataTypeConverterTest` / `DataTypeExtractorTest`: Table/SQL
auto-extraction of a `java.util.UUID` field to `DataTypes.UUID()`.
- `UuidITCase` (`flink-table-planner`): end-to-end `SELECT`, `ARRAY[...]`,
multi-row `VALUES`, `UNION`/`CASE`, `MAP` value, and nested `ROW`, collecting
`java.util.UUID` back through the full plan/codegen/execution stack.
## Design notes / trade-offs for reviewers
- **DataStream extraction is opt-in.** Registering `java.util.UUID` for
automatic reflective extraction in the DataStream API would silently switch the
serializer of existing `java.util.UUID` fields from Kryo to `UuidSerializer`,
breaking savepoint compatibility on upgrade. DataStream users therefore opt in
via `Types.UUID`; automatic extraction is planned for the next major version.
Table/SQL does auto-extract, since a plain `UUID` field there otherwise
requires an explicit RAW annotation, so there is no silent change.
- **Comparator order** is unsigned big-endian to match the type's
canonical order and the SQL side, deliberately differing from
`java.util.UUID.compareTo` (signed per half).
## Related tickets (FLIP-604)
- Depends on: FLINK-40486 — Introduce the UUID logical type (merged,
#29041).
- Follow-ups, handled in separate subtasks:
- FLINK-40487 — casts `UUID <-> STRING` / `BINARY(16)` (note:
`.print()` depends on the
to-string cast).
- FLINK-40488 — comparison and ordering in SQL (`ORDER BY` / `GROUP
BY` / joins);
`TypeCheckUtils.isComparable` still excludes `UUID` in this PR.
- FLINK-40489 — `UUID_V4` / `UUID_V7` functions.
- FLINK-40494 — documentation.
- Not addressed here: compiled-plan `RexNode` literal serde, left as a
follow-up as it was for `VARIANT`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: yes — adds
`Types.UUID` and `BasicTypeInfo.UUID_TYPE_INFO`
- The serializers: yes — new `UuidSerializer`; `UUID` internal state uses
the byte-array serializer
- The runtime per-record code paths (performance sensitive): yes —
per-record field getters/
writers and generated field access for `UUID`
- Anything that affects deployment or recovery: JobManager (and its
components),
Checkpointing, Kubernetes/Yarn, ZooKeeper: no (new type; no state
migration for existing jobs)
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes (as part of FLIP-604)
- If yes, how is the feature documented? not documented in this PR;
user-facing docs are
handled in a separate FLIP-604 subtask (FLINK-40494)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
