yhyyz opened a new pull request, #18721: URL: https://github.com/apache/iceberg/pull/18721
Follow-up to #18610, as suggested by @mxm in https://github.com/apache/iceberg/pull/18610#discussion_r4360024332. `StructLikeSerializer.serializeKey(StructLike, StructType)` required values in Iceberg's internal representation, but the readers pass generic `Record`s, which hold `java.time` values. That mismatch caused #18609, and the only safeguard after #18610 is a javadoc note at the call site. This PR makes the key API type-safe: - New `RecordSerializer`, constructed with the key `StructType`. It owns the `InternalRecordWrapper` (created once) and exposes only `serializeKey(Record key)`, so callers can no longer pass non-internal values. - `StructLikeSerializer` keeps only the partition encoding (`encodePartition` / `decodePartition`). Partition tuples from `ContentFile.partition()` already hold internal values, so they stay `StructLike`-based, and `DVPosition`, `EqualityConvertDVWriter` and their tests are unchanged. Both classes share one `writeField`, so the byte format is defined in one place. - `EqualityConvertReader` no longer wraps records itself. The serialized key format is unchanged, which matters because `SerializedEqualityValues` is Flink keyed state. ### Testing New `TestRecordSerializer` covers every primitive type with generic Java values (boolean, int, long, float, double, date, time, timestamp, timestamptz, timestamp_ns, timestamptz_ns, string, uuid, fixed, binary, decimal), with and without nulls. Each case compares the output byte-for-byte with an independent reference encoder of the previous format, built from internal values computed without the wrapper. It also checks that equivalent records produce equal keys and that a trailing `_pos` column on data-file records is ignored. The key tests moved out of `TestStructLikeSerializer`; its partition tests are unchanged. Flink 2.3 only; I will backport to 1.20 and 2.2 once this is merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
