tchivs commented on PR #4530: URL: https://github.com/apache/flink-cdc/pull/4530#issuecomment-5903352442
Thanks — the state-safety concern is addressed in three new commits. The unsafe surface was narrower than "TIME in state": 1. Nested TIME serializers were rejected by an `equals()`-based compatibility gate in the composite snapshots (`NullableSerializerWrapper`, `ArrayDataSerializer`, `MapDataSerializer`). They now resolve the nested serializer's compatibility from the previous instance's snapshot. 2. The pre-upgrade array payload for `TIME(p > 3)` carried 4-byte element slots. For (2) the layout is detected from the payload itself, not from a snapshot: a TIME array has no var-length region, so `sizeInBytes < round8(header + 8 * N)` is exact rather than a heuristic. Reader-side only — the writer, the snapshots and the new layout are unchanged. Compatibility: - `TIME(<= 3)`: `isCompatibleAsIs`, no migration. Snapshot class name and envelope unchanged; `getCurrentVersion()` 3 -> 4 and `readSnapshot` still accepts 2/3. - `TIME(> 3)`: `compatibleAfterMigration`, restored at millisecond fidelity, which is all the old writer carried. - Savepoints taken with this build cannot be restored on the previous build (`Unrecognized version: 4`) — inherent to the version bump, worth a release note. Worth stating explicitly why this does not go through the migration channel: the event serializers are `TypeSerializerSingleton` with `SimpleTypeSerializerSnapshot`, so `resolveSchemaCompatibility` always resolves `compatibleAsIs` and the rewrite never triggers; and `BinaryRecordDataSerializer.serialize` copies the row segment verbatim (`BinarySegmentUtils.copyToView`), so a nested array payload would not be re-encoded even if it did. Hence the reader-side fix. Not covered here: the Postgres snapshot read still truncates `TIME(> 3)`; that stays with FLINK-39748 / #4412, as noted earlier. The added tests drive real bytes (pre-upgrade payload into the new reader) instead of `equals()`, and fail before the fix. Verified on the JDK 11 baseline (`openjdk 11.0.24`): `flink-cdc-common` 8 tests and the affected `flink-cdc-runtime` serializer suites 96 tests, 0 failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
