weiqingy commented on issue #872:
URL: https://github.com/apache/flink-agents/issues/872#issuecomment-4897967684
Following up on the portability check I said I'd run before locking the
format.
Outcome: the Kryo binary does not need to be portable across the Flink
lines, because the recovery state never crosses them. `ActionState` is a
checkpoint-scoped durable-execution journal — pruned on
`notifyCheckpointComplete` and rebuilt from the checkpointed marker on
`initializeState()` — so it is always written and read within one deployment,
by the same code and the same Flink version. It never survives a Flink-version
move or a code upgrade.
That makes both Kryo liabilities moot here: the shaded-Kryo difference
across the 1.x→2.x boundary is never exercised, and neither is Kryo's
class-schema-evolution brittleness, since write and read are always the same
code version.
So the format is unchanged from what we converged on — `{ "serde": "kryo",
"version": 1, "payload": "<base64>" }`, field-scoped to `MemoryUpdate.value`,
with the Event field staying JSON. The `version` tag carries the on-disk format
marker for the recovery-compat contract, and Kryo stays pinned within the major
release via the cluster's Flink.
On the build side: `ActionStateSerde` is compiled once against the default
Flink (now 2.3.0) and re-shaded, not recompiled, into each `dist/flink-*` line
(1.20/2.0/2.1/2.2/2.3). The construction path —
`TypeInformation.of(Object.class).createSerializer(SerializerConfig)` →
`KryoSerializer` — is present and identical across those lines, so the single
compiled artifact links on all of them.
PR: #874. It supersedes #871's narrow `__flink_agents_bytes__` envelope,
base `main`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
