weiqingy commented on issue #872: URL: https://github.com/apache/flink-agents/issues/872#issuecomment-4889883721
Thanks @wenjin272! Agreed the root cause is the untyped `MemoryUpdate.value` plus a type-info-less mapper, and that a binary serde is the more complete fix. I'm on board with the direction. A few things I'd want to pin down so we pick the mechanism deliberately: 1. For an untyped `Object`, isn't "Flink native State Serializer" effectively Kryo already? `GenericTypeInfo<Object>` resolves to `KryoSerializer` since we have no per-value `TypeInformation` at this layer — so are the two options actually distinct here, or should we just settle on Kryo deliberately? 2. Given `ActionState` is durable recovery state persisted to the Kafka/Fluss `ActionStateStore` (not scratch), how do we feel about Kryo's durability caveats? Flink's own guidance discourages Kryo for state that must survive schema evolution / version upgrades — registration order and Kryo-version bumps can silently break state compat across a Flink upgrade. If we go binary, is it worth pinning the Kryo version and documenting the recovery-compat contract, or would a more evolution-friendly path be better? 3. For composition, could we keep `ActionState` as a single JSON document and embed the binary payload as a base64 field via the same field-scoped mixin #871 already added — generalizing the payload from `byte[]` to serialized `Object`? That would reuse the insertion point we have and avoid a separate binary sidecar. Or do you see a reason to split the container instead? 4. On sequencing, would it make sense to merge #871 as-is first — it's narrow and ships the reported break now — and land the binary generalization here in #872, which then retires the `__flink_agents_bytes__` envelope? Or would you rather fold everything into one binary change? Happy to prototype the binary-envelope path if the direction on 3 and 4 sounds right. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
