ferenc-csaky commented on PR #130: URL: https://github.com/apache/flink-connector-pulsar/pull/130#issuecomment-6341160165
I would suggest an append-only V2 format instead, see [this commit](https://github.com/ferenc-csaky/flink-connector-pulsar/commit/9c65687d1da30e3df32bc2cc87ada0930d217f71), as an example impl. V2 keeps the V1 layout as a prefix (the topic field is written empty) and appends the message-ID map, which is read only when `version >= 2`. ### Reasons - **Downgrade keeps working.** Flink passes the stored version to the serializer, and the V1 deserializer ignores it and any trailing bytes. An older connector would restore V2 state as `(txnID, "")` and commit the transaction, so the Compatibility caveat can be dropped. With the current layout, it misreads the leading version `int` and fails. - **No version on `PulsarCommittable`.** Flink already stores the serializer version, so repeating it in the payload and on the domain object is redundant. It also doesn't help the follow-up: a V2 committable can have an empty map too (e.g. every send failed before `prepareCommit()`), so an empty map has to mean "no recovery info" either way. - **Simpler.** One read path instead of two decoders plus an inner-version check, and a future V3 is just another append. The cost is 2 bytes for the empty topic field. All validation from your latest commit is kept. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
