poorbarcode opened a new pull request, #130: URL: https://github.com/apache/flink-connector-pulsar/pull/130
## What is the purpose of the change Fixes FLINK-41009. The Pulsar sink previously created one transaction per target topic within a Flink checkpoint. A partial commit could therefore leave topics from the same checkpoint in inconsistent transaction states. This change reuses one Pulsar transaction for all records written in a checkpoint and emits a single committable at checkpoint handoff. ## Brief change log - Reuse a single transaction across topics for each checkpoint. - Persist the latest published message ID per topic in the committable. - Add V2 JSON committable serialization while retaining V1 deserialization compatibility. - Extend writer, producer-register, and committable serializer test coverage. ## Verifying this change - `mvn -pl flink-connector-pulsar test` ## Does this pull request potentially affect the following parts: - Dependencies (does it add or upgrade a dependency): No - The public API, i.e., is any changed class annotated with @Public(Evolving): No - The serializers: Yes - The runtime: No - The `state` module: No - The `RPC` module: No - The `REST` module: No - The `quiesce` module: No - The `metrics` module: No - The `configuration` module: No - The `docs` module: No - The `ecosystem` module: No - Tests: Yes - Anything that affects deployment or Kubernetes: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
