SEZ9 commented on issue #13044: URL: https://github.com/apache/seatunnel/issues/13044#issuecomment-6487634767
Thanks for the detailed report and the clear reproduction. The analysis looks plausible: `generateTransactionId` only combines `transactionPrefix` and the checkpoint id, so with `parallelism = 2` both writers end up with the same transactionalId (e.g. `st310-eo-tx-1`) and Kafka fences one of them. Including the subtask index from `getIndexOfSubtask()` in the generated id is the right direction, and the same formula needs to be applied on the restore/abort paths so recovery still aborts the correct transactions. Since @zephyr45 has volunteered to work on this, it would be great if the PR includes the two tests suggested here: a unit test asserting two writers with the same prefix produce distinct transaction ids, and an E2E Kafka sink case with `EXACTLY_ONCE` and `parallelism = 2` verifying the job completes with exactly the 100 source records and no `ProducerFencedException`. If you get a chance to run the same job against 2.3.13, confirming whether it fails there too would help clarify which branches need the fix. <!-- streview-comment:2123 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
