[
https://issues.apache.org/jira/browse/FLINK-27318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527412#comment-17527412
]
Arvid Heise commented on FLINK-27318:
-------------------------------------
Are you sure your test is realistic? In a production setup, you'd have each
transaction id only used once and only on restart, you'd abort 1 transaction.
If you ofc restart your application over and over again without a checkpoint
without using a different transaction id, then you'd get a large accumulation
of started transactions. But this is not how the sink should be used. Use one
transaction id prefix per unique start (and keep for restarts from checkpoints).
> KafkaSink: when init transaction, it take too long to get a producerId with
> epoch=0
> -----------------------------------------------------------------------------------
>
> Key: FLINK-27318
> URL: https://issues.apache.org/jira/browse/FLINK-27318
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.14.4
> Reporter: Zhengqi Zhang
> Priority: Major
> Attachments: image-2022-04-20-17-34-48-207.png,
> image-2022-04-20-17-59-27-397.png
>
>
> as we can see, the new KafkaSink aborts all transactions that have been
> created by a subtask in a previous run, only return when get a producerId was
> unused before(epoch=0). But this can take a long time, especially if the task
> has been started and cancelled many times before. In my tests, it even took
> {*}10 minutes{*}. Is there a better way to solve this problem, or {*}do what
> FlinkKafkaProducer did{*}.
> !image-2022-04-20-17-59-27-397.png|width=534,height=256!
> !image-2022-04-20-17-34-48-207.png|width=556,height=412!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
