[ https://issues.apache.org/jira/browse/FLINK-13498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nico Kruber closed FLINK-13498. ------------------------------- Resolution: Fixed Fix Version/s: 1.10.0 fixed on master via d774fea > Reduce Kafka producer startup time by aborting transactions in parallel > ----------------------------------------------------------------------- > > Key: FLINK-13498 > URL: https://issues.apache.org/jira/browse/FLINK-13498 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka > Affects Versions: 1.8.1, 1.9.0 > Reporter: Nico Kruber > Assignee: Nico Kruber > Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a Flink job with a Kafka producer starts up without previous state, it > currently starts 5 * kafkaPoolSize number of Kafka producers (per sink > instance) to abort potentially existing transactions from a first run without > a completed snapshot. > Apparently, this is quite slow and it is also done sequentially. Until there > is a better way of aborting these transactions with Kafka, we could do this > in parallel quite easily and at least make use of lingering CPU resources. -- This message was sent by Atlassian JIRA (v7.6.14#76016)