[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[
https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18048806#comment-18048806
]
sanghyeok An edited comment on KAFKA-2 at 1/3/26 3:38 AM:
--
Hi! [~jolshan] [~chia7712] [~francisgodinho] , Allow me to add a perspective
here.
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
was (Author: JIRAUSER303328):
[~jolshan] [~chia7712] [~francisgodinho]
Allow me to add a perspective here.
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
> Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
>
>
> Key: KAFKA-2
> URL: https://issues.apache.org/jira/browse/KAFKA-2
> Project: Kafka
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Francis Godinho
>Priority: Major
> Fix For: 4.3.0
>
>
> Transaction V2 introduces frequent state transitions (epoch bumps) that
> briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default
> client retry backoff (100ms) is excessive for these transient locks, leading
> to unnecessary latency and degraded throughput. Reducing the backoff allows
> faster retries and smoother performance during state transitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[
https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18048806#comment-18048806
]
sanghyeok An edited comment on KAFKA-2 at 1/3/26 3:18 AM:
--
[~jolshan] [~chia7712] [~francisgodinho]
Allow me to add a perspective here.
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
was (Author: JIRAUSER303328):
[~jolshan] [~chia7712] [~francisgodinho]
Jumping in here to share my thoughts.
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
> Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
>
>
> Key: KAFKA-2
> URL: https://issues.apache.org/jira/browse/KAFKA-2
> Project: Kafka
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Francis Godinho
>Priority: Major
> Fix For: 4.3.0
>
>
> Transaction V2 introduces frequent state transitions (epoch bumps) that
> briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default
> client retry backoff (100ms) is excessive for these transient locks, leading
> to unnecessary latency and degraded throughput. Reducing the backoff allows
> faster retries and smoother performance during state transitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[
https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18048806#comment-18048806
]
sanghyeok An edited comment on KAFKA-2 at 1/3/26 3:17 AM:
--
[~jolshan] [~chia7712] [~francisgodinho]
Jumping in here to share my thoughts.
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
was (Author: JIRAUSER303328):
[~jolshan] [~chia7712] [~francisgodinho]
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
> Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
>
>
> Key: KAFKA-2
> URL: https://issues.apache.org/jira/browse/KAFKA-2
> Project: Kafka
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Francis Godinho
>Priority: Major
> Fix For: 4.3.0
>
>
> Transaction V2 introduces frequent state transitions (epoch bumps) that
> briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default
> client retry backoff (100ms) is excessive for these transient locks, leading
> to unnecessary latency and degraded throughput. Reducing the backoff allows
> faster retries and smoother performance during state transitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[
https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18048806#comment-18048806
]
sanghyeok An edited comment on KAFKA-2 at 1/3/26 3:16 AM:
--
[~jolshan] [~chia7712] [~francisgodinho]
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
was (Author: JIRAUSER303328):
I think it would be better to keep the client implementation as is and focus on
the server side.
Instead of modifying the client, we should implement the retry logic for
{{CONCURRENT_TRANSACTIONS}} on the server side (specifically, within the Group
Coordinator), similar to how the broker currently handles the produce path in
TV2. It seems that while the Group Coordinator currently sends
{{AddPartitionsToTxn}} requests to the Transaction Coordinator, it doesn't
handle retries for {{CONCURRENT_TRANSACTIONS}} internally.
If we handle this on the server:
# We can strictly reuse the {{add.partitions.to.txn.retry.backoff.ms}} config
for {{TxnOffsetCommit}} as intended.
# It aligns with the design goal of TV2 by keeping clients thin. This
significantly simplifies the implementation for other Kafka clients (Go, Rust,
Python, etc.), as they won't need to implement complex backoff logic for this
specific error.
What do you think?
> Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
>
>
> Key: KAFKA-2
> URL: https://issues.apache.org/jira/browse/KAFKA-2
> Project: Kafka
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Francis Godinho
>Priority: Major
> Fix For: 4.3.0
>
>
> Transaction V2 introduces frequent state transitions (epoch bumps) that
> briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default
> client retry backoff (100ms) is excessive for these transient locks, leading
> to unnecessary latency and degraded throughput. Reducing the backoff allows
> faster retries and smoother performance during state transitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[ https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18047100#comment-18047100 ] Justine Olshan edited comment on KAFKA-2 at 12/22/25 7:56 PM: -- Correct – the new config is meant to replace this original value but on the server side since we no longer send AddPartitionsToTxn from the client side in TV2 was (Author: jolshan): Correct – the new config is meant to replace this original config but on the server side since we no longer send AddPartitionsToTxn from the client side in TV2 > Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput > > > Key: KAFKA-2 > URL: https://issues.apache.org/jira/browse/KAFKA-2 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Francis Godinho >Priority: Major > Fix For: 4.3.0 > > > Transaction V2 introduces frequent state transitions (epoch bumps) that > briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default > client retry backoff (100ms) is excessive for these transient locks, leading > to unnecessary latency and degraded throughput. Reducing the backoff allows > faster retries and smoother performance during state transitions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (KAFKA-20000) Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput
[ https://issues.apache.org/jira/browse/KAFKA-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18046980#comment-18046980 ] Francis Godinho edited comment on KAFKA-2 at 12/22/25 6:49 AM: --- [~jolshan], [~chia7712] I noticed that the AddPartitionsToTxnHandler class has similar logic, but uses a fixed value: [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java#L1649] Since I'm adding the add.partitions.to.txn.retry.backoff.ms parameter to the client side for TxnOffsetCommitHandler, would it also make sense to change the logic in AddPartitionsToTxnHandler.maybeOverrideRetryBackoffMs to use that parameter too? Should I just do that in this ticket as well? was (Author: JIRAUSER308403): [~jolshan], [~chia7712] I noticed that the AddPartitionsToTxnHandler class has similar logic, but uses a fixed value: [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java#L1649] Since I'm adding the add.partitions.to.txn.retry.backoff.ms parameter to the client side for TxnOffsetCommitHandler, would it also make sense to change the logic in AddPartitionsToTxnHandler to use that parameter too? Should I just do that in this ticket as well? > Optimize retry backoff for CONCURRENT_TRANSACTIONS to improve TV2 throughput > > > Key: KAFKA-2 > URL: https://issues.apache.org/jira/browse/KAFKA-2 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Francis Godinho >Priority: Major > Fix For: 4.3.0 > > > Transaction V2 introduces frequent state transitions (epoch bumps) that > briefly reject concurrent requests with CONCURRENT_TRANSACTIONS. The default > client retry backoff (100ms) is excessive for these transient locks, leading > to unnecessary latency and degraded throughput. Reducing the backoff allows > faster retries and smoother performance during state transitions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
