Kowshik Prakasam created KAFKA-12867:
----------------------------------------
Summary: Trogdor ConsumeBenchWorker quits prematurely with
maxMessages config
Key: KAFKA-12867
URL: https://issues.apache.org/jira/browse/KAFKA-12867
Project: Kafka
Issue Type: Bug
Reporter: Kowshik Prakasam
The trogdor
[ConsumeBenchWorker|https://github.com/apache/kafka/commits/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java]
has a bug. If one of the consumption tasks completes executing successfully
due to [maxMessages being
consumed|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L245],
then, the consumption task [notifies the
doneFuture|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L285]
causing the ConsumeBenchWorker to halt. This becomes a problem when more than
1 consumption task is running in parallel, because the successful completion of
1 of the tasks shuts down the entire worker while the other tasks are still
running. When the worker is shut down, it
[kills|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L482]
all the active consumption tasks, which is not the desired behavior.
The fix is to not notify the doneFuture when 1 of the consumption tasks
complete without error. Instead, we should defer the notification to the
[CloseStatusUpdater|https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf64444c9/trogdor/src/main/java/org/apache/kafka/trogdor/workload/ConsumeBenchWorker.java#L299]
thread.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
