[
https://issues.apache.org/jira/browse/KAFKA-19054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Boquan Tang updated KAFKA-19054:
--------------------------------
Description:
When configured with SHUTDOWN_APPLICATION for uncaught exception handler:
{code:java}
kafkaStreams.setUncaughtExceptionHandler(e ->
StreamsUncaughtExceptionHandler.StreamThreadExceptionResponse.SHUTDOWN_APPLICATION);
{code}
The kafka streams application may fall in a tight loop where inside
StreamThread#runLoop()
{code:java}
while (isRunning() || taskManager.rebalanceInProgress()) { //continue
to loop because rebalanceInProgress even though the thread is already
PENDING_SHUTDOWN
try {
checkForTopologyUpdates();
// If we received the shutdown signal while waiting for a
topology to be added, we can
// stop polling regardless of the rebalance status since we
know there are no tasks left
if (!isRunning() && topologyMetadata.isEmpty()) {
log.info("Shutting down thread with empty topology.");
break;
}
maybeSendShutdown();
// omitted code, returns very quickly because consumer#poll
will return immediately due to it receiving shutdown request
}
{code}
Inside maybeSendShutdown(), logs are printed to flood the log. We received more
than 13k logs in a short period of 50ms.
Please add logic to avoid the tight loop. Thank you.
was:
When configured with SHUTDOWN_APPLICATION for uncaught exception handler:
{code:java}
kafkaStreams.setUncaughtExceptionHandler(e ->
StreamsUncaughtExceptionHandler.StreamThreadExceptionResponse.SHUTDOWN_APPLICATION);
{code}
The kafka streams application may fall in a tight loop where inside
StreamThread#runLoop()
{code:java}
while (isRunning() || taskManager.rebalanceInProgress()) { //continue
to loop because rebalanceInProgress even though the thread is already
PENDING_SHUTDOWN
try {
checkForTopologyUpdates();
// If we received the shutdown signal while waiting for a
topology to be added, we can
// stop polling regardless of the rebalance status since we
know there are no tasks left
if (!isRunning() && topologyMetadata.isEmpty()) {
log.info("Shutting down thread with empty topology.");
break;
}
maybeSendShutdown();
// omitted code, not executed because consumer#poll will return
immediately due to it receiving shutdown request
}
{code}
Inside maybeSendShutdown(), logs are printed to flood the log. We received more
than 13k logs in a short period of 50ms.
Please add logic to avoid the tight loop. Thank you.
> StreamThread exception handling with SHUTDOWN_APPLICATION may trigger a tight
> loop with MANY logs
> -------------------------------------------------------------------------------------------------
>
> Key: KAFKA-19054
> URL: https://issues.apache.org/jira/browse/KAFKA-19054
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.7.1
> Reporter: Boquan Tang
> Priority: Minor
>
> When configured with SHUTDOWN_APPLICATION for uncaught exception handler:
> {code:java}
> kafkaStreams.setUncaughtExceptionHandler(e ->
> StreamsUncaughtExceptionHandler.StreamThreadExceptionResponse.SHUTDOWN_APPLICATION);
> {code}
> The kafka streams application may fall in a tight loop where inside
> StreamThread#runLoop()
> {code:java}
> while (isRunning() || taskManager.rebalanceInProgress()) { //continue
> to loop because rebalanceInProgress even though the thread is already
> PENDING_SHUTDOWN
> try {
> checkForTopologyUpdates();
> // If we received the shutdown signal while waiting for a
> topology to be added, we can
> // stop polling regardless of the rebalance status since we
> know there are no tasks left
> if (!isRunning() && topologyMetadata.isEmpty()) {
> log.info("Shutting down thread with empty topology.");
> break;
> }
> maybeSendShutdown();
> // omitted code, returns very quickly because consumer#poll
> will return immediately due to it receiving shutdown request
> }
> {code}
> Inside maybeSendShutdown(), logs are printed to flood the log. We received
> more than 13k logs in a short period of 50ms.
> Please add logic to avoid the tight loop. Thank you.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
