Bruno Cadonna created KAFKA-17109:
-------------------------------------
Summary: Reduce log message load for failed locking
Key: KAFKA-17109
URL: https://issues.apache.org/jira/browse/KAFKA-17109
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 3.8.0
Reporter: Bruno Cadonna
The following exception with stack traces is logged many times when state
updater is enabled:
{code}
01:08:03 INFO [KAFKA] TaskManager - stream-thread [acme-StreamThread-4]
Encountered lock exception. Reattempting locking the state in the next
iteration.
org.apache.kafka.streams.errors.LockException: stream-thread
[acme-StreamThread-4] standby-task [1_15] Failed to lock the state directory
for task 1_15
at
org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:96)
at
org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:114)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTaskToStateUpdater(TaskManager.java:1008)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:995)
at
org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:911)
at
org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1188)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:996)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:711)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670)
{code}
The exception is expected since it happens because a lock on the task state
directory is not yet been freed by a different stream thread on the same Kafka
Streams client after an assignment. But with the state updater acquiring the
lock is attempted in each poll iteration which is every 100 ms by default.
One option to reduce the log messages is to reduce the rate at which a lock is
attempted to be acquired. The other is to reduce the logging.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
