[
https://issues.apache.org/jira/browse/KAFKA-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-10564:
------------------------------------
Labels: beginner newbie (was: )
> Continuous logging about deleting obsolete state directories
> ------------------------------------------------------------
>
> Key: KAFKA-10564
> URL: https://issues.apache.org/jira/browse/KAFKA-10564
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.6.0
> Reporter: Michael Bingham
> Priority: Minor
> Labels: beginner, newbie
> Fix For: 2.7.0, 2.6.1
>
>
> The internal process which automatically cleans up obsolete task state
> directories was modified in https://issues.apache.org/jira/browse/KAFKA-6647.
> The current logic in 2.6 is to remove all files from the task directory
> except the {{.lock}} file:
> [https://github.com/apache/kafka/blob/2.6/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L335]
> However, the directory is only removed in its entirely for a manual cleanup:
> [https://github.com/apache/kafka/blob/2.6/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L349-L353]
> The result of this is that Streams will continue revisiting this directory
> and trying to clean it up, since it determines what to clean based on
> last-modification time of the task directory (which is now no longer deleted
> during the automatic cleanup). So a user will see log messages like:
> stream-thread [app-c2d773e6-8ac3-4435-9777-378e0ec0ab82-CleanupThread]
> Deleting obsolete state directory 0_8 for task 0_8 as 60600001ms has elapsed
> (cleanup delay is 600000ms)
> repeated again and again.
> This issue doesn't seem to break anything - it's more about avoiding
> unnecessary logging and cleaning up empty task directories.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
