[
https://issues.apache.org/jira/browse/KAFKA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-17251:
------------------------------------
Description:
`KafkaStreams#cleanup()` method is designed to delete an instance local state
directory, including the `app.dir` itself if it's empty (cf corresponding unit
tests in StateDirectoryTest.java).
If the top level `app.dir` could not be deleted, a WARN is logged.
However, in a later version we started to persist the `processId` what implies
that the state directory won't be empty for many cases, because the
`StateDirectory#clean()` method does not explicitly delete the processId-file.
It's unclear right now, if `clean()` should actually try to delete the
`processId` file, too, and it's a bug that it does not, or if there are cases
for which the `processId` should actually be preserved (for this case, we
should not log a WARN as it's expected that the `app.dir` is not deleted).
Maybe there is not even a strict yes/no answer to this question, but we should
extend `KafkaStreams#cleanup()` with a parameter and let users pick? (This
would require a KIP.)
was:
`KafkaStreams#cleanup()` method is designed to delete an instance local state
directory, including the `app.dir` itself if it's empty (cf corresponding unit
tests in StateDirectoryTest.java).
If the top level `app.dir` could not be deleted, a WARN is logged.
However, in a later version we started to persist the `processId` what implies
that the state directory won't be empty for many cases, because the
`StateDirectory#clean()` method does not explicitly delete it.
It's unclear right now, if `clean()` should actually try to delete the
`processId` file, too, and it's a bug that it does not, or if there are cases
for which the `processId` should actually be preserved (for this case, we
should not log a WARN as it's expected that the `app.dir` is not deleted).
Maybe there is not even a strict yes/no answer to this question, but we should
extend `KafkaStreams#cleanup()` with a parameter and let users pick? (This
would require a KIP.)
> KafkaStreams.cleanup() semantics unclear
> ----------------------------------------
>
> Key: KAFKA-17251
> URL: https://issues.apache.org/jira/browse/KAFKA-17251
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Major
> Labels: needs-kip
>
> `KafkaStreams#cleanup()` method is designed to delete an instance local state
> directory, including the `app.dir` itself if it's empty (cf corresponding
> unit tests in StateDirectoryTest.java).
> If the top level `app.dir` could not be deleted, a WARN is logged.
> However, in a later version we started to persist the `processId` what
> implies that the state directory won't be empty for many cases, because the
> `StateDirectory#clean()` method does not explicitly delete the processId-file.
> It's unclear right now, if `clean()` should actually try to delete the
> `processId` file, too, and it's a bug that it does not, or if there are cases
> for which the `processId` should actually be preserved (for this case, we
> should not log a WARN as it's expected that the `app.dir` is not deleted).
> Maybe there is not even a strict yes/no answer to this question, but we
> should extend `KafkaStreams#cleanup()` with a parameter and let users pick?
> (This would require a KIP.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
