ableegoldman commented on a change in pull request #9380:
URL: https://github.com/apache/kafka/pull/9380#discussion_r501393724
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorStateManager.java
##########
@@ -603,7 +603,11 @@ public void checkpoint() {
try {
checkpointFile.write(checkpointingOffsets);
} catch (final IOException e) {
- log.warn("Failed to write offset checkpoint file to [{}]",
checkpointFile, e);
+ log.warn("Failed to write offset checkpoint file to [{}]." +
+ " This may occur if OS cleaned the state.dir in case when it
located in /tmp directory." +
+ " You can change location for state.dir to resolve problem." +
+ " This can also occur due to running multiple instances on the
same machine using the same state dir.",
Review comment:
Currently, running multiple threads within an instance is _not_ the same
as running multiple instances on the same machine in the same state dir. I
suppose technically if you could configure the two instances to have the same
client UUID then the task assignor would make sure not to assign the same
active/standby task to anyone on that machine, but I think there might be some
unexpected side effects to running two separates instances with the same UUID.
Anyways, if you're aware of the problem enough to configure the client Ids
directly, you should be aware enough to just configure to use separate
state.dirs. I don't think I buy the argument about it being wasteful just
because there's really no reason to ever do this in a production app. But
people seem to do it in testing all the time, and run into this issue, hence
the warning here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
