John Roesler created KAFKA-6978:
-----------------------------------
Summary: Make Streams Window retention time strict
Key: KAFKA-6978
URL: https://issues.apache.org/jira/browse/KAFKA-6978
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: John Roesler
Currently, the configured retention time for windows is a lower bound. We
actually keep the window around until it's time to roll a new segment. At that
time, we drop all windows in the oldest segment.
As long as a window is still in a segment, we will continue to add
late-arriving records to it and also serve IQ queries from it. This is sort of
nice, because it makes optimistic use of the fact that the windows live for
some time after their retention expires. However, it is also a source of
(apparent) non-determinism, and it's arguably better for programability if we
adhere strictly to the configured constraints.
Therefore, the new behavior will be:
* once the retention time for a window passes, Streams will drop any
later-arriving records (with a warning log and a metric)
* likewise, IQ will first check whether the window is younger than its
retention time before answering queries.
No changes need to be made to the underlying segment management, this is purely
to make the behavior more strict wrt the configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
