vamossagar12 commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957932804
> Personally I'd suggest we enforce the rentention at the inner store, which means: > > 1. for built-in in-memory stores, enforce at the InMemoryXXStore level. > 2. for built-in persistent stores, enforce at the SegmentStore level (which is used by all window stores). > 3. for user customized stores, they should enforced the retention themselves: this is via `WindowStoreSupplier#retentionPeriod` call which is accessible to users. > > The rationale of doing that is: > > * enforcing at the highest layer means that lower layers would still return more data than necessary which then need to be filtered at a per-record basis, which is less efficient. > * when more records are returned from the inner layer, it would also impact our intermediate layers such as polluting the caching layer. Thanks @guozhangwang for the response. While I agree with doing this enforcement at the inner most layer due to all the reasons mentioned, one of the problems highlighted on this PR by Mathias is that `InMemoryWindowStore does not enforce strict retention for IQ if caching is enabled`. While InMemoryWindowStore does enable strict retention but when combined with the caching store, it breaks. That's where doing it on MeteredStore might be beneficial. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
