[
https://issues.apache.org/jira/browse/KAFKA-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Bingham updated KAFKA-9524:
-----------------------------------
Description:
In a windowed aggregation, if you specify a window size larger than the default
window retention (1 day), Streams will implicitly set retention accordingly to
accommodate windows of that size. For example,
{code:java}
.windowedBy(TimeWindows.of(Duration.ofDays(20)))
{code}
In this case, Streams will implicitly set window retention to 20 days, and no
exceptions will occur.
However, if you also include a non-zero grace period on the window, such as:
{code:java}
.windowedBy(TimeWindows.of(Duration.ofDays(20)).grace(Duration.ofMinutes(5)))
{code}
In this case, Streams will still implicitly set the window retention 20 days
(not 20 days + 5 minutes grace), and an exception will be thrown:
{code:java}
Exception in thread "main" java.lang.IllegalArgumentException: The retention
period of the window store KSTREAM-KEY-SELECT-0000000002 must be no smaller
than its window size plus the grace period. Got size=[1728000000],
grace=[300000], retention=[1728000000]{code}
Ideally, Streams should include grace period when implicitly setting window
retention.
was:
In a windowed aggregation, if you specify a window size larger than the default
window retention (1 day), Streams will implicitly set retention accordingly to
accommodate windows of that size. For example,
{code:java}
.windowedBy(TimeWindows.of(Duration.ofDays(20)))
{code}
In this case, Streams will implicitly set window retention to 20 days, and no
exceptions will occur.
However, if you also include a non-zero grace period on the window, such as:
{code:java}
.windowedBy(TimeWindows.of(Duration.ofDays(20)).grace(Duration.ofMinutes(5)))
{code}
In this case, Streams will still implicitly set the window retention 20 days
(not 20 days + 5 minutes grace), and an exception will be thrown:
Exception in thread "main" java.lang.IllegalArgumentException: The retention
period of the window store KSTREAM-KEY-SELECT-0000000002 must be no smaller
than its window size plus the grace period. Got size=[1728000000],
grace=[300000], retention=[1728000000]
Ideally, Streams should include grace period when implicitly setting window
retention.
> Default window retention does not consider grace period
> -------------------------------------------------------
>
> Key: KAFKA-9524
> URL: https://issues.apache.org/jira/browse/KAFKA-9524
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.4.0
> Reporter: Michael Bingham
> Priority: Minor
>
> In a windowed aggregation, if you specify a window size larger than the
> default window retention (1 day), Streams will implicitly set retention
> accordingly to accommodate windows of that size. For example,
> {code:java}
> .windowedBy(TimeWindows.of(Duration.ofDays(20)))
> {code}
> In this case, Streams will implicitly set window retention to 20 days, and no
> exceptions will occur.
> However, if you also include a non-zero grace period on the window, such as:
> {code:java}
> .windowedBy(TimeWindows.of(Duration.ofDays(20)).grace(Duration.ofMinutes(5)))
> {code}
> In this case, Streams will still implicitly set the window retention 20 days
> (not 20 days + 5 minutes grace), and an exception will be thrown:
> {code:java}
> Exception in thread "main" java.lang.IllegalArgumentException: The retention
> period of the window store KSTREAM-KEY-SELECT-0000000002 must be no smaller
> than its window size plus the grace period. Got size=[1728000000],
> grace=[300000], retention=[1728000000]{code}
> Ideally, Streams should include grace period when implicitly setting window
> retention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
