Thanks Leah!

This kind of assumes an implicit answer to Matthias's question, but I was
wondering
if we should take this opportunity to choose a better default value for the
grace period.
Note that the default of -1 in the TimeWindows class, for example,
ultimately gets
translated into a default value of 24 hours. Which is kind of a long time.

This has hit users of suppression especially hard, since it means you won't
see *any*
output for 24 hours. It's pretty frustrating when you're trying to unit
test your topology
and nothing happens just because you didn't explicitly override the default
grace period.

Unfortunately, we're stuck with the 24hr grace period for our existing
operators for
compatibility reasons. But since this is a new kind of aggregation, we have
the opportunity
to consider alternatives and try to improve on this pain point for users.

Of course, the obvious question now is: what would be a good grace period?
We've
discussed this a number of times before and as far as I know were never
came up with
a good answer. It might also be somewhat confusing for users if different
kinds of
windowed aggregations had different default grace periods, although that's
not a
great reason to keep doing something that's obviously causing problems.

On the other hand, someone recently brought up what I thought was a good
suggestion:
just make the grace period a required parameter. This seems to solve the
existing
problem while dodging the question of what a "good" universal default would
be.

WDYT?

Cheers,
Sophie

On Thu, Jul 9, 2020 at 9:47 PM Matthias J. Sax <mj...@apache.org> wrote:

> Leah,
>
> thanks a lot for the KIP. Very well written.
>
> The KIP does not talk about the handling of out-of-order data though.
> How do you propose to address this?
>
>
> -Matthias
>
> On 7/8/20 5:33 PM, Leah Thomas wrote:
> > Hi all,
> > I'd like to kick-off the discussion for KIP-450, adding sliding window
> > aggregation support to Kafka Streams.
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-450%3A+Sliding+Window+Aggregations+in+the+DSL
> >
> > Let me know what you think,
> > Leah
> >
>
>

Reply via email to