[
https://issues.apache.org/jira/browse/FLINK-29692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713726#comment-17713726
]
Chalres Tan commented on FLINK-29692:
-------------------------------------
hi [~martijnvisser], there are many use cases that I believe can benefit from a
feature like this. One use case example would be fraud detection. Let's say you
want to be notified if a withdrawal from a bank account happens 3 times in an
hour. With a TUMBLING window of 1 hour, it isn't acceptable that you should
need to wait an hour for the window to end before realizing that a potential
fraud has happened. Of course CUMULATE window can reduce this delay, but like I
mentioned above we would be opening many more windows depending on how often we
want to emit an intermediate result which can be resource intensive. I think
supporting an early fire configuration, as was supported with older windowing
functions, for the newer TVFs would be a more streamlined way of supporting use
cases like these.
A secondary point is that windows in ksqldb have early fire by design (they
struggle in the opposite way here in that they don't easily support emitting
one correct result per window). Supporting an early fire configuration makes it
much easier for users who are trying to migrate their use cases from ksqldb to
Flink.
> Support early/late fires for Windowing TVFs
> -------------------------------------------
>
> Key: FLINK-29692
> URL: https://issues.apache.org/jira/browse/FLINK-29692
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
> Affects Versions: 1.15.3
> Reporter: Canope Nerda
> Priority: Major
>
> I have cases where I need to 1) output data as soon as possible and 2) handle
> late arriving data to achieve eventual correctness. In the logic, I need to
> do window deduplication which is based on Windowing TVFs and according to
> source code, early/late fires are not supported yet in Windowing TVFs.
> Actually 1) contradicts with 2). Without early/late fires, we had to
> compromise, either live with fresh incorrect data or tolerate excess latency
> for correctness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
