[
https://issues.apache.org/jira/browse/KAFKA-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906530#comment-16906530
]
Lital Eldan commented on KAFKA-6556:
------------------------------------
[~mjsax] i don't see the issue in the plan for release 2.3.1 , is it really in
the plan ?
Our use-case description:
We want in addition for (and after) sending event *per key* when window ends
(using suppress) we also want to send an event that *marks the end of the
window*, so consumers will know that the window currently ended and that they
can start using the new version of data.
> allow to attach callbacks in kafka streams, to be triggered when a window is
> expired
> -------------------------------------------------------------------------------------
>
> Key: KAFKA-6556
> URL: https://issues.apache.org/jira/browse/KAFKA-6556
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Reporter: igor mazor
> Assignee: John Roesler
> Priority: Major
> Labels: kip
>
> Allowing to attach callbacks in kafka streams, to be triggered when a window
> is expired,
> would help in use cases when the final state of the window is required.
> It would be also useful if together with that functionally the user would be
> able to control whether the callback would be triggered in addition to
> emitting the normal change log down the stream, or only triggering the
> callback when the window is expired. (for example in cases when only the
> final window state is required, and any updates to the window state during
> the window time interval are not important)
> An example for use case could be left join with proper sql semantics:
> A left join B currently would emit (a1,null) and (a1,b1) if b1 arrives
> within the defined join time window.
> what I would like is to have ONLY ONE result:
> (a1,null) if no b1 arrived during the defined join time window, OR ONLY
> (a1,b1) if b1 did arrived in the specified join time window.
> One possible solution could be to use the current kafka streams left join
> with downstream processor which would put the results in a windowed Ktable.
> The window size would be same as for the left join operation, however, only
> the final state of that window would be emitted down the stream once the time
> window is expired.
> So if the left join produces (a1, null) and after X minutes no (a1, b1) was
> produced, eventually only (a1, null) would be emitted, on the other hand, if
> the left join produces (a1, null) and after X-t minutes (a1, b1) was produced
> by the left join operation => only (a1, b1) would be emitted eventually down
> the stream after X minutes.
>
> Another use case is when the window state is written to another kafka topic
> which is then used to persist the window states into a db, However, many
> times only the final window state
> is required, and functionality to get only the last window state would help
> in reducing load from the db, since only the final window state would be
> persisted to the db, instead of multiple db writes for each window state
> update.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
