[
https://issues.apache.org/jira/browse/BEAM-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948727#comment-15948727
]
Etienne Chauchot commented on BEAM-1261:
----------------------------------------
I have one possible use case of this. I'm sure [~kenn] you know the one I'am
talking about :)
In nexmark query3 uses state to do an incremental join of the auctions and the
people. Auctions and person events can arrive out of order and in different
(fixed) windows
- person element is stored in state in order to match future auctions by that
person
- auction elements are stored in state until we have seen the corresponding
person record
But state seem to become useless for this query because it is actually scoped
to a window, so when the stored element will be needed in a future window, it
will not be there anymore.
> State API should allow state to be managed in different windows
> ---------------------------------------------------------------
>
> Key: BEAM-1261
> URL: https://issues.apache.org/jira/browse/BEAM-1261
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, sdk-java-core
> Reporter: Ben Chambers
>
> For example, even if the elements are being processed in fixed windows of an
> hour, it may be desirable for the state to "roll over" between windows (or be
> available to all windows).
> It will also be necessary to figure out when this state should be deleted
> (TTL? maximum retention?)
> Another problem is how to deal with out of order data. If data comes in from
> the 10:00 AM window, should its state changes be visible to the data in the
> 9:00 AM window?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
