[
https://issues.apache.org/jira/browse/BEAM-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Kedin updated BEAM-3360:
------------------------------
Description:
Currently when parsing HOP/TUMBLE/SESSION expressions we create a repeating
trigger for the defined windows, see:
{code:java|title=BeamAggregationRule.java}
private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
return
Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
.pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
}
{code}
This will not work correctly with joins, as joins with multiple trigger firings
are currently broken: https://issues.apache.org/jira/browse/BEAM-3190 .
Even if joins with multiple firings worked correctly, SQL parsing stage is
still probably an incorrect place to infer them.
Better alternatives:
- inherit the user-defined triggers for the input pcollection without
modification;
- triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define
a way to backpropagate triggers with correct semantics;
was:
Currently when parsing HOP/TUMBLE expressions we create a repeating trigger for
the defined windows, see:
{code:java|title=BeamAggregationRule.java}
private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
return
Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
.pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
}
{code}
This will not work correctly with joins, as joins with multiple trigger firings
are currently broken: https://issues.apache.org/jira/browse/BEAM-3190 .
Even if joins with multiple firings worked correctly, SQL parsing stage is
still probably an incorrect place to infer them.
Better alternatives:
- inherit the user-defined triggers for the input pcollection without
modification;
- triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define
a way to backpropagate triggers with correct semantics;
> [SQL] Do not assign triggers for HOP/TUMBLE
> -------------------------------------------
>
> Key: BEAM-3360
> URL: https://issues.apache.org/jira/browse/BEAM-3360
> Project: Beam
> Issue Type: Task
> Components: dsl-sql
> Reporter: Anton Kedin
>
> Currently when parsing HOP/TUMBLE/SESSION expressions we create a repeating
> trigger for the defined windows, see:
> {code:java|title=BeamAggregationRule.java}
> private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
> return
> Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
>
> .pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
> }
> {code}
> This will not work correctly with joins, as joins with multiple trigger
> firings are currently broken: https://issues.apache.org/jira/browse/BEAM-3190
> .
> Even if joins with multiple firings worked correctly, SQL parsing stage is
> still probably an incorrect place to infer them.
> Better alternatives:
> - inherit the user-defined triggers for the input pcollection without
> modification;
> - triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might
> define a way to backpropagate triggers with correct semantics;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
