[
https://issues.apache.org/jira/browse/BEAM-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles updated BEAM-2504:
----------------------------------
Description:
The affects accumulating mode most acutely: when a processing time trigger
fires and processing is ahead of the watermark (which it usually will be) there
will be a final pane from the premature GC. Then, if no more data comes in, the
GC timer fires and outputs _another_ pane marked "final" but this time with an
empty (aka zero) accumulator. This is a likely scenario since the processing
time timer is probably just a smidge ahead of the watermark. So any sink
writing the accumulated value will fill with zeros mostly, some correct values,
and other values that are just the data from the last moments before GC.
This would also have a severe affect on merging windows, since merging
terminates upon GC.
> Processing time timers with timestamp past GC time are interpreted as GC
> timers
> -------------------------------------------------------------------------------
>
> Key: BEAM-2504
> URL: https://issues.apache.org/jira/browse/BEAM-2504
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
> Priority: Blocker
> Fix For: 2.1.0
>
>
> The affects accumulating mode most acutely: when a processing time trigger
> fires and processing is ahead of the watermark (which it usually will be)
> there will be a final pane from the premature GC. Then, if no more data comes
> in, the GC timer fires and outputs _another_ pane marked "final" but this
> time with an empty (aka zero) accumulator. This is a likely scenario since
> the processing time timer is probably just a smidge ahead of the watermark.
> So any sink writing the accumulated value will fill with zeros mostly, some
> correct values, and other values that are just the data from the last moments
> before GC.
> This would also have a severe affect on merging windows, since merging
> terminates upon GC.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
