[
https://issues.apache.org/jira/browse/BEAM-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh resolved BEAM-1721.
-------------------------------
Resolution: Fixed
Fix Version/s: First stable release
> Reshuffle can shift elements in time
> ------------------------------------
>
> Key: BEAM-1721
> URL: https://issues.apache.org/jira/browse/BEAM-1721
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Thomas Groh
> Assignee: Thomas Groh
> Fix For: First stable release
>
>
> The reshuffle transform is meant to have no visible effects on the data that
> it processes. However, due to the use of a {{GroupByKey}}, the timestamp of
> the output elements is determined by the {{OutputTimeFn}} of the input
> {{WindowingStrategy}}
> Elements should not be shifted in time when being processed in {{Reshuffle}}.
> Currently this would require reifying all timestamps before applying the
> GroupByKey and reapplying them after. As an intermediate solution, elements
> should never be shifted forwards in time, as doing so permits the watermark
> to advance improperly (if the elements already contain their timestamps, for
> example), and prevents the timestamps from being reassigned within a {{DoFn}}
> or via the {{WithTimestamps}} transform.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
