[
https://issues.apache.org/jira/browse/BEAM-7972?focusedWorklogId=294280&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-294280
]
ASF GitHub Bot logged work on BEAM-7972:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Aug/19 23:38
Start Date: 13/Aug/19 23:38
Worklog Time Spent: 10m
Work Description: y1chi commented on pull request #9334: [BEAM-7972]
Always use Global window in reshuffle and then apply wind…
URL: https://github.com/apache/beam/pull/9334#discussion_r313655400
##########
File path: sdks/python/apache_beam/transforms/util.py
##########
@@ -634,12 +630,13 @@ def restore_timestamps(element, window=DoFn.WindowParam):
ungrouped = pcoll | Map(reify_timestamps)
ungrouped._windowing = Windowing(
- window_fn,
+ window.GlobalWindows(),
triggerfn=AfterCount(1),
accumulation_mode=AccumulationMode.DISCARDING,
timestamp_combiner=TimestampCombiner.OUTPUT_AT_EARLIEST)
result = (ungrouped
| GroupByKey()
+ | WindowInto(window_fn)
Review comment:
seems line 641 is restoring the window, is applying window_fn here makes a
difference?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 294280)
Time Spent: 0.5h (was: 20m)
> Portable Python Reshuffle does not work with with windowed pcollection
> ----------------------------------------------------------------------
>
> Key: BEAM-7972
> URL: https://issues.apache.org/jira/browse/BEAM-7972
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Ankur Goenka
> Assignee: Ankur Goenka
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Streaming pipeline gets stuck when using Reshuffle with windowed pcollection.
> The issue happen because of window function gets deserialized on java side
> which is not possible and hence default to global window function and result
> into window function mismatch later down the code.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
