[
https://issues.apache.org/jira/browse/BEAM-4605?focusedWorklogId=113958&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-113958
]
ASF GitHub Bot logged work on BEAM-4605:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Jun/18 21:57
Start Date: 20/Jun/18 21:57
Worklog Time Spent: 10m
Work Description: angoenka commented on a change in pull request #5702:
[BEAM-4605] Remove subtransform of GBK.
URL: https://github.com/apache/beam/pull/5702#discussion_r196955071
##########
File path: sdks/python/apache_beam/runners/portability/portable_runner.py
##########
@@ -90,6 +91,15 @@ def run_pipeline(self, pipeline):
pcoll.coder_id = proto_context.coders.get_id(coder)
proto_context.coders.populate_map(proto_pipeline.components.coders)
+ # Some runners won't detect the GroupByKey transform unless it has no
+ # subtransforms. Remove all sub-transforms until BEAM-4605 is resolved.
+ for transform_id, transform_proto in list(
+ proto_pipeline.components.transforms.items()):
+ if transform_proto.spec.urn == common_urns.primitives.GROUP_BY_KEY.urn:
+ for sub_transform in transform_proto.subtransforms:
Review comment:
Should we also log these removal for better debug-ability of the pipeline?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 113958)
Time Spent: 0.5h (was: 20m)
> Make GroupByKey the primitive in the Python SDK
> -----------------------------------------------
>
> Key: BEAM-4605
> URL: https://issues.apache.org/jira/browse/BEAM-4605
> Project: Beam
> Issue Type: Task
> Components: sdk-py-core
> Reporter: Robert Bradshaw
> Assignee: Ahmet Altay
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently it is still a composite, with GBKOnly as a primitive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
