[
https://issues.apache.org/jira/browse/BEAM-4605?focusedWorklogId=114431&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-114431
]
ASF GitHub Bot logged work on BEAM-4605:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jun/18 19:06
Start Date: 21/Jun/18 19:06
Worklog Time Spent: 10m
Work Description: robertwb closed pull request #5702: [BEAM-4605] Remove
subtransform of GBK.
URL: https://github.com/apache/beam/pull/5702
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/runners/portability/portable_runner.py
b/sdks/python/apache_beam/runners/portability/portable_runner.py
index 295b985bfe0..82c47b46da4 100644
--- a/sdks/python/apache_beam/runners/portability/portable_runner.py
+++ b/sdks/python/apache_beam/runners/portability/portable_runner.py
@@ -23,6 +23,7 @@
from apache_beam import coders
from apache_beam.internal import pickler
+from apache_beam.portability import common_urns
from apache_beam.portability.api import beam_job_api_pb2
from apache_beam.portability.api import beam_job_api_pb2_grpc
from apache_beam.runners import pipeline_context
@@ -90,6 +91,15 @@ def run_pipeline(self, pipeline):
pcoll.coder_id = proto_context.coders.get_id(coder)
proto_context.coders.populate_map(proto_pipeline.components.coders)
+ # Some runners won't detect the GroupByKey transform unless it has no
+ # subtransforms. Remove all sub-transforms until BEAM-4605 is resolved.
+ for _, transform_proto in list(
+ proto_pipeline.components.transforms.items()):
+ if transform_proto.spec.urn == common_urns.primitives.GROUP_BY_KEY.urn:
+ for sub_transform in transform_proto.subtransforms:
+ del proto_pipeline.components.transforms[sub_transform]
+ del transform_proto.subtransforms[:]
+
job_service = self._create_job_service()
prepare_response = job_service.Prepare(
beam_job_api_pb2.PrepareJobRequest(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 114431)
Time Spent: 50m (was: 40m)
> Make GroupByKey the primitive in the Python SDK
> -----------------------------------------------
>
> Key: BEAM-4605
> URL: https://issues.apache.org/jira/browse/BEAM-4605
> Project: Beam
> Issue Type: Task
> Components: sdk-py-core
> Reporter: Robert Bradshaw
> Assignee: Ahmet Altay
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently it is still a composite, with GBKOnly as a primitive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
