[
https://issues.apache.org/jira/browse/BEAM-6978?focusedWorklogId=223573&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223573
]
ASF GitHub Bot logged work on BEAM-6978:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Apr/19 12:50
Start Date: 05/Apr/19 12:50
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #8204: [BEAM-6978]
Have dataflow service receive additional arguments needed to support
SplittableDoFn.
URL: https://github.com/apache/beam/pull/8204#discussion_r272572759
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -781,6 +783,16 @@ def run_ParDo(self, transform_node, options):
step.add_property(PropertyNames.OUTPUT_INFO, outputs)
+ # Add the restriction encoding if we are a splittable DoFn
+ # and are using the Fn API on the unified worker.
+ from apache_beam.runners.common import DoFnSignature
+ signature = DoFnSignature(transform_node.transform.fn)
+ if (use_fnapi and use_unified_worker and signature.is_splittable_dofn()):
+ restriction_coder = (
+ signature.get_restriction_provider().restriction_coder())
+ step.add_property('restriction_encoding',
Review comment:
Could you put this constant in PropertyNames?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 223573)
Time Spent: 1h (was: 50m)
> Make Dataflow service receive required restriction encoding parameter on
> SplittableParDo
> ----------------------------------------------------------------------------------------
>
> Key: BEAM-6978
> URL: https://issues.apache.org/jira/browse/BEAM-6978
> Project: Beam
> Issue Type: Sub-task
> Components: runner-dataflow
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: Major
> Labels: portability
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The Dataflow service expects a "restriction_encoding" field to be populated
> with the coder CloudObject representation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
