[
https://issues.apache.org/jira/browse/BEAM-8240?focusedWorklogId=313168&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313168
]
ASF GitHub Bot logged work on BEAM-8240:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Sep/19 17:02
Start Date: 16/Sep/19 17:02
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #9583:
[BEAM-8240] Ensure that the pipeline proto contains the
worker_harness_container_image override
URL: https://github.com/apache/beam/pull/9583#discussion_r324783913
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
##########
@@ -176,6 +176,19 @@ def test_create_runner(self):
isinstance(create_runner('TestDataflowRunner'),
TestDataflowRunner))
+ def test_environment_override_translation(self):
+ self.default_properties.append("--experiments=beam_fn_api")
+ self.default_properties.append("--worker_harness_container_image=FOO")
+ remote_runner = DataflowRunner()
+ p = Pipeline(remote_runner,
+ options=PipelineOptions(self.default_properties))
+ (p | ptransform.Create([1, 2, 3]) # pylint:
disable=expression-not-assigned
+ | 'Do' >> ptransform.FlatMap(lambda x: [(x, x)])
+ | ptransform.GroupByKey())
+ p.run()
+ pipeline_proto = remote_runner.pipeline_proto
+ self.assertEqual(pipeline_proto.components.envirnoments.values(), ["FOO"])
Review comment:
Just to clarify, does "["FOO"]" somehow end up being equal to a
DockerPayload with "FOO" set to container_image ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 313168)
Time Spent: 50m (was: 40m)
> SDK Harness
> -----------
>
> Key: BEAM-8240
> URL: https://issues.apache.org/jira/browse/BEAM-8240
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: Minor
> Time Spent: 50m
> Remaining Estimate: 0h
>
> SDK harness incorrectly identifies itself when using custom SDK container
> within environment field when building pipeline proto.
>
> Passing in the experiment *worker_harness_container_image=YYY* doesn't
> override the pipeline proto environment field and it is still being populated
> with *gcr.io/cloud-dataflow/v1beta3/python-fnapi:beam-master-20190802*
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
