[
https://issues.apache.org/jira/browse/BEAM-7084?focusedWorklogId=246780&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-246780
]
ASF GitHub Bot logged work on BEAM-7084:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/May/19 14:50
Start Date: 22/May/19 14:50
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #8639: [BEAM-7084] Allow
custom environments for external transforms
URL: https://github.com/apache/beam/pull/8639#discussion_r286531920
##########
File path: sdks/python/apache_beam/runners/portability/portable_runner.py
##########
@@ -95,31 +95,40 @@ def default_docker_image():
@staticmethod
def _create_environment(options):
portable_options = options.view_as(PortableOptions)
+ return PortableRunner._resolve_environment(
+ portable_options.environment_type,
+ portable_options.environment_config)
+
+ @staticmethod
+ def _resolve_environment(environment_type, environment_config):
environment_urn = common_urns.environments.DOCKER.urn
- if portable_options.environment_type == 'DOCKER':
+ if environment_type == 'DOCKER':
environment_urn = common_urns.environments.DOCKER.urn
- elif portable_options.environment_type == 'PROCESS':
+ elif environment_type == 'PROCESS':
environment_urn = common_urns.environments.PROCESS.urn
- elif portable_options.environment_type in ('EXTERNAL', 'LOOPBACK'):
+ elif environment_type in ('EXTERNAL', 'LOOPBACK'):
environment_urn = common_urns.environments.EXTERNAL.urn
- elif portable_options.environment_type:
- if portable_options.environment_type.startswith('beam:env:'):
- environment_urn = portable_options.environment_type
+ elif environment_type == 'EMBEDDED':
+ environment_urn = environment_type
+ elif environment_type:
+ # Allow embedded environment for testing
+ if environment_type.startswith('beam:env:') or \
+ environment_type == 'EMBEDDED':
Review comment:
No, the environment_type is a URN. `Embedded` is a special testing URN that
I didn't want to add as a standard urn. So this alongside with the payload
works across all SDKs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 246780)
Time Spent: 2h (was: 1h 50m)
> Support custom environment types when expanding external transforms
> -------------------------------------------------------------------
>
> Key: BEAM-7084
> URL: https://issues.apache.org/jira/browse/BEAM-7084
> Project: Beam
> Issue Type: Bug
> Components: beam-model, sdk-java-core
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The {{ExpansionService}} registers a default Docker JAVA SDK environment.
> Users should be able to specify other types of environment, i.e.
> Process/External/Embedded.
> We can add an {{environment_type}} field to the {{ExpansionRequest}} Proto
> message.
> CC [~robertwb]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
