[
https://issues.apache.org/jira/browse/BEAM-7084?focusedWorklogId=246811&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-246811
]
ASF GitHub Bot logged work on BEAM-7084:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/May/19 15:45
Start Date: 22/May/19 15:45
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #8639: [BEAM-7084] Allow
custom environments for external transforms
URL: https://github.com/apache/beam/pull/8639#discussion_r286561017
##########
File path: sdks/python/apache_beam/runners/portability/portable_runner.py
##########
@@ -95,31 +95,40 @@ def default_docker_image():
@staticmethod
def _create_environment(options):
portable_options = options.view_as(PortableOptions)
+ return PortableRunner._resolve_environment(
+ portable_options.environment_type,
+ portable_options.environment_config)
+
+ @staticmethod
+ def _resolve_environment(environment_type, environment_config):
environment_urn = common_urns.environments.DOCKER.urn
- if portable_options.environment_type == 'DOCKER':
+ if environment_type == 'DOCKER':
environment_urn = common_urns.environments.DOCKER.urn
- elif portable_options.environment_type == 'PROCESS':
+ elif environment_type == 'PROCESS':
environment_urn = common_urns.environments.PROCESS.urn
- elif portable_options.environment_type in ('EXTERNAL', 'LOOPBACK'):
+ elif environment_type in ('EXTERNAL', 'LOOPBACK'):
environment_urn = common_urns.environments.EXTERNAL.urn
- elif portable_options.environment_type:
- if portable_options.environment_type.startswith('beam:env:'):
- environment_urn = portable_options.environment_type
+ elif environment_type == 'EMBEDDED':
+ environment_urn = environment_type
+ elif environment_type:
+ # Allow embedded environment for testing
+ if environment_type.startswith('beam:env:') or \
+ environment_type == 'EMBEDDED':
Review comment:
Agree, we should give it a proper, private URN. Yes, reusing the existing
Python code here has some problematic implications. We probably only want to
accept the URN, or come up with a better way to specify that we are using the
embedded environment of an SDK (e.g. `Embedded:Java`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 246811)
Time Spent: 2h 50m (was: 2h 40m)
> Support custom environment types when expanding external transforms
> -------------------------------------------------------------------
>
> Key: BEAM-7084
> URL: https://issues.apache.org/jira/browse/BEAM-7084
> Project: Beam
> Issue Type: Bug
> Components: beam-model, sdk-java-core
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> The {{ExpansionService}} registers a default Docker JAVA SDK environment.
> Users should be able to specify other types of environment, i.e.
> Process/External/Embedded.
> We can add an {{environment_type}} field to the {{ExpansionRequest}} Proto
> message.
> CC [~robertwb]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
