[
https://issues.apache.org/jira/browse/BEAM-8396?focusedWorklogId=335587&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-335587
]
ASF GitHub Bot logged work on BEAM-8396:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Oct/19 16:11
Start Date: 29/Oct/19 16:11
Worklog Time Spent: 10m
Work Description: ibzib commented on pull request #9833: [BEAM-8396]
Default to LOOPBACK mode for local flink runner.
URL: https://github.com/apache/beam/pull/9833#discussion_r340176458
##########
File path: sdks/python/apache_beam/runners/portability/flink_runner.py
##########
@@ -34,7 +34,9 @@ class FlinkRunner(portable_runner.PortableRunner):
def default_job_server(self, options):
flink_master = options.view_as(FlinkRunnerOptions).flink_master
if flink_master == '[local]' or sys.version_info < (3, 6):
- # TOOD(BEAM-8396): Also default to LOOPBACK for [local].
+ portable_options = options.view_as(pipeline_options.PortableOptions)
+ if flink_master == '[local]' and not portable_options.environment_type:
+ portable_options.environment_type == 'LOOPBACK'
Review comment:
There's a little blurb on loopback here:
https://beam.apache.org/roadmap/portability/#sdk-harness-config
I think the main goal of this PR was to prevent local file system issues
when using Docker workers (files not being shared between workers, "Where'd my
output go?") for people using the Flink runner for the first time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 335587)
Time Spent: 1h (was: 50m)
> Default to LOOPBACK mode for local flink (spark, ...) runner.
> -------------------------------------------------------------
>
> Key: BEAM-8396
> URL: https://issues.apache.org/jira/browse/BEAM-8396
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Robert Bradshaw
> Assignee: Robert Bradshaw
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> As well as being lower overhead, this will avoid surprises about workers
> operating within the docker filesystem, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
