[
https://issues.apache.org/jira/browse/BEAM-5440?focusedWorklogId=314542&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314542
]
ASF GitHub Bot logged work on BEAM-5440:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Sep/19 18:13
Start Date: 18/Sep/19 18:13
Worklog Time Spent: 10m
Work Description: chadrik commented on pull request #8982: [BEAM-5440]
Pass docker run options to SDK harness containers
URL: https://github.com/apache/beam/pull/8982#discussion_r325821356
##########
File path:
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerEnvironmentFactory.java
##########
@@ -136,6 +138,10 @@ public RemoteEnvironment createEnvironment(Environment
environment) throws Excep
dockerArgsBuilder.add("--rm");
}
+ if (!dockerOptions.isEmpty()) {
+ dockerArgsBuilder.addAll(Arrays.asList(dockerOptions.split("\\s+")));
Review comment:
In reviewing this I'm a bit concerned about the splitting on whitespace
issue that @sambvfx pointed out. It's going to break for someone eventually
with something like:
```json
{
"container_name": "MY_CONTAINER_NAME",
"run_options": "-v \"/Volumes/mnt/terrible folder name\""
}
```
This stack overflow post on the subject does not have any easy solutions:
https://stackoverflow.com/questions/1082953/shlex-alternative-for-java
Our options are:
1. deal with this later. it's a solvable problem but it does not
_necessarily_ need to be solved now
2. try to write a `shlex.split` java port now as part of this PR. @mxm do
you know of any open source options for this?
3. use a list of strings for the run options in the docker payload, which
obviously makes this even harder for a user to type the json equivalent, but
makes things easier to code. The json would look like this:
```json
{
"container_name": "MY_CONTAINER_NAME",
"run_options": ["-v", "/Volumes/mnt/terrible folder name"]
}
```
4. same as 3, but as a convenience, use `shlex.split()` in python to prep
the run_options if they are given as a string
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 314542)
Time Spent: 3h (was: 2h 50m)
> Add option to mount a directory inside SDK harness containers
> -------------------------------------------------------------
>
> Key: BEAM-5440
> URL: https://issues.apache.org/jira/browse/BEAM-5440
> Project: Beam
> Issue Type: New Feature
> Components: java-fn-execution, sdk-java-core
> Reporter: Maximilian Michels
> Priority: Major
> Labels: portability, portability-flink
> Time Spent: 3h
> Remaining Estimate: 0h
>
> While experimenting with the Python SDK locally, I found it inconvenient that
> I can't mount a host directory to the Docker containers, i.e. the input must
> already be in the container and the results of a Write remain inside the
> container. For local testing, users may want to mount a host directory.
> Since BEAM-5288 the {{Environment}} carries explicit environment information,
> we could a) add volume args to the {{DockerPayload}}, or b) provide a general
> Docker arguments field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
