[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341127#comment-16341127
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5346


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
> Fix For: 1.5.0
>
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340240#comment-16340240
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on the issue:

https://github.com/apache/flink/pull/5346
  
@tillrohrmann Could you take a final look?


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337961#comment-16337961
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163624908
  
--- Diff: 
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
 ---
@@ -365,6 +386,31 @@ public String call(String s) {
}
}
 
+   public static List 
buildDockerParameters(Option dockerParameters) {
+   if (dockerParameters.isEmpty()) {
+   return Collections.emptyList();
+   } else {
+   String[] dockerParameterSpecifications = 
dockerParameters.get().split(",");
+
+   List parameters = new 
ArrayList<>(dockerParameterSpecifications.length);
+
+   for (String dockerParameterSpecification : 
dockerParameterSpecifications) {
+   if 
(!dockerParameterSpecification.trim().isEmpty()) {
+   String[] match = 
dockerParameterSpecification.split("=", 2);
--- End diff --

I see, thanks for the explanation!


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337843#comment-16337843
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user lishim commented on the issue:

https://github.com/apache/flink/pull/5346
  
Thanks @joerg84 for your review and comments.  I have updated the PR to 
address your concerns.


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337837#comment-16337837
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user lishim commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163596501
  
--- Diff: 
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
 ---
@@ -365,6 +386,31 @@ public String call(String s) {
}
}
 
+   public static List 
buildDockerParameters(Option dockerParameters) {
+   if (dockerParameters.isEmpty()) {
+   return Collections.emptyList();
+   } else {
+   String[] dockerParameterSpecifications = 
dockerParameters.get().split(",");
+
+   List parameters = new 
ArrayList<>(dockerParameterSpecifications.length);
+
+   for (String dockerParameterSpecification : 
dockerParameterSpecifications) {
+   if 
(!dockerParameterSpecification.trim().isEmpty()) {
+   String[] match = 
dockerParameterSpecification.split("=", 2);
--- End diff --

Some Docker parameters are key=value pairs themselves, so key2=val1=val3 is 
actually valid specification.  I added a comment to clarify this, and included 
a test case.


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337839#comment-16337839
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user lishim commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163596583
  
--- Diff: 
flink-mesos/src/test/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParametersTest.java
 ---
@@ -84,6 +84,35 @@ public void testContainerVolumes() throws Exception {
assertEquals(Protos.Volume.Mode.RO, 
params.containerVolumes().get(0).getMode());
}
 
+   @Test
+   public void testContainerDockerParameter() throws Exception {
+   Configuration config = new Configuration();
+   
config.setString(MesosTaskManagerParameters.MESOS_RM_CONTAINER_DOCKER_PARAMETERS,
 "testKey=testValue");
+
+   MesosTaskManagerParameters params = 
MesosTaskManagerParameters.create(config);
+   assertEquals(params.dockerParameters().get(0).getKey(), 
"testKey");
--- End diff --

Added a size check.


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337834#comment-16337834
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user lishim commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163595893
  
--- Diff: docs/ops/deployment/mesos.md ---
@@ -264,6 +264,8 @@ May be set to -1 to disable this feature.
 
 `mesos.resourcemanager.tasks.container.volumes`: A comma separated list of 
`[host_path:]`container_path`[:RO|RW]`. This allows for mounting additional 
volumes into your container. (**NO DEFAULT**)
 
+`mesos.resourcemanager.tasks.container.docker.parameters`: Custom 
parameters to be passed into docker run command when using the docker 
containerizer.  Comma separated list of key=value pairs. (**NO DEFAULT**)
--- End diff --

Fixed


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337832#comment-16337832
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user lishim commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163595840
  
--- Diff: docs/ops/config.md ---
@@ -508,6 +508,8 @@ May be set to -1 to disable this feature.
 
 - `mesos.resourcemanager.tasks.container.volumes`: A comma separated list 
of `[host_path:]`container_path`[:RO|RW]`. This allows for mounting additional 
volumes into your container. (**NO DEFAULT**)
 
+- `mesos.resourcemanager.tasks.container.docker.parameters`: Custom 
parameters to be passed into docker run command when using the docker 
containerizer.  Comma separated list of key=value pairs. (**NO DEFAULT**)
--- End diff --

Fixed


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337276#comment-16337276
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163495632
  
--- Diff: 
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
 ---
@@ -365,6 +386,31 @@ public String call(String s) {
}
}
 
+   public static List 
buildDockerParameters(Option dockerParameters) {
+   if (dockerParameters.isEmpty()) {
+   return Collections.emptyList();
+   } else {
+   String[] dockerParameterSpecifications = 
dockerParameters.get().split(",");
+
+   List parameters = new 
ArrayList<>(dockerParameterSpecifications.length);
+
+   for (String dockerParameterSpecification : 
dockerParameterSpecifications) {
+   if 
(!dockerParameterSpecification.trim().isEmpty()) {
+   String[] match = 
dockerParameterSpecification.split("=", 2);
--- End diff --

I would use split without the limit, i.e., .split("=")
1. You already test for length below
2. this way you would miss invalid specifications such as key2=val1=val3


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337277#comment-16337277
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163483150
  
--- Diff: 
flink-mesos/src/test/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParametersTest.java
 ---
@@ -84,6 +84,35 @@ public void testContainerVolumes() throws Exception {
assertEquals(Protos.Volume.Mode.RO, 
params.containerVolumes().get(0).getMode());
}
 
+   @Test
+   public void testContainerDockerParameter() throws Exception {
+   Configuration config = new Configuration();
+   
config.setString(MesosTaskManagerParameters.MESOS_RM_CONTAINER_DOCKER_PARAMETERS,
 "testKey=testValue");
+
+   MesosTaskManagerParameters params = 
MesosTaskManagerParameters.create(config);
+   assertEquals(params.dockerParameters().get(0).getKey(), 
"testKey");
--- End diff --

maybe check the size first, same below.


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337278#comment-16337278
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163477651
  
--- Diff: docs/ops/config.md ---
@@ -508,6 +508,8 @@ May be set to -1 to disable this feature.
 
 - `mesos.resourcemanager.tasks.container.volumes`: A comma separated list 
of `[host_path:]`container_path`[:RO|RW]`. This allows for mounting additional 
volumes into your container. (**NO DEFAULT**)
 
+- `mesos.resourcemanager.tasks.container.docker.parameters`: Custom 
parameters to be passed into docker run command when using the docker 
containerizer.  Comma separated list of key=value pairs. (**NO DEFAULT**)
--- End diff --

2 blanks between "containerizer.  Comma"


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337279#comment-16337279
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on a diff in the pull request:

https://github.com/apache/flink/pull/5346#discussion_r163477722
  
--- Diff: docs/ops/deployment/mesos.md ---
@@ -264,6 +264,8 @@ May be set to -1 to disable this feature.
 
 `mesos.resourcemanager.tasks.container.volumes`: A comma separated list of 
`[host_path:]`container_path`[:RO|RW]`. This allows for mounting additional 
volumes into your container. (**NO DEFAULT**)
 
+`mesos.resourcemanager.tasks.container.docker.parameters`: Custom 
parameters to be passed into docker run command when using the docker 
containerizer.  Comma separated list of key=value pairs. (**NO DEFAULT**)
--- End diff --

Same here.


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336554#comment-16336554
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user EronWright commented on the issue:

https://github.com/apache/flink/pull/5346
  
+1


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336477#comment-16336477
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

Github user joerg84 commented on the issue:

https://github.com/apache/flink/pull/5346
  
Looks great! I need to test that tomorrow in more detail. 
@tillrohrmann @EronWright 


> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336460#comment-16336460
 ] 

ASF GitHub Bot commented on FLINK-8490:
---

GitHub user lishim opened a pull request:

https://github.com/apache/flink/pull/5346

[FLINK-8490] [mesos] Allow custom docker parameters for docker tasks on 
Mesos

## What is the purpose of the change
- This pull request introduces a feature that allows passing custom 
parameters to docker when using docker containerizer.

## Brief change log
- A new parameter mesos.resourcemanager.tasks.container.docker.parameters 
is introduced

## Verifying this change
This change added tests and can be verified as follows:
- Added unit tests that verify correct behavior of the new parameter
- Manually verified the change by running it in an Mesos cluster with 
Docker containerizer and 2 task managers, setting 
mesos.resourcemanager.tasks.container.docker.parameters=privileged=true and 
verifying that the task manager container is able to run a privileged command, 
then removing the parameter and verifying that the privileged command can no 
longer run.

## Does this pull request potentially affect one of the following parts:
  - Dependencies (does it add or upgrade a dependency): no
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
  - The serializers: no
  - The runtime per-record code paths (performance sensitive): no
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: yes
  - The S3 file system connector: no

## Documentation

  - Does this pull request introduce a new feature? yes
  - If yes, how is the feature documented? docs


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lishim/flink master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5346.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5346


commit bf392edcd446dee375d2e80c7597c13f1479696b
Author: Leonid Ishimnikov 
Date:   2018-01-23T18:55:17Z

[FLINK-8490] Allow custom docker parameters for docker tasks on Mesos




> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-8490) Allow custom docker parameters for docker tasks on Mesos

[JIRA]

[ 
https://issues.apache.org/jira/browse/FLINK-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335667#comment-16335667
 ] 

Jörg Schad commented on FLINK-8490:
---

I would propose the following parameter name:
{{mesos.resourcemanager.tasks.container.docker.parameters}}

> Allow custom docker parameters for docker tasks on Mesos
> 
>
> Key: FLINK-8490
> URL: https://issues.apache.org/jira/browse/FLINK-8490
> Project: Flink
>  Issue Type: Improvement
>  Components: Mesos
>Reporter: Jörg Schad
>Priority: Major
>
> It would be great to pass custom parameters to Mesos when using the Docker 
> Containerizer.
> This could be similar to this spark example: 
> `spark.mesos.executor.docker.parameters privileged=true`
>  
> Originally brought up here: 
> https://stackoverflow.com/questions/48393153/passing-custom-parameters-to-docker-when-running-flink-on-mesos-marathon?noredirect=1#comment83777480_48393153



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)