[jira] [Commented] (MESOS-7693) DEBUG container does not inherit env variable properly for command tasks.

2018-04-10 Thread Kevin Klues (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432898#comment-16432898
 ] 

Kevin Klues commented on MESOS-7693:


The underlying mesos implementation of debug containers doesn't allow an exec'd 
container to inherit environment variables from those defined in the parent 
container's base docker image.

For example, when running the following directly on docker, we see the 
{{LD_LIBRARY_PATH}} set from the base {{nvidia/cuda}} image inside the exec'd 
process:

{noformat}
[klueska@core-dev ~]$ docker run -d nvidia/cuda sleep 
1a78ee1eed45947952bc0a80354d3419423e6d5a52ee43909a54f54e3b751d3e
[klueska@core-dev ~]$ docker ps
CONTAINER IDIMAGE   COMMAND CREATED 
STATUS  PORTS   NAMES
1a78ee1eed45nvidia/cuda "sleep "9 seconds ago   
Up 4 secondsvibrant_volhard
[klueska@core-dev ~]$ docker exec -it 1a78ee1eed45 bash -c "echo 
\$LD_LIBRARY_PATH"
/usr/local/nvidia/lib:/usr/local/nvidia/lib64
{noformat}

But when running the equivalent on {{dcos}} via marathon:

{noformat}
$ cat cuda.json
{
  "id": "nvidia-test",
  "mem": 2048,
  "disk": 0,
  "instances": 1,
  "cmd": "sleep 999",
  "container": {
"type": "MESOS",
"docker": {
  "image": "nvidia/cuda"
}
  }
}

$ dcos marathon app add cuda.json
Created deployment 3d02ceb1-519c-4f08-a85c-bb0b8134c630
$ dcos task 
NAME HOST USER  STATE  ID   
 MESOS ID   REGION  ZONE
  
nvidia-test  10.10.0.183  rootR
nvidia-test.3f941a9f-0a8c-11e8-b838-f638b54f6560  
e896367a-8dbe-40d6-9867-1af79b77fb04-S0  us-west-2  us-west-2c
$ dcos task exec -it nvidia-test bash -c "echo \$LD_LIBRARY_PATH"
{noformat}

We need to update the logic for debug containers to inherit this properly:
This will likely involve an update to 
https://github.com/apache/mesos/blob/56100adc14a8acbcf2ee0aae967f758f592fcd31/src/slave/containerizer/mesos/containerizer.cpp#L1613


> DEBUG container does not inherit env variable properly for command tasks.
> -
>
> Key: MESOS-7693
> URL: https://issues.apache.org/jira/browse/MESOS-7693
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Jie Yu
>Assignee: Alexander Rukletsov
>Priority: Major
>
> I can repo the issue:
> {code}
> sudo /home/vagrant/workspace/dist/mesos-1.4.0/bin/mesos-execute 
> --master=172.28.128.3:5050 --name=java8 --docker_image=java:8 
> --command="sleep 1000"
> I0618 17:42:21.410598  3356 scheduler.cpp:184] Version: 1.4.0
> I0618 17:42:21.413465  3356 scheduler.cpp:470] New master detected at 
> master@172.28.128.3:5050
> Subscribed with ID cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> Submitted task 'java8' to agent 'cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0'
> Received status update TASK_RUNNING for task 'java8'
>   source: SOURCE_EXECUTOR
> Jies-MacBook-Pro:script jie$ ./dcos task
> NAME   HOST  USER  STATE  ID
> java8  172.28.128.3  rootRjava8
> Jies-MacBook-Pro:script jie$ ./dcos task exec -t -i java8 bash
> root@vagrant-ubuntu-trusty-64:/mnt/mesos/sandbox# env
> LIBPROCESS_IP=172.28.128.3
> MESOS_AGENT_ENDPOINT=172.28.128.3:5051
> MESOS_DIRECTORY=/tmp/mesos/slave/slaves/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0/frameworks/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006/executors/java8/runs/1b06c661-20f3-460a-8cfd-475dc3e60aa3
> MESOS_EXECUTOR_ID=java8
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> PWD=/mnt/mesos/sandbox
> MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD=5secs
> MESOS_NATIVE_JAVA_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_NATIVE_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_HTTP_COMMAND_EXECUTOR=0
> MESOS_SLAVE_PID=slave(1)@172.28.128.3:5051
> MESOS_FRAMEWORK_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> MESOS_CHECKPOINT=0
> SHLVL=1
> LIBPROCESS_PORT=0
> MESOS_SLAVE_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0
> MESOS_SANDBOX=/mnt/mesos/sandbox
> _=/usr/bin/env
> {code}
> As you can see, environment variables like JAVA_HOME defined in the docker 
> image are not in the debug container.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-7693) DEBUG container does not inherit env variable properly for command tasks.

2017-06-20 Thread Alexander Rukletsov (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16055622#comment-16055622
 ] 

Alexander Rukletsov commented on MESOS-7693:


Does not this happen because of MESOS-7692, i.e. the parent container does not 
have the env var set and hence it is not inherited, [~jieyu]?

> DEBUG container does not inherit env variable properly for command tasks.
> -
>
> Key: MESOS-7693
> URL: https://issues.apache.org/jira/browse/MESOS-7693
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Jie Yu
>Assignee: Alexander Rukletsov
>
> I can repo the issue:
> {code}
> sudo /home/vagrant/workspace/dist/mesos-1.4.0/bin/mesos-execute 
> --master=172.28.128.3:5050 --name=java8 --docker_image=java:8 
> --command="sleep 1000"
> I0618 17:42:21.410598  3356 scheduler.cpp:184] Version: 1.4.0
> I0618 17:42:21.413465  3356 scheduler.cpp:470] New master detected at 
> master@172.28.128.3:5050
> Subscribed with ID cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> Submitted task 'java8' to agent 'cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0'
> Received status update TASK_RUNNING for task 'java8'
>   source: SOURCE_EXECUTOR
> Jies-MacBook-Pro:script jie$ ./dcos task
> NAME   HOST  USER  STATE  ID
> java8  172.28.128.3  rootRjava8
> Jies-MacBook-Pro:script jie$ ./dcos task exec -t -i java8 bash
> root@vagrant-ubuntu-trusty-64:/mnt/mesos/sandbox# env
> LIBPROCESS_IP=172.28.128.3
> MESOS_AGENT_ENDPOINT=172.28.128.3:5051
> MESOS_DIRECTORY=/tmp/mesos/slave/slaves/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0/frameworks/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006/executors/java8/runs/1b06c661-20f3-460a-8cfd-475dc3e60aa3
> MESOS_EXECUTOR_ID=java8
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> PWD=/mnt/mesos/sandbox
> MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD=5secs
> MESOS_NATIVE_JAVA_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_NATIVE_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_HTTP_COMMAND_EXECUTOR=0
> MESOS_SLAVE_PID=slave(1)@172.28.128.3:5051
> MESOS_FRAMEWORK_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> MESOS_CHECKPOINT=0
> SHLVL=1
> LIBPROCESS_PORT=0
> MESOS_SLAVE_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0
> MESOS_SANDBOX=/mnt/mesos/sandbox
> _=/usr/bin/env
> {code}
> As you can see, environment variables like JAVA_HOME defined in the docker 
> image are not in the debug container.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7693) DEBUG container does not inherit env variable properly for command tasks.

2017-06-18 Thread Jie Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MESOS-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053268#comment-16053268
 ] 

Jie Yu commented on MESOS-7693:
---

[~alexr] Can you take a look at this issue?

> DEBUG container does not inherit env variable properly for command tasks.
> -
>
> Key: MESOS-7693
> URL: https://issues.apache.org/jira/browse/MESOS-7693
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Jie Yu
>Assignee: Alexander Rukletsov
>
> I can repo the issue:
> {code}
> sudo /home/vagrant/workspace/dist/mesos-1.4.0/bin/mesos-execute 
> --master=172.28.128.3:5050 --name=java8 --docker_image=java:8 
> --command="sleep 1000"
> I0618 17:42:21.410598  3356 scheduler.cpp:184] Version: 1.4.0
> I0618 17:42:21.413465  3356 scheduler.cpp:470] New master detected at 
> master@172.28.128.3:5050
> Subscribed with ID cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> Submitted task 'java8' to agent 'cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0'
> Received status update TASK_RUNNING for task 'java8'
>   source: SOURCE_EXECUTOR
> Jies-MacBook-Pro:script jie$ ./dcos task
> NAME   HOST  USER  STATE  ID
> java8  172.28.128.3  rootRjava8
> Jies-MacBook-Pro:script jie$ ./dcos task exec -t -i java8 bash
> root@vagrant-ubuntu-trusty-64:/mnt/mesos/sandbox# env
> LIBPROCESS_IP=172.28.128.3
> MESOS_AGENT_ENDPOINT=172.28.128.3:5051
> MESOS_DIRECTORY=/tmp/mesos/slave/slaves/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0/frameworks/cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006/executors/java8/runs/1b06c661-20f3-460a-8cfd-475dc3e60aa3
> MESOS_EXECUTOR_ID=java8
> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> PWD=/mnt/mesos/sandbox
> MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD=5secs
> MESOS_NATIVE_JAVA_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_NATIVE_LIBRARY=/home/vagrant/workspace/dist/mesos-1.4.0/lib/libmesos-1.4.0.so
> MESOS_HTTP_COMMAND_EXECUTOR=0
> MESOS_SLAVE_PID=slave(1)@172.28.128.3:5051
> MESOS_FRAMEWORK_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-0006
> MESOS_CHECKPOINT=0
> SHLVL=1
> LIBPROCESS_PORT=0
> MESOS_SLAVE_ID=cacf5c08-cbbc-401a-a84d-2cfc4edc6519-S0
> MESOS_SANDBOX=/mnt/mesos/sandbox
> _=/usr/bin/env
> {code}
> As you can see, environment variables like JAVA_HOME defined in the docker 
> image are not in the debug container.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)