The problem happens to me if I don't specify the --docker_mesos_image flag.
However, specifying the flag only makes things worse: the task is failed
again and agin, but there does exist a container for this task.
master and zookeeper is running on host, and slave is running inside a
docker image:
```
sudo docker run -it --rm \
-e MESOS_HOSTNAME=localhost \
-e MESOS_IP=127.0.0.1 \
-e MESOS_MASTER=zk://127.0.0.1:2181/mesos \
-v /sys/fs/cgroup:/sys/fs/cgroup \
-v /var/run/docker.sock:/var/run/docker.sock \
--name mesos-slave \
--net host \
--privileged \
mesoscloud/mesos-slave:0.24.1-ubuntu-14.04
```
However my setup may affect the outcome: master is 0.25.0 and slave is
0.24.1 (can't find a public docker image for mesos 2.5.1)
Output of http http://127.0.0.1:8080/v2/apps (unrelevant part ommited)
```
{
"apps": [
"container": {
"docker": {
"parameters": [],
"privileged": false,
"network": "BRIDGE",
"image": "testapp:latest"
},
"volumes": [],
"type": "DOCKER"
},
"uris": [
"https://google.com/robots.txt";
],
}
]
}
```
On Fri, Dec 18, 2015 at 7:11 PM, Grzegorz Graczyk <[email protected]>
wrote:
> I've tried to use this flag, but cannot really run any container when this
> flag is set.
> I've raised this issue here:
> https://www.mail-archive.com/[email protected]/msg04975.html and
> here:
> https://github.com/mesosphere/docker-containers/issues/6#issuecomment-155364351
> but
> sadly no one was able to help me...
>
> pt., 18.12.2015 o 11:33 użytkownik Marica Antonacci <
> [email protected]> napisał:
>
>> OK, the problem I spotted is related to the usage of the
>> flag —docker_mesos_image that allows the executor to
>>
>>
>> --docker_mesos_image=VALUEThe docker image used to launch this mesos
>> slave instance. If an image is specified, the docker containerizer assumes
>> the slave is running in a docker container, and launches executors with
>> docker containers in order to recover them when the slave restarts and
>> recovers.
>> Has anyone used this flag and tested the behavior of the fetcher?
>>
>> Thank you
>> Marica
>>
>>
>> Il giorno 18/dic/2015, alle ore 10:38, tommy xiao <[email protected]> ha
>> scritto:
>>
>> no docker_mesos_image flag in my docker run, and the docker image is
>> build by myself.
>>
>>
>>
>> 2015-12-18 17:20 GMT+08:00 Marica Antonacci <[email protected]>
>> :
>>
>> Yes, I did check inside the container and the csv file was not downloaded
>>> as shown also by the app details (see the screenshot below).
>>>
>>> Are you running your slave with the --docker_mesos_image flag? Can you
>>> please provide me the docker run command you are using to run your
>>> dockerized slave?
>>>
>>> Thank you very much
>>>
>> Marica
>>>
>>>
>>> <Schermata 2015-12-18 alle 10.15.58.png>
>>>
>>
>>>
>>> Il giorno 18/dic/2015, alle ore 10:00, tommy xiao <[email protected]> ha
>>> scritto:
>>>
>>> Hi Marica,
>>>
>>> use your test-app json, i can run it correctly, the csv is truely
>>> download by mesos slave. please check mesos-master:5050 to check the task
>>> detail download files.
>>>
>>> you describe the app container why not found the csv, because the csv is
>>> download in slave container's folder, not in app container. so if you run
>>>
>>> cd $MESOS_SANDBOX;
>>>
>>> the folder in app container is default value:
>>>
>>> MESOS_SANDBOX=/mnt/mesos/sandbox
>>> but in real world, the sandbox is in slave container, not in app
>>> container.
>>>
>>>
>>>
>>> 2015-12-18 16:11 GMT+08:00 Marica Antonacci <[email protected]
>>> >:
>>>
>>>> Thank you very much,
>>>>
>>>> I’m using a sample application definition file, just for testing
>>>> purpose:
>>>>
>>>> {
>>>> "id": "test-app",
>>>> "container": {
>>>> "type": "DOCKER",
>>>> "docker": {
>>>> "image": "libmesos/ubuntu"
>>>> }
>>>> },
>>>> "cpus": 1,
>>>> "mem": 512,
>>>> *"uris": [
>>>> "http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv
>>>> <http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv>"
>>>> ],*
>>>> "cmd": "cd $MESOS_SANDBOX; ls -latr; while sleep 10; do date -u +%T;
>>>> done"
>>>> }
>>>>
>>>> Here is the docker run command line:
>>>>
>>>> # docker run -d -e MESOS_HOSTNAME=<SLAVE_IP> -e MESOS_IP=<SLAVE_IP> -e
>>>> MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos
>>>> -e MESOS_CONTAINERIZERS=docker,mesos \
>>>> -e MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e
>>>> MESOS_LOG_DIR=/var/log -e MESOS_docker_mesos_image=mesos-slave
>>>> -v /sys/fs/cgroup:/sys/fs/cgroup -v
>>>> /var/run/docker.sock:/var/run/docker.sock --name slave --net host
>>>> --privileged --pid host mesos-slave
>>>>
>>>>
>>>> As already mentioned, if I remove the environment variable
>>>> MESOS_docker_mesos_image the fetcher works fine and I can see the file
>>>> snoqualmie.csv inside the sandbox.
>>>>
>>>> Thank you again! I’m looking forward to hearing about your outcomes.
>>>> Best regards,
>>>> Marica
>>>>
>>>>
>>>> Il giorno 18/dic/2015, alle ore 04:51, tommy xiao <[email protected]>
>>>> ha scritto:
>>>>
>>>> Hi Marica,
>>>>
>>>> Could you please give a sample marathon json, i can test it asap.
>>>>
>>>> 2015-12-18 5:11 GMT+08:00 Marica Antonacci <[email protected]
>>>> >:
>>>>
>>>>> No, using the socket:
>>>>>
>>>>> -v /var/run/docker.sock:/var/run/docker.sock
>>>>>
>>>>>
>>>>> Il giorno 17/dic/2015, alle ore 18:07, tommy xiao <[email protected]>
>>>>> ha scritto:
>>>>>
>>>>> docker in docker mode?
>>>>>
>>>>> 2015-12-17 19:08 GMT+08:00 Marica Antonacci <
>>>>> [email protected]>:
>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> I'm testing the URIs fetching mechanism for both Marathon
>>>>>> applications and Chronos jobs and I have found that if the slave is
>>>>>> running
>>>>>> inside a docker container (using *docker_mesos_image* startup flag)
>>>>>> and you submit the deployment of a dockerized application or job the
>>>>>> fetcher step is not performed. On the other hand, if I request the
>>>>>> deployment of a non-dockerized application, the URIs are correctly
>>>>>> fetched.
>>>>>> Moreover, if I don’t provide the docker_mesos_image flag, the fetcher
>>>>>> works
>>>>>> fine again for both dockerized and non-dockerized applications.
>>>>>>
>>>>>> Therefore, it seems that the information about the URIs gets lost
>>>>>> when the dockerized mesos slave spawns the executor docker container that
>>>>>> in turn launches the application docker container…Has anyone seen this
>>>>>> problem before? I would like to know if there is a workaround or a
>>>>>> fixing.
>>>>>>
>>>>>> Thanks a lot in advance for you help
>>>>>> Best Regards,
>>>>>> Marica
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------
>>>>>>
>>>>>> Marica ANTONACCI
>>>>>> INFN - National Institute of Nuclear Physics
>>>>>> Via Orabona 4
>>>>>> 70126 Bari - ITALY
>>>>>> Phone +39 080 5443244
>>>>>> Skype: marica.antonacci
>>>>>> e-mail [email protected]
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Deshi Xiao
>>>>> Twitter: xds2000
>>>>> E-mail: xiaods(AT)gmail.com
>>>>>
>>>>>
>>>>> ----------------------------------------------------------
>>>>>
>>>>> Marica ANTONACCI
>>>>> INFN - National Institute of Nuclear Physics
>>>>> Via Orabona 4
>>>>> 70126 Bari - ITALY
>>>>> Phone +39 080 5443244
>>>>> Skype: marica.antonacci
>>>>> e-mail [email protected]
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Deshi Xiao
>>>> Twitter: xds2000
>>>> E-mail: xiaods(AT)gmail.com
>>>>
>>>>
>>>> ----------------------------------------------------------
>>>>
>>>> Marica ANTONACCI
>>>> INFN - National Institute of Nuclear Physics
>>>> Via Orabona 4
>>>> 70126 Bari - ITALY
>>>> Phone +39 080 5443244
>>>> Skype: marica.antonacci
>>>> e-mail [email protected]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Deshi Xiao
>>> Twitter: xds2000
>>> E-mail: xiaods(AT)gmail.com
>>> <屏幕快照 2015-12-18 下午4.55.16.png><屏幕快照 2015-12-18 下午4.55.30.png><屏幕快照
>>> 2015-12-18 下午4.55.38.png>
>>>
>>>
>>> ----------------------------------------------------------
>>>
>>> Marica ANTONACCI
>>> INFN - National Institute of Nuclear Physics
>>> Via Orabona 4
>>> 70126 Bari - ITALY
>>> Phone +39 080 5443244
>>> Skype: marica.antonacci
>>> e-mail [email protected]
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Deshi Xiao
>> Twitter: xds2000
>> E-mail: xiaods(AT)gmail.com
>>
>>
>> ----------------------------------------------------------
>>
>> Marica ANTONACCI
>> INFN - National Institute of Nuclear Physics
>> Via Orabona 4
>> 70126 Bari - ITALY
>> Phone +39 080 5443244
>> Skype: marica.antonacci
>> e-mail [email protected]
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
