I am trying to integrate the Docker containerizer into the Singularity
framework (https://github.com/HubSpot/Singularity)
I have filled in my containerInfo:
"containerInfo": {
"type": "DOCKER",
"docker": {
"image": "registry.mesos-vpcqa.otenv.com/demo-server:latest-master"
}
}
I am leaving ‘command’ blank as I wish to have the default command executed.
Interestingly enough, the Docker containerizer seems to pass on the TaskInfo:
I0904 23:12:13.941849 32497 exec.cpp:132] Version: 0.20.0
I0904 23:12:13.947185 32500 exec.cpp:206] Executor registered on slave
20140826-180432-1594050058-5050-23467-6
F0904 23:12:13.949040 32502 executor.cpp:134] Check failed:
task.command().has_value() Shell command of task
DemoService-DemoService.18-1409872328807-2-10-us_west_2a is not specified!
*** Check failure stack trace: ***
@ 0x7f54dc76b4bd google::LogMessage::Fail()
@ 0x7f54dc76d2fd google::LogMessage::SendToLog()
@ 0x7f54dc76b0ac google::LogMessage::Flush()
@ 0x7f54dc76dbf9 google::LogMessageFatal::~LogMessageFatal()
@ 0x42abc2 mesos::internal::CommandExecutorProcess::launchTask()
@ 0x7f54dc7160e2 process::ProcessManager::resume()
@ 0x7f54dc7163ec process::schedule()
@ 0x7f54db190182 start_thread
@ 0x7f54daebd38d (unknown)
Aborted
but if I add some (otherwise unnecessary) URIs to fetch it then is happy to
accept it
"uris": [ "https://my.s3.amazonaws.com/something.tar.gz”]
then I start to see Docker containerizer output in the slave log:
I0904 22:58:24.092098 1288 docker.cpp:815] Starting container
'479dedef-4312-4441-be72-6570caaf2b8a' for task
'DemoService-DemoService.15-1409871503777-2-10-us_west_2a' (and executor
'DemoService-DemoService.15-1409871503777-2-10-us_west_2a') of framework
'Singularity'
I0904 22:58:28.327430 1288 slave.cpp:2535] Monitoring executor
'DemoService-DemoService.15-1409871503777-2-10-us_west_2a' of framework
'Singularity' in container '479dedef-4312-4441-be72-6570caaf2b8a'
I0904 22:58:28.368155 1291 status_update_manager.cpp:320] Received status
update TASK_RUNNING (UUID: a428c917-973d-4c7e-a8a3-43fdf9b49e99) for task
DemoService-DemoService.15-1409871503777-2-10-us_west_2a of framework
Singularity
I0904 22:58:29.331042 1291 slave.cpp:2087] Handling status update
TASK_FINISHED (UUID: de9b8003-9b61-4b87-98fc-32b01f875a95) for task
DemoService-DemoService.15-1409871503777-2-10-us_west_2a of framework
Singularity from executor(1)@10.70.6.186:56465
I0904 22:58:30.689954 1291 docker.cpp:1720] Executor for container
'479dedef-4312-4441-be72-6570caaf2b8a' has exited
I0904 22:58:30.690076 1291 docker.cpp:1545] Destroying container
'479dedef-4312-4441-be72-6570caaf2b8a'
I0904 22:58:30.690500 1291 docker.cpp:1642] Running docker kill on container
'479dedef-4312-4441-be72-6570caaf2b8a'
stderr:
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0904 22:58:24.317865 26387 fetcher.cpp:126] Downloading ‘xxx' to
'/mnt/mesos/slaves/20140826-180432-1594050058-5050-23467-8/frameworks/Singularity/executors/DemoService-DemoService.15-1409871503777-2-10-us_west_2a/runs/479dedef-4312-4441-be72-6570caaf2b8a/xxx'
I0904 22:58:24.387418 26387 fetcher.cpp:64] Extracted resource
'/mnt/mesos/slaves/20140826-180432-1594050058-5050-23467-8/frameworks/Singularity/executors/DemoService-DemoService.15-1409871503777-2-10-us_west_2a/runs/479dedef-4312-4441-be72-6570caaf2b8a/xxx'
into
'/mnt/mesos/slaves/20140826-180432-1594050058-5050-23467-8/frameworks/Singularity/executors/DemoService-DemoService.15-1409871503777-2-10-us_west_2a/runs/479dedef-4312-4441-be72-6570caaf2b8a'
I0904 22:58:27.870132 26811 exec.cpp:132] Version: 0.20.0
I0904 22:58:28.155580 26819 exec.cpp:206] Executor registered on slave
20140826-180432-1594050058-5050-23467-8
stdout:
Registered executor on 10.70.6.186
Starting task DemoService-DemoService.15-1409871503777-2-10-us_west_2a
Forked command at 26949
/bin/sh -c exit `docker wait mesos-479dedef-4312-4441-be72-6570caaf2b8a`
Command exited with status 0 (pid: 26949)
When I try to lookup the Docker container with ‘docker ps -a’ it looks like no
container was ever created, and then the ‘docker wait’ exits immediately.
Any clues as to how I could debug such a situation?
Thanks!
Steven
