Giulio D'Ippolito created MESOS-4899:
----------------------------------------
Summary: Mesos slave crash after killing docker container
Key: MESOS-4899
URL: https://issues.apache.org/jira/browse/MESOS-4899
Project: Mesos
Issue Type: Bug
Affects Versions: 0.27.1
Reporter: Giulio D'Ippolito
I have experienced an issue where a Mesos slave crashed because a docker task
could not be killed properly.
I'm using marathon to launch the task.
The setup is the following:
OS verison: Centos 7.2
Docker version: 1.8.2
Mesos-slave: 0.27.1
Mesos-master: 0.27.1
Marathon 0.15.3
The mesos slave crashed (which is not great). This is the log from the mesos
slave (both mesos slave had the same issue):
{code}
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.441756
30870 slave.cpp:1890] Asked to kill task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8
-d7536e319ea2-0000
Mar 07 15:25:16 marathon_mesos-2 docker[13019]:
time="2016-03-07T15:25:16.553925302Z" level=info msg="POST
/v1.20/containers/mesos-d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145.104860a3-4630-4cbb-8e68-87ecca13fcad/s
top?t=0"
Mar 07 15:25:16 marathon_mesos-2 docker[13019]:
time="2016-03-07T15:25:16.559932661Z" level=info msg="Container
fe412634ec92bb641a18b4c48d399895f703af29492804b927943646bd81ab8a failed to exit
within 0 seconds of
SIGTERM - using the force"
Mar 07 15:25:16 marathon_mesos-2 systemd[1]: Stopped docker container
fe412634ec92bb641a18b4c48d399895f703af29492804b927943646bd81ab8a.
Mar 07 15:25:16 marathon_mesos-2 systemd[1]: Stopping docker container
fe412634ec92bb641a18b4c48d399895f703af29492804b927943646bd81ab8a.
Mar 07 15:25:16 marathon_mesos-2 kernel: docker0: port 4(vetha772033) entered
disabled state
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <warn> (vethd55899a):
failed to find device 106 'vethd55899a' with udev
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <info> (vethd55899a):
new Veth device (carrier: OFF, driver: 'veth', ifindex: 106)
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <info> (vetha772033):
link disconnected
Mar 07 15:25:16 marathon_mesos-2 kernel: docker0: port 4(vetha772033) entered
disabled state
Mar 07 15:25:16 marathon_mesos-2 avahi-daemon[11714]: Withdrawing address
record for fe80::b06a:2fff:fecc:87f1 on vetha772033.
Mar 07 15:25:16 marathon_mesos-2 kernel: device vetha772033 left promiscuous
mode
Mar 07 15:25:16 marathon_mesos-2 kernel: docker0: port 4(vetha772033) entered
disabled state
Mar 07 15:25:16 marathon_mesos-2 avahi-daemon[11714]: Withdrawing workstation
service for vethd55899a.
Mar 07 15:25:16 marathon_mesos-2 avahi-daemon[11714]: Withdrawing workstation
service for vetha772033.
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <warn> (vethd55899a):
failed to disable userspace IPv6LL address handling
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <info> (docker0): bridge
port vetha772033 was detached
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <info> (vetha772033):
released from master docker0
Mar 07 15:25:16 marathon_mesos-2 NetworkManager[788]: <warn> (vetha772033):
failed to disable userspace IPv6LL address handling
Mar 07 15:25:16 marathon_mesos-2 kernel: XFS (dm-8): Unmounting Filesystem
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.879240
30868 slave.cpp:3001] Handling status update TASK_KILLED (UUID:
1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task giulio.httpd.test.tag.a47
c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 from executor(1)@XXX.XXX.XXX.XXX:57543
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: E0307 15:25:16.880144
30868 slave.cpp:3205] Failed to update resources for container
104860a3-4630-4cbb-8e68-87ecca13fcad of executor 'giulio.httpd.test.tag.a
47c55be-db01-11e5-a92a-0242eb705eb2' running task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 on status update for
terminal task, destroying container: Failed to determine cgroup for the 'cpu'
sub
system: Failed to read /proc/4390/cgroup: Failed to open file
'/proc/4390/cgroup': No such file or directory
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.880388
30868 status_update_manager.cpp:320] Received status update TASK_KILLED (UUID:
1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task giulio.htt
pd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.880416
30870 docker.cpp:1455] Destroying container
'104860a3-4630-4cbb-8e68-87ecca13fcad'
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.880487
30870 docker.cpp:1515] Sending SIGTERM to executor with pid: 4318
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.880437
30868 status_update_manager.cpp:824] Checkpointing UPDATE for status update
TASK_KILLED (UUID: 1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.899090
30868 slave.cpp:3353] Forwarding the update TASK_KILLED (UUID:
1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 to [email protected]:5051
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.899364
30868 slave.cpp:3263] Sending acknowledgement for status update TASK_KILLED
(UUID: 1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 to executor(1)@XXX.XXX.XXX.XXX:57543
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.923931
30867 status_update_manager.cpp:392] Received status update acknowledgement
(UUID: 1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.924121
30867 status_update_manager.cpp:824] Checkpointing ACK for status update
TASK_KILLED (UUID: 1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.950709
30870 docker.cpp:1557] Running docker stop on container
'104860a3-4630-4cbb-8e68-87ecca13fcad'
Mar 07 15:25:16 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:16.955036
30870 slave.cpp:3481] executor(1)@XXX.XXX.XXX.XXX:57543 exited
Mar 07 15:25:17 marathon_mesos-2 docker[13019]:
time="2016-03-07T15:25:17.003800525Z" level=info msg="POST
/v1.20/containers/mesos-d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145.104860a3-4630-4cbb-8e68-87ecca13fcad/stop?t=0"
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:17.042575
30872 docker.cpp:1654] Executor for container
'104860a3-4630-4cbb-8e68-87ecca13fcad' has exited
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:17.043073
30868 slave.cpp:3816] Executor
'giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2' of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 terminated with signal Terminated
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: I0307 15:25:17.043175
30868 slave.cpp:3920] Cleaning up executor
'giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2' of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 at executor(1)@XXX.XXX.XXX.XXX:57543
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: F0307 15:25:17.043442
30868 slave.cpp:3945] CHECK_SOME(os::touch(path)): Failed to open file: No such
file or directory
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: *** Check failure stack
trace: ***
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f73133d
google::LogMessage::Fail()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f73307c
google::LogMessage::SendToLog()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f730f2c
google::LogMessage::Flush()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f733979
google::LogMessageFatal::~LogMessageFatal()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f0faf1c
mesos::internal::slave::Slave::removeExecutor()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f112970
mesos::internal::slave::Slave::executorTerminated()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f6dee21
process::ProcessManager::resume()
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5f6df127
_ZNSt6thread5_ImplISt12_Bind_simpleIFSt5_BindIFZN7process14ProcessManager12init_threadsEvEUlRKSt11atomic_boolE_St17reference_wrapperIS6_EEEvEEE6_M_runEv
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5e023220
(unknown)
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5e27ddc5
start_thread
Mar 07 15:25:17 marathon_mesos-2 mesos-slave[30866]: @ 0x7f4a5da8d28d
__clone
Mar 07 15:25:17 marathon_mesos-2 systemd[1]: mesos-slave.service: main process
exited, code=killed, status=6/ABRT
Mar 07 15:25:17 marathon_mesos-2 systemd[1]: Unit mesos-slave.service entered
failed state.
Mar 07 15:25:17 marathon_mesos-2 systemd[1]: mesos-slave.service failed.
Mar 07 15:25:17 marathon_mesos-2 ntpd[728]: Deleting interface #42 vetha772033,
fe80::b06a:2fff:fecc:87f1#123, interface stats: received=0, sent=0, dropped=0,
active_time=1040732 secs
Mar 07 15:25:37 marathon_mesos-2 systemd[1]: mesos-slave.service holdoff time
over, scheduling restart.
Mar 07 15:25:37 marathon_mesos-2 systemd[1]: Started Mesos Slave.
Mar 07 15:25:37 marathon_mesos-2 systemd[1]: Starting Mesos Slave...
{code}
Mesos master log
{code}
I0307 15:25:16.443166 8 master.cpp:3855] Telling slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com) to kill task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663
I0307 15:25:16.443385 8 master.cpp:3855] Telling slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com) to kill task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663
I0307 15:25:16.443476 8 master.cpp:3855] Telling slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com) to kill task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663
I0307 15:25:16.903249 8 master.cpp:4789] Status update TASK_KILLED (UUID:
1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 from slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com)
I0307 15:25:16.903378 8 master.cpp:4837] Forwarding status update
TASK_KILLED (UUID: 1b9446db-dcdc-47a7-aa05-0372e84e1b4d) for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
I0307 15:25:16.903561 8 master.cpp:6445] Updating the state of task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (latest state: TASK_KILLED, status
update state: TASK_KILLED)
I0307 15:25:16.924135 8 master.cpp:3947] Processing ACKNOWLEDGE call
1b9446db-dcdc-47a7-aa05-0372e84e1b4d for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145
I0307 15:25:16.924257 8 master.cpp:6511] Removing task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 with resources
cpus(*):0.04; mem(*):32; ports(*):[31511-31511] of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com)
I0307 15:25:17.059509 8 master.cpp:4789] Status update TASK_KILLED (UUID:
6ab5a70a-9690-43df-8d05-f4ea08803ac2) for task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 from slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.059617 8 master.cpp:4837] Forwarding status update
TASK_KILLED (UUID: 6ab5a70a-9690-43df-8d05-f4ea08803ac2) for task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
I0307 15:25:17.059797 8 master.cpp:6445] Updating the state of task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (latest state: TASK_KILLED, status
update state: TASK_KILLED)
I0307 15:25:17.073132 8 master.cpp:3947] Processing ACKNOWLEDGE call
6ab5a70a-9690-43df-8d05-f4ea08803ac2 for task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146
I0307 15:25:17.073266 8 master.cpp:6511] Removing task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 with resources
cpus(*):0.04; mem(*):32; ports(*):[31833-31833] of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.088891 8 master.cpp:1172] Slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com) disconnected
I0307 15:25:17.088999 8 master.cpp:2633] Disconnecting slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com)
I0307 15:25:17.089064 8 master.cpp:2652] Deactivating slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-2.domain.com)
I0307 15:25:17.089191 8 hierarchical.cpp:560] Slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S145 deactivated
I0307 15:25:17.121345 8 master.cpp:4789] Status update TASK_KILLED (UUID:
582f00d2-037c-43a1-88c1-2f610717a5d5) for task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 from slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.121451 8 master.cpp:4837] Forwarding status update
TASK_KILLED (UUID: 582f00d2-037c-43a1-88c1-2f610717a5d5) for task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000
I0307 15:25:17.121590 8 master.cpp:6445] Updating the state of task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (latest state: TASK_KILLED, status
update state: TASK_KILLED)
I0307 15:25:17.145149 8 master.cpp:3947] Processing ACKNOWLEDGE call
582f00d2-037c-43a1-88c1-2f610717a5d5 for task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 (marathon) at
[email protected]:55663 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146
I0307 15:25:17.145256 8 master.cpp:6511] Removing task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 with resources
cpus(*):0.04; mem(*):32; ports(*):[31167-31167] of framework
ef1354df-7ecc-41ac-82d8-d7536e319ea2-0000 on slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.287336 8 master.cpp:1172] Slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com) disconnected
I0307 15:25:17.287485 8 master.cpp:2633] Disconnecting slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.287531 8 master.cpp:2652] Deactivating slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 at slave(1)@XXX.XXX.XXX.XXX:5051
(marathon_mesos-1.domain.com)
I0307 15:25:17.287618 8 hierarchical.cpp:560] Slave
d0f20f55-bc6e-43e8-babb-250b0176f5f6-S146 deactivated
{code}
Marathon framework log:
{code}
[2016-03-07 15:25:16,366] INFO Upgrade group id:/
version:2016-03-07T15:25:16.366Z with force:true
(mesosphere.marathon.state.GroupManager:marathon-akka.actor.default-dispatcher-46207)
[2016-03-07 15:25:16,368] INFO Computed new deployment plan:
[2016-03-07 15:25:16,368] INFO Deploy plan with force=true:
[2016-03-07 15:25:16,374] INFO [/giulio.httpd.test.tag] expunging app
(mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-5)
[2016-03-07 15:25:16,458] INFO Updated groups/apps according to deployment plan
bc0e6fa3-f1cc-4658-9787-efe85e957869
(mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-5)
[2016-03-07 15:25:16,459] INFO Deployment acknowledged. Waiting to get
processed:
[2016-03-07 15:25:16,690] INFO XXX.XXX.XXX.XXX - - [07/Mar/2016:15:25:16 +0000]
"DELETE //XXX.XXX.XXX.XXX:8080/v2/apps//giulio.httpd.test.tag HTTP/1.1" 200 92
"http://XXX.XXX.XXX.XXX:8080/ui/"; "Mozilla/5.0 (X11; Fedora; Linux x86_64;
rv:44.0) Gecko/20100101 Firefox/44.0" 32
(mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$562bf4ff:qtp1361931872-249367)
[2016-03-07 15:25:16,863] INFO XXX.XXX.XXX.XXX - - [07/Mar/2016:15:25:16 +0000]
"GET //XXX.XXX.XXX.XXX:8080/ui/img/marathon-favicon.ico HTTP/1.1" 200 32038 "-"
"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0"
(mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$562bf4ff:qtp1361931872-143659)
[2016-03-07 15:25:16,904] INFO Received status update for task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2: TASK_KILLED ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$4304ca77:Thread-63362)
[2016-03-07 15:25:16,910] INFO XXX.XXX.XXX.XXX - - [07/Mar/2016:15:25:16 +0000]
"GET //XXX.XXX.XXX.XXX:8080/v2/deployments HTTP/1.1" 200 308
"http://XXX.XXX.XXX.XXX:8080/ui/"; "Mozilla/5.0 (X11; Fedora; Linux x86_64;
rv:44.0) Gecko/20100101 Firefox/44.0"
(mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$562bf4ff:qtp1361931872-123617)
[2016-03-07 15:25:16,912] INFO XXX.XXX.XXX.XXX - - [07/Mar/2016:15:25:16 +0000]
"GET //XXX.XXX.XXX.XXX:8080/v2/apps HTTP/1.1" 200 4360
"http://XXX.XXX.XXX.XXX:8080/ui/"; "Mozilla/5.0 (X11; Fedora; Linux x86_64;
rv:44.0) Gecko/20100101 Firefox/44.0"
(mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$562bf4ff:qtp1361931872-116139)
[2016-03-07 15:25:16,912] INFO XXX.XXX.XXX.XXX - - [07/Mar/2016:15:25:16 +0000]
"GET //XXX.XXX.XXX.XXX:8080/v2/queue HTTP/1.1" 200 12
"http://XXX.XXX.XXX.XXX:8080/ui/"; "Mozilla/5.0 (X11; Fedora; Linux x86_64;
rv:44.0) Gecko/20100101 Firefox/44.0"
(mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$562bf4ff:qtp1361931872-143659)
[2016-03-07 15:25:16,921] INFO Sending event notification for task
[giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2] of app
[/giulio.httpd.test.tag]: TASK_KILLED
(mesosphere.marathon.core.task.update.impl.steps.PostToEventStreamStepImpl$$EnhancerByGuice$$8fcb1905:ForkJoinPool-2-worker-25)
[2016-03-07 15:25:16,921] INFO initiating a scale check for app
[/giulio.httpd.test.tag] after task
[giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2}] terminated
(mesosphere.marathon.core.task.update.impl.steps.ScaleAppUpdateStepImpl$$EnhancerByGuice$$fc6d4908:ForkJoinPool-2-worker-25)
[2016-03-07 15:25:16,922] INFO Task
giulio.httpd.test.tag.a47c55be-db01-11e5-a92a-0242eb705eb2 has been killed.
Waiting for 2 more tasks to be killed.
(mesosphere.marathon.upgrade.AppStopActor:marathon-akka.actor.default-dispatcher-46201)
[2016-03-07 15:25:17,060] INFO Received status update for task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2: TASK_KILLED ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$4304ca77:Thread-63363)
[2016-03-07 15:25:17,070] INFO Sending event notification for task
[giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2] of app
[/giulio.httpd.test.tag]: TASK_KILLED
(mesosphere.marathon.core.task.update.impl.steps.PostToEventStreamStepImpl$$EnhancerByGuice$$8fcb1905:ForkJoinPool-2-worker-5)
[2016-03-07 15:25:17,071] INFO initiating a scale check for app
[/giulio.httpd.test.tag] after task
[giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2}] terminated
(mesosphere.marathon.core.task.update.impl.steps.ScaleAppUpdateStepImpl$$EnhancerByGuice$$fc6d4908:ForkJoinPool-2-worker-5)
[2016-03-07 15:25:17,071] INFO Task
giulio.httpd.test.tag.a47d401f-db01-11e5-a92a-0242eb705eb2 has been killed.
Waiting for 1 more tasks to be killed.
(mesosphere.marathon.upgrade.AppStopActor:marathon-akka.actor.default-dispatcher-46207)
[2016-03-07 15:25:17,122] INFO Received status update for task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2: TASK_KILLED ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$4304ca77:Thread-63364)
[2016-03-07 15:25:17,132] INFO Removed app [/giulio.httpd.test.tag] from
tracker
(mesosphere.marathon.core.task.tracker.TaskTracker$TasksByApp$:marathon-akka.actor.default-dispatcher-46161)
[2016-03-07 15:25:17,133] INFO Sending event notification for task
[giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2] of app
[/giulio.httpd.test.tag]: TASK_KILLED
(mesosphere.marathon.core.task.update.impl.steps.PostToEventStreamStepImpl$$EnhancerByGuice$$8fcb1905:ForkJoinPool-2-worker-11)
[2016-03-07 15:25:17,133] INFO initiating a scale check for app
[/giulio.httpd.test.tag] after task
[giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2}] terminated
(mesosphere.marathon.core.task.update.impl.steps.ScaleAppUpdateStepImpl$$EnhancerByGuice$$fc6d4908:ForkJoinPool-2-worker-11)
[2016-03-07 15:25:17,134] INFO Task
giulio.httpd.test.tag.a47c079d-db01-11e5-a92a-0242eb705eb2 has been killed.
Waiting for 0 more tasks to be killed.
(mesosphere.marathon.upgrade.AppStopActor:marathon-akka.actor.default-dispatcher-46195)
[2016-03-07 15:25:17,134] INFO Successfully killed all the tasks
(mesosphere.marathon.upgrade.AppStopActor:marathon-akka.actor.default-dispatcher-46195)
[2016-03-07 15:25:17,184] INFO Stopping app /giulio.httpd.test.tag
(mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-46195)
[2016-03-07 15:25:17,188] INFO Deployment of / successful
(mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-46161)
[2016-03-07 15:25:17,188] INFO Removing bc0e6fa3-f1cc-4658-9787-efe85e957869
from list of running deployments
(mesosphere.marathon.upgrade.DeploymentManager:marathon-akka.actor.default-dispat
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
