[
https://issues.apache.org/jira/browse/MESOS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638842#comment-14638842
]
Adam B edited comment on MESOS-3136 at 7/23/15 5:21 PM:
--------------------------------------------------------
And Marathon logs of the task on 0.22.1:
{code}
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,786] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:08 srv004 marathon[1375]: [WARN] [07/23/2015 10:08:08.820]
[pool-2-thread-76] [akka://marathon/user/$b] Failed to post
MesosStatusUpdateEvent(20150723-081517-100663306-5050-2386-S1,company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,TASK_RUNNING,,/company-master/databases/redis,srv004,Buffer(31578),2015-07-23T10:08:04.839Z,status_update_event,2015-07-23T10:08:08.797Z)
to http://10.0.0.4:8000/api/marathon/event_callback WARNING arguments left: 1
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,944] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,947] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,947] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:08.947Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:08 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:08.949]
[marathon-akka.actor.default-dispatcher-13] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:08.947Z)]
Jul 23 10:08:08 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:08.949]
[marathon-akka.actor.default-dispatcher-27]
[akka://marathon/user/MarathonScheduler/$a/DeploymentManager/ca1e1c64-7453-4f47-af42-c066c88bbc0a/$a]
Killing old task
company-master_databases_redis.762a438f-311a-11e5-917e-cee15558ba83 because
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83 became
reachable
Jul 23 10:08:08 srv004 marathon[1375]: [WARN] [07/23/2015 10:08:08.970]
[pool-2-thread-73] [akka://marathon/user/$b] Failed to post
HealthStatusChanged(/company-master/databases/redis,company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,true,health_status_changed_event,2015-07-23T10:08:08.949Z)
to http://10.0.0.4:8000/api/marathon/event_callback WARNING arguments left: 1
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,529] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,530] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,531] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.530Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:12 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:12.531]
[marathon-akka.actor.default-dispatcher-21] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.530Z)]
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,585] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,587] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,588] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.588Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:12 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:12.588]
[marathon-akka.actor.default-dispatcher-18] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.588Z)]
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,500] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,501] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,502] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.501Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:42 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:42.502]
[marathon-akka.actor.default-dispatcher-18] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.501Z)]
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,537] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,539] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,539] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.539Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
{code}
was (Author: sttts):
And Marathon logs of the task on 0.22.1:
```
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,786] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:08 srv004 marathon[1375]: [WARN] [07/23/2015 10:08:08.820]
[pool-2-thread-76] [akka://marathon/user/$b] Failed to post
MesosStatusUpdateEvent(20150723-081517-100663306-5050-2386-S1,company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,TASK_RUNNING,,/company-master/databases/redis,srv004,Buffer(31578),2015-07-23T10:08:04.839Z,status_update_event,2015-07-23T10:08:08.797Z)
to http://10.0.0.4:8000/api/marathon/event_callback WARNING arguments left: 1
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,944] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING ()
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,947] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:08 srv004 marathon[1375]: [2015-07-23 10:08:08,947] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:08.947Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:08 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:08.949]
[marathon-akka.actor.default-dispatcher-13] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:08.947Z)]
Jul 23 10:08:08 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:08.949]
[marathon-akka.actor.default-dispatcher-27]
[akka://marathon/user/MarathonScheduler/$a/DeploymentManager/ca1e1c64-7453-4f47-af42-c066c88bbc0a/$a]
Killing old task
company-master_databases_redis.762a438f-311a-11e5-917e-cee15558ba83 because
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83 became
reachable
Jul 23 10:08:08 srv004 marathon[1375]: [WARN] [07/23/2015 10:08:08.970]
[pool-2-thread-73] [akka://marathon/user/$b] Failed to post
HealthStatusChanged(/company-master/databases/redis,company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,true,health_status_changed_event,2015-07-23T10:08:08.949Z)
to http://10.0.0.4:8000/api/marathon/event_callback WARNING arguments left: 1
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,529] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,530] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,531] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.530Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:12 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:12.531]
[marathon-akka.actor.default-dispatcher-21] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.530Z)]
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,585] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,587] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:12 srv004 marathon[1375]: [2015-07-23 10:08:12,588] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.588Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:12 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:12.588]
[marathon-akka.actor.default-dispatcher-18] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:12.588Z)]
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,500] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,501] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,502] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.501Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
Jul 23 10:08:42 srv004 marathon[1375]: [INFO] [07/23/2015 10:08:42.502]
[marathon-akka.actor.default-dispatcher-18] [akka://marathon/user/$Rb] Received
health result:
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.501Z)]
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,537] INFO Received
status update for task
company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83:
TASK_RUNNING (Reconciliation: Latest task state)
(mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$36c5b88a:99)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,539] INFO Received
status for
[company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83] with
version [2015-07-23T10:08:04.839Z] and healthy [true]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:165)
Jul 23 10:08:42 srv004 marathon[1375]: [2015-07-23 10:08:42,539] INFO
Forwarding health result
[Healthy(company-master_databases_redis.b787889b-3122-11e5-917e-cee15558ba83,2015-07-23T10:08:04.839Z,2015-07-23T10:08:42.539Z)]
to health check actor [Actor[akka://marathon/user/$Rb#521772329]]
(mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$2f9a72a2:186)
```
> COMMAND health checks with Marathon 0.10.0rc3 are broken
> --------------------------------------------------------
>
> Key: MESOS-3136
> URL: https://issues.apache.org/jira/browse/MESOS-3136
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Dr. Stefan Schimanski
>
> When deploying Mesos 0.23rc4 with latest Marathon 0.10.0 RC3 command health
> check stop working. Rolling back to Mesos 0.22.1 fixes the problem.
> Containerizer is Docker.
> All packages are from official Mesosphere Ubuntu 14.04 sources.
> The issue must be analyzed further.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
