[
https://issues.apache.org/jira/browse/MESOS-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262187#comment-15262187
]
Travis Hegner commented on MESOS-5294:
--------------------------------------
While marathon shows the task in an "unknown" state, during it's grace period,
the {{statuses}} array looks like this:
{code}
"statuses": [
{
"state": "TASK_RUNNING",
"timestamp": 1461850431.76103,
"labels": [
{
"key": "Docker.NetworkSettings.IPAddress",
"value": "10.1.9.16"
}
],
"container_status": {
"network_infos": [
{
"ip_address": "10.1.9.16",
"ip_addresses": [
{
"ip_address": "10.1.9.16"
}
]
}
]
}
}
],
{code}
Mind you, I'm running some custom networking, and I do _want_ docker's IP
address to come through, not the hosts IP. Therefore my {{config.json}} for
{{mesos-dns}} contains {{"IPSources":\["docker"\]}}
After, the task status is marked healthy, my status array looks like this:
{code}
"statuses": [
{
"state": "TASK_RUNNING",
"timestamp": 1461850492.06737,
"container_status": {
"network_infos": [
{
"ip_address": "10.1.8.54",
"ip_addresses": [
{
"ip_address": "10.1.8.54"
}
]
}
]
},
"healthy": true
}
],
{code}
Notice the IP change in {{network_infos}}. This new IP is of the host, and not
of the container itself. It's as if the status update that gets generated by
the health check process is doing it's own IP address discovery, different from
the ip discovery code in MESOS-4370.
> Status updates after a health check are incomplete or invalid
> -------------------------------------------------------------
>
> Key: MESOS-5294
> URL: https://issues.apache.org/jira/browse/MESOS-5294
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 0.28.0, docker 1.11, marathon 0.15.3, mesos-dns,
> ubuntu 14.04
> Reporter: Travis Hegner
> Assignee: Travis Hegner
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> With command health checks enabled via marathon, mesos-dns will resolve the
> task correctly until the task is reported as "healthy". At that point,
> mesos-dns stops resolving the task correctly.
> -Digging through src/docker/executor.cpp, I found that in the
> {{taskHealthUpdated()}} function is attempting to copy the taskID to the new
> status instance with-
> {code}status.mutable_task_id()->CopyFrom(taskID);{code}
> -but other instances of status updates have a similar line-
> {code}status.mutable_task_id()->CopyFrom(taskID.get());{code}
> -My assumption is that this difference is causing the status update after a
> health check to not have a proper taskID, which in turn is causing an
> incorrect state.json output.-
> -I'll try to get a patch together soon.-
> UPDATE:
> None of the above assumption are correct. Something else is causing the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
