[
https://issues.apache.org/jira/browse/MESOS-5961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407800#comment-15407800
]
Alexander Rukletsov edited comment on MESOS-5961 at 8/4/16 2:38 PM:
--------------------------------------------------------------------
h1. Summary / Suggested Solution
To keep the implementation simple and consistent, I propose that the Docker
executor's health check thread should always enter the task's net namespace and
connect to {{localhost}} on the port specified in the health check proto.
The framework would then be responsible for deciding if the health checker
should connect to the {{containerPort}} or the {{hostPort}}.
h1. Testing results
h2. HOST networking
# Connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<hostPort>}}.
NOTE: In this case, both the docker executor and the container are in the same
network, hence `setns` call will be a noop. We still suggest not to treat this
case differently for consistency and readability.
h2. BRIDGED networking
# If the task specifies port mappings, connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<containerPort>}}.
h2. USER networking (DC/OS Overlay network)
# If the task specifies a {{hostPort}}, connect to {{localhost:<hostPort>}}.
# If the task doesn't use a host port, enter the container's net ns and connect
to {{localhost:<containerPort>}}. I expect this to be the most common case.
h1. Testing notes
h2. Getting the networking namespace
{code}
NETNS="/proc/`docker inspect -f '{{.State.Pid}}' <container id>`/ns/net"
{code}
h2. Test Utility
The following utility enters a network namespace and does an HTTP request.
{code}
package main
import (
"fmt"
"io"
"net/http"
"os"
"syscall"
"github.com/opencontainers/runc/libcontainer/system"
)
func doNetns(nsPath string) error {
ns, err := os.Open(nsPath)
if err != nil {
return err
}
return system.Setns(ns.Fd(), syscall.CLONE_NEWNET)
}
func doGet(port string) {
response, err := http.Get("http://localhost:"; + port)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
} else {
defer response.Body.Close()
_, err := io.Copy(os.Stderr, response.Body)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
}
}
}
func main() {
doNetns(os.Args[1])
doGet(os.Args[2])
}
{code}
h2. Marathon App Definitions
h3. HOST networking
{code}
{
"id": "/host",
"cmd": "python -m SimpleHTTPServer $PORT0",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "HOST"
}
},
"portDefinitions": [
{
"name": "http",
"protocol": "tcp"
}
]
}
{code}
h3. BRIDGE networking
{code}
{
"id": "/bridged",
"cmd": "python -m SimpleHTTPServer 31337",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 31337,
"hostPort": 0,
"name": "http"
}
]
}
}
}
{code}
h3. USER networking (DC/OS Overlay network)
{code}
{
"id": "/overlay",
"cpus": 0.1,
"mem": 32,
"cmd": "python -m SimpleHTTPServer 31337",
"container": {
"type": "DOCKER",
"docker": {
"network": "USER",
"image": "python:2",
"portMappings": [
{
"containerPort": 31337,
"protocol": "tcp",
"name": "http"
}
]
}
},
"ipAddress": {
"networkName": "dcos"
}
}
{code}
was (Author: gkleiman):
h1. Summary / Suggested Solution
To keep the implementation simple and consistent, I propose that the Docker
executor's health check thread should always enter the task's net namespace and
connect to {{localhost}} on the port specified in the health check proto. This
solution covers all our current use cases and hence allows for consistency and
smaller code.
The framework would then be responsible for deciding if the health checker
should connect to the {{containerPort}} or the {{hostPort}}.
h1. Testing results
h2. HOST networking
# Connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<hostPort>}}.
NOTE: In this case, both the docker executor and the container are in the same
network, hence `setns` call will be a noop. We still suggest not to treat this
case differently for consistency and readability.
h2. BRIDGED networking
# If the task specifies port mappings, connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<containerPort>}}.
h2. USER networking (DC/OS Overlay network)
# If the task specifies a {{hostPort}}, connect to {{localhost:<hostPort>}}.
# If the task doesn't use a host port, enter the container's net ns and connect
to {{localhost:<containerPort>}}. I expect this to be the most common case.
h1. Testing notes
h2. Getting the networking namespace
{code}
NETNS="/proc/`docker inspect -f '{{.State.Pid}}' <container id>`/ns/net"
{code}
h2. Test Utility
The following utility enters a network namespace and does an HTTP request.
{code}
package main
import (
"fmt"
"io"
"net/http"
"os"
"syscall"
"github.com/opencontainers/runc/libcontainer/system"
)
func doNetns(nsPath string) error {
ns, err := os.Open(nsPath)
if err != nil {
return err
}
return system.Setns(ns.Fd(), syscall.CLONE_NEWNET)
}
func doGet(port string) {
response, err := http.Get("http://localhost:"; + port)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
} else {
defer response.Body.Close()
_, err := io.Copy(os.Stderr, response.Body)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
}
}
}
func main() {
doNetns(os.Args[1])
doGet(os.Args[2])
}
{code}
h2. Marathon App Definitions
h3. HOST networking
{code}
{
"id": "/host",
"cmd": "python -m SimpleHTTPServer $PORT0",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "HOST"
}
},
"portDefinitions": [
{
"name": "http",
"protocol": "tcp"
}
]
}
{code}
h3. BRIDGE networking
{code}
{
"id": "/bridged",
"cmd": "python -m SimpleHTTPServer 31337",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 31337,
"hostPort": 0,
"name": "http"
}
]
}
}
}
{code}
h3. USER networking (DC/OS Overlay network)
{code}
{
"id": "/overlay",
"cpus": 0.1,
"mem": 32,
"cmd": "python -m SimpleHTTPServer 31337",
"container": {
"type": "DOCKER",
"docker": {
"network": "USER",
"image": "python:2",
"portMappings": [
{
"containerPort": 31337,
"protocol": "tcp",
"name": "http"
}
]
}
},
"ipAddress": {
"networkName": "dcos"
}
}
{code}
> HTTP and TCP health checks should support docker executor and bridged mode.
> ---------------------------------------------------------------------------
>
> Key: MESOS-5961
> URL: https://issues.apache.org/jira/browse/MESOS-5961
> Project: Mesos
> Issue Type: Improvement
> Reporter: Alexander Rukletsov
> Assignee: haosdent
> Labels: health-check, mesosphere
>
> If an executor and a task, e.g. the docker executor and docker container in
> bridged mode, exist in different network namespaces, HTTP and TCP health
> checks using {{localhost}} may not work properly. One solution would be to
> enter the container's network in the health check binary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
