[
https://issues.apache.org/jira/browse/MESOS-5961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407800#comment-15407800
]
Alexander Rukletsov edited comment on MESOS-5961 at 8/4/16 2:34 PM:
--------------------------------------------------------------------
h1. Summary / Suggested Solution
To keep the implementation simple and consistent, I propose that the Docker
executor's health check thread should always enter the task's net namespace and
connect to {{localhost}} on the port specified in the health check proto.
The framework would then be responsible for deciding if the health checker
should connect to the {{containerPort}} or the {{hostPort}}.
h1. Testing results
h2. HOST networking
# Connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<hostPort>}}.
NOTE: In this case, both the docker executor and the container are in the same
network, hence `setns` call will be a noop. We still suggest not to treat this
case differently for consistency and readability.
h2. BRIDGED networking
# If the task specified port mappings, connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<containerPort>}}.
h2. USER networking (DC/OS Overlay network)
# If the task specifies a {{hostPort}}, connect to {{localhost:<hostPort>}}.
# If the task doesn't use a host port, enter the container's net ns and connect
to {{localhost:<containerPort>}}. I expect this to be the most common case.
h1. Testing notes
h2. Getting the networking namespace
{code}
NETNS="/proc/`docker inspect -f '{{.State.Pid}}' <container id>`/ns/net"
{code}
h2. Test Utility
The following utility enters a network namespace and does an HTTP request.
{code}
package main
import (
"fmt"
"io"
"net/http"
"os"
"syscall"
"github.com/opencontainers/runc/libcontainer/system"
)
func doNetns(nsPath string) error {
ns, err := os.Open(nsPath)
if err != nil {
return err
}
return system.Setns(ns.Fd(), syscall.CLONE_NEWNET)
}
func doGet(port string) {
response, err := http.Get("http://localhost:"; + port)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
} else {
defer response.Body.Close()
_, err := io.Copy(os.Stderr, response.Body)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
}
}
}
func main() {
doNetns(os.Args[1])
doGet(os.Args[2])
}
{code}
h2. Marathon App Definitions
h3. HOST networking
{code}
{
"id": "/host",
"cmd": "python -m SimpleHTTPServer $PORT0",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "HOST"
}
},
"portDefinitions": [
{
"name": "http",
"protocol": "tcp"
}
]
}
{code}
h3. BRIDGE networking
{code}
{
"id": "/bridged",
"cmd": "python -m SimpleHTTPServer 31337",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 31337,
"hostPort": 0,
"name": "http"
}
]
}
}
}
{code}
h3. USER networking (DC/OS Overlay network)
{code}
{
"id": "/overlay",
"cpus": 0.1,
"mem": 32,
"cmd": "python -m SimpleHTTPServer 31337",
"container": {
"type": "DOCKER",
"docker": {
"network": "USER",
"image": "python:2",
"portMappings": [
{
"containerPort": 31337,
"protocol": "tcp",
"name": "http"
}
]
}
},
"ipAddress": {
"networkName": "dcos"
}
}
{code}
was (Author: gkleiman):
h1. Summary / Suggested Solution
To keep the implementation simple and consistent, I propose that the Docker
executor's health check thread should always enter the task's net namespace and
connect to {{localhost}} on the port specified in the health check proto.
The framework would then be responsible for deciding if the health checker
should connect to the {{containerPort}} or the {{hostPort}}.
h1. Testing results
h2. HOST networking
# Connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<hostPort>}}.
NOTE: In this case, both the docker executor and the container are in the same
network, hence `setns` call will be a noop. We still suggest not to treat this
case differently for consistency and readability.
h2. BRIDGED networking
# Connect to {{localhost:<hostPort>}}.
# Enter the container's net ns and connect to {{localhost:<containerPort>}}.
h2. USER networking (DC/OS Overlay network)
# If the task specifies a {{hostPort}}, connect to {{localhost:<hostPort>}}.
# If the task doesn't use a host port, enter the container's net ns and connect
to {{localhost:<containerPort>}}. I expect this to be the most common case.
h1. Testing notes
h2. Getting the networking namespace
{code}
NETNS="/proc/`docker inspect -f '{{.State.Pid}}' <container id>`/ns/net"
{code}
h2. Test Utility
The following utility enters a network namespace and does an HTTP request.
{code}
package main
import (
"fmt"
"io"
"net/http"
"os"
"syscall"
"github.com/opencontainers/runc/libcontainer/system"
)
func doNetns(nsPath string) error {
ns, err := os.Open(nsPath)
if err != nil {
return err
}
return system.Setns(ns.Fd(), syscall.CLONE_NEWNET)
}
func doGet(port string) {
response, err := http.Get("http://localhost:"; + port)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
} else {
defer response.Body.Close()
_, err := io.Copy(os.Stderr, response.Body)
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n", err)
}
}
}
func main() {
doNetns(os.Args[1])
doGet(os.Args[2])
}
{code}
h2. Marathon App Definitions
h3. HOST networking
{code}
{
"id": "/host",
"cmd": "python -m SimpleHTTPServer $PORT0",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "HOST"
}
},
"portDefinitions": [
{
"name": "http",
"protocol": "tcp"
}
]
}
{code}
h3. BRIDGE networking
{code}
{
"id": "/bridged",
"cmd": "python -m SimpleHTTPServer 31337",
"cpus": 0.1,
"mem": 32,
"container": {
"type": "DOCKER",
"docker": {
"image": "python:2",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 31337,
"hostPort": 0,
"name": "http"
}
]
}
}
}
{code}
h3. USER networking (DC/OS Overlay network)
{code}
{
"id": "/overlay",
"cpus": 0.1,
"mem": 32,
"cmd": "python -m SimpleHTTPServer 31337",
"container": {
"type": "DOCKER",
"docker": {
"network": "USER",
"image": "python:2",
"portMappings": [
{
"containerPort": 31337,
"protocol": "tcp",
"name": "http"
}
]
}
},
"ipAddress": {
"networkName": "dcos"
}
}
{code}
> HTTP and TCP health checks should support docker executor and bridged mode.
> ---------------------------------------------------------------------------
>
> Key: MESOS-5961
> URL: https://issues.apache.org/jira/browse/MESOS-5961
> Project: Mesos
> Issue Type: Improvement
> Reporter: Alexander Rukletsov
> Assignee: haosdent
> Labels: health-check, mesosphere
>
> If an executor and a task, e.g. the docker executor and docker container in
> bridged mode, exist in different network namespaces, HTTP and TCP health
> checks using {{localhost}} may not work properly. One solution would be to
> enter the container's network in the health check binary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
