[CNI] Marathon task with IP from host-local plugin fails

[Frank Scholten]

Hi,

I have a Mesos 1.0.1 cluster with the following host-local CNI configuration:

{
    "cniVersion": "0.3.0",
    "name": "ipv4",
    "type": "host-local",
    "isGateway": "true",
    "ipMasq": "true",
    "ipam": {
        "type": "host-local",
        "subnet": "10.22.0.0/16",
        "routes": [
            { "dst": "0.0.0.0/0" }
        ]
    }
}

Now I try to deploy the following app in Marathon:

{
  "id": "netcat",
  "cpus": 1,
  "mem": 128,
  "instances": 1,
  "ipAddress": {
    "networkName": "ipv4"
  },
  "cmd": "nc -l 8080"
}

In the agent logs I can see the task gets an IP address from the
host-local plugin

I1108 18:46:51.919564 24354 containerizer.cpp:1319] Checkpointing
executor's forked pid 31587 to
'/tmp/mesos/meta/slaves/32f8badc-9771-4d3c-b2f5-083e63619946-S0/frameworks/32f8badc-9771-4d3c-b2f5-083e63619946-0000/executors/netcat.b678ad45-a5e3-11e6-8a46-0242ac110006/runs/324b867e-9464-4d32-bccb-f7f3940335dc/pids/forked.pid'
I1108 18:46:51.920114 24355 cni.cpp:716] Bind mounted
'/proc/31587/ns/net' to
'/run/mesos/isolators/network/cni/324b867e-9464-4d32-bccb-f7f3940335dc/ns'
for container 324b867e-9464-4d32-bccb-f7f3940335dc
I1108 18:46:51.920394 24355 cni.cpp:1030] Invoking CNI plugin
'host-local' with network configuration
'{"args":{"org.apache.mesos":{"network_info":{"ip_addresses":[{}],"labels":{},"name":"ipv4"}}},"cniVersion":"0.3.0","ipMasq":"true","ipam":{"routes":[{"dst":"0.0.0.0\/0"}],"subnet":"10.22.0.0\/16","type":"host-local"},"isGateway":"true","name":"ipv4","type":"host-local"}'
to attach container 324b867e-9464-4d32-bccb-f7f3940335dc to network
'ipv4'
I1108 18:46:51.941431 24354 cni.cpp:1109] Got assigned IPv4 address
'10.22.0.2/16' from CNI network 'ipv4' for container
324b867e-9464-4d32-bccb-f7f3940335dc
I1108 18:46:51.941766 24352 cni.cpp:838] Unable to find DNS
nameservers for container 324b867e-9464-4d32-bccb-f7f3940335dc. Using
host '/etc/resolv.conf'

However the task status is FAILED and it gets restarted over and over again.

The sandbox stdout logs are empty. This is the contents of the stderr logs:

I1108 18:46:32.717352 30482 exec.cpp:161] Version: 1.0.0
E1108 18:46:32.717542 30482 process.cpp:2105] Failed to shutdown
socket with fd 7: Transport endpoint is not connected
E1108 18:46:32.717600 30482 process.cpp:2105] Failed to shutdown
socket with fd 7: Transport endpoint is not connected
I1108 18:46:32.717619 30482 exec.cpp:495] Agent exited ... shutting down

Any ideas?

Cheers,

Frank