Oliver, you can't have the agent running on 127.0.0.1. The agent needs to be running in a routeabl IP address (choose an IP from one of the interfaces).
Reason being that if agent is on local host the executor running in its own network namespace will try to make a connection in its own network namespace and fail. On Wed, Aug 24, 2016 at 5:15 AM Olivier Sallou <[email protected]> wrote: > I have the same behavior with Calico. Task get IP from CNI plugin, but > task remains in STAGING and same logs. > > mesos-execute --containerizer=mesos \ > > --name=cni \ > > --master=127.0.0.1:5050 \ > > --networks=calico-net-1 \ > > --command="ifconfig" > I0824 14:12:03.202328 24912 scheduler.cpp:172] Version: 1.0.0 > I0824 14:12:03.203009 24911 scheduler.cpp:461] New master detected at > [email protected]:5050 > Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0017' > Submitted task 'cni' to agent 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' > > REMAINS STAGING! > > > I0824 14:12:03.857158 24806 cni.cpp:1109] Got assigned IPv4 address > '192.168.0.0/32' from CNI network 'calico-net-1' for container > bdbb275a-ec5f-4a50-aca0-5e694ae57324 > I0824 14:12:03.857348 24805 cni.cpp:838] Unable to find DNS nameservers > for container bdbb275a-ec5f-4a50-aca0-5e694ae57324. Using host > '/etc/resolv.conf' > > No more logs > > > Olivier > > On 08/24/2016 08:23 AM, Olivier Sallou wrote: > > > > On 08/23/2016 06:13 PM, Jie Yu wrote: > >> The DNS related logging means that the weave plugin does not return DNS > >> information, the agent uses the host resolv.conf for the container. So I > >> think is irrelevant to your problem. > >> > >> Mesos requires that executor can talk to agent. Can you see if there is > a > >> route from 10.32.0.1 to the agent IP? > > How can I check this as task does not start ? I have exposed weave > > network on host: > > > > osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo ./weave expose > > 10.32.0.2 > > osallou@tifenn~/Development/NOSAVE/go-docker/weave $ ping 10.32.0.2 > > PING 10.32.0.2 (10.32.0.2) 56(84) bytes of data. > > 64 bytes from 10.32.0.2: icmp_seq=1 ttl=64 time=0.032 ms > > 64 bytes from 10.32.0.2: icmp_seq=2 ttl=64 time=0.029 ms > > 64 bytes from 10.32.0.2: icmp_seq=3 ttl=64 time=0.029 ms > > 64 bytes from 10.32.0.2: icmp_seq=4 ttl=64 time=0.031 ms > > > > And why is it blocking? > > > > I am on a single host environement, so agent is on 127.0.0.1. > > > > Olivier > >> On Tue, Aug 23, 2016 at 9:05 AM, Olivier Sallou < > [email protected]> > >> wrote: > >> > >>> HI, > >>> > >>> I have setup Mesos 1.0.0-2 to use CNI with Weave (1.6.1) > >>> > >>> Weave works nicely with the Docker containerizer. > >>> > >>> When I try to launch a task via my framework with unified > containerizer, > >>> the job remains waiting forever (no RUNNING message). I can see however > >>> that weave cni allocated an IP address to Mesos. > >>> > >>> I tried with a simple mesos-execute test. > >>> > >>> Example with a mesos-execute with no CNI, everything is OK > >>> > >>> > >>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo > mesos-execute > >>> --command="sleep 2" -docker_image=centos:latest --master= > 127.0.0.1:5050 > >>> --name=test0 I0823 17:56:50.067520 28815 scheduler.cpp:172] Version: > 1.0.0 > >>> I0823 17:56:50.068260 28822 scheduler.cpp:461] New master detected at > >>> [email protected]:5050 > >>> Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0005' > >>> Submitted task 'test0' to agent > 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' > >>> Received status update TASK_RUNNING for task 'test0' > >>> source: SOURCE_EXECUTOR > >>> Received status update TASK_FINISHED for task 'test0' > >>> message: 'Command exited with status 0' > >>> > >>> > >>> Sample example specifying the weave network > >>> > >>> > >>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo > mesos-execute > >>> --command="sleep 2" -docker_image=centos:latest --master= > 127.0.0.1:5050 > >>> --name=test0 --networks=weave > >>> I0823 17:57:15.845304 28856 scheduler.cpp:172] Version: 1.0.0 > >>> I0823 17:57:15.846248 28857 scheduler.cpp:461] New master detected at > >>> [email protected]:5050 > >>> Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0006' > >>> Submitted task 'test0' to agent > 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' > >>> ==> REMAINS WAITING HERE, job is in STAGING in Mesos UI > >>> > >>> mesos-slave logs: > >>> > >>> I0823 17:57:15.873872 26522 cni.cpp:716] Bind mounted > >>> '/proc/28869/ns/net' to > >>> > '/run/mesos/isolators/network/cni/4f91a5df-2e9a-4cfc-93f5-aa197646db09/ns' > >>> for container 4f91a5df-2e9a-4cfc-93f5-aa197646db09 > >>> I0823 17:57:16.257063 26519 cni.cpp:1109] Got assigned IPv4 address > >>> '10.32.0.1/12' from CNI network 'weave' for container > >>> 4f91a5df-2e9a-4cfc-93f5-aa197646db09 > >>> I0823 17:57:16.257258 26525 cni.cpp:838] Unable to find DNS nameservers > >>> for container 4f91a5df-2e9a-4cfc-93f5-aa197646db09. Using host > >>> '/etc/resolv.conf' > >>> > >>> There are no other logs until I kill the job. > >>> We can see that Mesos container got an IP but it seems to block on DNS, > >>> > >>> Thanks for hints.... > >>> > >>> -- > >>> > >>> gpg key id: 4096R/326D8438 (keyring.debian.org) > >>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 > >>> > >>> > >>> > > -- > Olivier Sallou > IRISA / University of Rennes 1 > Campus de Beaulieu, 35000 RENNES - FRANCE > Tel: 02.99.84.71.95 > > gpg key id: 4096R/326D8438 (keyring.debian.org) > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 > >
