On 08/24/2016 04:04 PM, Avinash Sridharan wrote: > Oliver, you can't have the agent running on 127.0.0.1. The agent needs to > be running in a routeabl IP address (choose an IP from one of the > interfaces). > > Reason being that if agent is on local host the executor running in its own > network namespace will try to make a connection in its own network > namespace and fail.
Thanks! modying ip address to reachable IP works..... > On Wed, Aug 24, 2016 at 5:15 AM Olivier Sallou <[email protected]> > wrote: > >> I have the same behavior with Calico. Task get IP from CNI plugin, but >> task remains in STAGING and same logs. >> >> mesos-execute --containerizer=mesos \ >>> --name=cni \ >>> --master=127.0.0.1:5050 \ >>> --networks=calico-net-1 \ >>> --command="ifconfig" >> I0824 14:12:03.202328 24912 scheduler.cpp:172] Version: 1.0.0 >> I0824 14:12:03.203009 24911 scheduler.cpp:461] New master detected at >> [email protected]:5050 >> Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0017' >> Submitted task 'cni' to agent 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' >> >> REMAINS STAGING! >> >> >> I0824 14:12:03.857158 24806 cni.cpp:1109] Got assigned IPv4 address >> '192.168.0.0/32' from CNI network 'calico-net-1' for container >> bdbb275a-ec5f-4a50-aca0-5e694ae57324 >> I0824 14:12:03.857348 24805 cni.cpp:838] Unable to find DNS nameservers >> for container bdbb275a-ec5f-4a50-aca0-5e694ae57324. Using host >> '/etc/resolv.conf' >> >> No more logs >> >> >> Olivier >> >> On 08/24/2016 08:23 AM, Olivier Sallou wrote: >>> On 08/23/2016 06:13 PM, Jie Yu wrote: >>>> The DNS related logging means that the weave plugin does not return DNS >>>> information, the agent uses the host resolv.conf for the container. So I >>>> think is irrelevant to your problem. >>>> >>>> Mesos requires that executor can talk to agent. Can you see if there is >> a >>>> route from 10.32.0.1 to the agent IP? >>> How can I check this as task does not start ? I have exposed weave >>> network on host: >>> >>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo ./weave expose >>> 10.32.0.2 >>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ ping 10.32.0.2 >>> PING 10.32.0.2 (10.32.0.2) 56(84) bytes of data. >>> 64 bytes from 10.32.0.2: icmp_seq=1 ttl=64 time=0.032 ms >>> 64 bytes from 10.32.0.2: icmp_seq=2 ttl=64 time=0.029 ms >>> 64 bytes from 10.32.0.2: icmp_seq=3 ttl=64 time=0.029 ms >>> 64 bytes from 10.32.0.2: icmp_seq=4 ttl=64 time=0.031 ms >>> >>> And why is it blocking? >>> >>> I am on a single host environement, so agent is on 127.0.0.1. >>> >>> Olivier >>>> On Tue, Aug 23, 2016 at 9:05 AM, Olivier Sallou < >> [email protected]> >>>> wrote: >>>> >>>>> HI, >>>>> >>>>> I have setup Mesos 1.0.0-2 to use CNI with Weave (1.6.1) >>>>> >>>>> Weave works nicely with the Docker containerizer. >>>>> >>>>> When I try to launch a task via my framework with unified >> containerizer, >>>>> the job remains waiting forever (no RUNNING message). I can see however >>>>> that weave cni allocated an IP address to Mesos. >>>>> >>>>> I tried with a simple mesos-execute test. >>>>> >>>>> Example with a mesos-execute with no CNI, everything is OK >>>>> >>>>> >>>>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo >> mesos-execute >>>>> --command="sleep 2" -docker_image=centos:latest --master= >> 127.0.0.1:5050 >>>>> --name=test0 I0823 17:56:50.067520 28815 scheduler.cpp:172] Version: >> 1.0.0 >>>>> I0823 17:56:50.068260 28822 scheduler.cpp:461] New master detected at >>>>> [email protected]:5050 >>>>> Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0005' >>>>> Submitted task 'test0' to agent >> 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' >>>>> Received status update TASK_RUNNING for task 'test0' >>>>> source: SOURCE_EXECUTOR >>>>> Received status update TASK_FINISHED for task 'test0' >>>>> message: 'Command exited with status 0' >>>>> >>>>> >>>>> Sample example specifying the weave network >>>>> >>>>> >>>>> osallou@tifenn~/Development/NOSAVE/go-docker/weave $ sudo >> mesos-execute >>>>> --command="sleep 2" -docker_image=centos:latest --master= >> 127.0.0.1:5050 >>>>> --name=test0 --networks=weave >>>>> I0823 17:57:15.845304 28856 scheduler.cpp:172] Version: 1.0.0 >>>>> I0823 17:57:15.846248 28857 scheduler.cpp:461] New master detected at >>>>> [email protected]:5050 >>>>> Subscribed with ID 'b1925e13-76db-4225-a3dc-39ce65c79b3c-0006' >>>>> Submitted task 'test0' to agent >> 'b1925e13-76db-4225-a3dc-39ce65c79b3c-S0' >>>>> ==> REMAINS WAITING HERE, job is in STAGING in Mesos UI >>>>> >>>>> mesos-slave logs: >>>>> >>>>> I0823 17:57:15.873872 26522 cni.cpp:716] Bind mounted >>>>> '/proc/28869/ns/net' to >>>>> >> '/run/mesos/isolators/network/cni/4f91a5df-2e9a-4cfc-93f5-aa197646db09/ns' >>>>> for container 4f91a5df-2e9a-4cfc-93f5-aa197646db09 >>>>> I0823 17:57:16.257063 26519 cni.cpp:1109] Got assigned IPv4 address >>>>> '10.32.0.1/12' from CNI network 'weave' for container >>>>> 4f91a5df-2e9a-4cfc-93f5-aa197646db09 >>>>> I0823 17:57:16.257258 26525 cni.cpp:838] Unable to find DNS nameservers >>>>> for container 4f91a5df-2e9a-4cfc-93f5-aa197646db09. Using host >>>>> '/etc/resolv.conf' >>>>> >>>>> There are no other logs until I kill the job. >>>>> We can see that Mesos container got an IP but it seems to block on DNS, >>>>> >>>>> Thanks for hints.... >>>>> >>>>> -- >>>>> >>>>> gpg key id: 4096R/326D8438 (keyring.debian.org) >>>>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 >>>>> >>>>> >>>>> >> -- >> Olivier Sallou >> IRISA / University of Rennes 1 >> Campus de Beaulieu, 35000 RENNES - FRANCE >> Tel: 02.99.84.71.95 >> >> gpg key id: 4096R/326D8438 (keyring.debian.org) >> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 >> >> -- Olivier Sallou IRISA / University of Rennes 1 Campus de Beaulieu, 35000 RENNES - FRANCE Tel: 02.99.84.71.95 gpg key id: 4096R/326D8438 (keyring.debian.org) Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
