Hmmm… that doesn’t seem to work for me. What version of mesos does this work in? I am running 0.27.1.
When using this approach, I still get the following error when the kafka mess framework is starting up. "Scheduler driver bound to loopback interface! Cannot communicate with remote master(s). You might want to set 'LIBPROCESS_IP' environment variable to use a routable IP address.” I tried setting LIBPROCESS_IP to ‘0.0.0.0’ and LIBPROCESS_ADVERTISE_IP=‘the public ip’ and this works. But the host variations don’t seem to work. (i.e. set LIBPROCESS_IP=0.0.0.0 and LIBPROCESS_ADVERTISE_HOST=$HOST) It seems lib process doesn’t support using host names. I think I might have to run the framework outside of docker, but I would really like to avoid this. This problem would be solved if the docker executor was able to set the same environment variables as the command executor. Is there a way to make this happen? I saw that mesos can be extended with a Hook ‘module’ to set extra environment variables in docker containers. This might be a solution, but seems over wrought for a simple problem. > On 5 Jun 2016, at 12:50 am, Sivaram Kannan <[email protected]> wrote: > > > Hi, > > Can you try adding && after the LIBPROCESS_HOST variable and the actual > command. We have been using this for sometime now. > > "cmd": "LIBPROCESS_HOST=$HOST && ./kafka-mesos.sh .. > > Thanks, > ./Siva. > > > On Sat, Jun 4, 2016 at 8:34 AM, Eli Jordan <[email protected] > <mailto:[email protected]>> wrote: > Hi @haosdent > > Based on my testing, this is not the case. > > I ran a task (from marathon) without using a docker container that just > printed out all environment variables. i.e. while [ true ]; do env; sleep 2; > done > > I then run a task that executed the same command inside an alpine docker > image. > > When running without a docker image LIBPROCESS_IP was defined along with many > other variables. > > Sample output when running without docker (note LIBPROCESS_IP) is defined > > Registered executor on mesos-slave0 > Starting task plain-test.5e5b00cc-2645-11e6-a3dd-080027aa149e > sh -c 'while [ true ]; do env; sleep 2; done' > Forked command at 16571 > LIBPROCESS_IP=192.168.3.16 > MESOS_AGENT_ENDPOINT=192.168.3.16:5051 <http://192.168.3.16:5051/> > MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins > HOST=mesos-slave0 > SHELL=/bin/sh > MESOS_DIRECTORY=/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/plain-test.5e5b00cc-2645-11e6-a3dd-080027aa149e/runs/c9b6ef86-b37d-4e3c-b1ca-bd680aed779f > PORT0=31082 > PORT_10001=31082 > LC_ALL=en_US.UTF-8 > … more > > > Sample output when running with docker (note LIBPROCESS_IP is not defined) > > --container="mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42" > --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" > --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" > --logbufsecs="0" --logging_level="INFO" > --mapped_directory="/mnt/mesos/sandbox" --quiet="false" > --sandbox_directory="/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e/runs/f3a94ab4-dfff-4e97-b806-f1cc501ecf42" > --stop_timeout="0ns" > --container="mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42" > --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" > --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" > --logbufsecs="0" --logging_level="INFO" > --mapped_directory="/mnt/mesos/sandbox" --quiet="false" > --sandbox_directory="/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e/runs/f3a94ab4-dfff-4e97-b806-f1cc501ecf42" > --stop_timeout="0ns" > Registered docker executor on mesos-slave0 > Starting task alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e > HOSTNAME=984809b0b720 > SHLVL=1 > HOME=/root > PORT=31295 > MESOS_CONTAINER_NAME=mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42 > MARATHON_APP_ID=/alpine-test > PORTS=31295 > PORT0=31295 > MARATHON_APP_DOCKER_IMAGE=alpine > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > MESOS_SANDBOX=/mnt/mesos/sandbox > MARATHON_APP_RESOURCE_DISK=0.0 > MARATHON_APP_RESOURCE_MEM=128.0 > HOST=mesos-slave0 > PORT_10000=31295 > MARATHON_APP_VERSION=2016-05-30T08:56:59.065Z > MARATHON_APP_LABELS= > PWD=/ > MESOS_TASK_ID=alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e > MARATHON_APP_RESOURCE_CPUS=1.0 > > Is there some other config I need to do for the docker containerizer? Any > help greatly appreciated. > >> On 4 Jun 2016, at 7:28 pm, haosdent <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, @Jordan. I think not matter you use MesosContainerizer or >> DockerContainerizer, LIBPROCESS_IP always would be set if you launch you >> Agent with `--ip` flag. >> >> On Fri, Jun 3, 2016 at 8:23 PM, Eli Jordan <[email protected] >> <mailto:[email protected]>> wrote: >> >> The reason I need to set LIBPROCESS_IP is because the slaves have 2 network >> interfaces, and the docker container is running in host networking mode. So >> libmesos doesn’t know which IP to advertise. The hostnames of the slaves are >> all resolvable. >> >> I have noticed that if I run a marathon app that doesn’t use docker, e.g. >> while [ true ]; do env; sleep 2; done, that LIBPROCESS_IP is defined in the >> environment. However, when running a docker image this variable is not >> defined. >> >> Is there a way to have marathon pass along all environment variables defined >> by mesos? >> Thanks >> Eli >> >> On 4 Apr 2016, at 14:12, Eli Jordan <[email protected] >> <mailto:[email protected]>> wrote: >> >>> @haosdent I’m not sure how this works internally, but it seems the mess >>> master needs to send requests to the framework for resource offers, and >>> therefore needs to know the external IP (i.e. the host IP) >>> >>> @craig w >>> Would I need to do this in the cmd portion of the marathon JSON? I >>> currently have the config below >>> { >>> "container": ..., >>> "id":"kafka-mesos-scheduler", >>> "cpus": 0.5, >>> "mem": 256, >>> "ports": [9999], >>> "cmd": "./kafka-mesos.sh scheduler --master=mesos-master:5050 >>> --zk=mesos-master:2181 --api=http://mesos-slave0:9999 >>> <http://mesos-slave0:9999/> --storage=zk:/kafka-mesos", >>> "instances": 1, >>> "constraints": [["hostname", "LIKE", "mesos-slave0"]], >>> "env": { >>> "LIBPROCESS_IP": "192.168.3.16" >>> } >>> } >>> >>> @Chris Baker Currently we don’t have mess-dns setup but if this works it >>> would seem to be a nice solution. However, I did try setting LIBPROCESS_IP >>> to the slave hostname and it seems to produce a parse error. So I think it >>> needs to be an actual IP address. >>> >>> I was hoping there would be a configuration for the slave that would >>> automatically populate this env variable when starting the docker >>> container. So I don’t need to complicate the marathon file, and can reuse >>> it in different clusters. >>> >>> >>>> On 4 Apr 2016, at 11:25 am, Chris Baker <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Alternatively, because the $HOST username is indirect, which would require >>>> a runtime element to "export $LIBPROCESS_IP=$HOST", another alternative is >>>> to fallback on Mesos-DNS, if that's part of the cluster deployment, >>>> setting $LIBPROCESS_IP to the (a priori) Mesos-DNS entry corresponding to >>>> the service. >>>> >>>> On Sun, Apr 3, 2016 at 5:06 PM craig w <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> Hi, marathon sets the HOST env var. If it's not the ip address you can use >>>> getent with the value from HOST to figure it out. >>>> >>>> >However, in order for the frameworks to receive resource offers I need to >>>> >set the LIBPROCESS_IP environment variable to the hosts IP address for >>>> >the docker container running the frameworks. >>>> >>>> Hi, @Gmail. Could you provide more details about this? >>>> >>>> On Sun, Apr 3, 2016 at 10:40 PM, Rad Gruchalski <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> Hi Gmail, >>>> >>>> AFAIK not. The only way to do so is setting up the env variable as you do >>>> now. >>>> >>>> >>>> >>>> Kind regards, >>>> >>>> Radek Gruchalski >>>> >>>> [email protected] <mailto:[email protected]> >>>> >>>> <mailto:[email protected]> >>>> de.linkedin.com/in/radgruchalski/ >>>> <http://de.linkedin.com/in/radgruchalski/> >>>> >>>> Confidentiality: >>>> This communication is intended for the above-named person and may be >>>> confidential and/or legally privileged. >>>> If it has come to you in error you must take no action based on it, nor >>>> must you copy or show it to anyone; please delete/destroy and inform the >>>> sender immediately. >>>> On Sunday, 3 April 2016 at 16:09, Gmail wrote: >>>> >>>>> I'm pretty new to mesos and marathon, and I'm running a couple of >>>>> frameworks with marathon (Kafka and elastic search). However, in order >>>>> for the frameworks to receive resource offers I need to set the >>>>> LIBPROCESS_IP environment variable to the hosts IP address for the docker >>>>> container running the frameworks. Currently I am working around me this >>>>> by using a constraint to hard wire the slave that the framework gets >>>>> launched on, so then I can put the slaves ip in the marathon json file. >>>>> >>>>> Obviously this is not ideal. Is there a better way to define the host ip >>>>> Inside the docker container? >>>>> >>>>> Sent from my iPad >>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Haosdent Huang >>> >> >> >> >> -- >> Best Regards, >> Haosdent Huang > > > > > -- > ever tried. ever failed. no matter. > try again. fail again. fail better. > -- Samuel Beckett
