I have been using this way from 0.23.0 to the 0.28.0. This has been surely working (although for a different framework). Inside the docker container can you see the $HOST variable defined??
The ticket you referred says that the apps definition needs to define LIBPROCESS_HOST=$HOST to be make the framework take the proper IP - you are describing a different problem. Thanks, ./Siva. On Sun, Jun 5, 2016 at 4:30 AM, Eli Jordan <[email protected]> wrote: > I found this issue on the mesos jira that describes the exact issue I am > hitting. > > https://issues.apache.org/jira/browse/MESOS-3740 > > It doesn't appear to be resolved. > > Thanks > Eli > > On 5 Jun 2016, at 16:46, Eli Jordan <[email protected]> wrote: > > Hmmm… that doesn’t seem to work for me. What version of mesos does this > work in? I am running 0.27.1. > > When using this approach, I still get the following error when the kafka > mess framework is starting up. > > "Scheduler driver bound to loopback interface! Cannot communicate with > remote master(s). You might want to set 'LIBPROCESS_IP' environment > variable to use a routable IP address.” > > I tried setting LIBPROCESS_IP to ‘0.0.0.0’ and > LIBPROCESS_ADVERTISE_IP=‘the public ip’ and this works. But the host > variations don’t seem to work. (i.e. set LIBPROCESS_IP=0.0.0.0 and > LIBPROCESS_ADVERTISE_HOST=$HOST) > > It seems lib process doesn’t support using host names. > > I think I might have to run the framework outside of docker, but I would > really like to avoid this. > > This problem would be solved if the docker executor was able to set the > same environment variables as the command executor. Is there a way to make > this happen? > > I saw that mesos can be extended with a Hook ‘module’ to set extra > environment variables in docker containers. This might be a solution, but > seems over wrought for a simple problem. > > > On 5 Jun 2016, at 12:50 am, Sivaram Kannan <[email protected]> wrote: > > > Hi, > > Can you try adding && after the LIBPROCESS_HOST variable and the actual > command. We have been using this for sometime now. > > "cmd": "LIBPROCESS_HOST=$HOST && ./kafka-mesos.sh .. > > Thanks, > ./Siva. > > > On Sat, Jun 4, 2016 at 8:34 AM, Eli Jordan <[email protected]> > wrote: > >> Hi @haosdent >> >> Based on my testing, this is not the case. >> >> I ran a task (from marathon) without using a docker container that just >> printed out all environment variables. i.e. while [ true ]; do env; sleep >> 2; done >> >> I then run a task that executed the same command inside an alpine docker >> image. >> >> When running without a docker image LIBPROCESS_IP was defined along with >> many other variables. >> >> Sample output when running without docker (note LIBPROCESS_IP) is defined >> >> Registered executor on mesos-slave0 >> Starting task plain-test.5e5b00cc-2645-11e6-a3dd-080027aa149e >> sh -c 'while [ true ]; do env; sleep 2; done' >> Forked command at 16571 >> LIBPROCESS_IP=192.168.3.16 >> MESOS_AGENT_ENDPOINT=192.168.3.16:5051 >> MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins >> HOST=mesos-slave0 >> SHELL=/bin/sh >> >> MESOS_DIRECTORY=/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/plain-test.5e5b00cc-2645-11e6-a3dd-080027aa149e/runs/c9b6ef86-b37d-4e3c-b1ca-bd680aed779f >> PORT0=31082 >> PORT_10001=31082 >> LC_ALL=en_US.UTF-8 >> … more >> >> >> Sample output when running with docker (note LIBPROCESS_IP is not defined) >> >> --container="mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42" >> --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" >> --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" >> --logbufsecs="0" --logging_level="INFO" >> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >> --sandbox_directory="/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e/runs/f3a94ab4-dfff-4e97-b806-f1cc501ecf42" >> --stop_timeout="0ns" >> --container="mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42" >> --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" >> --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" >> --logbufsecs="0" --logging_level="INFO" >> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >> --sandbox_directory="/var/mesos/slaves/7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0/frameworks/aae929c7-24a5-4463-9ae0-bc7b044973c5-0000/executors/alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e/runs/f3a94ab4-dfff-4e97-b806-f1cc501ecf42" >> --stop_timeout="0ns" >> Registered docker executor on mesos-slave0 >> Starting task alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e >> HOSTNAME=984809b0b720 >> SHLVL=1 >> HOME=/root >> PORT=31295 >> >> MESOS_CONTAINER_NAME=mesos-7ad17efe-0f9e-4703-9d2e-7fb9ee03f64c-S0.f3a94ab4-dfff-4e97-b806-f1cc501ecf42 >> MARATHON_APP_ID=/alpine-test >> PORTS=31295 >> PORT0=31295 >> MARATHON_APP_DOCKER_IMAGE=alpine >> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin >> MESOS_SANDBOX=/mnt/mesos/sandbox >> MARATHON_APP_RESOURCE_DISK=0.0 >> MARATHON_APP_RESOURCE_MEM=128.0 >> HOST=mesos-slave0 >> PORT_10000=31295 >> MARATHON_APP_VERSION=2016-05-30T08:56:59.065Z >> MARATHON_APP_LABELS= >> PWD=/ >> MESOS_TASK_ID=alpine-test.77d5a3d9-2644-11e6-a3dd-080027aa149e >> MARATHON_APP_RESOURCE_CPUS=1.0 >> >> Is there some other config I need to do for the docker containerizer? Any >> help greatly appreciated. >> >> On 4 Jun 2016, at 7:28 pm, haosdent <[email protected]> wrote: >> >> Hi, @Jordan. I think not matter you use MesosContainerizer or >> DockerContainerizer, LIBPROCESS_IP always would be set if you launch you >> Agent with `--ip` flag. >> >> On Fri, Jun 3, 2016 at 8:23 PM, Eli Jordan <[email protected]> >> wrote: >> >>> >>> The reason I need to set LIBPROCESS_IP is because the slaves have 2 >>> network interfaces, and the docker container is running in host networking >>> mode. So libmesos doesn’t know which IP to advertise. The hostnames of the >>> slaves are all resolvable. >>> >>> I have noticed that if I run a marathon app that doesn’t use docker, >>> e.g. while [ true ]; do env; sleep 2; done, that LIBPROCESS_IP is defined >>> in the environment. However, when running a docker image this variable is >>> not defined. >>> >>> Is there a way to have marathon pass along all environment variables >>> defined by mesos? >>> Thanks >>> Eli >>> >>> On 4 Apr 2016, at 14:12, Eli Jordan <[email protected]> wrote: >>> >>> @haosdent I’m not sure how this works internally, but it seems the mess >>> master needs to send requests to the framework for resource offers, and >>> therefore needs to know the external IP (i.e. the host IP) >>> >>> @craig w >>> Would I need to do this in the cmd portion of the marathon JSON? I >>> currently have the config below >>> { >>> "container": ..., >>> "id":"kafka-mesos-scheduler", >>> "cpus": 0.5, >>> "mem": 256, >>> "ports": [9999], >>> "cmd": "./kafka-mesos.sh scheduler --master=mesos-master:5050 >>> --zk=mesos-master:2181 --api=http://mesos-slave0:9999 >>> --storage=zk:/kafka-mesos", >>> "instances": 1, >>> "constraints": [["hostname", "LIKE", "mesos-slave0"]], >>> "env": { >>> "LIBPROCESS_IP": "192.168.3.16" >>> } >>> } >>> >>> @Chris Baker Currently we don’t have mess-dns setup but if this works it >>> would seem to be a nice solution. However, I did try setting LIBPROCESS_IP >>> to the slave hostname and it seems to produce a parse error. So I think it >>> needs to be an actual IP address. >>> >>> I was hoping there would be a configuration for the slave that would >>> automatically populate this env variable when starting the docker >>> container. So I don’t need to complicate the marathon file, and can reuse >>> it in different clusters. >>> >>> >>> On 4 Apr 2016, at 11:25 am, Chris Baker <[email protected]> wrote: >>> >>> Alternatively, because the $HOST username is indirect, which would >>> require a runtime element to "export $LIBPROCESS_IP=$HOST", another >>> alternative is to fallback on Mesos-DNS, if that's part of the cluster >>> deployment, setting $LIBPROCESS_IP to the (a priori) Mesos-DNS entry >>> corresponding to the service. >>> >>> On Sun, Apr 3, 2016 at 5:06 PM craig w <[email protected]> wrote: >>> >>>> Hi, marathon sets the HOST env var. If it's not the ip address you can >>>> use getent with the value from HOST to figure it out. >>>> >However, in order for the frameworks to receive resource offers I >>>> need to set the LIBPROCESS_IP environment variable to the hosts IP address >>>> for the docker container running the frameworks. >>>> >>>> Hi, @Gmail. Could you provide more details about this? >>>> >>>> On Sun, Apr 3, 2016 at 10:40 PM, Rad Gruchalski <[email protected]> >>>> wrote: >>>> >>>>> Hi Gmail, >>>>> >>>>> AFAIK not. The only way to do so is setting up the env variable as you >>>>> do now. >>>>> >>>>> >>>>> >>>>> Kind regards, >>>>> Radek Gruchalski >>>>> [email protected] <[email protected]> >>>>> de.linkedin.com/in/radgruchalski/ >>>>> >>>>> >>>>> *Confidentiality:*This communication is intended for the above-named >>>>> person and may be confidential and/or legally privileged. >>>>> If it has come to you in error you must take no action based on it, >>>>> nor must you copy or show it to anyone; please delete/destroy and inform >>>>> the sender immediately. >>>>> >>>>> On Sunday, 3 April 2016 at 16:09, Gmail wrote: >>>>> >>>>> I'm pretty new to mesos and marathon, and I'm running a couple of >>>>> frameworks with marathon (Kafka and elastic search). However, in order for >>>>> the frameworks to receive resource offers I need to set the LIBPROCESS_IP >>>>> environment variable to the hosts IP address for the docker container >>>>> running the frameworks. Currently I am working around me this by using a >>>>> constraint to hard wire the slave that the framework gets launched on, so >>>>> then I can put the slaves ip in the marathon json file. >>>>> >>>>> Obviously this is not ideal. Is there a better way to define the host >>>>> ip Inside the docker container? >>>>> >>>>> Sent from my iPad >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Haosdent Huang >>>> >>> >>> >> >> >> -- >> Best Regards, >> Haosdent Huang >> >> >> > > > -- > ever tried. ever failed. no matter. > try again. fail again. fail better. > -- Samuel Beckett > > > -- ever tried. ever failed. no matter. try again. fail again. fail better. -- Samuel Beckett
