Thanks. It does not complain about LIBPROCESS any more. It is able to connect to the master, but hangs before the framework can register.
Can you please help me understand where I am going wrong? I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master (UPID=master@my_ip:5050) is detected I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my _ip:5050 I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided. Attempting to register without authentication On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <[email protected]> wrote: > Hi Haripiya, > > When you run Spark on Mesos it needs to run > > spark driver > mesos scheduler > > and both need to be visible to outside world on public iface IP > > you need to tell Spark and Mesos on which interface to bind - by default > they resolve node hostname to ip - this is loopback address in your case > > Possible solutions - on slave node with public IP 192.168.56.50 > > 1. Set > > export LIBPROCESS_IP=192.168.56.50 > export SPARK_LOCAL_IP=192.168.56.50 > > 2. Ensure your hostname resolves to public iface IP - (for testing) edit > /etc/hosts to resolve your domain name to 192.168.56.50 > 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer > > > > > 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula < > [email protected]>: > >> It quits before it writes any logs, when I look at the directory for >> logs, it has empty files! >> >> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <[email protected]> wrote: >> >>> can you paste the logs? >>> >>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula < >>> [email protected]> wrote: >>> >>>> Well, when I try doing that I get this error: >>>> >>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot >>>> assign requested address: Cannot assign requested address [99] >>>> >>>> When I do a >>>> >>>> ps -ef | grep mesos >>>> >>>> on both my master and slave nodes, it works fine. And, I am also able >>>> to ping both the nodes from each other - they are reachable to one another. >>>> >>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <[email protected]> >>>> wrote: >>>> >>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) >>>>> to bind to. It has nothing to do with the ZooKeeper address. >>>>> >>>>> In other words, do >>>>> >>>>> export LIBPROCESS_IP=scheduler_ip_address >>>>> >>>>> >>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula < >>>>> [email protected]> wrote: >>>>> >>>>>> I am trying to do this >>>>>> >>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos >>>>>> >>>>>> ./bin/spark-shell >>>>>> >>>>>> It gives me this error and aborts >>>>>> >>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR >>>>>> >>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing >>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP >>>>>> >>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <[email protected]> wrote: >>>>>> >>>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell, >>>>>>> or you can just set it before you call spark-shell yourself. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats >>>>>>>> the only place I can think of. Can you please point me to any related >>>>>>>> documentation? >>>>>>>> >>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> If you are not using any dns like service under >>>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put >>>>>>>>> the ip >>>>>>>>> of the eth interface. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent from my Samsung device >>>>>>>>> >>>>>>>>> >>>>>>>>> -------- Original message -------- >>>>>>>>> From: Haripriya Ayyalasomayajula <[email protected]> >>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00) >>>>>>>>> To: [email protected] >>>>>>>>> Subject: Problems connecting with Mesos Master >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0. >>>>>>>>> >>>>>>>>> When I try to start my spark-shell, it gives me the following >>>>>>>>> warning : >>>>>>>>> >>>>>>>>> ************************************************** >>>>>>>>> >>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate >>>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' >>>>>>>>> environment >>>>>>>>> variable to use a routable IP address. >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Spark-shell works fine on the node where I run master, but if I >>>>>>>>> start running on any of the other slave nodes it gives me the >>>>>>>>> following >>>>>>>>> error: >>>>>>>>> >>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on >>>>>>>>> fd=6: Transport endpoint is not connected [107] >>>>>>>>> >>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on >>>>>>>>> fd=6: Transport endpoint is not connected [107] >>>>>>>>> >>>>>>>>> I have the following configs: >>>>>>>>> >>>>>>>>> >>>>>>>>> - zookeeper configured to the mesos master >>>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip. >>>>>>>>> >>>>>>>>> I am not sure if I have to set the ip flag and where I have to >>>>>>>>> set the --ip flag? >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Haripriya Ayyalasomayajula >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Haripriya Ayyalasomayajula >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Haripriya Ayyalasomayajula >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Haripriya Ayyalasomayajula >>>> >>>> >>> >> >> >> -- >> Regards, >> Haripriya Ayyalasomayajula >> >> > -- Regards, Haripriya Ayyalasomayajula
