Are you enabling authenticate_slaves flag on your master? If so, are cyrus-sasl-devel cyrus-sasl-md5 rpms installed? On the masters and service restarted? Have you cleaned up previous meta data on the slaves? rm -vf /tmp/mesos/meta/slaves/latest
Shafay Latif On Jul 29, 2015, at 9:37 AM, Haripriya Ayyalasomayajula <[email protected]> wrote: Thanks. It does not complain about LIBPROCESS any more. It is able to connect to the master, but hangs before the framework can register. Can you please help me understand where I am going wrong? I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master (UPID=master@my_ip:5050) is detected I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my _ip:5050 I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided. Attempting to register without authentication > On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <[email protected]> wrote: > Hi Haripiya, > > When you run Spark on Mesos it needs to run > > spark driver > mesos scheduler > > and both need to be visible to outside world on public iface IP > > you need to tell Spark and Mesos on which interface to bind - by default they > resolve node hostname to ip - this is loopback address in your case > > Possible solutions - on slave node with public IP 192.168.56.50 > > 1. Set > > export LIBPROCESS_IP=192.168.56.50 > export SPARK_LOCAL_IP=192.168.56.50 > > 2. Ensure your hostname resolves to public iface IP - (for testing) edit > /etc/hosts to resolve your domain name to 192.168.56.50 > 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer > > > > > 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <[email protected]>: >> It quits before it writes any logs, when I look at the directory for logs, >> it has empty files! >> >>> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <[email protected]> wrote: >>> can you paste the logs? >>> >>>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula >>>> <[email protected]> wrote: >>>> Well, when I try doing that I get this error: >>>> >>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot assign >>>> requested address: Cannot assign requested address [99] >>>> >>>> When I do a >>>> >>>> ps -ef | grep mesos >>>> >>>> on both my master and slave nodes, it works fine. And, I am also able to >>>> ping both the nodes from each other - they are reachable to one another. >>>> >>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <[email protected]> wrote: >>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to >>>>> bind to. It has nothing to do with the ZooKeeper address. >>>>> >>>>> In other words, do >>>>> >>>>> export LIBPROCESS_IP=scheduler_ip_address >>>>> >>>>> >>>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula >>>>>> <[email protected]> wrote: >>>>>> I am trying to do this >>>>>> >>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos >>>>>> ./bin/spark-shell >>>>>> >>>>>> It gives me this error and aborts >>>>>> >>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR >>>>>> >>>>>> >>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing >>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP >>>>>> >>>>>> >>>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <[email protected]> wrote: >>>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell, or >>>>>>> you can just set it before you call spark-shell yourself. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula >>>>>>>> <[email protected]> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats >>>>>>>> the only place I can think of. Can you please point me to any related >>>>>>>> documentation? >>>>>>>> >>>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus >>>>>>>>> <[email protected]> wrote: >>>>>>>>> If you are not using any dns like service under /etc/mesos-master/ >>>>>>>>> create two files called ip and hostname and put the ip of the eth >>>>>>>>> interface. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent from my Samsung device >>>>>>>>> >>>>>>>>> >>>>>>>>> -------- Original message -------- >>>>>>>>> From: Haripriya Ayyalasomayajula <[email protected]> >>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00) >>>>>>>>> To: [email protected] >>>>>>>>> Subject: Problems connecting with Mesos Master >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0. >>>>>>>>> >>>>>>>>> When I try to start my spark-shell, it gives me the following warning >>>>>>>>> : >>>>>>>>> ************************************************** >>>>>>>>> >>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate with >>>>>>>>> remote master(s). You might want to set 'LIBPROCESS_IP' environment >>>>>>>>> variable to use a routable IP address. >>>>>>>>> >>>>>>>>> --- >>>>>>>>> >>>>>>>>> Spark-shell works fine on the node where I run master, but if I start >>>>>>>>> running on any of the other slave nodes it gives me the following >>>>>>>>> error: >>>>>>>>> >>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6: >>>>>>>>> Transport endpoint is not connected [107] >>>>>>>>> >>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6: >>>>>>>>> Transport endpoint is not connected [107] >>>>>>>>> >>>>>>>>> I have the following configs: >>>>>>>>> >>>>>>>>> zookeeper configured to the mesos master >>>>>>>>> /etc/mesos/zk on all nodes pointing to mesos master ip. >>>>>>>>> I am not sure if I have to set the ip flag and where I have to set >>>>>>>>> the --ip flag? >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Haripriya Ayyalasomayajula >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Haripriya Ayyalasomayajula >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Haripriya Ayyalasomayajula >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Haripriya Ayyalasomayajula >> >> >> >> -- >> Regards, >> Haripriya Ayyalasomayajula -- Regards, Haripriya Ayyalasomayajula
