Got it working with LIBPROCESS_IP parameter in the marathon. Thanks a lot for pointing out.
On Fri, Mar 6, 2015 at 12:50 PM, Sivaram Kannan <[email protected]> wrote: > > Opps. You are right on target. These are all run as containers. The > mesos-master is unable to reach the marathon when it is in another node. > Did not see the master logs what it is complaining about. > > I0305 17:31:52.019635 16 authenticator.hpp:166] Creating new server > SASL connection > W0305 17:31:53.022302 14 master.cpp:3954] Failed to authenticate > [email protected]:9092: Failed > to communicate with authenticatee > I0305 17:31:53.497721 13 http.cpp:478] HTTP request for > '/master/state.json' > I0305 17:31:57.023936 10 master.cpp:3901] Authenticating > [email protected]:9092 > I0305 17:31:57.024039 10 master.cpp:3912] Using default CRAM-MD5 > authenticator > I0305 17:31:57.024209 10 authenticator.hpp:166] Creating new server > SASL connection > W0305 17:32:00.034343 15 master.cpp:3954] Failed to authenticate > [email protected]:9092: Failed > to communicate with authenticatee > I0305 17:32:02.033409 11 master.cpp:3901] Authenticating > [email protected]:9092 > I0305 17:32:02.033514 11 master.cpp:3912] Using default CRAM-MD5 > authenticator > I0305 17:32:02.033685 11 authenticator.hpp:166] Creating new server > SASL connection > W0305 17:32:03.041254 9 master.cpp:3954] Failed to authenticate > [email protected]:9092: Failed > to communicate with authenticatee > I0305 17:32:03.507251 14 http.cpp:478] HTTP request for > '/master/state.json' > > I guess LIBPROCESS_IP would solve the problem for me. Let me try and will > get back. Thanks a lot. > > ./Siva. > > On Fri, Mar 6, 2015 at 12:39 PM, Adam Bordelon <[email protected]> wrote: > >> Seems like a networking issue. Please check that all of the nodes can >> ping each other, and that there aren't any iptables rules preventing >> connections on ports 5050 and 8080. >> >> Can you also provide the master log to see if the master receives the >> authentication request from the framework, and what pid/IP it thinks the >> framework is using? You may need to set --hostname and/or LIBPROCESS_IP in >> Marathon, or modify your /etc/hosts file, especially it's advertising >> itself as 127.0.x.1. >> >> On Thu, Mar 5, 2015 at 10:48 PM, Sivaram Kannan <[email protected]> >> wrote: >> >>> >>> All the nodes has the credentials and authentication_secret_file where >>> the mesos-master and marathon pick it from a standard location. >>> >>> On Fri, Mar 6, 2015 at 12:15 PM, Sivaram Kannan <[email protected]> >>> wrote: >>> >>>> >>>> Hi, >>>> >>>> Parameters in the mesos-master specifically for authorization and >>>> authentication >>>> >>>> --acls=/root/acl.json --authenticate --credentials=/root/passwd_file >>>> --roles="apps,dev-ops" --quorum=2 >>>> >>>> Parameters I pass in marathon >>>> >>>> --mesos_role "apps" --framework_name "marathon" >>>> --mesos_authentication_principal "user1" --mesos_authentication_secret_file >>>> "/root/marathon_password" >>>> >>>> If run from the same system, authentication works. For experimentation, >>>> I bought up mesos-master and marathon in the same node and another marathon >>>> instance in HA in another node - it came up. But when I brought down the >>>> marathon running in the same node as mesos-master, the other marathon >>>> instances started failing with the below log >>>> >>>> I0305 16:55:28.619690 47 sched.cpp:284] Authenticating with master >>>> [email protected]:5050 >>>> I0305 16:55:28.620254 47 authenticatee.hpp:133] Creating new client >>>> SASL connection >>>> W0305 16:55:33.626118 49 sched.cpp:379] Authentication timed out >>>> I0305 16:55:33.626292 49 sched.cpp:339] Failed to authenticate with >>>> master [email protected]:5050: Authentication discarded >>>> I0305 16:55:33.626947 49 sched.cpp:284] Authenticating with master >>>> [email protected]:5050 >>>> I0305 16:55:33.627034 49 authenticatee.hpp:133] Creating new client >>>> SASL connection >>>> W0305 16:55:38.631783 46 sched.cpp:379] Authentication timed out >>>> I0305 16:55:38.631957 46 sched.cpp:339] Failed to authenticate with >>>> master [email protected]:5050: Authentication discarded >>>> I0305 16:55:38.632975 46 sched.cpp:284] Authenticating with master >>>> [email protected]:5050 >>>> I0305 16:55:38.633229 46 authenticatee.hpp:133] Creating new client >>>> SASL connection >>>> >>>> Thanks, >>>> ./Siva >>>> >>>> On Fri, Mar 6, 2015 at 12:08 PM, Adam Bordelon <[email protected]> >>>> wrote: >>>> >>>>> What parameters are you setting on each of the masters and framework? >>>>> You'll need --credentials on the master and >>>>> --mesos_authentication_principal and --mesos_authentication_secret_file on >>>>> the (Marathon) framework node, and of course you'll need the credentials >>>>> file on the master and a matching secret file on the framework node. >>>>> >>>>> What exactly is the error message you see? >>>>> >>>>> On Thu, Mar 5, 2015 at 10:14 PM, Sivaram Kannan <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I have a 3 node mesos-master cluster. Say if node-1 is elected as >>>>>> mesos-master and if I run a framework(Marathon) in the same node, >>>>>> authentication succeeds. But if I run the framework from the node-2, >>>>>> authentication for the framework fails. The machines are all similarly >>>>>> configured in terms of hardware, same OS - everything is same except for >>>>>> the ip between hosts. >>>>>> >>>>>> Any idea why is this?? >>>>>> >>>>>> Thanks, >>>>>> ./Siva. >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> ever tried. ever failed. no matter. >>>> try again. fail again. fail better. >>>> -- Samuel Beckett >>>> >>> >>> >>> >>> -- >>> ever tried. ever failed. no matter. >>> try again. fail again. fail better. >>> -- Samuel Beckett >>> >> >> > > > -- > ever tried. ever failed. no matter. > try again. fail again. fail better. > -- Samuel Beckett > -- ever tried. ever failed. no matter. try again. fail again. fail better. -- Samuel Beckett
