Opps. You are right on target. These are all run as containers. The mesos-master is unable to reach the marathon when it is in another node. Did not see the master logs what it is complaining about.
I0305 17:31:52.019635 16 authenticator.hpp:166] Creating new server SASL connection W0305 17:31:53.022302 14 master.cpp:3954] Failed to authenticate [email protected]:9092: Failed to communicate with authenticatee I0305 17:31:53.497721 13 http.cpp:478] HTTP request for '/master/state.json' I0305 17:31:57.023936 10 master.cpp:3901] Authenticating [email protected]:9092 I0305 17:31:57.024039 10 master.cpp:3912] Using default CRAM-MD5 authenticator I0305 17:31:57.024209 10 authenticator.hpp:166] Creating new server SASL connection W0305 17:32:00.034343 15 master.cpp:3954] Failed to authenticate [email protected]:9092: Failed to communicate with authenticatee I0305 17:32:02.033409 11 master.cpp:3901] Authenticating [email protected]:9092 I0305 17:32:02.033514 11 master.cpp:3912] Using default CRAM-MD5 authenticator I0305 17:32:02.033685 11 authenticator.hpp:166] Creating new server SASL connection W0305 17:32:03.041254 9 master.cpp:3954] Failed to authenticate [email protected]:9092: Failed to communicate with authenticatee I0305 17:32:03.507251 14 http.cpp:478] HTTP request for '/master/state.json' I guess LIBPROCESS_IP would solve the problem for me. Let me try and will get back. Thanks a lot. ./Siva. On Fri, Mar 6, 2015 at 12:39 PM, Adam Bordelon <[email protected]> wrote: > Seems like a networking issue. Please check that all of the nodes can ping > each other, and that there aren't any iptables rules preventing connections > on ports 5050 and 8080. > > Can you also provide the master log to see if the master receives the > authentication request from the framework, and what pid/IP it thinks the > framework is using? You may need to set --hostname and/or LIBPROCESS_IP in > Marathon, or modify your /etc/hosts file, especially it's advertising > itself as 127.0.x.1. > > On Thu, Mar 5, 2015 at 10:48 PM, Sivaram Kannan <[email protected]> > wrote: > >> >> All the nodes has the credentials and authentication_secret_file where >> the mesos-master and marathon pick it from a standard location. >> >> On Fri, Mar 6, 2015 at 12:15 PM, Sivaram Kannan <[email protected]> >> wrote: >> >>> >>> Hi, >>> >>> Parameters in the mesos-master specifically for authorization and >>> authentication >>> >>> --acls=/root/acl.json --authenticate --credentials=/root/passwd_file >>> --roles="apps,dev-ops" --quorum=2 >>> >>> Parameters I pass in marathon >>> >>> --mesos_role "apps" --framework_name "marathon" >>> --mesos_authentication_principal "user1" --mesos_authentication_secret_file >>> "/root/marathon_password" >>> >>> If run from the same system, authentication works. For experimentation, >>> I bought up mesos-master and marathon in the same node and another marathon >>> instance in HA in another node - it came up. But when I brought down the >>> marathon running in the same node as mesos-master, the other marathon >>> instances started failing with the below log >>> >>> I0305 16:55:28.619690 47 sched.cpp:284] Authenticating with master >>> [email protected]:5050 >>> I0305 16:55:28.620254 47 authenticatee.hpp:133] Creating new client >>> SASL connection >>> W0305 16:55:33.626118 49 sched.cpp:379] Authentication timed out >>> I0305 16:55:33.626292 49 sched.cpp:339] Failed to authenticate with >>> master [email protected]:5050: Authentication discarded >>> I0305 16:55:33.626947 49 sched.cpp:284] Authenticating with master >>> [email protected]:5050 >>> I0305 16:55:33.627034 49 authenticatee.hpp:133] Creating new client >>> SASL connection >>> W0305 16:55:38.631783 46 sched.cpp:379] Authentication timed out >>> I0305 16:55:38.631957 46 sched.cpp:339] Failed to authenticate with >>> master [email protected]:5050: Authentication discarded >>> I0305 16:55:38.632975 46 sched.cpp:284] Authenticating with master >>> [email protected]:5050 >>> I0305 16:55:38.633229 46 authenticatee.hpp:133] Creating new client >>> SASL connection >>> >>> Thanks, >>> ./Siva >>> >>> On Fri, Mar 6, 2015 at 12:08 PM, Adam Bordelon <[email protected]> >>> wrote: >>> >>>> What parameters are you setting on each of the masters and framework? >>>> You'll need --credentials on the master and >>>> --mesos_authentication_principal and --mesos_authentication_secret_file on >>>> the (Marathon) framework node, and of course you'll need the credentials >>>> file on the master and a matching secret file on the framework node. >>>> >>>> What exactly is the error message you see? >>>> >>>> On Thu, Mar 5, 2015 at 10:14 PM, Sivaram Kannan <[email protected]> >>>> wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>> I have a 3 node mesos-master cluster. Say if node-1 is elected as >>>>> mesos-master and if I run a framework(Marathon) in the same node, >>>>> authentication succeeds. But if I run the framework from the node-2, >>>>> authentication for the framework fails. The machines are all similarly >>>>> configured in terms of hardware, same OS - everything is same except for >>>>> the ip between hosts. >>>>> >>>>> Any idea why is this?? >>>>> >>>>> Thanks, >>>>> ./Siva. >>>>> >>>>> >>>> >>> >>> >>> -- >>> ever tried. ever failed. no matter. >>> try again. fail again. fail better. >>> -- Samuel Beckett >>> >> >> >> >> -- >> ever tried. ever failed. no matter. >> try again. fail again. fail better. >> -- Samuel Beckett >> > > -- ever tried. ever failed. no matter. try again. fail again. fail better. -- Samuel Beckett
