There are currently three components of Mesos that use Zookeeper: *Master Detector:* This object is used by the Mesos Master, Agent, and Scheduler to find which Master is the leader. The existing detector code will parse a "zk://" URL if given here: https://github.com/apache/mesos/blob/1.7.x/src/master/detector/detector.cpp#L62
Not including tests, there are four call sites which pass in a ZK URL to the detector: - Master: https://github.com/apache/mesos/blob/1.7.x/src/master/main.cpp#L430-L433 - Agent: https://github.com/apache/mesos/blob/1.7.x/src/slave/main.cpp#L487-L490 - Scheduler: https://github.com/apache/mesos/blob/1.7.x/src/sched/sched.cpp#L152 - (Deprecated) CLI helper binary: https://github.com/apache/mesos/blob/1.7.x/src/cli/resolve.cpp#L95-L96 *Master Contender:* This object is used by the Mesos Master to contend for leadership of the cluster. The contender will parse a ZK URL just like the detector: https://github.com/apache/mesos/blob/1.7.x/src/master/contender/contender.cpp#L53 Unlike the detector, there is only a single call site for the contender: https://github.com/apache/mesos/blob/1.7.x/src/master/main.cpp#L418-L421 *Replicated Log Library:* This is a library which is used by the Mesos Master and some custom frameworks, to persist data via the Paxos algorithm. The Master's call site is straightforward: https://github.com/apache/mesos/blob/1.7.x/src/master/main.cpp#L383-L391 The library is built into a JAR for use by java frameworks, so there are two references in this JNI code: https://github.com/apache/mesos/blob/1.7.x/src/java/jni/org_apache_mesos_Log.cpp#L673 https://github.com/apache/mesos/blob/1.7.x/src/java/jni/org_apache_mesos_state_LogState.cpp#L75 Some other files that you will likely need to modify include: - The zookeeper::Authentication class: https://github.com/apache/mesos/blob/1.7.x/include/mesos/zookeeper/authentication.hpp This will need to be extended to allow non-digest schemes. It will currently exit if a non-digest scheme is passed in the URL. - The zookeeper::URL class: https://github.com/apache/mesos/blob/1.7.x/include/mesos/zookeeper/url.hpp Depending on how flexible the authentication schemes are, you may need to update the URL parsing logic, or scrap the URL altogether if there are authentication schemes that cannot be encoded in a URL. - The "--zk" flag for the Master: https://github.com/apache/mesos/blob/1.7.x/src/master/flags.cpp#L666-L673 You may need to update the documentation of this flag, or perhaps add new flags. - The "--master" flag for the Agent: https://github.com/apache/mesos/blob/1.7.x/src/slave/flags.cpp#L1421-L1427 This will look similar to the "--zk" Master flag, but it also supports non-ZK masters. Hopefully this list of code locations will give you some idea of where to start. Feel free to ping us in Slack too. On Fri, Dec 7, 2018 at 6:01 AM Kishchukov, Dmitrii (NIH/NLM/NCBI) [C] < dmitrii.kishchu...@nih.gov> wrote: > Yes. I want to do it. And it would be good if someone could give an advise > how to do it. For example is there one place where Authentication object > constructed for Zookeeper? > For me it looks like there many places which is strange. > > -- > > Dmitrii Kishchukov. > Leading software developer > Submission Portal Team > > > On 12/6/18, 12:56 PM, "Vinod Kone" <vinodk...@apache.org> wrote: > > Dmitrii. > > That approach sounds reasonable. Would you like to work on this? Are > you > looking for a reviewer/shepherd? > > On Thu, Dec 6, 2018 at 11:28 AM Kishchukov, Dmitrii (NIH/NLM/NCBI) [C] > < > dmitrii.kishchu...@nih.gov> wrote: > > > Mesos allow using only digest authentication scheme for Zookeeper. > Which > > is bad because Zookeeper has quite a flexible security model. > > It is easy to make you own authenticator with its own scheme name. > > > > To support fully Zookeeper authentication, Mesos has pass two items > into > > Zookeeper: > > scheme and credentials. > > credentials can have different format depending on authentication > scheme. > > For digest scheme it is ‘login:password’ > > > > All Mesos should do just pass scheme and credentials to Zookeeper. > > > > Another improvement might be be to configure credentials via file > instead > > of URI > > > > For example it can be two command line options: > > --zk_auth_scheme and –zk_auth_credentials > > > > It can be used like this: > > --zk_auth_scheme=some_custome_scheme –zk_auth_credentials=filename > > > > --zk_auth_credentials can just get all contents of the file as > credentials > > string. > > > > Class Authentication in Mesos already contains all that we need. The > > problem is what Mesos pass to the constructor. > > > > > > -- > > > > Dmitrii Kishchukov. > > > > > > >
