Following on from the discussion on GitHub and here on the mailing-list,
here is the proposal from me and my team:
------------------------------
Choice of approach
The “mediator” of every interaction with ZooKeeper in Mesos is the ZooKeeper
class, declared in include/mesos/zookeeper/zookeeper.hpp.
Of note are the following two differences in the *styles* of API provided
by ZooKeeper class and liboffkv:
-
Push-style mechanism of notifications on changes in “watched” data,
versus pull-style one in liboffkv. In Mesos, the notifications are
delivered via the Watcher interface, defined in the same file as
ZooKeeper. This interface has the process method, which is invoked by an
instance of ZooKeeper at most once for each watch. There is also a
special event which informs the watcher that the connection has been
dropped. An optional instance of Watcher is passed to the constructor of
ZooKeeper.
-
Asynchronous session establishment process in ZooKeeper versus
synchronous one (if at all — e.g. for Consul there is no concept of
“session” currently defined at all) in liboffkv.
The two users of the ZooKeeper are:
1.
GroupProcess;
2.
ZooKeeperStorageProcess.
We will thus evaluate the possible approaches of integrating liboffkv into
Mesos through the prism of details of their usage.
The two possible approaches are:
1.
Replace all usages of ZooKeeper with liboffkv-specific code under #ifdef
guards.
This approach would scale badly, as alternative liboffkv-specific
implementations will be needed for both of the users.
Moreover, we think that conditional compilation results in maintenance
nightmare; see, e.g.:
-
RealWaitForChar() in vim <https://geoff.greer.fm/vim/>;
-
“#ifdef Considered Harmful, or Portability Experience With C News”
paper by Henry Spencer and Geoff Collyer
<http://doc.cat-v.org/henry_spencer/ifdef_considered_harmful.pdf>.
The creators of the C programming language, which introduced the concept
in the first place, have also spoken against conditional compilation:
-
In “The Practice of Programming” by Brian W. Kernighan and Rob Pike,
the following advice is given: “Avoid conditional compilation.
Conditional
compilation with #ifdef and similar preprocessor directives is hard
to manage, because information tends to get sprinkled throughout the
source.”
-
In “Plan 9 from Bell Labs” paper by Rob Pike, Ken Thompson et al.
<https://pdos.csail.mit.edu/archive/6.824-2012/papers/plan9.pdf>, the
following is said: “Conditional compilation, even with #ifdef, is
used sparingly in Plan 9. The only architecture-dependent #ifdefs in
the system are in low-level routines in the graphics library. Instead, we
avoid such dependencies or, when necessary, isolate them in
separate source
files or libraries. Besides making code hard to read, #ifdefs make it
impossible to know what source is compiled into the binary or whether
source protected by them will compile or work properly. They
make it harder
to maintain software.”
2.
Modify the *implementation* of the ZooKeeper class to use liboffkv,
possibly renaming the class to something akin to KvClient to reflect the
fact that would no longer be ZooKeeper-specific (this also includes the
renaming of error codes and other similar nomenclature). The old version of
the implementation would be put under an #ifdef guard, thus minimising
the number — and maintenance impact — of #ifdefs.
Naturally there are some advantages to taking the ifdef approach, namely
that we can guarantee no difference in builds between before offscale's
contribution and after, unless a compiler flag is provided.
However to avoid polluting the code, we are recommending the second
approach.
Incompatibilities
The following is the list of incompatibilities between the interfaces of
ZooKeeper class and liboffkv. Some of those features should be implemented
in liboffkv; others should be emulated inside the ZooKeeper/KvClient class;
and for others still, the change of the interface of ZooKeeper/KvClient is
the preferred solution.
-
Asynchronous session establishment. We propose to emulate this through
spawning a new thread in the constructor of ZooKeeper/KvClient.
-
Push-style watch notification API. We propose to emulate this through
spawning a new thread for each watch; such a thread would then do the wait
and then invoke watcher->process() under a mutex. The number of threads
should not be a concern here, as the only user that uses watches at all (
GroupProcess) only registers at most one watch.
-
Multiple servers in URL string. We propose to implement this in liboffkv.
-
Authentication. We propose to implement this in liboffkv.
-
ACLs (access control lists). The following ACLs are in fact used for
everything:
_auth.isSome()
? zookeeper::EVERYONE_READ_CREATOR_ALL
: ZOO_OPEN_ACL_UNSAFE
We thus propose to:
1.
implement rudimentary support for ACLs in liboffkv in the form of an
optional parameter to create(),
bool protect_modify = false
2.
change the interface of ZooKeeper/KvClient so that protect_modify
flag is used instead of ACLs.
-
Configurable session timeout. We propose to implement this in liboffkv.
-
Getting the actual session timeout, which might differ from the
user-provided as a result of timeout negotiation with server. We propose to
implement this in liboffkv.
-
Getting the session ID. We propose to implement this in liboffkv, with
session ID being std::string; and to modify the interface accordingly.
It is possible to hash a string into a 64-bit number, but in the
circumstances given, we think it is just not worth it.
-
Getting the status of the connection to the server. We propose to
implement this in liboffkv.
-
Sequenced nodes. We propose to emulate this in the class. Here is the
pseudo-code of our solution:
while (true) {
[counter, version] = get("/counter")
seqnum = counter + 1
name = "label" + seqnum
try {
commit {
check "/counter" version,
set "/counter" seqnum,
create name value
}
break
} catch (TxnAborted) {}
}
-
“Recursive” creation of each parent in create(), akin to mkdir -p. This
is already emulated in the class, as ZooKeeper does not natively support
it; we propose to extend this emulation to work with liboffkv.
-
The semantics of the “set” operation if the entry does not exist:
ZooKeeper fails with ZNONODE in this case, while liboffkv creates a new
node. We propose to emulate this in-class with a transaction.
-
The semantics of the “erase” operation: ZooKeeper fails with ZNOTEMPTY
if node has children, while liboffkv removes the subtree recursively. As
neither of users ever attempts to remove node with children, we propose to
change the interface so that it declares (and actually implements) the
liboffkv-compatible semantics.
-
Return of ZooKeeper-specific Stat structures instead of just versions.
As both users only use the version field of this structure, we propose to
simply alter the interface so that only the version is returned.
-
Explicit “session drop” operation that also immediately erases all the
“leased” nodes. We propose to implement this in liboffkv.
-
Check if the node being created has leased parent. Currently, liboffkv
declares this to be unspecified behavior: it may either throw (if ZooKeeper
is used as the back-end) or successfully create the node (otherwise). As
neither of users ever attempts to create such a node, we propose to leave
this as is.
Estimates
We estimate that—including tests—this will be ready by the end of next
month.
------------------------------
Open to alternative suggestions, otherwise we'll begin.
Samuel Marks
Charity <https://sydneyscientific.org> | consultancy <https://offscale.io>
| open-source <https://github.com/offscale> | LinkedIn
<https://linkedin.com/in/samuelmarks>
On Sat, May 2, 2020 at 4:04 AM Benjamin Mahler <bmah...@apache.org> wrote:
> So it sounds like:
>
> Zookeeper: Official C library has an async API. Are we gaining a lot with
> the third party C++ wrapper you pointed to? Maybe it "just works", but it
> looks very inactive and it's hard to tell how maintained it is.
>
> Consul: No official C or C++ library. Only some third party C++ ones that
> look pretty inactive. The ppconsul one you linked to does have an issue
> about an async API, I commented on it:
> https://github.com/oliora/ppconsul/issues/26.
>
> etcd: Can use gRPC c++ client async API.
>
> Since 2 of 3 provide an async API already, I would lean more towards an
> async API so that we don't have to change anything with the mesos code when
> the last one gets an async implementation. However, we currently use the
> synchronous ZK API so I realize this would be more work to first adjust the
> mesos code to use the async zookeeper API. I agree that a synchronous
> interface is simpler to start with since that will be an easier integration
> and we currently do not perform many concurrent operations (and probably
> won't anytime soon).
>
> On Sun, Apr 26, 2020 at 11:17 PM Samuel Marks <sam...@offscale.io> wrote:
>
> > In terms of asynchronous vs synchronous interfacing, when we started
> > liboffkv, it had an asynchronous interface. Then we decided to drop it
> and
> > implemented a synchronous one, due to the dependent libraries which
> > liboffkv uses under the hood.
> >
> > Our ZooKeeper implementation uses the zookeeper-cpp library
> > <https://github.com/tgockel/zookeeper-cpp>—a well-maintained C++ wrapper
> > around common Zookeeper C bindings [which we contributed to vcpkg
> > <https://github.com/microsoft/vcpkg/pull/7001>]. It has an asynchronous
> > interface based on std::future
> > <https://en.cppreference.com/w/cpp/thread/future>. Since std::future
> does
> > not provide chaining or any callbacks, a Zookeeper-specific result cannot
> > be asynchronously mapped to liboffkv result. In early versions of
> liboffkv
> > we used thread pool to do the mapping.
> >
> > Consul implementation is based on the ppconsul
> > <https://github.com/oliora/ppconsul> library [which we contributed to
> > vcpkg
> > <
> >
> https://github.com/microsoft/vcpkg/pulls?q=is%3Apr+author%3ASamuelMarks+ppconsul
> > >],
> > which in turn utilizes libcurl <https://curl.haxx.se/libcurl>.
> > Unfortunately, ppconsul uses libcurl's easy interface, and consequently
> it
> > is synchronous by design. Again, in the early version of the library we
> > used a thread pool to overcome this limitation.
> >
> > As for etcd, we autogenerated the gRPC C++ client
> > <https://github.com/offscale/etcd-client-cpp> [which we contributed to
> > vcpkg
> > <https://github.com/microsoft/vcpkg/pull/6999>]. gRPC provides an
> > asynchronous interface, so a "fair" async client can be implemented on
> top
> > of it.
> >
> > To sum up, the chosen toolkit provided two of three implementations
> require
> > thread pool. After careful consideration, we have preferred to give the
> > user control over threading and opted out of the asynchrony.
> >
> > Nevertheless, there are some options. zookeeper-cpp allows building with
> > custom futures/promises, so we can create a custom build to use in
> > liboffkv/Mesos. Another variant is to use plain C ZK bindings
> > <
> >
> https://gitbox.apache.org/repos/asf?p=zookeeper.git;a=tree;f=zookeeper-client/zookeeper-client-c;h=c72b57355c977366edfe11304067ff35f5cf215d;hb=HEAD
> > >
> > instead of the C++ library.
> > As for the Consul client, the only meaningful option is to opt out of
> using
> > ppconsul and operate through libcurl's multi interface.
> >
> > At this point implementing asynchronous interfaces will require rewriting
> > liboffkv from the ground up. I can allocate the budget for doing this,
> as I
> > have done to date. However, it would be good to have some more
> > back-and-forth before reengaging.
> >
> > Design Doc:
> >
> >
> https://docs.google.com/document/d/1NOfyt7NzpMxxatdFs3f9ixKUS81DHHDVEKBbtVfVi_0
> > [feel free to add it to
> > http://mesos.apache.org/documentation/latest/design-docs/]
> >
> > Thanks,
> >
> > *SAMUEL MARKS*
> > Sydney Medical School | Westmead Institute for Medical Research |
> > https://linkedin.com/in/samuelmarks
> > Director | Sydney Scientific Foundation Ltd <
> https://sydneyscientific.org>
> > | Offscale.io of Sydney Scientific Pty Ltd <https://offscale.io>
> >
> > PS: Damien - not against contributing to FoundationDB, but priorities are
> > Mesos and the Mesos ecosystem, followed by Kuberentes and its ecosystem.
> >
> > On Tue, Apr 21, 2020 at 3:19 AM Benjamin Mahler <bmah...@apache.org>
> > wrote:
> >
> > > Samuel: One more thing I forgot to mention, we would prefer to use an
> > > asynchronous client interface rather than a synchronous one. Is that
> > > something you have considered?
> > >
> > > On Fri, Apr 17, 2020 at 6:11 PM Vinod Kone <vinodk...@apache.org>
> wrote:
> > >
> > > > Hi Samuel,
> > > >
> > > > Thanks for showing interest in contributing to the project. Having
> > > > optionality between ZooKeeper and Etcd would be great for the project
> > and
> > > > something that has been brought up a few times before, as you noted.
> > > >
> > > > I echo everything that BenM said. As part of the design it would be
> > great
> > > > to see the migration path for users currently using Mesos with
> > ZooKeeper
> > > to
> > > > Etcd. Ideally, the migration can happen without much user
> intervention.
> > > >
> > > > Additionally, from our past experience, efforts like these are more
> > > > successful if the people writing the code have experience with how
> > things
> > > > work in Mesos code base. So I would recommend starting small, maybe
> > have
> > > a
> > > > few engineers work on a couple "newbie" tickets and do some small
> > > projects
> > > > and have those committed to the project. That gives the committers
> some
> > > > level of confidence about quality of the code and be more open to
> > bigger
> > > > changes like etcd integration. It would also help contributors get a
> > > better
> > > > feeling for the lay of the land and see if they are truly interested
> in
> > > > maintaining this piece of integration for the long haul. This is a
> bit
> > > of a
> > > > longer path but I think it would be more a fruitful one.
> > > >
> > > > Looking forward to seeing new contributions to Mesos including the
> > above
> > > > design!
> > > >
> > > > Thanks,
> > > >
> > > > On Fri, Apr 17, 2020 at 4:52 PM Samuel Marks <sam...@offscale.io>
> > wrote:
> > > >
> > > > > Happy to build a design doc,
> > > > >
> > > > > To answer your question on what Offscale.io is, it's my software
> and
> > > > > biomedical engineering consultancy. Currently it's still rather
> > small,
> > > > with
> > > > > only 8 engineers, but I'm expecting & preparing to grow rapidly.
> > > > >
> > > > > My philosophy is always open-source and patent-free, so that's what
> > my
> > > > > consultancy—and for that matter, the charitable research that I
> fund
> > > > > through it <https://sydneyscientific.org>—follows.
> > > > >
> > > > > The goal of everything we create is: interoperable (cross-platform,
> > > > > cross-technology, cross-language, multi-cloud); open-source
> > (Apache-2.0
> > > > OR
> > > > > MIT); with a view towards scaling:
> > > > >
> > > > > - teams;
> > > > > - software-development <https://compilers.com.au>;
> > > > > - infrastructure [this proposed Mesos contribution + our DevOps
> > > > > tooling];
> > > > > - [in the charity's case] facilitating very large-scale medical
> > > > > diagnostic screening.
> > > > >
> > > > > Technologies like Mesos we expect to both optimise resource
> > > > > allocation—reducing costs and increasing data locality—and award us
> > > > > 'bragging rights' with which we can gain clients that are already
> > using
> > > > > Mesos (which, from my experience, is always big corporates… though
> > > > > hopefully contributions like these will make it attractive to small
> > > > > companies also).
> > > > >
> > > > > So no, we're not going anywhere, and are planning to maintain this
> > > > library
> > > > > into the future
> > > > >
> > > > > PS: Once accepted by Mesos, we'll be making similar contributions
> to
> > > > other
> > > > > Mesos ecosystem projects like Chronos <
> > https://mesos.github.io/chronos
> > > >,
> > > > > Marathon <https://github.com/mesosphere/marathon>, and Aurora
> > > > > <https://github.com/aurora-scheduler/aurora> as well as to
> unrelated
> > > > > projects (e.g., removing etcd as a hard-dependency from Kubernetes
> > > > > <https://kubernetes.io>… enabling them to choose between
> ZooKeeper,
> > > > etcd,
> > > > > and Consul).
> > > > >
> > > > > Thanks for your continual feedback,
> > > > >
> > > > > *SAMUEL MARKS*
> > > > > Sydney Medical School | Westmead Institute for Medical Research |
> > > > > https://linkedin.com/in/samuelmarks
> > > > > Director | Sydney Scientific Foundation Ltd <
> > > > https://sydneyscientific.org>
> > > > > | Offscale.io of Sydney Scientific Pty Ltd <https://offscale.io>
> > > > >
> > > > >
> > > > > On Sat, Apr 18, 2020 at 6:58 AM Benjamin Mahler <
> bmah...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Oh ok, could you tell us a little more about how you're using
> > Mesos?
> > > > And
> > > > > > what offscale.io is?
> > > > > >
> > > > > > Strictly speaking, we don't really need packaging and releases as
> > we
> > > > can
> > > > > > bundle the dependency in our repo and that's what we do for many
> of
> > > our
> > > > > > dependencies.
> > > > > > To me, the most important thing is the commitment to maintain the
> > > > library
> > > > > > and address issues that come up.
> > > > > > I also would lean more towards a run-time flag rather than a
> build
> > > > level
> > > > > > flag, if possible.
> > > > > >
> > > > > > I think the best place to start would be to put together a design
> > > doc.
> > > > > The
> > > > > > act of writing that will force the author to think through the
> > > details
> > > > > (and
> > > > > > there are a lot of them!), and we'll then get a chance to give
> > > > feedback.
> > > > > > You can look through the mailing list for past examples of design
> > > docs
> > > > > (in
> > > > > > terms of which sections to include, etc).
> > > > > >
> > > > > > How does that sound?
> > > > > >
> > > > > > On Tue, Apr 14, 2020 at 8:44 PM Samuel Marks <sam...@offscale.io
> >
> > > > wrote:
> > > > > >
> > > > > > > Dear Benjamin Mahler [and *Developers mailing-list for Apache
> > > > Mesos*],
> > > > > > >
> > > > > > > Thanks for responding so quickly.
> > > > > > >
> > > > > > > Actually this entire project I invested—time & money,
> including a
> > > > > > > development team—explicitly in order to contribute this to
> Apache
> > > > > Mesos.
> > > > > > So
> > > > > > > no releases yet, because I wanted to ensure it was up to the
> > > > > > specification
> > > > > > > requirements referenced in dev@mesos.apache.org before
> > proceeding
> > > > with
> > > > > > > packaging and releases.
> > > > > > >
> > > > > > > Tests have been setup in Travis CI for Linux (Ubuntu 18.04) and
> > > > macOS,
> > > > > > > happy to set them up elsewhere also. There are also some
> Windows
> > > > builds
> > > > > > > that need a bit of tweaking, then they will be pushed into CI
> > also.
> > > > We
> > > > > > are
> > > > > > > just starting to do some work on reducing build & test times.
> > > > > > >
> > > > > > > Would be great to build a checklist of things you want to see
> > > before
> > > > we
> > > > > > > send the PR, e.g.,
> > > > > > >
> > > > > > > - ☐ hosted docs;
> > > > > > > - ☐ CI/CD—including packaging—for Windows, Linux, and macOS;
> > > > > > > - ☐ releases on GitHub;
> > > > > > > - ☐ consistent session and auth interface
> > > > > > > - ☐ different tests [can you expand here?]
> > > > > > >
> > > > > > > This is just an example checklist, would be best if you and
> > others
> > > > can
> > > > > > > flesh it out, so when we do send the PR it's in an immediately
> > > > mergable
> > > > > > > state.
> > > > > > >
> > > > > > > BTW: Originally had a debate with my team about whether to
> send a
> > > PR
> > > > > out
> > > > > > of
> > > > > > > the blue—like Microsoft famously did for Node.js
> > > > > > > <https://github.com/nodejs/node/pull/4765>—or start an *offer
> > > > thread*
> > > > > on
> > > > > > > the developers mailing-list.
> > > > > > >
> > > > > > > Looking forward to contributing 🦀
> > > > > > >
> > > > > > > *SAMUEL MARKS*
> > > > > > > Sydney Medical School | Westmead Institute for Medical
> Research |
> > > > > > > https://linkedin.com/in/samuelmarks
> > > > > > > Director | Sydney Scientific Foundation Ltd <
> > > > > > https://sydneyscientific.org>
> > > > > > > | Offscale.io of Sydney Scientific Pty Ltd <
> https://offscale.io>
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Apr 15, 2020 at 2:38 AM Benjamin Mahler <
> > > bmah...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks for reaching out, a well maintained and well written
> > > wrapper
> > > > > > > > interface to the three backends would certainly make this
> > easier
> > > > for
> > > > > us
> > > > > > > vs
> > > > > > > > implementing such an interface ourselves.
> > > > > > > >
> > > > > > > > Is this the client interface?
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/offscale/liboffkv/blob/d31181a1e74c5faa0b7f5d7001879640b4d9f111/liboffkv/client.hpp#L115-L142
> > > > > > > >
> > > > > > > > At a quick glance, three ZK things that we rely on but seem
> to
> > be
> > > > > > absent
> > > > > > > > from the common interface is the ZK session, authentication,
> > and
> > > > > > > > authorization. How will these be provided via the common
> > > interface?
> > > > > > > >
> > > > > > > > Here is our ZK interface wrapper if you want to see what
> kinds
> > of
> > > > > > things
> > > > > > > we
> > > > > > > > use:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/mesos/blob/1.9.0/include/mesos/zookeeper/zookeeper.hpp#L72-L339
> > > > > > > >
> > > > > > > > The project has 0 releases and 0 issues, what kind of usage
> has
> > > it
> > > > > > seen?
> > > > > > > > Has there been any testing yet? Would Offscale.io be doing
> some
> > > of
> > > > > the
> > > > > > > > testing?
> > > > > > > >
> > > > > > > > On Mon, Apr 13, 2020 at 7:54 PM Samuel Marks <
> > sam...@offscale.io
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Apache ZooKeeper <https://zookeeper.apache.org> is a large
> > > > > > dependency.
> > > > > > > > > Enabling developers and operations to use etcd <
> > > https://etcd.io
> > > > >,
> > > > > > > Consul
> > > > > > > > > <https://consul.io>, or ZooKeeper should reduce resource
> > > > > utilisation
> > > > > > > and
> > > > > > > > > enable new use cases.
> > > > > > > > >
> > > > > > > > > There have already been a number of suggestions to get rid
> of
> > > > hard
> > > > > > > > > dependency on ZooKeeper. For example, see: MESOS-1806
> > > > > > > > > <https://issues.apache.org/jira/browse/MESOS-1806>,
> > MESOS-3574
> > > > > > > > > <https://issues.apache.org/jira/browse/MESOS-3574>,
> > MESOS-3797
> > > > > > > > > <https://issues.apache.org/jira/browse/MESOS-3797>,
> > MESOS-5828
> > > > > > > > > <https://issues.apache.org/jira/browse/MESOS-5828>,
> > MESOS-5829
> > > > > > > > > <https://issues.apache.org/jira/browse/MESOS-5829>.
> However,
> > > > there
> > > > > > are
> > > > > > > > > difficulties in supporting a few implementations for
> > different
> > > > > > services
> > > > > > > > > with quite distinct data models.
> > > > > > > > >
> > > > > > > > > A few months ago offscale.io invested in a solution to
> this
> > > > > problem
> > > > > > -
> > > > > > > > > liboffkv <https://github.com/offscale/liboffkv> – a *C++*
> > > > library
> > > > > > > which
> > > > > > > > > provides a *uniform interface over ZooKeeper, Consul KV and
> > > > etcd*.
> > > > > It
> > > > > > > > > abstracts common features of these services into its own
> data
> > > > model
> > > > > > > which
> > > > > > > > > is very similar to ZooKeeper’s one. Careful attention was
> > paid
> > > to
> > > > > > keep
> > > > > > > > > methods both efficient and consistent. It is
> cross-platform,
> > > > > > > > > open-source (*Apache-2.0
> > > > > > > > > OR MIT*), and is written in C++, with vcpkg packaging, *C
> > > library
> > > > > > > output
> > > > > > > > > <
> > > > > > > >
> > > > > >
> > > >
> > https://github.com/offscale/liboffkv/blob/d3d549e/CMakeLists.txt#L29-L35
> > > > > > > > > >*,
> > > > > > > > > and additional interfaces in *Go <
> > > > > > https://github.com/offscale?q=goffkv
> > > > > > > > >*,
> > > > > > > > > *Java
> > > > > > > > > <https://github.com/offscale/liboffkv-java>*, and *Rust
> > > > > > > > > <https://github.com/offscale/rsoffkv>*.
> > > > > > > > >
> > > > > > > > > Offscale.io proposes to replace all ZooKeeper usages in
> Mesos
> > > > with
> > > > > > > usages
> > > > > > > > > of liboffkv. Since all interactions which require ZooKeeper
> > in
> > > > > Mesos
> > > > > > > are
> > > > > > > > > conducted through the class Group (and GroupProcess) with a
> > > clear
> > > > > > > > interface
> > > > > > > > > the obvious way to introduce changes is to provide another
> > > > > > > implementation
> > > > > > > > > of the class which uses liboffkv instead of ZooKeeper. In
> > this
> > > > case
> > > > > > the
> > > > > > > > > original implementation may be left unchanged in the
> codebase
> > > and
> > > > > > build
> > > > > > > > > flags to select from ZK-only and liboffkv variants may be
> > > > > introduced.
> > > > > > > > Once
> > > > > > > > > the community is confident, you can decide to remove the
> > > ZK-only
> > > > > > > option,
> > > > > > > > > and instead only support liboffkv [which internally has
> build
> > > > flags
> > > > > > for
> > > > > > > > > each service].
> > > > > > > > >
> > > > > > > > > Removing the hard dependency on ZooKeeper will simplify
> local
> > > > > > > deployment
> > > > > > > > > for testing purposes as well as enable using Mesos in
> > clusters
> > > > > > without
> > > > > > > > > ZooKeeper, e.g. where etcd or Consul is used for
> > coordination.
> > > We
> > > > > > > expect
> > > > > > > > > this to greatly reduce the amount of resource—network, CPU,
> > > disk,
> > > > > > > > > memory—usage in a datacenter environment.
> > > > > > > > >
> > > > > > > > > If the community accepts the initiative, we will integrate
> > > > liboffkv
> > > > > > > into
> > > > > > > > > Mesos. We are also ready to develop the library and
> consider
> > > any
> > > > > > > > suggested
> > > > > > > > > improvements.
> > > > > > > > > *SAMUEL MARKS*
> > > > > > > > > Sydney Medical School | Westmead Institute for Medical
> > > Research |
> > > > > > > > > https://linkedin.com/in/samuelmarks
> > > > > > > > > Director | Sydney Scientific Foundation Ltd <
> > > > > > > > https://sydneyscientific.org>
> > > > > > > > > | Offscale.io of Sydney Scientific Pty Ltd <
> > > https://offscale.io>
> > > > > > > > > *SYDNEY SCIENTIFIC FOUNDATION and THE UNIVERSITY OF SYDNEY*
> > > > > > > > >
> > > > > > > > > PS: We will be offering similar contributions to Chronos
> > > > > > > > > <https://mesos.github.io/chronos>, Marathon
> > > > > > > > > <https://github.com/mesosphere/marathon>, Aurora
> > > > > > > > > <https://github.com/aurora-scheduler/aurora>, and related
> > > > > projects.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
