Re: [AREA1 SUSPICIOUS] [OFFER] Remove ZooKeeper as hard-dependency, support etcd, Consul, OR ZooKeeper

[Samuel Marks]

Dear Benjamin Mahler [and *Developers mailing-list for Apache Mesos*],

Thanks for responding so quickly.

Actually this entire project I invested—time & money, including a
development team—explicitly in order to contribute this to Apache Mesos. So
no releases yet, because I wanted to ensure it was up to the specification
requirements referenced in dev@mesos.apache.org before proceeding with
packaging and releases.

Tests have been setup in Travis CI for Linux (Ubuntu 18.04) and macOS,
happy to set them up elsewhere also. There are also some Windows builds
that need a bit of tweaking, then they will be pushed into CI also. We are
just starting to do some work on reducing build & test times.

Would be great to build a checklist of things you want to see before we
send the PR, e.g.,

   - ☐ hosted docs;
   - ☐ CI/CD—including packaging—for Windows, Linux, and macOS;
   - ☐ releases on GitHub;
   - ☐ consistent session and auth interface
   - ☐ different tests [can you expand here?]

This is just an example checklist, would be best if you and others can
flesh it out, so when we do send the PR it's in an immediately mergable
state.

BTW: Originally had a debate with my team about whether to send a PR out of
the blue—like Microsoft famously did for Node.js
—or start an *offer thread* on
the developers mailing-list.

Looking forward to contributing 呂

*SAMUEL MARKS*
Sydney Medical School | Westmead Institute for Medical Research |
https://linkedin.com/in/samuelmarks
Director | Sydney Scientific Foundation Ltd 
| Offscale.io of Sydney Scientific Pty Ltd 


On Wed, Apr 15, 2020 at 2:38 AM Benjamin Mahler  wrote:

> Thanks for reaching out, a well maintained and well written wrapper
> interface to the three backends would certainly make this easier for us vs
> implementing such an interface ourselves.
>
> Is this the client interface?
>
> https://github.com/offscale/liboffkv/blob/d31181a1e74c5faa0b7f5d7001879640b4d9f111/liboffkv/client.hpp#L115-L142
>
> At a quick glance, three ZK things that we rely on but seem to be absent
> from the common interface is the ZK session, authentication, and
> authorization. How will these be provided via the common interface?
>
> Here is our ZK interface wrapper if you want to see what kinds of things we
> use:
>
> https://github.com/apache/mesos/blob/1.9.0/include/mesos/zookeeper/zookeeper.hpp#L72-L339
>
> The project has 0 releases and 0 issues, what kind of usage has it seen?
> Has there been any testing yet? Would Offscale.io be doing some of the
> testing?
>
> On Mon, Apr 13, 2020 at 7:54 PM Samuel Marks  wrote:
>
> > Apache ZooKeeper  is a large dependency.
> > Enabling developers and operations to use etcd , Consul
> > , or ZooKeeper should reduce resource utilisation and
> > enable new use cases.
> >
> > There have already been a number of suggestions to get rid of hard
> > dependency on ZooKeeper. For example, see: MESOS-1806
> > , MESOS-3574
> > , MESOS-3797
> > , MESOS-5828
> > , MESOS-5829
> > . However, there are
> > difficulties in supporting a few implementations for different services
> > with quite distinct data models.
> >
> > A few months ago offscale.io invested in a solution to this problem -
> > liboffkv  – a *C++* library which
> > provides a *uniform interface over ZooKeeper, Consul KV and etcd*. It
> > abstracts common features of these services into its own data model which
> > is very similar to ZooKeeper’s one. Careful attention was paid to keep
> > methods both efficient and consistent. It is cross-platform,
> > open-source (*Apache-2.0
> > OR MIT*), and is written in C++, with vcpkg packaging, *C library output
> > <
> https://github.com/offscale/liboffkv/blob/d3d549e/CMakeLists.txt#L29-L35
> > >*,
> > and additional interfaces in *Go  >*,
> > *Java
> > *, and *Rust
> > *.
> >
> > Offscale.io proposes to replace all ZooKeeper usages in Mesos with usages
> > of liboffkv. Since all interactions which require ZooKeeper in Mesos are
> > conducted through the class Group (and GroupProcess) with a clear
> interface
> > the obvious way to introduce changes is to provide another implementation
> > of the class which uses liboffkv instead of ZooKeeper. In this case the
> > original implementation may be left unchanged in the codebase and build
> > flags to select from ZK-only and liboffkv variants may be introduced.
> Once
> > the community is confident, you can decide to remove the ZK-only option,
> > 

Re: [AREA1 SUSPICIOUS] [OFFER] Remove ZooKeeper as hard-dependency, support etcd, Consul, OR ZooKeeper

[Benjamin Mahler]

Thanks for reaching out, a well maintained and well written wrapper
interface to the three backends would certainly make this easier for us vs
implementing such an interface ourselves.

Is this the client interface?
https://github.com/offscale/liboffkv/blob/d31181a1e74c5faa0b7f5d7001879640b4d9f111/liboffkv/client.hpp#L115-L142

At a quick glance, three ZK things that we rely on but seem to be absent
from the common interface is the ZK session, authentication, and
authorization. How will these be provided via the common interface?

Here is our ZK interface wrapper if you want to see what kinds of things we
use:
https://github.com/apache/mesos/blob/1.9.0/include/mesos/zookeeper/zookeeper.hpp#L72-L339

The project has 0 releases and 0 issues, what kind of usage has it seen?
Has there been any testing yet? Would Offscale.io be doing some of the
testing?

On Mon, Apr 13, 2020 at 7:54 PM Samuel Marks  wrote:

> Apache ZooKeeper  is a large dependency.
> Enabling developers and operations to use etcd , Consul
> , or ZooKeeper should reduce resource utilisation and
> enable new use cases.
>
> There have already been a number of suggestions to get rid of hard
> dependency on ZooKeeper. For example, see: MESOS-1806
> , MESOS-3574
> , MESOS-3797
> , MESOS-5828
> , MESOS-5829
> . However, there are
> difficulties in supporting a few implementations for different services
> with quite distinct data models.
>
> A few months ago offscale.io invested in a solution to this problem -
> liboffkv  – a *C++* library which
> provides a *uniform interface over ZooKeeper, Consul KV and etcd*. It
> abstracts common features of these services into its own data model which
> is very similar to ZooKeeper’s one. Careful attention was paid to keep
> methods both efficient and consistent. It is cross-platform,
> open-source (*Apache-2.0
> OR MIT*), and is written in C++, with vcpkg packaging, *C library output
>  >*,
> and additional interfaces in *Go *,
> *Java
> *, and *Rust
> *.
>
> Offscale.io proposes to replace all ZooKeeper usages in Mesos with usages
> of liboffkv. Since all interactions which require ZooKeeper in Mesos are
> conducted through the class Group (and GroupProcess) with a clear interface
> the obvious way to introduce changes is to provide another implementation
> of the class which uses liboffkv instead of ZooKeeper. In this case the
> original implementation may be left unchanged in the codebase and build
> flags to select from ZK-only and liboffkv variants may be introduced. Once
> the community is confident, you can decide to remove the ZK-only option,
> and instead only support liboffkv [which internally has build flags for
> each service].
>
> Removing the hard dependency on ZooKeeper will simplify local deployment
> for testing purposes as well as enable using Mesos in clusters without
> ZooKeeper, e.g. where etcd or Consul is used for coordination. We expect
> this to greatly reduce the amount of resource—network, CPU, disk,
> memory—usage in a datacenter environment.
>
> If the community accepts the initiative, we will integrate liboffkv into
> Mesos. We are also ready to develop the library and consider any suggested
> improvements.
> *SAMUEL MARKS*
> Sydney Medical School | Westmead Institute for Medical Research |
> https://linkedin.com/in/samuelmarks
> Director | Sydney Scientific Foundation Ltd 
> | Offscale.io of Sydney Scientific Pty Ltd 
> *SYDNEY SCIENTIFIC FOUNDATION and THE UNIVERSITY OF SYDNEY*
>
> PS: We will be offering similar contributions to Chronos
> , Marathon
> , Aurora
> , and related projects.
>