Does it have to be a single project with functionality provided by multiple plugins? Designing a plugin API at this point seems to be a bit early and comes with additional complexity around managing plugins in general.
I was more thinking into the direction of: "what can we do to enable people to create any kind of side car or tooling solution?". Thinks like: Common cluster discovery and management API * Detect local Cassandra processes * Discover and receive events on cluster topology * Get assigned tokens for nodes * Read node configuration * Health checks (as already proposed) Any side cars should be easy to install on nodes that already run Cassandra * Scripts for packaging (tar, deb, rpm) * Templates for systemd support, optionally with auto-startup dependency on the Cassandra main process Integration testing * Provide basic testing framework for mocking cluster state and messages Support for other languages / avoid having to use JMX * JMX bridge (HTTP? gRPC?, already implemented in #14346?) Obviously the whole side car discussion is not moving into a direction everyone's happy with. Would it be an option to take a step back and start implementing such a tooling framework with scripts and libraries for the features described above, as a small GitHub project, instead of putting an existing side-car solution up for vote? If that would work and we get people collaborating on code shared between existing side-cars, then we could take the next step and think about either revisit the "official Cassandra side-car" topic, or add the created client tooling framework as official sub-project to the Cassandra project (maybe via Apache incubator). On 08.09.18 02:49, Joseph Lynch wrote: > On Fri, Sep 7, 2018 at 5:03 PM Jonathan Haddad <j...@jonhaddad.com> wrote: >> We haven’t even defined any requirements for an admin tool. It’s hard to >> make a case for anything without agreement on what we’re trying to build. >> > We were/are trying to sketch out scope/requirements in the #14395 and > #14346 tickets as well as their associated design documents. I think > the general proposed direction is a distributed 1:1 management sidecar > process similar in architecture to Netflix's Priam except explicitly > built to be general and pluggable by anyone rather than tightly > coupled to AWS. > > Dinesh, Vinay and I were aiming for low amounts of scope at first and > take things in an iterative approach with just enough upfront design > but not so much we are unable to make any progress at all. For example > maybe something like: > > 1. Get a super simple and non controversial sidecar process that ships > with Cassandra and exposes a lightweight HTTP interface to e.g. some > basic JMX endpoints > 2a. Add a pluggable execution engine for cron/oneshot/scheduled jobs > with the basic interfaces and state store and such > 2b. Start scoping and implementing the full HTTP interface, e.g. > backup status, cluster health status, etc ... > 3a. Start integrating implementations of the jobs from 2a such as > snapshot, backup, cluster restart, daemon + sstable upgrade, repair, > etc > 3b. Start integrating UI components that pair with the HTTP interface from 2b > 4. ?? Perhaps start unlocking next generation operations like moving > "background" activities like compaction, streaming, repair etc into > one or more sidecar contained processes to ensure the main daemon only > handles read+write requests > > There are going to be a lot of questions to answer, and I think trying > to answer them all up front will mean that we get nowhere or make > unfortunate compromises that cripple the project from the start. If > people think we need to do more design and discussion than we have > been doing then we can spend more time on the design, but personally > I'd rather start iterating on code and prove value incrementally. If > it doesn't work out we won't release it GA to the community ... > > -Joey > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
