It's interesting to me that someone would consider features of Reaper as "technical debt"... an odd way to word it. When we had proposed donating Reaper to the project, I had made the assumption people would realize the benefit of folding in a successful project. A working codebase with a large user base gives us an immediate tool, and it would let us migrate everyone from using our tool to using an official tool.
>From the discussions we've had so far, nobody except Blake seems to care about those users. To me, the highest priority of this process is considering what's best for the Cassandra community. Providing an easy, clear migration path forward for the thousands of Reaper users, supporting all 4 backends and cassandra versions is the logical the way of transitioning everyone to using the official tool. If supporting everyone using 2.x with a postgres backend for storing schedules isn't something anyone cares about, there's not much benefit from using Reaper. Gutting a bunch of the project won't help people migrate over, and if we're not migrating everyone over, then we (TLP) will still have to maintain Reaper. That means we'll be indefinitely maintaining a fork of the official admin, and probably not contributing to the main one, our time is limited. That's exactly what's going to happen if we start with anything other than Reaper. We've got a lot of teams asking for features, and if people pay us to keep working on Reaper, we'll be doing that. So, with that in mind, if we're putting this to a vote, I'd like to be clear - taking Reaper as-is about community - not it's technical prowess. If we're not enabling users to move off our version and directly to the official tool, I don't see a point in using Reaper at all except as a reference to correctly run subrange repair and have a nice UI. There's plenty of things I'd do differently if I built it from scratch, I'm not putting the codebase up on a pedestal. This entire discussion has been incredibly frustrating for me, considering we're talking about building an alternative to a well tested and widely deployed codebase that likely won't produce anything of value for at least a year (Cassandra 5?). I'm also pretty sure most of the folks that have cried out "tech debt" have spent less than 5 minutes looking at the codebase. I hope your enthusiasm at the moment would carry over if you build a tech-debt free admin tool from the ground up. TL;DR: If nobody cares about the Reaper community (which is a very large subset of the cassandra community) there's no reason to start with Reaper, it's not about the tech it's about the people. Jon On Sat, Sep 8, 2018 at 4:57 PM sankalp kohli <kohlisank...@gmail.com> wrote: > Most of the discussions have been around whether we take some side car or > do a cherry pick approach where we add a feature at a time to side car and > use the best implementation. > We have been discussing this first in April and now for several days. I > think we all want some progress here. We will start a vote soon..may be > next week to decide which approach we will take. I don't see any other > option to make progress besides a vote!! > > PS: I think these discussions are very good for the community and it shows > people care about Apache Cassandra and it is a sign of healthy community. > Several people offering to donate the side car or help build is showing the > interest everyone has in it. > > > On Sat, Sep 8, 2018 at 11:17 AM Joseph Lynch <joe.e.ly...@gmail.com> > wrote: > > > On Fri, Sep 7, 2018 at 10:00 PM Blake Eggleston <beggles...@apple.com> > > wrote: > > > > > > Right, I understand the arguments for starting a new project. I’m not > > saying reaper is, technically speaking, the best place to start. The > point > > I’m trying to make is that the non-technical advantages of using an > > existing project as a starting point may outweigh the technical benefits > of > > a clean slate. Whether that’s the case or not, it’s not a strictly > > technical decision, and the non-technical advantages of starting with > > reaper need to be weighed. > > > > > > > Technical debt doesn't just refer to the technical solutions, having > > an existing user base means that a project has made promises in the > > past (in the case of Priam 5+ years ago) that the new project would > > have to keep if we make keeping users of those sidecars a goal (which > > for the record I don't think should be a goal, I think the goal is to > > make Cassandra the database work out of the box in the 4.x series). > > > > For example, Priam has to continue supporting the following as users > > actively use them (including Netflix): > > * Parallel token assignment and creation allowing parallel bootstrap > > and parallel doubling of hundred node clusters at once (as long as you > > use single tokens and run in AWS with asgs). > > * 3+ backup solutions, as well as assumptions about where in e.g. S3 > > backups are present and what format they're present in. > > * Numerous configuration options and UI elements (mostly 5 year old JSON > > APIs) > > * Support for Cassandra 2.0, 2.1, 2.2, 3.0, 3.11 and soon 4.0 > > > > Reaper has to continue supporting the following as users actively use > them: > > * Postgres and h2 storage backends. These were the original storage > > engines and users may not have (probably haven't?) migrated to the > > relatively recently added Cassandra backend (which is probably the > > only one an official sidecar should support imo). > > * The three historical modes of running Reaper [1] all of which > > involve remote JMX (disallowed by many companies security policies > > including Netflix's) and none of which are a sidecar design (although > > Mick says we can add that back in, at which point it has to support > > four). > > * Numerous configuration options and UI elements (e.g. a UI around a > > single Reaper instance controlling many Cassandra clusters instead of > > each cluster having a separate UI more consistent with how Cassandra > > architecture works) > > * Support for Cassandra 2.2, 3.0, 3.11, and soon 4.0 > > > > [1] http://cassandra-reaper.io/docs/usage/multi_dc/ > > [2] https://github.com/hashbrowncipher/cassandra-mirror > > > > We can't "get the community" of these sidecars and drop support for > > 90+% of the supported configurations and features at the same time ... > > These projects have made promises to users, as per the name technical > > debt these choices made over years have explicit costs that we have to > > take into account if we accept a project as is with the goal of taking > > the community with us. If we don't have the goal of taking the > > existing community with us and are instead aiming to simply make > > Cassandra work out of the box without external tools, then we don't > > have to continue fulfilling these promises. > > > > Instead I think the organizations that made those promises (TLP and > > Netflix in these specific cases) should continue keeping them, and the > > Cassandra management process should be incrementally built by the > > Cassandra community with decisions made as a community and we only GA > > it when the PMC/community is happy with making a promise of support > > for the features that we've merged (and since we're starting from a > > fresh start if people have strong opinions about fundamental > > architecture we can try to take those into account like we did with > > the months of feedback on repair scheduling > > runtime/architecture/design). If we can't prove value over other > > community tools for running 4.x, which is definitely a possibility, > > then we don't mark the management process GA, we re-focus on > > individual community tools, and imo failure is a reasonable result of > > attempted innovation. > > > > -Joey > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > -- Jon Haddad http://www.rustyrazorblade.com twitter: rustyrazorblade
