Hm. We have a wrinkle that's thankfully testable. I've been talking back and forth w/Mick about his and the TLP team's experience upgrading clusters to 3.0 vs. upgrading to 3.11. The observations and assertion: 3.0 has performance regressions from 2.1 that were fixed through the tick-tock line as well as several significant optimizations in 3.11 not present on 3.0. Some examples:
- https://issues.apache.org/jira/browse/CASSANDRA-12269 - https://issues.apache.org/jira/browse/CASSANDRA-11206 - https://issues.apache.org/jira/browse/CASSANDRA-12731 - https://issues.apache.org/jira/browse/CASSANDRA-9766 - https://issues.apache.org/jira/browse/CASSANDRA-11623 Do we as a project have a sense of whether 3.0's performance is, at this time, on parity with the 2.1/2.2 line of releases? As I framed it to Mick on slack, a hypothesis we can test: "3.11 performs close to parity with 2.1/2.2. 3.0 does not. If we recommend people upgrade from 2.1 -> 3.0 -> 4.0, we are asking them to have a cluster in a regressed performance state for potentially months as they execute their upgrade." Did I get anything wrong here Mick? ^ I'll work with some folks to test this assertion. *pending results*, what that would amount to is the following change to the latest state on this thread: >From → To: 2.1 → 3.11 → 4.0 3.0 → 4.0 3.11 → 4.0 Fundamentally it doesn't change that we need to confirm 3.0 → 4.0 as well as 3.11 → 4.0, it strictly amounts to a change in recommendation for 2.1 cluster destinations. Until we have more information on this, I believe we shouldn't document a recommendation yet. I'll treat this with urgency. On Sat, Oct 10, 2020 at 6:05 AM, Benedict Elliott Smith <bened...@apache.org > wrote: > This sounds eminently sensible to me. > > On 09/10/2020, 19:42, "Joshua McKenzie" <jmcken...@apache.org> wrote: > > Fair point on uncertainties and delaying decisions until strictly required > so we have more data. > > I want to nuance my earlier proposal and what we document (sorry for the > multiple messages; my time is fragmented enough these days that I only have > thin slices to engage w/stuff like this). > > I think we should do a "From → To" model for both testing and supporting > upgrades and have a point of view as a project for each currently supported > version of C* in the "From" list. Specifically - we test and recommend the > following paths: > > 1. 2.1 → 3.0 → 4.0 > 2. 3.0 → 4.0 (subset of 1) > 3. 3.11 → 4.0 > > There's no value whatsoever in hopping through an interim version if a > leapfrog is expected to be as tested and stable. The only other alternative > would be to recommend 2.1 → 3.11 → 4.0 (as Mick alluded to) but that just > exposes to more deltas from the tick-tock .X line for no added value as you > mentioned. > > We could re-apply the "from-to" testing and support model in future > releases w/whatever is supported at that time. That way users will be able > to have a single source of truth on what the project recommends and vets > for going from wherever they are to the latest. > > On Fri, Oct 09, 2020 at 12:05 PM, Benedict Elliott Smith < benedict@ > apache.org> wrote: > > There is a sizeable cohort of us who I expect to be primarily focused on > 3.0->4.0, so if you have a cohort focusing primarily on 3.11->4.0 I think > we'll be in good shape. > > For all subsequent major releases, we test and officially support only 1 > major back > > I think we should wait to see what happens before committing ourselves to > something like this - things like release cadence etc will matter a lot. > That is *definitely* not to say that I disagree with you, just that I think > more project future-context is needed to make a decision like this. I > expect we'll have lots more fun (hopefully positive) conversations around > topics like this in the coming year, as I have no doubt we all want to > evolve our approach to releases, and there's no knowing what we'll end up > deciding (we have done some crazy things in the past __ ). > > On 09/10/2020, 16:46, "Joshua McKenzie" <jmcken...@apache.org> wrote: > > I think it's a clean and simple heuristic for the project to say "you can > safely upgrade to adjacent major versions". > > The difficulty we face with 3.0 is that it has made many contributors very > wary of pre 4.0 code and with good reason. Your point about conservative > users upgrading later in a cycle resonates Benedict, and reflects on the > confidence we should or should not have in 3.11. I think it's also > important to realize that many cluster upgrades can take months, so it's > not a transient exposure to unknowns in a release. > > I propose the following compromise: > > 1. For 4.0 GA, we consider the following upgrade paths "tested and > supported": 2.1 → 3.0 → 3.11 → 4.0, and 2.1 → 3.0 → 4.0 > 2. For all subsequent major releases, we test and officially support only > 1 major back > 3. Any contributor can optionally meet whatever bar we set for "tested and > supported" to allow leapfrogging versions, but we won't constrain GA on > that. > > We have to pay down our debt right now, but if we have to continue to do > this in the future we're not learning from our mistakes. > > Speaking for DataStax, we don't have enough resources to work through the > new testing work on 40_quality_test, the defects that David is surfacing > like crazy (well done!), and validating 2 major upgrade paths. If you and a > set of contributors could take on the 3.0 → 4.0 path Benedict, that'd be a > great help. I also assume we could all collaborate on the tooling / infra / > approaches we use for this validation so it wouldn't be a complete re-work. > > On Fri, Oct 09, 2020 at 11:02 AM, Benedict Elliott Smith < benedict@ > apache.org> wrote: > > Since email is very unclear and context gets lost, I'm personally OK with > officially supporting all of these upgrade paths, but the spectre was > raised that this might lead to lost labour due to an increased support > burden. My view is that 3.0->4.0 is probably a safer upgrade path for users > and as a result a lower support cost to the project, so I would be happy to > deprecate 3.0->3.11 if this helps alleviate the concerns of others that > this would be costly to the project. Alternatively, if we want to support > both but some feel 3.0->4.0 is burdensome, I would be happy to focus on > 3.0->4.0 while they focus on the paths I would be happy to deprecate. > > On 09/10/2020, 15:49, "Benedict Elliott Smith" <bened...@apache.org> > wrote: > > Yeah, and perhaps even drop 2.1 (2.2) -> 3.11 when 4.0 appears. > > I think there's anyway a big difference between supported and encouraged. > I think we should encourage 2.1->3.0->4.0, while maintaining support for > 2.2->3.0 and 3.0->3.11 for critical bugs only, and 3.11->4.0 in the normal > way given the userbase that is already on 3.11. > > we can expect it to be *stable enough to upgrade through* > > I don't know that this is true at all. Most bugs are not found by the > general userbase, and the most conservative (hence most likely to spot > problems on upgrade) are generally very late to the party. 2.1(2.2)->3.0 is > still discovering bugs today, many years after this metric was passed for > 3.0 - largely as the more sophisticated users upgrade. > > On 09/10/2020, 15:40, "Marcus Eriksson" <marc...@apache.org> wrote: > > My suggestion for "supported" upgrade paths would be; > > 2.1 (2.2) -> 3.0 -> 4.0 > 2.1 (2.2) -> 3.11 -> 4.0 > > and drop support for 3.0 -> 3.11 when we release 4.0 > > /Marcus > > On 9 October 2020 at 16:12:12, Joshua McKenzie (jmcken...@apache.org) > wrote: > > Some data that I believe is relevant here. > > Numerically it's safe to assume there's over 10,000 ASF C* clusters out in > the world (5,500 in China alone). In surveys (both informal polling and > primary research), at least 1/3rd of folks are running the 3.X latest if I > recall correctly. > > Basic conclusions we can draw from these data points: > 1) There are thousands of clusters running some form of post 3.0, so we > can expect it to be *stable enough to upgrade through* > 2) We have to support at least 3.11 → 4.0 > > If 1/3rd of our users are running 2.1, 1/3rd 3.0, and 1/3rd 3.11 > (hand-waving, probably more in the 25 vs. 40 etc but splitting hairs), > there's clearly a significant value-add in usability of skipping majors > (3.0->4.0). Depending on how we define "done" and "supported" for upgrade > testing, this will represent a significant development burden. > > From a *functional MVP* perspective on what upgrade paths we need to > support, the absolute minimum would be 2.1 → 3.0 → 3.11 → 4.0 > > If anyone wants to step in and officially support the 3.0 → 4.0 line, > that's fantastic both for the project community and for users. But as far > as basic table stakes, I can't think of a logical reason 3.0 → 4.0 as an > upgraded path should be considered a blocker for releasing 4.0 GA. > > On Fri, Oct 09, 2020 at 9:53 AM, Mick Semb Wever wrote: > > At The Last Pickle we have always recommended avoiding 3.0, including > upgrading from 2.2 directly to 3.11. > We (now DataStax) will continue to recommend that folk upgrade to the > latest 3.11 before upgrading to 4.0. > > To clarify that^, if it wasn't obvious, I wasn't making a statement about > DataStax at at large, but about those of us at TLP and now the team > providing the consulting for Apache Cassandra from DataStax. > > --------------------------------------------------------------------- To > unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional > commands, e-mail: dev-h...@cassandra.apache.org > > --------------------------------------------------------------------- To > unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional > commands, e-mail: dev-h...@cassandra.apache.org > > --------------------------------------------------------------------- To > unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional > commands, e-mail: dev-h...@cassandra.apache.org > > --------------------------------------------------------------------- To > unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional > commands, e-mail: dev-h...@cassandra.apache.org > > --------------------------------------------------------------------- To > unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional > commands, e-mail: dev-h...@cassandra.apache.org >
