> ACCORD in particular was hyped in numerous talks and presentations and noone > cautioned it might not hit 5.0, quite the opposite We need to be very careful in the future about how we communicate the availability of future novel work, especially when the ones promoting the delivery of that work and timelines aren't the ones actively working on the code. And to be explicit - I don't think there's any bad actors here; I think this is a natural consequence of specialization of skills and focus in the community as well as disjoint between different groups of people.
Also, it's become clear to me that we still weren't all in alignment on our view of "do we ship 5.0 based on a date or do we ship 5.0 based on feature availability". Since we're still going through some evolution in our release philosophy (train vs. feature, etc), this is to be expected. We're getting there. Having a marketing working group has helped bridge this gap, and getting more participation from other people in the community on that effort would help align more of us. On Fri, Oct 27, 2023, at 5:00 PM, German Eichberger via dev wrote: > Definitely want to second Josh. When I reached out on the ACCORD channel > about testing folks were super helpful and transparent about bugs, etc. > > Frankly, I was pretty frustrated that ACCORD+TCM slipped. I was looking > forward to it and felt let down - but I also haven't done anything to help > other than trying it out. So, I only have myself to blame... > > That there was a surprise for many of us that it slipped is an indication > there wasn't enough communication - we should probably rethink how we > communicate progress, especially on long running and highly anticipated > initiatives. Maybe a paragraph in the "Project Status Update" (but then we > need more frequent updates 🙂) -- or send a separate update e-mail or as Maxim > is suggesting to some newly created release list. > > A highly anticipated feature has more visibility and we need to account for > that with more communication other than the usual channels. ACCORD in > particular was hyped in numerous talks and presentations and noone cautioned > it might not hit 5.0, quite the opposite --so we need to ask ourselves how > people who go on stage as Cassandra experts are not aware that it could slip. > That's where I think more communication could help -- > > > Thanks, > German > > > > > > *From:* Josh McKenzie <jmcken...@apache.org> > *Sent:* Friday, October 27, 2023 10:13 AM > *To:* dev <dev@cassandra.apache.org> > *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and > cut an immediate 5.1-alpha1) > > Lots of threads of thought have popped up here. The big one I feel needs to > be clearly addressed and inspected is the implication of development not > happening transparently and not being inclusive or available for > participation by the community on major features. > > The CEP process + dedicated development channels on ASF slack + public JIRA's > + feature branches in the ASF repo we've seen with specifically TCM and > Accord are the most transparent this kind of development has *ever been* on > this project, and I'd argue right at the sweet spot or past where the degree > of reaching out to external parties to get feedback starts to not just hit > diminishing returns but starts to actively hurt a small group of peoples' > ability to make rapid progress on something. > > No-one can expect to review everything, and no-one can expect to follow every > JIRA, commit, or update. This is why we have the role of a committer; a > person in this community we've publicly communicated we trust based on earned > merit (and in our project's case, at least 2 people who's opinion we trust) > to do quality work, validate it, and reach our expected bar for correctness, > performance, and maintainability. If a CEP is voted in and 2 committers have > an implementation they feel meets the goals, CI is green, and nobody has a > serious technical concern that warrants a binding -1, we're good. It doesn't, > and shouldn't, matter who currently employs or sponsors their work. It > doesn't, and shouldn't, matter whether individuals on the project who were > interested in collaborating on that work missed one or multiple > announcements, or whether they saw those announcements and just didn't have > the cycles to engage when they wanted to. > > Now - we can always improve. We can always try and be proactive, knowing each > other and our interests and reaching out to specific folks to make sure > they're aware that work has hit a collaboration point or inflection point. I > can do (apparently much) better about sending out more consistent project > status updates with calls to action around when these inflection points occur > as well. > > At the end of the day, this is an Apache project, and trust and lazy > consensus are the backbone for how a lot of this stuff works when distributed > and at scale. > > On Fri, Oct 27, 2023, at 10:51 AM, Alex Petrov wrote: >> Firstly, when talking about quality, many folks mention the risk of >> releasing bugs together with TCM and Accord. While I agree this risk is >> real, I would like to remind that TCM and Accord were extensively tested and >> simulated, for *many* hours. Just an example, we’ve recently filed an issue >> we found during our Harry testing, and this issue was introduced outside >> TCM. So a lot of validation work will be invalidated by having two releases. >> And I am not certain how many organisations have capacity for internally >> vetting two subsequent major releases. But I am also not compeltely opposed >> to having 5.0 and 5.1 if this is something a majority of contributors >> prefers. >> >> As regards developing CEPs in public, this is how it was already done this >> time. Both Accord and TCM were published for a substantial amount of time. I >> did read an argument that the code was somehow not ready for review, but by >> the same logic neither is it ready right now, as the interesting parts >> haven't changed in a while. Even the feedback on the CEP itself, which was >> published in April 2023, was minimal. There were multiple sessions about the >> TCM and Accord in New Orleans in 2023, and the interested parties (including >> many folks form this discussion) couldn't help but learn about their status >> and progress. Still, there was very little engagement (which, I claim, is >> absolutely fine). So, since one can't say that we (collectively) are not >> pubslihing CEPs and code early enough, the only argument is that the people >> choose to prioritise things based on what is important for their businesses >> today, and this is, again, completely fine. >> >> If you are interested in a CEP, make sure you engage with its authors from >> the first time they publish something. There are many patches and CEPs I >> wish I have reviewed, but did not have time for. For those, I am reading the >> available discussions, talking to their authors, and writing Harry tests. I >> would not, however, ask someone to postpone a feature based on my past or >> future availability. >> >> On Fri, Oct 27, 2023, at 10:14 AM, Jacek Lewandowski wrote: >>> I've been thinking about this and I believe that if we ever decide to delay >>> a release to include some CEPs, we should make the plan and status of those >>> CEPs public. This should include publishing a branch, creating tickets for >>> the remaining work required for feature completion in Jira, and notifying >>> the mailing list. >>> >>> By doing this, we can make an informed decision about whether delivering a >>> CEP in a release x.y planned for some time z is feasible. This approach >>> would also be beneficial for improving collaboration, as we will all be >>> aware of what is left to be done and can adjust our focus accordingly to >>> participate in the remaining work. >>> >>> Thanks, >>> - - -- --- ----- -------- ------------- >>> Jacek Lewandowski >>> >>> >>> pt., 27 paź 2023 o 10:26 Benjamin Lerer <ble...@apache.org> napisał(a): >>>> I would be interested in testing Maxim's approach. We need more visibility >>>> on big features and their progress to improve our coordination. Hopefully >>>> it will also open the door to more collaboration on those big projects. >>>> >>>> Le jeu. 26 oct. 2023 à 21:35, German Eichberger via dev >>>> <dev@cassandra.apache.org> a écrit : >>>>> +1 to Maxim's idea >>>>> >>>>> Like Stefan my assumption was that we would get some version of TCM + >>>>> ACCORD in 5.0 but it wouldn't be ready for production use. My own testing >>>>> and conversations at Community over Code in Halifax confirmed this. >>>>> >>>>> From this perspective as disappointing as TCM+ACCORD slipping is moving >>>>> it to 5.1 makes sense and I am supporting of this - but I am worried if >>>>> 5.1 is basically 5.0 + TCM/ACCORD and this slips again we draw ourselves >>>>> into a corner where we can't release 5.2 before 5.1 or something. I would >>>>> like some more elaboration on that. >>>>> >>>>> I am also very worried about ANN vector search being in jeopardy for 5.0 >>>>> which is an important feature for me to win some internal company bet 🙂 >>>>> >>>>> My 2 cents, >>>>> German >>>>> >>>>> >>>>> >>>>> *From:* Miklosovic, Stefan via dev <dev@cassandra.apache.org> >>>>> *Sent:* Thursday, October 26, 2023 4:23 AM >>>>> *To:* dev@cassandra.apache.org <dev@cassandra.apache.org> >>>>> *Cc:* Miklosovic, Stefan <stefan.mikloso...@netapp.com> >>>>> *Subject:* [EXTERNAL] Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 >>>>> (and cut an immediate 5.1-alpha1) >>>>> >>>>> What Maxim proposes in the last paragraph would be definitely helpful. >>>>> Not for the project only but for a broader audience, companies etc., too. >>>>> >>>>> Until this thread was started, my assumption was that "there will be 5.0 >>>>> on summit with TCM and Accord and it somehow just happens". More >>>>> transparent communication where we are at with high-profile CEPs like >>>>> these and knowing if deadlines are going to be met would be welcome. >>>>> >>>>> I don't want to be that guy and don't take me wrong here, but really, >>>>> these CEPs are being developed, basically, by devs from two companies, >>>>> which have developers who do not have any real need to explain themselves >>>>> like what they do, regularly, to outsiders. (or maybe you do, you just >>>>> don't have time?) I get that. But on the other hand, you can not >>>>> realistically expect that other folks will have any visibility into what >>>>> is going on there and that there is a delay on the horizon and so on. >>>>> >>>>> ________________________________________ >>>>> From: Maxim Muzafarov <mmu...@apache.org> >>>>> Sent: Thursday, October 26, 2023 12:21 >>>>> To: dev@cassandra.apache.org >>>>> Subject: Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an >>>>> immediate 5.1-alpha1) >>>>> >>>>> NetApp Security WARNING: This is an external email. Do not click links or >>>>> open attachments unless you recognize the sender and know the content is >>>>> safe. >>>>> >>>>> >>>>> >>>>> >>>>> Personally, I think frequent releases (2-3 per year) are better than >>>>> infrequent big releases. I can understand all the concerns from a >>>>> marketing perspective, as smaller major releases may not shine as >>>>> brightly as a single "game changer" release. However, smaller >>>>> releases, especially if they don't have backwards compatibility >>>>> issues, are better for the engineering and SRE teams because if a >>>>> long-awaited feature is delayed for any reason, there should be no >>>>> worry about getting it in right into the next release. >>>>> >>>>> An analogy here might be that if you miss your train (small release) >>>>> due to circumstances, you can wait right here for the next one, but if >>>>> you miss a flight (big release), you will go back home :-) This is why >>>>> I think that the 5.0, 5.1, 5.2, etc. are better and I support Mick's >>>>> plan with the caveat that we should release 5.1 when we think we are >>>>> ready to do so. Here is an example of the Postgres releases [1]. >>>>> >>>>> [1] >>>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbucardo.org%2Fpostgres_all_versions.html&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187354112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zjMpuN%2FQMhBtFTemLswn8BRaLyQ9eLZTIeZfeWYwhQk%3D&reserved=0 >>>>> <https://bucardo.org/postgres_all_versions.html> >>>>> >>>>> >>>>> Another little thing that I'd like to mention is a release management >>>>> story. In the Apache Ignite project, we've got used to creating a >>>>> release thread and posting the release status updates and/or problems, >>>>> and/or delays there, and maybe some of the benchmarks at the end. Of >>>>> course, this was done by the release manager who volunteered to do >>>>> this work. I'm not saying we're doing anything wrong here, no, but the >>>>> publicity and openness, coupled with regular updates, could help >>>>> create a real sense of the remaining work in progress. These are my >>>>> personal feelings, and definitely not actions to be taken. The example >>>>> is here: [2]. >>>>> >>>>> [2] >>>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm11m0nxq701f2cj8xxdcsc4nnn2sm8ql&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187360611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BG0wgMItsMv83XDLzRgbfJoi%2FiwSywWU0qAzN%2BmMBZU%3D&reserved=0 >>>>> <https://lists.apache.org/thread/m11m0nxq701f2cj8xxdcsc4nnn2sm8ql> >>>>> >>>>> On Thu, 26 Oct 2023 at 11:15, Benjamin Lerer <b.le...@gmail.com> wrote: >>>>> >> >>>>> >> Regarding the release of 5.1, I understood the proposal to be that we >>>>> >> cut an actual alpha, thereby sealing the 5.1 release from new >>>>> >> features. Only features merged before we cut the alpha would be >>>>> >> permitted, and the alpha should be cut as soon as practicable. What >>>>> >> exactly would we be waiting for? >>>>> > >>>>> > >>>>> > The problem I believe is about expectations. It seems that your >>>>> > expectation is that a release with only TCM and Accord will reach GA >>>>> > quickly. Based on the time it took us to release 4.1, I am simply >>>>> > expecting more delays (a GA around end of May, June). In which case it >>>>> > seems to me that we could be interested in shipping more stuff in the >>>>> > meantime (thinking of CASSANDRA-15254 or CEP-29 for example). >>>>> > I do not have a strong opinion, I just want to make sure that we all >>>>> > share the same understanding and fully understand what we agree upon. >>>>> > >>>>> > Le jeu. 26 oct. 2023 à 10:59, Benjamin Lerer <b.le...@gmail.com> a >>>>> > écrit : >>>>> >>> >>>>> >>> I am surprised this needs to be said, but - especially for >>>>> >>> long-running CEPs - you must involve yourself early, and certainly >>>>> >>> within some reasonable time of being notified the work is ready for >>>>> >>> broader input and review. In this case, more than six months ago. >>>>> >> >>>>> >> >>>>> >> It is unfortunately more complicated than that because six month ago >>>>> >> Ekaterina and I were working on supporting Java 17 and dropping Java 8 >>>>> >> which was needed by different ongoing works. We both missed the >>>>> >> announcement that TCM was ready for review and anyway would not have >>>>> >> been available at that time. Maxim has asked me ages ago for a review >>>>> >> of CASSANDRA-15254 more than 6 months ago and I have not been able to >>>>> >> help him so far. We all have a limited bandwidth and can miss some >>>>> >> announcements. >>>>> >> >>>>> >> The project has grown and a lot of things are going on in parallel. >>>>> >> There are also more interdependencies between the different projects. >>>>> >> In my opinion what we are lacking is a global overview of the >>>>> >> different things going on in the project and some rough ideas of the >>>>> >> status of the different significant pieces. It would allow us to >>>>> >> better organize ourselves. >>>>> >> >>>>> >> Le jeu. 26 oct. 2023 à 00:26, Benedict <bened...@apache.org> a écrit : >>>>> >>> >>>>> >>> I have spoken privately with Ekaterina, and to clear up some possible >>>>> >>> ambiguity: I realise nobody has demanded a delay to this work to >>>>> >>> conduct additional reviews; a couple of folk have however said they >>>>> >>> would prefer one. >>>>> >>> >>>>> >>> >>>>> >>> My point is that, as a community, we need to work on ensuring folk >>>>> >>> that care about a CEP participate at an appropriate time. If they >>>>> >>> aren’t able to, the consequences of that are for them to bear. >>>>> >>> >>>>> >>> >>>>> >>> We should be working to avoid surprises as CEP start to land. To this >>>>> >>> end, I think we should work on some additional paragraphs for the >>>>> >>> governance doc covering expectations around the landing of CEPs. >>>>> >>> >>>>> >>> >>>>> >>> On 25 Oct 2023, at 21:55, Benedict <bened...@apache.org> wrote: >>>>> >>> >>>>> >>>  >>>>> >>> >>>>> >>> I am surprised this needs to be said, but - especially for >>>>> >>> long-running CEPs - you must involve yourself early, and certainly >>>>> >>> within some reasonable time of being notified the work is ready for >>>>> >>> broader input and review. In this case, more than six months ago. >>>>> >>> >>>>> >>> >>>>> >>> This isn’t the first time this has happened, and it is disappointing >>>>> >>> to see it again. Clearly we need to make this explicit in the >>>>> >>> guidance docs. >>>>> >>> >>>>> >>> >>>>> >>> Regarding the release of 5.1, I understood the proposal to be that we >>>>> >>> cut an actual alpha, thereby sealing the 5.1 release from new >>>>> >>> features. Only features merged before we cut the alpha would be >>>>> >>> permitted, and the alpha should be cut as soon as practicable. What >>>>> >>> exactly would we be waiting for? >>>>> >>> >>>>> >>> >>>>> >>> If we don’t have a clear and near-term trigger for branching 5.1 for >>>>> >>> its own release, shortly after Accord and TCM merge, then I am in >>>>> >>> favour of instead delaying 5.0. >>>>> >>> >>>>> >>> >>>>> >>> On 25 Oct 2023, at 19:40, Mick Semb Wever <m...@apache.org> wrote: >>>>> >>> >>>>> >>>  >>>>> >>> I'm open to the suggestions of not branching cassandra-5.1 and/or >>>>> >>> naming a preview release something other than 5.1-alpha1. >>>>> >>> >>>>> >>> But… the codebases and release process (and upgrade tests) do not >>>>> >>> currently support releases with qualifiers that are not alpha, beta, >>>>> >>> or rc. We can add a preview qualifier to this list, but I would not >>>>> >>> want to block getting a preview release out only because this wasn't >>>>> >>> yet in place. >>>>> >>> >>>>> >>> Hence the proposal used 5.1-alpha1 simply because that's what we know >>>>> >>> we can do today. An alpha release also means (typically) the branch. >>>>> >>> >>>>> >>> Is anyone up for looking into adding a "preview" qualifier to our >>>>> >>> release process? >>>>> >>> This may also solve our previous discussions and desire to have >>>>> >>> quarterly releases that folk can use through the trunk dev cycle. >>>>> >>> >>>>> >>> Personally, with my understanding of timelines in front of us to >>>>> >>> fully review and stabilise tcm, it makes sense to branch it as soon >>>>> >>> as it's merged. It's easiest to stabilise it on a branch, and >>>>> >>> there's definitely the desire and demand to do so, so it won't be >>>>> >>> getting forgotten or down-prioritised. >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> On Wed, 25 Oct 2023 at 18:07, Jeremiah Jordan >>>>> >>> <jeremiah.jor...@gmail.com> wrote: >>>>> >>>>> >>>>> >>>>> If we do a 5.1 release why not take it as an opportunity to release >>>>> >>>>> more things. I am not saying that we will. Just that we should let >>>>> >>>>> that door open. >>>>> >>>> >>>>> >>>> >>>>> >>>> Agreed. This is the reason I brought up the possibility of not >>>>> >>>> branching off 5.1 immediately. >>>>> >>>> >>>>> >>>> >>>>> >>>> On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer <b.le...@gmail.com> >>>>> >>>> wrote: >>>>> >>>>> >>>>> >>>>> The proposal includes 3 things: >>>>> >>>>> 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0 >>>>> >>>>> 2. The next release will be 5.1 and will include only Accord and TCM >>>>> >>>>> 3. Merge TCM and Accord right now in 5.1 (making an initial release) >>>>> >>>>> >>>>> >>>>> I am fine with question 1 and do not have a strong opinion on which >>>>> >>>>> way to go. >>>>> >>>>> 2. Means that every new feature will have to wait for post 5.1 even >>>>> >>>>> if it is ready before 5.1 is stabilized and shipped. If we do a 5.1 >>>>> >>>>> release why not take it as an opportunity to release more things. I >>>>> >>>>> am not saying that we will. Just that we should let that door open. >>>>> >>>>> 3. There is a need to merge TCM and Accord as maintaining those >>>>> >>>>> separate branches is costly in terms of time and energy. I fully >>>>> >>>>> understand that. On the other hand merging TCM and Accord will make >>>>> >>>>> the TCM review harder and I do believe that this second round of >>>>> >>>>> review is valuable as it already uncovered a valid issue. >>>>> >>>>> Nevertheless, I am fine with merging TCM as soon as it passes CI >>>>> >>>>> and continuing the review after the merge. If we cannot meet at >>>>> >>>>> least that quality level (Green CI) we should not merge just for >>>>> >>>>> creating an 5.1.alpha release for the summit. >>>>> >>>>> >>>>> >>>>> Now, I am totally fine with a preview release without numbering and >>>>> >>>>> with big warnings that will only serve as a preview for the summit. >>>>> >>>>> >>>>> >>>>> Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi >>>>> >>>>> <berenguerbl...@gmail.com> a écrit : >>>>> >>>>>> >>>>> >>>>>> I also think there's many good new features in 5.0 already they'd >>>>> >>>>>> make a >>>>> >>>>>> good release even on their own. My 2 cts. >>>>> >>>>>> >>>>> >>>>>> On 24/10/23 23:20, Brandon Williams wrote: >>>>> >>>>>> > The catch here is that we don't publish docker images currently. >>>>> >>>>>> > The >>>>> >>>>>> > C* docker images available are not made by us. >>>>> >>>>>> > >>>>> >>>>>> > Kind Regards, >>>>> >>>>>> > Brandon >>>>> >>>>>> > >>>>> >>>>>> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin >>>>> >>>>>> > <pmcfa...@gmail.com> wrote: >>>>> >>>>>> >> Let me make that really easy. Hell yes >>>>> >>>>>> >> >>>>> >>>>>> >> Not everybody runs CCM, I've tried but I've met resistance. >>>>> >>>>>> >> >>>>> >>>>>> >> Compiling your own version usually involves me saying the words >>>>> >>>>>> >> "Yes, ant realclean exists. I'm not trolling you" >>>>> >>>>>> >> >>>>> >>>>>> >> docker pull <image> works on every OS and curates a single node >>>>> >>>>>> >> experience. >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie >>>>> >>>>>> >> <jmcken...@apache.org> wrote: >>>>> >>>>>> >>> In order for the project to advertise the release outside the >>>>> >>>>>> >>> dev@ list it needs to be a formal release. >>>>> >>>>>> >>> >>>>> >>>>>> >>> That's my reading as well: >>>>> >>>>>> >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23release-definition&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187365573%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NG4J76LSzq6jP34phPPpQav7kTnXcaSytE0wDAZJd30%3D&reserved=0 >>>>> >>>>>> >>> >>>>> >>>>>> >>> <https://www.apache.org/legal/release-policy.html#release-definition> >>>>> >>>>>> >>> >>>>> >>>>>> >>> I wonder if there'd be value in us having a cronned job that'd >>>>> >>>>>> >>> do nightly docker container builds on trunk + feature >>>>> >>>>>> >>> branches, archived for N days, and we make that generally >>>>> >>>>>> >>> known to the dev@ list here so folks that want to poke at the >>>>> >>>>>> >>> current state of trunk or other branches could do so with very >>>>> >>>>>> >>> low friction. We'd probably see more engagement on feature >>>>> >>>>>> >>> branches if it was turn-key easy for other C* devs to spin the >>>>> >>>>>> >>> up and check them out. >>>>> >>>>>> >>> >>>>> >>>>>> >>> For what you're talking about here Patrick (a docker image for >>>>> >>>>>> >>> folks outside the dev@ audience and more user-facing), we'd >>>>> >>>>>> >>> want to vote on it and go through the formal process. >>>>> >>>>>> >>> >>>>> >>>>>> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> In order for the project to advertise the release outside the >>>>> >>>>>> >>> dev@ list it needs to be a formal release. That just means >>>>> >>>>>> >>> that there was a release vote and at least 3 PMC members +1’ed >>>>> >>>>>> >>> it, and there are more +1 than there are -1, and we follow all >>>>> >>>>>> >>> the normal release rules. The ASF release process doesn’t >>>>> >>>>>> >>> care what branch you cut the artifacts from or what version >>>>> >>>>>> >>> you call it. >>>>> >>>>>> >>> >>>>> >>>>>> >>> So the project can cut artifacts for and release a 5.1-alpha1, >>>>> >>>>>> >>> 5.1-dev-preview1, what ever we want to version this thing, >>>>> >>>>>> >>> from trunk or any other branch name we want. >>>>> >>>>>> >>> >>>>> >>>>>> >>> -Jeremiah >>>>> >>>>>> >>> >>>>> >>>>>> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin >>>>> >>>>>> >>> <pmcfa...@gmail.com> wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> I would like to have something for developers to use ASAP to >>>>> >>>>>> >>> try the Accord syntax. Very few people have seen it, and I >>>>> >>>>>> >>> think there's a learning curve we can start earlier. >>>>> >>>>>> >>> >>>>> >>>>>> >>> It's my understanding that ASF policy is that it needs to be a >>>>> >>>>>> >>> project release to create a docker image. >>>>> >>>>>> >>> >>>>> >>>>>> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan >>>>> >>>>>> >>> <jeremiah.jor...@gmail.com> wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> If we decide to go the route of not merging TCM to the 5.0 >>>>> >>>>>> >>> branch. Do we actually need to immediately cut a 5.1 branch? >>>>> >>>>>> >>> Can we work on stabilizing things while it is in trunk and cut >>>>> >>>>>> >>> the 5.1 branch when we actually think we are near releasing? >>>>> >>>>>> >>> I don’t see any reason we can not cut “preview” artifacts from >>>>> >>>>>> >>> trunk? >>>>> >>>>>> >>> >>>>> >>>>>> >>> -Jeremiah >>>>> >>>>>> >>> >>>>> >>>>>> >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad >>>>> >>>>>> >>> <rustyrazorbl...@apache.org> wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> I guess at the end of the day, shipping a release with a bunch >>>>> >>>>>> >>> of awesome features is better than holding it back. If >>>>> >>>>>> >>> there's 2 big releases in 6 months the community isn't any >>>>> >>>>>> >>> worse off. >>>>> >>>>>> >>> >>>>> >>>>>> >>> We either ship something, or nothing, and something is >>>>> >>>>>> >>> probably better. >>>>> >>>>>> >>> >>>>> >>>>>> >>> Jon >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> On 2023/10/24 16:27:04 Patrick McFadin wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> +1 to what you are saying, Josh. Based on the last survey, >>>>> >>>>>> >>> yes, everyone >>>>> >>>>>> >>> >>>>> >>>>>> >>> was excited about Accord, but SAI and UCS were pretty high on >>>>> >>>>>> >>> the list. >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> Benedict and I had a good conversation last night, and now I >>>>> >>>>>> >>> understand >>>>> >>>>>> >>> >>>>> >>>>>> >>> more essential details for this conversation. TCM is taking >>>>> >>>>>> >>> far more work >>>>> >>>>>> >>> >>>>> >>>>>> >>> than initially scoped, and Accord depends on a stable TCM. TCM >>>>> >>>>>> >>> is months >>>>> >>>>>> >>> >>>>> >>>>>> >>> behind and that's a critical fact, and one I personally just >>>>> >>>>>> >>> learned of. I >>>>> >>>>>> >>> >>>>> >>>>>> >>> thought things were wrapping up this month, and we were in the >>>>> >>>>>> >>> testing >>>>> >>>>>> >>> >>>>> >>>>>> >>> phase. I get why that's a topic we are dancing around. Nobody >>>>> >>>>>> >>> wants to say >>>>> >>>>>> >>> >>>>> >>>>>> >>> ship dates are slipping because that's part of our culture. >>>>> >>>>>> >>> It's >>>>> >>>>>> >>> >>>>> >>>>>> >>> disappointing and, if new information, an unwelcome surprise, >>>>> >>>>>> >>> but none of >>>>> >>>>>> >>> >>>>> >>>>>> >>> us should be angry or in a blamey mood because I guarantee >>>>> >>>>>> >>> every one of us >>>>> >>>>>> >>> >>>>> >>>>>> >>> has shipped the code late. My reaction yesterday was based on >>>>> >>>>>> >>> an incorrect >>>>> >>>>>> >>> >>>>> >>>>>> >>> assumption. Now that I have a better picture, my point of view >>>>> >>>>>> >>> is changing. >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> Josh's point about what's best for users is crucial. Users >>>>> >>>>>> >>> deserve stable >>>>> >>>>>> >>> >>>>> >>>>>> >>> code with a regular cadence of features that make their lives >>>>> >>>>>> >>> easier. If we >>>>> >>>>>> >>> >>>>> >>>>>> >>> put 5.0 on hold for TCM + Accord, users will get neither for a >>>>> >>>>>> >>> very long >>>>> >>>>>> >>> >>>>> >>>>>> >>> time. And I mentioned a disaster yesterday. A bigger disaster >>>>> >>>>>> >>> would be >>>>> >>>>>> >>> >>>>> >>>>>> >>> shipping Accord with a major bug that causes data loss, >>>>> >>>>>> >>> eroding community >>>>> >>>>>> >>> >>>>> >>>>>> >>> trust. Accord has to be the most bulletproof of all >>>>> >>>>>> >>> bulletproof features. >>>>> >>>>>> >>> >>>>> >>>>>> >>> The pressure to ship is only going to increase and that's >>>>> >>>>>> >>> fertile ground >>>>> >>>>>> >>> >>>>> >>>>>> >>> for that sort of bug. >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> So, taking a step back and with a clearer picture, I support >>>>> >>>>>> >>> the 5.0 + 5.1 >>>>> >>>>>> >>> >>>>> >>>>>> >>> plan mainly because I don't think 5.1 is (or should be) a fast >>>>> >>>>>> >>> follow. >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> For the user community, the communication should be >>>>> >>>>>> >>> straightforward. TCM + >>>>> >>>>>> >>> >>>>> >>>>>> >>> Accord are turning out to be much more complicated than was >>>>> >>>>>> >>> originally >>>>> >>>>>> >>> >>>>> >>>>>> >>> scoped, and for good reasons. Our first principle is to >>>>> >>>>>> >>> provide a stable >>>>> >>>>>> >>> >>>>> >>>>>> >>> and reliable system, so as a result, we'll be de-coupling TCM >>>>> >>>>>> >>> + Accord from >>>>> >>>>>> >>> >>>>> >>>>>> >>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 >>>>> >>>>>> >>> while >>>>> >>>>>> >>> >>>>> >>>>>> >>> additional hardening and testing is done. We can communicate >>>>> >>>>>> >>> this in a blog >>>>> >>>>>> >>> >>>>> >>>>>> >>> post., >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> To make this much more palatable to our use community, if we >>>>> >>>>>> >>> can get a >>>>> >>>>>> >>> >>>>> >>>>>> >>> build and docker image available ASAP with Accord, it will >>>>> >>>>>> >>> allow developers >>>>> >>>>>> >>> >>>>> >>>>>> >>> to start playing with the syntax. Up to this point, that >>>>> >>>>>> >>> hasn't been widely >>>>> >>>>>> >>> >>>>> >>>>>> >>> available unless you compile the code yourself. Developers >>>>> >>>>>> >>> need to >>>>> >>>>>> >>> >>>>> >>>>>> >>> understand how this will work in an application, and up to >>>>> >>>>>> >>> this point, the >>>>> >>>>>> >>> >>>>> >>>>>> >>> syntax is text they see in my slides. We need to get some >>>>> >>>>>> >>> hands-on and that >>>>> >>>>>> >>> >>>>> >>>>>> >>> will get our user community engaged on Accord this calendar >>>>> >>>>>> >>> year. The >>>>> >>>>>> >>> >>>>> >>>>>> >>> feedback may even uncover some critical changes we'll need to >>>>> >>>>>> >>> make. Lack of >>>>> >>>>>> >>> >>>>> >>>>>> >>> access to Accord by developers is a critical problem we can >>>>> >>>>>> >>> fix soon and >>>>> >>>>>> >>> >>>>> >>>>>> >>> there will be plenty of excitement there and start building >>>>> >>>>>> >>> use cases >>>>> >>>>>> >>> >>>>> >>>>>> >>> before the final code ships. >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> I'm bummed but realistic. It sucks that I won't have a pony >>>>> >>>>>> >>> for Christmas, >>>>> >>>>>> >>> >>>>> >>>>>> >>> but maybe one for my birthday? >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> Patrick >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie >>>>> >>>>>> >>> <jmcken...@apache.org> wrote: >>>>> >>>>>> >>> >>>>> >>>>>> >>> >>>>> >>>>>> >>>> Maybe it won't be a glamorous release but shipping >>>>> >>>>>> >>>> 5.0 mitigates our worst case scenario. >>>>> >>>>>> >>>> I disagree with this characterization of 5.0 personally. UCS, >>>>> >>>>>> >>>> SAI, Trie >>>>> >>>>>> >>>> memtables and sstables, maybe vector ANN if the sub-tasks on >>>>> >>>>>> >>>> C-18715 are >>>>> >>>>>> >>>> accurate, all combine to make 5.0 a pretty glamorous release >>>>> >>>>>> >>>> IMO >>>>> >>>>>> >>>> independent of TCM and Accord. Accord is a true >>>>> >>>>>> >>>> paradigm-shift game-changer >>>>> >>>>>> >>>> so it's easy to think of 5.0 as uneventful in comparison, and >>>>> >>>>>> >>>> TCM helps >>>>> >>>>>> >>>> resolve one of the biggest pain-points in our system for over >>>>> >>>>>> >>>> a decade, but >>>>> >>>>>> >>>> I think 5.0 is a very meaty release in its own right today. >>>>> >>>>>> >>>> Anyway - I agree with you Brandon re: timelines. If things >>>>> >>>>>> >>>> take longer >>>>> >>>>>> >>>> than we'd hope (which, if I think back, they do roughly 100% >>>>> >>>>>> >>>> of the time on >>>>> >>>>>> >>>> this project), blocking on these features could both lead to >>>>> >>>>>> >>>> a significant >>>>> >>>>>> >>>> delay in 5.0 going out as well as increasing pressure and >>>>> >>>>>> >>>> risk of burnout >>>>> >>>>>> >>>> on the folks working on it. While I believe we all need some >>>>> >>>>>> >>>> balanced >>>>> >>>>>> >>>> urgency to do our best work, being under the gun for >>>>> >>>>>> >>>> something with a hard >>>>> >>>>>> >>>> deadline or having an entire project drag along blocked on >>>>> >>>>>> >>>> you is not where >>>>> >>>>>> >>>> I want any of us to be. >>>>> >>>>>> >>>> Part of why we talked about going to primarily annual >>>>> >>>>>> >>>> calendar-based >>>>> >>>>>> >>>> releases was to avoid precisely this situation, where >>>>> >>>>>> >>>> something that >>>>> >>>>>> >>>> *feels* right at the cusp of merging leads us to delay a >>>>> >>>>>> >>>> release >>>>> >>>>>> >>>> repeatedly. We discussed this a couple times this year: >>>>> >>>>>> >>>> 1: >>>>> >>>>>> >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2F9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187370031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LsDhZ%2FEwjf2DfA6Fl9UqmGZAOqLMbEBoPhOAtWe4MYE%3D&reserved=0 >>>>> >>>>>> >>>> >>>>> >>>>>> >>>> <https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3>, >>>>> >>>>>> >>>> where Mick proposed a "soft-freeze" for everything w/out >>>>> >>>>>> >>>> exception and 1st >>>>> >>>>>> >>>> week October "hard-freeze", and there was assumed to be lazy >>>>> >>>>>> >>>> consensus >>>>> >>>>>> >>>> 2: >>>>> >>>>>> >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fmzj3dq8b7mzf60k6mkby88b9n9ywmsgw&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187374062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lVOtweysN%2FsUBxg9VmslQEKmJqRpd9vhmI%2FxeWEH1o0%3D&reserved=0 >>>>> >>>>>> >>>> >>>>> >>>>>> >>>> <https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw>, >>>>> >>>>>> >>>> where we kept along with what we discussed in 1 but added in >>>>> >>>>>> >>>> CEP-30 to be >>>>> >>>>>> >>>> waivered in as well. >>>>> >>>>>> >>>> So. We're at a crossroads here where we need to either follow >>>>> >>>>>> >>>> through with >>>>> >>>>>> >>>> what we all agreed to earlier this year, or acknowledge that >>>>> >>>>>> >>>> our best >>>>> >>>>>> >>>> intention of calendar-based releases can't stand up to our >>>>> >>>>>> >>>> optimism and >>>>> >>>>>> >>>> desire to get these features into the next major. >>>>> >>>>>> >>>> There's no immediate obvious better path to me in terms of >>>>> >>>>>> >>>> what's best for >>>>> >>>>>> >>>> our users. This is a situation of risk tolerance with a lot >>>>> >>>>>> >>>> of unknowns >>>>> >>>>>> >>>> that could go either way. >>>>> >>>>>> >>>> Any light that folks active on TCM and Accord could shed in >>>>> >>>>>> >>>> terms of their >>>>> >>>>>> >>>> best and worst-case scenarios on timelines for those features >>>>> >>>>>> >>>> might help us >>>>> >>>>>> >>>> narrow this down a bit. Otherwise, I'm inclined to defer to >>>>> >>>>>> >>>> our past selves >>>>> >>>>>> >>>> and fall back to "we agreed to yearly calendar releases for >>>>> >>>>>> >>>> good reason. >>>>> >>>>>> >>>> Let's stick to our guns." >>>>> >>>>>> >>>> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: >>>>> >>>>>> >>>> The concern I have with this is that is a big slippery 'if' >>>>> >>>>>> >>>> that >>>>> >>>>>> >>>> involves development time estimation, and if it tends to take >>>>> >>>>>> >>>> longer >>>>> >>>>>> >>>> than we estimate (as these things tend to do), then I can see >>>>> >>>>>> >>>> a future >>>>> >>>>>> >>>> where 5.0 is delayed until the middle of 2024, and I really >>>>> >>>>>> >>>> don't want >>>>> >>>>>> >>>> that to happen. Maybe it won't be a glamorous release but >>>>> >>>>>> >>>> shipping >>>>> >>>>>> >>>> 5.0 mitigates our worst case scenario. >>>>> >>>>>> >>>> Kind Regards, >>>>> >>>>>> >>>> Brandon >>>>> >>>>>> >>>> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi >>>>> >>>>>> >>>> <djo...@apache.org> wrote: >>>>> >>>>>> >>>>> I have a strong preference to move out the 5.0 date to have >>>>> >>>>>> >>>>> accord and >>>>> >>>>>> >>>> TCM. I don’t see the point in shipping 5.0 without these >>>>> >>>>>> >>>> features >>>>> >>>>>> >>>> especially if 5.1 is going to follow close behind it. >>>>> >>>>>> >>>>> Dinesh >>>>> >>>>>> >>>>> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever >>>>> >>>>>> >>>>> <m...@apache.org> wrote: >>>>> >>>>>> >>>>>  >>>>> >>>>>> >>>>> The TCM work (CEP-21) is in its review stage but being well >>>>> >>>>>> >>>>> past our >>>>> >>>>>> >>>> cut-off date¹ for merging, and now jeopardising 5.0 GA >>>>> >>>>>> >>>> efforts, I would >>>>> >>>>>> >>>> like to propose the following. >>>>> >>>>>> >>>>> We merge TCM and Accord only to trunk. Then branch >>>>> >>>>>> >>>>> cassandra-5.1 and >>>>> >>>>>> >>>> cut an immediate 5.1-alpha1 release. >>>>> >>>>>> >>>>> I see this as a win-win scenario for us, considering our >>>>> >>>>>> >>>>> current >>>>> >>>>>> >>>> situation. (Though it is unfortunate that Accord is included >>>>> >>>>>> >>>> in this >>>>> >>>>>> >>>> scenario because we agreed it to be based upon TCM.) >>>>> >>>>>> >>>>> This will mean… >>>>> >>>>>> >>>>> - We get to focus on getting 5.0 to beta and GA, which >>>>> >>>>>> >>>>> already has a >>>>> >>>>>> >>>> ton of features users want. >>>>> >>>>>> >>>>> - We get an alpha release with TCM and Accord into users >>>>> >>>>>> >>>>> hands quickly >>>>> >>>>>> >>>> for broader testing and feedback. >>>>> >>>>>> >>>>> - We isolate GA efforts on TCM and Accord – giving oss and >>>>> >>>>>> >>>>> downstream >>>>> >>>>>> >>>> engineers time and patience reviewing and testing. TCM will >>>>> >>>>>> >>>> be the biggest >>>>> >>>>>> >>>> patch ever to land in C*. >>>>> >>>>>> >>>>> - Give users a choice for a more incremental upgrade >>>>> >>>>>> >>>>> approach, given >>>>> >>>>>> >>>> just how many new features we're putting on them in one year. >>>>> >>>>>> >>>>> - 5.1 w/ TCM and Accord will maintain its upgrade >>>>> >>>>>> >>>>> compatibility with >>>>> >>>>>> >>>> all 4.x versions, just as if it had landed in 5.0. >>>>> >>>>>> >>>>> The risks/costs this introduces are >>>>> >>>>>> >>>>> - If we cannot stabilise TCM and/or Accord on the >>>>> >>>>>> >>>>> cassandra-5.1 branch, >>>>> >>>>>> >>>> and at some point decide to undo this work, while we can >>>>> >>>>>> >>>> throw away the >>>>> >>>>>> >>>> cassandra-5.1 branch we would need to do a bit of work >>>>> >>>>>> >>>> reverting the >>>>> >>>>>> >>>> changes in trunk. This is a _very_ edge case, as confidence >>>>> >>>>>> >>>> levels on the >>>>> >>>>>> >>>> design and implementation of both are already tested and high. >>>>> >>>>>> >>>>> - We will have to maintain an additional branch. I >>>>> >>>>>> >>>>> propose that we >>>>> >>>>>> >>>> treat the 5.1 branch in the same maintenance window as 5.0 >>>>> >>>>>> >>>> (like we have >>>>> >>>>>> >>>> with 3.0 and 3.11). This also adds the merge path overhead. >>>>> >>>>>> >>>>> - Reviewing of TCM and Accord will continue to happen >>>>> >>>>>> >>>>> post-merge. This >>>>> >>>>>> >>>> is not our normal practice, but this work will have already >>>>> >>>>>> >>>> received its >>>>> >>>>>> >>>> two +1s from committers, and such ongoing review effort is >>>>> >>>>>> >>>> akin to GA >>>>> >>>>>> >>>> stabilisation work on release branches. >>>>> >>>>>> >>>>> I see no other ok solution in front of us that gets us at >>>>> >>>>>> >>>>> least both the >>>>> >>>>>> >>>> 5.0 beta and TCM+Accord alpha releases this year. Keeping in >>>>> >>>>>> >>>> mind users >>>>> >>>>>> >>>> demand to start experimenting with these features, and our >>>>> >>>>>> >>>> Cassandra Summit >>>>> >>>>>> >>>> in December. >>>>> >>>>>> >>>>> 1) >>>>> >>>>>> >>>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2F9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cc811f6a430d1466acc3f08dbd61639c2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638339163187377903%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=x67jPkkAhNQHdbY%2FAXTaBr42stNwpA2NHdyckR%2F%2FQCI%3D&reserved=0 >>>>> >>>>>> >>>>> >>>>> >>>>>> >>>>> <https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3> >>>>> >>>>>> >>> >>>>> >>>>>> >>> >> >
