> > If we do a 5.1 release why not take it as an opportunity to release more > things. I am not saying that we will. Just that we should let that door > open. >
Agreed. This is the reason I brought up the possibility of not branching off 5.1 immediately. On Oct 25, 2023 at 3:17:13 AM, Benjamin Lerer <b.le...@gmail.com> wrote: > The proposal includes 3 things: > 1. Do not include TCM and Accord in 5.0 to avoid delaying 5.0 > 2. The next release will be 5.1 and will include only Accord and TCM > 3. Merge TCM and Accord right now in 5.1 (making an initial release) > > I am fine with question 1 and do not have a strong opinion on which way to > go. > 2. Means that every new feature will have to wait for post 5.1 even if it > is ready before 5.1 is stabilized and shipped. If we do a 5.1 release why > not take it as an opportunity to release more things. I am not saying that > we will. Just that we should let that door open. > 3. There is a need to merge TCM and Accord as maintaining those separate > branches is costly in terms of time and energy. I fully understand that. On > the other hand merging TCM and Accord will make the TCM review harder and I > do believe that this second round of review is valuable as it already > uncovered a valid issue. Nevertheless, I am fine with merging TCM as soon > as it passes CI and continuing the review after the merge. If we cannot > meet at least that quality level (Green CI) we should not merge just for > creating an 5.1.alpha release for the summit. > > Now, I am totally fine with a preview release without numbering and with > big warnings that will only serve as a preview for the summit. > > Le mer. 25 oct. 2023 à 06:33, Berenguer Blasi <berenguerbl...@gmail.com> > a écrit : > >> I also think there's many good new features in 5.0 already they'd make a >> good release even on their own. My 2 cts. >> >> On 24/10/23 23:20, Brandon Williams wrote: >> > The catch here is that we don't publish docker images currently. The >> > C* docker images available are not made by us. >> > >> > Kind Regards, >> > Brandon >> > >> > On Tue, Oct 24, 2023 at 3:31 PM Patrick McFadin <pmcfa...@gmail.com> >> wrote: >> >> Let me make that really easy. Hell yes >> >> >> >> Not everybody runs CCM, I've tried but I've met resistance. >> >> >> >> Compiling your own version usually involves me saying the words "Yes, >> ant realclean exists. I'm not trolling you" >> >> >> >> docker pull <image> works on every OS and curates a single node >> experience. >> >> >> >> >> >> >> >> On Tue, Oct 24, 2023 at 12:37 PM Josh McKenzie <jmcken...@apache.org> >> wrote: >> >>> In order for the project to advertise the release outside the dev@ >> list it needs to be a formal release. >> >>> >> >>> That's my reading as well: >> >>> https://www.apache.org/legal/release-policy.html#release-definition >> >>> >> >>> I wonder if there'd be value in us having a cronned job that'd do >> nightly docker container builds on trunk + feature branches, archived for N >> days, and we make that generally known to the dev@ list here so folks >> that want to poke at the current state of trunk or other branches could do >> so with very low friction. We'd probably see more engagement on feature >> branches if it was turn-key easy for other C* devs to spin the up and check >> them out. >> >>> >> >>> For what you're talking about here Patrick (a docker image for folks >> outside the dev@ audience and more user-facing), we'd want to vote on it >> and go through the formal process. >> >>> >> >>> On Tue, Oct 24, 2023, at 3:10 PM, Jeremiah Jordan wrote: >> >>> >> >>> In order for the project to advertise the release outside the dev@ >> list it needs to be a formal release. That just means that there was a >> release vote and at least 3 PMC members +1’ed it, and there are more +1 >> than there are -1, and we follow all the normal release rules. The ASF >> release process doesn’t care what branch you cut the artifacts from or what >> version you call it. >> >>> >> >>> So the project can cut artifacts for and release a 5.1-alpha1, >> 5.1-dev-preview1, what ever we want to version this thing, from trunk or >> any other branch name we want. >> >>> >> >>> -Jeremiah >> >>> >> >>> On Oct 24, 2023 at 2:03:41 PM, Patrick McFadin <pmcfa...@gmail.com> >> wrote: >> >>> >> >>> I would like to have something for developers to use ASAP to try the >> Accord syntax. Very few people have seen it, and I think there's a learning >> curve we can start earlier. >> >>> >> >>> It's my understanding that ASF policy is that it needs to be a >> project release to create a docker image. >> >>> >> >>> On Tue, Oct 24, 2023 at 11:54 AM Jeremiah Jordan < >> jeremiah.jor...@gmail.com> wrote: >> >>> >> >>> If we decide to go the route of not merging TCM to the 5.0 branch. >> Do we actually need to immediately cut a 5.1 branch? Can we work on >> stabilizing things while it is in trunk and cut the 5.1 branch when we >> actually think we are near releasing? I don’t see any reason we can not >> cut “preview” artifacts from trunk? >> >>> >> >>> -Jeremiah >> >>> >> >>> On Oct 24, 2023 at 11:54:25 AM, Jon Haddad < >> rustyrazorbl...@apache.org> wrote: >> >>> >> >>> I guess at the end of the day, shipping a release with a bunch of >> awesome features is better than holding it back. If there's 2 big releases >> in 6 months the community isn't any worse off. >> >>> >> >>> We either ship something, or nothing, and something is probably >> better. >> >>> >> >>> Jon >> >>> >> >>> >> >>> On 2023/10/24 16:27:04 Patrick McFadin wrote: >> >>> >> >>> +1 to what you are saying, Josh. Based on the last survey, yes, >> everyone >> >>> >> >>> was excited about Accord, but SAI and UCS were pretty high on the >> list. >> >>> >> >>> >> >>> Benedict and I had a good conversation last night, and now I >> understand >> >>> >> >>> more essential details for this conversation. TCM is taking far more >> work >> >>> >> >>> than initially scoped, and Accord depends on a stable TCM. TCM is >> months >> >>> >> >>> behind and that's a critical fact, and one I personally just learned >> of. I >> >>> >> >>> thought things were wrapping up this month, and we were in the testing >> >>> >> >>> phase. I get why that's a topic we are dancing around. Nobody wants >> to say >> >>> >> >>> ship dates are slipping because that's part of our culture. It's >> >>> >> >>> disappointing and, if new information, an unwelcome surprise, but >> none of >> >>> >> >>> us should be angry or in a blamey mood because I guarantee every one >> of us >> >>> >> >>> has shipped the code late. My reaction yesterday was based on an >> incorrect >> >>> >> >>> assumption. Now that I have a better picture, my point of view is >> changing. >> >>> >> >>> >> >>> Josh's point about what's best for users is crucial. Users deserve >> stable >> >>> >> >>> code with a regular cadence of features that make their lives easier. >> If we >> >>> >> >>> put 5.0 on hold for TCM + Accord, users will get neither for a very >> long >> >>> >> >>> time. And I mentioned a disaster yesterday. A bigger disaster would be >> >>> >> >>> shipping Accord with a major bug that causes data loss, eroding >> community >> >>> >> >>> trust. Accord has to be the most bulletproof of all bulletproof >> features. >> >>> >> >>> The pressure to ship is only going to increase and that's fertile >> ground >> >>> >> >>> for that sort of bug. >> >>> >> >>> >> >>> So, taking a step back and with a clearer picture, I support the 5.0 >> + 5.1 >> >>> >> >>> plan mainly because I don't think 5.1 is (or should be) a fast follow. >> >>> >> >>> >> >>> For the user community, the communication should be straightforward. >> TCM + >> >>> >> >>> Accord are turning out to be much more complicated than was originally >> >>> >> >>> scoped, and for good reasons. Our first principle is to provide a >> stable >> >>> >> >>> and reliable system, so as a result, we'll be de-coupling TCM + >> Accord from >> >>> >> >>> 5.0 into a 5.1 branch, which is available in parallel to 5.0 while >> >>> >> >>> additional hardening and testing is done. We can communicate this in >> a blog >> >>> >> >>> post., >> >>> >> >>> >> >>> To make this much more palatable to our use community, if we can get a >> >>> >> >>> build and docker image available ASAP with Accord, it will allow >> developers >> >>> >> >>> to start playing with the syntax. Up to this point, that hasn't been >> widely >> >>> >> >>> available unless you compile the code yourself. Developers need to >> >>> >> >>> understand how this will work in an application, and up to this >> point, the >> >>> >> >>> syntax is text they see in my slides. We need to get some hands-on >> and that >> >>> >> >>> will get our user community engaged on Accord this calendar year. The >> >>> >> >>> feedback may even uncover some critical changes we'll need to make. >> Lack of >> >>> >> >>> access to Accord by developers is a critical problem we can fix soon >> and >> >>> >> >>> there will be plenty of excitement there and start building use cases >> >>> >> >>> before the final code ships. >> >>> >> >>> >> >>> I'm bummed but realistic. It sucks that I won't have a pony for >> Christmas, >> >>> >> >>> but maybe one for my birthday? >> >>> >> >>> >> >>> Patrick >> >>> >> >>> >> >>> >> >>> >> >>> On Tue, Oct 24, 2023 at 7:23 AM Josh McKenzie <jmcken...@apache.org> >> wrote: >> >>> >> >>> >> >>>> Maybe it won't be a glamorous release but shipping >> >>>> 5.0 mitigates our worst case scenario. >> >>>> I disagree with this characterization of 5.0 personally. UCS, SAI, >> Trie >> >>>> memtables and sstables, maybe vector ANN if the sub-tasks on C-18715 >> are >> >>>> accurate, all combine to make 5.0 a pretty glamorous release IMO >> >>>> independent of TCM and Accord. Accord is a true paradigm-shift >> game-changer >> >>>> so it's easy to think of 5.0 as uneventful in comparison, and TCM >> helps >> >>>> resolve one of the biggest pain-points in our system for over a >> decade, but >> >>>> I think 5.0 is a very meaty release in its own right today. >> >>>> Anyway - I agree with you Brandon re: timelines. If things take >> longer >> >>>> than we'd hope (which, if I think back, they do roughly 100% of the >> time on >> >>>> this project), blocking on these features could both lead to a >> significant >> >>>> delay in 5.0 going out as well as increasing pressure and risk of >> burnout >> >>>> on the folks working on it. While I believe we all need some balanced >> >>>> urgency to do our best work, being under the gun for something with >> a hard >> >>>> deadline or having an entire project drag along blocked on you is >> not where >> >>>> I want any of us to be. >> >>>> Part of why we talked about going to primarily annual calendar-based >> >>>> releases was to avoid precisely this situation, where something that >> >>>> *feels* right at the cusp of merging leads us to delay a release >> >>>> repeatedly. We discussed this a couple times this year: >> >>>> 1: https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3, >> >>>> where Mick proposed a "soft-freeze" for everything w/out exception >> and 1st >> >>>> week October "hard-freeze", and there was assumed to be lazy >> consensus >> >>>> 2: https://lists.apache.org/thread/mzj3dq8b7mzf60k6mkby88b9n9ywmsgw, >> >>>> where we kept along with what we discussed in 1 but added in CEP-30 >> to be >> >>>> waivered in as well. >> >>>> So. We're at a crossroads here where we need to either follow >> through with >> >>>> what we all agreed to earlier this year, or acknowledge that our best >> >>>> intention of calendar-based releases can't stand up to our optimism >> and >> >>>> desire to get these features into the next major. >> >>>> There's no immediate obvious better path to me in terms of what's >> best for >> >>>> our users. This is a situation of risk tolerance with a lot of >> unknowns >> >>>> that could go either way. >> >>>> Any light that folks active on TCM and Accord could shed in terms of >> their >> >>>> best and worst-case scenarios on timelines for those features might >> help us >> >>>> narrow this down a bit. Otherwise, I'm inclined to defer to our past >> selves >> >>>> and fall back to "we agreed to yearly calendar releases for good >> reason. >> >>>> Let's stick to our guns." >> >>>> On Tue, Oct 24, 2023, at 9:37 AM, Brandon Williams wrote: >> >>>> The concern I have with this is that is a big slippery 'if' that >> >>>> involves development time estimation, and if it tends to take longer >> >>>> than we estimate (as these things tend to do), then I can see a >> future >> >>>> where 5.0 is delayed until the middle of 2024, and I really don't >> want >> >>>> that to happen. Maybe it won't be a glamorous release but shipping >> >>>> 5.0 mitigates our worst case scenario. >> >>>> Kind Regards, >> >>>> Brandon >> >>>> On Mon, Oct 23, 2023 at 4:02 PM Dinesh Joshi <djo...@apache.org> >> wrote: >> >>>>> I have a strong preference to move out the 5.0 date to have accord >> and >> >>>> TCM. I don’t see the point in shipping 5.0 without these features >> >>>> especially if 5.1 is going to follow close behind it. >> >>>>> Dinesh >> >>>>> On Oct 23, 2023, at 4:52 AM, Mick Semb Wever <m...@apache.org> >> wrote: >> >>>>> >> >>>>> The TCM work (CEP-21) is in its review stage but being well past our >> >>>> cut-off date¹ for merging, and now jeopardising 5.0 GA efforts, I >> would >> >>>> like to propose the following. >> >>>>> We merge TCM and Accord only to trunk. Then branch cassandra-5.1 >> and >> >>>> cut an immediate 5.1-alpha1 release. >> >>>>> I see this as a win-win scenario for us, considering our current >> >>>> situation. (Though it is unfortunate that Accord is included in this >> >>>> scenario because we agreed it to be based upon TCM.) >> >>>>> This will mean… >> >>>>> - We get to focus on getting 5.0 to beta and GA, which already >> has a >> >>>> ton of features users want. >> >>>>> - We get an alpha release with TCM and Accord into users hands >> quickly >> >>>> for broader testing and feedback. >> >>>>> - We isolate GA efforts on TCM and Accord – giving oss and >> downstream >> >>>> engineers time and patience reviewing and testing. TCM will be the >> biggest >> >>>> patch ever to land in C*. >> >>>>> - Give users a choice for a more incremental upgrade approach, >> given >> >>>> just how many new features we're putting on them in one year. >> >>>>> - 5.1 w/ TCM and Accord will maintain its upgrade compatibility >> with >> >>>> all 4.x versions, just as if it had landed in 5.0. >> >>>>> The risks/costs this introduces are >> >>>>> - If we cannot stabilise TCM and/or Accord on the cassandra-5.1 >> branch, >> >>>> and at some point decide to undo this work, while we can throw away >> the >> >>>> cassandra-5.1 branch we would need to do a bit of work reverting the >> >>>> changes in trunk. This is a _very_ edge case, as confidence levels >> on the >> >>>> design and implementation of both are already tested and high. >> >>>>> - We will have to maintain an additional branch. I propose that >> we >> >>>> treat the 5.1 branch in the same maintenance window as 5.0 (like we >> have >> >>>> with 3.0 and 3.11). This also adds the merge path overhead. >> >>>>> - Reviewing of TCM and Accord will continue to happen >> post-merge. This >> >>>> is not our normal practice, but this work will have already received >> its >> >>>> two +1s from committers, and such ongoing review effort is akin to GA >> >>>> stabilisation work on release branches. >> >>>>> I see no other ok solution in front of us that gets us at least >> both the >> >>>> 5.0 beta and TCM+Accord alpha releases this year. Keeping in mind >> users >> >>>> demand to start experimenting with these features, and our Cassandra >> Summit >> >>>> in December. >> >>>>> 1) https://lists.apache.org/thread/9c5cnn57c7oqw8wzo3zs0dkrm4f17lm3 >> >>> >> >>> >> >
