I defer to the judgement of the folks that are most impacted by it - ones that are in the code, working on the next release. If you all think it's good to merge, then I am 100% in support of it. I suspect merging will help get it out faster, and I don't see any future in which we don't ship this in the next release.
I will be happy to help answer the "how does it compare to paxos v2" question post-merge. Jon On Fri, Mar 7, 2025 at 5:52 AM Josh McKenzie <jmcken...@apache.org> wrote: > 3.5 years is an incredible amount of time and work; it really is > significant and thanks to everyone involved for the investment of time and > energy. > > We have a rocky history with large, disruptive contributions in the past > that have either blocked forward progress post-merge (CASSANDRA-8099), or > lingered in the code-base increasing maintenance burden on other > contributors for minimal or no user benefit (early open post SSD > transition, witness replicas, materialized views). I'm sympathetic to where > Jordan's questions stem from, as our history of leaving things in the > codebase long after they've become vestigial or abandoned has slowed down > our collective momentum maintaining the project on actively used features. > > That said, I don't think Accord will run afoul of some of those same > patterns. Aside from the degree of investment already in it and sheer > number of pmc members and committers involved, I believe it's a feature > that's universally impactful and that if we had a metaphorical bus-factor > change (entire group of people working on it disappeared the day after > merge or decided to go on vacation for 5 years), others in the community > would be willing to pick things up and keep it moving given its proximity > to release readiness. > > The 2 questions Jordan asked resonate with me: 1) do we have line of sight > to a fix on the schema issues, and I'll take the liberty of reframing 2) do > we have line of sight to improvement on the performance front to be usable > for multi-key transactions? (subtle: I don't think "parity with PaxosV2" is > the right target, but rather "fast enough to be usable for multi-key > transactions" since it's a new query paradigm). > > Given the context on contributor backing and if the answer is yes to those > 2 questions (which I believe it is), I think we should generally be > comfortable with merging the feature as experimental at this time. > > On Fri, Mar 7, 2025, at 12:54 AM, Benedict wrote: > > > There are essentially three possible timelines to choose from here: > > 1) We agree in the next few days to merge to trunk. We will then > prioritise rebasing onto trunk and resolving any pre-merge items starting > next week. > 2) There’s some more debate and agreement to merge to trunk in a week or > two. In the meantime we will shift to internal-first development but we’ll > likely prioritise the above work as soon as we can, which may be in a few > weeks, so we can shift to trunk first development. > 3) We don’t agree to merge accord anytime soon, so we shift to > internal-first development for the time being. I’m not sure when we will > prioritise any of the above. > > Our resources are finite and we’ve exhausted them (literally), so it’s > pretty much pick one of the above. I don’t really mind which you pick, but > I won’t personally be prioritising merge after this third attempt. > > > On 6 Mar 2025, at 22:01, Jon Haddad <j...@rustyrazorblade.com> wrote: > > > Hmm... I took a look at the cep-15-accord branch in GitHub, it looks like > it's several hundred commits behind trunk. Since you'll need to rebase > again before merge *anyways*, would it make sense to do it once more, and I > can publish easy-cass-lab with the latest branch? If folks have concerns, > it's easy to fire up a cluster (I do it constantly) and try it out. > > I think if we were to do this, out of consideration we should time box the > amount of time for an evaluation and unless someone raises an objection, > consider lazy consensus achieved. > > Jon > > > > On Thu, Mar 6, 2025 at 12:46 PM Benedict Elliott Smith < > bened...@apache.org> wrote: > > Because we want to validate against the latest code in trunk, else we are > validating stale behaviours. The cost of rebasing is high, so we do not do > it frequently. That means we will likely stop developing OSS-first, as the > focus will have to move to our internal branch that satisfies these > criteria. > > Exactly what this might be for upstreaming I cannot say. Personally, I aim > to work exclusively on the branch we are stabilising. If that is not trunk, > the latency for my contributions being made public might be high, as I have > a huge imbalance of over-investment to recoup, and anything unnecessary > will be deferred. > > Since the feature is disabled, and the code is almost entirely isolated, I > cannot imagine the cost to the community to removing this work would be > very high. But, I do not intend to argue Accord’s case here. I will let you > all decide. > > Please decide soon though, as it shapes our work planning. The positive > reception so far had lead me to consider prioritising a move to trunk-first > development within the next week or two, and the associated work that > entails. However, if that was optimistic we will have to shift our plans. > > > > On 6 Mar 2025, at 20:16, Jordan West <jw...@apache.org> wrote: > > The work and effort in accord has been amazing. And I’m sure it sets a new > standard for code quality and correctness testing which I’m also entirely > behind. I also trust the folks working on it want to take it to the a fully > production ready solution. But I’m worried about circumstances out of our > control leaving us with a very complex feature that isn’t complete. > > I do have some questions. Could folks help me better understand why > testing real workloads necessitates a merge (my understanding from the > original reason is this is the impetus for why we would merge now)? Also I > think the performance and scheme change caveats are rather large ones. One > of accords promise was better performance and I think making schema changes > with nodes down not being supported is a big gap. Could we have some > criteria like “supports all the operations PaxosV2 supports” or “performs > as well or better than PaxosV2 on [workload(s)]”? > > I understand waiting asks a lot of the authors in terms of baring the > burden of a more complex merge. But I think we also need to consider what > merging is asking the community to bear if the worst happens and we are > unable to take the feature from its current state to something that can be > widely used in production. > > Jordan > > > On Wed, Mar 5, 2025 at 15:52 Blake Eggleston <bl...@ultrablake.com> wrote: > > > +1 to merging it > > On Wed, Mar 5, 2025, at 12:22 PM, Patrick McFadin wrote: > > You have my +1 > > On Wed, Mar 5, 2025 at 12:16 PM Benedict <bened...@apache.org> wrote: > > > > Correct, these caveats should only apply to tables that have opted-in to > accord. > > > > On 5 Mar 2025, at 20:08, Jeremiah Jordan <jerem...@apache.org> wrote: > > > > > > So great to see all this hard work about to pay off! > > > > On the questions/concerns front, the only concern I would have towards > merging this to trunk is if any of the caveats apply when someone is not > using Accord. Assuming they only apply when the feature flag is enabled, I > see no reason not to get this merged into trunk once everyone involved is > happy with the state of it. > > > > -Jeremiah > > > > On Mar 5, 2025 at 12:15:23 PM, Benedict Elliott Smith < > bened...@apache.org> wrote: > >> > >> That depends on all of you lovely people :D > >> > >> I think we should have finished merging everything we want before QA by > ~Monday; certainly not much later. > >> > >> I think we have some upgrade and python dtest failures to address as > well. > >> > >> So it could be pretty soon if the community is supportive. > >> > >> On 5 Mar 2025, at 17:22, Patrick McFadin <pmcfa...@gmail.com> wrote: > >> > >> > >> What is the timing for starting the merge process? I'm asking because > >> > >> I have (yet another) presentation and this would be a cool update. > >> > >> > >> On Wed, Mar 5, 2025 at 1:22 AM Benedict Elliott Smith > >> > >> <bened...@apache.org> wrote: > >> > >> > > >> > >> > Thanks everyone. > >> > >> > > >> > >> > Jon - your help will be greatly appreciated. We’ll let you know when > we’ve got the cycles to invest in performance work (hopefully fairly soon). > I expect the first step will be improving visibility so we can better > understand what the system is doing (particularly the caching layers), but > we can dig in together when ready. > >> > >> > > >> > >> > On 4 Mar 2025, at 18:15, Jon Haddad <j...@rustyrazorblade.com> wrote: > >> > >> > > >> > >> > Very exciting! > >> > >> > > >> > >> > I have a client that's very interested in Accord, so I should have > budget to dig into it, especially on the performance side of things. > >> > >> > > >> > >> > Jon > >> > >> > > >> > >> > On Tue, Mar 4, 2025 at 9:57 AM Dmitry Konstantinov < > netud...@gmail.com> wrote: > >> > >> >> > >> > >> >> Thank you to all Accord and TCM contributors, it is really exciting > to see a development of such huge and wonderful features moving forward and > opening the door to the new Cassandra epoch! > >> > >> >> > >> > >> >> On Tue, 4 Mar 2025 at 20:45, Blake Eggleston <bl...@ultrablake.com> > wrote: > >> > >> >>> > >> > >> >>> Thanks Benedict! > >> > >> >>> > >> > >> >>> I’m really excited to see accord reach this milestone, even with > these caveats. You seem to have left yourself off the list of contributors > though, even though you’ve been a central figure in its development :) So > thanks to all accord & tcm contributors, including Benedict, for making > this possible! > >> > >> >>> > >> > >> >>> On Tue, Mar 4, 2025, at 8:00 AM, Benedict Elliott Smith wrote: > >> > >> >>> > >> > >> >>> Hi everyone, > >> > >> >>> > >> > >> >>> It’s been exactly 3.5 years since the first commit to > cassandra-accord. Yes, really, it’s been that long. > >> > >> >>> > >> > >> >>> We will be starting to validate the feature against real workloads > in the near future, so we can’t sensibly push off merging much longer. The > following is a brief run-down of the state of play. There are no known > bugs, but there remain a number of caveats we will be incrementally > addressing in the run-up to a full release: > >> > >> >>> > >> > >> >>> [1] Accord is likely to be SLOW until further optimisations are > implemented > >> > >> >>> [2] Schema changes have a number of hard edges > >> > >> >>> [3] Validation is ongoing, so there are likely still a number of > bugs to shake out > >> > >> >>> [4] Many operator visibility/tooling/documentation improvements are > pending > >> > >> >>> > >> > >> >>> To expand a little: > >> > >> >>> > >> > >> >>> [1] As of the last experiment we conducted, accord’s throughput was > poor - also leading to higher LAN latencies. We have done no WAN > experiments to date, but the protocol guarantees should already achieve > better round-trip performance, in particular under contention. Improving > throughput will be the main focus of attention once we are satisfied the > protocol is otherwise stable, but our focus remains validation for the > moment. > >> > >> >>> [2] Schema changes have not yet been well integrated with TCM. > Dropping a table for instance will currently cause problems if nodes are > offline. > >> > >> >>> [3] We have a range of validations we are already performing > against cassandra-accord directly, and against its integration with > Cassandra in cep-15-accord. We have run hundreds of billions of simulated > transactions, and are still discovering some minor fault every few billion > simulated transactions or so. There remains a lot more simulated validation > to explore, as well as with real clusters serving real workloads. > >> > >> >>> [4] There are already a range of virtual tables for exploring > internal state in Accord, and reasonably good metric support. However, > tracing is not yet supported, and our metric and virtual table integrations > need some further development. > >> > >> >>> [5] There are also other edge cases to address such as ensuring we > do not reuse HLCs after restart, supporting ByteOrderPartitioner, and live > migration from/to Paxos is undergoing fine-tuning and validation; probably > there are some other things I am forgetting. > >> > >> >>> > >> > >> >>> Altogether the feature is fairly mature, despite these caveats. > This is the fruit of the labour of a long list of contributors, including > Aleksey Yeschenko, Alex Petrov, Ariel Weisberg, Blake Eggleston, Caleb > Rackliffe and David Capwell, and represents a huge undertaking. It also > wouldn’t have been possible without the work of Alex Petrov, Marcus > Eriksson and Sam Tunnicliffe on delivering transactional cluster metadata. > I hope you will join me in thanking them all for their contributions. > >> > >> >>> > >> > >> >>> Alex has also kindly produced some initial overview documentation > for developers, that can be found here: > https://github.com/apache/cassandra/blob/cep-15-accord/doc/modules/cassandra/pages/developing/accord/index.adoc. > This will be expanded as time permits. > >> > >> >>> > >> > >> >>> Does anyone have any questions or concerns? > >> > >> >>> > >> > >> >>> > >> > >> >> > >> > >> >> > >> > >> >> -- > >> > >> >> Dmitry Konstantinov > >> > >> > > >> > >> > > >> > >> > > > >
