Hi Dave, I appreciate these performance/cost considerations and I believe these should be taken into account when evaluating default changes.
I am trying to frame this as an usability issue with the database by shipping with STCS by default. I think it's possible to classify workloads into two types: a) Mutable (superset) b) Immutable (or semi-immutable) The majority of current use cases might be b) Immutable, but shipping with STCS provides a bad user experience to users of a) Mutable use cases. In turn, this reinforces that "cassandra is not good for mutable use cases". I believe the use cases that will be covered by CQL Transactions tend to be a) mutable, and it might make sense to optimize to this reality. Existing users of immutable use cases are familiar with STCS and can remain using this choice. Thanks, Paulo On Sun, Dec 8, 2024 at 10:33 AM Dave Herrington <he...@rhinosource.com> wrote: > …the analysis I describe would need to be weighted by table size. I have > several representative production cluster tablestats analyses that show r:w > ratio by table, including table size. I can check to see how this analysis > plays out on a few of these. > > -Dave > > David A. Herrington II > President and Chief Engineer > RhinoSource, Inc. > > *Data Lake Architecture, Cloud Computing and Advanced Analytics.* > > www.rhinosource.com > > > On Sun, Dec 8, 2024 at 7:22 AM Dave Herrington <he...@rhinosource.com> > wrote: > >> Paulo, >> >> I understand your perspective. >> >> Short of waiting for UCS to prove itself out, I guess it comes down to >> the assertion that a strong majority of Cassandra use cases would benefit >> from using LCS vs. STCS. >> >> The conventional wisdom is that workloads need to be read-heavy to make >> the extra resource consumption of LCS pay off. 4:1 read:write is the >> threshold I use to decide whether or not to use LCS. >> >> I think this ratio is important in this analysis. Has this LCS “payoff” >> threshold changed to 2:1 or better, in favor of LCS? This would be good to >> know. >> >> With an up-to-date threshold in hand, what is the fraction of Cassandra >> use cases that meet this updatedthreshold? >> >> For example, say this LCS payoff r:w ratio has improved to 2:1. What >> percentage of Cassandra tables across all clusters currently in operation >> are 2:1 read-to-write or more? >> >> If the answer is a solid majority, I think this would justify the default >> change. >> >> -Dave >> >> David A. Herrington II >> President and Chief Engineer >> RhinoSource, Inc. >> >> *Data Lake Architecture, Cloud Computing and Advanced Analytics.* >> >> www.rhinosource.com >> >> >> On Sun, Dec 8, 2024 at 5:43 AM Paulo Motta <pa...@apache.org> wrote: >> >>> Hi Dave, >>> >>> I'm also in the field and my experience is different. >>> >>> I have seen new users shooting themselves in the foot with the default >>> compaction strategy STCS on a regular basis over the past few years and >>> have been recommending them to switch to LCS and they no longer encounter >>> issues after making this switch. I would like to generalize this >>> recommendation to prevent new users from having bad experiences and >>> abandoning the database. >>> >>> This is not a cost issue, it's an ease of use matter. STCS does not work >>> for mutable workloads and this is a massive functional limitation with the >>> database. >>> >>> I don't want people to download Cassandra 5.1 to try out transactions >>> and start facing issues due to bad STCS performance on mutable data. >>> >>> If you would like to optimize for cost, then you can read the docs or >>> hire a consultant to optimize the cost for you. Otherwise, the database >>> should work out of the box and this is provided by LCS. If LCS can not keep >>> up, it means the cluster is under provisioned and needs to be expanded, >>> it's not a functional issue but a capacity issue. >>> >>> Cheers, >>> >>> Paulo >>> >>> On Sun, Dec 8, 2024 at 1:26 AM Dave Herrington <he...@rhinosource.com> >>> wrote: >>> >>>> Chiming in from the field, I think maintaining the familiar status quo >>>> until a panacea compaction strategy proves itself out (could that be UCS?) >>>> makes sense to me. I feel it could be maddening to customers if LCS >>>> started showing up in schemas after an upgrade just because the default >>>> changed. If UCS proves itself as the fits-all solution, then we’d be doing >>>> them a favor by making the default. In time. >>>> >>>> -Dave >>>> >>>> David A. Herrington II >>>> President and Chief Engineer >>>> RhinoSource, Inc. >>>> >>>> *Data Lake Architecture, Cloud Computing and Advanced Analytics.* >>>> >>>> www.rhinosource.com >>>> >>>> >>>> On Sat, Dec 7, 2024 at 7:32 PM Jeff Jirsa <jji...@gmail.com> wrote: >>>> >>>>> >>>>> >>>>> On Dec 7, 2024, at 7:08 PM, Mick Semb Wever <m...@apache.org> wrote: >>>>> >>>>> Chiming in with my two cents… >>>>> >>>>> >>>>> When people have the luxury of working in environments where clusters >>>>>> are massively over provisioned, LCS as a default makes a lot of sense, >>>>>> because there's not much downside. The use cases where you'd actually >>>>>> fall >>>>>> behind in compaction are pretty slim, so the negative impact isn't felt. >>>>>> >>>>>> Most people aren't doing this. Putting LCS as the default >>>>>> significantly changes the performance profile of new clusters in a way >>>>>> that >>>>>> actively harms a portion of the community. >>>>>> >>>>> >>>>> >>>>> Haddad's statement here resonates above everything else that's been >>>>> said so far. It is this particular audience that I'm thinking first about >>>>> not screwing over, everyone else is a step in front of them wrt knowing >>>>> what compaction is and making an informed decision into changing it. >>>>> >>>>> >>>>> “You have to over-provision (iops) to use LCS” isn’t that different >>>>> from “you have to over-provision (space) to use LCS” (by perhaps 50%). >>>>> >>>>> Both of them are sub-optimal and you’re trading off either extra space >>>>> or extra compute/ops. >>>>> >>>>> >>>>>
