Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Jeff Jirsa
I’m without laptop this week but looks like CompactionTask#reduceScopeForLimitedSpace So maybe it just comes for free with UCS > On Mar 17, 2023, at 6:21 PM, Jeremy Hanna wrote: > > You're right that it doesn't handle it in the sense that it doesn't resolve > it the problem, but it also

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Jeremy Hanna
You're right that it doesn't handle it in the sense that it doesn't resolve it the problem, but it also doesn't do what STCS does. From what I've seen, STCS blindly tries to compact and then the disk will fill up triggering the disk failure policy. With UCS it's much less likely and if it

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Jeff Jirsa
> On Mar 17, 2023, at 1:46 PM, Jeremy Hanna wrote: > > > > One much more graceful element of UCS is that instead of what was previously > done with compaction strategies where the server just shuts down when running > out of space - forcing system administrators to be paranoid about

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Mick Semb Wever
On Fri, 17 Mar 2023 at 17:24, Brandon Williams wrote: > On Fri, Mar 17, 2023 at 9:25 AM Mick Semb Wever wrote: > > Question/Suggestion: should we improve gossip to include what the oldest > format a node has, and ensure newer versioned node joining fail/warn if it > does > not support that

Re: [VOTE] Release Apache Cassandra 4.1.1

2023-03-17 Thread Brandon Williams
+1 Kind Regards, Brandon On Thu, Mar 16, 2023 at 3:11 AM Miklosovic, Stefan wrote: > > Proposing the test build of Cassandra 4.1.1 for release. > > sha1: 8d91b469afd3fcafef7ef85c10c8acc11703ba2d > Git: > https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.1.1-tentative

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Jeremy Hanna
I think it's important to highlight a few of practical elements of this work that may not be completely clear. Sharding ranges allows for parallel compactions - taking advantage of additional CPUs on the server. This allows compactions to keep up much easier. One much more graceful element of

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Josh McKenzie
> we (including me) have done a lot of stupid shit over the years on this > project. Half the time “this is how we’ve historically done X” to me is a > strong argument to start doing things differently. Oof. The truth (when applied to myself) hurts doesn't it? :) > I suggest we should have a

Re: [VOTE] Release Apache Cassandra 4.1.1

2023-03-17 Thread Josh McKenzie
+1 On Fri, Mar 17, 2023, at 12:18 PM, Aleksey Yeshchenko wrote: > +1 > >> On 17 Mar 2023, at 13:54, Mick Semb Wever wrote: >> >>> The vote will be open for 72 hours (longer if needed). Everyone who has >>> tested the build is invited to vote. Votes by PMC members are considered >>> binding.

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Josh McKenzie
Could we get a JIRA for this too so we can get some reviewers collaborating on this? Only see Lorina's ticket for documenting it in JIRA atm. On Fri, Mar 17, 2023, at 9:53 AM, Branimir Lambov wrote: > The prototype of UCS can now be found in this pull request: >

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Jeremiah D Jordan
> As for precedent - we (including me) have done a lot of stupid shit over the > years on this project. Half the time “this is how we’ve historically done X” > to me is a strong argument to start doing things differently. This is one > such case. +1. I definitely agree that this is one area

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Brandon Williams
On Fri, Mar 17, 2023 at 9:25 AM Mick Semb Wever wrote: > Question/Suggestion: should we improve gossip to include what the oldest > format a node has, and ensure newer versioned node joining fail/warn if it > does > not support that older format? That is, should we give a clear signal > back

Re: [VOTE] Release Apache Cassandra 4.1.1

2023-03-17 Thread Aleksey Yeshchenko
+1 > On 17 Mar 2023, at 13:54, Mick Semb Wever wrote: > >> The vote will be open for 72 hours (longer if needed). Everyone who has >> tested the build is invited to vote. Votes by PMC members are considered >> binding. A vote passes if there are at least three binding +1s and no -1's. > > >

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Aleksey Yeshchenko
> Saying that there is never any complexity and we should keep formats in > perpetuity, and I'm sitting here having a heart attack, srsly. Nobody is claiming that. Don’t let a straw man give you a heart attack. > Though I _always_ recommend users upgrade all sstables, before and after > every

Re: [DISCUSS] Drop support for sstable formats m* (in trunk)

2023-03-17 Thread Mick Semb Wever
Ok ok, there's a number of strong arguments to keep sstable formats around for much longer than the previous major Cassandra version, I will unset fixVersion on 18312 :-) Taking a look at the history of sstable formats. They were first introduced in version 0.7, and minor versions introduced in

Re: [VOTE] Release Apache Cassandra 4.1.1

2023-03-17 Thread Mick Semb Wever
> > The vote will be open for 72 hours (longer if needed). Everyone who has > tested the build is invited to vote. Votes by PMC members are considered > binding. A vote passes if there are at least three binding +1s and no -1's. > +1 Checked - signing correct - checksums are correct - source

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2023-03-17 Thread Branimir Lambov
The prototype of UCS can now be found in this pull request: https://github.com/apache/cassandra/pull/2228 Its description is given in the included markdown documentation: https://github.com/blambov/cassandra/blob/UCS-density/src/java/org/apache/cassandra/db/compaction/UnifiedCompactionStrategy.md

Re: Role of Hadoop code in Cassandra 5.0

2023-03-17 Thread Miklosovic, Stefan
You can initiate that vote if you want, I do not see any problem with that. Please be aware that there is ongoing vote on 4.1.1 release so we have couple of options: a) someone posts binding -1 there which will render that vote as failed. b) the voting on 4.1.1 just passes and we release it as