Re: Request to review feature-freeze proposed tickets

[kurt greaves]

Thanks Vinay. While I suspect this will be subject to heated debate, I'm
also for this. The time to review for this project is incredibly
demotivating, and it stems from a lack of contributors that are interested
in the general health of the project. I think this can be quite easily
remedied by making more committers/PMC, however there is a catch-22 that to
achieve this our existing set of committers needs to be dedicated to
reviewing contributions from non-committers.

If we can get dedicated reviewers for the listed tickets I'll take on some
of the work to get the tickets up to scratch.

On Wed, 21 Nov 2018 at 02:12, Ariel Weisberg  wrote:

> Hi,
>
> I would like to get as many of these as is feasible in. Before the feature
> freeze started 1 out of 17 JIRAs that were patch available were reviewed
> and committed.
>
> If you didn’t have access reviewers and committers, as the one out of the
> 17 did, it has been essentially impossible to get your problems with
> Cassandra fixed in 4.0.
>
> This is basically the same as saying that despite the fact Cassandra is
> open source it does you no good because it will be years before the issues
> impacting you get fixed even if you contribute the fixes yourself.
>
> Pulling up the ladder after getting “your own” fixes in is a sure fire way
> to fracture the community into a collection of private forks containing the
> fixes people can’t live without, and pushing people to look at alternatives.
>
> Private forks are a serious threat to the project. The people on them are
> at risk of getting left behind and Cassandra stagnates for them and becomes
> uncompetitive. Those with the resources to maintain a seriously diverged
> fork are also the ones better positioned to be active contributors.
>
> Regards,
> Ariel
>
> > On Nov 18, 2018, at 9:18 PM, Vinay Chella 
> wrote:
> >
> > Hi,
> >
> > We still have 15 Patch Available/ open tickets which were requested for
> > reviews before the Sep 1, 2018 freeze. I am starting this email thread to
> > resurface and request a review of community tickets as most of these
> > tickets address vital correctness, performance, and usability bugs that
> > help avoid critical production issues. I tried to provide context on why
> we
> > feel these tickets are important to get into 4.0. If you would like to
> > discuss the technical details of a particular ticket, let's try to do
> that
> > in JIRA.
> >
> > CASSANDRA-14525: Cluster enters an inconsistent state after bootstrap
> > failures. (Correctness bug, Production impact, Ready to Commit)
> >
> > CASSANDRA-14459: DES sends requests to the wrong nodes routinely. (SLA
> > breaking latencies, Production impact, Review in progress)
> >
> > CASSANDRA-14303 and CASSANDRA-14557: Currently production 3.0+ clusters
> > cannot be rebuilt after node failure due to 3.0’s introduction of the
> > system_auth keyspace with rf of 1. These tickets both fix the regression
> > introduced in 3.0 by letting operators configure rf=3 and prevent future
> > outages (Usability bug, Production impact, Patch Available).
> >
> > CASSANDRA-14096: Cassandra 3.11.1 Repair Causes Out of Memory. We believe
> > this may also impact 3.0 (Title says it all, Production impact, Patch
> > Available)
> >
> > CASSANDRA-10023: It is impossible to accurately determine local
> read/write
> > calls on C*. This patch allows users to detect when they are choosing
> > incorrect coordinators. (Usability bug (troubleshoot), Review in
> progress)
> >
> > CASSANDRA-10789: There is no way to safely stop bad clients bringing down
> > C* nodes. This patch would give operators a very important tool to use
> > during production incidents to mitigate impact. (Usability bug,
> Production
> > Impact (recovery), Patch Available)
> >
> > CASSANDRA-13010: No visibility into which disk is being compacted to.
> > (Usability bug, Production Impact (troubleshoot), Review in progress)
> >
> > CASSANDRA-12783 - Break up large MV mutations to prevent OOMs (Title says
> > it all, Production Impact, Patch InProgress/ Awaiting Feedback)
> >
> > CASSANDRA-14319 - nodetool rebuild from DC lets you pass invalid
> > datacenters (Usability bug, Production impact, Patch available)
> >
> > CASSANDRA-13841 - Smarter nodetool rebuild. Kind of a bug but would be
> nice
> > to get it in 4.0. (Production Impact (recovery), Patch Available)
> >
> > CASSANDRA-9452: Cleanup of old configuration, confusing to new C*
> > operators. (Cleanup, Patch Available)
> >
> > CASSANDRA-14309: Hint window persistence across the record. This way
> hints
> > that are accumulated over a period of time when nodes are creating are
> less
> > likely to take down the entire cluster. (Potential Production Impact,
> Patch
> > Available)
> >
> > CASSANDRA-14291: Bug from CASSANDRA-11163? (Usability Bug, Patch
> Available)
> >
> > CASSANDRA-10540: RangeAware compaction. 256 vnode clusters really need
> this
> > to be able to do basic things like repair. The patch needs some rework
> > after tr

Re: 4.0 Testing Signup

[kurt greaves]

Been thinking about this for a while and agree it's how we should approach
it. BIkeshedding but seems like a nice big table would be suitable here,
and I think rather than a separate confluence page per component we just
create separate JIRA tickets that detail what's being tested and the
approach, and discussion can be kept in JIRA.

I'm happy to volunteer for testing repair. I can also add lots of different
components to the list if you're happy for me to attack the page.

On Thu, 8 Nov 2018 at 08:51, sankalp kohli  wrote:

> This is good start and we should use this approach each component. Once we
> have volunteers for each area, it will be important to also publish a
> confluence page per component by the volunteer so we can know/discuss how
> it was tested.
>
> On Wed, Nov 7, 2018 at 12:14 PM Joseph Lynch 
> wrote:
>
> > Following up on Jon's call
> > <
> >
> https://lists.apache.org/thread.html/17e57d1666393d961a15502a648a1174a1b145a4bf0a8e07fd8bb760@%3Cdev.cassandra.apache.org%3E
> > >
> > for QA, I put together the start of a confluence page
> > for
> > people to list out important components that they think should be tested
> > before 4.0 releases and hopefully committers and contributors can signup
> > and present their progress to the community. I've certainly missed a ton
> of
> > components that need testing but I figured that it may be good to get the
> > conversation started and moving forward.
> >
> > What do people think? Is there a more effective way to list these out or
> if
> > people like this maybe folks can start contributing sections and
> > volunteering to shepherd or test them?
> >
> > Let me know,
> > -Joey
> >
>

Re: CASSANDRA-13241 lower default chunk_length_in_kb

[kurt greaves]

I think if we're going to drop it to 16k, we should invest in the compact
sequencing as well. Just lowering it to 16k will have potentially a painful
impact on anyone running low memory nodes, but if we can do it without the
memory impact I don't think there's any reason to wait another major
version to implement it.

Having said that, we should probably benchmark the two representations
Ariel has come up with.

On Wed, 17 Oct 2018 at 20:17, Alain RODRIGUEZ  wrote:

> +1
>
> I would guess a lot of C* clusters/tables have this option set to the
> default value, and not many of them are having the need for reading so big
> chunks of data.
> I believe this will greatly limit disk overreads for a fair amount (a big
> majority?) of new users. It seems fair enough to change this default value,
> I also think 4.0 is a nice place to do this.
>
> Thanks for taking care of this Ariel and for making sure there is a
> consensus here as well,
>
> C*heers,
> ---
> Alain Rodriguez - al...@thelastpickle.com
> France / Spain
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> Le sam. 13 oct. 2018 à 08:52, Ariel Weisberg  a écrit :
>
> > Hi,
> >
> > This would only impact new tables, existing tables would get their
> > chunk_length_in_kb from the existing schema. It's something we record in
> a
> > system table.
> >
> > I have an implementation of a compact integer sequence that only requires
> > 37% of the memory required today. So we could do this with only slightly
> > more than doubling the memory used. I'll post that to the JIRA soon.
> >
> > Ariel
> >
> > On Fri, Oct 12, 2018, at 1:56 AM, Jeff Jirsa wrote:
> > >
> > >
> > > I think 16k is a better default, but it should only affect new tables.
> > > Whoever changes it, please make sure you think about the upgrade path.
> > >
> > >
> > > > On Oct 12, 2018, at 2:31 AM, Ben Bromhead 
> wrote:
> > > >
> > > > This is something that's bugged me for ages, tbh the performance gain
> > for
> > > > most use cases far outweighs the increase in memory usage and I would
> > even
> > > > be in favor of changing the default now, optimizing the storage cost
> > later
> > > > (if it's found to be worth it).
> > > >
> > > > For some anecdotal evidence:
> > > > 4kb is usually what we end setting it to, 16kb feels more reasonable
> > given
> > > > the memory impact, but what would be the point if practically, most
> > folks
> > > > set it to 4kb anyway?
> > > >
> > > > Note that chunk_length will largely be dependent on your read sizes,
> > but 4k
> > > > is the floor for most physical devices in terms of ones block size.
> > > >
> > > > +1 for making this change in 4.0 given the small size and the large
> > > > improvement to new users experience (as long as we are explicit in
> the
> > > > documentation about memory consumption).
> > > >
> > > >
> > > >> On Thu, Oct 11, 2018 at 7:11 PM Ariel Weisberg 
> > wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> This is regarding
> > https://issues.apache.org/jira/browse/CASSANDRA-13241
> > > >>
> > > >> This ticket has languished for a while. IMO it's too late in 4.0 to
> > > >> implement a more memory efficient representation for compressed
> chunk
> > > >> offsets. However I don't think we should put out another release
> with
> > the
> > > >> current 64k default as it's pretty unreasonable.
> > > >>
> > > >> I propose that we lower the value to 16kb. 4k might never be the
> > correct
> > > >> default anyways as there is a cost to compression and 16k will still
> > be a
> > > >> large improvement.
> > > >>
> > > >> Benedict and Jon Haddad are both +1 on making this change for 4.0.
> In
> > the
> > > >> past there has been some consensus about reducing this value
> although
> > maybe
> > > >> with more memory efficiency.
> > > >>
> > > >> The napkin math for what this costs is:
> > > >> "If you have 1TB of uncompressed data, with 64k chunks that's 16M
> > chunks
> > > >> at 8 bytes each (128MB).
> > > >> With 16k chunks, that's 512MB.
> > > >> With 4k chunks, it's 2G.
> > > >> Per terabyte of data (pre-compression)."
> > > >>
> > > >>
> >
> https://issues.apache.org/jira/browse/CASSANDRA-13241?focusedCommentId=15886621&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15886621
> > > >>
> > > >> By way of comparison memory mapping the files has a similar cost per
> > 4k
> > > >> page of 8 bytes. Multiple mappings makes this more expensive. With a
> > > >> default of 16kb this would be 4x less expensive than memory mapping
> a
> > file.
> > > >> I only mention this to give a sense of the costs we are already
> > paying. I
> > > >> am not saying they are directly related.
> > > >>
> > > >> I'll wait a week for discussion and if there is consensus make the
> > change.
> > > >>
> > > >> Regards,
> > > >> Ariel
> > > >>
> > > >>
> -
> > > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>

Re: Moving tickets out of 4.0 post freeze

[kurt greaves]

My JIRA experience tells me it would be better to do a blanket update and
then go back and fix anything that shouldn't have been updated. Shouldn't
be hard that way either because everyones inbox will be flooded with
emails, so as long as we all sanity check what got changed we should catch
most of the tickets. Alternatively we could go through and tag all the ones
that should go to 4.0 beforehand so we can use the tag to filter them out
when we do the bulk change. But the former seems easier to crowdsource to
me.

On Tue, 25 Sep 2018 at 21:19, Jason Brown  wrote:

> Before we blanket update all of these, let's make sure they are not
> relevant for 4.0 - because some of them actually are. Some are docs, some
> are parent tickets, some are testing.
>
> Naively, here are some that are still 4.0:
> CASSANDRA-14746 
> CASSANDRA-13254 
> CASSANDRA-14606 
>
> Thanks,
>
> -Jason
>
> On Tue, Sep 25, 2018 at 4:05 AM, Benedict Elliott Smith <
> bened...@apache.org
> > wrote:
>
> > Do we want to manage more versions than we do now?  Why don’t we simply
> > align these things to majors, like we’ve typically tried to anyway?
> >
> > I think it’s anyway better to decide on a strategy and find a versioning
> > scheme that matches it, rather than to look for a strategy that matches
> our
> > current versioning scheme.
> >
> >
> >
> >
> > > On 25 Sep 2018, at 03:07, Mick Semb Wever  wrote:
> > >
> > >
> > >
> > >
> > >> I'm totally spitballing here on possible uses of meaningful minors.
> > >
> > > Continuing the splitballing…
> > >
> > > What about aligning native protocol or sstable format changes with the
> > minor version?
> > >
> > >
> > >> Regardless, the OP's statement that new features and improvements
> > should go to 4.0.x seems wrong
> > >
> > > Yeah, I instinctively thought features and improvements would be moved
> > to either 4.x or 5.x (not to any subsequent patch version 4.0.x).
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [DISCUSS] changing default token behavior for 4.0

[kurt greaves]

This was exactly the kind of problem I was foreseeing. I don't see any
simple way of fixing it without introducing some shuffle-like nightmare
that does a whole bunch of token movements though. On the other hand we
could just document best practice, and also make it so that by default you
have to choose between random and the algorithm for allocation initially,
essentially the way I mentioned before. At least this way people are aware
of the advantages and disadvantages from the start, rather than everyone
just ending up with random allocation because that's what they were given.
Anyway, this isn't a simple problem so we could probably come up with
something better than that with a bit more thought

On Tue, 25 Sep 2018 at 05:43, Benedict Elliott Smith 
wrote:

> This sounds worthy of a bug report!  We should at least document any such
> inadequacy, and come up with a plan to fix it.  It would be great if you
> could file a ticket with a detailed example of the problem.
>
> > On 24 Sep 2018, at 14:57, Tom van der Woerdt <
> tom.vanderwoe...@booking.com> wrote:
> >
> > Late comment, but I'll write it anyway.
> >
> > The main advantage of random allocation over the new allocation strategy
> is
> > that it seems to be significantly better when dealing with node
> *removals*,
> > when the order of removal is not the inverse of the order of addition.
> This
> > can lead to severely unbalanced clusters when the new strategy is
> enabled.
> >
> > I tend to go with the random allocation for this reason: you can freely
> > add/remove nodes when needed, and the data distribution will remain "good
> > enough". It's only when the data density becomes high enough that the new
> > token allocation strategy really matters, imho.
> >
> > Hope that helps!
> >
> > Tom van der Woerdt
> > Site Reliability Engineer
> >
> > Booking.com B.V.
> > Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
> > [image: Booking.com] <https://www.booking.com/>
> > The world's #1 accommodation site
> > 43 languages, 198+ offices worldwide, 120,000+ global destinations,
> > 1,550,000+ room nights booked every day
> > No booking fees, best price always guaranteed
> > Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
> >
> >
> > On Sat, Sep 22, 2018 at 8:12 PM Jonathan Haddad 
> wrote:
> >
> >> Is there a use case for random allocation? How does it help with
> testing? I
> >> can’t see a reason to keep it around.
> >>
> >> On Sat, Sep 22, 2018 at 3:06 AM kurt greaves 
> wrote:
> >>
> >>> +1. I've been making a case for this for some time now, and was
> actually
> >> a
> >>> focus of my talk last week. I'd be very happy to get this into 4.0.
> >>>
> >>> We've tested various num_tokens with the algorithm on various sized
> >>> clusters and we've found that typically 16 works best. With lower
> numbers
> >>> we found that balance is good initially but as a cluster gets larger
> you
> >>> have some problems. E.g We saw that on a 60 node cluster with 8 tokens
> >> per
> >>> node we were seeing a difference of 22% in token ownership, but on a
> <=12
> >>> node cluster a difference of only 12%. 16 tokens on the other hand
> wasn't
> >>> perfect but generally gave a better balance regardless of cluster size
> at
> >>> least up to 100 nodes. TBH we should probably do some proper testing
> and
> >>> record all the results for this before we pick a default (I'm happy to
> do
> >>> this - think we can use the original testing script for this).
> >>>
> >>> But anyway, I'd say Jon is on the right track. Personally how I'd like
> to
> >>> see it is that we:
> >>>
> >>>   1. Change allocate_tokens_for_keyspace to allocate_tokens_for_rf in
> >> the
> >>>   same way that DSE does it. Allowing a user to specify a RF to
> allocate
> >>>   from, and allowing multiple DC's.
> >>>   2. Add a new boolean property random_token_allocation, defaults to
> >>> false.
> >>>   3. Make allocate_tokens_for_rf default to *unset**.
> >>>   4. Make allocate_tokens_for_rf *required*** if num_tokens > 1 and
> >>>   random_token_allocation != true.
> >>>   5. Default num_tokens to 16 (or whatever we find appropriate)
> >>>
> >>> * I think setting a default is asking for trouble. When people are
> going
> >> to
> &g

Re: [DISCUSS] changing default token behavior for 4.0

[kurt greaves]

Sounds good to me. I'm going to play around with the algorithm and actually
record some numbers/evidence over the next week to help us decide.

On Tue, 25 Sep 2018 at 05:38, Joseph Lynch  wrote:

> I am a big fan of lowering the default number of tokens for many
> reasons (availability, repair, etc...). I also agree there are some
> usability blockers to "just lowering the number today", but I very
> much agree that the current default of 256 random tokens is a huge bug
> I hope we fix by 4.0 release.
>
> It sounds like Kurt and Jon have done a lot of work already on this
> problem, and internally I've worked on this as well (Netflix's
> internal token allocation as well as evaluating vnodes that resulted
> in the paper I sent out) so I would be excited to help fix this for
> 4.0. Maybe the three of us (plus any others that are interested) can
> put together a short proposal over the next few days including the
> following:
>
> 1. What precisely should we change the defaults to
> 2. Given the new defaults how would a user bootstrap a new cluster
> 3. Given the new defaults how would a user add capacity to an existing
> cluster
> 4. Concrete jiras that would implement #1 with minimal possible scope
>
> Then we could send the proposal to the dev list for feedback and if
> there is consensus that the scope is not too large/dangerous and a
> committer (Jon perhaps) can commit to reviewing/merging, we can work
> on them and be accountable to merge them before the 4.0 release?
>
> -Joey
> On Sun, Sep 23, 2018 at 1:42 PM Nate McCall  wrote:
> >
> > Let's pick a default setup that works for most people (IME clusters <
> > 30 nodes, but TLP and Instaclustr peeps probably have the most insight
> > here).
> >
> > Then we just explain the heck out of it in the comments. I would also
> > like to see this include some details add/remove a DC to change the
> > values (perhaps we sub-task a doc creation for that?).
> >
> > Good discussion though - thanks folks.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Moving tickets out of 4.0 post freeze

[kurt greaves]

Yeah so lets not derail any further and just put them all to 4.x (as I'm
sure OP originally intended) and we can figure out versioning later.

On Tue, 25 Sep 2018 at 12:07, Mick Semb Wever  wrote:

>
>
>
> > I'm totally spitballing here on possible uses of meaningful minors.
>
> Continuing the splitballing…
>
> What about aligning native protocol or sstable format changes with the
> minor version?
>
>
> > Regardless, the OP's statement that new features and improvements should
> go to 4.0.x seems wrong
>
> Yeah, I instinctively thought features and improvements would be moved to
> either 4.x or 5.x (not to any subsequent patch version 4.0.x).
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Measuring Release Quality

[kurt greaves]

Yep agreed with that. Count me in.

On Sun., 23 Sep. 2018, 00:33 Benedict Elliott Smith, 
wrote:

> Thanks Kurt.  I think the goal would be to get JIRA into a state where it
> can hold all the information we want, and for it to be easy to get all the
> information correct when filing.
>
> My feeling is that it would be easiest to do this with a small group, so
> we can make rapid progress on an initial proposal, then bring that to the
> community for final tweaking / approval (or, perhaps, rejection - but I
> hope it won’t be a contentious topic).  I don’t think it should be a huge
> job to come up with a proposal - though we might need to organise a
> community effort to clean up the JIRA history!
>
> It would be great if we could get a few more volunteers from other
> companies/backgrounds to participate.
>
>
> > On 22 Sep 2018, at 11:54, kurt greaves  wrote:
> >
> > I'm interested. Better defining the components and labels we use in our
> > docs would be a good start and LHF. I'd prefer if we kept all the
> > information within JIRA through the use of fields/labels though, and
> > generated reports off those tags. Keeping all the information in one
> place
> > is much better in my experience. Not applicable for CI obviously, but
> > ideally we can generate testing reports directly from the testing
> systems.
> >
> > I don't see this as a huge amount of work so I think the overall risk is
> > pretty small, especially considering it can easily be done in a way that
> > doesn't affect anyone until we get consensus on methodology.
> >
> >
> >
> > On Sat, 22 Sep 2018 at 03:44, Scott Andreas 
> wrote:
> >
> >> Josh, thanks for reading and sharing feedback. Agreed with starting
> simple
> >> and measuring inputs that are high-signal; that’s a good place to begin.
> >>
> >> To the challenge of building consensus, point taken + agreed. Perhaps
> the
> >> distinction is between producing something that’s “useful” vs. something
> >> that’s “authoritative” for decisionmaking purposes. My motivation is to
> >> work toward something “useful” (as measured by the value contributors
> >> find). I’d be happy to start putting some of these together as part of
> an
> >> experiment – and agreed on evaluating “value relative to cost” after we
> see
> >> how things play out.
> >>
> >> To Benedict’s point on JIRA, agreed that plotting a value from messy
> input
> >> wouldn’t produce useful output. Some questions a small working group
> might
> >> take on toward better categorization might look like:
> >>
> >> –––
> >> – Revisiting the list of components: e.g., “Core” captures a lot right
> now.
> >> – Revisiting which fields should be required when filing a ticket – and
> if
> >> there are any that should be removed from the form.
> >> – Reviewing active labels: understanding what people have been trying to
> >> capture, and how they could be organized + documented better.
> >> – Documenting “priority”: (e.g., a common standard we can point to, even
> >> if we’re pretty good now).
> >> – Considering adding a "severity” field to capture the distinction
> between
> >> priority and severity.
> >> –––
> >>
> >> If there’s appetite for spending a little time on this, I’d put effort
> >> toward it if others are interested; is anyone?
> >>
> >> Otherwise, I’m equally fine with an experiment to measure basics via the
> >> current structure as Josh mentioned, too.
> >>
> >> – Scott
> >>
> >>
> >> On September 20, 2018 at 8:22:55 AM, Benedict Elliott Smith (
> >> bened...@apache.org<mailto:bened...@apache.org>) wrote:
> >>
> >> I think it would be great to start getting some high quality info out of
> >> JIRA, but I think we need to clean up and standardise how we use it to
> >> facilitate this.
> >>
> >> Take the Component field as an example. This is the current list of
> >> options:
> >>
> >> 4.0
> >> Auth
> >> Build
> >> Compaction
> >> Configuration
> >> Core
> >> CQL
> >> Distributed Metadata
> >> Documentation and Website
> >> Hints
> >> Libraries
> >> Lifecycle
> >> Local Write-Read Paths
> >> Materialized Views
> >> Metrics
> >> Observability
> >> Packaging
> >> Repair
> >> SASI
> >> Secondary Indexes
> >> Streaming and Messa

Re: [DISCUSS] changing default token behavior for 4.0

[kurt greaves]

Only that it makes it easier to spin up a cluster.

I'm for removing it entirely as well, however I think we should keep it
around at least until the next major just as a safety precaution until the
algorithm is properly battle tested.

This is not a strongly held opinion though, I'm just foreseeing the "new
defaults don't work for my edge case" problem.

On Sun., 23 Sep. 2018, 04:12 Jonathan Haddad,  wrote:

> Is there a use case for random allocation? How does it help with testing? I
> can’t see a reason to keep it around.
>
> On Sat, Sep 22, 2018 at 3:06 AM kurt greaves  wrote:
>
> > +1. I've been making a case for this for some time now, and was actually
> a
> > focus of my talk last week. I'd be very happy to get this into 4.0.
> >
> > We've tested various num_tokens with the algorithm on various sized
> > clusters and we've found that typically 16 works best. With lower numbers
> > we found that balance is good initially but as a cluster gets larger you
> > have some problems. E.g We saw that on a 60 node cluster with 8 tokens
> per
> > node we were seeing a difference of 22% in token ownership, but on a <=12
> > node cluster a difference of only 12%. 16 tokens on the other hand wasn't
> > perfect but generally gave a better balance regardless of cluster size at
> > least up to 100 nodes. TBH we should probably do some proper testing and
> > record all the results for this before we pick a default (I'm happy to do
> > this - think we can use the original testing script for this).
> >
> > But anyway, I'd say Jon is on the right track. Personally how I'd like to
> > see it is that we:
> >
> >1. Change allocate_tokens_for_keyspace to allocate_tokens_for_rf in
> the
> >same way that DSE does it. Allowing a user to specify a RF to allocate
> >from, and allowing multiple DC's.
> >2. Add a new boolean property random_token_allocation, defaults to
> > false.
> >3. Make allocate_tokens_for_rf default to *unset**.
> >4. Make allocate_tokens_for_rf *required*** if num_tokens > 1 and
> >random_token_allocation != true.
> >5. Default num_tokens to 16 (or whatever we find appropriate)
> >
> > * I think setting a default is asking for trouble. When people are going
> to
> > add new DC's/nodes we don't want to risk them adding a node with the
> wrong
> > RF. I think it's safe to say that a user should have to think about this
> > before they spin up their cluster.
> > ** Following above, it should be required to be set so that we don't have
> > people accidentally using random allocation. I think we should really be
> > aiming to get rid of random allocation completely, but provide a new
> > property to enable it for backwards compatibility (also for testing).
> >
> > It's worth noting that a smaller number of tokens *theoretically*
> decreases
> > the time for replacement/rebuild, so if we're considering QUORUM
> > availability with vnodes there's an argument against having a very low
> > num_tokens. I think it's better to utilise NTS and racks to reduce the
> > chance of a QUORUM outage over banking on having a lower number of
> tokens,
> > as with just a low number of tokens unless you go all the way to 1 you
> are
> > just relying on luck that 2 nodes don't overlap. Guess what I'm saying is
> > that I think we should be choosing a num_tokens that gives the best
> > distribution for most cluster sizes rather than choosing one that
> > "decreases" the probability of an outage.
> >
> > Also I think we should continue using CASSANDRA-13701 to track this. TBH
> I
> > think in general we should be a bit better at searching for and using
> > existing tickets...
> >
> > On Sat, 22 Sep 2018 at 18:13, Stefan Podkowinski 
> wrote:
> >
> > > There already have been some discussions on this here:
> > > https://issues.apache.org/jira/browse/CASSANDRA-13701
> > >
> > > The mentioned blocker there on the token allocation shouldn't exist
> > > anymore. Although it would be good to get more feedback on it, in case
> > > we want to enable it by default, along with new defaults for number of
> > > tokens.
> > >
> > >
> > > On 22.09.18 06:30, Dinesh Joshi wrote:
> > > > Jon, thanks for starting this thread!
> > > >
> > > > I have created CASSANDRA-14784 to track this.
> > > >
> > > > Dinesh
> > > >
> > > >> On Sep 21, 2018, at 9:18

Re: Measuring Release Quality

[kurt greaves]

I'm interested. Better defining the components and labels we use in our
docs would be a good start and LHF. I'd prefer if we kept all the
information within JIRA through the use of fields/labels though, and
generated reports off those tags. Keeping all the information in one place
is much better in my experience. Not applicable for CI obviously, but
ideally we can generate testing reports directly from the testing systems.

I don't see this as a huge amount of work so I think the overall risk is
pretty small, especially considering it can easily be done in a way that
doesn't affect anyone until we get consensus on methodology.



On Sat, 22 Sep 2018 at 03:44, Scott Andreas  wrote:

> Josh, thanks for reading and sharing feedback. Agreed with starting simple
> and measuring inputs that are high-signal; that’s a good place to begin.
>
> To the challenge of building consensus, point taken + agreed. Perhaps the
> distinction is between producing something that’s “useful” vs. something
> that’s “authoritative” for decisionmaking purposes. My motivation is to
> work toward something “useful” (as measured by the value contributors
> find). I’d be happy to start putting some of these together as part of an
> experiment – and agreed on evaluating “value relative to cost” after we see
> how things play out.
>
> To Benedict’s point on JIRA, agreed that plotting a value from messy input
> wouldn’t produce useful output. Some questions a small working group might
> take on toward better categorization might look like:
>
> –––
> – Revisiting the list of components: e.g., “Core” captures a lot right now.
> – Revisiting which fields should be required when filing a ticket – and if
> there are any that should be removed from the form.
> – Reviewing active labels: understanding what people have been trying to
> capture, and how they could be organized + documented better.
> – Documenting “priority”: (e.g., a common standard we can point to, even
> if we’re pretty good now).
> – Considering adding a "severity” field to capture the distinction between
> priority and severity.
> –––
>
> If there’s appetite for spending a little time on this, I’d put effort
> toward it if others are interested; is anyone?
>
> Otherwise, I’m equally fine with an experiment to measure basics via the
> current structure as Josh mentioned, too.
>
> – Scott
>
>
> On September 20, 2018 at 8:22:55 AM, Benedict Elliott Smith (
> bened...@apache.org) wrote:
>
> I think it would be great to start getting some high quality info out of
> JIRA, but I think we need to clean up and standardise how we use it to
> facilitate this.
>
> Take the Component field as an example. This is the current list of
> options:
>
> 4.0
> Auth
> Build
> Compaction
> Configuration
> Core
> CQL
> Distributed Metadata
> Documentation and Website
> Hints
> Libraries
> Lifecycle
> Local Write-Read Paths
> Materialized Views
> Metrics
> Observability
> Packaging
> Repair
> SASI
> Secondary Indexes
> Streaming and Messaging
> Stress
> Testing
> Tools
>
> In some cases there's duplication (Metrics + Observability, Coordination
> (=“Storage Proxy, Hints, Batchlog, Counters…") + Hints, Local Write-Read
> Paths + Core)
> In others, there’s a lack of granularity (Streaming + Messaging, Core,
> Coordination, Distributed Metadata)
> In others, there’s a lack of clarity (Core, Lifecycle, Coordination)
> Others are probably missing entirely (Transient Replication, …?)
>
> Labels are also used fairly haphazardly, and there’s no clear definition
> of “priority”
>
> Perhaps we should form a working group to suggest a methodology for
> filling out JIRA, standardise the necessary components, labels etc, and put
> together a wiki page with step-by-step instructions on how to do it?
>
>
> > On 20 Sep 2018, at 15:29, Joshua McKenzie  wrote:
> >
> > I've spent a good bit of time thinking about the above and bounced off
> both
> > different ways to measure quality and progress as well as trying to
> > influence community behavior on this topic. My advice: start small and
> > simple (KISS, YAGNI, all that). Get metrics for pass/fail on
> > utest/dtest/flakiness over time, perhaps also aggregate bug count by
> > component over time. After spending a predetermined time doing that (a
> > couple months?) as an experiment, we retrospect as a project and see if
> > these efforts are adding value commensurate with the time investment
> > required to perform the measurement and analysis.
> >
> > There's a lot of really good ideas in that linked wiki article / this
> email
> > thread. The biggest challenge, and risk of failure, is in translating
> good
> > ideas into action and selling project participants on the value of
> changing
> > their behavior. The latter is where we've fallen short over the years;
> > building consensus (especially regarding process /shudder) is Very Hard.
> >
> > Also - thanks for spearheading this discussion Scott. It's one we come
> back
> > to with some re

Re: Proposing an Apache Cassandra Management process

[kurt greaves]

Is this something we're moving ahead with despite the feature freeze?

On Sat, 22 Sep 2018 at 08:32, dinesh.jo...@yahoo.com.INVALID
 wrote:

> I have created a sub-task - CASSANDRA-14783. Could we get some feedback
> before we begin implementing anything?
>
> Dinesh
>
> On Thursday, September 20, 2018, 11:22:33 PM PDT, Dinesh Joshi <
> dinesh.jo...@yahoo.com.INVALID> wrote:
>
>  I have updated the doc with a short paragraph providing the
> clarification. Sankalp's suggestion is already part of the doc. If there
> aren't further objections could we move this discussion over to the jira
> (CASSANDRA-14395)?
>
> Dinesh
>
> > On Sep 18, 2018, at 10:31 AM, sankalp kohli 
> wrote:
> >
> > How about we start with a few basic features in side car. How about
> starting with this
> > 1. Bulk nodetool commands: User can curl any sidecar and be able to run
> a nodetool command in bulk across the cluster.
> >
> :/bulk/nodetool/tablestats?arg0=keyspace_name.table_name&arg1= required>
> >
> > And later
> > 2: Health checks.
> >
> > On Thu, Sep 13, 2018 at 11:34 AM dinesh.jo...@yahoo.com.INVALID <
> dinesh.jo...@yahoo.com.invalid> wrote:
> > I will update the document to add that point. The document did not mean
> to serve as a design or architectural document but rather something that
> would spark a discussion on the idea.
> > Dinesh
> >
> >On Thursday, September 13, 2018, 10:59:34 AM PDT, Jonathan Haddad <
> j...@jonhaddad.com > wrote:
> >
> >  Most of the discussion and work was done off the mailing list - there's
> a
> > big risk involved when folks disappear for months at a time and resurface
> > with big pile of code plus an agenda that you failed to loop everyone in
> > on. In addition, by your own words the design document didn't accurately
> > describe what was being built.  I don't write this to try to argue about
> > it, I just want to put some perspective for those of us that weren't part
> > of this discussion on a weekly basis over the last several months.  Going
> > forward let's keep things on the ML so we can avoid confusion and
> > frustration for all parties.
> >
> > With that said - I think Blake made a really good point here and it's
> > helped me understand the scope of what's being built better.  Looking at
> it
> > from a different perspective it doesn't seem like there's as much overlap
> > as I had initially thought.  There's the machinery that runs certain
> tasks
> > (what Joey has been working on) and the user facing side of exposing that
> > information in management tool.
> >
> > I do appreciate (and like) the idea of not trying to boil the ocean, and
> > working on things incrementally.  Putting a thin layer on top of
> Cassandra
> > that can perform cluster wide tasks does give us an opportunity to move
> in
> > the direction of a general purpose user-facing admin tool without
> > committing to trying to write the full stack all at once (or even make
> > decisions on it now).  We do need a sensible way of doing rolling
> restarts
> > / scrubs / scheduling and Reaper wasn't built for that, and even though
> we
> > can add it I'm not sure if it's the best mechanism for the long term.
> >
> > So if your goal is to add maturity to the project by making cluster wide
> > tasks easier by providing a framework to build on top of, I'm in favor of
> > that and I don't see it as antithetical to what I had in mind with
> Reaper.
> > Rather, the two are more complementary than I had originally realized.
> >
> > Jon
> >
> >
> >
> >
> > On Thu, Sep 13, 2018 at 10:39 AM dinesh.jo...@yahoo.com.INVALID
> > mailto:dinesh.jo...@yahoo.com>.invalid> wrote:
> >
> > > I have a few clarifications -
> > > The scope of the management process is not to simply run repair
> > > scheduling. Repair scheduling is one of the many features we could
> > > implement or adopt from existing sources. So could we please split the
> > > Management Process discussion and the repair scheduling?
> > > After re-reading the management process proposal, I see we missed to
> > > communicate a basic idea in the document. We wanted to take a pluggable
> > > approach to various activities that the management process could
> perform.
> > > This could accommodate different implementations of common activities
> such
> > > as repair. The management process would provide the basic framework
> and it
> > > would have default implementations for some of the basic activities.
> This
> > > would allow for speedier iteration cycles and keep things extensible.
> > > Turning to some questions that Jon and others have raised, when I +1,
> my
> > > intention is to fully contribute and stay with this community. That
> said,
> > > things feel rushed for some but for me it feels like analysis
> paralysis.
> > > We're looking for actionable feedback and to discuss the management
> process
> > > _not_ repair scheduling solutions.
> > > Thanks,
> > > Dinesh
> > >
> > >
> > >
> > > On Sep 12, 2018, at 6:24 PM, sankalp kohli 

Re: [DISCUSS] changing default token behavior for 4.0

[kurt greaves]

+1. I've been making a case for this for some time now, and was actually a
focus of my talk last week. I'd be very happy to get this into 4.0.

We've tested various num_tokens with the algorithm on various sized
clusters and we've found that typically 16 works best. With lower numbers
we found that balance is good initially but as a cluster gets larger you
have some problems. E.g We saw that on a 60 node cluster with 8 tokens per
node we were seeing a difference of 22% in token ownership, but on a <=12
node cluster a difference of only 12%. 16 tokens on the other hand wasn't
perfect but generally gave a better balance regardless of cluster size at
least up to 100 nodes. TBH we should probably do some proper testing and
record all the results for this before we pick a default (I'm happy to do
this - think we can use the original testing script for this).

But anyway, I'd say Jon is on the right track. Personally how I'd like to
see it is that we:

   1. Change allocate_tokens_for_keyspace to allocate_tokens_for_rf in the
   same way that DSE does it. Allowing a user to specify a RF to allocate
   from, and allowing multiple DC's.
   2. Add a new boolean property random_token_allocation, defaults to false.
   3. Make allocate_tokens_for_rf default to *unset**.
   4. Make allocate_tokens_for_rf *required*** if num_tokens > 1 and
   random_token_allocation != true.
   5. Default num_tokens to 16 (or whatever we find appropriate)

* I think setting a default is asking for trouble. When people are going to
add new DC's/nodes we don't want to risk them adding a node with the wrong
RF. I think it's safe to say that a user should have to think about this
before they spin up their cluster.
** Following above, it should be required to be set so that we don't have
people accidentally using random allocation. I think we should really be
aiming to get rid of random allocation completely, but provide a new
property to enable it for backwards compatibility (also for testing).

It's worth noting that a smaller number of tokens *theoretically* decreases
the time for replacement/rebuild, so if we're considering QUORUM
availability with vnodes there's an argument against having a very low
num_tokens. I think it's better to utilise NTS and racks to reduce the
chance of a QUORUM outage over banking on having a lower number of tokens,
as with just a low number of tokens unless you go all the way to 1 you are
just relying on luck that 2 nodes don't overlap. Guess what I'm saying is
that I think we should be choosing a num_tokens that gives the best
distribution for most cluster sizes rather than choosing one that
"decreases" the probability of an outage.

Also I think we should continue using CASSANDRA-13701 to track this. TBH I
think in general we should be a bit better at searching for and using
existing tickets...

On Sat, 22 Sep 2018 at 18:13, Stefan Podkowinski  wrote:

> There already have been some discussions on this here:
> https://issues.apache.org/jira/browse/CASSANDRA-13701
>
> The mentioned blocker there on the token allocation shouldn't exist
> anymore. Although it would be good to get more feedback on it, in case
> we want to enable it by default, along with new defaults for number of
> tokens.
>
>
> On 22.09.18 06:30, Dinesh Joshi wrote:
> > Jon, thanks for starting this thread!
> >
> > I have created CASSANDRA-14784 to track this.
> >
> > Dinesh
> >
> >> On Sep 21, 2018, at 9:18 PM, Sankalp Kohli 
> wrote:
> >>
> >> Putting it on JIRA is to make sure someone is assigned to it and it is
> tracked. Changes should be discussed over ML like you are saying.
> >>
> >> On Sep 21, 2018, at 21:02, Jonathan Haddad  wrote:
> >>
>  We should create a JIRA to find what other defaults we need revisit.
> >>> Changing a default is a pretty big deal, I think we should discuss any
> >>> changes to defaults here on the ML before moving it into JIRA.  It's
> nice
> >>> to get a bit more discussion around the change than what happens in
> JIRA.
> >>>
> >>> We (TLP) did some testing on 4 tokens and found it to work surprisingly
> >>> well.   It wasn't particularly formal, but we verified the load stays
> >>> pretty even with only 4 tokens as we added nodes to the cluster.
> Higher
> >>> token count hurts availability by increasing the number of nodes any
> given
> >>> node is a neighbor with, meaning any 2 nodes that fail have an
> increased
> >>> chance of downtime when using QUORUM.  In addition, with the recent
> >>> streaming optimization it seems the token counts will give a greater
> chance
> >>> of a node streaming entire sstables (with LCS), meaning we'll do a
> better
> >>> job with node density out of the box.
> >>>
> >>> Next week I can try to put together something a little more convincing.
> >>> Weekend time.
> >>>
> >>> Jon
> >>>
> >>>
> >>> On Fri, Sep 21, 2018 at 8:45 PM sankalp kohli 
> >>> wrote:
> >>>
>  +1 to lowering it.
>  Thanks Jon for starting this.We should create a JIRA to find what
> other
> >>>

Re: QA signup

[kurt greaves]

It's pretty much only third party plugins. I need it for the LDAP
authenticator, and StratIO's lucene plugin will also need it. I know there
are users out there with their own custom plugins that would benefit from
it as well (and various other open source projects). It would make it
easier, however it certainly is feasible for these devs to just build the
jars themselves (and I've done this so far). If it's going to be easy I
think there's value in generating and hosting nightly jars, but if it's
difficult I can just write some docs for DIY.

On Thu, 20 Sep 2018 at 12:20, Mick Semb Wever  wrote:

> Sorry about the terrible english in my last email.
>
>
> > On the target audience:
> >
> > [snip]
> >  For developers building automation around testing and
> > validation, it’d be great to have a common build to work from rather
> > than each developer producing these builds themselves.
>
>
> Sure. My question was only in context of maven artefacts.
> It seems to me all the use-cases you highlight would be for the binary
> artefacts.
>
> If that's the case we don't need to worry about publishing snapshots maven
> artefacts, and can just focus on uploading nightly builds to
> https://dist.apache.org/repos/dist/dev/cassandra/
>
> Or is there a use-case I'm missing that needs the maven artefacts?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Accept GoCQL driver donation and begin incubation process

[kurt greaves]

In the previous thread we seemed to come to the conclusion it would be
under the same project with same committers/pmc. I don't know if sending it
through incubation changes that?

On Wed., 12 Sep. 2018, 13:03 Jeremy Hanna, 
wrote:

> I don’t know if others have this same question, but what does accepting
> the gocql driver donation mean?  It becomes a full Apache project separate
> from Cassandra and there exists a separate set of PMC members and such?  Or
> does it become part of the Cassandra project itself?  From Sylvain and
> Jon’s responses, it seems like it’s the latter.  I have some memories of
> the Apache Extras and some things that lived in there for a time and those
> were eventually phased out so I didn’t know if that applied to this
> discussion as well.
>
> > On Sep 12, 2018, at 10:12 AM, Nate McCall  wrote:
> >
> > This will be the same process used for dtest. We will need to walk
> > this through the incubator per the process outlined here:
> >
> > https://incubator.apache.org/guides/ip_clearance.html
> >
> > Pending the outcome of this vote, we will create the JIRA issues for
> > tracking and after we go through the process, and discuss adding
> > committers in a separate thread (we need to do this atomically anyway
> > per general ASF committer adding processes).
> >
> > Thanks,
> > -Nate
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Accept GoCQL driver donation and begin incubation process

[kurt greaves]

+1. I do share the same concerns as Sylvain, but I think it's important we
get more driver experience/exposure in the project, as it seems lacking
since datastax moved off, and I don't think this will be something we can
just leave to the userbase to solve.

I'm definitely looking to contribute as well.

On Wed., 12 Sep. 2018, 12:48 Ben Bromhead,  wrote:

> +1
>
> On Wed, Sep 12, 2018 at 1:55 PM Jonathan Haddad  wrote:
>
> > I'm +0, and I share the same concerns as Sylvain.
> >
> > For those of you that have +1'ed, are you planning on contributing to the
> > driver?  Docs, code, QA?  It's easy to throw a +1 down to make the driver
> > the responsibility of the project if you're asking others to do the work.
> > I vote this way because I already know I won't contribute to it and it's
> > irresponsible to vote something in if I have no intent on helping
> maintain
> > it.
> >
> > On Wed, Sep 12, 2018 at 10:45 AM Sumanth Pasupuleti
> >  wrote:
> >
> > > +1
> > >
> > > On Wed, Sep 12, 2018 at 10:37 AM Dinesh Joshi
> > >  wrote:
> > >
> > > > +1
> > > >
> > > > Dinesh
> > > >
> > > > > On Sep 12, 2018, at 10:23 AM, Jaydeep Chovatia <
> > > > chovatia.jayd...@gmail.com> wrote:
> > > > >
> > > > > +1
> > > > >
> > > > > On Wed, Sep 12, 2018 at 10:00 AM Roopa Tangirala
> > > > >  wrote:
> > > > >
> > > > >> +1
> > > > >>
> > > > >>
> > > > >> *Regards,*
> > > > >>
> > > > >> *Roopa Tangirala*
> > > > >>
> > > > >> Engineering Manager CDE
> > > > >>
> > > > >> *(408) 438-3156 - mobile*
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Sep 12, 2018 at 8:51 AM Sylvain Lebresne <
> > lebre...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> -0
> > > > >>>
> > > > >>> The project seems to have a hard time getting on top of reviewing
> > his
> > > > >>> backlog
> > > > >>> of 'patch available' issues, so that I'm skeptical adopting more
> > code
> > > > to
> > > > >>> maintain is the thing the project needs the most right now.
> > Besides,
> > > > I'm
> > > > >>> also
> > > > >>> generally skeptical that augmenting the scope of a project makes
> it
> > > > >> better:
> > > > >>> I feel
> > > > >>> keeping this project focused on the core server is better. I see
> > > risks
> > > > >>> here, but
> > > > >>> the upsides haven't been made very clear for me, even for end
> > users:
> > > > yes,
> > > > >>> it
> > > > >>> may provide a tiny bit more clarity around which Golang driver to
> > > > choose
> > > > >> by
> > > > >>> default, but I'm not sure users are that lost, and I think there
> is
> > > > other
> > > > >>> ways to
> > > > >>> solve that if we really want.
> > > > >>>
> > > > >>> Anyway, I reckon I may be overly pessimistic here and it's not
> that
> > > > >> strong
> > > > >>> of
> > > > >>> an objection if a large majority is on-board, so giving my
> opinion
> > > but
> > > > >> not
> > > > >>> opposing.
> > > > >>>
> > > > >>> --
> > > > >>> Sylvain
> > > > >>>
> > > > >>>
> > > > >>> On Wed, Sep 12, 2018 at 5:36 PM Jeremiah D Jordan <
> > > > >>> jeremiah.jor...@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > >  +1
> > > > 
> > > >  But I also think getting this through incubation might take a
> > > while/be
> > > >  impossible given how large the contributor list looks…
> > > > 
> > > > > On Sep 12, 2018, at 10:22 AM, Jeff Jirsa 
> > wrote:
> > > > >
> > > > > +1
> > > > >
> > > > > (Incubation looks like it may be challenging to get acceptance
> > from
> > > > >> all
> > > >  existing contributors, though)
> > > > >
> > > > > --
> > > > > Jeff Jirsa
> > > > >
> > > > >
> > > > >> On Sep 12, 2018, at 8:12 AM, Nate McCall 
> > > > >> wrote:
> > > > >>
> > > > >> This will be the same process used for dtest. We will need to
> > walk
> > > > >> this through the incubator per the process outlined here:
> > > > >>
> > > > >>
> > > > 
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__incubator.apache.org_guides_ip-5Fclearance.html&d=DwIFAg&c=adz96Xi0w1RHqtPMowiL2g&r=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=g-MlYFZVJ7j5Dj_ZfPfa0Ik8Nxco7QsJhTG1TnJH7xI&s=rk5T_t1HZY6PAhN5XgflBhfEtNrcZkVTIvQxixDlw9o&e=
> > > > >>
> > > > >> Pending the outcome of this vote, we will create the JIRA
> issues
> > > for
> > > > >> tracking and after we go through the process, and discuss
> adding
> > > > >> committers in a separate thread (we need to do this atomically
> > > > >> anyway
> > > > >> per general ASF committer adding processes).
> > > > >>
> > > > >> Thanks,
> > > > >> -Nate
> > > > >>
> > > > >>
> > > > >>
> > -
> > > > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > >> For additional commands, e-mail:
> dev-h...@cassandra.apache.org
> > > > >>
> > > > >
> > > > >
> > > ---

Re: TTL of paxos table

[kurt greaves]

It's just that compaction hasn't occured. TTL works even though GCGS is 0.
There's been multiple discussions around this in the past, but we haven't
come to a consensus yet though. Some proposals to change compaction
strategy, some want to make it customisable. Treating paxos as sub-tables
has also come up and the improvements listed in CASSANDRA-13548.
TBH we should probably do 13548 and see where that gets us.

On 30 August 2018 at 13:14, Yuji Ito  wrote:

> Hi,
>
> I wonder why records on the system.paxos table aren't removed, though all
> records are updated with TTL (at least 3 hours).
> That's because gc_grace_seconds of system.paxos table is always 0 and users
> can't change that value of System keyspace!
>
> Why is the TTL of paxos record invalidated?
> Or, I'm misunderstanding?
>
> Related to:
> https://issues.apache.org/jira/browse/CASSANDRA-5451
> https://issues.apache.org/jira/browse/CASSANDRA-13548
>
> Thanks,
> Yuji
>

Re: Reaper as cassandra-admin

[kurt greaves]

2c: There's a lot to think about here, and as Blake already mentioned most
people don't have time to dedicate a lot of thought to this at the moment.
There appear to be a lot of voices missing from the discussion, and I think
it's pretty clear this isn't super tied to the freeze, so maybe we should
leave this discussion until next week when everyone can take part? This
kind of goes for every sidecar related discussion going on at the moment
IMO.

On 29 August 2018 at 16:44, Vinay Chella 
wrote:

> > I haven’t settled on a position yet (will have more time think about
> things after the 9/1 freeze), but I wanted to point out that the argument
> that something new should be written because an existing project has tech
> debt, and we'll do it the right way this time, is a pretty common software
> engineering mistake. The thing you’re replacing usually needs to have some
> really serious problems to make it worth replacing.
>
> Agreed, Yes, I don’t think we should write everything from the scratch, but
> carry forwarding tech debt (if any) and design decisions which makes new
> features in future difficult to develop is something that we need to
> consider. I second Dinesh’s thought on taking the best parts from available
> projects to move forward with the right solution which works great and
> easily pluggable.
>
> -
> Vinay Chella
>
>
> On Tue, Aug 28, 2018 at 10:03 PM Mick Semb Wever  wrote:
>
> >
> > > the argument that something new should be written because an existing
> > project has tech debt, and we'll do it the right way this time, is a
> pretty
> > common software engineering mistake. The thing you’re replacing usually
> > needs to have some really serious problems to make it worth replacing.
> >
> >
> > Thanks for writing this Blake. I'm no fan of writing from scratch.
> Working
> > with other people's code is the joy of open-source, imho.
> >
> > Reaper is not a big project. None of its java files are large or
> > complicated.
> > This is not the C* codebase we're talking about.
> >
> > It comes with strict code style in place (which the build enforces), unit
> > and integration tests. The tech debt that I think of first is removing
> > stuff that we would no longer want to support if it were inside the
> > Cassandra project. A number of recent refactorings  have proved it's an
> > easy codebase to work with.
> >
> > It's also worth noting that Cassandra-4.x adoption is still some away, in
> > which time Reaper will only continue to grow and gain users.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Side Car New Repo vs not

[kurt greaves]

+1 separate repo. I think in-tree only works if you're *not* supporting
multiple versions and each sidecar release is directly tied to a
corresponding C* release. Considering this case is also completely
achievable in a separate repo anyway with minimal overhead we may as well
start that way and see where it goes.

On 24 August 2018 at 17:33, Sam Tunnicliffe  wrote:

> +1 for a separate repo
>
> On Fri, 24 Aug 2018 at 06:40, Michael Shuler 
> wrote:
>
> > +1 for a separate repository.
> >
> > Michael
> >
> > On 08/23/2018 07:30 PM, Murukesh Mohanan wrote:
> > > FWIW, I think it's possible to merge in a separate repository into a
> > > subdirectory while keeping git history, but I don't know if the other
> way
> > > will be possible if commits span other parts of the repo as well\*
> (which
> > > will likely happen sooner or later). So a separate repo is a choice we
> > can
> > > backtrack from if it proves problematic later.
> > >
> > >
> > > \* it may be possible, but the commit messages might not make much
> sense
> > > after that.
> > >
> > > On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, <
> bened...@apache.org>
> > > wrote:
> > >
> > >> +1 also for separate repo
> > >>
> > >>> On 24 Aug 2018, at 01:11, Jeff Jirsa  wrote:
> > >>>
> > >>> +1 for separate repo
> > >>>
> > >>>
> > >>> --
> > >>> Jeff Jirsa
> > >>>
> > >>>
> >  On Aug 23, 2018, at 1:00 PM, sankalp kohli 
> > >> wrote:
> > 
> >  Separate repo is in a majority so far. Please reply to this thread
> > with
> >  your responses.
> > 
> >  On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
> > >> rahul.xavier.si...@gmail.com>
> >  wrote:
> > 
> > > +1 for separate repo. Especially on git. Maybe make it a submodule.
> > >
> > > Rahul
> > > On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski <
> s...@apache.org
> > >,
> > > wrote:
> > >> I'm also currently -1 on the in-tree option.
> > >>
> > >> Additionally to what Aleksey mentioned, I also don't see how we
> > could
> > >> make this work with the current build and release process. Our
> > scripts
> > >> [0] for creating releases (tarballs and native packages), would
> need
> > >> significant work to add support for an independent side-car. Our
> ant
> > >> based build process is also not a great start for adding new
> tasks,
> > >> let
> > >> alone integrating other tool chains for web components for a
> > potential
> > > UI.
> > >>
> > >> [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.
> git
> > >>
> > >>
> > >>> On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > >>> Sure, allow me to elaborate - at least a little bit. But before I
> > do,
> > > just let me note that this wasn’t a veto -1, just a shorthand for
> “I
> > >> don’t
> > > like this option”.
> > >>>
> > >>> It would be nice to have sidecar and C* version and release
> cycles
> > > fully decoupled. I know it *can* be done when in-tree, but the way
> we
> > >> vote
> > > on releases with tags off current branches would have to change
> > >> somehow.
> > > Probably painfully. It would be nice to be able to easily enforce
> > >> freezes,
> > > like the upcoming one, on the whole C* repo, while allowing feature
> > > development on the sidecar. It would be nice to not have sidecar
> > >> commits in
> > > emails from commits@ mailing list. It would be nice to not have C*
> > CI
> > > trigger necessarily on sidecar commits. Groups of people working on
> > >> the two
> > > repos will mostly be different too, so what’s the point in sharing
> > the
> > >> repo?
> > >>>
> > >>> Having an extra repo with its own set of branches is cheap and
> > easy -
> > > we already do that with dtests. I like cleanly separated things
> when
> > > coupling is avoidable. As such I would prefer the sidecar to live
> in
> > a
> > > separate new repo, while still being part of the C* project.
> > >>>
> > >>> —
> > >>> AY
> > >>>
> > >>> On 21 August 2018 at 17:06:39, sankalp kohli (
> > kohlisank...@gmail.com
> > >> )
> > > wrote:
> > >>>
> > >>> Hi Aleksey,
> > >>> Can you please elaborate on the reasons for your -1? This
> > >>> way we can make progress towards any one approach.
> > >>> Thanks,
> > >>> Sankalp
> > >>>
> > >>> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
> > >> alek...@apple.com>
> > >>> wrote:
> > >>>
> >  FWIW I’m strongly -1 on in-tree approach, and would much prefer
> a
> > > separate
> >  repo, dtest-style.
> > 
> >  —
> >  AY
> > 
> >  On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> >  jeremiah.jor...@gmail.com) wrote:
> > 
> >  I think the following is a very big plus of it being in tree:
> > >> * Faster iteration speed in general. For example when we need
> to
> > > add a
> > >

Re: URGENT: CASSANDRA-14092 causes Data Loss

[kurt greaves]

I definitely think we should include it in 4.0. TBH I think it's reasonable
for it to get done after the feature freeze seeing as it is a bug.

On 17 August 2018 at 21:06, Anuj Wadehra 
wrote:

> Hi,
>
> I think CASSANDRA-14227 is pending for long time now. Though, the  data
> loss issue was addressed in CASSANDRA-14092, Cassandra users are still
> prohibited to use long TTLs (20+ years) as the maximum expiration timestamp
> that can be represented by the storage engine is 2038-01-19T03:14:06+00:00
> (due to the encoding of localExpirationTime as an int32). As per JIRA
> comments, the fix seems relatively simple. Considering high impact/returns
> and relatively less efforts, are there any plans to prioritize this fix for
> upcoming releases?
>
> Thanks
> Anuj
>
>
>
>
> On Saturday, 27 January, 2018, 8:35:20 PM IST, Anuj Wadehra <
> anujw_2...@yahoo.co.in.INVALID> wrote:
>
>
>
>
>
> Hi Paulo,
>
> Thanks for coming out with the Emergency Hot Fix!!
> The patch will help many Cassandra users in saving their precious data.
> I think the criticality and urgency of the bug is too high. How can we
> make sure that maximum Cassandra users are alerted about the silent
> deletion problem? What are formal ways of working for broadcasting such
> critical alerts?
> I still see that the JIRA is marked as a "Major" defect and not a
> "Blocker". What worst can happen to a database than irrecoverable silent
> deletion of successfully inserted data. I hope you understand.
>
>
>
> ThanksAnuj
>
>
>
>
>   On Fri, 26 Jan 2018 at 18:57, Paulo Motta
> wrote:  > I have serious concerns regarding reducing the TTL to 15 yrs.The
> patch will immediately break all existing applications in Production which
> are using 15+ yrs TTL.
>
> In order to prevent applications from breaking I will update the patch
> to automatically set the maximum TTL to '03:14:08 UTC 19 January 2038'
> when it overflows and log a warning as a initial measure.  We will
> work on extending this limit or lifting this limitation, probably for
> the 3.0+ series due to the large scale compatibility changes required
> on lower versions, but community patches are always welcome.
>
> Companies that cannot upgrade to a version with the proper fix will
> need to workaround this limitation in some other way: do a batch job
> to delete old data periodically, perform deletes with timestamps in
> the future, etc.
>
> > If its a 32 bit timestamp, can't we just save/read localDeletionTime as
> unsinged int?
>
> The proper fix will likely be along these lines, but this involve many
> changes throughout the codebase where localDeletionTime is consumed
> and extensive testing, reviewing, etc, so we're now looking into a
> emergency hot fix to prevent silent data loss while the permanent fix is
> not in place.
>
> 2018-01-26 6:27 GMT-02:00 Anuj Wadehra :
> > Hi Jeff,
> > One correction in my last message: "it may be more feasible to SUPPORT
> (not extend) the 20 year limit in Cassandra in 2.1/2.2".
> > I completely agree that the existing 20 years TTL support is okay for
> older versions.
> >
> > If I have understood your last message correctly, upcoming patches are
> on following lines :
> >
> > 1. New Patches shall be released for 2.1, 2.2 and 3.x.2. The patches for
> 2.1 & 2.2 would support the existing 20 year TTL limit and ensure that
> there is no data loss when 20 year is set as TTL.3. The patches for 2.1 and
> 2.2 are unlikely to update the sstable format.
> > 4. 3.x patches may even remove the 20 year TTL constraint (and extend
> TTL support beyond 2038).
> > I think that the JIRA priority should be increased from "Major" to
> "Blocker" as the JIRA may cause unexpected data loss. Also, all impacted
> versions should be included in the JIRA. This will attract the due
> attention of all Cassandra users.
> > ThanksAnuj
> >On Friday 26 January 2018, 12:47:18 PM IST, Anuj Wadehra <
> anujw_2...@yahoo.co.in.INVALID> wrote:
> >
> >  Hi Jeff,
> >
> > Thanks for the prompt action! I agree that patching an application MAY
> have a shorter life cycle than patching Cassandra in production. But, in
> the interest of the larger Cassandra user community, we should put our best
> effort to avoid breaking all the affected applications in production. We
> should also consider that updating business logic as per the new 15 year
> TTL constraint may have business implications for many users. I have a
> limited understanding about the complexity of the code patch, but it may be
> more feasible to extend the 20 year limit in Cassandra in 2.1/2.2 rather
> than asking all impacted users to do an immediate business logic
> adaptation. Moreover, now that we officially support Cassandra 2.1 & 2.2
> until 4.0 release and provide critical fixes for 2.1, it becomes even more
> reasonable to provide this extremely critical patch for 2.1 & 2.2 (unless
> its absolutely impossible). Still, many users use Cassandra 2.1 and 2.2 in
> their most critical production systems.
> >
> > Thanks
> > Anuj
> >
> > 

Re: GitHub PR ticket spam

[kurt greaves]

+1

> Or perhaps we should require committers to summarise in the comments.  For
> most tickets, perhaps just stating ’nits from GitHub comments’.  But for
> any complex tickets, summarising the conclusions of any unexpected logical
> or structural discussion would be really helpful for posterity.  This has
> always been true, but especially so now, given how hard the replicated
> comments are to parse.

Yeah this. I think design discussion should continue to at least be
summarised in the comments. The worklog is still going to be painful to
parse, so should really only be used for the minor details and a record
IMO. I don't think we need to make this a hard requirement though, people
already do this so as long as we keep doing it setting the example should
be good enough.


On 7 August 2018 at 02:52, Benedict Elliott Smith 
wrote:

> Also +1
>
> It might perhaps be nice to (just once) have ASF Bot comment that a GitHub
> discussion has been replicated to the worklog? In the Arrow example, for
> instance, it isn’t immediately obvious that there are any worklog comments
> to look at.
>
> Or perhaps we should require committers to summarise in the comments.  For
> most tickets, perhaps just stating ’nits from GitHub comments’.  But for
> any complex tickets, summarising the conclusions of any unexpected logical
> or structural discussion would be really helpful for posterity.  This has
> always been true, but especially so now, given how hard the replicated
> comments are to parse.
>
>
> > On 6 Aug 2018, at 17:18, Jeremiah D Jordan 
> wrote:
> >
> > Oh nice.  I like the idea of keeping it but moving it to the worklog
> tab.  +1 on that from me.
> >
> >> On Aug 6, 2018, at 5:34 AM, Stefan Podkowinski  wrote:
> >>
> >> +1 for worklog option
> >>
> >> Here's an example ticket from Arrow, where they seem to be using the
> >> same approach:
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_ARROW-2D2583&d=DwICaQ&c=adz96Xi0w1RHqtPMowiL2g&r=
> CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=wYZwHSze6YITTXgzOrEvfr_
> onojahtjeJRzGAt8ByzM&s=KWt0xsOv9ESaieg432edGvPhktGkWHxVuLAdNyORiYY&e= <
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__issues.apache.org_jira_browse_ARROW-2D2583&d=DwICaQ&
> c=adz96Xi0w1RHqtPMowiL2g&r=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=
> wYZwHSze6YITTXgzOrEvfr_onojahtjeJRzGAt8ByzM&s=
> KWt0xsOv9ESaieg432edGvPhktGkWHxVuLAdNyORiYY&e=>
> >>
> >>
> >> On 05.08.2018 09:56, Mick Semb Wever wrote:
>  I find this a bit annoying while subscribed to commits@,
>  especially since we created pr@ for these kind of messages. Also I
> don't
>  really see any value in mirroring all github comments to the ticket.
> >>>
> >>>
> >>> I agree with you Stefan. It makes the jira tickets quite painful to
> read. And I tend to make comments on the commits rather than the PRs so to
> avoid spamming back to the jira ticket.
> >>>
> >>> But the linking to the PR is invaluable. And I can see Ariel's point
> about a chronological historical archive.
> >>>
> >>>
>  Ponies would be for this to be mirrored to a tab
>  separate from comments in JIRA.
> >>>
> >>>
> >>> Ariel, that would be the the "worklog" option.
> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__
> reference.apache.org_pmc_github&d=DwICaQ&c=adz96Xi0w1RHqtPMowiL2g&r=
> CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=wYZwHSze6YITTXgzOrEvfr_
> onojahtjeJRzGAt8ByzM&s=1lWQawAO9fITzakpnmdzERuCbZs6IGQsUH_EEIMCMqs&e= <
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__reference.apache.org_pmc_github&d=DwICaQ&c=adz96Xi0w1RHqtPMowiL2g&r=
> CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=wYZwHSze6YITTXgzOrEvfr_
> onojahtjeJRzGAt8ByzM&s=1lWQawAO9fITzakpnmdzERuCbZs6IGQsUH_EEIMCMqs&e=>
> >>>
> >>> If this works for you, and others, I can open a INFRA to switch to
> worklog.
> >>> wdyt?
> >>>
> >>>
> >>> Mick.
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org  dev-unsubscr...@cassandra.apache.org>
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> >>>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org  dev-unsubscr...@cassandra.apache.org>
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org  dev-h...@cassandra.apache.org>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 3.11.3 (Take 2)

[kurt greaves]

+1 nb

On Fri., 27 Jul. 2018, 00:20 Sam Tunnicliffe,  wrote:

> +1
>
> On 25 July 2018 at 08:16, Michael Shuler  wrote:
>
> > I propose the following artifacts for release as 3.11.3.
> >
> > sha1: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.11.3-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1164/org/apache/cassandra/apache-cassandra/3.11.3/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1164/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> > [2]: NEWS.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 2.2.13

[kurt greaves]

+1 nb

On Fri., 27 Jul. 2018, 00:20 Sam Tunnicliffe,  wrote:

> +1
>
> On 25 July 2018 at 08:17, Michael Shuler  wrote:
>
> > I propose the following artifacts for release as 2.2.13.
> >
> > sha1: 3482370df5672c9337a16a8a52baba53b70a4fe8
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.2.13-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1167/org/apache/cassandra/apache-cassandra/2.2.13/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1167/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> > [2]: NEWS.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.0.17 (Take 2)

[kurt greaves]

+1 nb

On Fri., 27 Jul. 2018, 00:20 Sam Tunnicliffe,  wrote:

> +1
>
> On 25 July 2018 at 08:17, Michael Shuler  wrote:
>
> > I propose the following artifacts for release as 3.0.17.
> >
> > sha1: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.0.17-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1165/org/apache/cassandra/apache-cassandra/3.0.17/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1165/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> > [2]: NEWS.txt:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: RangeAwareCompaction for manual token management

[kurt greaves]

I've had similar thoughts in the past about RACS and manual tokens. I think
it would be a good idea to be able to split it based on some configurable
factor other than vnodes. I think Marcus may have already addressed this to
some extent as well but if not it's theoretically possible.


On 20 July 2018 at 07:34, Carl Mueller  wrote:

> I don't want to comment on the 10540 ticket since it seems very well
> focused on vnode-aligned sstable partitioning and compaction. I'm pretty
> excited about that ticket. RACS should enable:
>
> - smaller scale LCS, more constrained I/O consumption
> - less sstables to hit in read path
> - multithreaded/multiprocessor compactions and even serving of data based
> on individual vnode or pools of vnodes
> - better alignment of tombstones with data they should be
> nullifying/eventually removing
> - repair streaming efficiency
> - backups have more granularity for not uploading sstables that didn't
> change for the range since last backup snapshot
>
> There is ongoing discussions as to using Priam for cluster management where
> I am, and as I understand it (superficially) Priam does not use vnodes and
> use manual tokens, and expands via node multiples. I believe it has certain
> advantages over vnodes including expanding by multiple machines at once,
> backups could possibly do (nodecount / RF) number of nodes for data backups
> rather than the mess of vnodes where you have to do basically all of them.
>
> But we could still do some divisor split of the manual range and apply RACS
> to that. I guess this would be vnode-lite. We could have some number like
> 100 subranges on a  node and expansion might just involve temporary lower
> bound count of subranges until the sstables can be reprocessed to the
> typical subrange count.
>
> Is this theoretically correct, or are there glaring things I might have
> missed with respect to RACS-style compaction and manual tokens?
>

Re: JIRAs in Review

[kurt greaves]

Cheers Dinesh, not too worried about that one in 4.0's case though as it's
a bug but it will need a committer.

As for improvements for 4.0:
https://issues.apache.org/jira/browse/CASSANDRA-13010 - More verbose
nodetool compactionstats. Speaks for itself.
https://issues.apache.org/jira/browse/CASSANDRA-10023 - Metrics for number
of local reads/writes. For detecting when you're choosing wrong
coordinators. Useful for those of us without access to clients.
https://issues.apache.org/jira/browse/CASSANDRA-10789 - nodetool blacklist
command to stop bad clients. Would be great for the sysadmin toolbox.

https://issues.apache.org/jira/browse/CASSANDRA-13841 - Smarter nodetool
rebuild. Kind of a bug but would be nice to get it in 4.0 *at least*. (I
probably need to rebase this)
https://issues.apache.org/jira/browse/CASSANDRA-14309 - Hint window
persistence. Would be nice to get some thoughts on this.
https://issues.apache.org/jira/browse/CASSANDRA-12783 - Batchlog refactor
to better support MV's. Been meaning to get back to this one, but it's
pretty much there except needs rebase and a bit more testing. Someone else
to go over it and see if it makes sense would be useful.

May have traction? but worth keeping an eye on.
https://issues.apache.org/jira/browse/CASSANDRA-14291 - Nodetool command to
regenerate SSTable components. Mostly important for efficient
summary/bloomfilter regeneration which doesn't exist apart from using
upgradesstables. Other than that it's effectively upgradesstables but with
a cleaner interface. Chris has started looking at this but would probably
be nice to make sure it gets in before 4.0 seeing as we have no way to
regenerate bloomfilter/summary without re-writing the entire SSTable ATM.

Other than that hoping to get
https://issues.apache.org/jira/browse/CASSANDRA-10540 (RangeAwareCS) in. On
Markus' plate ATM but I'm fairly sure its been decently reviewed.

On 19 July 2018 at 10:07, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> Kurt was looking at some help with this ticket -
> https://issues.apache.org/jira/browse/CASSANDRA-14525
> Dinesh
>
> On Tuesday, July 17, 2018, 12:35:25 PM PDT, sankalp kohli <
> kohlisank...@gmail.com> wrote:
>
>  Hi,
> We are 7 weeks away from 4.0 freeze and there are ~150 JIRAs waiting
> for review. It is hard to know which ones should be prioritized as some of
> them could be not valid(fixes 2.0 bug), some of them will have the assignee
> who no longer is active, etc.
>
> If anyone is *not* getting traction on the JIRA to get it reviewed, please
> use this thread to send your JIRA number and optionally why it is
> important.
>
> Thanks,
> Sankalp
>
>

Re: [VOTE] Release Apache Cassandra 2.2.13

[kurt greaves]

Actually taking back my +1, seems like we've got a fair amount of dtests
(at least 7) failing consistently on 2.2, and another one that's very flaky.
Last completed runs are here:
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-2.2-dtest/116/testReport/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-2.2-dtest-large/68/testReport/

Seeing as we committed to green tests at some point we should probably look
into these.

Also yay for the test results analyzer not working.

On 4 July 2018 at 09:06, Stefan Podkowinski  wrote:

> +1
>
> On 02.07.2018 22:10, Michael Shuler wrote:
> > I propose the following artifacts for release as 2.2.13.
> >
> > sha1: 9ff78249a0a5e87bd04bf9804ef1a3b29b5e1645
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/2.2.13-tentative
> >
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1159/org/apache/cassandra/apache-cassandra/2.2.13/
> >
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1159/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler/
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt)
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> >
> > [2]: (NEWS.txt)
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=NEWS.txt;hb=refs/tags/2.2.13-tentative
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Testing 4.0 Post-Freeze

[kurt greaves]

Correct.

On Wed., 4 Jul. 2018, 15:21 sankalp kohli,  wrote:

> Hi Kurt,
>Thank you for your feedback. I am reading your comment as +1 on
> the idea of working on making a solid release but +0 on branching model. Is
> that correct?
> Thanks,
> Sankalp
>
> On Tue, Jul 3, 2018 at 10:09 PM kurt greaves  wrote:
>
> > >
> > > but by committing to reserving trunk for stabilizing changes until the
> > > beta, we focus our effort on polishing and delivering the release.
> >
> > No, committing to testing, bug fixing, and validating focuses our effort
> on
> > polishing and delivering the release.
> > Committing to reserving trunk for stabilising changes doesn't actually do
> > anything. We could reserve trunk and no one could ever commit a single
> > patch to it, or we could reserve trunk and everyone continues working on
> > features. Whatever we do to trunk doesn't have any impact on how our
> > attention is directed.
> >
> > Granted, you could argue that there is some great beneficial
> psychological
> > aspect and having no feature branch will make everyone suddenly start
> > working on bug fixes, but to people this actually affects (committers),
> > it's not going to mean much to them. They are either already going to be
> > motivated and tasked with getting 4.0 out the door, or they are going to
> do
> > their own thing.
> >
> > I'm fairly positive that better communication around the release, and
> > progress towards the release would be a better motivator. And it would go
> > further to getting new contributors (who definitely don't care about the
> > branches) involved, because anyone on the ML could see progress getting
> > made and be a part of any call to arms.
> >
> > But at the end of the day, bikeshedding about this is not important. This
> > is one of those issues which someone should just stand up and say "We're
> > doing it this way because I said so", because either way, it really
> doesn't
> > matter and the impact is going to be minimal, so we may as well get on
> with
> > our lives.
> >
> > On 4 July 2018 at 04:06, Scott Andreas  wrote:
> >
> > > Here’s why I really like this proposal:
> > >
> > > – It focuses our thought and effort on what it’ll take for each of us
> to
> > > become confident in running Apache Cassandra 4.0 for our most critical
> > > applications *prior* to cutting the dot-zero release.
> > > – It marks a commitment we can make to our user community: delivering
> > > 4.0’s featureset with safety and stability is our top priority.
> > > – There are many different perspectives each contributor can bring:
> > > correctness, performance, durability, automation, documentation,
> > > operational safety, etc; and many ways to gain confidence in each –
> perf
> > > tests, profiling, shadow writes/traffic mirroring, diff tests, replay
> > > tests, fuzz tests, fault injection, chaos engineering, and so many
> > others.
> > > By bringing diverse methods to bear on the same class of problem
> > (quality),
> > > we’re more likely to identify issues that could impact our users and to
> > > ship a better release.
> > > – It’s disciplined and courageous!
> > >
> > > Here’s why I think this won’t discourage new contributors from joining
> –
> > > in fact, the opposite:
> > >
> > > – It provides a very simple step that any contributor can take toward
> > > shipping 4.0: running it.
> > > – We raise spirits by delivering enhancements the community has been
> > > working on for two years rather than fracturing our attention.
> > > – If members of the community are interested in post-4.0 work, they’re
> > not
> > > blocked from making progress toward that – but it’s not the focus, and
> > > unlikely that review bandwidth would be available given that focus. I
> > agree
> > > with Kurt’s note that nobody can be “forced" to do anything - but by
> > > committing to reserving trunk for stabilizing changes until the beta,
> we
> > > focus our effort on polishing and delivering the release.
> > >
> > > On the last question:
> > > > On another note, we are still planning on accepting/reviewing bug
> fixes
> > > for previous versions as well correct?
> > >
> > > I’d expect so – and to take it a step further, it’s likely that this
> > rigor
> > > will result in us identifying serious issues that were not regressions
> in

Re: Testing 4.0 Post-Freeze

[kurt greaves]

>
> but by committing to reserving trunk for stabilizing changes until the
> beta, we focus our effort on polishing and delivering the release.

No, committing to testing, bug fixing, and validating focuses our effort on
polishing and delivering the release.
Committing to reserving trunk for stabilising changes doesn't actually do
anything. We could reserve trunk and no one could ever commit a single
patch to it, or we could reserve trunk and everyone continues working on
features. Whatever we do to trunk doesn't have any impact on how our
attention is directed.

Granted, you could argue that there is some great beneficial psychological
aspect and having no feature branch will make everyone suddenly start
working on bug fixes, but to people this actually affects (committers),
it's not going to mean much to them. They are either already going to be
motivated and tasked with getting 4.0 out the door, or they are going to do
their own thing.

I'm fairly positive that better communication around the release, and
progress towards the release would be a better motivator. And it would go
further to getting new contributors (who definitely don't care about the
branches) involved, because anyone on the ML could see progress getting
made and be a part of any call to arms.

But at the end of the day, bikeshedding about this is not important. This
is one of those issues which someone should just stand up and say "We're
doing it this way because I said so", because either way, it really doesn't
matter and the impact is going to be minimal, so we may as well get on with
our lives.

On 4 July 2018 at 04:06, Scott Andreas  wrote:

> Here’s why I really like this proposal:
>
> – It focuses our thought and effort on what it’ll take for each of us to
> become confident in running Apache Cassandra 4.0 for our most critical
> applications *prior* to cutting the dot-zero release.
> – It marks a commitment we can make to our user community: delivering
> 4.0’s featureset with safety and stability is our top priority.
> – There are many different perspectives each contributor can bring:
> correctness, performance, durability, automation, documentation,
> operational safety, etc; and many ways to gain confidence in each – perf
> tests, profiling, shadow writes/traffic mirroring, diff tests, replay
> tests, fuzz tests, fault injection, chaos engineering, and so many others.
> By bringing diverse methods to bear on the same class of problem (quality),
> we’re more likely to identify issues that could impact our users and to
> ship a better release.
> – It’s disciplined and courageous!
>
> Here’s why I think this won’t discourage new contributors from joining –
> in fact, the opposite:
>
> – It provides a very simple step that any contributor can take toward
> shipping 4.0: running it.
> – We raise spirits by delivering enhancements the community has been
> working on for two years rather than fracturing our attention.
> – If members of the community are interested in post-4.0 work, they’re not
> blocked from making progress toward that – but it’s not the focus, and
> unlikely that review bandwidth would be available given that focus. I agree
> with Kurt’s note that nobody can be “forced" to do anything - but by
> committing to reserving trunk for stabilizing changes until the beta, we
> focus our effort on polishing and delivering the release.
>
> On the last question:
> > On another note, we are still planning on accepting/reviewing bug fixes
> for previous versions as well correct?
>
> I’d expect so – and to take it a step further, it’s likely that this rigor
> will result in us identifying serious issues that were not regressions in
> 4.0 warranting backports. As with any investment in testing / validation,
> I’m hoping we do … but also hoping we don’t. :-)
>
>
> > On Jul 3, 2018, at 7:21 PM, kurt greaves  wrote:
> >
> > tl;dr: +1 for committing to testing + bugfixing after feature freeze,
> but I
> > can't really see how changing the branching strategy is going to have any
> > impact at all.
> >
> > I really don't think it's a big deal. People will work based on whatever
> > priorities they've been assigned, regardless of what you do to the
> > branching. That's essentially just red tape and adding unnecessary
> > complexity to an already established process. Granted, you'll have to do
> > one less merge, but it should be an effortless merge, and it saves there
> > being any confusion about what people should do with features. If someone
> > decided to commit a feature, they could, and we wouldn't have to be all
> > over every ticket making sure people don't commit things that shouldn't
> be
> > committed

Re: Testing 4.0 Post-Freeze

[kurt greaves]

tl;dr: +1 for committing to testing + bugfixing after feature freeze, but I
can't really see how changing the branching strategy is going to have any
impact at all.

I really don't think it's a big deal. People will work based on whatever
priorities they've been assigned, regardless of what you do to the
branching. That's essentially just red tape and adding unnecessary
complexity to an already established process. Granted, you'll have to do
one less merge, but it should be an effortless merge, and it saves there
being any confusion about what people should do with features. If someone
decided to commit a feature, they could, and we wouldn't have to be all
over every ticket making sure people don't commit things that shouldn't be
committed.

Cutting to the point, all we're really aiming for here is a commitment from
the community to only dev on bugfixes, only review bugfixes, and
testing/validation during the freeze period. That's not going to be
enforced by a change in branching strategy, it's going to be enforced by
the contributors themselves (and their respective priority-setters).
The best we can do is encourage a commitment to testing and bugfixing for
the freeze period. This is a volunteer based project after all, so we can't
really force anyone to do anything. If contributors make proposals for
features in the freeze period we can just tell them that there is a focus
on 4.0 testing and it's unlikely to be reviewed until 4.0 is released at
best.

On another note, we are still planning on accepting/reviewing bug fixes for
previous versions as well correct?  Just to clarify because it doesn't seem
to be mentioned here and we don't want people getting in arguments over
reviewing patches that only affect older releases.

I think not having your patch reviewed for months is more
> discouraging than following the community and helping with stability of
> 4.0.

Patches not getting reviewed for months on end already occurs. Changing how
we branch isn't going to change this or make anyone feel better about it.
The best we can do is communicate why their patches aren't getting
reviewed, and we should be doing this on individual feature tickets that
get raised and on the ML.

On 4 July 2018 at 00:44, Mick Semb Wever  wrote:

> >
> >
> > We propose that between the September freeze date and beta, a new branch
> > would not be created and trunk would only have bug fixes and performance
> > improvements committed to it.
> >
>
>
> +1, like really yes!! because it's a step towards a more stable-master
> project.
>
> If trunk is focused on stabilising (which ideally it shouldn't have to, if
> stable-master were true) then new features remaining in their respective
> branches shouldn't be a big cost or risk (nor seen as anything negative).
> And hopefully any additional practices/lessons learnt from during the
> freeze period get carried on for good.
>
> Although, and unfortunately, there's just not the testing infrastructure
> available to the public to ensure feature branches are solid before
> merging. But I struggle to see that as an excuse to only "stabilise" on
> release branches.
>
> 2¢,
> mick
>

Re: [VOTE] Release Apache Cassandra 2.2.13

[kurt greaves]

+1 nb

On Wed., 4 Jul. 2018, 03:26 Brandon Williams,  wrote:

> +1
>
> On Mon, Jul 2, 2018 at 3:10 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 2.2.13.
> >
> > sha1: 9ff78249a0a5e87bd04bf9804ef1a3b29b5e1645
> > Git: http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=sho
> > rtlog;h=refs/tags/2.2.13-tentative
> > Artifacts: https://repository.apache.org/content/repositories/orgapache
> > cassandra-1159/org/apache/cassandra/apache-cassandra/2.2.13/
> > Staging repository: https://repository.apache.org/
> > content/repositories/orgapachecassandra-1159/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler/
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) http://git-wip-us.apache.org/r
> > epos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/
> > tags/2.2.13-tentative
> > [2]: (NEWS.txt) http://git-wip-us.apache.org/r
> > epos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tag
> > s/2.2.13-tentative
> >
> > --
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Tombstone passed GC period causes un-repairable inconsistent data

[kurt greaves]

That makes sense going forward (assuming it works), but this is still
pretty surprising behaviour. Although disregarding the read repair factor
entirely, the result will *eventually* come true when the tombstones are
purged, we're still returning a result that doesn't match up with what we
have on disk. I guess the question becomes whether it is less surprising to
return what is actually on disk versus what the data will look like in the
future. Read repair throws a spanner in the works because it could fix this
issue, but there's no guarantee that read repair would fix this issue
(tombstones may be purged prior to the read). TBH the read repair not
working is what makes it most noticeable, and more surprising. If it did
work you'd probably be in for potentially nastier surprises in the future,
and harder to identify. Ideally the read repair wouldn't be attempted at
all and no one would ever investigate what's in the SSTables :p. That may
be possible and not terribly disruptive, but in the scheme of things it's
just nitpicking.

Anyway, at the very least we should document (if we haven't aleady) this
behaviour, because it's pretty complicated and someone is bound to run into
it again. I know I'm probably going to forget about it and get confused
when we run into it again.



On 20 June 2018 at 16:35, sankalp kohli  wrote:

> I agree with Stefan that we should use incremental repair and use patches
> from Marcus to drop tombstones only from repaired data.
> Regarding deep repair, you can bump the read repair and run the repair. The
> issue will be that you will stream lot of data and also your blocking read
> repair will go up when you bump the gc grace to higher value.
>
> On Wed, Jun 20, 2018 at 1:10 AM Stefan Podkowinski 
> wrote:
>
> > Sounds like an older issue that I tried to address two years ago:
> > https://issues.apache.org/jira/browse/CASSANDRA-11427
> >
> > As you can see, the result hasn't been as expected and we got some
> > unintended side effects based on the patch. I'm not sure I'd be willing
> > to give this another try, considering the behaviour we like to fix in
> > the first place is rather harmless and the read repairs shouldn't happen
> > at all to any users who regularly run repairs within gc_grace.
> >
> > What I'd suggest is to think more into the direction of a
> > post-full-repair-world and to fully embrace incremental repairs, as
> > fixed by Blake in 4.0. In that case, we should stop doing read repairs
> > at all for repaired data, as described in
> > https://issues.apache.org/jira/browse/CASSANDRA-13912. RRs are certainly
> > useful, but can be very risky if not very very carefully implemented. So
> > I'm wondering if we shouldn't disable RRs for everything but unrepaired
> > data. I'd btw also be interested to hear any opinions on this in context
> > of transient replicas.
> >
> >
> > On 20.06.2018 03:07, Jay Zhuang wrote:
> > > Hi,
> > >
> > > We know that the deleted data may re-appear if repair is not run within
> > > gc_grace_seconds. When the tombstone is not propagated to all nodes,
> the
> > > data will re-appear. But it's also causing following 2 issues before
> the
> > > tombstone is compacted away:
> > > a. inconsistent query result
> > >
> > > With consistency level ONE or QUORUM, it may or may not return the
> value.
> > > b. lots of read repairs, but doesn't repair anything
> > >
> > > With consistency level ALL, it always triggers a read repair.
> > > With consistency level QUORUM, it also very likely (2/3) causes a read
> > > repair. But it doesn't repair the data, so it's causing repair every
> > time.
> > >
> > >
> > > Here are the reproducing steps:
> > >
> > > 1. Create a 3 nodes cluster
> > > 2. Create a table (with small gc_grace_seconds):
> > >
> > > CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> > > 'replication_factor': 3};
> > > CREATE TABLE foo.bar (
> > > id int PRIMARY KEY,
> > > name text
> > > ) WITH gc_grace_seconds=30;
> > >
> > > 3. Insert data with consistency all:
> > >
> > > INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');
> > >
> > > 4. stop 1 node
> > >
> > > $ ccm node2 stop
> > >
> > > 5. Delete the data with consistency quorum:
> > >
> > > DELETE FROM foo.bar WHERE id=1;
> > >
> > > 6. Wait 30 seconds and then start node2:
> > >
> > > $ ccm node2 start
> > >
> > > Now the tombstone is on node1 and node3 but not on node2.
> > >
> > > With quorum read, it may or may not return value, and read repair will
> > send
> > > the data from node2 to node1 and node3, but it doesn't repair anything.
> > >
> > > I'd like to discuss a few potential solutions and workarounds:
> > >
> > > 1. Can hints replay sends GCed tombstone?
> > >
> > > 2. Can we have a "deep repair" which detects such issue and repair the
> > GCed
> > > tombstone? Or temperately increase the gc_grace_seconds for repair?
> > >
> > > What other suggestions you have if the user is having such issue?
> > >
> > >
> > > Thanks,
> > >
> > > Jay
> > >
> >

Re: Tombstone passed GC period causes un-repairable inconsistent data

[kurt greaves]

We've seen this before but couldn't tie it to GCGS so we ended up
forgetting about it. Now with a reproducible test case things make much
more sense and we should be able to fix this.
Seems that it's most certainly a bug with partition deletions and handling
of GC grace seconds. It seems that the read repair isn't propagating the
partition level deletion in the case where the deletion is after GCGS.
Increasing GCGS to beyond the timestamp of the problematic deletion and
then causing a read repair will fix the issue as a workaround. Only way to
avoid the issue entirely is to repair the deletion within GCGS.

Looking at finding the bug now... I took the liberty of creating a JIRA for
the issue: https://issues.apache.org/jira/browse/CASSANDRA-14532


On 20 June 2018 at 01:07, Jay Zhuang  wrote:

> Hi,
>
> We know that the deleted data may re-appear if repair is not run within
> gc_grace_seconds. When the tombstone is not propagated to all nodes, the
> data will re-appear. But it's also causing following 2 issues before the
> tombstone is compacted away:
> a. inconsistent query result
>
> With consistency level ONE or QUORUM, it may or may not return the value.
> b. lots of read repairs, but doesn't repair anything
>
> With consistency level ALL, it always triggers a read repair.
> With consistency level QUORUM, it also very likely (2/3) causes a read
> repair. But it doesn't repair the data, so it's causing repair every time.
>
>
> Here are the reproducing steps:
>
> 1. Create a 3 nodes cluster
> 2. Create a table (with small gc_grace_seconds):
>
> CREATE KEYSPACE foo WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE TABLE foo.bar (
> id int PRIMARY KEY,
> name text
> ) WITH gc_grace_seconds=30;
>
> 3. Insert data with consistency all:
>
> INSERT INTO foo.bar (id, name) VALUES(1, 'cstar');
>
> 4. stop 1 node
>
> $ ccm node2 stop
>
> 5. Delete the data with consistency quorum:
>
> DELETE FROM foo.bar WHERE id=1;
>
> 6. Wait 30 seconds and then start node2:
>
> $ ccm node2 start
>
> Now the tombstone is on node1 and node3 but not on node2.
>
> With quorum read, it may or may not return value, and read repair will send
> the data from node2 to node1 and node3, but it doesn't repair anything.
>
> I'd like to discuss a few potential solutions and workarounds:
>
> 1. Can hints replay sends GCed tombstone?
>
> 2. Can we have a "deep repair" which detects such issue and repair the GCed
> tombstone? Or temperately increase the gc_grace_seconds for repair?
>
> What other suggestions you have if the user is having such issue?
>
>
> Thanks,
>
> Jay
>

Re: 3.11.3

[kurt greaves]

Another one worth mentioning (that I missed because I neglected 3.0) would
be https://issues.apache.org/jira/browse/CASSANDRA-14419

Tommy has provided a backport of CASSANDRA-11960 which supposedly fixes the
issue, just needs a review.

On Thu., 7 Jun. 2018, 11:29 kurt greaves,  wrote:

> Oh yeah forgot about those branches, but yes definitely +1 to those.
>
> On 6 June 2018 at 14:55, Michael Shuler  wrote:
>
>> +1 for all the active branches. It will also be a good chance to review
>> the release doc addition Mick so graciously wrote up.
>> https://github.com/apache/cassandra/pull/230
>>
>> --
>> Michael
>>
>>
>> On 06/06/2018 07:30 AM, Jason Brown wrote:
>>
>>> wrt releasing 3.11, +1 to this ... and additionally, 3.0 and 2.2.
>>>
>>> I'd propose we cut early next week (Monday) and anything that can get
>>> reviewed/committed from Kurt's list would be great.
>>>
>>> On Wed, Jun 6, 2018 at 5:22 AM, kurt greaves 
>>> wrote:
>>>
>>> We probably need to release 3.11.3 at some point soon because there's
>>>> some
>>>> pretty major bug fixes in there and interest has been expressed multiple
>>>> times now for a release.
>>>>
>>>> Currently the only tickets marked for 3.11.3 that aren't complete are:
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14355 - No work has
>>>> started
>>>> on this yet.
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13929 - A lot of work
>>>> has
>>>> been done here... just need a final review
>>>>
>>>> However we've got quite a few tickets with patches that should probably
>>>> be
>>>> considered - can't say all are necessary to get into 3.11.3 but seeing
>>>> as
>>>> they all have patch available and have been around :
>>>> for a while they are worth considering for anyone who could kindly
>>>> review:
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14242 - Inconsistent
>>>> indexing on static columns :(. needs review
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14415 - performance
>>>> regression fix for DISTINCT
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14204 - Straightforward
>>>> fix
>>>> for nodetool garbagecollect
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14167 - Pretty much
>>>> ready
>>>> for commit - has been given +1 by Sylvain
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14099 - Needs only a
>>>> unit
>>>> test
>>>> https://issues.apache.org/jira/browse/CASSANDRA-14085 - performance
>>>> fix,
>>>> small patch needs review
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13935 - Simple fix to
>>>> CQL
>>>> output for indexes when dumping schema
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13917 - Needs review,
>>>> fix
>>>> for compact storage inserts.
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13843 - Needs review.
>>>> Small
>>>> debian packaging fix for heapdump location.
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13834 - Needs an
>>>> "official"
>>>> reviewer and probably a test.
>>>> https://issues.apache.org/jira/browse/CASSANDRA-13618 - Can likely be
>>>> committed. Jeff has already done all the work for it just need to verify
>>>> tests.
>>>> https://issues.apache.org/jira/browse/CASSANDRA-11479 - Waiting for
>>>> review
>>>> for quite a long time.. unit test fix but seems to include changes to
>>>> the
>>>> server
>>>>
>>>> Handy JQL:
>>>> project = CASSANDRA AND ((fixVersion = 3.11.x and status = "Patch
>>>> Available" ) OR (fixVersion = 3.11.3 AND status != Resolved)) and Type =
>>>> Bug
>>>>
>>>>
>>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
>

Re: 3.11.3

[kurt greaves]

Oh yeah forgot about those branches, but yes definitely +1 to those.

On 6 June 2018 at 14:55, Michael Shuler  wrote:

> +1 for all the active branches. It will also be a good chance to review
> the release doc addition Mick so graciously wrote up.
> https://github.com/apache/cassandra/pull/230
>
> --
> Michael
>
>
> On 06/06/2018 07:30 AM, Jason Brown wrote:
>
>> wrt releasing 3.11, +1 to this ... and additionally, 3.0 and 2.2.
>>
>> I'd propose we cut early next week (Monday) and anything that can get
>> reviewed/committed from Kurt's list would be great.
>>
>> On Wed, Jun 6, 2018 at 5:22 AM, kurt greaves 
>> wrote:
>>
>> We probably need to release 3.11.3 at some point soon because there's some
>>> pretty major bug fixes in there and interest has been expressed multiple
>>> times now for a release.
>>>
>>> Currently the only tickets marked for 3.11.3 that aren't complete are:
>>> https://issues.apache.org/jira/browse/CASSANDRA-14355 - No work has
>>> started
>>> on this yet.
>>> https://issues.apache.org/jira/browse/CASSANDRA-13929 - A lot of work
>>> has
>>> been done here... just need a final review
>>>
>>> However we've got quite a few tickets with patches that should probably
>>> be
>>> considered - can't say all are necessary to get into 3.11.3 but seeing as
>>> they all have patch available and have been around :
>>> for a while they are worth considering for anyone who could kindly
>>> review:
>>> https://issues.apache.org/jira/browse/CASSANDRA-14242 - Inconsistent
>>> indexing on static columns :(. needs review
>>> https://issues.apache.org/jira/browse/CASSANDRA-14415 - performance
>>> regression fix for DISTINCT
>>> https://issues.apache.org/jira/browse/CASSANDRA-14204 - Straightforward
>>> fix
>>> for nodetool garbagecollect
>>> https://issues.apache.org/jira/browse/CASSANDRA-14167 - Pretty much
>>> ready
>>> for commit - has been given +1 by Sylvain
>>> https://issues.apache.org/jira/browse/CASSANDRA-14099 - Needs only a
>>> unit
>>> test
>>> https://issues.apache.org/jira/browse/CASSANDRA-14085 - performance fix,
>>> small patch needs review
>>> https://issues.apache.org/jira/browse/CASSANDRA-13935 - Simple fix to
>>> CQL
>>> output for indexes when dumping schema
>>> https://issues.apache.org/jira/browse/CASSANDRA-13917 - Needs review,
>>> fix
>>> for compact storage inserts.
>>> https://issues.apache.org/jira/browse/CASSANDRA-13843 - Needs review.
>>> Small
>>> debian packaging fix for heapdump location.
>>> https://issues.apache.org/jira/browse/CASSANDRA-13834 - Needs an
>>> "official"
>>> reviewer and probably a test.
>>> https://issues.apache.org/jira/browse/CASSANDRA-13618 - Can likely be
>>> committed. Jeff has already done all the work for it just need to verify
>>> tests.
>>> https://issues.apache.org/jira/browse/CASSANDRA-11479 - Waiting for
>>> review
>>> for quite a long time.. unit test fix but seems to include changes to the
>>> server
>>>
>>> Handy JQL:
>>> project = CASSANDRA AND ((fixVersion = 3.11.x and status = "Patch
>>> Available" ) OR (fixVersion = 3.11.3 AND status != Resolved)) and Type =
>>> Bug
>>>
>>>
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

3.11.3

[kurt greaves]

We probably need to release 3.11.3 at some point soon because there's some
pretty major bug fixes in there and interest has been expressed multiple
times now for a release.

Currently the only tickets marked for 3.11.3 that aren't complete are:
https://issues.apache.org/jira/browse/CASSANDRA-14355 - No work has started
on this yet.
https://issues.apache.org/jira/browse/CASSANDRA-13929 - A lot of work has
been done here... just need a final review

However we've got quite a few tickets with patches that should probably be
considered - can't say all are necessary to get into 3.11.3 but seeing as
they all have patch available and have been around :
for a while they are worth considering for anyone who could kindly review:
https://issues.apache.org/jira/browse/CASSANDRA-14242 - Inconsistent
indexing on static columns :(. needs review
https://issues.apache.org/jira/browse/CASSANDRA-14415 - performance
regression fix for DISTINCT
https://issues.apache.org/jira/browse/CASSANDRA-14204 - Straightforward fix
for nodetool garbagecollect
https://issues.apache.org/jira/browse/CASSANDRA-14167 - Pretty much ready
for commit - has been given +1 by Sylvain
https://issues.apache.org/jira/browse/CASSANDRA-14099 - Needs only a unit
test
https://issues.apache.org/jira/browse/CASSANDRA-14085 - performance fix,
small patch needs review
https://issues.apache.org/jira/browse/CASSANDRA-13935 - Simple fix to CQL
output for indexes when dumping schema
https://issues.apache.org/jira/browse/CASSANDRA-13917 - Needs review, fix
for compact storage inserts.
https://issues.apache.org/jira/browse/CASSANDRA-13843 - Needs review. Small
debian packaging fix for heapdump location.
https://issues.apache.org/jira/browse/CASSANDRA-13834 - Needs an "official"
reviewer and probably a test.
https://issues.apache.org/jira/browse/CASSANDRA-13618 - Can likely be
committed. Jeff has already done all the work for it just need to verify
tests.
https://issues.apache.org/jira/browse/CASSANDRA-11479 - Waiting for review
for quite a long time.. unit test fix but seems to include changes to the
server

Handy JQL:
project = CASSANDRA AND ((fixVersion = 3.11.x and status = "Patch
Available" ) OR (fixVersion = 3.11.3 AND status != Resolved)) and Type = Bug

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[kurt greaves]

Seems pretty straightforward to me. Create a python 3 version as soon as
possible and make it available, keep the python 2.7 version as default
until the next major release after 4.0 (assuming around/after python 2.7
EOL), then switch default and leave continued support for 2.7 cqlsh up to
the community and whoever needs it.

On Sat., 2 Jun. 2018, 04:45 J. D. Jordan,  wrote:

> All for using six and supporting both. Sorry, I read your initial email as
> wanting to drop support for 2 at the end of the year.
>
> > On Jun 1, 2018, at 1:01 PM, Jonathan Haddad  wrote:
> >
> > And that's why I said supporting both with six is the right path
> > forward, later dropping support for 2.  I'm not advocating we drop 2
> > support now, and I'm not asking for any sort of commitment.  I didn't
> > think adding support for 3 would be so controversial.
> > On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
> >  wrote:
> >>
> >> The community of people doing python development and the community of
> people running Cassandra servers are not the same.  I am not fine riding
> the coat tails of libraries used in python development.  As others have
> stated we need to be following the lead of the OS vendors that people will
> be deploying Cassandra on top of.  And those will not be dropping Python 2
> at the end of the year.
> >>
> >> -Jeremiah
> >>
> >>> On Jun 1, 2018, at 12:37 PM, Jonathan Haddad 
> wrote:
> >>>
> >>> Both can work.  I did a lot of the work on the port of the Python
> >>> driver's object mapper (formerly cqlengine) to Python 3.  It's
> >>> reasonably straightforward if you use the six library.
> >>>
> >>> Both pandas and numpy are dropping support for Python 2 at the end of
> >>> this year.  I'm fine with riding on their coattails.
>  On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman 
> wrote:
> 
>  Support for, but not the very script, right? Because, as gently
> pointed
>  out by several realists here, Python 2 is far from dead and arguably
>  still the majority usage. That's only just now beginning to change. I
>  think it will be more than 2 years before people begin asking what
>  Python 2 was.
> 
> 
> > On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > Supporting both as a next step is logical, removing support for 2 in
> the
> > next year or two seems reasonable enough. Gotta rip the band aid off
> at
> > some point.
> >
> >> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman 
> wrote:
> >>
> >> Hi,
> >>
> >> Deprecating in this context does not mean removing it or it being
> >> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> >> refers to future versions (>7), but there are none at this point. It
> >> appears Ubuntu has deviated from Debian in this sense, but Debian
> has
> >> not changed yet (likely Debian 10 will, but that's not out yet and
> has
> >> no announced release date).
> >>
> >> Thus, 2.x still remains the most used version for servers. And
> servers
> >> deployed at this point of time will use these versions for years.
> >>
> >>   - Micke
> >>
> >>
> >>> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
>  On 2018/06/01 07:40:04, Michael Burman 
> wrote:
>  IIRC, there's no major distribution yet that defaults to Python 3
> (I
>  think Ubuntu & Debian are still defaulting to Python 2 also).
> This will
>  happen eventually (maybe), but not yet. Discarding Python 2
> support
>  would mean more base-OS work for most people wanting to run
> Cassandra
>  and that's not a positive thing.
> 
> >>> Ubuntu since 16.04 defaults to Python 3:
> >>>
>  Python2 is not installed anymore by default on the server, cloud
> and
> >> the touch images, long live Python3! Python3 itself has been
> upgraded to
> >> the 3.5 series. -
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA&s=9OWAbO26grwiI2ly_-gAGBqJP9Mv6KPAKJyQu_OEDPc&e=
> >>> RHEL 7.5 deprecates Python 2 (
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_documentation_en-2Dus_red-5Fhat-5Fenterprise-5Flinux_7_html_7.5-5Frelease-5Fnotes_chap-2Dred-5Fhat-5Fenterprise-5Flinux-2D7.5-5Frelease-5Fnotes-2Ddeprecated-5Ffunctionality&d=DwIBaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CNZK3RiJDLqhsZDG6FQGnXn8WyPRCQhp4x_uBICNC0g&m=J5Su6wvm91QrOBcici7HyIiFiyzjrg8UnamYu8qtSRA&s=CDFufWbcvq6VpoLJQVbCQP9rpvIv3ssNtKMQce-1vwU&e=
> >> ).
> >>>
> >>>
> >>>
> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>
> >>
> 

Re: [DISCUSS] Cassandra and future Java

[kurt greaves]

I suppose so, I guess just highlighting that we shouldn't bother dedicating
much testing resources to 4.0 on 11.

I will note however that if we wanted to start making changes to support 11
it'd have to be (as Robert alluded to) based off an incomplete 11 as I'm
assuming we'd make no further changes after 4.0 is frozen and instead
they'd go to 4.1.

On Wed., 30 May 2018, 20:40 Stefan Podkowinski,  wrote:

> That's probably not far off what Robert suggested:
>
> "The idea here is to default to Java 8, but the code also runs on 11"
>
> "Initially, only the combination of C* 4.0 + Java 8 would be labeled as
> "stable" and the combination of C* 4.0 + Java 11 as "experimental"."
>
> If Robert wants to go ahead making the code "also run Java 11", while we
> keep testing for and officially supporting Java 8, then I can't really
> think of any argument against that, as long as we don't end up with tons
> of version based code toggles, or too risky changes regarding stability
> in general. We would still release 4.0 for Java 8 and afterwards
> officially switch to 11 in 4.1, based on the work done in #9608 and
> already merged into 4.0. Downloads and packages for 4.0 would be
> released for Java 8, while running 4.0 with Java 11 would produce a
> warning message indicating that it's experimental.
>
>
> On 30.05.2018 08:54, kurt greaves wrote:
> > So for anyone that missed it, Java 11 will be released in September 2018.
> >
> > I'd prefer we target one Java release only. This is purely because we
> don't
> > have the capacity or capability to test both releases. We hardly do a
> good
> > enough job as it is of testing and lumping another JVM into the mix is
> just
> > going to complicate things a lot and all the effort we expend into
> testing
> > both releases is probably better off just spent focusing on one.
> >
> > At this point I don't think there is *much* value in supporting 11 for
> 4.0,
> > seeing as we won't be able to fully utilise features in 11 as our feature
> > freeze for 4.0 will occur before 11 is released. There is obviously the
> > support problem but adoptOpenJDK are claiming they'll support Java 8
> until
> > September 2022 (https://adoptopenjdk.net/support.html) - which on top of
> > all the existing releases is probably good enough for us, and 2022 is far
> > enough away that hopefully 4.0 will be EOL'd by then. I don't think it's
> a
> > big risk that support for Java 8 will stop anytime soon, it's pretty
> > widespread and it's going to take people a *long* time to get off 8.
> >
> > It would make much more sense to me to support 11 in 4.1 that way we can
> > actually utilise any benefits of 11.
> >
> > On 29 May 2018 at 12:22, Robert Stupp  wrote:
> >
> >> Ideally, CI would run against both Java 8 and 11. I’ve no clue about
> b.a.o
> >> though.
> >>
> >> There will definitely be a log of smaller issues - both for OpenJDK 8
> and
> >> 11.
> >> I think, it’s sufficient to deal with the Linux distros' (RH/deb)
> openjdk
> >> dependencies - just making sure, that we’re using the right Java
> version -
> >> and not let the package manger just pull the newest available.
> >> The version-string from adoptopenjdk for example is one of these “minor
> >> issues"...
> >>
> >> —
> >> Robert Stupp
> >> @snazy
> >>
> >> On 28. May 2018, at 15:46, Stefan Podkowinski  wrote:
> >>
> >> The main issue that I see, for supporting both Java 8 + 11, is testing.
> >> We should first decide how this would effect builds.apache.org, or how
> >> we're going to do CI testing in general for that situation.
> >>
> >> There are probably also smaller issues that we're not aware of yet, such
> >> as which Java dependency to use for our deb and rpm packages,
> >> differences in Java distributions (Oracle, AdoptOpenJDK, Redhat,..) and
> >> so on. I'd expect we could deal with this on the Java side, but the
> >> infra, scripting and testing implications give me a greater headache
> >> when thinking of it.
> >>
> >>
> >> On 25.05.2018 15:33, J. D. Jordan wrote:
> >>
> >> +1 for “Option 3: both 8 + 11” it shouldn’t be too hard to maintain code
> >> wise, and leaves people’s options open.
> >>
> >> -Jeremiah
> >>
> >> On May 25, 2018, at 6:31 AM, Robert Stupp  wrote:
> >>
> >> I'd like t

Re: [DISCUSS] Cassandra and future Java

[kurt greaves]

So for anyone that missed it, Java 11 will be released in September 2018.

I'd prefer we target one Java release only. This is purely because we don't
have the capacity or capability to test both releases. We hardly do a good
enough job as it is of testing and lumping another JVM into the mix is just
going to complicate things a lot and all the effort we expend into testing
both releases is probably better off just spent focusing on one.

At this point I don't think there is *much* value in supporting 11 for 4.0,
seeing as we won't be able to fully utilise features in 11 as our feature
freeze for 4.0 will occur before 11 is released. There is obviously the
support problem but adoptOpenJDK are claiming they'll support Java 8 until
September 2022 (https://adoptopenjdk.net/support.html) - which on top of
all the existing releases is probably good enough for us, and 2022 is far
enough away that hopefully 4.0 will be EOL'd by then. I don't think it's a
big risk that support for Java 8 will stop anytime soon, it's pretty
widespread and it's going to take people a *long* time to get off 8.

It would make much more sense to me to support 11 in 4.1 that way we can
actually utilise any benefits of 11.

On 29 May 2018 at 12:22, Robert Stupp  wrote:

> Ideally, CI would run against both Java 8 and 11. I’ve no clue about b.a.o
> though.
>
> There will definitely be a log of smaller issues - both for OpenJDK 8 and
> 11.
> I think, it’s sufficient to deal with the Linux distros' (RH/deb) openjdk
> dependencies - just making sure, that we’re using the right Java version -
> and not let the package manger just pull the newest available.
> The version-string from adoptopenjdk for example is one of these “minor
> issues"...
>
> —
> Robert Stupp
> @snazy
>
> On 28. May 2018, at 15:46, Stefan Podkowinski  wrote:
>
> The main issue that I see, for supporting both Java 8 + 11, is testing.
> We should first decide how this would effect builds.apache.org, or how
> we're going to do CI testing in general for that situation.
>
> There are probably also smaller issues that we're not aware of yet, such
> as which Java dependency to use for our deb and rpm packages,
> differences in Java distributions (Oracle, AdoptOpenJDK, Redhat,..) and
> so on. I'd expect we could deal with this on the Java side, but the
> infra, scripting and testing implications give me a greater headache
> when thinking of it.
>
>
> On 25.05.2018 15:33, J. D. Jordan wrote:
>
> +1 for “Option 3: both 8 + 11” it shouldn’t be too hard to maintain code
> wise, and leaves people’s options open.
>
> -Jeremiah
>
> On May 25, 2018, at 6:31 AM, Robert Stupp  wrote:
>
> I'd like to bring up the C*/Java discussion again. It's been a while since
> we've discussed this.
>
> To me it sounds like there's still the question about which version(s) of
> Java we want to support beginning with C* 4.0.
>
> I assume, that it's legit (and probably very necessary) to assume that
> OpenJDK is now (i.e. after Java 6) considered as "production ready" for C*.
> The public (and legal and free) availability of Oracle's Java 8 will end in
> January 2019 (unless you're using it privately on your desktop). Java 9 and
> 10 are not a thing, as both will be EOL when the C* 4.0 branch is about to
> be cut. The most recent available Java version will be 11, which is meant
> to be publicly available from Oracle until March 2019 and should get LTS
> support for OpenJDK 11 from major Linux distros (RHEL and derivates,
> Ubuntu, Azul Zulu).
>
> (Side note: adoptopenjdk is different here, because it does not include
> the patch version in the version banner (java.version=1.8.0-adoptopenjdk),
> so difficult to check the minimum patch version on startup of C*.)
>
> (Attn, rant: I'm not particularly happy with the new release and support
> model for Java, because developing something now, that's about to release
> end of the year on a Java version that has not even reached
> feature-complete status, is, gently speaking, difficult. But sticking to an
> "antique" Java version (8) has its own risks as well.)
>
> I'm silently ignoring any Java release, that's not aimed to get any
> LTS(-ish?) support from anybody - so only Java 8 + 11 remain.
>
> There are generally three (IMO legit) options here: only support Java 8,
> only support Java 11, support both Java 8 and Java 11. All three options
> have a bunch of pros and cons.
>
> Option 1, only Java 8: Probably the safest option. Considering the
> potential lifetimes of Java 8 and C* 4.0, even the most enthusiastic
> maintainers may stop backporting security or bug fixes to OpenJDK 8. It
> might not be an issue in practice, but if there's for example a severe
> issue in the SSL/TLS area and nobody fixes it in 8, well, good luck.
>
> Option 2, only Java 11: The option with the most risks IMO. Java 11 is not
> even feature complete, and there a bunch of big projects that still may
> make it into 11 (think: Valhalla). There's no guarantee whether the C* code
> or any

Re: Resource Intensive and Upgrade tests (Dtest Cassandra)

[kurt greaves]

Moving thread to dev.

We have setup a dtest environment to run against Cassandra db version
> 3.11.1 and 3.0.5
> During the test run we see that tests under the category of
> Resource intensive and Upgrade tests
> Are skipped by default during the collection phase.

FWIW you can see these results for the tests (excluding upgrade tests) on
the ASF jenkins here: https://builds.apache.org/view/A-D/view/Cassandra/
(dtest-large jobs are resource-intensive)

Do you recommend, turning on these tests using the options above?

IMO those options suck. I've got a patch which provides some options that
will *only* run the resource-intensive and/or upgrade tests. The current
ones will run all non-resource-intensive plus resource-intensive (and same
for upgrade tests). Having said that this is probably fine if you don't
care how long the tests take and you have a big enough machine, but you're
looking at well over 20 hours of running tests and praying nothing goes
wrong in that time.
Patch is on CASSANDRA-14443
 if you're
interested.

> Are there any special case/scenarios when these tests need to be run?

We should be running them all the time/before any commit, but the testing
environment isn't really in a good position for that at the moment. We'll
be working on this for 4.0.

What should be the minimum system configuration for the resource intensive
> tests to run without any issues?

For resource-intensive you'll need at least 32gb of RAM and at least a 4
core machine to run them reliably. Upgrade tests are currently not
completely working and you're probably best avoiding them for now.


On 21 May 2018 at 05:11, Rajiv Dimri  wrote:

> Hi All,
>
>
>
> We have setup a dtest environment to run against Cassandra db version
> 3.11.1 and 3.0.5
>
> During the test run we see that tests under the category of
>
> Resource intensive and Upgrade tests
>
> Are skipped by default during the collection phase.
>
>
>
> And this is a huge number (1100+ out of the 1900+ tests written) more than
> 50%
>
>
>
> If we have to enable these tests then we have to use
> ‘--force-resource-intensive-tests’ and ‘--execute-upgrade-tests’ option.
>
>
>
> My questions are as below:
>
> Do you recommend, turning on these tests using the options above?
>
> Are there any special case/scenarios when these tests need to be run?
>
> What should be the minimum system configuration for the resource intensive
> tests to run without any issues?
>
>
>
> Regards,
>
> Rajiv
>
>
>

Re: In need of reviewers

[kurt greaves]

Oh I know, there are just tickets relevant to people I work with.

On Fri., 11 May 2018, 22:10 Josh McKenzie,  wrote:

> May be easier to just provide a link to the JQL, since there's quite a few
> more tickets than just what you've mentioned above:
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20cassandra%20AND%20resolution%20%3D%20unresolved%20AND%20status%20in%20(%22Patch%20Available%22%2C%20%22Awaiting%20Feedback%22)%20AND%20reviewer%20is%20EMPTY%20ORDER%20BY%20updated%20DESC
>
> or JQL:
> project = cassandra AND resolution = unresolved AND status in ("Patch
> Available", "Awaiting Feedback") AND reviewer is EMPTY ORDER BY updated
> DESC
>
> On Fri, May 11, 2018 at 6:43 AM, kurt greaves 
> wrote:
>
> > We've got a bunch of tickets that are either in need of review or just a
> > bit of feedback. Would be very grateful for any help here :).
> >
> > Bugs:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14365&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=dzgugNQ_-Xp_IhoZ_
> > yXOhK90rU63tOa5VWtbNCJXB1I&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14204&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > pITz0kdAo5jrNc3TDzeXdBJ14yDl4HHi2aowk1yA4NI&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14162&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=tiN83_
> > Q68L9l6coMUD2KdBIptE4F-xfi--cWEzadVLM&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14126&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > LBThldSjUG6z912bxdp2202kNA-ma-TbReAfPuaKOho&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14365&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=dzgugNQ_-Xp_IhoZ_
> > yXOhK90rU63tOa5VWtbNCJXB1I&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14099&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=RYPlsfyk_
> > m9AUKPjTqS99FUIrHUR15T9gYFNkAvHmT4&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14073&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=YJs46FqZ1P_M9xiLcl-
> > ynUduvhhfzCWoElcDXMioThY&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14063&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > vjJGUzOXIG0UXbVZWOb42nx6kBHhHhJG-p6eyw2JpLs&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14056&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=XItQA2Xa1jdbjLCZhYRhdm-
> > HgfoKgJwspC99fdQv_iM&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14054&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=FvTGFmMgSIp4wS-
> > Knt7lzAJONsk2DthnBLPPcgyMEfg&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> > apache.org_jira_browse_CASSANDRA-2D14013&d=DwIBaQ&c=
> > adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=
> > PxaWSXkgaViGX9BMjOCmJhZSQCCVyOq0rlqievPEnHY&s=
> > ngN2nSGlKr5iNVL4NVGzH84CJvQABXjBqmvtqjVeiy0&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> &

In need of reviewers

[kurt greaves]

We've got a bunch of tickets that are either in need of review or just a
bit of feedback. Would be very grateful for any help here :).

Bugs:
https://issues.apache.org/jira/browse/CASSANDRA-14365
https://issues.apache.org/jira/browse/CASSANDRA-14204
https://issues.apache.org/jira/browse/CASSANDRA-14162
https://issues.apache.org/jira/browse/CASSANDRA-14126
https://issues.apache.org/jira/browse/CASSANDRA-14365
https://issues.apache.org/jira/browse/CASSANDRA-14099
https://issues.apache.org/jira/browse/CASSANDRA-14073
https://issues.apache.org/jira/browse/CASSANDRA-14063
https://issues.apache.org/jira/browse/CASSANDRA-14056
https://issues.apache.org/jira/browse/CASSANDRA-14054
https://issues.apache.org/jira/browse/CASSANDRA-14013
https://issues.apache.org/jira/browse/CASSANDRA-13841
https://issues.apache.org/jira/browse/CASSANDRA-13698

Improvements:
https://issues.apache.org/jira/browse/CASSANDRA-14309
https://issues.apache.org/jira/browse/CASSANDRA-10789
https://issues.apache.org/jira/browse/CASSANDRA-14443
https://issues.apache.org/jira/browse/CASSANDRA-13010
https://issues.apache.org/jira/browse/CASSANDRA-11559
https://issues.apache.org/jira/browse/CASSANDRA-10789
https://issues.apache.org/jira/browse/CASSANDRA-10023
https://issues.apache.org/jira/browse/CASSANDRA-8460

Cheers,
Kurt

Re: Evolving the client protocol

[kurt greaves]

>
> 1. The protocol change is developed using the Cassandra process in a JIRA
> ticket, culminating in a patch to doc/native_protocol*.spec when consensus
> is achieved.

I don't think forking would be desirable (for anyone) so this seems the
most reasonable to me. For 1 and 2 it certainly makes sense but can't say I
know enough about sharding to comment on 3 - seems to me like it could be
locking in a design before anyone truly knows what sharding in C* looks
like. But hopefully I'm wrong and there are devs out there that have
already thought that through.

Do we have driver authors who wish to support both projects?

Surely, but I imagine it would be a minority.
​

Re: Proposing an Apache Cassandra Management process

[kurt greaves]

For anyone looking Dinesh made a ticket already: CASSANDRA-14395


On 18 April 2018 at 18:14, Vinay Chella  wrote:

> This is a good initiative. We have advocated for and run a sidecar for the
> past 5+ years, and we've learned and improved it a lot. We look forward to
> moving features from Priam (such as backup, HTTP -> JMX, etc) incrementally
> to this sidecar as they make sense.
>
>
> Thanks,
> Vinay Chella
>
> On Fri, Apr 13, 2018 at 7:01 AM, Eric Evans 
> wrote:
>
> > On Thu, Apr 12, 2018 at 4:41 PM, Dinesh Joshi
> >  wrote:
> > > Hey all -
> > > With the uptick in discussion around Cassandra operability and after
> > discussing potential solutions with various members of the community, we
> > would like to propose the addition of a management process/sub-project
> into
> > Apache Cassandra. The process would be responsible for common operational
> > tasks like bulk execution of nodetool commands, backup/restore, and
> health
> > checks, among others. We feel we have a proposal that will garner some
> > discussion and debate but is likely to reach consensus.
> > > While the community, in large part, agrees that these features should
> > exist “in the database”, there is debate on how they should be
> implemented.
> > Primarily, whether or not to use an external process or build on
> > CassandraDaemon. This is an important architectural decision but we feel
> > the most critical aspect is not where the code runs but that the operator
> > still interacts with the notion of a single database. Multi-process
> > databases are as old as Postgres and continue to be common in newer
> systems
> > like Druid. As such, we propose a separate management process for the
> > following reasons:
> > >
> > >- Resource isolation & Safety: Features in the management process
> > will not affect C*'s read/write path which is critical for stability. An
> > isolated process has several technical advantages including preventing
> use
> > of unnecessary dependencies in CassandraDaemon, separation of JVM
> resources
> > like thread pools and heap, and preventing bugs from adversely affecting
> > the main process. In particular, GC tuning can be done separately for the
> > two processes, hopefully helping to improve, or at least not adversely
> > affect, tail latencies of the main process.
> > >
> > >- Health Checks & Recovery: Currently users implement health checks
> > in their own sidecar process. Implementing them in the serving process
> does
> > not make sense because if the JVM running the CassandraDaemon goes south,
> > the healthchecks and potentially any recovery code may not be able to
> run.
> > Having a management process running in isolation opens up the possibility
> > to not only report the health of the C* process such as long GC pauses or
> > stuck JVM but also to recover from it. Having a list of basic health
> checks
> > that are tested with every C* release and officially supported will help
> > boost confidence in C* quality and make it easier to operate.
> > >
> > >- Reduced Risk: By having a separate Daemon we open the possibility
> > to contribute features that otherwise would not have been considered
> before
> > eg. a UI. A library that started many background threads and is operated
> > completely differently would likely be considered too risky for
> > CassandraDaemon but is a good candidate for the management process.
> >
> > Makes sense IMO.
> >
> > > What can go into the management process?
> > >- Features that are non-essential for serving reads & writes for eg.
> > Backup/Restore or Running Health Checks against the CassandraDaemon, etc.
> > >
> > >- Features that do not make the management process critical for
> > functioning of the serving process. In other words, if someone does not
> > wish to use this management process, they are free to disable it.
> > >
> > > We would like to initially build minimal set of features such as health
> > checks and bulk commands into the first iteration of the management
> > process. We would use the same software stack that is used to build the
> > current CassandraDaemon binary. This would be critical for sharing code
> > between CassandraDaemon & management processes. The code should live
> > in-tree to make this easy.
> > > With regards to more in-depth features like repair scheduling and
> > discussions around compaction in or out of CassandraDaemon, while the
> > management process may be a suitable host, it is not our goal to decide
> > that at this time. The management process could be used in these cases,
> as
> > they meet the criteria above, but other technical/architectural reasons
> may
> > exists for why it should not be.
> > > We are looking forward to your comments on our proposal,
> >
> > Sounds good to me.
> >
> > Personally, I'm a little less interested in things like
> > health/availability checks and metrics collection, because there are
> > already tools to solve

Re: Quantifying Virtual Node Impact on Cassandra Availability

[kurt greaves]

Great write up. Glad someone finally did the math for us. I don't think
this will come as a surprise for many of the developers. Availability is
only one issue raised by vnodes. Load distribution and performance are also
pretty big concerns.

I'm always a proponent for fixing vnodes, and removing them as a default
until we do. Happy to help on this and we have ideas in mind that at some
point I'll create tickets for...

On Tue., 17 Apr. 2018, 06:16 Joseph Lynch,  wrote:

> If the blob link on github doesn't work for the pdf (looks like mobile
> might not like it), try:
>
>
> https://github.com/jolynch/python_performance_toolkit/raw/master/notebooks/cassandra_availability/whitepaper/cassandra-availability-virtual.pdf
>
> -Joey
> <
> https://github.com/jolynch/python_performance_toolkit/raw/master/notebooks/cassandra_availability/whitepaper/cassandra-availability-virtual.pdf
> >
>
> On Mon, Apr 16, 2018 at 1:14 PM, Joseph Lynch 
> wrote:
>
> > Josh Snyder and I have been working on evaluating virtual nodes for large
> > scale deployments and while it seems like there is a lot of anecdotal
> > support for reducing the vnode count [1], we couldn't find any concrete
> > math on the topic, so we had some fun and took a whack at quantifying how
> > different choices of num_tokens impact a Cassandra cluster.
> >
> > According to the model we developed [2] it seems that at small cluster
> > sizes there isn't much of a negative impact on availability, but when
> > clusters scale up to hundreds of hosts, vnodes have a major impact on
> > availability. In particular, the probability of outage during short
> > failures (e.g. process restarts or failures) or permanent failure (e.g.
> > disk or machine failure) appears to be orders of magnitude higher for
> large
> > clusters.
> >
> > The model attempts to explain why we may care about this and advances a
> > few existing/new ideas for how to fix the scalability problems that
> vnodes
> > fix without the availability (and consistency—due to the effects on
> repair)
> > problems high num_tokens create. We would of course be very interested in
> > any feedback. The model source code is on github [3], PRs are welcome or
> > feel free to play around with the jupyter notebook to match your
> > environment and see what the graphs look like. I didn't attach the pdf
> here
> > because it's too large apparently (lots of pretty graphs).
> >
> > I know that users can always just pick whichever number they prefer, but
> I
> > think the current default was chosen when token placement was random,
> and I
> > wonder whether it's still the right default.
> >
> > Thank you,
> > -Joey Lynch
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRA-13701
> > [2] https://github.com/jolynch/python_performance_toolkit/
> > raw/master/notebooks/cassandra_availability/whitepaper/cassandra-
> > availability-virtual.pdf
> >
> > <
> https://github.com/jolynch/python_performance_toolkit/blob/master/notebooks/cassandra_availability/whitepaper/cassandra-availability-virtual.pdf
> >
> > [3] https://github.com/jolynch/python_performance_toolkit/tree/m
> > aster/notebooks/cassandra_availability
> >
>

Re: Roadmap for 4.0

[kurt greaves]

Seems to me we should put September first to an official vote so we can
finish up with this mess.

On 13 April 2018 at 02:27, Rahul Singh  wrote:

> I can commit some resources on my team - especially as we onboard some of
> our summer apprentices.
>
> I have some proprietary stress tools geared for Cassandra read / writes
> that are a little better and creates a little more realistic data than
> Cassandra stress.
>
> --
> Rahul Singh
> rahul.si...@anant.us
>
> Anant Corporation
>
> On Apr 12, 2018, 3:41 PM -0400, Nate McCall , wrote:
> > Ok. So who's willing to test 4.0 on June 2nd? Let's start a sign up.
> >
> > We (tlp) will put some resources on this via going through some canned
> > scenarios we have internally. We aren't in a position to test data
> validity
> > (yet) but we can do a lot around cluster behavior.
> >
> > Who else has specific stuff they are willing to do? Even if it's just
> > tee'ing prod traffic, that would be hugely valuable.
> >
> > On Fri, Apr 13, 2018, 6:15 AM Jeff Jirsa  wrote:
> >
> > > On Thu, Apr 12, 2018 at 9:41 AM, Jonathan Haddad  > > wrote:
> > >
> > > > It sounds to me (please correct me if I'm wrong) like Jeff is arguing
> > > that
> > > > releasing 4.0 in 2 months isn't worth the effort of evaluating it,
> > > because
> > > > it's a big task and there's not enough stuff in 4.0 to make it
> > > worthwhile.
> > > >
> > > >
> > > More like "not enough stuff in 4.0 to make it worthwhile for the
> people I
> > > personally know to be willing and able to find the weird bugs".
> > >
> > >
> > > > If that is the case, I'm not quite sure how increasing the surface
> area
> > > of
> > > > changed code which needs to be vetted is going to make the process
> any
> > > > easier.
> > >
> > >
> > > It changes the interest level of at least some of the people able to
> > > properly test it from "not willing" to "willing".
> > >
> > > Totally possible that there exist people who are willing and able to
> find
> > > and fix those bugs, who just haven't committed to it in this thread.
> That's
> > > probably why Sankalp keeps asking who's actually willing to do the
> testing
> > > on June 2 - if nobody's going to commit to doing real testing on June
> 2,
> > > all we're doing is adding inconvenience to those of us who'd be
> willing to
> > > do it later in the year.
> > >
>

Re: Roadmap for 4.0

[kurt greaves]

September also works for Instaclustr.

On Fri., 13 Apr. 2018, 08:27 Jon Haddad,  wrote:

> Sept works for me too.  I’ll be involved in the validation process before
> the cutoff date.
>
>
> > On Apr 12, 2018, at 3:17 PM, Carlos Rolo  wrote:
> >
> > I will commit time to test (not a full validation, but at least go
> through
> > operations) regardless of the date. Both seems fine to me.
> >
> > Regards,
> >
> > Carlos Juzarte Rolo
> > Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
> >
> > Pythian - Love your data
> >
> > rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
> > *linkedin.com/in/carlosjuzarterolo
> > *
> > Mobile: +351 918 918 100
> > www.pythian.com
> >
> > On Thu, Apr 12, 2018 at 11:00 PM, Joseph Lynch 
> > wrote:
> >
> >> The Netflix team prefers September as well. We don't have time before
> that
> >> to do a full certification (e2e and performance testing), but can
> probably
> >> work it into end of Q3 / start of Q4.
> >>
> >> I personally hope that the extra time gives us as a community a chance
> to
> >> come up with a compelling user story for why users would want to
> upgrade. I
> >> don't feel we have one right now.
> >>
> >> -Joey
> >>
> >>
> >> On Thu, Apr 12, 2018 at 2:51 PM, Ariel Weisberg 
> wrote:
> >>
> >>> Hi,
> >>>
> >>> +1 to September 1st. I know I will have much better availability then.
> >>>
> >>> Ariel
> >>> On Thu, Apr 12, 2018, at 5:15 PM, Sankalp Kohli wrote:
>  +1 with Sept 1st as I am seeing willingness for people to test it
> after
> >>> it
> 
> > On Apr 12, 2018, at 13:59, Ben Bromhead  wrote:
> >
> > While I would prefer earlier, if Sept 1 gets better buy-in and we can
> >>> have
> > broader commitment to testing. I'm super happy with that. As Nate
> >> said,
> > having a solid line to work towards is going to help massively.
> >
> > On Thu, Apr 12, 2018 at 4:07 PM Nate McCall 
> >>> wrote:
> >
> >>> If we push it to Sept 1 freeze, I'll personally spend a lot of time
> >> testing.
> >>>
> >>> What can I do to help convince the Jun1 folks that Sept1 is
> >>> acceptable?
> >>
> >> I can come around to that. At this point, I really just want us to
> >> have a date we can start talking to/planning around.
> >>
> >> 
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >> --
> > Ben Bromhead
> > CTO | Instaclustr 
> > +1 650 284 9692
> > Reliability at Scale
> > Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
> 
>  -
>  To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>  For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>>
> >>
> >
> > --
> >
> >
> > --
> >
> >
> >
> >
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[kurt greaves]

>
> I also don't see a place for minor releases as they exist today. It seems
> like they are almost all the overhead of a major release with unnecessary
> restrictions on what is possible.

Yeah this, I've never heard of anything that we don't do in "minors", and
it seems to me that everyone treats the minors as majors (hell, we're doing
that here re 4.1). It seems to me that we should just have .
versioning based on the way we treat minors.

Who can sign up for testing the alpha1. I know Ben has shown interest.

We can certainly do it, and we plan to do a lot of testing of 4.0, but I
doubt we'll have anything ready to properly test it by June 1st.
Another thing worth noting is dtests currently only partially run on
CircleCI and it seems to me that Apple is the only one that actually runs
them even there. It's going to take us a while to get the testing
environment in a state that covers all bases post-freeze, and it's a good
idea to have some commitment to doing that before we go ahead and freeze. I
agree there's little point freezing if we can't even test the system
properly.

Not to mention the total lack of any kind of standard performance testing
environment...
​

Re: Roadmap for 4.0

[kurt greaves]

Huh, I was writing my response for quite a while and getting distracted so
didn't see this, but yeah if I had a vote, this would obviously have it.

On 11 April 2018 at 03:03, Jeff Jirsa  wrote:

>
>
> --
> Jeff Jirsa
>
>
> On Apr 10, 2018, at 5:24 PM, Josh McKenzie  wrote:
>
> >>
> >> 50'ish days is too short to draw a line in the sand,
> >> especially as people balance work obligations with Cassandra feature
> >> development.
> >
> > What's a reasonable alternative / compromise for this? And what
> > non-disruptive-but-still-large patches are in flight that we would want
> to
> > delay the line in the sand for?
>
> I don’t care about non disruptive patches to be really honest. Nobody’s
> running trunk now, so it doesn’t matter to me if the patch landed 6 months
> ago or Jun 29, unless you can show me one person who’s ran a nontrivial
> multi-dc test cluster under real load that included correctness validation.
> Short of that, it’s untested, and the duration a patch has been in an
> untested repo is entirely irrelevant.
>
> If there’s really someone already testing trunk in a meaningful way (real
> workloads, and verifying correctness), and that person is really able to
> find and fix bugs, then tell me who it is and I’ll change my opinion (and
> I’m not even talking about thousand node clusters, just someone who’s
> actually using real data, like something upgraded from 2.1/3.0, and is
> checking to prove it matches expectations).
>
> Otherwise, when the time comes for real users to plan real upgrades to a
> hypothetical 4.1, they’ll have to do two sets of real, expensive, annoying
> testing - one for the stuff in 4.0 (chunk cache, file format changes,
> internode changes, etc), and a second for 4.0-4.1 changes for the invasive
> stuff I care about and you don’t want to wait for.
>
> I’d rather see us get all this stuff in and then spend real time testing
> and fixing in a 4-6 month alpha/beta phase (where real users can help,
> because its one real dedicated validation phase) than push this into two
> (probably inadequately tested) releases.
>
> But that’s just my opinion, and I’ll support it with my one vote, and I
> may get outvoted, but that’s what I’d rather see happen.
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[kurt greaves]

>
> In thinking about this, what is stopping us from branching 4.0 a lot
> sooner? Like now-ish? This will let folks start hacking on trunk with
> new stuff, and things we've gotten close on can still go in 4.0

Agree with Jeff here that this is not necessary. The branch point should be
the feature freeze point.

What's a reasonable alternative / compromise for this? And what
> non-disruptive-but-still-large patches are in flight that we would want to
> delay the line in the sand for?

Seeing as we are now back to where we started (because this was inevitable)
I'll start listing issues again. If we got this worked up over the release
back when I started this thread, June 1 probably would have been enough
warning for everyone to be happy. I started it the way I did so that we
avoided all the needless debate, but we went there anywhere.

Let's also not pretend that "we can drop 4.1 anytime", it will be exactly
as controversial as 4.0 and require as much time, discussion, validation,
and testing to get it out the window. I'd be expecting *at the earliest* that
4.1 landed in Q4 2019.
I also find the whole "only drop large features at the start of a branch"
curious. It's not like trunk has been verified in the meantime (ccm doesn't
count), and anything new that's been introduced should have tests
associated with it. As far as I'm aware, trunk is incredibly broken if you
consider the current testing environment, and we'll have to fix that prior
to releasing anything real. The point of a feature freeze is that you then
spend a lot of time validating, testing, and fixing after the freeze but
before you do the release. You're not in some great rush to release after
the freeze. The point of a feature freeze isn't to have all your features
in months/years before the freeze, it's just meant to stop you from
implementing more features while you fix existing issues.

I do understand the reluctance though, based on the history of the project.
Tick-tock consisted of almost no follow up testing and validation, and 3.0
wasn't much better - but I think it's still reasonable to give a proper
validation/testing strategy a chance, testing in our own environments and
actually making all the utests+dtests work.

Anyway, back to things which I think should land in 4.0 but have been in
the works for a long time:

RangeAwareCompaction - CASSANDRA-10540

We've seen *many* use cases that could benefit from this, and it's also
good for fixing some of the serious problems with vnodes, repairs, and
streaming.
Despite the lack of action I don't think it's far from being finished, but
with a June 1 deadline it's probably unlikely.

Large partition improvements - CASSANDRA-9754

I know this is a much awaited feature for a fair few users, but mostly it
will make operator life a lot easier. I look forward to the day where a
user can't kill their cluster by reading a few large partitions.

Idempotent DDL - CASSANDRA-13426

This will be handy but is really more important for CASSANDRA-10699
 (which probably
won't make it into 4.0).

More importantly, as I already noted, there's a large backlog of patches to
review that we simply won't get to before June 1st (this includes some
fairly large patches as well, like the pluggable storage refactors).


Side note June 1st puts us almost 2 years from our last feature release
(3.10, Oct 2016) - not "almost a year"/a year/around a year.

On 10 April 2018 at 22:34, Jeff Jirsa  wrote:

> Seriously, what's the rush to branch? Do we all love merging so much we
> want to do a few more times just for the sake of merging? If nothing
> diverges, there's nothing gained from the branch, and if it did diverge, we
> add work for no real gain.
>
> Beyond that, I still don't like June 1. Validating releases is hard. It
> sounds easy to drop a 4.1 and ask people to validate again, but it's a hell
> of a lot harder than it sounds. I'm not saying I'm a hard -1, but I really
> think it's too soon. 50'ish days is too short to draw a line in the sand,
> especially as people balance work obligations with Cassandra feature
> development.
>
>
>
>
> On Tue, Apr 10, 2018 at 3:18 PM, Nate McCall  wrote:
>
> > A lot of good points and everyone's input is really appreciated.
> >
> > So it sounds like we are building consensus towards June 1 for 4.0
> > branch point/feature freeze and the goal is stability. (No one has
> > come with a hard NO anyway).
> >
> > I want to reiterate Sylvain's point that we can do whatever we want in
> > terms of dropping a new feature 4.1/5.0 (or whatev.) whenever we want.
> >
> > In thinking about this, what is stopping us from branching 4.0 a lot
> > sooner? Like now-ish? This will let folks start hacking on trunk with
> > new stuff, and things we've gotten close on can still go in 4.0

Re: [Discuss] patch review virtual hackathon

[kurt greaves]

To add to the above, hackathons would make sense in the lead up to the
feature freeze IMO to get things through the door, but not necessarily
afterwards (debatable). And they can be flexible too; I suspect once people
start on reviewing something they'll be much more likely to see it through
to completion. It's unlikely we'd end up with many commits after 72 hours,
purely because any feedback may take longer than that to be actioned, but
that doesn't mean our time is wasted.

On 6 April 2018 at 05:09, kurt greaves  wrote:

> I like the idea and we would be willing to take part (no committers here
> but I'm sure we can help).
>
> I think it better to pick some JIRAs per 2-3 weeks and have people review
>> them. In my experience, it is hard to synchronize all people across
>> companies during one 72 hour slot.
>
>
> It is hard, but there are benefits. Mostly having many people all actively
> reviewing stuff at the "same" time so communication will be much
> easier/more natural. I think a plan like ^ should be done as well as a
> hackathon, but should be a continuous thing that we always do. E.g, each
> week someone is responsible for picking a few tickets that should be
> reviewed and we hunt for volunteers to start the review work.
>
>
>>
>
>
>>
>> On Thu, Apr 5, 2018 at 9:48 PM, Nate McCall  wrote:
>>
>> > Per Kurt's point in our release thread, we have a lot to do here.
>> >
>> > What do folks feel about setting aside a 72hr period at some point soon
>> > where we get some allotment from our employers to spend a window or two
>> of
>> > time therein reviewing patches?
>> >
>> > I have seen a couple of other ASF communities do this type of thing
>> > effectively in recent months. If we pull this off I think it could set
>> an
>> > excellent precedent for swarming on other things in the future.
>> >
>>
>
>

Re: [Discuss] patch review virtual hackathon

[kurt greaves]

I like the idea and we would be willing to take part (no committers here
but I'm sure we can help).

I think it better to pick some JIRAs per 2-3 weeks and have people review
> them. In my experience, it is hard to synchronize all people across
> companies during one 72 hour slot.


It is hard, but there are benefits. Mostly having many people all actively
reviewing stuff at the "same" time so communication will be much
easier/more natural. I think a plan like ^ should be done as well as a
hackathon, but should be a continuous thing that we always do. E.g, each
week someone is responsible for picking a few tickets that should be
reviewed and we hunt for volunteers to start the review work.


>


>
> On Thu, Apr 5, 2018 at 9:48 PM, Nate McCall  wrote:
>
> > Per Kurt's point in our release thread, we have a lot to do here.
> >
> > What do folks feel about setting aside a 72hr period at some point soon
> > where we get some allotment from our employers to spend a window or two
> of
> > time therein reviewing patches?
> >
> > I have seen a couple of other ASF communities do this type of thing
> > effectively in recent months. If we pull this off I think it could set an
> > excellent precedent for swarming on other things in the future.
> >
>

Re: Roadmap for 4.0

[kurt greaves]

>
> Lay our cards on the table about what we want included in 4.0 and work to
> get those in

Are you saying we're back to where we started?  😉

For those wanting to delay, are we just dancing around inclusion of
> some pet features? This is fine, I just think we need to communicate
> what we are after if so.


Mostly. There are numerous large tickets that have been being worked on *for
a long time*, and have been rumoured to be nearing completion for some
time, which would be beneficial for everyone. They aren't my pet tickets
but I'd sure like to see them finally land (see: Apple tickets).

But there's more to it than that. We've also got tickets we'd like to get
committed and It's an incredibly slow process to get anyone to review your
tickets (and commit) if you're not in the club, so to speak. There's 148
tickets that are Patch available ATM, 89 of which have no reviewer. I think
it's highly unlikely a lot of these will get committed before June 1st, and
I don't think that's fair on a lot of the people who have been trying to
contribute their patches for months, potentially years in some cases, but
have been stuck waiting on feedback/reviewers/committers.

Sure it's not the end of the world, but I know from experience that it's
damn discouraging. Even missing a minor release for a bug fix because of
this is annoying as hell.

I do want to remind everyone though that each new feature is at odds
> with our stability goals for 4.0.

With all the refactoring that's already gone into 4.0 and our current lack
of testing I think we're fighting an uphill battle here anyway. Adding a
few more metres is the least of our worries IMO. The
alpha/verification/testing period is already going to be a very long one.


On 6 April 2018 at 04:01, Nate McCall  wrote:

> >>
> >> So long as non-user-visible improvements, including big ones, can still
> go
> >> in 4.0 at that stage, I’m all for it.
> >
> >
> > My understanding is that after June 1st the 4.0 branch would be created
> and
> > would be bugfix only. It's not really a feature freeze if you allow
> > improvements after that, which is why they'd instead go to trunk.
> >
> > I'm also on the "too soon" train so pushing it back to August or so is
> > desirable to me.
> >
>
> For those wanting to delay, are we just dancing around inclusion of
> some pet features? This is fine, I just think we need to communicate
> what we are after if so.
>
> We can do two things:
> 1. Lay our cards on the table about what we want included in 4.0 and
> work to get those in
> 2. Agree to keep June 1 follow up a lot quicker with a 4.1
>
> I do want to remind everyone though that each new feature is at odds
> with our stability goals for 4.0.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Repair scheduling tools

[kurt greaves]

Vnodes is related and because we made it a default lots of people are using
it. Repairing a cluster with vnodes is a catastrophe (even a small one is
often problematic), but we have to deal with it if we build in repair
scheduling.

Repair scheduling is very important and we should definitely include it
with C* (sidecar long term makes most sense to me but only if we looked at
moving other background ops to the sidecar), but I'm positive it's not
going to work well with vnodes in their current state. Having said that, it
should still support scheduling repairs on vnode clusters, but the
vnode+repair problem should be fixed separately (and probably with more
attention than we've given it) because it's a major issue.

FWIW I know of 256 vnode clusters with > 100 nodes, yet I'd be surprised if
any of them are currently successfully repairing.

On 6 April 2018 at 03:03, Nate McCall  wrote:

> I think a take away here is that we can't assume a level of operation
> maturity will coincide automatically with scale. To make our core
> features robust, we have to account for less-experienced users.
>
> A lot of folks on this thread have *really* strong ops and OpsViz
> stories. Let's not forget that most of our users don't.
> ((Un)fortunately, as a consulting firm, we tend to see the worst of
> this).
>
> On Fri, Apr 6, 2018 at 2:52 PM, Jonathan Haddad  wrote:
> > Off the top of my head I can remember clusters with 600 or 700 nodes with
> > 256 tokens.
> >
> > Not the best situation, but it’s real. 256 has been the default for
> better
> > or worse.
> >
> > On Thu, Apr 5, 2018 at 7:41 PM Joseph Lynch 
> wrote:
> >
> >> >
> >> > We see this in larger clusters regularly. Usually folks have just
> >> > 'grown into it' because it was the default.
> >> >
> >>
> >> I could understand a few dozen nodes with 256 vnodes, but hundreds is
> >> surprising. I have a whitepaper draft lying around showing how vnodes
> >> decrease availability in large clusters by orders of magnitude, I'll
> polish
> >> it up and send it out to the list when I get a second.
> >>
> >> In the meantime, sorry for de-railing a conversation about repair
> >> scheduling to talk about vnodes, let's chat about that in a different
> >> thread :-)
> >>
> >> -Joey
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[kurt greaves]

>
> So long as non-user-visible improvements, including big ones, can still go
> in 4.0 at that stage, I’m all for it.


My understanding is that after June 1st the 4.0 branch would be created and
would be bugfix only. It's not really a feature freeze if you allow
improvements after that, which is why they'd instead go to trunk.

I'm also on the "too soon" train so pushing it back to August or so is
desirable to me.


On 5 April 2018 at 21:06, Aleksey Yeshchenko  wrote:

> So long as non-user-visible improvements, including big ones, can still go
> in 4.0 at that stage, I’m all for it.
>
> —
> AY
>
> On 5 April 2018 at 21:14:03, Nate McCall (zznat...@gmail.com) wrote:
>
> >>> My understanding, from Nate's summary, was June 1 is the freeze date
> for
> >>> features. I expect we would go for at least 4 months (if not longer)
> >>> testing, fixing bugs, early dogfooding, and so on. I also equated June
> 1
> >>> with the data which we would create a 'cassandra-4.0' branch, and thus
> the
> >>> merge order becomes: 3.0->3,11->4.0->trunk.
>
> This^ (apologies - 'freeze for alpha' was a bit open for interpretation :)
>
> The idea of making this point in time the 4.0 branch date and merge
> order switch is a good one.
>
> Can we move our gelling consensus here towards this goal?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[kurt greaves]

>
> Earlier than I’d have personally picked, but I’m +1 too

This but +1.

On 5 April 2018 at 03:04, J. D. Jordan  wrote:

> +1
>
> > On Apr 4, 2018, at 5:06 PM, Nate McCall  wrote:
> >
> > Top-posting as I think this summary is on point - thanks, Scott! (And
> > great to have you back, btw).
> >
> > It feels to me like we are coalescing on two points:
> > 1. June 1 as a freeze for alpha
> > 2. "Stable" is the new "Exciting" (and the testing and dogfooding
> > implied by such before a GA)
> >
> > How do folks feel about the above points?
> >
> >
> >> Re-raising a point made earlier in the thread by Jeff and affirmed by
> Josh:
> >>
> >> –––
> >> Jeff:
>  A hard date for a feature freeze makes sense, a hard date for a
> release
>  does not.
> >>
> >> Josh:
> >>> Strongly agree. We should also collectively define what "Done" looks
> like
> >>> post freeze so we don't end up in bike-shedding hell like we have in
> the
> >>> past.
> >> –––
> >>
> >> Another way of saying this: ensuring that the 4.0 release is of high
> quality is more important than cutting the release on a specific date.
> >>
> >> If we adopt Sylvain's suggestion of freezing features on a "feature
> complete" date (modulo a "definition of done" as Josh suggested), that will
> help us align toward the polish, performance work, and dog-fooding needed
> to feel great about shipping 4.0. It's a good time to start thinking about
> the approaches to testing, profiling, and dog-fooding various contributors
> will want to take on before release.
> >>
> >> I love how Ben put it:
> >>
> >>> An "exciting" 4.0 release to me is one that is stable and usable
> >>> with no perf regressions on day 1 and includes some of the big
> >>> internal changes mentioned previously.
> >>>
> >>> This will set the community up well for some awesome and exciting
> >>> stuff that will still be in the pipeline if it doesn't make it to 4.0.
> >>
> >> That sounds great to me, too.
> >>
> >> – Scott
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[kurt greaves]

>
> I'm also a bit sad that we seem to be getting back to our old demons of
> trying
> to shove as much as we possibly can in the next major as if having a
> feature
> miss it means it will never happen.

That wasn't the intention of this thread, but that's the response I got.
Thought I made it pretty clear that this was about compiling a list of
things that people are currently working on and can commit to getting
finished soon (which should be a relatively small list considering the
limited number of full time contributors).

Of course, we should probably (re-(re-(re-)))start a discussion on release
> "strategy" in parallel because it doesn't seem we have one right now, but
> that's imo a discussion we should keep separate.

This discussion was always about the release strategy. There is no
separation between the release strategy for 4.0 and the release strategy
for the project, they are the same thing and what is intended to be
discussed here. I don't think it's possible to have a separate discussion
on these two things as the release strategy has a pretty big influence on
how 4.0 is released.

I'm all for a feature freeze and KISS, but I feel that this really needs a
bit more thought before we just jump in and set another precedent for
future releases. IMO the Cassandra project has had a seriously bad track
record of releasing major versions in the past, and we should probably work
at resolving that properly, rather than just continuing the current "let's
just try something new every time without really thinking about it".

Some points:

   1.  This strategy means that we don't care about what improvements
   actually make it into any given major version. This means that we will have
   major releases with nothing/very little desirable for users, and thus
   little reason to upgrade other than to stay on a supported version (from
   experience this isn't terribly important to users of a database). I think
   this inevitably leads to supporting more versions than necessary, and in
   general a pretty poor experience for users as we spend more time fighting
   bugs in production rather than before we do a release (purely because of
   increased frequency of releases).
   2. We'll always be driven by feature deadlines, which for the most part
   is fine, as long as we handle verification/quality assurance/release
   candidates appropriately. The main problem here though is that we don't
   really know what's going to be in a certain release until we hit the
   freeze, and what's in it may not really make sense at that point in time.
   3. We'll pump out major versions fairly regularly and end up with even
   more users that are on EOL versions with complex upgrade paths to get to a
   supported version or a version with a feature they need (think all those
   people still out there on 1.2).
   4. This strategy has the positive effect of allowing developers to see
   their changes in production faster, but OTOH if no one really uses the new
   versions this doesn't really happen anyway.

I'd also note that if people hadn't noticed, users tend to be pretty
reluctant to upgrade their databases (hello everyone still running 2.1).
This tends to be the nature of a database to some extent (if it works on
version x, why upgrade to y?). IMO it would make more sense to support less
versions but for a longer period of time. I'm sure most users would
appreciate 2 years of bug fixes for only 2 branches with a new major
approximately every 2 years. Databases don't move that fast, there's not
much desirable in a feature release every year for users.

sidenote: 3.10 was released in January 2017, and while the changes list for
4.0 is getting quite large there's not much there that's going to win over
users. It's mostly refactorings and improvements that affect developers
more so than users. I'm really interested in why people believe there is an
actual benefit in pumping out feature releases on a yearly basis. Who
exactly does that benefit? From what I know, the majority of "major" users
are still backporting stuff they want to 2.1, so why rush releasing more
versions?

Regardless of whatever plan we do end up following it would still be
> valuable to have a list of tickets for 4.0 which is the overall goal of
> this email - so let's not get too worked up on the details just yet (save
> that for after I summarise/follow up).
>
 lol. dreaming.

On 4 April 2018 at 10:38, Aleksey Yeshchenko  wrote:

> 3.0 will be the most popular release for probably at least another couple
> years - I see no good reason to cap its support window. We aren’t Oracle.
>
> —
> AY
>
> On 3 April 2018 at 22:29:29, Michael Shuler (mich...@pbandjelly.org)
> wrote:
>
> Apache Cassandra 3.0 is supported until 6 months after 4.0 release (date
> TBD).
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[kurt greaves]

I say we give it a few weeks and let adoptopenjdk make an official
statement before we make any kind of plan. We're not in a great rush.

On Mon., 26 Mar. 2018, 05:49 Carl Mueller, 
wrote:

> And here we go!
>
> from here:
> https://www.reddit.com/r/java/comments/86ce66/java_long_term_support/
>
> AdoptOpenJDK (https://www.adoptopenjdk.net) is a new non-profit community
> build farm that will be providing LTS support (that means we will keep Java
> 11, 17 etc up to date with security and stability patches) for OpenJDK
> binaries for all platforms matching Oracle’s LTS roadmap (but for more
> platforms and OpenJDK derivatives.
>
> We’ve just produced our first TCK passing builds and hope to have
> everything ironed out by Java 11.
>
> Most of the major OpenJDK vendors are participating in this (Oracle, IBM,
> Red Hat et al). The goal is for this to be shared common infrastructure for
> all.
>
> Commercial vendors may choose to provide commercial support over and above
> this 🙂
>
> Disclaimer: I'm the chief Cat herder at the AdoptOpenJDK build farm.
>
> More tech details: https://ci.adoptopenjdk.net and
> https://www.github.com/AdoptOpenJDK (start with the TSC repo)
>
>
> On Sat, Mar 24, 2018 at 5:14 AM, Robert Stupp  wrote:
>
> > The current (old-ish) status of #9608 requires to be built on Java 8, but
> > runs on both Java 8 and Java 9 (and should on 10). It comes with the
> > inconvenience that there are two (new) jvm.options files for the
> specifics
> > of 8 and 9. This was done to make it a no-brainer to start in on either 8
> > or 9/10 - think: CI. It's not trivial but otoh also not complex and some
> > changes in #9608 are already superfluous (#14173). Multi-release jars
> > aren't required.
> >
> > At the moment I do not see any _major_ blocker that prevents a hybrid,
> but
> > having support for both 8 + 11 in 4.0 is not an appealing option IMO. It
> > was ok at the time I built the initial patch, but I no longer think it's
> > the "right" way.
> >
> > I think, option 1) here is the right way, considering the release date
> > (maybe end of this year?) and the lifetime of 4.0. It's certainly a risk
> at
> > this point, but maybe less than the hassle to support 4.0 w/ Java 8 in
> the
> > next years. However, if I understand option 3) correctly and 4.0 and 4.1
> > only differ by the required Java version, I agree that this is also a
> good
> > option.
> >
> > TL;DR I'd vote for option 1, but would also be fine with 3, not ok with 2
> >
> >
> > On 03/23/2018 10:03 AM, Stefan Podkowinski wrote:
> >
> >> I think it's pretty safe to assume that Java 8 will stay around much
> >> longer than by the end of the year, after Oracle dropped their official
> >> maintainer role. I also think that we don't have to worry that much how
> >> exactly Java 8 is going to be supported. It's a mature enough version
> >> that I wouldn't expect significant incompatibilities between back-ports
> >> or forks. In the best case, someone will even step up taking the
> >> official maintainer role as part of the OpenJDK project. But I'm pretty
> >> sure we'll manage to keeping up supporting Java 8 for Cassandra
> >> throughout the next years, if we decide to do so.
> >>
> >> At the beginning, we've discussed the elastic search plans of supporting
> >> the newest Java release and the latest LTS release at the same time.
> >> Maybe it's a good time to get back thinking about this idea again and
> >> ask us, do we really want to support the latest Java release, even if
> >> it's a non-LTS release? Given the likely overlap of new major Java
> >> releases and continued support for already released Cassandra branches,
> >> I'd expect this to become a strain for developers and possible source of
> >> confusion for users. Do we as developers or any users really care that
> >> much about non-LTS releases in general, that we want to commit us to
> that?
> >>
> >> Let's assume we're only going to support Java LTS releases for now. How
> >> exactly would we want to go on from here? Keep in mind that Java 11 LTS
> >> is already scheduled for September. Let's take a look at some LTS only
> >> options:
> >>
> >> 1) Release 4.0 for Java 11 exclusively (3.11 for Java 8)
> >>
> >> Start upgrading CI to initial Java 11 release candidate, merge Robert's
> >> Java 9 patch and start fixing all incompatibility issues. Release 4.0 at
> >> some point after Java 11. This is probably the most risky option, as we
> >> can't see yet how the Java 11 adoption rate will turn out to be. In the
> >> worst case, Java 8 will still dominate for times to come and depending
> >> on Java 11 as hard requirement may hurt 4.0 adoption.
> >>
> >> 2) Release 4.0 for Java 8 + 11
> >>
> >> Support both LTS versions for 4.0. I'd expect this to be non-trivial,
> >> but maybe Robert can share some first hand experience what would have to
> >> be done to make this possible. As described in the elastic search blog,
> >> what they plan to do is to create multi-release jars for code c

Re: Empty partition keys allowed in MV, but not in normal table

[kurt greaves]

Yeah definitely a problem there. Can you create a JIRA for it?

On Sat., 24 Mar. 2018, 11:00 Duarte Nunes,  wrote:

> Hi,
>
> Given the following table:
>
> cqlsh> create keyspace ks WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> create table t (p text, c int, v text, primary key (p));
> cqlsh> use ks;
>
> The following fails:
>
> cqlsh:ks> insert into t (p, c, v) values ('', 2, '');
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Key
> may not be empty"
>
> However, MVs don't appear to have this restriction:
>
> create materialized view mv as select * from t where v is not null and p
> is not null and c is not null primary key (v, p);
> insert into t (p, c, v) values ('a', 2, '');
> select * from mv;
>
>   v | p | c
> ---+---+---
> | a | 2
>
> I guess this is because an empty value can't be distinguished from null at
> the protocol level, but this distinction can
> be made internally.
>
> I think the behavior should be made consistent, if nothing else because
> querying the MV for the empty key is impossible:
>
> cqlsh:ks> select * from mv where v = '';
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Key
> may not be empty"
>
> Thoughts?
>
> Thanks,
> Duarte
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[kurt greaves]

I think Michael is right. It would be impossible to make everyone follow
such a fast release scheme, and supporting it will be pressured onto the
various distributions, M$ and Apple.
On the other hand https://adoptopenjdk.net has already done a lot of the
work and it's already rumoured they may take up backporting of security/bug
fixes. I'd fully expect a lot of users to collaborate around this (or
similar), and there's no reason we couldn't do our part to contribute.

On Fri., 23 Mar. 2018, 09:37 Gerald Henriksen,  wrote:

> On Thu, 22 Mar 2018 16:04:16 -0500, you wrote:
>
> >Is OpenJDK really not addressing this at all? Is that because OpenJDK is
> >beholden to Oracle somehow?
>
> I suspect it is more a case of OpenJDK (as in the entitites other than
> Oracle that are members) haven't historically been involved in the
> providing of JRE/JDK to the public.  Oracle (previously Sun) handled
> that entirely and so there is no group outside of Oracle prepared (at
> least at this point in time) to do anything.
>
> >There is no looming deadline on this, is there?
>
> Less than a year.  Java 8 is currently scheduled to get dropped (as in
> no more updates) in January 2019 (the actual plan is 4 months after
> Java 11, so if 11 is late then that date will get changed).
>
> > Can we just let the dust
> >settle on this in the overall ecosystem to see what happens?
>
> Not a lot of time to organize an alternative if too much time is spent
> seeing if someone else does anything.
>
> I don't see Apple picking up supporting Java again on MacOS, I suppose
> Microsoft might be convinced to pick up Java for Windows if they felt
> it helped either Windows or Azure (on the other hand though this might
> convince a lot of organizations to move to the now multi-platform
> .Net).
>
> The question really becomes who sees it in their interest to come up
> with the money to support (either through OpenJDK or an outside group)
> the process of building and maintaining a free set of Java binaries.
>
> Unless someone steps up then the best anyone can do (at least in terms
> of no-cost use of Java) is to work around where we know there will be
> long term Java support, that being the Linux/BSD versions that provide
> long term support through the distribution.
>
> If that is the case, then it might be worth someone creating an
> informal group to discuss between the distributions and the software
> projects that rely on Java what version of Java to support.  If for
> example Red Hat, Ubuntu, Debian, FreeBSD, etc. can't come to an
> informal agreement to all support the same version of Java we could in
> 5 years end up with Red Hat supporting X, Ubuntu X+1, Debian X-1, etc.
> with the result that the software have to try and pick and choose.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Paying off tech debt and correctly naming things

[kurt greaves]

As someone who came to the codebase post CQL but prior to thrift being
removed, +1 to refactor. The current mixing of terminology is a complete
nightmare. This would also give a good opportunity document a lot of code
that simply isn't documented (or incorrect). I'd say it's worth doing it in
multiple steps though, such as refactor of a single class at a time, then
followed by refactor of variable names. We've already done one pretty big
refactor (InetAddressAndPort) for 4.0, I don't see how a few more could
make it any worse (lol).

Row vs partition vs key vs PK is killing me

On 20 March 2018 at 22:04, Jon Haddad  wrote:

> Whenever I hop around in the codebase, one thing that always manages to
> slow me down is needing to understand the context of the variable names
> that I’m looking at.  We’ve now removed thrift the transport, but the
> variables, classes and comments still remain.  Personally, I’d like to go
> in and pay off as much technical debt as possible by refactoring the code
> to be as close to CQL as possible.  Rows should be rows, not partitions,
> I’d love to see the term column family removed forever in favor of always
> using tables.  That said, it’s a big task.  I did a quick refactor in a
> branch, simply changing the ColumnFamilyStore class to TableStore, and
> pushed it up to GitHub. [1]
>
> Didn’t click on the link?  That’s ok.  The TL;DR is that it’s almost 2K
> LOC changed across 275 files.  I’ll note that my branch doesn’t change any
> of the almost 1000 search results of “columnfamilystore” found in the
> codebase and hundreds of tests failed on my branch in CircleCI, so that 2K
> LOC change would probably be quite a bit bigger.  There is, of course, a
> lot more than just renaming this one class, there’s thousands of variable
> names using any manner of “cf”, “cfs”, “columnfamily”, names plus comments
> and who knows what else.  There’s lots of references in probably every file
> that would have to get updated.
>
> What are people’s thoughts on this?  We should be honest with ourselves
> and know this isn’t going to get any easier over time.  It’s only going to
> get more confusing for new people to the project, and having to figure out
> “what kind of row am i even looking at” is a waste of time.  There’s
> obviously a much bigger impact than just renaming a bunch of files, there’s
> any number of patches and branches that would become outdated, plus anyone
> pulling in Cassandra as a dependency would be affected.  I don’t really
> have a solution for the disruption other than “leave it in place”, but in
> my mind that’s not a great (or even good) solution.
>
> Anyways, enough out of me.  My concern for ergonomics and naming might be
> significantly higher than the rest of the folks working in the code, and I
> wanted to put a feeler out there before I decided to dig into this in a
> more serious manner.
>
> Jon
>
> [1]
> https://github.com/apache/cassandra/compare/trunk...rustyrazorblade:refactor_column_family_store?expand=1
> <
> https://github.com/apache/cassandra/compare/trunk...rustyrazorblade:refactor_column_family_store?expand=1
> >

Re: Website front page broken links

[kurt greaves]

A ticket already exists that covers this, with a patch. just needs some
consensus around the third party page.
https://issues.apache.org/jira/browse/CASSANDRA-14128

On 19 Mar. 2018 18:26, "Hannu Kröger"  wrote:

> Hello,
>
> I created this ticket
> https://issues.apache.org/jira/browse/CASSANDRA-14324
>
> Basically the website front page contains manybroken links to
> planetcassandra.org
>
> Hannu
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Debug logging enabled by default since 2.2

[kurt greaves]

On the same page as Michael here. We disable debug logs in production due
to the performance impact. Personally I think if debug logging is necessary
for users to use the software we're doing something wrong. Also in my
experience, if something doesn't reproduce it will not get fixed. Debug
logging helps, but I've never seen a case where an actual bug simply
*doesn't* reproduce eventually, and I'm sure if this issue appears debug
logging could be enabled after the fact for the relevant classes and
eventually it will reoccur and we could solve the problem. I've never seen
a user say no to helping debug a problem with patched jars/changes to their
system (like logging). I'd much prefer we pushed that harder rather than
just saying "Everyone gets debug logging!". I'm also really not sold that
debug logging saves the day. To me it mostly just speeds up the
identification process, it generally doesn't expose magical information
that wasn't available before, you just had to think about it a bit more.


In a way the real issue might be that we don’t have nightly performance
> runs that would make an accidentally introduced debug statement obvious.

This is an issue, but I don't think it's the *real* issue. As already
noted, debug logging is for debugging, which normal users shouldn't have to
think about when they are just operating the software. We shouldn't risk
performance regressions just for developer convenience.

On 19 March 2018 at 00:55, Ariel Weisberg  wrote:

> Hi,
>
> In a way the real issue might be that we don’t have nightly performance
> runs that would make an accidentally introduced debug statement obvious.
>
> A log statement that runs once or more per read or write should be easy to
> spot. I haven’t measured the impact though. And as a bonus by having this
> we can spot a variety of performance issues introduced by all kinds of
> changes.
>
> Ariel
>
> > On Mar 18, 2018, at 3:46 PM, Jeff Jirsa  wrote:
> >
> > In Cassandra-10241 I said I was torn on this whole ticket, since most
> people would end up turning it off if it had a negative impact. You said:
> >
> > “I'd like to emphasize that we're not talking about turning debug or
> trace on for client-generated request paths. There's way too much data
> generated and it's unlikely to be useful.
> > What we're proposing is enabling debug logging ONLY for cluster state
> changes like gossip and schema, and infrequent activities like repair. “
> >
> > Clearly there’s a disconnect here - we’ve turned debug logging on for
> everything and shuffled some stuff to trace, which is a one time action but
> is hard to protect against regression. In fact, just looking at the read
> callback shows two instances of debug log in the client request path
> (exercise for the reader to “git blame”).
> >
> > Either we can go clean up all the surprises that leaked through, or we
> can turn off debug and start backing out some of the changes in 10241.
> Putting stuff like compaction in the same bucket as digest mismatch and
> gossip state doesn’t make life materially better for most people.
> >
> >
> > --
> > Jeff Jirsa
> >
> >
> >> On Mar 18, 2018, at 11:21 AM, Jonathan Ellis  wrote:
> >>
> >> That really depends on whether you're judicious in deciding what to log
> at
> >> debug, doesn't it?
> >>
> >> On Sun, Mar 18, 2018 at 12:57 PM, Michael Kjellman 
> >> wrote:
> >>
> >>> +1. this is how it works.
> >>>
> >>> your computer doesn’t run at debug logging by default. your phone
> doesn’t
> >>> either. neither does your smart tv. your database can’t be running at
> debug
> >>> just because it makes our lives as engineers easier.
> >>>
>  On Mar 18, 2018, at 5:14 AM, Alexander Dejanovski <
> >>> a...@thelastpickle.com> wrote:
> 
>  It's a tiny bit unusual to turn on debug logging for all users by
> default
>  though, and there should be occasions to turn it on when facing issues
> >>> that
>  you want to debug (if they can be easily reproduced).
> >>>
> >>
> >>
> >>
> >> --
> >> Jonathan Ellis
> >> co-founder, http://www.datastax.com
> >> @spyced
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Action Required: We are sunsetting CircleCI 1.0 on August 31, 2018

[kurt greaves]

Not that much gets committed to 2.1 and 2.2 anymore, but is this also true
for those branches?

On 27 February 2018 at 22:58, Michael Kjellman  wrote:

> FYI: we're already fully on circleci 2.0 for the 3.0, 3.11, and trunk
> branches so no action required for us here!
>
> best,
> kjellman
>
> Begin forwarded message:
>
> From: The CircleCI Team  no-re...@circleci.com>>
> Subject: Action Required: We are sunsetting CircleCI 1.0 on August 31, 2018
> Date: February 27, 2018 at 2:44:01 PM PST
> To: mkjell...@internalcircle.com
> Reply-To: mailto:no-re...@circleci.com>>
>
>
> Dear customer,
>
> We wanted to let you know that we are planning on sunsetting CircleCI 1.0
> on August 31st, 2018. Our goal as a company for 2018 is to invest in
> delivering more features and better performance on CircleCI 2.0, which
> unlocks faster builds and greater control. For more information, you can
> read our blog post on
> sunsetting CircleCI 1.0.
>
> You’ll need to update all of your config files to the CircleCI 2.0 syntax
> in order to migrate your projects to CircleCI 2.0 over the next 6 months.
>
> If all of your projects are already on 2.0, congratulations! No action is
> necessary. We are sending this announcement to all active users to make
> sure you have all of the information you need. Take a look at your builds
> dashboard to see if your
> projects are still building on CircleCI 1.0:
>
> [https://www2.circleci.com/rs/485-ZMH-626/images/CircleCI%
> 20Version%20Number.png]
>
>
> These resources will help you get migrate your projects from 1.0 to 2.0:
>
> Config.yml translator.<
> http://go.circleci.com/ZMkaU018240m720GHZ0> Note: This will generate
> a baseline config.yml file that you can adjust to fit your needs.
> 1.0 to 2.0 migration documentation. X2G0U9M0k000H0am4021Z80>
> Language-specific 2.0 tutorials. com/Q00H0mM0Z92G40k1200aaU0>
>
> We will be sending you email reminders periodically with additional
> resources and links as they become available to help with your migration
> plan. We will also be updating this page ga0makG04201MH02Z000b0U> with information relevant to sunsetting CircleCI
> 1.0. If you need additional migration assistance, open a support request<
> http://go.circleci.com/R00Gam1M420020U0ZbH0ck0> and our support team will
> be in touch.
>
>
> Cheers,
>
> The CircleCI Team
>
>
>
>
>

Re: Why isn't there a separate JVM per table?

[kurt greaves]

>
>  ... compaction on its own jvm was also something I was thinking about, but
> then I realized even more JVM sharding could be done at the table level.


Compaction in it's own JVM makes sense. At the table level I'm not so sure
about. Gotta be some serious overheads from running that many JVM's.
Keyspace might be reasonable purely to isolate bad tables, but for the most
part I'd think isolating every table isn't that beneficial and pretty
complicated. In most cases people just fix their modelling so that they
don't generate large amounts of GC, and hopefully test enough so they know
how it will behave in production.

If we did at the table level we would inevitable have to make each
individual table incredibly tune-able which would be a bit tedious IMO.
There's no way for us to smartly decide how much heap/memtable space/etc
each table should use (not without some decent AI, anyway).
​

Re: Release votes

[kurt greaves]

It seems there has been a bit of a slip in testing as of recently, mostly
due to the fact that there's no canonical testing environment that isn't
flaky. We probably need to come up with some ideas and a plan on how we're
going to do testing in the future, and how we're going to make testing
accessible for all contributors. I think this is the only way we're really
going to change behaviour. Having an incredibly tedious process and then
being aggressive about it only leads to resentment and workarounds.

I'm completely unsure of where dtests are at since the conversion to
pytest, and there's a lot of failing dtests on the ASF jenkins jobs (which
appear to be running pytest). As there's currently not a lot of visibility
into what people are doing with CircleCI for this it's hard to say if
things are better over there. I'd like to help here if anyone wants to fill
me in.

On 15 February 2018 at 21:14, Josh McKenzie  wrote:

> >
> > We’ve said in the past that we don’t release without green tests. The PMC
> > gets to vote and enforce it. If you don’t vote yes without seeing the
> test
> > results, that enforces it.
>
> I think this is noble and ideal in theory. In practice, the tests take long
> enough, hardware infra has proven flaky enough, and the tests *themselves*
> flaky enough, that there's been a consistent low-level of test failure
> noise that makes separating signal from noise in this context very time
> consuming. Reference 3.11-test-all for example re:noise:
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-3.11-test-all/test/?width=1024&height=768
>
> Having spearheaded burning test failures to 0 multiple times and have them
> regress over time, my gut intuition is we should have one person as our
> Source of Truth with a less-flaky source for release-vetting CI (dedicated
> hardware, circle account, etc) we can use as a reference to vote on release
> SHA's.
>
> We’ve declared this a requirement multiple times
>
> Declaring things != changed behavior, and thus != changed culture. The
> culture on this project is one of having a constant low level of test
> failure noise in our CI as a product of our working processes. Unless we
> change those (actually block release w/out green board, actually
> aggressively block merge w/any failing tests, aggressively retroactively
> track down test failures on a daily basis and RCA), the situation won't
> improve. Given that this is a volunteer organization / project, that kind
> of daily time investment is a big ask.
>
> On Thu, Feb 15, 2018 at 1:10 PM, Jeff Jirsa  wrote:
>
> > Moving this to it’s own thread:
> >
> > We’ve declared this a requirement multiple times and then we occasionally
> > get a critical issue and have to decide whether it’s worth the delay. I
> > assume Jason’s earlier -1 on attempt 1 was an enforcement of that earlier
> > stated goal.
> >
> > It’s up to the PMC. We’ve said in the past that we don’t release without
> > green tests. The PMC gets to vote and enforce it. If you don’t vote yes
> > without seeing the test results, that enforces it.
> >
> > --
> > Jeff Jirsa
> >
> >
> > > On Feb 15, 2018, at 9:49 AM, Josh McKenzie 
> wrote:
> > >
> > > What would it take for us to get green utest/dtests as a blocking part
> of
> > > the release process? i.e. "for any given SHA, here's a link to the
> tests
> > > that passed" in the release vote email?
> > >
> > > That being said, +1.
> > >
> > >> On Wed, Feb 14, 2018 at 4:33 PM, Nate McCall 
> > wrote:
> > >>
> > >> +1
> > >>
> > >> On Thu, Feb 15, 2018 at 9:40 AM, Michael Shuler <
> mich...@pbandjelly.org
> > >
> > >> wrote:
> > >>> I propose the following artifacts for release as 3.0.16.
> > >>>
> > >>> sha1: 890f319142ddd3cf2692ff45ff28e71001365e96
> > >>> Git:
> > >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > >> shortlog;h=refs/tags/3.0.16-tentative
> > >>> Artifacts:
> > >>> https://repository.apache.org/content/repositories/
> > >> orgapachecassandra-1157/org/apache/cassandra/apache-cassandra/3.0.16/
> > >>> Staging repository:
> > >>> https://repository.apache.org/content/repositories/
> > >> orgapachecassandra-1157/
> > >>>
> > >>> Debian and RPM packages are available here:
> > >>> http://people.apache.org/~mshuler
> > >>>
> > >>> *** This release addresses an important fix for CASSANDRA-14092 ***
> > >>>"Max ttl of 20 years will overflow localDeletionTime"
> > >>>https://issues.apache.org/jira/browse/CASSANDRA-14092
> > >>>
> > >>> The vote will be open for 72 hours (longer if needed).
> > >>>
> > >>> [1]: (CHANGES.txt) https://goo.gl/rLj59Z
> > >>> [2]: (NEWS.txt) https://goo.gl/EkrT4G
> > >>>
> > >>> 
> -
> > >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >>>
> > >>
> > >> -
> > >> To unsubscribe, e-mail: dev-

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[kurt greaves]

CASSANDRA-14234 <https://issues.apache.org/jira/browse/CASSANDRA-14234> patch
in for ReadCommandTest if anyone wants to take a look.

On 14 February 2018 at 00:45, kurt greaves  wrote:

> ​ReadCommandTest is still not passing. I would be a happy maintainer if
>> someone could step up and analyze and contribute a patch; I will review.
>
>
> Having a look. Appears to be just a case of using the same CF for multiple
> tests and metrics (tombstone histograms) aren't cleared in between.
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[kurt greaves]

>
> ​ReadCommandTest is still not passing. I would be a happy maintainer if
> someone could step up and analyze and contribute a patch; I will review.


Having a look. Appears to be just a case of using the same CF for multiple
tests and metrics (tombstone histograms) aren't cleared in between.

Re: [VOTE] Release Apache Cassandra 3.11.2

[kurt greaves]

May as well just recut and include it IMO. Sure, the issue is
work-aroundable, but in a pretty terrible way. Only a few hours will be
lost (sorry Michael :/)​

Re: Coordinator Write Metrics per CF

[kurt greaves]

I've tried to search around for a reason for this in the past and haven't
found one. I don't think it's a bad idea. Would be a helpful metric to
diagnose internode networking issues - although I'll note that the read
metric will also achieve this assuming you have enough reads to get some
useful information out of it.

On 9 February 2018 at 17:43, Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> There is an existing CoordinatorReadLatency table metric. I am looking to
> add CoordinatorWriteLatency table metric - however, before I attempt a shot
> at it, would like to know if anyone has context around why we currently do
> not have such metric (while we have the read metric) - if someone has
> already attempted and realized its a bad idea for some reason.
>
> Thanks,
> Sumanth
>

Roadmap for 4.0

[kurt greaves]

Hi friends,
*TL;DR: Making a plan for 4.0, ideally everyone interested should provide
up to two lists, one for tickets they can contribute resources to getting
finished, and one for features they think would be desirable for 4.0, but
not necessarily have the resources to commit to helping with.*

So we had that Roadmap for 4.0 discussion last year, but there was never a
conclusion or a plan that came from it. Times getting on and the changes
list for 4.0 is getting pretty big. I'm thinking it would probably make
sense to define some goals to getting 4.0 released/have an actual plan. 4.0
is already going to be quite an unwieldy release with a lot of testing
required.

Note: the following is open to discussion, if people don't like the plan
feel free to speak up. But in the end it's a pretty basic plan and I don't
think we should over-complicate it, I also don't want to end up in a
discussion where we "make a plan to make a plan". Regardless of whatever
plan we do end up following it would still be valuable to have a list of
tickets for 4.0 which is the overall goal of this email - so let's not get
too worked up on the details just yet (save that for after I
summarise/follow up).

// TODO
I think the best way to go about this would be for us to come up with a
list of JIRA's that we want included in 4.0, tag these as 4.0, and all
other improvements as 4.x. We can then aim to release 4.0 once all the 4.0
tagged tickets (+bug fixes/blockers) are complete.

Now, the catch is that we obviously don't want to include too many tickets
in 4.0, but at the same time we want to make sure 4.0 has an appealing
feature set for both users/operators/developers. To minimise scope creep I
think the following strategy will help:

We should maintain two lists:

   1. JIRA's that people want in 4.0 and can commit resources to getting
   them implemented in 4.0.
   2. JIRA's that people simply think would be desirable for 4.0, but
   currently don't have anyone assigned to them or planned assignment. It
   would probably make sense to label these with an additional tag in
JIRA. *(User's
   please feel free to point out what you want here)*

>From list 1 will come our source of truth for when we release 4.0. (after
aggregating a list I will summarise and we can vote on it).

List 2 would be the "hopeful" list, where stories can be picked up from if
resourcing allows, or where someone comes along and decides it's good
enough to work on. I guess we can also base this on a vote system if we
reach the point of including some of them. (but for the moment it's purely
to get an idea of what users actually want).

Please don't refrain from listing something that's already been mentioned.
The purpose is to get an idea of everyone's priorities/interests and the
resources available. We will need multiple resources for each ticket, so
anywhere we share an interest will make for a lot easier work sharing.

Note that we are only talking about improvements here. Bugs will be treated
the same as always, and major issues/regressions we'll need to fix prior to
4.0 anyway.

TIME FRAME
Generally I think it's a bad idea to commit to any hard deadline, but we
should have some time frames in mind. My idea would be to aim for a Q3/4
2018 release, and as we go we just review the outstanding improvements and
decide whether it's worth pushing it back or if we've got enough to
release. I suppose keep this time frame in mind when choosing your tickets.

We can aim for an earlier date (midyear?) but I figure the
testing/validation/bugfixing period prior to release might drag on a bit so
being a bit conservative here.
The main goal would be to not let list 1 grow unless we're well ahead, and
only cull from it if we're heavily over-committed or we decide the
improvement can wait. I assume this all sounds like common sense but
figured it's better to spell it out now.


NEXT STEPS
After 2 weeks/whenever the discussion dies off I'll consolidate all the
tickets, relevant comments and follow up with a summary, where we can
discuss/nitpick issues and come up with a final list to go ahead with.

On a side note, in conjunction with this effort we'll obviously have to do
something about validation and testing. I'll keep that out of this email
for now, but there will be a follow up so that those of us willing to help
validate/test trunk can avoid duplicating effort.

REVIEW
This is the list of "huge/breaking" tickets that got mentioned in the last
roadmap discussion and their statuses. This is not terribly important but
just so we can keep in mind what we previously talked about. I think we
leave it up to the relevant contributors to decide whether they want to get
the still open tickets into 4.0.

CASSANDRA-9425 Immutable node-local schema
 - Committed
CASSANDRA-10699 Strongly consistent schema alterations
 - Open, no
discussion in quite some time.
CASSANDRA-12229 

Re: range queries on partition key supported?

[kurt greaves]

>
> So that means more than one nodes can be selected to fulfill a range query
> based on the token, correct?


Yes. When doing a token range query Cassandra will need to send requests to
any node that owns part of the token range requested. This could be just
one set of replicas or more, depending on how your token ring is arranged.
You could avoid querying multiple nodes by limiting the token() calls to be
within one token range.

And, then entire partition on each node will be searched based on the
> clustering key (i.e. "time" in this case).

No. it will skip to the section of the partition with time = '12:00'.
Cassandra should be smart enough to avoid reading the whole partition.


On 31 January 2018 at 06:57, Tyagi, Preetika 
wrote:

> So that means more than one nodes can be selected to fulfill a range query
> based on the token, correct?
>
> I was looking at this link: https://www.datastax.com/dev/
> blog/a-deep-look-to-the-cql-where-clause
>
> In the example query,
> SELECT * FROM numberOfRequests
> WHERE token(cluster, date) > token('cluster1', '2015-06-03')
> AND token(cluster, date) <= token('cluster1', '2015-06-05')
> AND time = '12:00'
>
> More than one nodes might get picked for this token based range query.
> And, then entire partition on each node will be searched based on the
> clustering key (i.e. "time" in this case).
> Is my understanding correct?
>
> Thanks,
> Preetika
>
> -Original Message-
> From: J. D. Jordan [mailto:jeremiah.jor...@gmail.com]
> Sent: Tuesday, January 30, 2018 10:13 AM
> To: dev@cassandra.apache.org
> Subject: Re: range queries on partition key supported?
>
> A range query can be performed on the token of a partition key, not on the
> value.
>
> -Jeremiah
>
> > On Jan 30, 2018, at 12:21 PM, Tyagi, Preetika 
> wrote:
> >
> > Hi All,
> >
> > I have a quick question on Cassandra's behavior in case of partition
> keys. I know that range queries are allowed in general, however, is it also
> allowed on partition keys as well? The partition key is used as an input to
> determine a node in a cluster, so I'm wondering how one can possibly
> perform range query on that.
> >
> > Thanks,
> > Preetika
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: cpp driver warning for create keyspace statement

[kurt greaves]

Seems like it might be a bug in the driver. Can you insert some data using
the driver and query it from that datacenter either via the driver
specifying a DC aware load balancing policy against DC-Ffm-2 with
consistency LOCAL_ONE? or if that doesn't work, using cqlsh on a node in
that datacenter with CONSISTENCY LOCAL_ONE?​

Also double check that the DC name reported in status is *exactly* the same
as the DC name in the replication details.

Re: CASSANDRA-8527

[kurt greaves]

I think tombstone rows is more appropriate. Calling them deleted rows
initially confused me and it took a few reads of the problem for me to work
it out. Especially when you put it in context and "tombstone cells"
follows.​

Re: CASSANDRA-8527

[kurt greaves]

I think that's a good idea for 4.x, but not so for current branches. I
think as long as we document it well for 4.0 upgrades it's not so much of a
problem. Obviously there will be cases of queries failing that were
previously succeeding but we can already use
tombstone_failure|warn_threshold to tune around that already. I don't think
we need another yaml setting to enable/disable counting deleted rows for
these thresholds, especially because it's going into 4.0. It *might* be a
good idea to bump the tombstone failure threshold default as a safety net
though (as well as put something in the NEWS.txt).

On 21 December 2017 at 20:11, Jon Haddad  wrote:

> I had suggested to Alex we kick this discussion over to the ML because the
> change will have a significant impact on the behavior of Cassandra when
> doing reads with range tombstones that cover a lot of rows.  The behavior
> now is a little weird, a single tombstone could shadow hundreds of
> thousands or even millions of rows, and the query would probably just time
> out.  Personally, I’m in favor of the change in behavior of this patch but
> I wanted to get some more feedback before committing to it.  Are there any
> objections to what Alex described?
>
> Regarding Mockito, I’ve been meaning to bring this up for a while, and I’m
> a solid +1 on introducing it to help with testing.  In an ideal world we’d
> have no singletons and could test everything in isolation, but
> realistically that’s a multi year process and we just aren’t there.
>
>
> > On Dec 19, 2017, at 11:07 PM, Alexander Dejanovski <
> a...@thelastpickle.com> wrote:
> >
> > Hi folks,
> >
> > I'm working on CASSANDRA-8527
> >  and would need to
> > discuss a few things.
> >
> > The ticket makes it visible in tracing and metrics that rows shadowed by
> > range tombstones were scanned during reads.
> > Currently, scanned range tombstones aren't reported anywhere which hides
> > the cause of performance issues during reads when the users perform
> primary
> > key deletes.
> > As such, they do not count in the warn and failure thresholds.
> >
> > While the number of live rows and tombstone cells is counted in the
> > ReadCommand class, it is currently not possible to count the number of
> > range tombstones there as they are merged with the rows they shadow
> before
> > reaching the class.
> > Instead, we can count the number of deleted rows that were read , which
> > already improves diagnosis and show that range tombstones were scanned :
> >
> > if (row.hasLiveData(ReadCommand.this.nowInSec(), enforceStrictLiveness))
> >++liveRows;
> > else if (!row.primaryKeyLivenessInfo().isLive(ReadCommand.this.
> nowInSec()))
> > {
> >// We want to detect primary key deletions only.
> >// If all cells have expired they will count as tombstones.
> >   ++deletedRows;
> > }
> >
> > Deleted rows would be part of the warning threshold so that we can spot
> the
> > range tombstone scans in the logs and tracing would look like this :
> >
> > WARN  [ReadStage-2] 2017-12-18 18:22:31,352 ReadCommand.java:491 -
> > Read 2086 live rows, 2440 deleted rows and 0 tombstone cells for
> > query..
> >
> >
> > Are there any caveats to that approach ?
> > Should we include the number of deleted rows in the failure threshold or
> > make it optional, knowing that it could make some queries fail while they
> > were passing before ?
> >
> > On a side note, is it ok to bring in Mockito in order to make it easier
> > writing tests ? I would like to use a Spy in order to write some tests
> > without starting the whole stack.
> >
> > Thanks,
> >
> >
> > --
> > -
> > Alexander Dejanovski
> > France
> > @alexanderdeja
> >
> > Consultant
> > Apache Cassandra Consulting
> > http://www.thelastpickle.com
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Proposal: github pull requests for all code changes

[kurt greaves]

It would definitely be nice if PR's were made against the actual repo.
Makes commenting/reviewing code a lot easier. I'd say as long as PR's are
kept permanently (after closing) so we can go back and look over
comments/patches would be a big plus. Not a fan of going through JIRA
tickets and finding the linked branches have been deleted.

However not sure about having different identifiers/labels everywhere.
Considering we'll still be creating JIRA tickets it makes sense to me to
use JIRA for labelling/categorisation, and github simply for reviewing
code. Don't really want to go assigning JIRA tickets then have to repeat
the proceduce for github and make sure all labels align.

Also, if we're adding "closes #123" to commit messages that implies we'll
have "patch by y, reviewed by x for CASSANDRA- closes #123"? I don't
know if there is a way to do it (doubtful), but it'd certainly be nice if
JIRA ticket numbers aligned with PR numbers.

On 21 December 2017 at 02:56, mck  wrote:

>
> > There's also no way for us to label, assign or otherwise use PR related
> > features, so I'm really wondering why it would make sense to more
> > heavily using them.
>
>
> Apache does now offer the GitBox service. This makes the github repository
> writable. The asf and github repos are kept in sync, rather than one being
> a mirror of the other.
>
> I'm not entirely sure this would be the right move for C*, given that
> patches and branches are merged in a manner that GitHub pull requests don't
> do (unless you're only committing to trunk). But it would mean PRs can be
> closed otherwise by any of the committers, and to use the reviewers,
> assignees, labels, projects and milestone fields.
>
> Mick.
>
>
> ref:
>  - http://apache-nifi.1125220.n5.nabble.com/DISCUSS-Apache-
> Gitbox-td18273.html
>  - https://s3.amazonaws.com/files-dist/AnnualReports/
> FY2017-ASF-AnnualReport-FINAL.pdf
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: How to fetch replication factor of a given keyspace

[kurt greaves]

Did you look at how status fetches and initialises the keyspace? Have a
look at org.apache.cassandra.service.StorageService#effectiveOwnership. It
uses a keyspace and its RF to measure ownership. The same method should
work for any nodetool command that needs a keyspace. Note that depending on
what you are using it for you may need to handle each strategy differently
(and also handle LocalStrategy).

On 21 December 2017 at 01:02, Tyagi, Preetika 
wrote:

> Hi,
>
> If I need to get the replication factor of a given keyspace in nodetool
> commands (e.g. status), how can I do that? I'm trying to figure it out for
> a JIRA item I'm working on.
>
> I tried using the below:
> Keyspace keyspace = Keyspace.open(keyspaceName);
> Int rf = keyspace.getReplicationStrategy().getReplicationFactor()
>
> However, it runs into some issues since internally something doesn't get
> initialized while looking up keyspaces/metatadata.
>
> Any ideas on how I can approach it differently?
>
> Thanks,
> Preetika
>
>

Reviewer for LWT bug

[kurt greaves]

Need a reviewer for CASSANDRA-14087


Pretty straight forward, we just get an NPE when comparing against a frozen
collection which is null and we expect a specific collection. Anyone with a
bit of knowledge around ColumnCondition.java should be able to review.

Patch is for 3.0 but should apply cleanly to 3.11. Trunk is unaffected.

Cheers,
Kurt

Re: [PROPOSAL] Migrate to pytest from nosetests for dtests

[kurt greaves]

+1nb, Never been that fond of nose from a usability perspective, and I
wouldn't be surprised if at least some of the problems running dtests were
related to issues in nose. I can't imagine it would be a lot of work to
port to py.test, if someone wants to do it they can go right ahead.

On 29 November 2017 at 17:55, Michael Kjellman  wrote:

> s/handling/hanging
>
> > On Nov 29, 2017, at 9:54 AM, Michael Kjellman <
> mkjell...@internalcircle.com> wrote:
> >
> > i keep seeing nose randomly handing after a test successfully completes
> execution. i’m very far from a python guru but i spent a few hours with gdb
> trying to debug the thing and get python stacks and got symbolicated native
> stacks but it’s random but root causing while nose is sitting on a lock
> forever alludes me. some tests are more reproducible than others. some i
> see fail 1 in 10 runs.
> >
> > the net of it all though is this makes people not trust dtests because
> it randomly hangs and shows tests with “failures” that actually succeeded.
> >
> > i’m not a huge fan of just blindly upgrading to fix a problem but in
> this case I found that there is quite a lot of mistrust and dislike for
> nosetests in the python community with most projects already moving to
> pytest. and if it is some complicated set of interactions between threads
> we use in the tests and how nose works do we really want to even debug it
> when the project appears to be abandoned?
> >
> > i think regardless of the root cause for making things more stable it
> seems like there is little motivation to stick around on nose...
> >
> > lmk!
> >
> > best,
> > kjellman
> >
> >> On Nov 29, 2017, at 5:33 AM, Philip Thompson <
> philip.thomp...@datastax.com> wrote:
> >>
> >> I don't have any objection to this, really. I know I rely on a handful
> of
> >> nose plugins, and possibly others do, but those should be easy enough to
> >> re-write. I am curious though, what's the impetus for this? Is there
> some
> >> pytest feature we want that nose lacks? Is there some nosetest bug or
> >> restriction getting in the way?
> >>
> >>> On Tue, Nov 28, 2017 at 8:34 PM, Jon Haddad  wrote:
> >>>
> >>> +1
> >>>
> >>> I stopped using nose a long time ago in favor of py.test.  It’s a
> >>> significant improvement.
> >>>
>  On Nov 28, 2017, at 10:49 AM, Michael Kjellman 
> >>> wrote:
> 
>  I'd like to propose we move from nosetest to pytest for the dtests. It
> >>> looks like nosetests is basically abandoned, the python community
> doesn't
> >>> like it, it hasn't been updated since 2015, and pytest even has
> nosetests
> >>> support which would help us greatly during migration (
> >>> https://docs.pytest.org/en/latest/nose.html).
> 
>  Thoughts?
> 
>  best,
>  kjellman
> >>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>

Re: Flakey Dtests

[kurt greaves]

Upgrade tests are probably broken because we haven't been running them
since the move to ASF jenkins (I believe). I'll start having a look at some
MV test failures.

On 17 November 2017 at 14:34, Josh McKenzie  wrote:

> >
> > Do we have any volunteers to fix the broken Materialized Views and CDC
> > DTests?
>
> I'll try to take a look at the CDC tests next week; looks like one of the
> base unit tests is failing as well.
>
> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
> mkjell...@internalcircle.com> wrote:
>
> > Quick update re: dtests and off-heap memtables:
> >
> > I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> > offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
> >
> > Looks like we’re gonna need to do some work to test this configuration
> and
> > right now it’s pretty broken...
> >
> > Do we have any volunteers to fix the broken Materialized Views and CDC
> > DTests?
> >
> > best,
> > kjellman
> >
> >
> > > On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> > mkjell...@internalcircle.com> wrote:
> > >
> > > yes - true- some are flaky, but almost all of the ones i filed fail
> 100%
> > (💯) of the time. i look forward to triaging just the remaining flaky
> ones
> > (hopefully - without powers combined - by the end of this month!!)
> > >
> > > appreciate everyone’s help - no matter how small... i already
> personally
> > did a few “fun” random-python-class-is-missing-return-after-method
> stuff.
> > >
> > > we’ve wanted this for a while and now is our time to actually execute
> > and make good on our previous dev list promises.
> > >
> > > best,
> > > kjellman
> > >
> > >> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa  wrote:
> > >>
> > >> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> > >>
> > >> If you haven't been paying attention to JIRA, you likely didn't notice
> > that
> > >> Josh went through and triage/categorized a bunch of issues by adding
> > >> components, and Michael took the time to open a bunch of JIRAs for
> > failing
> > >> tests.
> > >>
> > >> How many is a bunch? Something like 35 or so just for tests currently
> > >> failing on trunk.  If you're a regular contributor, you already know
> > that
> > >> dtests are flakey - it'd be great if a few of us can go through and
> fix
> > a
> > >> few. Even incremental improvements are improvements. Here's an easy
> > search
> > >> to find them:
> > >>
> > >> https://issues.apache.org/jira/secure/IssueNavigator.
> > jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+
> > component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> > DESC%2C+created+ASC&mode=hide
> > >>
> > >> If you're a new contributor, fixing tests is often a good way to
> learn a
> > >> new part of the codebase. Many of these are dtests, which live in a
> > >> different repo ( https://github.com/apache/cassandra-dtest ) and are
> in
> > >> python, but have no fear, the repo has instructions for setting up and
> > >> running dtests(
> > >> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> > >>
> > >> Normal contribution workflow applies: self-assign the ticket if you
> > want to
> > >> work on it, click on 'start progress' to indicate that you're working
> on
> > >> it, mark it 'patch available' when you've uploaded code to be reviewed
> > (in
> > >> a github branch, or as a standalone patch file attached to the JIRA).
> If
> > >> you have questions, feel free to email the dev list (that's what it's
> > here
> > >> for).
> > >>
> > >> Many thanks will be given,
> > >> - Jeff
> >
> >
>

Re: Somewhat Weekly Cassandra Dev Wrapup

[kurt greaves]

Looks like that solves that problem. Probably wouldn't be a bad idea to
include that list in the release posts as well as link to changes.txt? At
the very least it will make us a bit more careful about assigning fix
versions (maybe).

On 1 November 2017 at 00:36, Michael Shuler  wrote:

> On 10/31/2017 07:17 PM, kurt greaves wrote:
> > It's usually pretty clear what is a fix in the changes log but they
> > definitely require checking. However I'd say putting more boilerplate in
> > the changes log isn't really necessary and would be hard to enforce. If
> we
> > could generate a report with this information from JIRA with desired
> > information would be better (e.g all ticket names, type, maybe even
> > labels). I'd hope JIRA has this functionality...​
> >
>
> It does have release notes that is split up by issue type:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12310865&version=12336842
>
> --
> Michael
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Somewhat Weekly Cassandra Dev Wrapup

[kurt greaves]

It's usually pretty clear what is a fix in the changes log but they
definitely require checking. However I'd say putting more boilerplate in
the changes log isn't really necessary and would be hard to enforce. If we
could generate a report with this information from JIRA with desired
information would be better (e.g all ticket names, type, maybe even
labels). I'd hope JIRA has this functionality...​

Re: Using SCT to test cassandra clusters

[kurt greaves]

Thanks Roy, this actually looks pretty interesting. It seems that only AWS
is supported for Cassandra, and a lot of the code is obviously geared
towards scylla (e.g nemesis). Do you have any idea how much works w.r.t
Cassandra?

On 8 October 2017 at 17:00, Roy Dahan  wrote:

> Hi,
> My name is Roy and I'm one of the maintainers of SCT
> , a framework to deploy
> and test Scylla & Cassandra clusters on various backends like (AWS, GCE,
> OpenStack and libvirt).
>
> In general:
>
>- scdm.cluster: Contains cluster/node abstractions
>
>
>- scdm.nemesis: Cluster disruption procedures
>
>
>- scdm.remote: SSH library
>
>
>- scdm.tester: Contains base test class
>
>
> We regularly add more type of Nemesis (cluster disrupters) and tests.
>
> Important to note, It doesn't replace dtest  but it's a useful framework to
> test the cluster as a whole for longer period of time with various
> scenarios.
>
> Feel free to use and contribute back.
>
> Roy
> - QA Manager, Scylla
>

Re: What would be the appropriate number of vnodes (num_tokens) to use?

[kurt greaves]

>
> If you had more vnodes per
> machine, you could stream more ranges in parallel, taking advantage of more
> cores, streaming significantly faster. This is a very real gain if you are
> regularly adding or removing a FEW nodes.

In our experience 16 vnodes for streaming is much like 256. You tend to hit
other streaming bottlenecks in Cassandra pretty quickly, and 16 vnodes is
generally enough to give the streaming benefit (which isn't really that
much in practice).
I'd still say for any cluster over 20 nodes single tokens are the way to
go, despite the reduced streaming performance, but if you really don't want
to manage tokens certainly aim for 16 - 64 (depending on topology 16 may
not work that well until you get to 30ish nodes or more).
​

Re: Proposal to retroactively mark materialized views experimental

[kurt greaves]

>
> My concerns have consistently been more fundamental - the basic properties
> of a theoretically bug-free MV are simultaneously problematic and unknown.
> How can we say we have a working feature, when we our definition of
> ‘working’ is unknown?

Well, we know to some extent that MV's are theoretically possible, because
they work as is. There might be limitations to the design, as I know you've
raised before and were ignored, if that's what you're referring to here.
But for the most part they have been working just fine bar implementation
bugs. I wouldn't go as far as saying our definition of working is unknown,
that's a tad over the top. As I've already said, if you're aware of some
serious fundamental flaws, it shouldn't be too difficult to point them out.
I'd suspect they would have already been encountered so evidence shouldn't
be hard to find.

We have unsafe defaults

Yeah we do, but don't be changing them in a patch release. For the record,
MV's aren't something that is automatically created that everyone has to
deal with, it's still opt-in. I think incremental repairs and vnodes are
far worse defaults, and cause much more damage.

Nobody in the wild is even using MVs in the way they were designed, because
> they were too slow.  In no other endeavour have we gone “well, it’s too
> slow, so let’s just accept data loss by default”
>
 How exactly can you use MVs in a way different from how they were
designed? I don't understand, all the use cases I've seen have been
perfectly legitimate. I mean, you can use them with a bad data model, but I
wouldn't say that's not the way they were designed. That's possible for
normal tables as well.

In no other endeavour have we gone “well, it’s too slow, so let’s just
> accept data loss by default”
>
 Most people we've talked to prefer the simplicity of using MV's over the
speed. I don't know what the sentiment was at the beginning, but in the
past several months we've only cared about correctness, not speed.


Anyway, if there are fundamental flaws then they need to be addressed. As
soon as possible. I'm still going to tell you that telling existing users
who have already gone down this path that the feature is no longer
supported in production is a terrible idea. You are telling users "actually
you need to stop using that and re-write all your applications and do it
all yourself because we screwed up". It doesn't magically remove any fault
on our part for putting it in in the first place. The correct solution for
us and the users is to fix these issues, as soon as possible, and alert
everyone to the fact that these problems can occur. If we can't fix it,
then we can turn it off in the next major, otherwise we should just note
the flaws and restrict any functionality that just isn't going to work.

​

Re: Proposal to retroactively mark materialized views experimental

[kurt greaves]

>
> So you’d rather continue to lie to users about the stability of the
> feature rather than admitting it was merged in prematurely?

It was merged prematurely, but a lot has changed since then and a lot of
fixes have been made, and now it's really no more awful than any other
component of Cassandra. A lot of the commentary in here is coming from
people who have had no part of the recent changes to MV's, and as 3.11.1 is
not even out yet I doubt anyone really has any idea how stable they
currently are. In fact I doubt many people have even operated or consulted
on clusters with the most recent versions of 3.0 or 3.11. It's kind of
really annoying that this has come up now, after we've already done a lot
of the work to fix the known issues, and everyone seems to just be saying
"they are so broken" but no one can really provide any evidence why.

Ideally, we’d follow a process that looks a lot more like this:
> 1. New feature is built with an opt in flag.  Unknowns are documented, the
> risk of using the feature is known to the end user.
> 2. People test and use the feature that know what they’re doing.  They are
> able to read the code, submit patches, and help flush out the issues.  They
> do so in low risk environments.  In the case of MVs, they can afford to
> drop and rebuild the view over a week, or rebuild the cluster altogether.
> We may not even need to worry as much about backwards compatibility.
> 3. The feature matures.  More tests are written.  More people become aware
> of how to contribute to the feature’s stability.
> 4. After a while, we vote on removing the feature flag and declare it
> stable for general usage.


No I don't think this works very well for Cassandra because features are
often heavily intertwined with other components of Cassandra, and often a
new feature relies on making changes to other components of Cassandra. At
least this is true for any feature that is large enough to justify having
an opt-in flag. This will lead down the path of "oh it's only
experimental/opt-in so we don't need to worry about testing every single
component", which is wrong.
We provide a database, and users expect stability from all aspects of the
database, at all times. We should be working to fix bugs early so we can
have confidence in the entire database from very early in the release
branch. We shouldn't provide a database that people will say "don't use it
in production until at least the .15 patch release".

What I see is sufficient distrust coming from core committers, including
> the author of the v1 design, to warrant opt-in for MVs.

Core committers who have had almost nothing to do with MV since quite some
time ago.  Also I'm skeptical of how much first hand experience these core
committers have with MV's.

We already have those for UDFs and CDC.
> We should have more: for triggers, SASI, and MVs, at least. Operators need
> a way to disable features they haven’t validated.

After a bit more thought I've changed my mind on this. Operators do need a
way to disable features, but it makes a lot more sense to have that as part
of the auth/roles system rather than yaml properties. Plus as previously
noted, I'm not of the opinion we should release features (even in a
beta/experimental form) at all, and we should be reasonably confident in
the entire system and any new features being introduced prior to releasing
them. We should also better practice incremental release of features,
starting with a bare minimum, or subset of what we want the end product to
be, rather than releasing a massive change and then calling it experimental
for years until we can somehow deduce that it is stable enough. This could
have been done for MV's by starting with an append only use case, and then
moving onto the more complex transactional use case.
​

Re: Proposal to retroactively mark materialized views experimental

[kurt greaves]

>
> The flag name `cdc_enabled` is simple and, without adjectives, does not
> imply "experimental" or "beta" or anything like that.
> It does make life easier for both operators and the C* developers.

I would be all for a mv_enabled option, assuming it's enabled by default
for all existing branches. I don't think saying that you are meant to read
NEWS.txt before upgrading a patch is acceptable. Most people don't, and
expecting them to is a bit insane. Also Assuming that if they read it
they'd understand all implications is also a bit questionable. If deemed
suitable to turn it off that can be done in the next major/minor, but I
think that would be unlikely, as we should really require sufficient
evidence that it's dangerous which I just don't think we have. I'm still of
the opinion that MV in their current state are no worse off than a lot of
other features, and marking them as experimental and disabling now would
just be detrimental to their development and annoy users. Also if we give
them that treatment then there a whole load of other defaults we should
change and disable which is just not acceptable in a patch release. It's
not really necessary anyway, we don't have anyone crying bloody murder on
the mailing list about how everything went to hell because they used
feature x.

No one has really provided any counter evidence yet that MV's are in some
awful state and they are going to shoot users. There are a few existing
issues that I've brought up already, but they are really quite minor,
nothing comparable to "lol you can't repair if you use vnodes, sorry". I
think we really need some real examples/evidence before making calls like
"lets disable this feature in a patch release and mark it experimental"

>  I personally believe it is better to offer the feature as experimental
> until we iron out all of the problems

What problems are you referring to, and how exactly will we know when all
of them have been sufficiently ironed? If we mark it as experimental how
exactly are we going to get people to use said feature to find issues?
​

Re: CREATE INDEX without IF NOT EXISTS when snapshoting

[kurt greaves]

Certainly would make sense and should be trivial.  here

is
where you want to look. Just create a ticket for it and prod here for a
reviewer once you've got a change.​

Re: Proposal to retroactively mark materialized views experimental

[kurt greaves]

Lots of users are already using MV's, believe it or not in some cases quite
effectively and also on older versions which were still exposed to a lot of
the bugs that cause inconsistencies. 3.11.1 has come a long way since then
and I think with a bit more documentation around the current issues marking
MV's as experimental is unnecessary and likely annoying for current users.
On that note we've already had complaints about changing defaults and
behaviours willy nilly across majors and minors, I can't see this helping
our cause. Sure, you can make it "seamless" from an upgrade perspective,
but that doesn't account for every single way operators do things. I'm sure
someone will express surprise when they run up a new cluster or datacenter
for testing with default config and find out that they have to enable MV's.
Meanwhile they've been using them the whole time and haven't had any major
issues because they didn't touch the edge cases.

I'd like to point out that introducing "experimental" features sets a
precedent for future releases, and will likely result in using the
"experimental" tag to push out features that are not ready (again). In fact
we already routinely say >=3 isn't production ready yet, so why don't we
just mark 3+ as "experimental" as well? I don't think experimental is the
right approach for a database. The better solution, as I said, is more
verification and testing during the release process (by users!). A lot of
other projects take this approach, and it certainly makes sense. It could
also be coupled with beta releases, so people can start getting
verification of their new features at an earlier date. Granted this is
similar to experimental features, but applied to the whole release rather
than just individual features.

* There's no way to determine if a view is out of sync with the base table.
>
 As already pointed out by Jake, this is still true when you don't use
MV's. We should document this. I think it's entirely fair to say that
users *should
not *expect this to be done for them. There is also no way for a user to
determine they have inconsistencies short of their own verification. And
also a lot of the synchronisation problems have been resolved, undoubtedly
there are more unknowns out there but what MV's have is still better than
managing your own.

> * If you do determine that a view is out of sync, the only way to fix it
> is to drop and rebuild the view.
>
This is undoubtedly a problem, but also no worse than managing your own
views. Also at least there is still a way to fix your view. It certainly
shouldn't be as common in 3.11.1/3.0.15, and we have enough insight now to
be able to tell when out of sync will actually occur, so we can document
those cases.

> * There are liveness issues with updates being reflected in the view.

 What specific issues are you referring to here? The only one I'm aware of
is deletion of unselected columns in the view affecting out of order
updates. If we deem this a major problem we can document it or at least put
a restriction in place until it's fixed in CASSANDRA-13826

​

In this case, 'out of sync' means 'you lost data', since the current design
> + repair should keep things eventually consistent right?

I'd like Zhao or Paulo to confirm here but I believe the only way you can
really "lose data" (that can't be repaired) here would be partition
deletions on massively wide rows in the view that will not fit in the
batchlog (256mb/max value size) as it currently stands. Frankly this is
probably an anti-pattern for MV's at the moment anyway and one we should
advise against.

Re: Proposal to retroactively mark materialized views experimental

[kurt greaves]

Well this is all terribly interesting. I was actually going to get some
discussion going about this during my talk, which unfortunately didn't
happen, but I'll take this opportunity to push my agenda. My 99 cents:

*tl;dr: we should probably just focus on not releasing completely broken
features in the first place, and we should do that through user
engagement/testing wooo!*

Some context to begin with, because I think this needs to be spelled out.
Cassandra is a database. People treat databases as their prize possession.
It stores all their sweet sweet data, and undoubtedly that data is the most
important component in their system. Without it, there is no point in
having a system. Users expect their databases to be the most stable
component of their system, and generally they won't upgrade them without
being absolutely positively sure that a new version will work at least
exactly as the old one has. All our users treat their database in exactly
this same way. Change happens slowly in the database world, and generally
this is true both for the database and the users of the database. "C* 3.0.0
is out tomorrow! let's upgrade!" - said no one ever.

Anyway, with that out of the way, back to the crux of the issue. This may
get long and unwieldy, and derail the actual thread, but in this case I
think for good reason. Either way it's all relevant to the actual topic.

I think it's worth taking a step back and looking at the actual situation
and what brought us here, rather than just proposing a solution that's
really just a band-aid on the real issue. These are the problems I've seen
that have caused a lot of the pain with new features, and an indication
that we need to change the way we manage our releases and major changes.

   1. We pushed out large feature sets with minimal testing of said
   features. At this stage we had no requirement for clean passing tests on
   commit, and over all we didn't have a strong commitment to writing tests
   either. In 3.10 this changed, where we put forth that dtests and utests
   needed to pass, and new tests needed to be written for each change. Any
   change prior to 3.10 was subject to many flaky tests with minimal coverage.
   Many features only went partially tested and were committed anyway.

   2. We rushed features to meet deadlines, or simply didn't give them
   enough time + thought in the conception phase because of deadlines.
   I've never met an arbitrary deadline that made things better. From
   looking at lots of old tickets, there was "some" reason that even major
   changes had to be squeezed into 3.0 before it was released, which resulted
   in a lack of attention and testing for these features. We didn't just wait
   until things were ready before committing them, we just cut scope so it
   would fit. I honestly don't know how this could ever make sense for a
   volunteer driven project. In fact I don't really know how it works well for
   any software project. It generally just ends in bad software. It might make
   sense for a business pushing the feature agenda for $$, or where a projects
   users don't care about stability (lol), but it still results in bad
   software. It definitely doesn't make sense for an open source project.

   3. We didn't do any system-wide verification/integration testing of
   features. We essentially relied on dtests and unit tests. Touched on this
   in 1, but we don't have much system testing. dtests kind of covers it, but
   not really well. cstar is also used in some cases but is also limited in
   scope (performance only, really). We're lucky that we can cover a lot of
   cases with dtests, but it seems to me that we don't capture a lot of the
   cases where feature X affects feature Y. E.g: the effect of repairs against
   everything ever, but mostly vnodes. We really need a proper testing cluster
   with each version we put out, and to test new and existing features
   extensively to measure their worth. Instaclustr is looking at this but
   we're still a ways off having something up and running.
   On this note we also changed defaults prematurely, but we wouldn't know
   it was premature until we did so, as if we didn't change the default they
   probably wouldn't have received much usage.

   4. Our community is made up of mostly power users, and most of these are
   still on older versions (2.0, 2.1). There is little reason for these users
   to upgrade to newer versions, and little reason to use the new features
   (even if they were the ones developing them). This is actually great, that
   the power users have been adding functionality to Cassandra for new users,
   however we haven't really engaged with these users to go and verify this
   functionality, and we did a pretty half-arsed job of testing them
   ourselves. We essentially just rolled it out and waited for the bug reports.
   IMO this is where the "experimental flag" comes in. We rolled out a
   bunch of stuff, a year later some people started using

Re: rebuild constantly fails, 3.11

[kurt greaves]

cc'ing user back in...

On 12 Aug. 2017 01:55, "kurt greaves"  wrote:

> How much memory do these machines have?  Typically we've found that G1
> isn't worth it until you get to around 24G heaps, and even at that it's not
> really better than CMS. You could try CMS with an 8G heap and 2G new size.
>
> However as the oom is only happening on one node have you ensured there
> are no extra processes running on that node that could be consuming extra
> memory? Note that the oom killer will kill the process with the highest oom
> score, which generally corresponds to the process using the most memory,
> but not necessarily the problem.
>
> Also could you run nodetool info on the problem node and 1 other and dump
> the output in a gist? It would be interesting to see if there is a
> significant difference in off-heap.
>
> On 11 Aug. 2017 17:30, "Micha"  wrote:
>
>> It's an oom issue, the kernel kills the cassandra job.
>> The config was to use offheap buffers and 20G java heap, I changed this
>> to use heap buffers and 16G java heap. I added a  new node yesterday
>> which got streams from 4 other nodes. They all succeeded except on the
>> one node which failed before. This time again the db was killed by the
>> kernel. At the moment I don't know what is the reason here, since the
>> nodes are equal.
>>
>> For me it seems the g1gc is not able to free the memory fast enough.
>> The settings were for  MaxGCPauseMillis=600 and ParallelGCThreads=10
>> ConcGCThreads=10 which maybe are too high since the node has only 8
>> cores..
>> I changed this ParallelGCThreads=8 and ConcGCThreads=2 as is mentioned
>> in the comments of jvm.options
>>
>> Since the bootstrap of the fifth node did not complete I will start it
>> again and check if the memory is still decreasing over time.
>>
>>
>>
>>  Michael
>>
>>
>>
>> On 11.08.2017 01:25, Jeff Jirsa wrote:
>> >
>> >
>> > On 2017-08-08 01:00 (-0700), Micha  wrote:
>> >> Hi,
>> >>
>> >> it seems I'm not able to add add 3 node dc to a 3 node dc. After
>> >> starting the rebuild on a new node, nodetool netstats show it will
>> >> receive 1200 files from node-1 and 5000 from node-2. The stream from
>> >> node-1 completes but the stream from node-2 allways fails, after
>> sending
>> >> ca 4000 files.
>> >>
>> >> After restarting the rebuild it again starts to send the 5000 files.
>> >> The whole cluster is connected via one switch only , no firewall
>> >> between, the networks shows no errors.
>> >> The machines have 8 cores, 32GB RAM and two 1TB discs as raid0.
>> >> the logs show no errors. The size of the data is ca 1TB.
>> >
>> > Is there anything in `dmesg` ?  System logs? Nothing? Is node2 running?
>> Is node3 running?
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>

Re: rebuild constantly fails, 3.11

[kurt greaves]

How much memory do these machines have?  Typically we've found that G1
isn't worth it until you get to around 24G heaps, and even at that it's not
really better than CMS. You could try CMS with an 8G heap and 2G new size.

However as the oom is only happening on one node have you ensured there are
no extra processes running on that node that could be consuming extra
memory? Note that the oom killer will kill the process with the highest oom
score, which generally corresponds to the process using the most memory,
but not necessarily the problem.

Also could you run nodetool info on the problem node and 1 other and dump
the output in a gist? It would be interesting to see if there is a
significant difference in off-heap.

On 11 Aug. 2017 17:30, "Micha"  wrote:

> It's an oom issue, the kernel kills the cassandra job.
> The config was to use offheap buffers and 20G java heap, I changed this
> to use heap buffers and 16G java heap. I added a  new node yesterday
> which got streams from 4 other nodes. They all succeeded except on the
> one node which failed before. This time again the db was killed by the
> kernel. At the moment I don't know what is the reason here, since the
> nodes are equal.
>
> For me it seems the g1gc is not able to free the memory fast enough.
> The settings were for  MaxGCPauseMillis=600 and ParallelGCThreads=10
> ConcGCThreads=10 which maybe are too high since the node has only 8 cores..
> I changed this ParallelGCThreads=8 and ConcGCThreads=2 as is mentioned
> in the comments of jvm.options
>
> Since the bootstrap of the fifth node did not complete I will start it
> again and check if the memory is still decreasing over time.
>
>
>
>  Michael
>
>
>
> On 11.08.2017 01:25, Jeff Jirsa wrote:
> >
> >
> > On 2017-08-08 01:00 (-0700), Micha  wrote:
> >> Hi,
> >>
> >> it seems I'm not able to add add 3 node dc to a 3 node dc. After
> >> starting the rebuild on a new node, nodetool netstats show it will
> >> receive 1200 files from node-1 and 5000 from node-2. The stream from
> >> node-1 completes but the stream from node-2 allways fails, after sending
> >> ca 4000 files.
> >>
> >> After restarting the rebuild it again starts to send the 5000 files.
> >> The whole cluster is connected via one switch only , no firewall
> >> between, the networks shows no errors.
> >> The machines have 8 cores, 32GB RAM and two 1TB discs as raid0.
> >> the logs show no errors. The size of the data is ca 1TB.
> >
> > Is there anything in `dmesg` ?  System logs? Nothing? Is node2 running?
> Is node3 running?
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: jira rights

[kurt greaves]

I ran into this a few weeks ago. Basically contributors can assign any
Cassandra ticket to anyone with an account, however non-contributors can't
do anything. So you still have to get a contributor to assign you a ticket
if you are not one yourself (or get added to contributors).

Re: State of Materialized Views

[kurt greaves]

I'm going to do my best to review all the changes Zhao is making under
CASSANDRA-11500 ,
but yeah definitely need a committer nominee as well. On that note, Zhao is
going to try address a lot of the current issues I listed above in #11500.​
Thanks Zhao!

RE: CHANGES.txt

[kurt greaves]

I agree that all patches should be added to changes.txt, just to rule out
any ambiguities. When people look at Changes.txt it's usually to find
something specific, not to browse the list of changes. Anything significant
should make it into news.txt, which is more appropriate for users.
changes.txt is more aimed at developers in my opinion.

On that note, messages like "Remove unused method" should be more specific,
that's just a bad commit message in general, and doesn't give at context.

Re: State of Materialized Views

[kurt greaves]

Thanks for the input Benjamin. Sounds like you've come to a lot of the same
conclusions I have. I'm certainly keen on fixing up MV's and I don't really
see a way we could avoid it, as I know they are already widely being used
in production. I think we would have had a much easier time if we went with
a basic implementation (append only) first, but y'know, hindsight.
Unfortunately I'd say we're kind of stuck with fixing what we've got or
have a really angry userbase that jumps ship.

*What I miss is a central consensus about "MV business rules" + a central
> set of proofs and or tests that support these rules and proof or falsify
> assumptions in a reproducible way.*
>
>From what I gathered from JIRA the goals in my original post are the ones
outlined during initial development of MV's. The general design and goals
were also documented here
,
however doesn't completely cover the current state of MV's.
I'm with you that we certainly need a set of proofs/tests to support these
rules. At the moment a lot of the open tickets have patches that contribute
good tests that cover many cases however we're almost kind of defining
rules as we go (granted it is difficult when we need to test every possible
write you could make in Cassandra).

In regards to your "tickler", a colleague has been working on something
similar however we haven't deemed it quite production ready yet so we
haven't released it to the public. It may be useful to compare notes if
you're interested!

​

State of Materialized Views

[kurt greaves]

wall of text inc.
*tl;dr: *Aiming to come to some conclusions about what we are doing with
MV's and how we are going to make them stable in production. But really
just trying to raise awareness/involvement for MV's.

It seems we've got an excess of MV bugs that pretty much make them
completely unusable in production, or at least incredibly risky and also
limited. It also appears that we don't have many people totally across MV's
either (or at least a lack of people currently looking at them). To avoid
us "forgetting" about MV's I'd like to raise the current issues and get
opinions on the direction we should go with MV's. I know historically there
was a lot of discussion about this, but it seems a lot of the originally
involved are currently less involved, and thus before making wild changes
to MV's it might be worth going back to the start and think through the
original requirements and implementation.

Probably worth summarising the original goals of MV's:

   - Maintain eventual consistency between base table and view tables
   - Provide mechanisms to repair consistency between base and views
   - Aim to keep convergence between base and view fast without sacrificing
   availability (low MTTR)
   Goals that weren't explicitly mentioned but more or less implied:
   - Performance must be at least good enough to justify using them over
   rolling-your-own. (we haven't really tried to measure this yet - only
   measured in comparison to not-a-MV)
   - Allow a user to redefine their partitioning key

And also a quick summary of *some *of the limitations in our implementation
(there are more, but majority of our current problems revolve around these):

   1. Primary key of the base table must be included in the view,
   optionally one non-primary key column can be included in the view primary
   key.
   2. All columns in the view primary key must be declared NOT NULL.
   3. Base tables and views are one-to-one. That is, a *primary key* in a
   base maps to exactly one *primary key *in the view. Therefore you should
   never expect multiple rows in the view for a partition with multiple rows
   in the base.


I've summarised the bulk of the outstanding bugs below (may have missed
some), but notably it would be useful to get some decision-making happening
on them. Fixing these bugs is a bit more involved and there is likely a few
possible solutions and implications. Also they all pretty much touch the
same parts of the code, so needs to be some collaboration across the
patches (part of the reason I'm trying to bring more attention to them).

CASSANDRA-13657  -
Using a non-PK column in the view PK means that you can TTL that column in
the base without TTLing the resulting view row. Potential solution is to
change the definition of liveness info for view rows. This would probably
work but makes moving away from the NOT NULL requirement on view PK's
harder. Need to decide if that's what we want to do or if we pursue a
different solution.

CASSANDRA-13127  -
Inserting with key with a TTL then updating the TTL on a column from the
base that doesn't exist in the view doesn't update the liveness of the row
in the MV, and thus the MV row expires before the base. The current
proposed solution should work but will increase the amount of cases where
we need to read the existing data. Needs some reviewing and wouldn't hurt
to benchmark the changes.

CASSANDRA-13547  -
Being able to leave a column out of your SELECT but including it in the
view filters causes some serious issues. Proposed fix is to force user to
select all columns also included in where clause. This will potentially be
a compatibility issue but *should *be fine as it only is checked on MV
creation - so people upgrading shouldn't be affected (needs reviewing).
Also another issue is addressed in the patch regarding timestamps - choice
of timestamps led to rows not being deleted in the view. This comes back to
the fact that we allow a non-PK column in the view PK. Needs more reviewing.
Also related somewhat to 11500.

CASSANDRA-13409  -
Issues with shadowable tombstones. Has a patch but not sure if resolved
based on Zhao's last comment. Another case of bringing data back in the
view and thus making base and view inconsistent. Needs reviewing.

CASSANDRA-11500 
CASSANDRA-10965  -
Both these appear to be instances of the same issue. Got a couple of
potential solutions. Back to that problem of shadowable tombstones and
timestamps. Pretty involved and would require an in depth review as
decisions could greatly impact the complexity/usefulness of MV's.

CASSANDRA-13069  -
Node movements c

Re: [DISCUSS] Should we do a 2.2 release?

[kurt greaves]

+1 what Jeff said.

Re: Potential block issue for 3.0.13: schema version id mismatch while upgrading

[kurt greaves]

On 31 May 2017 at 03:41, Jeremiah D Jordan 
wrote:

> If 3.0.13 causes schema mismatch on upgrade, then maybe we should pull
> that and release 3.0.14 once 13559 is fixed.  As that is a pretty bad place
> to get into.


i think that would be a good idea

Re: Why does CockroachDB github website say Cassandra has no Availability on datacenter failure?

[kurt greaves]

Marketing never lies. Ever

Re: Proposals for releases - 4.0 and beyond

[kurt Greaves]

yep +1 to this. The LTS release solves my previous concern