Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

+1 to everything Blake said. Bonus points for property-based state testing (a la ScalaCheck/QuickCheck). This being said, are we drifting from the original topic of this thread? Should we decide features for 4.0? It seems to me testing concerns might be for another thread? Is it worthwhile voting

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

> I'm not sure how the apache team does this. Perhaps individual engineers  can run some modern version at a company of theirs, altho that seems  unlikely, but as an Apache org, i just don't see how that happens.  > To me it seems like the Apache Cassandra infrastructure itself needs to  stand up

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

This was not for the Dev list :) > On Nov 20, 2016, at 09:06, Sankalp Kohli wrote: > > I have asked him to calm down as these things are never constructive for the > community. Making personal comments put him in bad light more than anytime > else. > I will speak with

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

I have asked him to calm down as these things are never constructive for the community. Making personal comments put him in bad light more than anytime else. I will speak with him in person when we are in office. Thanks for keeping an eye on these things for us. I will setup another meeting

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

>> We fully intend to "engineer and test the snot out of" the changes we are working on as the whole point of us working on them is so we *can* run them in production, at our scale. I'm not sure how the apache team does this. Perhaps individual engineers can run some modern version at a

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Hey all, One of the goals on my team, when working on large patches, is to get community feedback on these initiatives before throwing them into prod. This gets us a wider net of feedback (see Sylvain's continuing excellent rounds of feedback to my work on CASSANDRA-8457), as well as making sure

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Jason has asked for review and feedback many times. Maybe be constructive and review his code instead of just complaining (once again)? Sent from my iPhone > On Nov 19, 2016, at 1:49 PM, Edward Capriolo wrote: > > I would say start with a mindset like 'people will run

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

I would say start with a mindset like 'people will run this in production' not like 'why would you expect this to work'. Now how does this logic effect feature develement? Maybe use gossip 2.0 as an example. I will play my given debby downer role. I could imagine 1 or 2 dtests and the logic of

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Any proposal to solve the problem you describe? -- Jeff Jirsa > On Nov 19, 2016, at 8:50 AM, Edward Capriolo wrote: > > This is especially relevant if people wish to focus on removing things. > > For example, gossip 2.0 sounds great, but seems geared toward huge

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Honest question: are you *ever* positive Ed? Maybe give it a shot once in a while. It will be good for your mental health. Sent from my iPhone > On Nov 19, 2016, at 11:50 AM, Edward Capriolo wrote: > > This is especially relevant if people wish to focus on removing

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

This is especially relevant if people wish to focus on removing things. For example, gossip 2.0 sounds great, but seems geared toward huge clusters which is not likely a majority of users. For those with a 20 node cluster are the indirect benefits woth it? Also there seems to be a first push to

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

On Friday, November 18, 2016, Jeff Jirsa wrote: > We should assume that we’re ditching tick/tock. I’ll post a thread on > 4.0-and-beyond here in a few minutes. > > The advantage of a prod release every 6 months is fewer incentive to push > unfinished work into a

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

We should assume that we’re ditching tick/tock. I’ll post a thread on 4.0-and-beyond here in a few minutes. The advantage of a prod release every 6 months is fewer incentive to push unfinished work into a release. The disadvantage of a prod release every 6 months is then we either have a very

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

> While stability is important if we push back large "core" changes until later > we're just setting ourselves up to face the same issues later on In theory, yes. In practice, when incomplete features are earmarked for a certain release, those features are often rushed out, and not always fully

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

On 18 November 2016 at 18:25, Jason Brown wrote: > #11559 (enhanced node representation) - decided it's *not* something we > need wrt #7544 storage port configurable per node, so we are punting on > #12344 - Forward writes to replacement node with same address during

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Introducing all of these in a single release seems pretty risky. I think it would be safer to spread these out over a few 4.x releases (as they’re finished) and give them time to stabilize before including them in an LTS release. The downside would be having to maintain backwards compatibility

Re: Rough roadmap for 4.0

It's not marked fixed, it's marked resolved and the resolution is duplicate. This is how all dupes are marked in jira. On Fri, Nov 18, 2016 at 2:30 PM, Edward Capriolo wrote: > These tickets claim to duplicate each other: > >

Re: Rough roadmap for 4.0

These tickets claim to duplicate each other: https://issues.apache.org/jira/browse/CASSANDRA-12674 https://issues.apache.org/jira/browse/CASSANDRA-12746 But one is marked fixed and the other is still open. What is the status here? On Thu, Nov 17, 2016 at 5:20 PM, DuyHai Doan

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

Hi Nate, Most of the JIRAs in the middle are being rebased or being reviewed and code is already out there. These will make 4.0 a very solid release. Thanks, Sankalp On Thu, Nov 17, 2016 at 5:10 PM, Ben Bromhead wrote: > We are happy to start testing against

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

We are happy to start testing against completed features. Ideally once everything is ready for an RC (to catch interaction bugs), but we can do sooner for features where it make sense and are finished earlier. On Thu, 17 Nov 2016 at 16:47 Nate McCall wrote: > To sum up that

Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

To sum up that other thread (I very much appreciate everyone's input, btw), here is an aggregate list of large, breaking 4.0 proposed changes: CASSANDRA-9425 Immutable node-local schema CASSANDRA-10699 Strongly consistent schema alterations -- CASSANDRA-12229 NIO streaming CASSANDRA-8457 NIO

Re: Rough roadmap for 4.0

s/materialised views/aggregates/ also we expect to have our first larger production 3.7 LTS cluster in the next few months. On Thu, 17 Nov 2016 at 15:38 Ben Bromhead wrote: > We have a few small customers clusters running on our 3.7 LTS release... > though we are not

Re: Rough roadmap for 4.0

We have a few small customers clusters running on our 3.7 LTS release... though we are not calling it production ready yet. We also just moved our internal metrics cluster from 2.2 to 3.7 LTS to get materialised views and to get some 3.x production experience. On Thu, 17 Nov 2016 at 14:27 Carlos

Re: Rough roadmap for 4.0

No Cluster in tick-tock. Actually reverted a couple to 3.0.x Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: *linkedin.com/in/carlosjuzarterolo

Re: Rough roadmap for 4.0

Be very careful, there is a serious bug about AND/OR semantics, not solved yet and not going to be solved any soon: https://issues.apache.org/jira/browse/CASSANDRA-12674 On Thu, Nov 17, 2016 at 7:32 PM, Jeff Jirsa wrote: > > We’ll be voting in the very near future on

Re: Rough roadmap for 4.0

We should continue with 3.X until all the 4.0 blockers have been >> committed - and there are quite a few of them remaining yet. > > And… are we right to presume that all the "roadmap 4.0" issues that don't break any compatibility will be released in the 3.X tock releases leading up to 4.0?

Re: Rough roadmap for 4.0

@Jeff "But since you asked, I have ONE tick/tock (3.9) cluster being qualified for production because it needs SASI." You are brave :) On Thu, Nov 17, 2016 at 10:32 AM, Jeff Jirsa wrote: > > We’ll be voting in the very near future on timing of major releases and >

Re: Rough roadmap for 4.0

We’ll be voting in the very near future on timing of major releases and release strategy. 4.0 won’t happen until that vote takes place. But since you asked, I have ONE tick/tock (3.9) cluster being qualified for production because it needs SASI. - Jeff On 11/17/16, 9:59 AM, "Jonathan

Re: Rough roadmap for 4.0

I think it might be worth considering adopting the release strategy before 4.0 release. Are any PMC members putting tick tock in prod? Does anyone even trust it? What's the downside of changing the release cycle independently from 4.0? On Thu, Nov 17, 2016 at 9:03 AM Jason Brown

Re: Rough roadmap for 4.0

Jason, That's a separate topic, but we will have a different vote on how the branching/release strategy should be for the future. On Thursday, November 17, 2016, jason zhao yang wrote: > Hi, > > Will we still use tick-tock release for 4.x and 4.0.x ? > > Stefan

Re: Rough roadmap for 4.0

Hi, Will we still use tick-tock release for 4.x and 4.0.x ? Stefan Podkowinski 于2016年11月16日周三 下午4:52写道： > From my understanding, this will also effect EOL dates of other branches. > > "We will maintain the 2.2 stability series until 4.0 is released, and 3.0 > for six months

Re: Rough roadmap for 4.0

>From my understanding, this will also effect EOL dates of other branches. "We will maintain the 2.2 stability series until 4.0 is released, and 3.0 for six months after that.". On Wed, Nov 16, 2016 at 5:34 AM, Nate McCall wrote: > Agreed. As long as we have a goal I don't

Re: Rough roadmap for 4.0

Agreed. As long as we have a goal I don't see why we have to adhere to arbitrary date for 4.0. On Nov 16, 2016 1:45 PM, "Aleksey Yeschenko" wrote: > I’ll comment on the broader issue, but right now I want to elaborate on > 3.11/January/arbitrary cutoff date. > > Doesn’t

Re: Rough roadmap for 4.0

On 16 November 2016 at 11:45, Aleksey Yeschenko wrote: > > Doesn’t matter what the original plan was. We should continue with 3.X > until all the 4.0 blockers have been > committed - and there are quite a few of them remaining yet. Thanks for the clarity, the quick

Re: Rough roadmap for 4.0

I’ll comment on the broader issue, but right now I want to elaborate on 3.11/January/arbitrary cutoff date. Doesn’t matter what the original plan was. We should continue with 3.X until all the 4.0 blockers have been committed - and there are quite a few of them remaining yet. So given all the

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

On Sun, Nov 13, 2016 at 1:13 AM, Ben Slater wrote: > For anyone that’s interested, I’ve submitted my doc changes for point 2 > below (emphasising contributions other than new features) here: > https://issues.apache.org/jira/browse/CASSANDRA-12906 > > I haven’t added

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

For anyone that’s interested, I’ve submitted my doc changes for point 2 below (emphasising contributions other than new features) here: https://issues.apache.org/jira/browse/CASSANDRA-12906 I haven’t added anything about the sponsor/shepherd idea as doesn’t seem to be agreed at this point.

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

> > I strongly feel that there should be a better way e.g. a summary field in > JIRA which filters out the discussions, arguments, solutions etc.and just > crisply summarizes the problem, solution under discussion and the current > status. I've personally found that attaching a design doc for

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Thanks for the information Jeremy. My main concern is around making JIRAs easy to understand. I am not sure how community feels about it. But, I have personally observed that long discussion thread on JIRAs is not user friendly for someone trying to understand the ticket or may be trying to

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Regarding low hanging fruit, on the How To Contribute page [1] we’ve tried to keep a list of lhf tickets [2] linked to help people get started. They are usually good starting points and don’t require much context. I rarely see duplicates from lhf tickets. Regarding duplicates, in my

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Hi, We need to understand that time is precious for all of us. Even if a developer has intentions to contribute, he may take months to contribute his first patch or may be longer. Some common entry barriers are: 1. Difficult to identify low hanging fruits. 30 JIRA comments on a ticket and a

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

I like the idea of a goal-based approach. I think that would make coming to a consensus a bit easier particularly if a larger number of people are involved. On Tue, Nov 8, 2016 at 8:04 PM, Dikang Gu wrote: > My 2 cents. I'm wondering is it a good idea to have some high level

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

My 2 cents. I'm wondering is it a good idea to have some high level goals for the major release? For example, the goals could be something like: 1. Improve the scalability/reliability/performance by X%. 2. Add Y new features (feature A, B, C, D...). 3. Fix Z known issues (issue A, B, C, D...). I

Re: Rough roadmap for 4.0

Also https://issues.apache.org/jira/browse/CASSANDRA-12676 -- Jeff Jirsa > On Nov 4, 2016, at 11:51 AM, Jason Brown wrote: > > +1 to epaxos > >> On Fri, Nov 4, 2016 at 11:40 AM, Jonathan Haddad wrote: >> >> epaxos would be nice, it's been about

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Recently there was another discussion on documentation and comments [1] On one hand, documentation and comments will help newcomers to familiarise themselves with the codebase. On the other - one may get up to speed by reading the code and adding some docs. Such things may require less oversight

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Agreed. --  AY On 7 November 2016 at 16:38:07, Jeff Jirsa (jeff.ji...@crowdstrike.com) wrote: ‘Accepted’ JIRA status seems useful, but would encourage something more explicit like ‘Concept Accepted’ or similar to denote that the concept is agreed upon, but the actual patch itself may not be

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

‘Accepted’ JIRA status seems useful, but would encourage something more explicit like ‘Concept Accepted’ or similar to denote that the concept is agreed upon, but the actual patch itself may not be accepted yet. /bikeshed. On 11/7/16, 2:56 AM, "Ben Slater" wrote:

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Thanks Dave. The shepherd concept sounds a lot like I had in mind (and a better name). One other thing I noted from the Mesos process - they have an “Accepted” jira status that comes after open and means “at least one Mesos developer thought that the ideas proposed in the issue are worth pursuing

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Hi Ben, A few ideas to add to your suggestions [inline]: On 2016-11-06 13:51 (-0800), Ben Slater wrote: > Hi All, > > I thought I would add a couple of observations and suggestions as someone > who has both personally made my first contributions to the project in

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Ben, Thank you for providing two thoughtful, concrete recommendations. There is some good feedback in general on this thread, but I'm calling Ben's response out because point #1 is important to discuss and point #2 is immediately actionable. > 1) I think some process of assigning a committer of a

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Hi All, I thought I would add a couple of observations and suggestions as someone who has both personally made my first contributions to the project in the last few months and someone in a leadership role in an organisation (Instaclustr) that is feeling it’s way through increasing our

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

On 11/04/2016 06:43 PM, Jeff Beck wrote: > I run the local Cassandra User Group and I would love to help get the > community more involved. I would propose holding a night to add patches to > Cassandra some will be simple things like making sure some asserts have > proper messages with them etc,

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

On Sat, Nov 5, 2016 at 9:19 AM, Benedict Elliott Smith wrote: > Hi Ed, > > I would like to try and clear up what I perceive to be some > misunderstandings. > > Aleksey is relating that for *complex* tickets there are desperately few > people with the expertise necessary to

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Hi Tyler, There is a nice guide now in the docs on how to contribute[1]. If you try it and find holes you can also help by contributing to those docs. -Jake [1]: http://cassandra.apache.org/doc/latest/development/index.html On Sat, Nov 5, 2016 at 11:08 AM, Tyler Tolley

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Just want to weigh in my 2 cents. I've been following the dev list for quite a while and wanted to contribute. As I approached trying to handle some lhf, I couldn't find any instructions on how to check out, build, test or any guidance on coding standards and best practices. Maybe these existed

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Hi Ed, I would like to try and clear up what I perceive to be some misunderstandings. Aleksey is relating that for *complex* tickets there are desperately few people with the expertise necessary to review them. In some cases it can amount to several weeks' work, possibly requiring multiple

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

"I’m sure users running Cassandra in production would prefer actual proper reviews to non-review +1s." Again, you are implying that only you can do a proper job. Lets be specific here: You and I are working on this one: https://issues.apache.org/jira/browse/CASSANDRA-10825 Now, Ariel reported

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

I run the local Cassandra User Group and I would love to help get the community more involved. I would propose holding a night to add patches to Cassandra some will be simple things like making sure some asserts have proper messages with them etc, but some may be slightly larger. The goal being

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

Dunno. A sneaky correctness or data corruption bug. A performance regression. Or something that can take a node/cluster down. Of course no process is bullet-proof. The purpose of review is to minimise the odds of such a thing happening. I’m sure users running Cassandra in production would

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

"There is also the issue of specialisation. Very few people can be trusted with review of arbitrary Cassandra patches. I can count them all on fingers of one hand." I have to strongly disagree here. The Cassandra issue tracker is over 12000 tickets. I do not think that cassandra has added 12000

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

I’m sure that impactful, important, and/or potentially destabilising patches will continue getting reviewed by those engineers. As for patches that no organisation with a strong enough commercial interest cares about, they probably won’t. Engineering time is quite expensive, most employers are

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

This has always been a concern. We’ve always had a backlog on unreviewed patches. Reviews (real reviews, not rubber-stamping a +1 formally) are real work, often taking as much work as creating the patch in question. And taking as much expertise (or more). It’s also not ‘fun’ and doesn’t lend

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

To be clear, getting the community more involved is a super hard, critically important problem to which I don't have a concrete answer other than I'm going to keep reaching out for opinions, ideas and involvement. Just like this. Please speak up here if you have ideas on how this could work. On

Re: Rough roadmap for 4.0

Hi Stefan, Thank you very much for bringing this up. I moved this to a new thread because, though closely related, I think it is a very important discussion to have on it's own as it does have a big impact. Thanks, -Nate On Sat, Nov 5, 2016 at 9:15 AM, Stefan Podkowinski

Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

[Moved to a new thread because this topic is important by itself] There are some excellent points here - thanks for bringing this up. > What can inspiring developers contribute to 4.0 > that would move the project forward to it’s goals and would be very likely > included in the final release?

Re: Rough roadmap for 4.0

There has been a lot of discussions about diversity and getting new contributors and I think this aspect should be kept in mind as well when talking about a roadmap, additionally to the listed tickets that are already in the pipeline. What can inspiring developers contribute to 4.0 that would move

Re: Rough roadmap for 4.0

+1 to epaxos On Fri, Nov 4, 2016 at 11:40 AM, Jonathan Haddad wrote: > epaxos would be nice, it's been about 2 years since it was started, and > Blake asked for a first review well over a year ago. the benchmarks and > thorough test suite look like a huge improvement over

Re: Rough roadmap for 4.0

epaxos would be nice, it's been about 2 years since it was started, and Blake asked for a first review well over a year ago. the benchmarks and thorough test suite look like a huge improvement over the current situation. https://issues.apache.org/jira/browse/CASSANDRA-6246 On Fri, Nov 4, 2016

Re: Rough roadmap for 4.0

I would like to propose features around seeds: https://issues.apache.org/jira/browse/CASSANDRA-12627 I have other follow up issues like getting seeds from Amazon API, or from JNDI/ DNS, etc. I was hoping 12627 was an easy way to grease the wheels. On Fri, Nov 4, 2016 at 8:39 AM, Jason Brown

Re: Rough roadmap for 4.0

Hey Nate, I'd like to add CASSANDRA-11559 (Enhance node representation) to the list, including the comment I made on the ticket (different storage ports for each node). For us, it's a great "would really like to have" as our group will probably need this in production within the next 1 year or

Re: Rough roadmap for 4.0

List looks really good. I will let you know if there is something else we plan to add to this list. On Thu, Nov 3, 2016 at 7:47 PM, Nate McCall wrote: > It was brought up recently at the PMC level that our goals as a > project are not terribly clear. > > This is a pretty

Rough roadmap for 4.0

It was brought up recently at the PMC level that our goals as a project are not terribly clear. This is a pretty good point as outside of Jira 'Fix Version' labelling (which we actually suck less at compared to a lot of other ASF projects) this really isnt tracked anywhere outside of general