I’d prefer to time box it as well. I like Sylvain’s suggestion, although I’d also be comfortable with setting a more aggressive cutoff date for features (maybe a month), given all the stuff that’s already in.
If we plan on a follow up (4.1/5.0) in 6 months I *hope* there will be less of a desire to do a bunch of last minute feature merges, maybe I’m too optimistic. Jon > On Apr 3, 2018, at 9:48 AM, Ben Bromhead <b...@instaclustr.com> wrote: > > +1 > > Even though I suggested clearing blockers, I'm equally happy with a > time-boxed event to draw the line in the sand. As long as its something > clear to work towards with appropriate commitment from folk. > > On Tue, Apr 3, 2018 at 8:10 AM Sylvain Lebresne <slebre...@apache.org > <mailto:slebre...@apache.org>> > wrote: > >> For what it's worth (and based on the project experience), I think the >> strategy >> of "let's agree on a list of tickets everyone would love to get in before >> we >> freeze 4.0" doesn't work very well (it's largely useless, expect for making >> us >> feel good about not releasing anything). Those lists always end up being >> too >> big especially given we have no control on people's ability to contribute >> (some stuffs will always lag for a very long time, even when they sound >> really cool on paper). >> >> I'm also a bit sad that we seem to be getting back to our old demons of >> trying >> to shove as much as we possibly can in the next major as if having a >> feature >> miss it means it will never happen. The 4.0 changelog is big already and we >> haven't made a release with new features in almost a year now, so I >> personally >> think we should start being a bit more aggressive with it and learn to get >> comfortable letting feature slip if they are not ready. >> >> My concrete proposal would be to declare a feature freeze for 4.0 in 2 >> months, >> so say June 1th. That leave some time for finishing features that are in >> progress, but not too much to get derailed. And let's be strict on that >> freeze. >> After that, we'll see how quickly we can get stuffs to stabilize but I'd >> suggest aiming for an alpha 3-4 weeks after that. >> >> Of course, we should probably (re-(re-(re-)))start a discussion on release >> "strategy" in parallel because it doesn't seem we have one right now, but >> that's imo a discussion we should keep separate. >> >> -- >> Sylvain >> >> >> On Mon, Apr 2, 2018 at 4:54 PM DuyHai Doan <doanduy...@gmail.com> wrote: >> >>> My wish list: >>> >>> * Add support for arithmetic operators (CASSANDRA-11935) >>> * Allow IN restrictions on column families with collections >>> (CASSANDRA-12654) >>> * Add support for + and - operations on dates (CASSANDRA-11936) >>> * Add the currentTimestamp, currentDate, currentTime and currentTimeUUID >>> functions (CASSANDRA-13132) >>> * Allow selecting Map values and Set elements (CASSANDRA-7396) >>> >>> Those are mostly useful for timeseries data models and I guess has no >>> significant impact on the internals and operations so the risk of >>> regression is low >>> >>> On Mon, Apr 2, 2018 at 4:33 PM, Jeff Jirsa <jji...@gmail.com> wrote: >>> >>>> 9608 (java9) >>>> >>>> -- >>>> Jeff Jirsa >>>> >>>> >>>>> On Apr 2, 2018, at 3:45 AM, Jason Brown <jasedbr...@gmail.com> >> wrote: >>>>> >>>>> The only additional tickets I'd like to mention are: >>>>> >>>>> https://issues.apache.org/jira/browse/CASSANDRA-13971 - Automatic >>>>> certificate management using Vault >>>>> - Stefan's Vault integration work. A sub-ticket, CASSANDRA-14102, >>>> addresses >>>>> encryption at-rest, subsumes CASSANDRA-9633 (SSTable encryption) - >>> which >>>> I >>>>> doubt I would be able to get to any time this year. It would >> definitely >>>> be >>>>> nice to have a clarified encryption/security story for 4.0. >>>>> >>>>> https://issues.apache.org/jira/browse/CASSANDRA-11990 - Address rows >>>> rather >>>>> than partitions in SASI >>>>> - a nice update for SASI, but not critical. >>>>> >>>>> -Jason >>>>> >>>>>> On Sat, Mar 31, 2018 at 6:53 PM, Ben Bromhead <b...@instaclustr.com> >>>> wrote: >>>>>> >>>>>> Apologies all, I didn't realize I was responding to this discussion >>>> only on >>>>>> the @user list. One of the perils of responding to a thread that is >> on >>>> both >>>>>> user and dev... >>>>>> >>>>>> For context, I have included my response to Kurt's previous >> discussion >>>> on >>>>>> this topic as it only ended up on the user list. >>>>>> >>>>>> *After some further discussions with folks offline, I'd like to >> revive >>>> this >>>>>> discussion. * >>>>>> >>>>>> *As Kurt mentioned, to keep it simple I if we can simply build >>> consensus >>>>>> around what is in for 4.0 and what is out. We can then start the >>>> process of >>>>>> working off a 4.0 branch towards betas and release candidates. Again >>> as >>>>>> Kurt mentioned, assigning a timeline to it right now is difficult, >> but >>>>>> having a firm line in the sand around what features/patches are in, >>> then >>>>>> limiting future 4.0 work to bug fixes will give folks a less >> nebulous >>>>>> target to work on. * >>>>>> >>>>>> *The other thing to mention is that once we have a 4.0 branch to >> work >>>> off, >>>>>> we at Instaclustr have a commitment to dogfooding the release >>>> candidates on >>>>>> our internal staging and internal production workloads before 4.0 >>>> becomes >>>>>> generally available. I know other folks have similar commitments and >>>> simply >>>>>> having a 4.0 branch with a clear list of things that are in or out >>> will >>>>>> allow everyone to start testing and driving towards a quality >>> release. * >>>>>> >>>>>> *The other thing is that there are already a large number of changes >>>> ready >>>>>> for 4.0, I would suggest not recommending tickets for 4.0 that have >>> not >>>> yet >>>>>> been finished/have outstanding work unless you are the person >> working >>>> on it >>>>>> (or are offering to work on it instead) and can get it ready for >>> review >>>> in >>>>>> a timely fashion. That way we can build a more realistic working >>> target. >>>>>> For other major breaking changes, there is always 5.0 or 4.1 or >>>> whatever we >>>>>> end up doing :)* >>>>>> >>>>>> Thinking further about it, I would suggest a similar process that >> was >>>>>> applied to releasing 3.0, in order to get to 4.0: >>>>>> >>>>>> - Clean up ticket labeling. Move tickets unlikely to make it / be >>>> worked >>>>>> on for 4.0 to something else (e.g. 4.x or whatever). >>>>>> - Tickets labeled 4.0 will be the line in the sand, with some >>> trigger >>>>>> ("done") event where all features not done by a certain event will >>>>>> simply >>>>>> move into the next release. For the 3.0 branch, this occurred >> after >>> a >>>>>> large review of 8099. For 4.0 it could simply be resolving all >>> current >>>>>> blockers/major tickets tagged 4.0... doesn't have to be / nor is >> it >>>>>> something I would strongly advocate. >>>>>> - Once we hit this "done" event. Cut a Cassandra-4.0 branch and >>> start >>>>>> the alpha/beta/rc cycle from that branch, with only bugfixes going >>>> into >>>>>> it >>>>>> - This, in my mind, is similar to the 3.0 approach >>>>>> https://mail-archives.apache.org/mod_mbox/cassandra-dev/ >>>>>> 201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_ >>>>>> 4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E, >>>>>> but without the subsequent tick-tock :) >>>>>> >>>>>> There are currently 3 open blockers tagged 4.0, some are old and >>>> probably >>>>>> not really blockers anymore, there are other tickets that may/should >>> be >>>>>> blockers on 4.0: >>>>>> >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-13951 >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-13994 >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-12042 >>>>>> >>>>>> In terms of major tickets that I would like to see land: >>>>>> >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual >>> Tables >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode >>>> netty >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable >>>>>> Storage >>>>>> - https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable >>>>>> encryption >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa <jji...@gmail.com> >>> wrote: >>>>>>> >>>>>>> >>>>>>> Advantages of cutting a release sooner than later: >>>>>>> 1) The project needs to constantly progress forward. Releases are >> the >>>>>> most >>>>>>> visible part of that. >>>>>>> 2) Having a huge changelog in a release increases the likelihood of >>>> bugs >>>>>>> that take time to find. >>>>>>> >>>>>>> Advantages of a slower release: >>>>>>> 1) We don't do major versions often, and when we do breaking >> changes >>>>>>> (protocol, file format, etc), we should squeeze in as many as >>> possible >>>> to >>>>>>> avoid having to roll new majors >>>>>>> 2) There are probably few people actually running 3.11 at scale, so >>>>>>> probably few people actually testing trunk. >>>>>>> >>>>>>> In terms of "big" changes I'd like to see land, the ones that come >> to >>>>>> mind >>>>>>> are: >>>>>>> >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch" >>>> (changes >>>>>>> file format) >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient >>>>>>> Replicas (probably adds new replication strategy or similar) >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of >> the >>>>>>> internode netty stuff (no idea if this changes internode stuff, >> but I >>>> bet >>>>>>> it's a lot easier if it lands on a major) >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual >>> Tables >>>>>>> (selfish inclusion, probably doesn't need to be a major at all, >> and I >>>>>>> wouldn't even lose sleep if it slips, but I'd like to see it land) >>>>>>> >>>>>>> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to >> land >>>> on a >>>>>>> major because we'll change something big (like gossip, or the way >>>> schema >>>>>> is >>>>>>> passed, etc): >>>>>>> >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly >>>>>>> consistent membership >>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly >>>>>>> consistent schema >>>>>>> >>>>>>> All that said, what I really care about is building confidence in >> the >>>>>>> release, which means an extended testing cycle. If all of those >>> patches >>>>>>> landed tomorrow, I'd still expect us to be months away from a >>> release, >>>>>>> because we need to bake the next major - there's too many changes >> to >>>>>> throw >>>>>>> out an alpha/beta/rc and hope someone actually runs it. >>>>>>> >>>>>>> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). >>>> It's >>>>>>> possible Q3/Q4 alpha/beta is realistic, but definitely not a >> release. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves < >> k...@instaclustr.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi friends, >>>>>>>> *TL;DR: Making a plan for 4.0, ideally everyone interested should >>>>>> provide >>>>>>>> up to two lists, one for tickets they can contribute resources to >>>>>> getting >>>>>>>> finished, and one for features they think would be desirable for >>> 4.0, >>>>>> but >>>>>>>> not necessarily have the resources to commit to helping with.* >>>>>>>> >>>>>>>> So we had that Roadmap for 4.0 discussion last year, but there was >>>> never >>>>>>>> a conclusion or a plan that came from it. Times getting on and the >>>>>> changes >>>>>>>> list for 4.0 is getting pretty big. I'm thinking it would probably >>>> make >>>>>>>> sense to define some goals to getting 4.0 released/have an actual >>>> plan. >>>>>> 4.0 >>>>>>>> is already going to be quite an unwieldy release with a lot of >>> testing >>>>>>>> required. >>>>>>>> >>>>>>>> Note: the following is open to discussion, if people don't like >> the >>>> plan >>>>>>>> feel free to speak up. But in the end it's a pretty basic plan >> and I >>>>>> don't >>>>>>>> think we should over-complicate it, I also don't want to end up >> in a >>>>>>>> discussion where we "make a plan to make a plan". Regardless of >>>> whatever >>>>>>>> plan we do end up following it would still be valuable to have a >>> list >>>> of >>>>>>>> tickets for 4.0 which is the overall goal of this email - so let's >>> not >>>>>> get >>>>>>>> too worked up on the details just yet (save that for after I >>>>>>>> summarise/follow up). >>>>>>>> >>>>>>>> // TODO >>>>>>>> I think the best way to go about this would be for us to come up >>> with >>>> a >>>>>>>> list of JIRA's that we want included in 4.0, tag these as 4.0, and >>> all >>>>>>>> other improvements as 4.x. We can then aim to release 4.0 once all >>> the >>>>>> 4.0 >>>>>>>> tagged tickets (+bug fixes/blockers) are complete. >>>>>>>> >>>>>>>> Now, the catch is that we obviously don't want to include too many >>>>>>>> tickets in 4.0, but at the same time we want to make sure 4.0 has >> an >>>>>>>> appealing feature set for both users/operators/developers. To >>> minimise >>>>>>>> scope creep I think the following strategy will help: >>>>>>>> >>>>>>>> We should maintain two lists: >>>>>>>> >>>>>>>> 1. JIRA's that people want in 4.0 and can commit resources to >>>> getting >>>>>>>> them implemented in 4.0. >>>>>>>> 2. JIRA's that people simply think would be desirable for 4.0, >> but >>>>>>>> currently don't have anyone assigned to them or planned >>> assignment. >>>>>> It >>>>>>>> would probably make sense to label these with an additional tag >> in >>>>>> JIRA. *(User's >>>>>>>> please feel free to point out what you want here)* >>>>>>>> >>>>>>>> From list 1 will come our source of truth for when we release 4.0. >>>>>> (after >>>>>>>> aggregating a list I will summarise and we can vote on it). >>>>>>>> >>>>>>>> List 2 would be the "hopeful" list, where stories can be picked up >>>> from >>>>>>>> if resourcing allows, or where someone comes along and decides >> it's >>>> good >>>>>>>> enough to work on. I guess we can also base this on a vote system >> if >>>> we >>>>>>>> reach the point of including some of them. (but for the moment >> it's >>>>>> purely >>>>>>>> to get an idea of what users actually want). >>>>>>>> >>>>>>>> Please don't refrain from listing something that's already been >>>>>>>> mentioned. The purpose is to get an idea of everyone's >>>>>> priorities/interests >>>>>>>> and the resources available. We will need multiple resources for >>> each >>>>>>>> ticket, so anywhere we share an interest will make for a lot >> easier >>>> work >>>>>>>> sharing. >>>>>>>> >>>>>>>> Note that we are only talking about improvements here. Bugs will >> be >>>>>>>> treated the same as always, and major issues/regressions we'll >> need >>> to >>>>>> fix >>>>>>>> prior to 4.0 anyway. >>>>>>>> >>>>>>>> TIME FRAME >>>>>>>> Generally I think it's a bad idea to commit to any hard deadline, >>> but >>>> we >>>>>>>> should have some time frames in mind. My idea would be to aim for >> a >>>> Q3/4 >>>>>>>> 2018 release, and as we go we just review the outstanding >>> improvements >>>>>> and >>>>>>>> decide whether it's worth pushing it back or if we've got enough >> to >>>>>>>> release. I suppose keep this time frame in mind when choosing your >>>>>> tickets. >>>>>>>> >>>>>>>> We can aim for an earlier date (midyear?) but I figure the >>>>>>>> testing/validation/bugfixing period prior to release might drag >> on a >>>>>> bit so >>>>>>>> being a bit conservative here. >>>>>>>> The main goal would be to not let list 1 grow unless we're well >>> ahead, >>>>>>>> and only cull from it if we're heavily over-committed or we decide >>> the >>>>>>>> improvement can wait. I assume this all sounds like common sense >> but >>>>>>>> figured it's better to spell it out now. >>>>>>>> >>>>>>>> >>>>>>>> NEXT STEPS >>>>>>>> After 2 weeks/whenever the discussion dies off I'll consolidate >> all >>>> the >>>>>>>> tickets, relevant comments and follow up with a summary, where we >>> can >>>>>>>> discuss/nitpick issues and come up with a final list to go ahead >>> with. >>>>>>>> >>>>>>>> On a side note, in conjunction with this effort we'll obviously >> have >>>> to >>>>>>>> do something about validation and testing. I'll keep that out of >>> this >>>>>> email >>>>>>>> for now, but there will be a follow up so that those of us willing >>> to >>>>>> help >>>>>>>> validate/test trunk can avoid duplicating effort. >>>>>>>> >>>>>>>> REVIEW >>>>>>>> This is the list of "huge/breaking" tickets that got mentioned in >>> the >>>>>>>> last roadmap discussion and their statuses. This is not terribly >>>>>> important >>>>>>>> but just so we can keep in mind what we previously talked about. I >>>>>> think we >>>>>>>> leave it up to the relevant contributors to decide whether they >> want >>>> to >>>>>> get >>>>>>>> the still open tickets into 4.0. >>>>>>>> >>>>>>>> CASSANDRA-9425 Immutable node-local schema >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - >> Committed >>>>>>>> CASSANDRA-10699 Strongly consistent schema alterations >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, >> no >>>>>>>> discussion in quite some time. >>>>>>>> CASSANDRA-12229 NIO streaming >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - >> Committed >>>>>>>> CASSANDRA-8457 NIO messaging >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - >> Committed >>>>>>>> CASSANDRA-12345 Gossip 2.0 >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, >> no >>>> sign >>>>>>>> of any action. >>>>>>>> CASSANDRA-9754 Make index info heap friendly for large CQL >>> partitions >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In >>> progress >>>>>> but >>>>>>>> no update in a long time. >>>>>>>> CASSANDRA-11559 enhanced node representation >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, >> no >>>>>>>> change since early 2016. >>>>>>>> CASSANDRA-6246 epaxos >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In >>> progress >>>>>> but >>>>>>>> no update since Feb 2017. >>>>>>>> CASSANDRA-7544 storage port configurable per node >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - >> Committed >>>>>>>> CASSANDRA-11115 remove thrift support >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - >> Committed >>>>>>>> CASSANDRA-10857 dropping compact storage >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - >> Committed >>>>>>>> >>>>>>>> To start us off... >>>>>>>> And here are my lists to get us started. >>>>>>>> 1. >>>>>>>> CASSANDRA-8460 - Tiered/Cold storage for TWCS >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-8460> >>>>>>>> CASSANDRA-12783 - Batchlog redesign >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-12783> >>>>>>>> CASSANDRA-11559 - Enchance node representation >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> >>>>>>>> CASSANDRA-12344 - Forward writes to replacement node with same >>>>>>>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344> >>>>>>>> CASSANDRA-8119 - More expressive Consistency Levels >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-8119> >>>>>>>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-14210> >>>>>>>> CASSANDRA-10540 - RangeAwareCompaction >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-10540> >>>>>>>> >>>>>>>> >>>>>>>> 2: >>>>>>>> CASSANDRA-10726 - Read repair inserts should not be blocking >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-10726> >>>>>>>> CASSANDRA-9754 - Make index info heap friendly for large CQL >>>> partitions >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> >>>>>>>> CASSANDRA-12294 - LDAP auth >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-12294> >>>>>>>> CASSANDRA-12151 - Audit logging >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-12151> >>>>>>>> CASSANDRA-10495 - Fix streaming with vnodes >>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-10495> >>>>>>>> >>>>>>>> Also, here's some handy JQL to start you off: >>>>>>>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in >>>>>>>> watchedIssues() AND status != Resolved >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>> Ben Bromhead >>>>>> CTO | Instaclustr <https://www.instaclustr.com/> >>>>>> +1 650 284 9692 <(650)%20284-9692> >>>>>> Reliability at Scale >>>>>> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer >>>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >>>> <mailto:dev-unsubscr...@cassandra.apache.org> >>>> For additional commands, e-mail: dev-h...@cassandra.apache.org >>>> <mailto:dev-h...@cassandra.apache.org> >>>> >>>> >>> >> > -- > Ben Bromhead > CTO | Instaclustr <https://www.instaclustr.com/ > <https://www.instaclustr.com/>> > +1 650 284 9692 > Reliability at Scale > Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
