Our messages crossed, so I didn’t see the message where you said “versioning doesn’t need to be semantic”.
I did not say that Druid should do SemVer; I said that you should have a policy, and discuss how that policy differs from SemVer. Julian > On Dec 21, 2018, at 10:57 AM, Roman Leventov <leventov...@gmail.com> wrote: > > Julian, I mentioned semantic versioning in the previous message. The > comparison with IntelliJ may seem shocking at first, but actually Druid may > be semantically closer to IntelliJ than to Hadoop or Spark. Druid is a > sever-side *application*, not a library or framework. Like Druid has > extensions API, IntelliJ has plugin API, that is also very unstable and > broken in every IntelliJ release, as far as I know. > > Comparison with Guava doesn't make a lot of sense - exactly because of > that. Guava is a library, half of the Java world depends on it. Almost > nothing depends on Druid. > > On Fri, 21 Dec 2018 at 19:46, Julian Hyde <jh...@apache.org> wrote: > >> No one has mentioned Semantic Versioning (semver [1]) yet, so I will. >> I don't know whether the Druid developers think in terms of semver, >> but a lot of your audience will. In semver, the shift from 0. to 1. is >> a big event, because the "only remove APIs in major releases" rule >> does not apply for versions < 1.0. >> >> It would be good if Druid had a policy for how long APIs and will be >> around. It does not need to be based on semver, but if it isn't, it >> should explain how it is different than semver. >> >> It should also spell out the planned release cadence. It sounds like >> you're thinking of two major versions per year, which sounds great. >> But note that Guava did exactly that, and got flak from a lot of >> people because features would move from supported to deprecated to >> gone in just six months. >> >> Regarding combining release 1.0 with Druid's graduation. My gut says >> no. A lot of people mistakenly think that graduation is a sign of >> product maturity, whereas it's actually a sign of *community* >> maturity. You don't want to play into those misconceptions and make >> people think that Druid is less mature than it really is. (For the >> record, I think Druid's product and community are both very mature.) I >> think of 1.0 and graduation as two separate opportunities to generate >> some news. For 1.0 you can talk about how Druid is industry strength, >> has wide adoption at large scales; for graduation you can talk about >> community and what that brings to governance, and adapting to market >> needs. >> >> Also regarding that "1.0" thing. How about going straight to 14.0? >> >> Julian >> >> [1] https://semver.org/ >> On Fri, Dec 21, 2018 at 10:10 AM Charles Allen <cral...@apache.org> wrote: >>> >>> If I'm greedily honest, I don't want to maintain that many backport >>> channels. I'd rather have "If you want XYZ backport for version 14, then >>> you have to take the latest minor version for 14" and have a policy to >>> where someone can upgrade from 14.x --> 14.latest with (hopefully) no >>> config changes. >>> >>> >>> >>> >>> On Fri, Dec 21, 2018 at 9:03 AM David Glasser <glas...@apollographql.com >>> >>> wrote: >>> >>>> One nice advantage to moving out of 0.x is that it frees up a digit on >> the >>>> right side to more cleanly differentiate between "minor release (a >> random >>>> assortment of bug fixes, small features, etc)" and "patch release >>>> (literally the minimum delta to give you a security fix or high impact >> bug >>>> fix)". >>>> >>>> --dave >>>> >>>> On Fri, Dec 21, 2018 at 8:58 AM Gian Merlino <g...@apache.org> wrote: >>>> >>>>> I'm not too fussy around whether we do a 1.0 or simply drop the 0. >> and >>>> have >>>>> it be a 14.0 or 15.0 or 16.0 or wherever we are at the time we do >> it. I >>>>> also like the quarterly cadence of release-from-master we had before >> we >>>> got >>>>> blocked on the ASF transition, and would like to pick that back up >> again >>>>> (with the next branch cut from master at the end of January, since >> we did >>>>> the 0.13.0 branch cut in late October). >>>>> >>>>> Seems to me that good points of discussion are, what should we use >> as the >>>>> rule for incrementing the major version? Do we do what we've been >> doing >>>>> (incrementing whenever there's either an incompatible change in >> extension >>>>> APIs, or in query APIs, or when necessary to preserve the ability to >>>> always >>>>> be able to roll forward/back one major release). Or do we do >> something >>>> else >>>>> (Roman seems to be suggesting dropping extension APIs from >>>> consideration). >>>>> >>>>> And also, what does 1.0 or 14.0 or 15.0 or what-have-you mean to us? >> Is >>>> it >>>>> something that should be tied to ASF graduation? Completeness of >> vision? >>>>> Stability of APIs or operational characteristics? Something else? >> You are >>>>> right that it is sort of a marketing/mentality thing, so it's an >>>>> opportunity for us to declare that we feel Druid has reached some >>>>> milestone. My feeling at this time is probably ASF graduation or >>>>> completeness of vision (see my earlier mail for thoughts there) are >> the >>>>> ones that make most sense to me. >>>>> >>>>> On Fri, Dec 21, 2018 at 10:41 AM Charles Allen <cral...@apache.org> >>>> wrote: >>>>> >>>>>> Is there any feeling in the community that the logic behind the >>>> releases >>>>>> needs to change? >>>>>> >>>>>> If so then I think we should discuss what that release cadence >> needs to >>>>>> look like. >>>>>> >>>>>> If not then dropping the 0. prefix is a marketing / mental item. >> Kind >>>> of >>>>>> like the 3.x->4.x Linux kernel upgrade. If this is the case then >> would >>>> we >>>>>> even want to go with 1.x? I think Roman's proposal would work fine >> in >>>>> this >>>>>> case. Where we just call it Apache Druid 14 (or 15 or whatever it >> is >>>> when >>>>>> we get there) and just keep the same logic for when we release >> stuff, >>>>> which >>>>>> has been something like: >>>>>> >>>>>> For a X.Y release, going to a X.? release should be very straight >>>> forward >>>>>> for anyone running stock Druid. >>>>>> For a X.Y release, going to a (X+1).? or from a (X+1).? back to an >> X.Y >>>>>> release should be feasible. It might require running a tool >> supported >>>> by >>>>>> the community. >>>>>> For a X.Y release, going to an (X+2).? or an (X-2).? is not >> supported. >>>>> Some >>>>>> things that will not have tools might have warning logs printed >> that >>>> the >>>>>> functionality will change (should we change these to alerts?) >>>>>> >>>>>> If this sounds reasonable then jumping straight to Apache Druid 14 >> on >>>> the >>>>>> first official apache release would make a lot of sense. >>>>>> >>>>>> Cheers, >>>>>> Charles Allen >>>>>> >>>>>> >>>>>> On Thu, Dec 20, 2018 at 11:07 PM Gian Merlino <g...@apache.org> >> wrote: >>>>>> >>>>>>> I think it's a good point. Culturally we have been willing to >> break >>>>>>> extension APIs for relatively small benefits. But we have >> generally >>>>> been >>>>>>> unwilling to make breaking changes on the operations side quite >> so >>>>>>> liberally. Also, most cluster operators don't have their own >> custom >>>>>>> extensions, in my experience. So it does make sense to >> differentiate >>>>>> them. >>>>>>> I'm not sure how it makes sense to differentiate them, though. It >>>> could >>>>>> be >>>>>>> done through the version number (only increment the major >> version for >>>>>>> operations breaking changes) or it could be done through an >>>> "upgrading" >>>>>>> guide in the documentation (increment the major version for >>>> operations >>>>> or >>>>>>> extension breaking changes, but, have a guide that tells people >> which >>>>>>> versions have operations breaking changes to aid in upgrades). >>>>>>> >>>>>>> Coming back to the question in the subject of your mail: IMO, for >>>>>>> "graduation" out of 0.x, we should talk as a community about what >>>> that >>>>>>> means to us. It is a milestone that on the one hand, doesn't mean >>>> much, >>>>>> but >>>>>>> on the other hand, can be deeply symbolic. Some things that it >> has >>>>> meant >>>>>> to >>>>>>> other projects: >>>>>>> >>>>>>> 1) Production readiness. Obviously Druid is well past this. If >> this >>>> is >>>>>> what >>>>>>> dropping the 0. means, then we should do it immediately. >>>>>>> >>>>>>> 2) Belief that the APIs have become relatively stable. Like you >> said, >>>>> the >>>>>>> extension APIs don't seem particularly close to stable, but maybe >>>>> that's >>>>>>> okay. However, the pace of breaking changes on the operations and >>>> query >>>>>>> side for non-experimental features has been relatively calm for >> the >>>>> past >>>>>>> couple of years, so if we focus on that then we can make a case >> here. >>>>>>> >>>>>>> 3) Completeness of vision. This one is the most interesting to >> me. I >>>>>>> suspect that different people in the community have different >> visions >>>>> for >>>>>>> Druid. It is also the kind of project that may never truly be >>>> complete >>>>> in >>>>>>> vision (in principle, the platform could become a competitive >> data >>>>>>> warehouse, search engine, etc, …). For what it's worth, my >> vision of >>>>>> Druid >>>>>>> for the next year at least involves robust stream ingestion >> being a >>>>> first >>>>>>> class ingestion method (Kafka / Kinesis indexing service style) >> and >>>> SQL >>>>>>> being a first class query language. These are both, today, still >>>>>>> experimental features. So are lookups. All of these 3 features, >> from >>>>>> what I >>>>>>> can see, are quite popular amongst Druid users despite being >>>>>> experimental. >>>>>>> For a 'completeness of vision' based 1.0 I would want to lift >> all of >>>>>> those >>>>>>> out of experimental status and, for SQL in particular, to have >> its >>>>>>> functionality rounded out a bit more (to support the native query >>>>>> features >>>>>>> it doesn't currently support, like multi-value dimensions, >>>>> datasketches, >>>>>>> etc). >>>>>>> >>>>>>> 4) Marketing / timing. Like, doing a 1.0 around the time we >> graduate >>>>> from >>>>>>> the Incubator. Not sure how much this really matters, but >> projects do >>>>> it >>>>>>> sometimes. >>>>>>> >>>>>>> Another question is, how often do we intend to rev the version? >> At >>>> the >>>>>> rate >>>>>>> we're going, we rev 2-3 major versions a year. Would we intend to >>>> keep >>>>>> that >>>>>>> up, or slow it down by making more of an effort to avoid breaking >>>>>> changes? >>>>>>> >>>>>>> On Thu, Dec 20, 2018 at 2:17 PM Roman Leventov < >>>> leventov...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> It may also make sense to distinguish "operations" breaking >> changes >>>>>> from >>>>>>>> API breaking changes. Operations breaking changes establish the >>>>> minimum >>>>>>>> cadence of Druid cluster upgrades, that allow rolling Druid >>>> versions >>>>>> back >>>>>>>> and forward. I. e. it's related to segment format, the format >> of >>>> the >>>>>> data >>>>>>>> kept in ZooKeeper and the SQL database, or events such as >> stopping >>>>>>> support >>>>>>>> of ZooKeeper for certain things (e. g. forcing using of HTTP >>>>>>>> announcements). So Druid cluster operators cannot update Druid >> from >>>>>>> version >>>>>>>> X to version Z skipping the version Y, if both Y and Z have >> some >>>>>>> operations >>>>>>>> breaking changes. (Any such changes should support rollback >> options >>>>> at >>>>>>>> least until the next version with operations breaking changes.) >>>>>>>> >>>>>>>> API breaking changes are just changes in Druid extensions APIs. >>>> Druid >>>>>>>> cluster operators could skip any number of releases with such >>>>> breaking >>>>>>>> changes, as long as their extension's code is updated for the >>>> latest >>>>>>>> version of API. >>>>>>>> >>>>>>>> On Thu, 20 Dec 2018 at 20:03, Roman Leventov < >> leven...@apache.org> >>>>>>> wrote: >>>>>>>> >>>>>>>>> It doesn't seem to me that Druid API is going to stabilize >> in the >>>>>> near >>>>>>>>> future (if ever), because there are so many extension points >> and >>>>>>>> something >>>>>>>>> is broken in every release. On the other hand, Druid is not >>>> Hadoop >>>>> or >>>>>>>>> Spark, which have applications API. Druid API for >> extensions, not >>>>>>>>> applications. It is used by people who are closer to Druid >>>>>> development >>>>>>>> and >>>>>>>>> fixing their extensions is routine. >>>>>>>>> >>>>>>>>> With that, I think it make sense to drop "0." from the Druid >>>>> version >>>>>>> and >>>>>>>>> call it Druid 14, Druid 15, etc. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org >> For additional commands, e-mail: dev-h...@druid.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org
