On Mon, Oct 22, 2018 at 2:11 AM, Robert Bradshaw <[email protected]> wrote:
> On Sat, Oct 20, 2018 at 12:24 AM Kenneth Knowles <[email protected]> wrote: > >> Pinging this. I think Beam should have a live LTS branch. >> >> I want to suggest a different approach: choose something already released >> to be LTS. This way it has had some usage and we have some confidence there >> are no critical problems. >> > > +1. This is what I suggested as well. > I agree, great suggestion to base on a known good release. > > >> So how about we make 2.7 the first LTS branch? >> > > I would be fine with that, however, I think our first stab at an LTS > branch should not promise a full year of support, more like a test run. > How could we cleanly communicate this to the users if we go with the test run approach? My worry is that if we label it as LTS people will treat is as it is supported. We could choose to not label it anything and still try to provide patches as a test run. However, that would not be very helpful because user would not necessarily stay on 2.7 unless they know that we have an intention of keeping the branch patched for a while. How about marking it LTS but with a special note on the release notes with a specifically shorter time period? (e.g. 2.7.0 release date + 6 months as the test run period.) > +1 to Thomas' point that concrete user input/feedback would be useful in > planning here. > > But I have another question: Are we ready to make LTS releases? Today, do most/all users (that upgrade) move to the latest release because they really need new features or other changes that would be incompatible with an LTS branch or do we see a sizable amount of users that would feel content with the changes an LTS release can provide them? Did we do a poll or have sufficient feedback? We attempted to collect feedback on the mailing lists but did not get much input. From my experience (mostly based on dataflow) there is a sizeable group of users who are less interested in new features and want a version that is stable, that does not have security issues, major data integrity issues etc. In Beam's existing release model that corresponds to the latest release. It would help a lot if we can hear the perspectives of other users who are not present here through the developers who work with them. > >> Kenn >> >> On Wed, Oct 10, 2018 at 8:18 AM Romain Manni-Bucau <[email protected]> >> wrote: >> >>> some times Ago JB spoke about Beam roadmap. I tend to think this >>> discussion does no make any sense without a clear roadmap. The rational >>> here is that a roadmap will give you the future changes >>> and the potential future versions (we spoke a few times of Beam 3). This >>> does not have to be very factual, a slice of 3 months is ok at that stage. >>> However, if you don't have that, >>> you can say 2.8 will be LTS and we support 2 versions but if 2.9 and >>> 2.10 introduce breaking changes, then >>> it leads to a LTS 2.8 no more supported. This is just an example but the >>> ratio "cost(project) / gain(user)" 100% depends the plans for the project, >>> technically there is no blocker to support all releases >>> for life but would any PMC have the will to release beam 0.x now? The >>> point of a LTS for an user is to plan investment, if we are in previous >>> case it does not help IMHO. >>> So maybe grab back the Beam enhancement plans and assign them some fix >>> versions before defining what support model of Beam can be. >>> >>> Just the 2 cts of an outsider. >>> >>> Romain Manni-Bucau >>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>> <https://rmannibucau.metawerx.net/> | Old Blog >>> <http://rmannibucau.wordpress.com> | Github >>> <https://github.com/rmannibucau> | LinkedIn >>> <https://www.linkedin.com/in/rmannibucau> | Book >>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>> >>> >>> Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <[email protected]> >>> a écrit : >>> >>>> >>>> >>>> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <[email protected]> >>>> wrote: >>>> >>>>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <[email protected]> >>>>> wrote: >>>>> >>>>>> The simplest thing we can do is just to pin all the deps of the LTS >>>>>> and not move them in any maintenance release if not a strong reason to >>>>>> do so. >>>>>> >>>>>> The next subject is to make maintainers aware of which release will be >>>>>> the LTS in advance so they decide what to do with the dependencies >>>>>> versions. In my previous mail I mentioned all the possible cases that >>>>>> can happen with dependencies and it is clear that one unified policy >>>>>> won’t satisfy every one. So better let the maintainers (who can also >>>>>> ask for user feedback in the ML) to decide about versions before the >>>>>> release. >>>>>> >>>>>> Alexey’s question is still a really important issue, and has been so >>>>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS. >>>>>> Options are: >>>>>> >>>>>> (1) We keep consistent with Experimental which means that they are >>>>>> still not guarantees (note that this does not mean that they will be >>>>>> broken arbitrarily). >>>>>> (2) We are consistent with the LTS approach which makes them ‘non >>>>>> experimental’ for the LTS so we will guarantee the functionality/API >>>>>> stable. >>>>>> >>>>>> I personally have conflicted opinions I would like to favor (1) but >>>>>> this is not consistent with the whole idea of LTS so probably (2) is >>>>>> wiser. >>>>>> >>>>> >>>>> Yeah, I think (2) is the only viable option. >>>>> >>>> >>>> I think important thing here is that future releases on a LTS branch >>>> will be patch (bugfix) releases so I don't think we can/should do >>>> API/functionality changes (even if the change is experimental and/or >>>> backwards compatible). >>>> >>>> I think same goes for dependency changes. If the change is to fix a >>>> known bug we can do that in a patch release but if it's to add more >>>> functionality probably that should come in a new minor release instead of a >>>> patch release. >>>> >>>> This is why I think we should be bit careful about "rushed" changes to >>>> major functionalities of Beam going into LTS releases. Just my 2 cents. >>>> >>>> Thanks, >>>> Cham >>>> >>>> >>>>> >>>>> >>>>>> Finally I also worry about Tim’s remarks on performance and quality, >>>>>> even if some of these things effectively can be fixed in a subsequent >>>>>> LTS release. Users will probably prefer a LTS to start with Beam and >>>>>> if the performance/quality of the LTS, this can hurt perception of the >>>>>> project. >>>>>> >>>>> >>>>> Yes, for this reason I think it's important to consider what goes into >>>>> an LTS as well as what happens after. Almost by definition, using an LTS >>>>> is >>>>> choosing stability over cutting edge features. I'd rather major feature X >>>>> goes in after LTS, and lives in a couple of releases gaining fixes and >>>>> improvements before being released as part of the next LTS, than quickly >>>>> making it into an LTS while brand new (both due to the time period before >>>>> we refine it, and the extra work of porting refinements back). >>>>> >>>>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway, >>>>> waiting for at least x.y.1? >>>>> >>>>> Come to think of it, do we even have to designate releases at LTS at >>>>> the time of release? Perhaps we could instead just keep with our normal >>>>> release cadence, and periodically choose one of them as LTS once we've >>>>> confirmed its stability out in the wild and start backporting to it. (I >>>>> can >>>>> think of several cons with this approach as well, e.g. generally it's >>>>> easier to backport bugfixes at the time a bugfix is done in master rather >>>>> than swapping in context at a later date, but just thought I'd throw it >>>>> out >>>>> there.) >>>>> >>>>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <[email protected]> >>>>>> wrote: >>>>>> > >>>>>> > I've seen two mentions that "rushing" is contrary to the goals of >>>>>> LTS. But I wouldn't worry about this. The fact is there is almost nothing >>>>>> you can do to stabilize *prior* to cutting the LTS branch. Stability >>>>>> comes >>>>>> from the branch being long-lived and having multiple releases. >>>>>> > >>>>>> > (I think this is pretty much my version of what JB is saying) >>>>>> > >>>>>> > What a conservative user will do if 2.8.x is declared LTS is to >>>>>> start using the 2.8.x branch after it has had a couple bugfix releases. I >>>>>> don't think it is useful or possible to try for an "extra stable" 2.x.0. >>>>>> > >>>>>> > The arguments about supporting the most widely used versions of >>>>>> runner backends apply regardless of LTS. We should support them if we >>>>>> have >>>>>> the resources to do so. >>>>>> > >>>>>> > Kenn >>>>>> > >>>>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <[email protected]> >>>>>> wrote: >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré < >>>>>> [email protected]> wrote: >>>>>> >>> >>>>>> >>> Hi, >>>>>> >>> >>>>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a >>>>>> branch >>>>>> >>> that we guarantee to have fixes on it for a long period of time. >>>>>> >>> >>>>>> >>> >>>>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of >>>>>> >>> releases on a LTS branch, the only constraint is to avoid to >>>>>> introduce >>>>>> >>> breaking changes. >>>>>> >> >>>>>> >> >>>>>> >> I agree with this perspective. Thank you for sharing this. However >>>>>> the other commenters also had a good point. Requiring users to upgrade >>>>>> their runner version maybe incompatible with the goals of an LTS branch. >>>>>> Ideally the fixes here should be very minimal and targeted. >>>>>> >> >>>>>> >>> >>>>>> >>> >>>>>> >>> So, IMHO, the key part is not release, it's branch. >>>>>> >>> >>>>>> >>> The first thing to decide is the branch. >>>>>> >>> >>>>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x >>>>>> LTS >>>>>> >>> branch. It's a branch where we will cherry-pick some important >>>>>> fixes in >>>>>> >>> the future and where we will cut release. It's the approach I use >>>>>> in >>>>>> >>> other Apache projects (especially Karaf) and it works fine. >>>>>> >> >>>>>> >> >>>>>> >> JB, does Karaf has a documented process that we can re-use? If not >>>>>> could you explain a bit more? >>>>>> >> >>>>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0 >>>>>> release out of that? >>>>>> >> >>>>>> >>> >>>>>> >>> >>>>>> >>> Just my $0.01 >>>>>> >>> >>>>>> >>> Regards >>>>>> >>> JB >>>>>> >>> >>>>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote: >>>>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath < >>>>>> [email protected] >>>>>> >>> > <mailto:[email protected]>> wrote: >>>>>> >>> > >>>>>> >>> > >>>>>> >>> > On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay < >>>>>> [email protected] >>>>>> >>> > <mailto:[email protected]>> wrote: >>>>>> >>> > >>>>>> >>> > I agree that LTS releases require more thought. Thank >>>>>> you for >>>>>> >>> > raising these questions. What other open questions do >>>>>> we have >>>>>> >>> > related LTS releases? >>>>>> >>> > >>>>>> >>> > One way to do this would be to add them to a particular >>>>>> tracking >>>>>> >>> > list (e.g. 2.9.0 blocking list) that way we would be >>>>>> ready for >>>>>> >>> > an LTS release ahead of time. >>>>>> >>> > >>>>>> >>> > Related to dependencies, I agree with Thomas. If we >>>>>> block on >>>>>> >>> > waiting for dependencies, we may end up taking a long >>>>>> time >>>>>> >>> > before making any LTS release. And the reality of Beam >>>>>> releases >>>>>> >>> > right now is that there are no supported releases today >>>>>> that and >>>>>> >>> > in the long term that might hurt user trust. In my >>>>>> opinion, we >>>>>> >>> > need to fix that sooner rather than later. >>>>>> >>> > >>>>>> >>> > >>>>>> >>> > Agree on the idea of focussing on stability instead of >>>>>> feature set >>>>>> >>> > when it comes to LTS releases. Based on the previous >>>>>> discussion on >>>>>> >>> > this [1] looks like the intended audience is enterprise >>>>>> customers >>>>>> >>> > that would like to maintain a stable environment and that >>>>>> usually >>>>>> >>> > have a long testing cycle before deploying a new version. >>>>>> Seems >>>>>> >>> > like, rushing in features or dependency upgrades for an >>>>>> LTS release >>>>>> >>> > will be counterintuitive for such use cases. >>>>>> >>> > >>>>>> >>> > On the other hand, I think a part of Ismaël point was that >>>>>> all major >>>>>> >>> > features of Beam (Spark Runner, Flink Runner, Kafka IO, etc >>>>>> ..) >>>>>> >>> > should be supportable for the duration of the time Beam LTS >>>>>> release >>>>>> >>> > is supported (1 year based on [1]). If a major dependency is >>>>>> >>> > expected to become unsupported within an year, it makes >>>>>> sense to >>>>>> >>> > upgrade that before the LTS release. I suggest we do this >>>>>> at least >>>>>> >>> > one release before the planned LTS release (so 2.9.0 if we >>>>>> want to >>>>>> >>> > make 2.10.0 a LTS release) so that we have all major >>>>>> >>> > dependency/feature updates battle tested before committing >>>>>> to them. >>>>>> >>> > >>>>>> >>> > >>>>>> >>> > Dependencies are a significant concern. What happens if one of >>>>>> our >>>>>> >>> > dependencies has no version they're going to support for a year >>>>>> hence? >>>>>> >>> > It could happen that there's never a time when all our >>>>>> dependencies will >>>>>> >>> > be supported for at least a year as of any given date. And >>>>>> rushing to >>>>>> >>> > get the "latest" version of a dependency in is often contrary >>>>>> to the >>>>>> >>> > goals of an LTS. (Better to have those depend on the >>>>>> battle-hardened >>>>>> >>> > versions.) >>>>>> >>> > >>>>>> >>> > The long-term goal is probably to avoid pinning specific >>>>>> versions as >>>>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6 >>>>>> (whichever the >>>>>> >>> > user provided), and considered backporting support for 1.7 when >>>>>> it came >>>>>> >>> > out, we'd be in much better shape here. How feasible this is >>>>>> depends on >>>>>> >>> > how backwards-compatible the dependency is. >>>>>> >>> > >>>>>> >>> > On the other hand, forcing a user to upgrade a (user-visible, >>>>>> which in >>>>>> >>> > particular includes runners) dependency seems contrary to the >>>>>> goals of >>>>>> >>> > an LTS. Even if that dependency becomes unsupported. >>>>>> >>> > >>>>>> >>> > >>>>>> >>> >>>>>> >>> -- >>>>>> >>> Jean-Baptiste Onofré >>>>>> >>> [email protected] >>>>>> >>> http://blog.nanthrax.net >>>>>> >>> Talend - http://www.talend.com >>>>>> >> >>>>>> >> >>>>>> >>>>>
