Hi all, Reminder, I will cut the release branch tomorrow. If you have not done so please take a look at the 2.8.0 issues assigned to you [1].
Thank you! Ahmet [1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%202.8.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC On Thu, Oct 4, 2018 at 9:27 AM, Ahmet Altay <al...@google.com> wrote: > Thank you all for the feedback. I will continue with 2.8.0 as a regular > release and separate the LTS discussion to a new thread. > > On Thu, Oct 4, 2018 at 7:58 AM, Thomas Weise <t...@apache.org> wrote: > >> Given the feedback so far, we should probably decouple LTS and 2.8.0 >> discussions. In case both converge before 10/10 then fine, but not >> necessary. I also agree that we should not jump the gun on LTS and minimum >> 72 hours feedback window for the topic looks appropriate. >> >> The issues raised by Tim look like blockers and unless we are confident >> that they can be addressed as a patch release may warrant to defer LTS? Can >> we start to tag such JIRAs with an LTS label? >> >> On the other hand, I think we could allow for a bit of experimentation >> error for the first LTS attempt and feed guidelines/policies from >> learnings/feedback. >> >> Dependency updates for LTS: I don't think we should block LTS because >> there is a newer version of a dependency out there or we should rush >> updates. If we prioritize stability, then the latest usually isn't the >> best. In the case of Flink, 1.5.x is probably what most users have at this >> time and it has seen 4 patch releases. If Flink community continues to >> support last two minor (X.Y) versions, then 1.5.x support may drop when 1.7 >> comes out, but that does not mean we cannot use it if we were to cut a Beam >> LTS release today. I generally think that LTS needs to focus more on the >> stability of Beam itself. >> >> Thanks, >> Thomas >> >> >> >> On Thu, Oct 4, 2018 at 6:59 AM Alexey Romanenko <aromanenko....@gmail.com> >> wrote: >> >>> Regarding LTS release - I agree that we need to have clear view what >>> kind of support will be provided for such releases. >>> >>> Despite of the concerns mentioned before, I have another one about API >>> labeled as “@Experimental". I think there are most of IOs, SQL, PCollection >>> with Schema, etc, labeled with this annotation. >>> According to definition, such API should be considered as unstable in >>> terms that it can be changed/removed in next releases. >>> >>> So, the question is - how “@Experimental” API affects LTS releases (if >>> it does)? What kind of support should be provided in this case, especially, >>> in case if API continued evolving after LTS has been issued? Do we need to >>> provide a guarantee (another annotation, for example) that API won’t be >>> changed between two LTS releases? >>> >>> And one more related question, which probably deserves another >>> discussion (or was already discussed) - what is criteria to remove >>> status “@Experimental” from API? How we decide that API is stable and not >>> changing anymore? >>> >>> >>> On 4 Oct 2018, at 12:35, Robert Bradshaw <rober...@google.com> wrote: >>> >>> +1 to cutting the release. >>> >>> I agree that the LTS label requires more discussion. I think it boils >>> down to the question of whether we are comfortable with encouraging people >>> to not upgrade to the latest Beam. It probably boils down to creating a >>> list of (potential) blockers and then going from there. Also, on this note, >>> I think we should be very conservative in updating dependencies for an LTS >>> release. >>> >>> We could also consider for this release doing an "LTS light" where we >>> prove the process, gain some experience, but don't promise a full 12 months >>> of support (say, cutting it to 6 months). >>> >>> - Robert >>> >>> >>> >>> >>> On Thu, Oct 4, 2018 at 11:25 AM Tim Robertson <timrobertson...@gmail.com> >>> wrote: >>> >>>> I was in the middle of writing something similar when Ismaël posted. >>>> >>>> Please do bear in mind that this is an international project and 7hrs >>>> is not long enough to decide upon something that affects us all. >>>> >>>> +1 on cutting 2.8.0 on 10/10 and thank you for pushing it forward >>>> >>>> -1 on designating it as LTS: >>>> While LTS is a statement of expectation in maintenance it also carries >>>> an element of trust. I propose we should have a separate discussion about >>>> what we might like to collectively achieve before announcing our first LTS >>>> edition. >>>> My concern stems from usability and first impressions - for example: >>>> - Beam has real issues with HDFS today (BEAM-5036) which I propose as >>>> blocker for announcing LTS >>>> - DirectRunner and the inability to run basic pipelines on a few GB of >>>> data is *really* putting people off our project - we might consider >>>> exploring that as it affects our "brand" >>>> >>>> >>>> >>>> >>>> On Thu, Oct 4, 2018 at 11:18 AM Ismaël Mejía <ieme...@gmail.com> wrote: >>>> >>>>> Hello, >>>>> >>>>> Thanks Ahmet for volunteering to do the release, and proposing this as >>>>> an LTS. >>>>> >>>>> I have still some questions on our LTS policies (which may have >>>>> consequences on the discussed release): >>>>> >>>>> What are the expected implications of upgrades in the LTS, e.g. If a >>>>> connector let’s say Kafka is released using the 1.0 dependency, can it >>>>> be moved upwards in a LTS to version 2.0 or this will be considered a >>>>> breaking change and we should only move in minor versions. Will this >>>>> rule be more relaxed for example for all cloud based dependencies >>>>> (GCP, AWS) for example if a security issue or correctness/performance >>>>> improvement? >>>>> >>>>> Given that this will last for a year maybe we should raise some of the >>>>> dependencies to the latest versions. Following the recent discussion >>>>> on dependencies that cannot be ‘automatically’ updated because of end >>>>> user consequences, I still think about what we should do with >>>>> (probably related to the previous paragraph): >>>>> >>>>> - Should we move Flink then to 1.6.x considering that 1.5.x won’t be >>>>> maintained in less than 6 months. >>>>> - Should we wait and upgrade Spark into version 2.4.0 (which is being >>>>> voted at this moment but not released but could make sense for a LTS) >>>>> or just stay in 2.3.x. Spark is less of an issue because it is a >>>>> provided dep but still worth. >>>>> - Should we update the IO connectors dependencies to the latest stable >>>>> versions who aren’t, e.g. Elasticsearch, HBase, >>>>> >>>>> Of course the goal is not a last minute rush to do this so it fits in >>>>> the LTS release, but to see that for LTS we may consider the ‘lasting >>>>> consequences'. >>>>> >>>>> One last comment, next time we discuss a proposal please ensure that >>>>> we wait at least 24h to reach conclusions or proceed, otherwise this >>>>> will exclude opinions from people who are not in the right time zone >>>>> (this is the reason why votes last 72h to ensure that everyone may be >>>>> aware of what is been voted). This is not a mandatory requirement, but >>>>> agreeing on a LTS in 7h seems a bit short. >>>>> On Thu, Oct 4, 2018 at 1:36 AM Ahmet Altay <al...@google.com> wrote: >>>>> > >>>>> > Great. I will do the cut on 10/10. >>>>> > >>>>> > Let's start by triaging the open issues targeted for 2.8.0 [1]. If >>>>> you have any issues in this list please resolve them or move to the next >>>>> release. If you are aware of any critical issues please add to this list. >>>>> > >>>>> > Ahmet >>>>> > >>>>> > [1] https://issues.apache.org/jira/browse/BEAM-5456?jql=project% >>>>> 20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20f >>>>> ixVersion%20%3D%202.8.0%20ORDER%20BY%20priority%20DESC%2C% >>>>> 20updated%20DESC >>>>> > >>>>> > > +1 for the 2.7.0 release schedule. Thanks for volunteering. Do we >>>>> want a standing owner for the LTS branch (like the Linux kernel has) or >>>>> will we just take volunteers for each LTS release as they arise? >>>>> > >>>>> > We have not thought about this before. IMO, it is better to keep >>>>> things simple and use the same process (i.e. "we just take volunteers for >>>>> each LTS release as they arise") for patch releases in the future if/when >>>>> we happen to need those. >>>>> > >>>>> > >>>>> > On Wed, Oct 3, 2018 at 1:21 PM, Thomas Weise <t...@apache.org> wrote: >>>>> >> >>>>> >> +1 >>>>> >> >>>>> >> On Wed, Oct 3, 2018 at 12:33 PM Ted Yu <yuzhih...@gmail.com> wrote: >>>>> >>> >>>>> >>> +1 >>>>> >>> >>>>> >>> On Wed, Oct 3, 2018 at 9:52 AM Jean-Baptiste Onofré < >>>>> j...@nanthrax.net> wrote: >>>>> >>>> >>>>> >>>> +1 >>>>> >>>> >>>>> >>>> but we have to be fast in release process. 2.7.0 took more than 1 >>>>> month >>>>> >>>> to be cut ! >>>>> >>>> >>>>> >>>> If no blocker, we have to just move forward. >>>>> > >>>>> > >>>>> > +1 >>>>> > >>>>> >>>> >>>>> >>>> >>>>> >>>> Regards >>>>> >>>> JB >>>>> >>>> >>>>> >>>> On 03/10/2018 18:25, Ahmet Altay wrote: >>>>> >>>> > Hi all, >>>>> >>>> > >>>>> >>>> > Release cut date for the next release is 10/10 according to >>>>> Beam release >>>>> >>>> > calendar [1]. Since the previous release is already mostly >>>>> wrapped up >>>>> >>>> > (modulo blog post), I would like to propose starting the next >>>>> release on >>>>> >>>> > time (10/10). >>>>> >>>> > >>>>> >>>> > Additionally I propose designating this release as the first >>>>> >>>> > long-term-support (LTS) release [2]. This should have no impact >>>>> on the >>>>> >>>> > release process, however it would mean that we commit to patch >>>>> this >>>>> >>>> > release for the next 12 months for major issues. >>>>> >>>> > >>>>> >>>> > I volunteer to perform this release. >>>>> >>>> > >>>>> >>>> > What do you think? >>>>> >>>> > >>>>> >>>> > Ahmet >>>>> >>>> > >>>>> >>>> > [1] https://calendar.google.com/ca >>>>> lendar/embed?src=0p73sl034k80oob7seouanigd0%40group. >>>>> calendar.google.com&ctz=America%2FLos_Angeles >>>>> >>>> > [2] https://beam.apache.org/community/policies/#releases >>>>> >>>> >>>>> >>>> -- >>>>> >>>> Jean-Baptiste Onofré >>>>> >>>> jbono...@apache.org >>>>> >>>> http://blog.nanthrax.net >>>>> >>>> Talend - http://www.talend.com >>>>> > >>>>> > >>>>> >>>> >>> >
