Update: I re-cut the release branch. Only remaining issue on the 2.8.0 list is currently RabbitMQIO. JB, let me know if you would like to cherry pick that in to the release branch.
On Wed, Oct 10, 2018 at 1:44 PM, Ahmet Altay <al...@google.com> wrote: > Given the number of open issues, I will re-cut the release branch once the > blocking issues are resolved. Don't worry about cherry picking changes to > directly to the release branch for now. > > I will continue to update this thread. > > On Wed, Oct 10, 2018 at 12:12 PM, Niel Markwick <ni...@google.com> wrote: > >> The 3 spannerio issues (5445, 3516, 4796) are waiting for one last LGTM >> before the PRs can be merged, but are otherwise ready for 2.8... > > > Please work with the reviewers to get them in. I moved those issues to > 2.9.0 already. > > >> >> On Wed, 10 Oct 2018, 19:51 Ahmet Altay, <al...@google.com> wrote: >> >>> Thank you JB. >>> >>> It turns out there are 2 more blocker issues. I will look at them now >>> first. (So, I am not rushing towards cutting RC1 yet.) >>> >>> On Wed, Oct 10, 2018 at 11:42 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>>> Hey >>>> >>>> Etienne should do a new pass soon. I do my best to cherry pick >>>> RabbitMQIO. >>>> >>>> Thanks >>>> Regards >>>> JB >>>> Le 10 oct. 2018, à 21:25, Ahmet Altay <al...@google.com> a écrit: >>>>> >>>>> Update: >>>>> >>>>> I started cutting the branch. There are 2 open issues: >>>>> - RabbitMQIO - JB, if you plan to complete this soon I can cherry pick >>>>> to the branch. >>>>> - One new issue related to release process changes with respect to >>>>> beam-site deprecation. >>>>> >>>>> On Tue, Oct 9, 2018 at 11:38 AM, Jean-Baptiste Onofré <j...@nanthrax.net >>>>> > wrote: >>>>> >>>>>> Ok. Gonna move forward on RabbitMQIO asap. >>>>>> >>>>>> Thanks >>>>>> Regards >>>>>> JB >>>>>> Le 9 oct. 2018, à 21:00, Ahmet Altay <al...@google.com> a écrit: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Reminder, I will cut the release branch tomorrow. If you have not >>>>>>> done so please take a look at the 2.8.0 issues assigned to you [1]. >>>>>>> >>>>>>> Thank you! >>>>>>> Ahmet >>>>>>> >>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%3D >>>>>>> %20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVer >>>>>>> sion%20%3D%202.8.0%20ORDER%20BY%20priority%20DESC%2C% >>>>>>> 20updated%20DESC >>>>>>> >>>>>>> On Thu, Oct 4, 2018 at 9:27 AM, Ahmet Altay <al...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Thank you all for the feedback. I will continue with 2.8.0 as a >>>>>>>> regular release and separate the LTS discussion to a new thread. >>>>>>>> >>>>>>>> On Thu, Oct 4, 2018 at 7:58 AM, Thomas Weise <t...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Given the feedback so far, we should probably decouple LTS and >>>>>>>>> 2.8.0 discussions. In case both converge before 10/10 then fine, but >>>>>>>>> not >>>>>>>>> necessary. I also agree that we should not jump the gun on LTS and >>>>>>>>> minimum >>>>>>>>> 72 hours feedback window for the topic looks appropriate. >>>>>>>>> >>>>>>>>> The issues raised by Tim look like blockers and unless we are >>>>>>>>> confident that they can be addressed as a patch release may warrant to >>>>>>>>> defer LTS? Can we start to tag such JIRAs with an LTS label? >>>>>>>>> >>>>>>>>> On the other hand, I think we could allow for a bit of >>>>>>>>> experimentation error for the first LTS attempt and feed >>>>>>>>> guidelines/policies from learnings/feedback. >>>>>>>>> >>>>>>>>> Dependency updates for LTS: I don't think we should block LTS >>>>>>>>> because there is a newer version of a dependency out there or we >>>>>>>>> should >>>>>>>>> rush updates. If we prioritize stability, then the latest usually >>>>>>>>> isn't the >>>>>>>>> best. In the case of Flink, 1.5.x is probably what most users have at >>>>>>>>> this >>>>>>>>> time and it has seen 4 patch releases. If Flink community continues to >>>>>>>>> support last two minor (X.Y) versions, then 1.5.x support may drop >>>>>>>>> when 1.7 >>>>>>>>> comes out, but that does not mean we cannot use it if we were to cut >>>>>>>>> a Beam >>>>>>>>> LTS release today. I generally think that LTS needs to focus more on >>>>>>>>> the >>>>>>>>> stability of Beam itself. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Thomas >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Oct 4, 2018 at 6:59 AM Alexey Romanenko < >>>>>>>>> aromanenko....@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Regarding LTS release - I agree that we need to have clear view >>>>>>>>>> what kind of support will be provided for such releases. >>>>>>>>>> >>>>>>>>>> Despite of the concerns mentioned before, I have another one >>>>>>>>>> about API labeled as “@Experimental". I think there are most of IOs, >>>>>>>>>> SQL, >>>>>>>>>> PCollection with Schema, etc, labeled with this annotation. >>>>>>>>>> According to definition, such API should be considered as >>>>>>>>>> unstable in terms that it can be changed/removed in next releases. >>>>>>>>>> >>>>>>>>>> So, the question is - how “@Experimental” API affects LTS >>>>>>>>>> releases (if it does)? What kind of support should be provided in >>>>>>>>>> this >>>>>>>>>> case, especially, in case if API continued evolving after LTS has >>>>>>>>>> been >>>>>>>>>> issued? Do we need to provide a guarantee (another annotation, for >>>>>>>>>> example) >>>>>>>>>> that API won’t be changed between two LTS releases? >>>>>>>>>> >>>>>>>>>> And one more related question, which probably deserves another >>>>>>>>>> discussion (or was already discussed) - what is criteria to remove >>>>>>>>>> status “@Experimental” from API? How we decide that API is stable >>>>>>>>>> and not >>>>>>>>>> changing anymore? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 4 Oct 2018, at 12:35, Robert Bradshaw <rober...@google.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> +1 to cutting the release. >>>>>>>>>> >>>>>>>>>> I agree that the LTS label requires more discussion. I think it >>>>>>>>>> boils down to the question of whether we are comfortable with >>>>>>>>>> encouraging >>>>>>>>>> people to not upgrade to the latest Beam. It probably boils down to >>>>>>>>>> creating a list of (potential) blockers and then going from there. >>>>>>>>>> Also, on >>>>>>>>>> this note, I think we should be very conservative in updating >>>>>>>>>> dependencies >>>>>>>>>> for an LTS release. >>>>>>>>>> >>>>>>>>>> We could also consider for this release doing an "LTS light" >>>>>>>>>> where we prove the process, gain some experience, but don't promise >>>>>>>>>> a full >>>>>>>>>> 12 months of support (say, cutting it to 6 months). >>>>>>>>>> >>>>>>>>>> - Robert >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Oct 4, 2018 at 11:25 AM Tim Robertson < >>>>>>>>>> timrobertson...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> I was in the middle of writing something similar when Ismaël >>>>>>>>>>> posted. >>>>>>>>>>> >>>>>>>>>>> Please do bear in mind that this is an international project and >>>>>>>>>>> 7hrs is not long enough to decide upon something that affects us >>>>>>>>>>> all. >>>>>>>>>>> >>>>>>>>>>> +1 on cutting 2.8.0 on 10/10 and thank you for pushing it forward >>>>>>>>>>> >>>>>>>>>>> -1 on designating it as LTS: >>>>>>>>>>> While LTS is a statement of expectation in maintenance it also >>>>>>>>>>> carries an element of trust. I propose we should have a separate >>>>>>>>>>> discussion >>>>>>>>>>> about what we might like to collectively achieve before announcing >>>>>>>>>>> our >>>>>>>>>>> first LTS edition. >>>>>>>>>>> My concern stems from usability and first impressions - for >>>>>>>>>>> example: >>>>>>>>>>> - Beam has real issues with HDFS today (BEAM-5036) which I >>>>>>>>>>> propose as blocker for announcing LTS >>>>>>>>>>> - DirectRunner and the inability to run basic pipelines on a few >>>>>>>>>>> GB of data is *really* putting people off our project - we might >>>>>>>>>>> consider >>>>>>>>>>> exploring that as it affects our "brand" >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Oct 4, 2018 at 11:18 AM Ismaël Mejía <ieme...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> Thanks Ahmet for volunteering to do the release, and proposing >>>>>>>>>>>> this as an LTS. >>>>>>>>>>>> >>>>>>>>>>>> I have still some questions on our LTS policies (which may have >>>>>>>>>>>> consequences on the discussed release): >>>>>>>>>>>> >>>>>>>>>>>> What are the expected implications of upgrades in the LTS, e.g. >>>>>>>>>>>> If a >>>>>>>>>>>> connector let’s say Kafka is released using the 1.0 dependency, >>>>>>>>>>>> can it >>>>>>>>>>>> be moved upwards in a LTS to version 2.0 or this will be >>>>>>>>>>>> considered a >>>>>>>>>>>> breaking change and we should only move in minor versions. Will >>>>>>>>>>>> this >>>>>>>>>>>> rule be more relaxed for example for all cloud based >>>>>>>>>>>> dependencies >>>>>>>>>>>> (GCP, AWS) for example if a security issue or >>>>>>>>>>>> correctness/performance >>>>>>>>>>>> improvement? >>>>>>>>>>>> >>>>>>>>>>>> Given that this will last for a year maybe we should raise some >>>>>>>>>>>> of the >>>>>>>>>>>> dependencies to the latest versions. Following the recent >>>>>>>>>>>> discussion >>>>>>>>>>>> on dependencies that cannot be ‘automatically’ updated because >>>>>>>>>>>> of end >>>>>>>>>>>> user consequences, I still think about what we should do with >>>>>>>>>>>> (probably related to the previous paragraph): >>>>>>>>>>>> >>>>>>>>>>>> - Should we move Flink then to 1.6.x considering that 1.5.x >>>>>>>>>>>> won’t be >>>>>>>>>>>> maintained in less than 6 months. >>>>>>>>>>>> - Should we wait and upgrade Spark into version 2.4.0 (which is >>>>>>>>>>>> being >>>>>>>>>>>> voted at this moment but not released but could make sense for >>>>>>>>>>>> a LTS) >>>>>>>>>>>> or just stay in 2.3.x. Spark is less of an issue because it is a >>>>>>>>>>>> provided dep but still worth. >>>>>>>>>>>> - Should we update the IO connectors dependencies to the latest >>>>>>>>>>>> stable >>>>>>>>>>>> versions who aren’t, e.g. Elasticsearch, HBase, >>>>>>>>>>>> >>>>>>>>>>>> Of course the goal is not a last minute rush to do this so it >>>>>>>>>>>> fits in >>>>>>>>>>>> the LTS release, but to see that for LTS we may consider the >>>>>>>>>>>> ‘lasting >>>>>>>>>>>> consequences'. >>>>>>>>>>>> >>>>>>>>>>>> One last comment, next time we discuss a proposal please ensure >>>>>>>>>>>> that >>>>>>>>>>>> we wait at least 24h to reach conclusions or proceed, otherwise >>>>>>>>>>>> this >>>>>>>>>>>> will exclude opinions from people who are not in the right time >>>>>>>>>>>> zone >>>>>>>>>>>> (this is the reason why votes last 72h to ensure that everyone >>>>>>>>>>>> may be >>>>>>>>>>>> aware of what is been voted). This is not a mandatory >>>>>>>>>>>> requirement, but >>>>>>>>>>>> agreeing on a LTS in 7h seems a bit short. >>>>>>>>>>>> On Thu, Oct 4, 2018 at 1:36 AM Ahmet Altay <al...@google.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> > >>>>>>>>>>>> > Great. I will do the cut on 10/10. >>>>>>>>>>>> > >>>>>>>>>>>> > Let's start by triaging the open issues targeted for 2.8.0 >>>>>>>>>>>> [1]. If you have any issues in this list please resolve them or >>>>>>>>>>>> move to the >>>>>>>>>>>> next release. If you are aware of any critical issues please add >>>>>>>>>>>> to this >>>>>>>>>>>> list. >>>>>>>>>>>> > >>>>>>>>>>>> > Ahmet >>>>>>>>>>>> > >>>>>>>>>>>> > [1] https://issues.apache.org/jira >>>>>>>>>>>> /browse/BEAM-5456?jql=project%20%3D%20BEAM%20AND%20resolutio >>>>>>>>>>>> n%20%3D%20Unresolved%20AND%20fixVersion%20%3D%202.8.0%20ORDE >>>>>>>>>>>> R%20BY%20priority%20DESC%2C%20updated%20DESC >>>>>>>>>>>> > >>>>>>>>>>>> > > +1 for the 2.7.0 release schedule. Thanks for volunteering. >>>>>>>>>>>> Do we want a standing owner for the LTS branch (like the Linux >>>>>>>>>>>> kernel has) >>>>>>>>>>>> or will we just take volunteers for each LTS release as they arise? >>>>>>>>>>>> > >>>>>>>>>>>> > We have not thought about this before. IMO, it is better to >>>>>>>>>>>> keep things simple and use the same process (i.e. "we just take >>>>>>>>>>>> volunteers >>>>>>>>>>>> for each LTS release as they arise") for patch releases in the >>>>>>>>>>>> future >>>>>>>>>>>> if/when we happen to need those. >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > On Wed, Oct 3, 2018 at 1:21 PM, Thomas Weise <t...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >> >>>>>>>>>>>> >> +1 >>>>>>>>>>>> >> >>>>>>>>>>>> >> On Wed, Oct 3, 2018 at 12:33 PM Ted Yu <yuzhih...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> +1 >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> On Wed, Oct 3, 2018 at 9:52 AM Jean-Baptiste Onofré < >>>>>>>>>>>> j...@nanthrax.net> wrote: >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> +1 >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> but we have to be fast in release process. 2.7.0 took more >>>>>>>>>>>> than 1 month >>>>>>>>>>>> >>>> to be cut ! >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> If no blocker, we have to just move forward. >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > +1 >>>>>>>>>>>> > >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> Regards >>>>>>>>>>>> >>>> JB >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> On 03/10/2018 18:25, Ahmet Altay wrote: >>>>>>>>>>>> >>>> > Hi all, >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > Release cut date for the next release is 10/10 according >>>>>>>>>>>> to Beam release >>>>>>>>>>>> >>>> > calendar [1]. Since the previous release is already >>>>>>>>>>>> mostly wrapped up >>>>>>>>>>>> >>>> > (modulo blog post), I would like to propose starting the >>>>>>>>>>>> next release on >>>>>>>>>>>> >>>> > time (10/10). >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > Additionally I propose designating this release as the >>>>>>>>>>>> first >>>>>>>>>>>> >>>> > long-term-support (LTS) release [2]. This should have no >>>>>>>>>>>> impact on the >>>>>>>>>>>> >>>> > release process, however it would mean that we commit to >>>>>>>>>>>> patch this >>>>>>>>>>>> >>>> > release for the next 12 months for major issues. >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > I volunteer to perform this release. >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > What do you think? >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > Ahmet >>>>>>>>>>>> >>>> > >>>>>>>>>>>> >>>> > [1] https://calendar.google.com/ca >>>>>>>>>>>> lendar/embed?src=0p73sl034k80oob7seouanigd0%40group. >>>>>>>>>>>> calendar.google.com&ctz=America%2FLos_Angeles >>>>>>>>>>>> >>>> > [2] https://beam.apache.org/community/policies/#releases >>>>>>>>>>>> >>>> >>>>>>>>>>>> >>>> -- >>>>>>>>>>>> >>>> Jean-Baptiste Onofré >>>>>>>>>>>> >>>> jbono...@apache.org >>>>>>>>>>>> >>>> http://blog.nanthrax.net >>>>>>>>>>>> >>>> Talend - http://www.talend.com >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> -- >> >> <https://cloud.google.com/> >> * • **Niel Markwick* >> * • *Cloud Solutions Architect >> * • *Google Belgium >> * • *ni...@google.com >> * • *+32 2 894 6771 <+3228946771> >> >> Google Belgium NV/SA, Steenweg op Etterbeek 180, 1040 >> Brussel, Belgie. RPR: 0878.065.378 >> >> If you received this communication by mistake, please don't forward it to >> anyone else (it may contain confidential or privileged information), please >> erase all copies of it, including all attachments, and please let the >> sender know it went to the wrong person. Thanks >> > >
