Can you add a description and assign a reviewer (using R:). I think this was basically ready to merge before modulo breaking compilation.
On Wed, Mar 14, 2018 at 10:01 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > up, know it missed the 2.4 but can > https://github.com/apache/beam/pull/4790 have some love? it really makes > beam pretty unusable with direct runner, > I start to have "// workaround for BEAM-3409" everywhere in my codebase > which is quite bothering after 3 months :( > > > Romain Manni-Bucau > @rmannibucau <https://twitter.com/rmannibucau> | Blog > <https://rmannibucau.metawerx.net/> | Old Blog > <http://rmannibucau.wordpress.com> | Github > <https://github.com/rmannibucau> | LinkedIn > <https://www.linkedin.com/in/rmannibucau> | Book > <https://www.packtpub.com/application-development/java-ee-8-high-performance> > > 2018-03-06 14:44 GMT+01:00 Romain Manni-Bucau <rmannibu...@gmail.com>: > >> Hi guys, >> >> tried to reapply the waitUntilFinish fix - without breaking the >> compilation this time ;) - in https://github.com/apache/beam/pull/4790, >> anyone able to help to review and move that forward? >> >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibucau> | Blog >> <https://rmannibucau.metawerx.net/> | Old Blog >> <http://rmannibucau.wordpress.com> | Github >> <https://github.com/rmannibucau> | LinkedIn >> <https://www.linkedin.com/in/rmannibucau> | Book >> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >> >> 2018-03-02 9:28 GMT+01:00 Robert Bradshaw <rober...@google.com>: >> >>> On Fri, Mar 2, 2018 at 12:01 AM Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>>> Hi Ismaël, >>>> >>>> that's a good idea to show history. >>>> >>>> For me, the vote duration doesn't matter as we are in the release >>>> process already. >>>> >>> >>> A more relevant duration to track would probably be cut to final >>> release. This both measures our investment in the release process, as well >>> as how behind head is when we finally get the release out. >>> >>> >>>> The gap between two releases is more significant. >>> >>> >>> +1, this is what users see. (To clarify terminology, the "time between >>> release" is the time between actually releasing x.y and x.y+1 that is most >>> visible to end users, regardless of intermediate process like cutting and >>> voting that we have.) Of course this gets thrown off if our >>> time-to-prepare-a-release becomes a significant fraction of the desired >>> time-between-releases. >>> >>> And clearly with an average of >>>> 80 days (~ 3 months) it's two long. The idea is to reduce this clearly. >>>> I >>>> propose two months previously (including the vote period), so meaning 6 >>>> weeks >>>> between releases. >>>> >>> >>> It seems there have been proposals for monthly, every 6 weeks >>> (sesquimonthly?), and bimonthly. >>> >>> >>>> Regarding the time we take for the PR review and merge, I think it's a >>>> fair time >>>> to give us time to include new improvements and features, but also to >>>> fix bugs. >>>> >>> >>> Concrete deadlines can provide good motivation to get around to doing >>> reviews, cleaning up PRs, fixing bugs, etc. that you've been meaning to do >>> but for whatever reason keep putting off. So I think it's still good >>> practice to have some lead time that a release is coming for a chance for >>> folks to get stuff in, while still being clear that we're not holding >>> things back for new features and if you don't make the cut another one is >>> close behind. >>> >>> Regards >>>> JB >>>> >>>> On 03/01/2018 06:17 PM, Ismaël Mejía wrote: >>>> > The average time just in the vote process for Beam since we are out >>>> of the >>>> > incubator is 17.5 days with an average of 75 days between versions. >>>> > >>>> > Version Vote Period No. days >>>> > 2.3.0 30/01-16/02 17 days (83 days since last) >>>> > 2.2.0 27/10-25/11 29 days (101 days since last) >>>> > 2.1.0 11/07-16/08 36 days (92 days since last) >>>> > 2.0.0 08/05-16/05 8 days (62 days since last) >>>> > 0.6.0 10/03-15/03 5 days (37 days since last) >>>> > 0.5.0 27/01-06/02 10 days >>>> > >>>> > I think we should have these numbers into account to refine the >>>> distance between >>>> > releases. If we want to follow strict time-based releases, what we >>>> can probably >>>> > refine is how we cut the release so we try to reduce release overlaps >>>> and avoid >>>> > rushing unnecessarily. >>>> > >>>> > Maybe we should follow the proposed 6 weeks for the next release like >>>> this: >>>> > >>>> > - 4 weeks let’s say just after succesful vote and then cut the >>>> release branch. >>>> > - 1 week to burn the blocker list (good to have ones that don’t make >>>> will be >>>> > moved to the next release). >>>> > - 1 week for the vote + RCs (in case the vote takes longer at least >>>> the overlap >>>> > between vote + next dev cycle will be smaller). >>>> > >>>> > If we do this for the next cycle we will have a 6 week ‘dev’ period >>>> and then we >>>> > will have optimistically an average of 2 weeks of ‘releasing’ and 6 >>>> weeks ‘dev’ >>>> > cycles. >>>> > >>>> > On Thu, Mar 1, 2018 at 6:46 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >>>> wrote: >>>> >> About BEAM-3409, I did a review yesterday and it looks good to me. >>>> We are >>>> >> waiting for Thomas' feedback. >>>> >> >>>> >> Regards >>>> >> JB >>>> >> Le 1 mars 2018, à 09:38, Robert Bradshaw <rober...@google.com> a >>>> écrit: >>>> >>> >>>> >>> Looking at the burn-down list, we have 5 remaining issues. None of >>>> these >>>> >>> are blockers, but all look like they're really close (reviewed, >>>> review >>>> >>> comments were addressed, waiting for a final LGTM). Specifically: >>>> >>> >>>> >>> BEAM-3409 (teardown issues): Thomas Groh had some concerns, could >>>> you >>>> >>> verify they have all been addressed? >>>> >>> BEAM-3479 (DoFn classloader regression test): Kenn Knowles had >>>> minor >>>> >>> comments, looks like they were addressed, could you confirm? >>>> >>> BEAM-3735 (Missing gaming release archetypes): Lukasz Cwik had minor >>>> >>> comments, looks like they were addressed, could you confirm? >>>> >>> BEAM-3611 (KafkaIO.java splitting): Looks like this was resolved. >>>> >>> BEAM-3762 (unlimited JCE for Dataflow Worker): LGTM pending >>>> (currently >>>> >>> running) tests passing. >>>> >>> >>>> >>> Let's see how many of these we can get in by, say, noon PST >>>> tomorrow. >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>> On Wed, Feb 28, 2018 at 9:26 PM Robert Bradshaw < >>>> rober...@google.com> >>>> >>> wrote: >>>> >>>> >>>> >>>> I tend to fall into the "release early, release often" camp in >>>> general, >>>> >>>> but for this one I'm particularly anxious to get the faster Python >>>> direct >>>> >>>> runner out in the hands of TFT/TFX users (and in particular have >>>> an eye on >>>> >>>> https://www.tensorflow.org/dev-summit/ which I think can be a >>>> healthy source >>>> >>>> of Beam users). >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Feb 28, 2018 at 7:01 PM Jean-Baptiste Onofré < >>>> j...@nanthrax.net> >>>> >>>> wrote: >>>> >>>>> >>>> >>>>> Hi Gus, >>>> >>>>> >>>> >>>>> Thanks for the update, it makes sense. >>>> >>>>> >>>> >>>>> Regards >>>> >>>>> JB >>>> >>>>> >>>> >>>>> On 03/01/2018 02:59 AM, Konstantinos Katsiapis wrote: >>>> >>>>>> Hi Jean-Baptiste, >>>> >>>>>> >>>> >>>>>> I can speak from the perspective of tf.transform >>>> >>>>>> < https://github.com/tensorflow/transform> (TFT) in particular >>>> and TFX >>>> >>>>>> < https://research.google.com/pubs/pub46484.html> libs in >>>> general, in >>>> >>>>>> case it is >>>> >>>>>> useful. >>>> >>>>>> >>>> >>>>>> TFX distributed computation has 2 "large" dependencies, namely >>>> >>>>>> TensorFlow and >>>> >>>>>> Apache Beam, each on their own release schedule. >>>> >>>>>> As such, releasing of new TFX functionality often (but not >>>> always) >>>> >>>>>> depends on >>>> >>>>>> (and is blocked by) releases of *both* TensorFlow *and* Apache >>>> Beam. >>>> >>>>>> >>>> >>>>>> Synchronizing releases across such large projects and >>>> organizations is >>>> >>>>>> likely >>>> >>>>>> hard, so from our perspective having *frequent* releases of >>>> Tensorflow >>>> >>>>>> or Apache >>>> >>>>>> Beam (and better yet both) decreases the time for which we are >>>> blocked >>>> >>>>>> on >>>> >>>>>> releasing our features. >>>> >>>>>> >>>> >>>>>> In light of this, I would vote for more frequent releases in >>>> general, >>>> >>>>>> and for a >>>> >>>>>> Beam 2.4 release soon in particular (as TFT 0.6 depends on it). >>>> >>>>>> >>>> >>>>>> Thanks, >>>> >>>>>> Gus >>>> >>>>>> >>>> >>>>>> On Tue, Feb 27, 2018 at 10:29 PM, Jean-Baptiste Onofré < >>>> >>>>>> j...@nanthrax.net >>>> >>>>>> <mailto: j...@nanthrax.net>> wrote: >>>> >>>>>> >>>> >>>>>> By the way, if third party projects based on Beam (Google >>>> >>>>>> Dataflow, Talend >>>> >>>>>> DataStream, and others) need a release (including some >>>> features), >>>> >>>>>> it's better to >>>> >>>>>> clearly state this on the mailing list. >>>> >>>>>> >>>> >>>>>> At Apache Karaf, I have lot of projects based on it >>>> (OpenDaylight, >>>> >>>>>> OpenHAB, >>>> >>>>>> Websphere, ...). They just ask for the release schedule and >>>> they >>>> >>>>>> align with >>>> >>>>>> these release. As a best effort, I'm always trying to move >>>> fast >>>> >>>>>> when a release >>>> >>>>>> is asked. >>>> >>>>>> >>>> >>>>>> So, if 2.4.0 is required by third party, no problem to "ask >>>> for a >>>> >>>>>> release". >>>> >>>>>> >>>> >>>>>> Regards >>>> >>>>>> JB >>>> >>>>>> >>>> >>>>>> On 02/28/2018 04:17 AM, Reuven Lax wrote: >>>> >>>>>> > It's been six weeks since you proposed beam 2.3.0. so >>>> assuming >>>> >>>>>> the same time >>>> >>>>>> > scale for this release, that's 1.5 months between releases. >>>> >>>>>> Slightly faster than >>>> >>>>>> > 2 months, but not by much. >>>> >>>>>> > >>>> >>>>>> > I do seem to remember that the original goal for beam was >>>> >>>>>> monthly releases though. >>>> >>>>>> > >>>> >>>>>> > Reuven >>>> >>>>>> > >>>> >>>>>> > On Tue, Feb 27, 2018, 9:12 PM Jean-Baptiste Onofré < >>>> >>>>>> j...@nanthrax.net <mailto: j...@nanthrax.net> >>>> >>>>>> > <mailto: j...@nanthrax.net <mailto: j...@nanthrax.net>>> >>>> wrote: >>>> >>>>>> > >>>> >>>>>> > Hi Reuven, >>>> >>>>>> > >>>> >>>>>> > In a previous thread (about Beam project execution), I >>>> >>>>>> proposed a release every >>>> >>>>>> > two months (as a best effort), I will find the e-mail. >>>> >>>>>> > >>>> >>>>>> > Beam 2.3.0 has been released "officially" on February >>>> 16th, >>>> >>>>>> so two week ago >>>> >>>>>> > roughly. I would have expected 2.4.0 not before end of >>>> >>>>>> March. >>>> >>>>>> > >>>> >>>>>> > If we have issue we want to fix fast, then 2.3.1 is >>>> good. If >>>> >>>>>> it's a new release >>>> >>>>>> > in the pace, it's pretty fast and might "confuse" our >>>> users. >>>> >>>>>> > >>>> >>>>>> > That's why I'm curious ;) >>>> >>>>>> > >>>> >>>>>> > Regards >>>> >>>>>> > JB >>>> >>>>>> > >>>> >>>>>> > On 02/28/2018 03:50 AM, Reuven Lax wrote: >>>> >>>>>> > > Wasn't the original statement monthly releases? We've >>>> >>>>>> never realistically >>>> >>>>>> > > managed that, but Robert's proposed cut will be on a >>>> >>>>>> 6-week pace. >>>> >>>>>> > > >>>> >>>>>> > > On Tue, Feb 27, 2018, 8:48 PM Jean-Baptiste Onofré < >>>> >>>>>> j...@nanthrax.net <mailto: j...@nanthrax.net> >>>> >>>>>> > <mailto: j...@nanthrax.net <mailto: j...@nanthrax.net>> >>>> >>>>>> > > <mailto: j...@nanthrax.net <mailto: j...@nanthrax.net> >>>> >>>>>> <mailto: j...@nanthrax.net >>>> >>>>>> <mailto: j...@nanthrax.net>>>> wrote: >>>> >>>>>> > > >>>> >>>>>> > > Hi Robert, >>>> >>>>>> > > >>>> >>>>>> > > I'm just curious: it's pretty fast compared to >>>> the >>>> >>>>>> original plan of a >>>> >>>>>> > release >>>> >>>>>> > > every two months. What's the reason to cut 2.4.0 >>>> now >>>> >>>>>> instead of end of >>>> >>>>>> > March ? >>>> >>>>>> > > >>>> >>>>>> > > I will do the Jira triage and update today. >>>> >>>>>> > > >>>> >>>>>> > > Regards >>>> >>>>>> > > JB >>>> >>>>>> > > >>>> >>>>>> > > On 02/27/2018 09:21 PM, Robert Bradshaw wrote: >>>> >>>>>> > > > I'm planning on cutting the 2.4.0 release >>>> branch >>>> >>>>>> soon (tomorrow?). I >>>> >>>>>> > see 13 >>>> >>>>>> > > > open issues on JIRA [1], none of which are >>>> labeled >>>> >>>>>> as blockers. If there >>>> >>>>>> > > > are any that cannot be bumped to the next >>>> release, >>>> >>>>>> let me know soon. >>>> >>>>>> > > > >>>> >>>>>> > > > - Robert >>>> >>>>>> > > > >>>> >>>>>> > > > >>>> >>>>>> > > > [1] >>>> >>>>>> > > > >>>> >>>>>> > > >>>> >>>>>> > >>>> >>>>>> >>>> https://issues.apache.org/jira/browse/BEAM-3749?jql=project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0 >>>> >>>>>> < >>>> >>>>>> >>>> https://issues.apache.org/jira/browse/BEAM-3749?jql=project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0 >>>> > >>>> >>>>>> > > > >>>> >>>>>> > > >>>> >>>>>> > > -- >>>> >>>>>> > > Jean-Baptiste Onofré >>>> >>>>>> > > jbono...@apache.org <mailto: >>>> jbono...@apache.org> >>>> >>>>>> <mailto: jbono...@apache.org >>>> >>>>>> <mailto: jbono...@apache.org>> >>>> >>>>>> > <mailto: jbono...@apache.org <mailto: >>>> jbono...@apache.org> >>>> >>>>>> <mailto: jbono...@apache.org <mailto: jbono...@apache.org>>> >>>> >>>>>> > > http://blog.nanthrax.net >>>> >>>>>> > > Talend - http://www.talend.com >>>> >>>>>> > > >>>> >>>>>> > >>>> >>>>>> > -- >>>> >>>>>> > Jean-Baptiste Onofré >>>> >>>>>> > jbono...@apache.org <mailto: jbono...@apache.org> >>>> >>>>>> <mailto: jbono...@apache.org <mailto: jbono...@apache.org>> >>>> >>>>>> > http://blog.nanthrax.net >>>> >>>>>> > Talend - http://www.talend.com >>>> >>>>>> > >>>> >>>>>> >>>> >>>>>> -- >>>> >>>>>> Jean-Baptiste Onofré >>>> >>>>>> jbono...@apache.org <mailto: jbono...@apache.org> >>>> >>>>>> http://blog.nanthrax.net >>>> >>>>>> Talend - http://www.talend.com >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> -- >>>> >>>>>> Gus Katsiapis | Software Engineer | katsia...@google.com >>>> >>>>>> <mailto: katsia...@google.com> | 650-918-7487 >>>> >>>>> >>>> >>>>> -- >>>> >>>>> Jean-Baptiste Onofré >>>> >>>>> jbono...@apache.org >>>> >>>>> http://blog.nanthrax.net >>>> >>>>> Talend - http://www.talend.com >>>> >>>> -- >>>> Jean-Baptiste Onofré >>>> jbono...@apache.org >>>> http://blog.nanthrax.net >>>> Talend - http://www.talend.com >>>> >>> >> >
