Fully agree. If nobody jumps on it, I will tackle that tomorrow.
Regards JB Le 14 mars 2018 à 18:03, à 18:03, Reuven Lax <re...@google.com> a écrit: >Can you add a description and assign a reviewer (using R:). I think >this >was basically ready to merge before modulo breaking compilation. > > >On Wed, Mar 14, 2018 at 10:01 AM Romain Manni-Bucau ><rmannibu...@gmail.com> >wrote: > >> up, know it missed the 2.4 but can >> https://github.com/apache/beam/pull/4790 have some love? it really >makes >> beam pretty unusable with direct runner, >> I start to have "// workaround for BEAM-3409" everywhere in my >codebase >> which is quite bothering after 3 months :( >> >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibucau> | Blog >> <https://rmannibucau.metawerx.net/> | Old Blog >> <http://rmannibucau.wordpress.com> | Github >> <https://github.com/rmannibucau> | LinkedIn >> <https://www.linkedin.com/in/rmannibucau> | Book >> ><https://www.packtpub.com/application-development/java-ee-8-high-performance> >> >> 2018-03-06 14:44 GMT+01:00 Romain Manni-Bucau ><rmannibu...@gmail.com>: >> >>> Hi guys, >>> >>> tried to reapply the waitUntilFinish fix - without breaking the >>> compilation this time ;) - in >https://github.com/apache/beam/pull/4790, >>> anyone able to help to review and move that forward? >>> >>> >>> Romain Manni-Bucau >>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>> <https://rmannibucau.metawerx.net/> | Old Blog >>> <http://rmannibucau.wordpress.com> | Github >>> <https://github.com/rmannibucau> | LinkedIn >>> <https://www.linkedin.com/in/rmannibucau> | Book >>> ><https://www.packtpub.com/application-development/java-ee-8-high-performance> >>> >>> 2018-03-02 9:28 GMT+01:00 Robert Bradshaw <rober...@google.com>: >>> >>>> On Fri, Mar 2, 2018 at 12:01 AM Jean-Baptiste Onofré ><j...@nanthrax.net> >>>> wrote: >>>> >>>>> Hi Ismaël, >>>>> >>>>> that's a good idea to show history. >>>>> >>>>> For me, the vote duration doesn't matter as we are in the release >>>>> process already. >>>>> >>>> >>>> A more relevant duration to track would probably be cut to final >>>> release. This both measures our investment in the release process, >as well >>>> as how behind head is when we finally get the release out. >>>> >>>> >>>>> The gap between two releases is more significant. >>>> >>>> >>>> +1, this is what users see. (To clarify terminology, the "time >between >>>> release" is the time between actually releasing x.y and x.y+1 that >is most >>>> visible to end users, regardless of intermediate process like >cutting and >>>> voting that we have.) Of course this gets thrown off if our >>>> time-to-prepare-a-release becomes a significant fraction of the >desired >>>> time-between-releases. >>>> >>>> And clearly with an average of >>>>> 80 days (~ 3 months) it's two long. The idea is to reduce this >clearly. >>>>> I >>>>> propose two months previously (including the vote period), so >meaning 6 >>>>> weeks >>>>> between releases. >>>>> >>>> >>>> It seems there have been proposals for monthly, every 6 weeks >>>> (sesquimonthly?), and bimonthly. >>>> >>>> >>>>> Regarding the time we take for the PR review and merge, I think >it's a >>>>> fair time >>>>> to give us time to include new improvements and features, but also >to >>>>> fix bugs. >>>>> >>>> >>>> Concrete deadlines can provide good motivation to get around to >doing >>>> reviews, cleaning up PRs, fixing bugs, etc. that you've been >meaning to do >>>> but for whatever reason keep putting off. So I think it's still >good >>>> practice to have some lead time that a release is coming for a >chance for >>>> folks to get stuff in, while still being clear that we're not >holding >>>> things back for new features and if you don't make the cut another >one is >>>> close behind. >>>> >>>> Regards >>>>> JB >>>>> >>>>> On 03/01/2018 06:17 PM, Ismaël Mejía wrote: >>>>> > The average time just in the vote process for Beam since we are >out >>>>> of the >>>>> > incubator is 17.5 days with an average of 75 days between >versions. >>>>> > >>>>> > Version Vote Period No. days >>>>> > 2.3.0 30/01-16/02 17 days (83 days since last) >>>>> > 2.2.0 27/10-25/11 29 days (101 days since last) >>>>> > 2.1.0 11/07-16/08 36 days (92 days since last) >>>>> > 2.0.0 08/05-16/05 8 days (62 days since last) >>>>> > 0.6.0 10/03-15/03 5 days (37 days since last) >>>>> > 0.5.0 27/01-06/02 10 days >>>>> > >>>>> > I think we should have these numbers into account to refine the >>>>> distance between >>>>> > releases. If we want to follow strict time-based releases, what >we >>>>> can probably >>>>> > refine is how we cut the release so we try to reduce release >overlaps >>>>> and avoid >>>>> > rushing unnecessarily. >>>>> > >>>>> > Maybe we should follow the proposed 6 weeks for the next release >like >>>>> this: >>>>> > >>>>> > - 4 weeks let’s say just after succesful vote and then cut the >>>>> release branch. >>>>> > - 1 week to burn the blocker list (good to have ones that don’t >make >>>>> will be >>>>> > moved to the next release). >>>>> > - 1 week for the vote + RCs (in case the vote takes longer at >least >>>>> the overlap >>>>> > between vote + next dev cycle will be smaller). >>>>> > >>>>> > If we do this for the next cycle we will have a 6 week ‘dev’ >period >>>>> and then we >>>>> > will have optimistically an average of 2 weeks of ‘releasing’ >and 6 >>>>> weeks ‘dev’ >>>>> > cycles. >>>>> > >>>>> > On Thu, Mar 1, 2018 at 6:46 AM, Jean-Baptiste Onofré ><j...@nanthrax.net> >>>>> wrote: >>>>> >> About BEAM-3409, I did a review yesterday and it looks good to >me. >>>>> We are >>>>> >> waiting for Thomas' feedback. >>>>> >> >>>>> >> Regards >>>>> >> JB >>>>> >> Le 1 mars 2018, à 09:38, Robert Bradshaw <rober...@google.com> >a >>>>> écrit: >>>>> >>> >>>>> >>> Looking at the burn-down list, we have 5 remaining issues. >None of >>>>> these >>>>> >>> are blockers, but all look like they're really close >(reviewed, >>>>> review >>>>> >>> comments were addressed, waiting for a final LGTM). >Specifically: >>>>> >>> >>>>> >>> BEAM-3409 (teardown issues): Thomas Groh had some concerns, >could >>>>> you >>>>> >>> verify they have all been addressed? >>>>> >>> BEAM-3479 (DoFn classloader regression test): Kenn Knowles >had >>>>> minor >>>>> >>> comments, looks like they were addressed, could you confirm? >>>>> >>> BEAM-3735 (Missing gaming release archetypes): Lukasz Cwik had >minor >>>>> >>> comments, looks like they were addressed, could you confirm? >>>>> >>> BEAM-3611 (KafkaIO.java splitting): Looks like this was >resolved. >>>>> >>> BEAM-3762 (unlimited JCE for Dataflow Worker): LGTM pending >>>>> (currently >>>>> >>> running) tests passing. >>>>> >>> >>>>> >>> Let's see how many of these we can get in by, say, noon PST >>>>> tomorrow. >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> On Wed, Feb 28, 2018 at 9:26 PM Robert Bradshaw < >>>>> rober...@google.com> >>>>> >>> wrote: >>>>> >>>> >>>>> >>>> I tend to fall into the "release early, release often" camp >in >>>>> general, >>>>> >>>> but for this one I'm particularly anxious to get the faster >Python >>>>> direct >>>>> >>>> runner out in the hands of TFT/TFX users (and in particular >have >>>>> an eye on >>>>> >>>> https://www.tensorflow.org/dev-summit/ which I think can be a >>>>> healthy source >>>>> >>>> of Beam users). >>>>> >>>> >>>>> >>>> >>>>> >>>> On Wed, Feb 28, 2018 at 7:01 PM Jean-Baptiste Onofré < >>>>> j...@nanthrax.net> >>>>> >>>> wrote: >>>>> >>>>> >>>>> >>>>> Hi Gus, >>>>> >>>>> >>>>> >>>>> Thanks for the update, it makes sense. >>>>> >>>>> >>>>> >>>>> Regards >>>>> >>>>> JB >>>>> >>>>> >>>>> >>>>> On 03/01/2018 02:59 AM, Konstantinos Katsiapis wrote: >>>>> >>>>>> Hi Jean-Baptiste, >>>>> >>>>>> >>>>> >>>>>> I can speak from the perspective of tf.transform >>>>> >>>>>> < https://github.com/tensorflow/transform> (TFT) in >particular >>>>> and TFX >>>>> >>>>>> < https://research.google.com/pubs/pub46484.html> libs in >>>>> general, in >>>>> >>>>>> case it is >>>>> >>>>>> useful. >>>>> >>>>>> >>>>> >>>>>> TFX distributed computation has 2 "large" dependencies, >namely >>>>> >>>>>> TensorFlow and >>>>> >>>>>> Apache Beam, each on their own release schedule. >>>>> >>>>>> As such, releasing of new TFX functionality often (but not >>>>> always) >>>>> >>>>>> depends on >>>>> >>>>>> (and is blocked by) releases of *both* TensorFlow *and* >Apache >>>>> Beam. >>>>> >>>>>> >>>>> >>>>>> Synchronizing releases across such large projects and >>>>> organizations is >>>>> >>>>>> likely >>>>> >>>>>> hard, so from our perspective having *frequent* releases of >>>>> Tensorflow >>>>> >>>>>> or Apache >>>>> >>>>>> Beam (and better yet both) decreases the time for which we >are >>>>> blocked >>>>> >>>>>> on >>>>> >>>>>> releasing our features. >>>>> >>>>>> >>>>> >>>>>> In light of this, I would vote for more frequent releases >in >>>>> general, >>>>> >>>>>> and for a >>>>> >>>>>> Beam 2.4 release soon in particular (as TFT 0.6 depends on >it). >>>>> >>>>>> >>>>> >>>>>> Thanks, >>>>> >>>>>> Gus >>>>> >>>>>> >>>>> >>>>>> On Tue, Feb 27, 2018 at 10:29 PM, Jean-Baptiste Onofré < >>>>> >>>>>> j...@nanthrax.net >>>>> >>>>>> <mailto: j...@nanthrax.net>> wrote: >>>>> >>>>>> >>>>> >>>>>> By the way, if third party projects based on Beam >(Google >>>>> >>>>>> Dataflow, Talend >>>>> >>>>>> DataStream, and others) need a release (including some >>>>> features), >>>>> >>>>>> it's better to >>>>> >>>>>> clearly state this on the mailing list. >>>>> >>>>>> >>>>> >>>>>> At Apache Karaf, I have lot of projects based on it >>>>> (OpenDaylight, >>>>> >>>>>> OpenHAB, >>>>> >>>>>> Websphere, ...). They just ask for the release >schedule and >>>>> they >>>>> >>>>>> align with >>>>> >>>>>> these release. As a best effort, I'm always trying to >move >>>>> fast >>>>> >>>>>> when a release >>>>> >>>>>> is asked. >>>>> >>>>>> >>>>> >>>>>> So, if 2.4.0 is required by third party, no problem to >"ask >>>>> for a >>>>> >>>>>> release". >>>>> >>>>>> >>>>> >>>>>> Regards >>>>> >>>>>> JB >>>>> >>>>>> >>>>> >>>>>> On 02/28/2018 04:17 AM, Reuven Lax wrote: >>>>> >>>>>> > It's been six weeks since you proposed beam 2.3.0. so >>>>> assuming >>>>> >>>>>> the same time >>>>> >>>>>> > scale for this release, that's 1.5 months between >releases. >>>>> >>>>>> Slightly faster than >>>>> >>>>>> > 2 months, but not by much. >>>>> >>>>>> > >>>>> >>>>>> > I do seem to remember that the original goal for beam >was >>>>> >>>>>> monthly releases though. >>>>> >>>>>> > >>>>> >>>>>> > Reuven >>>>> >>>>>> > >>>>> >>>>>> > On Tue, Feb 27, 2018, 9:12 PM Jean-Baptiste Onofré < >>>>> >>>>>> j...@nanthrax.net <mailto: j...@nanthrax.net> >>>>> >>>>>> > <mailto: j...@nanthrax.net <mailto: j...@nanthrax.net>>> >>>>> wrote: >>>>> >>>>>> > >>>>> >>>>>> > Hi Reuven, >>>>> >>>>>> > >>>>> >>>>>> > In a previous thread (about Beam project >execution), I >>>>> >>>>>> proposed a release every >>>>> >>>>>> > two months (as a best effort), I will find the >e-mail. >>>>> >>>>>> > >>>>> >>>>>> > Beam 2.3.0 has been released "officially" on >February >>>>> 16th, >>>>> >>>>>> so two week ago >>>>> >>>>>> > roughly. I would have expected 2.4.0 not before >end of >>>>> >>>>>> March. >>>>> >>>>>> > >>>>> >>>>>> > If we have issue we want to fix fast, then 2.3.1 >is >>>>> good. If >>>>> >>>>>> it's a new release >>>>> >>>>>> > in the pace, it's pretty fast and might "confuse" >our >>>>> users. >>>>> >>>>>> > >>>>> >>>>>> > That's why I'm curious ;) >>>>> >>>>>> > >>>>> >>>>>> > Regards >>>>> >>>>>> > JB >>>>> >>>>>> > >>>>> >>>>>> > On 02/28/2018 03:50 AM, Reuven Lax wrote: >>>>> >>>>>> > > Wasn't the original statement monthly releases? >We've >>>>> >>>>>> never realistically >>>>> >>>>>> > > managed that, but Robert's proposed cut will be >on a >>>>> >>>>>> 6-week pace. >>>>> >>>>>> > > >>>>> >>>>>> > > On Tue, Feb 27, 2018, 8:48 PM Jean-Baptiste >Onofré < >>>>> >>>>>> j...@nanthrax.net <mailto: j...@nanthrax.net> >>>>> >>>>>> > <mailto: j...@nanthrax.net <mailto: >j...@nanthrax.net>> >>>>> >>>>>> > > <mailto: j...@nanthrax.net <mailto: >j...@nanthrax.net> >>>>> >>>>>> <mailto: j...@nanthrax.net >>>>> >>>>>> <mailto: j...@nanthrax.net>>>> wrote: >>>>> >>>>>> > > >>>>> >>>>>> > > Hi Robert, >>>>> >>>>>> > > >>>>> >>>>>> > > I'm just curious: it's pretty fast compared >to >>>>> the >>>>> >>>>>> original plan of a >>>>> >>>>>> > release >>>>> >>>>>> > > every two months. What's the reason to cut >2.4.0 >>>>> now >>>>> >>>>>> instead of end of >>>>> >>>>>> > March ? >>>>> >>>>>> > > >>>>> >>>>>> > > I will do the Jira triage and update today. >>>>> >>>>>> > > >>>>> >>>>>> > > Regards >>>>> >>>>>> > > JB >>>>> >>>>>> > > >>>>> >>>>>> > > On 02/27/2018 09:21 PM, Robert Bradshaw >wrote: >>>>> >>>>>> > > > I'm planning on cutting the 2.4.0 release >>>>> branch >>>>> >>>>>> soon (tomorrow?). I >>>>> >>>>>> > see 13 >>>>> >>>>>> > > > open issues on JIRA [1], none of which >are >>>>> labeled >>>>> >>>>>> as blockers. If there >>>>> >>>>>> > > > are any that cannot be bumped to the next >>>>> release, >>>>> >>>>>> let me know soon. >>>>> >>>>>> > > > >>>>> >>>>>> > > > - Robert >>>>> >>>>>> > > > >>>>> >>>>>> > > > >>>>> >>>>>> > > > [1] >>>>> >>>>>> > > > >>>>> >>>>>> > > >>>>> >>>>>> > >>>>> >>>>>> >>>>> >https://issues.apache.org/jira/browse/BEAM-3749?jql=project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0 >>>>> >>>>>> < >>>>> >>>>>> >>>>> >https://issues.apache.org/jira/browse/BEAM-3749?jql=project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0 >>>>> > >>>>> >>>>>> > > > >>>>> >>>>>> > > >>>>> >>>>>> > > -- >>>>> >>>>>> > > Jean-Baptiste Onofré >>>>> >>>>>> > > jbono...@apache.org <mailto: >>>>> jbono...@apache.org> >>>>> >>>>>> <mailto: jbono...@apache.org >>>>> >>>>>> <mailto: jbono...@apache.org>> >>>>> >>>>>> > <mailto: jbono...@apache.org <mailto: >>>>> jbono...@apache.org> >>>>> >>>>>> <mailto: jbono...@apache.org <mailto: >jbono...@apache.org>>> >>>>> >>>>>> > > http://blog.nanthrax.net >>>>> >>>>>> > > Talend - http://www.talend.com >>>>> >>>>>> > > >>>>> >>>>>> > >>>>> >>>>>> > -- >>>>> >>>>>> > Jean-Baptiste Onofré >>>>> >>>>>> > jbono...@apache.org <mailto: >jbono...@apache.org> >>>>> >>>>>> <mailto: jbono...@apache.org <mailto: >jbono...@apache.org>> >>>>> >>>>>> > http://blog.nanthrax.net >>>>> >>>>>> > Talend - http://www.talend.com >>>>> >>>>>> > >>>>> >>>>>> >>>>> >>>>>> -- >>>>> >>>>>> Jean-Baptiste Onofré >>>>> >>>>>> jbono...@apache.org <mailto: jbono...@apache.org> >>>>> >>>>>> http://blog.nanthrax.net >>>>> >>>>>> Talend - http://www.talend.com >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> -- >>>>> >>>>>> Gus Katsiapis | Software Engineer | katsia...@google.com >>>>> >>>>>> <mailto: katsia...@google.com> | 650-918-7487 >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Jean-Baptiste Onofré >>>>> >>>>> jbono...@apache.org >>>>> >>>>> http://blog.nanthrax.net >>>>> >>>>> Talend - http://www.talend.com >>>>> >>>>> -- >>>>> Jean-Baptiste Onofré >>>>> jbono...@apache.org >>>>> http://blog.nanthrax.net >>>>> Talend - http://www.talend.com >>>>> >>>> >>> >>
