Thanks to Kenn, I'm able to move forward on RC2. I will start the RC2 release process in the coming hour.
Regards JB On 02/06/2018 11:17 PM, Kenneth Knowles wrote: > I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621 > > On master it seems that HBase tests are failing reliably, but actually mostly > isolated to this PR. Hopefully the changes on the release branch will go > green. > > On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles <[email protected] > <mailto:[email protected]>> wrote: > > Changes are cherrypicked to the release-2.3.0 branch, or applied directly > there. > > On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov <[email protected] > <mailto:[email protected]>> wrote: > > Possibly stupid question: are new RCs created from master, or by > cherrypicks of desired changes on top of previous RCs? > (it affects whether or not my recently merged change on "master" will > be > in 2.3.0; I'd like it to be :) ) > > On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik <[email protected] > <mailto:[email protected]>> wrote: > > +1 for rollback as well. > > On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax <[email protected] > <mailto:[email protected]>> wrote: > > +1 for the rollback. > > On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles > <[email protected] > <mailto:[email protected]>> wrote: > > +1 for option 1. It is almost certainly repeated proto > parsing. That is fixed for ParDo via generalized caching > but > not for other transforms. Flink also has a proto round > trip, > but the design might avoid the problem anyhow. We should > investigate before releasing, or just preemptively roll it > back too. The proto round trips are helpful for maturing > portability and for preventing hacks from returning but > otherwise don't matter so much yet. > > Kenn > > On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré > <[email protected] <mailto:[email protected]>> wrote: > > Hi all, > > As you might have seen, I found the cause of > performance > degradation on the > direct runner (BEAM-3617). > > We have basically three options for RC2: > > 1. We revert the change for 2.3.0. I think it's the > fastest and more secure way. > I created a PR for that > (https://github.com/apache/beam/pull/4609 > <https://github.com/apache/beam/pull/4609>). It gives > us > time to investigate and provide a better fix in Runner > API for 2.4.0. > > 2. If the fix is easy and with limited impact in > Runner > API, we do that. > > 3. We leave as it is, adding a note that direct runner > has worse performance in > 2.3.0 compared to 2.2.0. > > In any case, I would like to cut RC2 later tonight or > tomorrow morning (my time). > > Thoughts ? > > Regards > JB > > On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote: > > Hi Reuven, > > > > it's what I'm suspecting. git bisect should give us > more information (still in > > progress, 3 more steps to complete ;)). I keep you > posted. > > > > Regards > > JB > > > > On 02/06/2018 08:36 AM, Reuven Lax wrote: > >> Could this be related to any of the portability > changes? > >> > >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste > Onofré > <[email protected] <mailto:[email protected]> > >> <mailto:[email protected] > <mailto:[email protected]>>> wrote: > >> > >> Created: > >> > >> https://issues.apache.org/jira/browse/BEAM-3617 > <https://issues.apache.org/jira/browse/BEAM-3617> > >> > <https://issues.apache.org/jira/browse/BEAM-3617 > <https://issues.apache.org/jira/browse/BEAM-3617>> > >> > >> Regards > >> JB > >> > >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote: > >> > What is the Jira for direct runner perf? > >> > > >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste > Onofré <[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > >> > <mailto:[email protected] > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>>>> wrote: > >> > > >> > Thanks ! > >> > > >> > I cherry-pick on release-2.3.0 branch. > >> > > >> > I'm on the direct runner perf test in the > mean time. > >> > > >> > Thanks again ! > >> > > >> > Regards > >> > JB > >> > > >> > On 02/05/2018 12:06 PM, Aljoscha Krettek > wrote: > >> > > I merged fixes for: > >> > > > - https://issues.apache.org/jira/browse/BEAM-3186 > <https://issues.apache.org/jira/browse/BEAM-3186> > >> > <https://issues.apache.org/jira/browse/BEAM-3186 > <https://issues.apache.org/jira/browse/BEAM-3186>> > >> > > <https://issues.apache.org/jira/browse/BEAM-3186 > <https://issues.apache.org/jira/browse/BEAM-3186> > >> > <https://issues.apache.org/jira/browse/BEAM-3186 > <https://issues.apache.org/jira/browse/BEAM-3186>>> > >> > > > - https://issues.apache.org/jira/browse/BEAM-3589 > <https://issues.apache.org/jira/browse/BEAM-3589> > >> > <https://issues.apache.org/jira/browse/BEAM-3589 > <https://issues.apache.org/jira/browse/BEAM-3589>> > >> > > <https://issues.apache.org/jira/browse/BEAM-3589 > <https://issues.apache.org/jira/browse/BEAM-3589> > >> > <https://issues.apache.org/jira/browse/BEAM-3589 > <https://issues.apache.org/jira/browse/BEAM-3589>>> > >> > > > >> > > @JB I didn't yet merge them on the > 2.3.0 > branch, though, but I can or you can go > >> > > ahead. > >> > > > >> > >> On 5. Feb 2018, at 06:30, > Jean-Baptiste > Onofré <[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > <mailto:[email protected] <mailto:[email protected]> > >> <mailto:[email protected] > <mailto:[email protected]>>> > >> > >> <mailto:[email protected] > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > >> <mailto:[email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>>>> wrote: > >> > >> > >> > >> Hi guys, > >> > >> > >> > >> Quick update on the RC2 preparation: > >> > >> > >> > >> * BEAM-3587 (TextIO with Flink) seems > related to a custom build with > >> > Gradle (not > >> > >> using artifacts created by Maven). > Anyway, I will take a look today. > >> > >> * BEAM-3186 has a PR. Aljoscha will do > the review pretty soon. > >> > >> * I'm also taking a look on the direct > runner performance change. > >> > >> > >> > >> The goal is to cut RC2 tomorrow or the > day after. > >> > >> > >> > >> Regards > >> > >> JB > >> > >> > >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste > Onofré wrote: > >> > >>> Hi guys, > >> > >>> > >> > >>> Especially due to BEAM-3587 & > BEAM-3186 > regressions, I cancel RC1. > >> > >>> > >> > >>> We will cherry-pick fixes on > release-2.3.0 branch. > >> > >>> > >> > >>> I'm updating Jira right now. When the > fixes will be cherry-picked, I > >> > will submit > >> > >>> a RC2 to vote. > >> > >>> > >> > >>> Thanks ! > >> > >>> Regards > >> > >>> JB > >> > >>> > >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste > Onofré wrote: > >> > >>>> Hi everyone, > >> > >>>> > >> > >>>> Please review and vote on the > release > candidate #1 for the version > >> > 2.3.0, as > >> > >>>> follows: > >> > >>>> > >> > >>>> [ ] +1, Approve the release > >> > >>>> [ ] -1, Do not approve the release > (please provide specific comments) > >> > >>>> > >> > >>>> > >> > >>>> The complete staging area is > available > for your review, which > >> includes: > >> > >>>> * JIRA release notes [1], > >> > >>>> * the official Apache source release > to be deployed to > >> dist.apache.org <http://dist.apache.org> > <http://dist.apache.org> > >> > <http://dist.apache.org> > >> > >>>> <http://dist.apache.org> [2], > >> > >>>> which is signed with the key with > fingerprint C8282E76 [3], > >> > >>>> * all artifacts to be deployed to > the > Maven Central Repository [4], > >> > >>>> * source code tag "v2.3.0-RC1" [5], > >> > >>>> * website pull request listing the > release and publishing the API > >> reference > >> > >>>> manual [6]. > >> > >>>> * Java artifacts were built with > Maven > 3.3.9 and Oracle JDK > >> 1.8.0_111. > >> > >>>> * Python artifacts are deployed > along > with the source release to the > >> > >>>> dist.apache.org > <http://dist.apache.org> <http://dist.apache.org> > <http://dist.apache.org> > >> <http://dist.apache.org> [2]. > >> > >>>> > >> > >>>> The vote will be open for at least > 72 > hours. It is adopted by majority approval, > >> > >>>> with at least 3 PMC affirmative > votes. > >> > >>>> > >> > >>>> Thanks, > >> > >>>> JB > >> > >>>> > >> > >>>> [1] > >> > >>>> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608> > >> > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>> > >> > > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608> > >> > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608 > > <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608>>> > >> > >>>> [2] > https://dist.apache.org/repos/dist/dev/beam/2.3.0/ > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/> > >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/ > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>> > >> > > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/ > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/> > >> > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/ > <https://dist.apache.org/repos/dist/dev/beam/2.3.0/>>> > >> > >>>> [3] > https://dist.apache.org/repos/dist/release/beam/KEYS > <https://dist.apache.org/repos/dist/release/beam/KEYS> > >> > <https://dist.apache.org/repos/dist/release/beam/KEYS > > <https://dist.apache.org/repos/dist/release/beam/KEYS>> > >> > > <https://dist.apache.org/repos/dist/release/beam/KEYS > <https://dist.apache.org/repos/dist/release/beam/KEYS> > >> > <https://dist.apache.org/repos/dist/release/beam/KEYS > > <https://dist.apache.org/repos/dist/release/beam/KEYS>>> > >> > >>>> [4] > > https://repository.apache.org/content/repositories/orgapachebeam-1026/ > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/> > >> > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/ > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/>> > >> > > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/ > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/> > >> > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/ > > <https://repository.apache.org/content/repositories/orgapachebeam-1026/>>> > >> > >>>> [5] > https://github.com/apache/beam/tree/v2.3.0-RC1 > <https://github.com/apache/beam/tree/v2.3.0-RC1> > >> <https://github.com/apache/beam/tree/v2.3.0-RC1 > <https://github.com/apache/beam/tree/v2.3.0-RC1>> > >> > > <https://github.com/apache/beam/tree/v2.3.0-RC1 > <https://github.com/apache/beam/tree/v2.3.0-RC1> > >> <https://github.com/apache/beam/tree/v2.3.0-RC1 > <https://github.com/apache/beam/tree/v2.3.0-RC1>>> > >> > >>>> [6] > https://github.com/apache/beam-site/pull/381 > <https://github.com/apache/beam-site/pull/381> > >> <https://github.com/apache/beam-site/pull/381 > <https://github.com/apache/beam-site/pull/381>> > >> > > <https://github.com/apache/beam-site/pull/381 > <https://github.com/apache/beam-site/pull/381> > >> <https://github.com/apache/beam-site/pull/381 > <https://github.com/apache/beam-site/pull/381>>> > >> > >>>> > >> > >>> > >> > >> > >> > >> -- > >> > >> Jean-Baptiste Onofré > >> > >> [email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>> > <mailto:[email protected] > <mailto:[email protected]> > >> <mailto:[email protected] > <mailto:[email protected]>>> > >> > <mailto:[email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>> > >> <mailto:[email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>>> > >> > >> http://blog.nanthrax.net > >> > >> Talend - http://www.talend.com > >> > > > >> > > >> > -- > >> > Jean-Baptiste Onofré > >> > [email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>> > >> <mailto:[email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>> > >> > http://blog.nanthrax.net > >> > Talend - http://www.talend.com > >> > > >> > > >> > >> -- > >> Jean-Baptiste Onofré > >> [email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>> > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > >> > > > > -- > Jean-Baptiste Onofré > [email protected] <mailto:[email protected]> > http://blog.nanthrax.net > Talend - http://www.talend.com > > > > > > -- Jean-Baptiste Onofré [email protected] http://blog.nanthrax.net Talend - http://www.talend.com
