I pushed binaries to the repositories. I started a blog post draft, please feel to make any changes directly, or comment on it [1]. I plan to publish the blog post along with an email to user@ on Monday 10/29.
Ahmet [1] https://github.com/apache/beam/pull/6852 On Fri, Oct 26, 2018 at 10:16 AM, Ahmet Altay <[email protected]> wrote: > +1 (binding) > > Thank you all for running validations and voting. > > I'm pleased to announce that the 2.8.0 RC1 is approved for release with 5 > +1 votes (4 binding) and no -1 votes. I will start pushing the bits > around. > > On Fri, Oct 26, 2018 at 9:20 AM, Maximilian Michels <[email protected]> > wrote: > >> +1 (binding) >> >> On 26.10.18 17:45, Kenneth Knowles wrote: >> >>> Nice. Thanks. >>> >>> +1 >>> >>> >>> On Fri, Oct 26, 2018 at 8:44 AM Robert Bradshaw <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Thanks Tim! >>> >>> This was my only hesitation, and sounds like we're in the clear here. >>> >>> +1 (binding) >>> On Fri, Oct 26, 2018 at 5:05 PM Tim Robertson >>> <[email protected] <mailto:[email protected]>> >>> wrote: >>> > >>> > A colleague and I tested on 2.7.0 and 2.8.0RC1: >>> > >>> > 1. Quickstart on Spark/YARN/HDFS (CDH 5.12.0) (commented in >>> spreadsheet) >>> > 2. Our Avro to Avro pipelines on Spark/YARN/HDFS (note we >>> backport the un-merged BEAM-5036 fix in our code) >>> > 3. Our Avro to Elasticsearch pipelines on Spark/YARN/HDFS >>> > >>> > Everything worked, and performance was similar on both. >>> > We built using maven pointing at >>> https://repository.apache.org/content/repositories/orgapache >>> beam-1049/ >>> > >>> > Based on this limited testing: +1 >>> > >>> > Thank you to the release managers, >>> > Tim >>> > >>> > >>> > On Thu, Oct 25, 2018 at 7:21 PM Tim <[email protected] >>> <mailto:[email protected]>> wrote: >>> >> >>> >> I can do some tests on Spark / YARN tomorrow (CEST timezone). >>> Sorry I’ve just been too busy to assist. >>> >> >>> >> Tim >>> >> >>> >> On 25 Oct 2018, at 18:59, Kenneth Knowles <[email protected] >>> <mailto:[email protected]>> wrote: >>> >> >>> >> I tried to do a more thorough job on this. >>> >> >>> >> - I could not reproduce the slowdown in Query 9. I believe the >>> variance was simply high given the parameters and environment >>> >> - I saw the same slowdown in Query 8 when running as part of >>> the suite, but it vanished when I ran repeatedly on its own, so >>> again it is not good methodology probably >>> >> >>> >> We do have the dashboard at >>> https://apache-beam-testing.appspot.com/dashboard-admin though no >>> anomaly detection set up AFAIK. >>> >> >>> >> - There is no issue easily visible in DirectRunner: >>> https://apache-beam-testing.appspot.com/explore?dashboard=50 >>> 84698770407424 >>> >> - There is a notable degradation in Spark runner on 10/5 for >>> many queries. >>> https://apache-beam-testing.appspot.com/explore?dashboard=51 >>> 38380291571712 >>> >> - Something minor happened for Dataflow around 10/1: >>> https://apache-beam-testing.appspot.com/explore?dashboard=56 >>> 70405876482048 >>> >> - Flink runner seems to have had some fantastic improvements >>> :-) >>> https://apache-beam-testing.appspot.com/explore?dashboard=56 >>> 99257587728384 >>> >> >>> >> So if there is a blocker it would really be the Spark runner >>> perf changes. Of course, all these except Dataflow are using local >>> instances so may not be representative of larger scale AFAIK. >>> >> >>> >> Kenn >>> >> >>> >> On Wed, Oct 24, 2018 at 9:48 AM Maximilian Michels >>> <[email protected] <mailto:[email protected]>> wrote: >>> >>> >>> >>> I've run WordCount using Quickstart with the FlinkRunner >>> (locally and >>> >>> against a Flink cluster). >>> >>> >>> >>> Would give a +1 but waiting what Kenn finds. >>> >>> >>> >>> -Max >>> >>> >>> >>> On 23.10.18 07:11, Ahmet Altay wrote: >>> >>> > >>> >>> > >>> >>> > On Mon, Oct 22, 2018 at 10:06 PM, Kenneth Knowles >>> <[email protected] <mailto:[email protected]> >>> >>> > <mailto:[email protected] <mailto:[email protected]>>> wrote: >>> >>> > >>> >>> > You two did so much verification I had a hard time >>> finding something >>> >>> > where my help was meaningful! :-) >>> >>> > >>> >>> > I did run the Nexmark suite on the DirectRunner against >>> 2.7.0 and >>> >>> > 2.8.0 following >>> >>> > >>> https://beam.apache.org/documentation/sdks/java/nexmark/#run >>> ning-smoke-suite-on-the-directrunner-local >>> >>> > <https://beam.apache.org/docu >>> mentation/sdks/java/nexmark/#running-smoke-suite-on-the-dire >>> ctrunner-local>. >>> >>> > >>> >>> > It is admittedly a very silly test - the instructions >>> leave >>> >>> > immutability enforcement on, etc. But it does appear that >>> there is a >>> >>> > 30% degradation in query 8 and 15% in query 9. These are >>> the pure >>> >>> > Java tests, not the SQL variants. The rest of the queries >>> are close >>> >>> > enough that differences are not meaningful. >>> >>> > >>> >>> > >>> >>> > (It would be a good improvement for us to have alerts on daily >>> >>> > benchmarks if we do not have such a concept already.) >>> >>> > >>> >>> > >>> >>> > I would ask a little more time to see what is going on >>> here - is it >>> >>> > a real performance issue or an artifact of how the tests >>> are >>> >>> > invoked, or ...? >>> >>> > >>> >>> > >>> >>> > Thank you! Much appreciated. Please let us know when you are >>> done with >>> >>> > your investigation. >>> >>> > >>> >>> > >>> >>> > Kenn >>> >>> > >>> >>> > On Mon, Oct 22, 2018 at 6:20 PM Ahmet Altay >>> <[email protected] <mailto:[email protected]> >>> >>> > <mailto:[email protected] <mailto:[email protected]>>> >>> wrote: >>> >>> > >>> >>> > Hi all, >>> >>> > >>> >>> > Did you have a chance to review this RC? Between me >>> and Robert >>> >>> > we ran a significant chunk of the validations. Let me >>> know if >>> >>> > you have any questions. >>> >>> > >>> >>> > Ahmet >>> >>> > >>> >>> > On Thu, Oct 18, 2018 at 5:26 PM, Ahmet Altay >>> <[email protected] <mailto:[email protected]> >>> >>> > <mailto:[email protected] <mailto:[email protected]>>> >>> >>> wrote: >>> >>> > >>> >>> > Hi everyone, >>> >>> > >>> >>> > Please review and vote on the release candidate >>> #1 for the >>> >>> > version 2.8.0, as follows: >>> >>> > [ ] +1, Approve the release >>> >>> > [ ] -1, Do not approve the release (please >>> provide specific >>> >>> > comments) >>> >>> > >>> >>> > The complete staging area is available for your >>> review, >>> >>> > which includes: >>> >>> > * JIRA release notes [1], >>> >>> > * the official Apache source release to be >>> deployed to >>> >>> > dist.apache.org <http://dist.apache.org> >>> <http://dist.apache.org> [2], which is >>> >>> > signed with the key with fingerprint 6096FA00 [3], >>> >>> > * all artifacts to be deployed to the Maven >>> Central >>> >>> > Repository [4], >>> >>> > * source code tag "v2.8.0-RC1" [5], >>> >>> > * website pull request listing the release and >>> publishing >>> >>> > the API reference manual [6]. >>> >>> > * Python artifacts are deployed along with the >>> source >>> >>> > release to the dist.apache.org >>> <http://dist.apache.org> <http://dist.apache.org> [2]. >>> >>> >>> > * Validation sheet with a tab for 2.8.0 release >>> to help with >>> >>> > validation [7]. >>> >>> > >>> >>> > The vote will be open for at least 72 hours. It >>> is adopted >>> >>> > by majority approval, with at least 3 PMC >>> affirmative votes. >>> >>> > >>> >>> > Thanks, >>> >>> > Ahmet >>> >>> > >>> >>> > [1] >>> >>> > >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje >>> ctId=12319527&version=12343985 >>> >>> > <https://issues.apache.org/ji >>> ra/secure/ReleaseNote.jspa?projectId=12319527&version=12343985> >>> >>> > [2] https://dist.apache.org/repos/ >>> dist/dev/beam/2.8.0 >>> >>> > <https://dist.apache.org/repo >>> s/dist/dev/beam/2.8.0> >>> >>> > [3] https://dist.apache.org/repos/ >>> dist/dev/beam/KEYS >>> >>> > <https://dist.apache.org/repos/dist/dev/beam/KEYS >>> > >>> >>> > [4] >>> >>> > >>> https://repository.apache.org/content/repositories/orgapache >>> beam-1049/ >>> >>> > <https://repository.apache.or >>> g/content/repositories/orgapachebeam-1049/> >>> >>> > [5] https://github.com/apache/beam >>> /tree/v2.8.0-RC1 >>> >>> > <https://github.com/apache/beam/tree/v2.8.0-RC1> >>> >>> > [6] https://github.com/apache/beam-site/pull/583 >>> >>> > <https://github.com/apache/beam-site/pull/583> >>> and >>> >>> > https://github.com/apache/beam/pull/6745 >>> >>> > <https://github.com/apache/beam/pull/6745> >>> >>> > [7] >>> >>> > >>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkS >>> ZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1854712816 >>> >>> > <https://docs.google.com/spre >>> adsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit >>> #gid=1854712816> >>> >>> > >>> >>> > >>> >>> > >>> >>> >
