On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <k...@apache.org> wrote:
> What can we do to make this part of day-to-day workflow instead of finding
> out during release validation? Was this just a failing test that was missed?
>
> Kenn
>
> On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <apill...@google.com>
> wrote:
>
>> It looks like Java Nexmark tests are on the validation sheet but we've
>> missed it the last few releases. Thanks for checking it Etienne!
>>
>
I only see nexmark perf on direct runner. Is this even a thing? Should we
also add major runners to that list?
> Does the current release process require everything to be tested before
>> making the release final?
>>
>
I do not think we agreed on this. Previous releases went out with partial
validations. It would be good to validate everything for the final release.
>
>> I fully agree with you on point 2. All of these issues were in RC1 and
>> could have been fixed for RC2.
>>
>> Andrew
>>
>> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you Andrew. I will suggest two improvements to the release process:
>>> 1. We can include benchmarks in the validation sheet ("Apache Beam
>>> Release Acceptance Criteria"). They are used part of the validation process
>>> and we can ensure that we check those for each release.
>>> 2. For RC validation, we can continue to exhaustively validate each RC
>>> even after the first -1 vote. Otherwise we end up with not discovering all
>>> issues in a given RC and find them a successive RC, increasing the number
>>> of iterations required.
>>>
>>>
>>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <apill...@google.com>
>>> wrote:
>>>
>>>> Please consider the vote for RC4 canceled. I'll quickly follow up with
>>>> a new RC.
>>>>
>>>> Thanks for the complete testing everyone!
>>>>
>>>> Andrew
>>>>
>>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote:
>>>>
>>>>> -1
>>>>>
>>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue
>>>>>
>>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <apill...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I believe the breakage of Nexmark on Dataflow is
>>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in
>>>>>> before the release was cut. It looks like this might be a release blocker
>>>>>> based on the fix: https://github.com/apache/beam/pull/8325.
>>>>>>
>>>>>> The degraded performance is after the release is cut, so we should be
>>>>>> good there.
>>>>>>
>>>>>> Andrew
>>>>>>
>>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ieme...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry
>>>>>>> picks on the spark runner afterwards so if there is a commit that
>>>>>>> degraded performance around 04/10 it is not part of the release we
>>>>>>> are
>>>>>>> voting, so please consider reverting your -1.
>>>>>>>
>>>>>>> However the issue you are reporting looks important, from a quick
>>>>>>> look
>>>>>>> I am guessing it could be related to BEAM-5775 that was merged on
>>>>>>> 12/04 however the performance regressions started happening since
>>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in
>>>>>>> our infrastructure. Maybe the change in the workers to be tracked,
>>>>>>> but
>>>>>>> definitely not a release blocker at least for the Spark runner.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot <
>>>>>>> echauc...@apache.org> wrote:
>>>>>>> >
>>>>>>> > Hi guys ,
>>>>>>> > I will vote -1 (binding) on this RC (although degradation is
>>>>>>> before RC4 cut date). I took a look at Nexmark graphs for the 3 major
>>>>>>> runners :
>>>>>>> > - there seem to have functional regressions on Dataflow:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856
>>>>>>> . 13 queries fail in batch mode starting on 04/17
>>>>>>> > - there is a perf degradation (+200%) in spark runner starting on
>>>>>>> 04/10 for all the queries:
>>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>>>>>>> >
>>>>>>> > Sorry Andrew for the added work
>>>>>>> >
>>>>>>> > Etienne
>>>>>>> >
>>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit :
>>>>>>> >
>>>>>>> > I signed the wheels files and updated the build process to not
>>>>>>> require giving travis apache credentials. (You should probably change
>>>>>>> your
>>>>>>> password if you haven't already.)
>>>>>>> >
>>>>>>> > Andrew
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Verified the python 2 wheel files with quick start examples.
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I built the wheel files. They are in the usual place along with
>>>>>>> other python artifacts. I will test them a bit and update here. Could
>>>>>>> someone else please try the wheel files as well?
>>>>>>> >
>>>>>>> > Andrew, could you sign and hash the wheel files?
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > I verified
>>>>>>> > - signatures and hashes.
>>>>>>> > - python streaming quickstart guide
>>>>>>> >
>>>>>>> > I would like to verify the wheel files before voting. Please let
>>>>>>> us know when they are ready. Also, if you need help with building wheel
>>>>>>> files I can help/build.
>>>>>>> >
>>>>>>> > Ahmet
>>>>>>> >
>>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <m...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > +1 (binding)
>>>>>>> >
>>>>>>> > Found a minor bug while testing, but not a blocker:
>>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128
>>>>>>> >
>>>>>>> > Thanks,
>>>>>>> > Max
>>>>>>> >
>>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote:
>>>>>>> > > +1
>>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java
>>>>>>> validatesrunner tests.
>>>>>>> > >
>>>>>>> > > -P.
>>>>>>> > >
>>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré <
>>>>>>> j...@nanthrax.net
>>>>>>> > > <mailto:j...@nanthrax.net>> wrote:
>>>>>>> > >
>>>>>>> > > +1 (binding)
>>>>>>> > >
>>>>>>> > > Quickly checked with beam-samples.
>>>>>>> > >
>>>>>>> > > Regards
>>>>>>> > > JB
>>>>>>> > >
>>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote:
>>>>>>> > > > Hi everyone,
>>>>>>> > > >
>>>>>>> > > > Please review and vote on the release candidate #4 for
>>>>>>> the version
>>>>>>> > > > 2.12.0, as follows:
>>>>>>> > > >
>>>>>>> > > > [ ] +1, Approve the release
>>>>>>> > > > [ ] -1, Do not approve the release (please provide
>>>>>>> specific comments)
>>>>>>> > > >
>>>>>>> > > > The complete staging area is available for your review,
>>>>>>> which
>>>>>>> > > includes:
>>>>>>> > > > * JIRA release notes [1],
>>>>>>> > > > * the official Apache source release to be deployed to
>>>>>>> > > dist.apache.org <http://dist.apache.org>
>>>>>>> > > > <http://dist.apache.org> [2], which is signed with the
>>>>>>> key with
>>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3],
>>>>>>> > > > * all artifacts to be deployed to the Maven Central
>>>>>>> Repository [4],
>>>>>>> > > > * source code tag "v2.12.0-RC4" [5],
>>>>>>> > > > * website pull request listing the release [6],
>>>>>>> publishing the API
>>>>>>> > > > reference manual [7], and the blog post [8].
>>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and
>>>>>>> OpenJDK/Oracle JDK
>>>>>>> > > > 1.8.0_181.
>>>>>>> > > > * Python artifacts are deployed along with the source
>>>>>>> release to the
>>>>>>> > > > dist.apache.org <http://dist.apache.org> <
>>>>>>> http://dist.apache.org>
>>>>>>> > > [2].
>>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help
>>>>>>> with
>>>>>>> > > validation
>>>>>>> > > > [9].
>>>>>>> > > >
>>>>>>> > > > The vote will be open for at least 72 hours. It is
>>>>>>> adopted by
>>>>>>> > > majority
>>>>>>> > > > approval, with at least 3 PMC affirmative votes.
>>>>>>> > > >
>>>>>>> > > > Thanks,
>>>>>>> > > > Andrew
>>>>>>> > > >
>>>>>>> > > > 1]
>>>>>>> > >
>>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944
>>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/
>>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> > > > [4]
>>>>>>> > >
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/
>>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4
>>>>>>> > > > [6] https://github.com/apache/beam/pull/8215
>>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588
>>>>>>> > > > [8] https://github.com/apache/beam/pull/8314
>>>>>>> > > > [9]
>>>>>>> > >
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984
>>>>>>> > >
>>>>>>> > > --
>>>>>>> > > Jean-Baptiste Onofré
>>>>>>> > > jbono...@apache.org <mailto:jbono...@apache.org>
>>>>>>> > > http://blog.nanthrax.net
>>>>>>> > > Talend - http://www.talend.com
>>>>>>> > >
>>>>>>>
>>>>>>
