Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thanks, I'm testing it as well. Regards JB On 15/06/2018 10:25, Charles Chen wrote: > Thank you and sorry for the delay.  Been testing the fix the past few > hours.  This CP PR fixes the > issue: https://github.com/apache/beam/pull/5658. > > On Thu, Jun 14, 2018 at 10:25 PM Jean-Baptiste Onofré

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thank you and sorry for the delay. Been testing the fix the past few hours. This CP PR fixes the issue: https://github.com/apache/beam/pull/5658 . On Thu, Jun 14, 2018 at 10:25 PM Jean-Baptiste Onofré wrote: > OK, I started the RC2, but I'm stopping the process to cut a new one. > > Is it ok f

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

OK, I started the RC2, but I'm stopping the process to cut a new one. Is it ok from your side ? Regards JB On 15/06/2018 01:54, Charles Chen wrote: > Looks like there is something wrong with PR 5636 >  which we cherry-picked > above.  It breaks leaderboa

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Looks like there is something wrong with PR 5636 which we cherry-picked above. It breaks leaderboard examples which previously passed. I've reopened the issue and will update this thread shortly. On Thu, Jun 14, 2018 at 12:55 PM Jean-Baptiste Onofré wro

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Sure, just in time ;) Regards JB On 14/06/2018 20:58, Charles Chen wrote: > Can you also merge the CP https://github.com/apache/beam/pull/5636 for > https://issues.apache.org/jira/browse/BEAM-4549? > > On Thu, Jun 14, 2018 at 6:52 AM Jean-Baptiste Onofré > wrote: > >

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Can you also merge the CP https://github.com/apache/beam/pull/5636 for https://issues.apache.org/jira/browse/BEAM-4549? On Thu, Jun 14, 2018 at 6:52 AM Jean-Baptiste Onofré wrote: > FYI, I'm starting RC2 right now. > > Stay tuned ! > > Regards > JB > > On 06/06/2018 10:44, Jean-Baptiste Onofré w

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

FYI, I'm starting RC2 right now. Stay tuned ! Regards JB On 06/06/2018 10:44, Jean-Baptiste Onofré wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version > 2.5.0, as follows: > > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please prov

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

It looks good to me, I'm merging and moving forward. Regards JB On 14/06/2018 00:45, Pablo Estrada wrote: > Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky > test. As JB is the release manager, I'l let him make the call on what to > do about it. > Best > -P. > > On Wed, Jun

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

FWIW I have a fix to the flaky test in https://github.com/apache/beam/pull/5585 (open) On Wed, Jun 13, 2018 at 5:26 PM Udi Meiri wrote: > +1 to ignoring flaky test. > > FYI there's a fourth cherrypick: https://github.com/apache/beam/pull/5624 > > On Wed, Jun 13, 2018 at 3:45 PM Pablo Estrada wr

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

+1 to ignoring flaky test. FYI there's a fourth cherrypick: https://github.com/apache/beam/pull/5624 On Wed, Jun 13, 2018 at 3:45 PM Pablo Estrada wrote: > Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky > test. As JB is the release manager, I'l let him make the call on wh

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky test. As JB is the release manager, I'l let him make the call on what to do about it. Best -P. On Wed, Jun 13, 2018 at 3:34 PM Ahmet Altay wrote: > I would vote for second option, not a release blocker and disable the test > i

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

I would vote for second option, not a release blocker and disable the test in the release branch. My reasoning is: - ReferenceRunner is not yet the official alternative to existing direct runners. - It is bad to have flaky tests on the release branch, and we would not get good signal during validat

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hello all, cherrypicks for the release branch seem to be going well, but thanks to them we were able to surface a flaky test in the release branch. JIRA is filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558 Given that test issue, I see the following options: - Consider that this

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hey all, Currently we have 3 PRs supposed to be cherrypicked into RC2: Pablo: https://github.com/apache/beam/pull/5609 (merged) Udi: https://github.com/apache/beam/pull/5607 (open) Charles: https://github.com/apache/beam/pull/5636 (open) Boyuan On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada wr

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Precommits for PR https://github.com/apache/beam/pull/5609 are now passing. For now I've simply set failOnWarning to false to cherrypick into the release, and fix in master later on. Best -P. On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner wrote: > From my understanding, the @SuppressFBWarnings usa

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

>From my understanding, the @SuppressFBWarnings usage is in a dependency (ByteBuddy) rather than directly in our code; so we're not able to modify the usage. Pablo, feel free to disable failOnWarning for the sdks-java-core project temporarily. This isn't a major regression since we've only recentl

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi Pablo, I'm afraid I couldn't find one either... there is an issue about it [1] which is old so it doesn't look likely to be resolved either. If you have time (sorry I am a bit busy) could you please verify the version does work if you install that version locally? I know the maven version of t

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi Tim, you're right. Thanks for pointing that out. There's just one problem that I'm running into now: The 3.0.1-1 version does not seem to be available in Maven Central[1]. Looking at the website, I am not quite sure if there's another repository where they do stage the newer versions?[2] -P [1

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi Pablo, I took only a quick look. "- The JAR from the non-LGPL findbugs does not contain the SuppressFBWarnings annotation" Unless I misunderstand you it looks like SuppressFBWarnings was added in Stephen's version in this commit [1] which was introduced in version 2.0.3-1 - I've checked is i

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi all, I'll humbly declare that after wrestling with he build to stop depending on the wrong findbugs_annotations, I feel somewhat lost. The issue is actually quite small: - The JAR from the non-LGPL findbugs does not contain the SuppressFBWarnings annotation. This means that when building, ByteB

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Yes, ok I was not aware it was already being addressed, nice. On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay wrote: > > Ismaël, > > I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing the > issue by changing the findbugs back to "com.github.stephenc.findbugs". Is > this what you a

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Ismaël, I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing the issue by changing the findbugs back to "com.github.stephenc.findbugs". Is this what you are referring to? Ahmet On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang wrote: > Hey JB, > > I added some instructions about ho

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hey JB, I added some instructions about how to create python wheels in this PR: https://github.com/apache/beam-site/pull/467 . Hope it would be helpful. Boyuan On Tue, Jun 12, 2018 at 12:33 PM Jean-Baptiste Onofré wrote: > Hi all, > > due to issues found by Udi and Pablo, I cancel RC1 to prepa

Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

There is another issue highlighted by Scott Wegner in a non-related to the vote PR discussion. https://github.com/apache/beam/pull/5540 It seems that in the migration to gradle we changed the findbugs-annotations library from com.github.stephenc.findbugs to com.google.code.findbugs:findbugs-annota

[CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi all, due to issues found by Udi and Pablo, I cancel RC1 to prepare a new release. Thanks Udi and Pablo for catching up. I will prepare a RC2 after cherry pick/fixed the found issues (tomorrow morning my time). Regards JB On 12/06/2018 21:04, Pablo Estrada wrote: > Hello all, > I've found a

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hello all, I've found another release blocker: The current findbugs dependency in build_rules.gradle has a GPL license. Here's the PR to depend on the correct findbugs: https://github.com/apache/beam/pull/5609 Best -P. On Tue, Jun 12, 2018 at 11:03 AM Udi Meiri wrote: > Cherrypick created: http

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Cherrypick created: https://github.com/apache/beam/pull/5607 Tests still running. On Tue, Jun 12, 2018 at 9:59 AM Udi Meiri wrote: > -1: Would like to cherry pick a fix for > https://issues.apache.org/jira/browse/BEAM-4536 > > On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré > wrote: > >> A

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

-1: Would like to cherry pick a fix for https://issues.apache.org/jira/browse/BEAM-4536 On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré wrote: > Any update about your vote and fix ? > > Thanks ! > Regards > JB > > On 12/06/2018 04:02, Udi Meiri wrote: > > Another bug: reading from PubSub wi

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Any update about your vote and fix ? Thanks ! Regards JB On 12/06/2018 04:02, Udi Meiri wrote: > Another bug: reading from PubSub with_attributes=True is broken on > Python with Dataflow. > https://issues.apache.org/jira/browse/BEAM-4536 > > JB, I'm making a PR that removes this keyword and I'd

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi Udi, are you voting -1 ? It would be great if you can cast your vote (-1): I will cancel RC1 based on your vote. Thanks, Regards JB On 12/06/2018 04:02, Udi Meiri wrote: > Another bug: reading from PubSub with_attributes=True is broken on > Python with Dataflow. > https://issues.apache.org/j

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thank you JB. For the wheel artifacts, Boyuan was trying to get the instructions from Robert and reproduce the artifacts. She can help you with this if you need. Ahmet On Mon, Jun 11, 2018 at 10:29 PM, Jean-Baptiste Onofré wrote: > Hi, > > sorry, I missed wheel artifact. Something to add on th

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi, sorry, I missed wheel artifact. Something to add on the release guide ;) I will add it this morning, I think I know how to generate it ;) Regards JB On 12/06/2018 02:45, Pablo Estrada wrote: > Thanks everyone who has pitched in to validate the release! > > Boyuan Zhang and I have also run

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi, no problem, I can cut RC2 as soon as the cherry pick is done. Thanks for catching up ! Please let me know when the cherry pick is done, or you can do the PR and I will do it, up to you. Regards JB On 12/06/2018 04:02, Udi Meiri wrote: > Another bug: reading from PubSub with_attributes=True

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Another bug: reading from PubSub with_attributes=True is broken on Python with Dataflow. https://issues.apache.org/jira/browse/BEAM-4536 JB, I'm making a PR that removes this keyword and I'd like to propose it as a cherrypick to 2.5.0. (feature should be fixed in the next release) On Mon, Jun 11,

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

FYI: looks like Python tests are failing for Windows. JIRA is https://issues.apache.org/jira/browse/BEAM-4535. I don't think this is a release blocker but this should probably go in release notes (for any user that tries to run tests on Python source build). And we should try to incorporate a fix

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thanks everyone who has pitched in to validate the release! Boyuan Zhang and I have also run a few pipelines, and verified that they work properly (see release validation spreadsheet[1]). We have also found that the Game Stats pipeline is failing in Python Streaming Dataflow. I have filed BEAM-45

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

+1 (non-binding) tested some of the quickstarts On Sun, Jun 10, 2018 at 1:39 AM Tim wrote: > Tested by our team: > - mvn inclusion > - Avro, ES, Hadoop IF IO > - Pipelines run on Spark (Cloudera 5.12.0 YARN cluster) > - Reviewed release notes > > +1 > > Thanks also to everyone who helped get ov

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Tested by our team: - mvn inclusion - Avro, ES, Hadoop IF IO - Pipelines run on Spark (Cloudera 5.12.0 YARN cluster) - Reviewed release notes +1 Thanks also to everyone who helped get over the gradle hurdle and in particular to JB. Tim > On 9 Jun 2018, at 05:56, Jean-Baptiste Onofré wrote: >

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

No problem Pablo. The vote period is a minimum, it can be extended as requested or if we don't have the minimum of 3 binding votes. Regards JB On 09/06/2018 01:54, Pablo Estrada wrote: > Hello all, > I'd like to request an extension of the voting period until Monday > evening (US time, so later

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hello all, I'd like to request an extension of the voting period until Monday evening (US time, so later in other geographical regions). This is because we were only now able to publish Dataflow Workers, and have not had the chance to run release validation tests on them. The extension will allow u

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thank you JB for your work! I tested running simple streaming (KafkaIO) and batch (TextIO / HDFS) pipelines with SparkRunner on YARN cluster - it works fine. WBR, Alexey > On 8 Jun 2018, at 10:00, Etienne Chauchot wrote: > > I forgot to vote: > +1 (non binding). > What I tested: > - no funct

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

I forgot to vote:+1 (non binding). What I tested: - no functional or performance regression comparing to v2.4- dependencies in the poms are ok EtienneLe vendredi 08 juin 2018 à 08:27 +0200, Romain Manni-Bucau a écrit : > +1 (non-binding), mainstream usage is not broken by the pom changes and > ru

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

+1 (non-binding), mainstream usage is not broken by the pom changes and runtime has no known regression compared to the 2.4.0 (side note: kudo to JB for this build tool change release, I know how it can hurt ;)) Romain Manni-Bucau @rmannibucau | Blog

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thanks for the details Etienne ! The good news is that the artifacts seem OK and the overall Nexmark results are consistent with the 2.4.0 release ones. I'm starting a complete review using the beam-samples as well. Regards JB On 07/06/2018 16:14, Etienne Chauchot wrote: > Hi, > I've just run t

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi,I've just run the nexmark queries on v2.5.0-RC1 tagWhat we can notice: - query 3 (exercises CoGroupByKey, state and timer) shows different output with DR between batch and streaming and with the other runners => I compared with v2.4 there were still these differences but with different output

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

I have added the 2.5.0 tab to the validation spreadsheet[1], please mark down which things you intend to validate for the release and update the community on progress. 1: https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807 On Wed, Jun 6, 2018 at

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Agreed 💯! It's not ready being the first to try something. Thank you so much for helping blaze the way! Reuven On Wed, Jun 6, 2018, 11:50 AM Etienne Chauchot wrote: > Thanks JB for all your work ! I believe doing the first gradle release > must have been hard. > I'll run Nexmark on the release

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

I updated dist.apache.org dev with Python distribution. Regards JB On 06/06/2018 18:19, Jean-Baptiste Onofré wrote: > Hi Robert, > > sorry, I missed this step, let me add on dist.apache.org. > > Thanks for the catch and sorry about that ! > > Regards > JB > > On 06/06/2018 18:06, Robert Brads

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi Robert, sorry, I missed this step, let me add on dist.apache.org. Thanks for the catch and sorry about that ! Regards JB On 06/06/2018 18:06, Robert Bradshaw wrote: > Thank you JB! Glad to see this finally rolling out. I don't see the > Python artifacts, did you mean to stage them > in https

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thank you JB! Glad to see this finally rolling out. I don't see the Python artifacts, did you mean to stage them in https://dist.apache.org/repos/dist/dev/beam/2.5.0/? If you want help building wheels, let me know. On Wed, Jun 6, 2018 at 1:50 AM Etienne Chauchot wrote: > Thanks JB for all your

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #1

Thanks JB for all your work ! I believe doing the first gradle release must have been hard. I'll run Nexmark on the release and keep you posted. Best Etienne Le mercredi 06 juin 2018 à 10:44 +0200, Jean-Baptiste Onofré a écrit : > Hi everyone, > > Please review and vote on the release candida

[VOTE] Apache Beam, version 2.5.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 2.5.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) NB: this is the first release using Gradle, so don't be too harsh ;) A PR about the release guide w