Re: [VOTE] Release 2.2.0, release candidate #4

FYI I'm still working on finalizing the release. The issue is that the Beam documentation underwent a large refactor while the release was pending, so the website PR (pr/337) conflicts. I think it's safer to give up on resolving these conflicts, so I'm simply going to regenerate this PR from scratc

Re: [VOTE] Release 2.2.0, release candidate #4

Sure, I was mostly surprised it was taking this long. However Robert says it sometimes takes three days. Reuven On Sat, Nov 25, 2017 at 9:11 PM, Jean-Baptiste Onofré wrote: > mvnrepository doesn't matter (it will sync later), the actual Central URL > is: > > http://repo.maven.apache.org/maven2/

Re: [VOTE] Release 2.2.0, release candidate #4

Historically mvn has taken 3+ days to finish syncing. Maybe some of the links are using RFC 1149 :). On Sat, Nov 25, 2017 at 9:11 PM, Jean-Baptiste Onofré wrote: > mvnrepository doesn't matter (it will sync later), the actual Central URL > is: > > http://repo.maven.apache.org/maven2/org/apache/be

Re: [VOTE] Release 2.2.0, release candidate #4

mvnrepository doesn't matter (it will sync later), the actual Central URL is: http://repo.maven.apache.org/maven2/org/apache/beam/beam-sdks-java-core/ And 2.2.0 is there. Regards JB On 11/25/2017 08:01 PM, Reuven Lax wrote: BTW, It's been over a day, and I still don't see 2.2.0 listed at htt

Re: [VOTE] Release 2.2.0, release candidate #4

Release instructions just show the "git tag" command for generating the new tag. However I'm assuming that I also need to push the new tag to the remote, right? Reuven On Sat, Nov 25, 2017 at 11:01 AM, Reuven Lax wrote: > Getting close. A few more steps left (source release, new git tag, mergin

Re: [VOTE] Release 2.2.0, release candidate #4

BTW, It's been over a day, and I still don't see 2.2.0 listed at https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-core How long does it usually take to promote the artifacts here? On Fri, Nov 24, 2017 at 3:43 PM, Reuven Lax wrote: > Appears to be a problem :) > > I tried publis

Re: [VOTE] Release 2.2.0, release candidate #4

Getting close. A few more steps left (source release, new git tag, merging the website PR, and marking it as released in JIRA). On Sat, Nov 25, 2017 at 10:12 AM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > So, is the release done now? Have both Java and Python artifacts been > uploa

Re: [VOTE] Release 2.2.0, release candidate #4

So, is the release done now? Have both Java and Python artifacts been uploaded? On Sat, Nov 25, 2017, 9:05 AM Robert Bradshaw wrote: > Upload to PyPi done. https://pypi.python.org/pypi/apache-beam > > On Sat, Nov 25, 2017 at 8:56 AM, Robert Bradshaw > wrote: > > I can do that. > > > > On Fri, N

Re: [VOTE] Release 2.2.0, release candidate #4

Upload to PyPi done. https://pypi.python.org/pypi/apache-beam On Sat, Nov 25, 2017 at 8:56 AM, Robert Bradshaw wrote: > I can do that. > > On Fri, Nov 24, 2017, 11:13 PM Reuven Lax wrote: >> >> I am not an owner or maintainer of the PyPi package, so I'm not sure if >> I'll be able to release Pyt

Re: [VOTE] Release 2.2.0, release candidate #4

I can do that. On Fri, Nov 24, 2017, 11:13 PM Reuven Lax wrote: > I am not an owner or maintainer of the PyPi package, so I'm not sure if > I'll be able to release Python artifacts. > > On Fri, Nov 24, 2017 at 10:08 PM, Jean-Baptiste Onofré > wrote: > > > Awesome, Thanks Reuven ! > > > > Regard

Re: [VOTE] Release 2.2.0, release candidate #4

I am not an owner or maintainer of the PyPi package, so I'm not sure if I'll be able to release Python artifacts. On Fri, Nov 24, 2017 at 10:08 PM, Jean-Baptiste Onofré wrote: > Awesome, Thanks Reuven ! > > Regards > JB > > On 11/25/2017 07:01 AM, Reuven Lax wrote: > >> I'll go ahead and send th

Re: [VOTE] Release 2.2.0, release candidate #4

Awesome, Thanks Reuven ! Regards JB On 11/25/2017 07:01 AM, Reuven Lax wrote: I'll go ahead and send the RESULT email right now. On Fri, Nov 24, 2017 at 9:56 PM, Jean-Baptiste Onofré wrote: It's not sync: promoting the artifacts takes some time (at least 30 minutes). So, the artifacts wil

[RESULT] [VOTE] Release 2.2.0, release candidate #4

I'm happy to announce that we have unanimously approved this release. There are 9 approving votes, 5 of which are binding: * Lukasz Cwik (binding) * Romain Manni-Bucau (non binding) * Jean-Baptiste Onofré (binding) * Ahmet Altay (binding) * Robert Bradshaw (binding) * Konstantinos Katsiapis (non b

Re: [VOTE] Release 2.2.0, release candidate #4

I'll go ahead and send the RESULT email right now. On Fri, Nov 24, 2017 at 9:56 PM, Jean-Baptiste Onofré wrote: > It's not sync: promoting the artifacts takes some time (at least 30 > minutes). So, the artifacts will be on central after a certain time. > > I confirm that it's OK because now th

Re: [VOTE] Release 2.2.0, release candidate #4

It's not sync: promoting the artifacts takes some time (at least 30 minutes). So, the artifacts will be on central after a certain time. I confirm that it's OK because now the artifacts are on Central: http://repo.maven.apache.org/maven2/org/apache/beam/beam-sdks-java-core/2.2.0/ By the way, y

Re: [VOTE] Release 2.2.0, release candidate #4

Appears to be a problem :) I tried publishing the latest artifact from Apache Nexus to Maven Central. After clicking publish, Nexus claimed that the operation has completed. However a look at the Maven Central page ( https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-core) does not

Re: [VOTE] Release 2.2.0, release candidate #4

+1 Run quickstart with Apex runner in embedded mode and on YARN. It needed couple tweaks to get there though. 1) Change quickstart pom.xml apex-runner profile: org.apache.hadoop hadoop-yarn-client ${hadoop.version} runtime

Re: [VOTE] Release 2.2.0, release candidate #4

Hi Eugene, I ran it on both standalone flink(non Yarn) and Flink on HDInsight Cluster(Yarn). Both ran successfully. :) Regards, Nishu Virus-free. www.avast.com

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks Nishu. So, if I understand correctly, your pipelines were running on non-YARN, but you're planning to run with YARN? I meanwhile was able to get Flink running on Dataproc (YARN), and validated quickstart and game examples. At this point we need validation for Spark and Flink non-YARN [I thi

Re: [VOTE] Release 2.2.0, release candidate #4

Hi Eugene, No, I didn't try with those instead I have my custom pipeline where Kafka topic is the source. I have defined a Global Window and processing time trigger to read the data. Further it runs some transformation i.e. GroupByKey and CoGroupByKey. on the windowed collections. I was running th

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (binding) On Wed, Nov 22, 2017 at 11:43 AM, Max Barrios wrote: > +1 (non-binding) > > Sent from my iPhone > > > On Nov 20, 2017, at 12:47 AM, Jean-Baptiste Onofré > wrote: > > > > Yeah, I have a Jira about that. > > > > You just have to update the existing symlink to point on the new release

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (non-binding) Sent from my iPhone > On Nov 20, 2017, at 12:47 AM, Jean-Baptiste Onofré wrote: > > Yeah, I have a Jira about that. > > You just have to update the existing symlink to point on the new release. > > I will update the release guide asap. > > Thanks ! > Regards > JB > >> On 11

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks Nishu! Can you clarify which pipeline you were running? The validation spreadsheet includes 1) the quickstart and 2) mobile game walkthroughs. Was it one of these, or your custom pipeline? On Wed, Nov 22, 2017 at 10:20 AM Nishu wrote: > Hi, > > Typo in previous mail. I meant Flink runner

Re: [VOTE] Release 2.2.0, release candidate #4

Please update the spreadsheet. On Wed, Nov 22, 2017 at 10:19 AM, Nishu wrote: > Hi, > > Typo in previous mail. I meant Flink runner. > > Thanks, > Nishu > On Wed, 22 Nov 2017 at 19.17, > > > Hi, > > > > I build a pipeline using RC 2.2 today and ran with runner on yarn. > > It worked seamlessly

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (non-binding) Since Beam 2.2 is blocking release of tensorflow.transform 0.4. On Wed, Nov 22, 2017 at 10:19 AM, Nishu wrote: > Hi, > > Typo in previous mail. I meant Flink runner. > > Thanks, > Nishu > On Wed, 22 Nov 2017 at 19.17, > > > Hi, > > > >

Re: [VOTE] Release 2.2.0, release candidate #4

Hi, Typo in previous mail. I meant Flink runner. Thanks, Nishu On Wed, 22 Nov 2017 at 19.17, > Hi, > > I build a pipeline using RC 2.2 today and ran with runner on yarn. > It worked seamlessly for unbounded sources. Couldn’t see any issues with > my pipeline so far :) > > > Thanks,Nishu > > On

Re: [VOTE] Release 2.2.0, release candidate #4

Hi, I build a pipeline using RC 2.2 today and ran with runner on yarn. It worked seamlessly for unbounded sources. Couldn’t see any issues with my pipeline so far :) Thanks,Nishu On Wed, 22 Nov 2017 at 18.57, Reuven Lax wrote: > Who is validating Flink and Yarn? > > On Tue, Nov 21, 2017 at 9:

Re: [VOTE] Release 2.2.0, release candidate #4

Who is validating Flink and Yarn? On Tue, Nov 21, 2017 at 9:26 AM, Kenneth Knowles wrote: > On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpichov < > kirpic...@google.com.invalid> wrote: > > > In the verification spreadsheet, I'm not sure I understand the difference > > between the "YARN" and "Stand

Re: [VOTE] Release 2.2.0, release candidate #4

On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > In the verification spreadsheet, I'm not sure I understand the difference > between the "YARN" and "Standalone cluster/service". Which is Dataproc? It > definitely uses YARN, but it is also a standalone clus

Re: [VOTE] Release 2.2.0, release candidate #4

We now have 3 PMC votes for yes. I'm just waiting for Flink runner validation (still missing on the spreadsheet) before finalizing the release. On Tue, Nov 21, 2017 at 9:39 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > +1 (binding) > > On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpich

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (binding) On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > I've additionally verified the game examples against Dataflow runner, and > verified UserScore and Leaderboard against Spark on Dataproc (YARN). All of > these worked* > So my +1 stands. > > In

Re: [VOTE] Release 2.2.0, release candidate #4

I've additionally verified the game examples against Dataflow runner, and verified UserScore and Leaderboard against Spark on Dataproc (YARN). All of these worked* So my +1 stands. In the verification spreadsheet, I'm not sure I understand the difference between the "YARN" and "Standalone cluster/

Re: [VOTE] Release 2.2.0, release candidate #4

Done On Tue, Nov 21, 2017 at 3:08 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > Thanks. You need to re-sign as well. > > On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax > wrote: > > FYI these generated files have been removed from the source distribution. > > > > On Sat, Nov 18, 2017

Re: [VOTE] Release 2.2.0, release candidate #4

+1 I verified the python quick start on Windows. I could not verify the documentation changes because the staged version expired. On Mon, Nov 20, 2017 at 12:08 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > Thanks Luke. I was able to validate quickstart on Dataflow and on Spark >

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks Luke. I was able to validate quickstart on Dataflow and on Spark cluster (using Cloud Dataproc). So +1 from me so far. On Sun, Nov 19, 2017 at 4:28 PM Lukasz Cwik wrote: > Eugene, you can setup your ~/.m2/settings.xml to point to the repository > containing the release candidate. > >

Re: [VOTE] Release 2.2.0, release candidate #4

Thanks. You need to re-sign as well. On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax wrote: > FYI these generated files have been removed from the source distribution. > > On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax wrote: > >> hmmm, I thought I removed those generated files from the zip file befor

Re: [VOTE] Release 2.2.0, release candidate #4

Yeah, I have a Jira about that. You just have to update the existing symlink to point on the new release. I will update the release guide asap. Thanks ! Regards JB On 11/20/2017 09:39 AM, Reuven Lax wrote: Good point. How do I update that link? I think this step might be missing from our rele

Re: [VOTE] Release 2.2.0, release candidate #4

Or is this simply a matter of copying from the dev to the release repository? On Mon, Nov 20, 2017 at 4:39 PM, Reuven Lax wrote: > Good point. How do I update that link? I think this step might be missing > from our release guide (or I simply can't find it). > > Reuven > > On Mon, Nov 20, 2017 a

Re: [VOTE] Release 2.2.0, release candidate #4

Good point. How do I update that link? I think this step might be missing from our release guide (or I simply can't find it). Reuven On Mon, Nov 20, 2017 at 4:24 PM, Jean-Baptiste Onofré wrote: > +1 (binding) > > Tested on Beam samples, it looks good. > > Checked the legal/license, content. OK

Re: [VOTE] Release 2.2.0, release candidate #4

+1 (binding) Tested on Beam samples, it looks good. Checked the legal/license, content. OK for me. Just a side note: don't forget to update the latest link on dist.apache.org. Thanks Regards JB On 11/17/2017 07:08 AM, Reuven Lax wrote: Hi everyone, Please review and vote on the release cand

Re: [VOTE] Release 2.2.0, release candidate #4

Tested on some custom transforms and integrations, +1 (non-binding), thanks for the perseverance Reuven! Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn 2017-11-20 9:14 GMT+01:00 Reuven Lax : > FYI these generated files have been removed from the source distribution. > > O

Re: [VOTE] Release 2.2.0, release candidate #4

FYI these generated files have been removed from the source distribution. On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax wrote: > hmmm, I thought I removed those generated files from the zip file before > sending this email. Let me check again. > > Reuven > > On Sat, Nov 18, 2017 at 8:52 AM, Robert

Re: [VOTE] Release 2.2.0, release candidate #4

Eugene, you can setup your ~/.m2/settings.xml to point to the repository containing the release candidate. release-repo true Release 2.2.0 RC4 Release 2.2.0 RC4 https://repository.apache.org/content/repositories/orgapachebeam

Re: [VOTE] Release 2.2.0, release candidate #4

hmmm, I thought I removed those generated files from the zip file before sending this email. Let me check again. Reuven On Sat, Nov 18, 2017 at 8:52 AM, Robert Bradshaw < rober...@google.com.invalid> wrote: > The source distribution contains a couple of files not on github (e.g. > folders that w

Re: [VOTE] Release 2.2.0, release candidate #4

The source distribution contains a couple of files not on github (e.g. folders that were added on master, Python generated files). The pom files differed only by missing -SNAPSHOT, other than that presumably the source release should just be "wget https://github.com/apache/beam/archive/release-2.2.

Re: [VOTE] Release 2.2.0, release candidate #4

How can I specify a dependency on the staged RC? E.g. I'm trying to validate the quickstart per https://beam.apache.org/get-started/quickstart-java/ and specifying version 2.2.0 doesn't work I suppose because it's not released yet. Should I pass some command-line flag to mvn to make it fetch the ve

Re: [VOTE] Release 2.2.0, release candidate #4

Its open to all, its just that there are binding votes and non-binding votes. On Fri, Nov 17, 2017 at 4:26 PM, Valentyn Tymofieiev < valen...@google.com.invalid> wrote: > I have a process question: is the vote open for committers only or for all > contributors? > > On Fri, Nov 17, 2017 at 4:06 PM

Re: [VOTE] Release 2.2.0, release candidate #4

I have a process question: is the vote open for committers only or for all contributors? On Fri, Nov 17, 2017 at 4:06 PM, Lukasz Cwik wrote: > +1, Approve the release > > I have verified the wordcount quickstart on the Apache Beam website using > Apex, DirectRunner, Flink & Spark on Linux. > > T

Re: [VOTE] Release 2.2.0, release candidate #4

+1, Approve the release I have verified the wordcount quickstart on the Apache Beam website using Apex, DirectRunner, Flink & Spark on Linux. The Gearpump runner is yet to have a quickstart listed on our website. Adding the quickstart is already represented by this existing issue: https://issues.

Re: [VOTE] Release 2.2.0, release candidate #4

I have verified: SHA & MD5 signatures of Python artifacts in [2], and checked Python side of the validation checklist on Linux. There is one known issue in UserScore example for Dataflow runner. The issue has been fixed on master branch and does not require a cherry-pick at this point. A workaroun

Re: [VOTE] Release 2.2.0, release candidate #4

Hi all, Following up on past discussions and https://issues.apache.org/jira/browse/BEAM-1189 I have prepared a spreadsheet so we can sign up for validation steps that must be done by a human. The spreadsheet for 2.2.0 is at https://s.apache.org/beam-2.2.0-release-validation. Everyone can edit, so

[VOTE] Release 2.2.0, release candidate #4

Hi everyone, Please review and vote on the release candidate #4 for the version 2.2.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes