I've additionally verified the game examples against Dataflow runner, and verified UserScore and Leaderboard against Spark on Dataproc (YARN). All of these worked* So my +1 stands.
In the verification spreadsheet, I'm not sure I understand the difference between the "YARN" and "Standalone cluster/service". Which is Dataproc? It definitely uses YARN, but it is also a standalone cluster/service. Does it count for both? Seems now we're missing just Apex and Flink cluster verifications. *though Spark runner took 6x longer to run UserScore, partially I guess because it didn't do autoscaling (Dataflow runner ramped up to 5 workers whereas Spark runner used 2 workers). For some reason Spark runner chose not to split the 10GB input files into chunks. On Mon, Nov 20, 2017 at 3:46 PM Reuven Lax <re...@google.com.invalid> wrote: > Done > > On Tue, Nov 21, 2017 at 3:08 AM, Robert Bradshaw < > rober...@google.com.invalid> wrote: > > > Thanks. You need to re-sign as well. > > > > On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax <re...@google.com.invalid> > > wrote: > > > FYI these generated files have been removed from the source > distribution. > > > > > > On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax <re...@google.com> wrote: > > > > > >> hmmm, I thought I removed those generated files from the zip file > before > > >> sending this email. Let me check again. > > >> > > >> Reuven > > >> > > >> On Sat, Nov 18, 2017 at 8:52 AM, Robert Bradshaw < > > >> rober...@google.com.invalid> wrote: > > >> > > >>> The source distribution contains a couple of files not on github > (e.g. > > >>> folders that were added on master, Python generated files). The pom > > >>> files differed only by missing -SNAPSHOT, other than that presumably > > >>> the source release should just be "wget > > >>> https://github.com/apache/beam/archive/release-2.2.0.zip";? > > >>> > > >>> diff -rq apache-beam-2.2.0 beam/ | grep -v pom.xml > > >>> > > >>> # OK? > > >>> > > >>> Only in apache-beam-2.2.0: DEPENDENCIES > > >>> > > >>> # Expected. > > >>> > > >>> Only in beam/: .git > > >>> Only in beam/: .gitattributes > > >>> Only in beam/: .gitignore > > >>> > > >>> # These folders are probably from switching around between master and > > >>> git branches. > > >>> > > >>> Only in apache-beam-2.2.0: model > > >>> Only in apache-beam-2.2.0/runners/flink: examples > > >>> Only in apache-beam-2.2.0/runners/flink: runner > > >>> Only in apache-beam-2.2.0/runners/gearpump: jarstore > > >>> Only in apache-beam-2.2.0/sdks/java/extensions: gcp-core > > >>> Only in apache-beam-2.2.0/sdks/java/extensions: sketching > > >>> Only in apache-beam-2.2.0/sdks/java/io: file-based-io-tests > > >>> Only in apache-beam-2.2.0/sdks/java/io: hdfs > > >>> Only in apache-beam-2.2.0/sdks/java/maven-archetypes/examples/src/ma > > >>> in/resources/archetype-resources: > > >>> src > > >>> Only in apache-beam-2.2.0/sdks/java/maven-archetypes/examples-java8/ > > >>> src/main/resources/archetype-resources: > > >>> src > > >>> Only in apache-beam-2.2.0/sdks/java: microbenchmarks > > >>> > > >>> # Here's the generated protos. > > >>> > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_artifact_api_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_artifact_api_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_fn_api_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_fn_api_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_job_api_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_job_api_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_provision_api_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_provision_api_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_runner_api_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> beam_runner_api_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> endpoints_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> endpoints_pb2.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> standard_window_fns_pb2_grpc.py > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > >>> standard_window_fns_pb2.py > > >>> > > >>> And some other sdist generated Python files. > > >>> > > >>> Only in apache-beam-2.2.0/sdks/python: .eggs > > >>> Only in apache-beam-2.2.0/sdks/python: LICENSE > > >>> Only in apache-beam-2.2.0/sdks/python: NOTICE > > >>> Only in apache-beam-2.2.0/sdks/python: README.md > > >>> > > >>> Presumably we should just purge these files from the rc? > > >>> > > >>> > > >>> FWIW, the Python tarball looks fine. > > >>> > > >>> On Fri, Nov 17, 2017 at 4:40 PM, Eugene Kirpichov > > >>> <kirpic...@google.com.invalid> wrote: > > >>> > How can I specify a dependency on the staged RC? E.g. I'm trying to > > >>> > validate the quickstart per > > >>> > https://beam.apache.org/get-started/quickstart-java/ and > specifying > > >>> version > > >>> > 2.2.0 doesn't work I suppose because it's not released yet. Should > I > > >>> pass > > >>> > some command-line flag to mvn to make it fetch the version from the > > >>> staging > > >>> > area? > > >>> > > > >>> > On Fri, Nov 17, 2017 at 4:37 PM Lukasz Cwik > <lc...@google.com.invalid > > > > > >>> > wrote: > > >>> > > > >>> >> Its open to all, its just that there are binding votes and > > non-binding > > >>> >> votes. > > >>> >> > > >>> >> On Fri, Nov 17, 2017 at 4:26 PM, Valentyn Tymofieiev < > > >>> >> valen...@google.com.invalid> wrote: > > >>> >> > > >>> >> > I have a process question: is the vote open for committers only > or > > >>> for > > >>> >> all > > >>> >> > contributors? > > >>> >> > > > >>> >> > On Fri, Nov 17, 2017 at 4:06 PM, Lukasz Cwik > > >>> <lc...@google.com.invalid> > > >>> >> > wrote: > > >>> >> > > > >>> >> > > +1, Approve the release > > >>> >> > > > > >>> >> > > I have verified the wordcount quickstart on the Apache Beam > > website > > >>> >> using > > >>> >> > > Apex, DirectRunner, Flink & Spark on Linux. > > >>> >> > > > > >>> >> > > The Gearpump runner is yet to have a quickstart listed on our > > >>> website. > > >>> >> > > Adding the quickstart is already represented by this existing > > >>> issue: > > >>> >> > > https://issues.apache.org/jira/browse/BEAM-2692 > > >>> >> > > > > >>> >> > > On Fri, Nov 17, 2017 at 11:50 AM, Valentyn Tymofieiev < > > >>> >> > > valen...@google.com.invalid> wrote: > > >>> >> > > > > >>> >> > > > I have verified: SHA & MD5 signatures of Python artifacts in > > >>> [2], and > > >>> >> > > > checked Python side of the validation checklist on Linux. > > >>> >> > > > > > >>> >> > > > There is one known issue in UserScore example for Dataflow > > >>> runner. > > >>> >> The > > >>> >> > > > issue has been fixed on master branch and does not require a > > >>> >> > cherry-pick > > >>> >> > > at > > >>> >> > > > this point. A workaround is to pass --save_main_session > > pipeline > > >>> >> > option. > > >>> >> > > > > > >>> >> > > > RC4 looks good to me so far. > > >>> >> > > > > > >>> >> > > > > > >>> >> > > > On Fri, Nov 17, 2017 at 11:30 AM, Kenneth Knowles > > >>> >> > <k...@google.com.invalid > > >>> >> > > > > > >>> >> > > > wrote: > > >>> >> > > > > > >>> >> > > > > Hi all, > > >>> >> > > > > > > >>> >> > > > > Following up on past discussions and > > >>> >> > > > > https://issues.apache.org/jira/browse/BEAM-1189 I have > > >>> prepared a > > >>> >> > > > > spreadsheet so we can sign up for validation steps that > > must be > > >>> >> done > > >>> >> > > by a > > >>> >> > > > > human. > > >>> >> > > > > > > >>> >> > > > > The spreadsheet for 2.2.0 is at > > >>> >> > > > > https://s.apache.org/beam-2.2.0-release-validation. > > Everyone > > >>> can > > >>> >> > edit, > > >>> >> > > > so > > >>> >> > > > > it is easy to sign up or add new rows. > > >>> >> > > > > > > >>> >> > > > > FYI the template is at > > >>> >> https://s.apache.org/beam-release-validation. > > >>> >> > > > After > > >>> >> > > > > this release, I will update the rows in the template to > > match > > >>> >> > whatever > > >>> >> > > we > > >>> >> > > > > have added to the validation for 2.2.0. > > >>> >> > > > > > > >>> >> > > > > Kenn > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > On Thu, Nov 16, 2017 at 10:08 PM, Reuven Lax > > >>> >> > <re...@google.com.invalid > > >>> >> > > > > > >>> >> > > > > wrote: > > >>> >> > > > > > > >>> >> > > > > > Hi everyone, > > >>> >> > > > > > > > >>> >> > > > > > Please review and vote on the release candidate #4 for > the > > >>> >> version > > >>> >> > > > 2.2.0, > > >>> >> > > > > > as follows: > > >>> >> > > > > > [ ] +1, Approve the release > > >>> >> > > > > > [ ] -1, Do not approve the release (please provide > > specific > > >>> >> > > comments) > > >>> >> > > > > > > > >>> >> > > > > > > > >>> >> > > > > > The complete staging area is available for your review, > > which > > >>> >> > > includes: > > >>> >> > > > > > * JIRA release notes [1], > > >>> >> > > > > > * the official Apache source release to be deployed to > > >>> >> > > > dist.apache.org > > >>> >> > > > > > [2], > > >>> >> > > > > > which is signed with the key with fingerprint B98B7708 > > [3], > > >>> >> > > > > > * all artifacts to be deployed to the Maven Central > > >>> Repository > > >>> >> > [4], > > >>> >> > > > > > * source code tag "v2.2.0-RC4" [5], > > >>> >> > > > > > * website pull request listing the release and > > publishing > > >>> the > > >>> >> API > > >>> >> > > > > > reference manual [6]. > > >>> >> > > > > > * Java artifacts were built with Maven 3.5.0 and > > >>> OpenJDK/Oracle > > >>> >> > JDK > > >>> >> > > > > > 1.8.0_144. > > >>> >> > > > > > * Python artifacts are deployed along with the source > > >>> release > > >>> >> to > > >>> >> > > the > > >>> >> > > > > > dist.apache.org [2]. > > >>> >> > > > > > > > >>> >> > > > > > The vote will be open for at least 72 hours. It is > > adopted by > > >>> >> > > majority > > >>> >> > > > > > approval, with at least 3 PMC affirmative votes. > > >>> >> > > > > > > > >>> >> > > > > > Thanks, > > >>> >> > > > > > Reuven > > >>> >> > > > > > > > >>> >> > > > > > [1] https://issues.apache.org/ > > jira/secure/ReleaseNote.jspa?p > > >>> >> > > > > > rojectId=12319527&version=12341044 > > >>> >> > > > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.2.0/ > > >>> >> > > > > > [3] > https://dist.apache.org/repos/dist/release/beam/KEYS > > >>> >> > > > > > [4] https://repository.apache.org/content/repositories/ > > >>> >> > > > > orgapachebeam-1025/ > > >>> >> > > > > > [5] https://github.com/apache/beam/tree/v2.2.0-RC4 > > >>> >> > > > > > <https://github.com/apache/beam/tree/v2.2.0-RC4> > > >>> >> > > > > > [6] https://github.com/apache/beam-site/pull/337 > > >>> >> > > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > >>> >> > > >>> > > >> > > >> > > >
