On Mon, Nov 20, 2017 at 5:01 PM, Eugene Kirpichov < [email protected]> wrote:
> In the verification spreadsheet, I'm not sure I understand the difference > between the "YARN" and "Standalone cluster/service". Which is Dataproc? It > definitely uses YARN, but it is also a standalone cluster/service. Does it > count for both? > No, it doesn't. A number of runners have their own non-YARN cluster mode. I would expect that the launching experience might be different and the portable container management to differ. If they are identical, experts in those systems should feel free to coalesce the rows. Conversely, as other platforms become supported, they could be added or not based on whether they are substantively different from a user experience or QA point of view. Kenn > Seems now we're missing just Apex and Flink cluster verifications. > > *though Spark runner took 6x longer to run UserScore, partially I guess > because it didn't do autoscaling (Dataflow runner ramped up to 5 workers > whereas Spark runner used 2 workers). For some reason Spark runner chose > not to split the 10GB input files into chunks. > > On Mon, Nov 20, 2017 at 3:46 PM Reuven Lax <[email protected]> > wrote: > > > Done > > > > On Tue, Nov 21, 2017 at 3:08 AM, Robert Bradshaw < > > [email protected]> wrote: > > > > > Thanks. You need to re-sign as well. > > > > > > On Mon, Nov 20, 2017 at 12:14 AM, Reuven Lax <[email protected] > > > > > wrote: > > > > FYI these generated files have been removed from the source > > distribution. > > > > > > > > On Sat, Nov 18, 2017 at 9:09 AM, Reuven Lax <[email protected]> > wrote: > > > > > > > >> hmmm, I thought I removed those generated files from the zip file > > before > > > >> sending this email. Let me check again. > > > >> > > > >> Reuven > > > >> > > > >> On Sat, Nov 18, 2017 at 8:52 AM, Robert Bradshaw < > > > >> [email protected]> wrote: > > > >> > > > >>> The source distribution contains a couple of files not on github > > (e.g. > > > >>> folders that were added on master, Python generated files). The pom > > > >>> files differed only by missing -SNAPSHOT, other than that > presumably > > > >>> the source release should just be "wget > > > >>> https://github.com/apache/beam/archive/release-2.2.0.zip";? > > > >>> > > > >>> diff -rq apache-beam-2.2.0 beam/ | grep -v pom.xml > > > >>> > > > >>> # OK? > > > >>> > > > >>> Only in apache-beam-2.2.0: DEPENDENCIES > > > >>> > > > >>> # Expected. > > > >>> > > > >>> Only in beam/: .git > > > >>> Only in beam/: .gitattributes > > > >>> Only in beam/: .gitignore > > > >>> > > > >>> # These folders are probably from switching around between master > and > > > >>> git branches. > > > >>> > > > >>> Only in apache-beam-2.2.0: model > > > >>> Only in apache-beam-2.2.0/runners/flink: examples > > > >>> Only in apache-beam-2.2.0/runners/flink: runner > > > >>> Only in apache-beam-2.2.0/runners/gearpump: jarstore > > > >>> Only in apache-beam-2.2.0/sdks/java/extensions: gcp-core > > > >>> Only in apache-beam-2.2.0/sdks/java/extensions: sketching > > > >>> Only in apache-beam-2.2.0/sdks/java/io: file-based-io-tests > > > >>> Only in apache-beam-2.2.0/sdks/java/io: hdfs > > > >>> Only in apache-beam-2.2.0/sdks/java/maven-archetypes/examples/src/ > ma > > > >>> in/resources/archetype-resources: > > > >>> src > > > >>> Only in apache-beam-2.2.0/sdks/java/maven-archetypes/examples- > java8/ > > > >>> src/main/resources/archetype-resources: > > > >>> src > > > >>> Only in apache-beam-2.2.0/sdks/java: microbenchmarks > > > >>> > > > >>> # Here's the generated protos. > > > >>> > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_artifact_api_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_artifact_api_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_fn_api_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_fn_api_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_job_api_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_job_api_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_provision_api_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_provision_api_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_runner_api_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> beam_runner_api_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> endpoints_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> endpoints_pb2.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> standard_window_fns_pb2_grpc.py > > > >>> Only in apache-beam-2.2.0/sdks/python/apache_beam/portability/api: > > > >>> standard_window_fns_pb2.py > > > >>> > > > >>> And some other sdist generated Python files. > > > >>> > > > >>> Only in apache-beam-2.2.0/sdks/python: .eggs > > > >>> Only in apache-beam-2.2.0/sdks/python: LICENSE > > > >>> Only in apache-beam-2.2.0/sdks/python: NOTICE > > > >>> Only in apache-beam-2.2.0/sdks/python: README.md > > > >>> > > > >>> Presumably we should just purge these files from the rc? > > > >>> > > > >>> > > > >>> FWIW, the Python tarball looks fine. > > > >>> > > > >>> On Fri, Nov 17, 2017 at 4:40 PM, Eugene Kirpichov > > > >>> <[email protected]> wrote: > > > >>> > How can I specify a dependency on the staged RC? E.g. I'm trying > to > > > >>> > validate the quickstart per > > > >>> > https://beam.apache.org/get-started/quickstart-java/ and > > specifying > > > >>> version > > > >>> > 2.2.0 doesn't work I suppose because it's not released yet. > Should > > I > > > >>> pass > > > >>> > some command-line flag to mvn to make it fetch the version from > the > > > >>> staging > > > >>> > area? > > > >>> > > > > >>> > On Fri, Nov 17, 2017 at 4:37 PM Lukasz Cwik > > <[email protected] > > > > > > > >>> > wrote: > > > >>> > > > > >>> >> Its open to all, its just that there are binding votes and > > > non-binding > > > >>> >> votes. > > > >>> >> > > > >>> >> On Fri, Nov 17, 2017 at 4:26 PM, Valentyn Tymofieiev < > > > >>> >> [email protected]> wrote: > > > >>> >> > > > >>> >> > I have a process question: is the vote open for committers > only > > or > > > >>> for > > > >>> >> all > > > >>> >> > contributors? > > > >>> >> > > > > >>> >> > On Fri, Nov 17, 2017 at 4:06 PM, Lukasz Cwik > > > >>> <[email protected]> > > > >>> >> > wrote: > > > >>> >> > > > > >>> >> > > +1, Approve the release > > > >>> >> > > > > > >>> >> > > I have verified the wordcount quickstart on the Apache Beam > > > website > > > >>> >> using > > > >>> >> > > Apex, DirectRunner, Flink & Spark on Linux. > > > >>> >> > > > > > >>> >> > > The Gearpump runner is yet to have a quickstart listed on > our > > > >>> website. > > > >>> >> > > Adding the quickstart is already represented by this > existing > > > >>> issue: > > > >>> >> > > https://issues.apache.org/jira/browse/BEAM-2692 > > > >>> >> > > > > > >>> >> > > On Fri, Nov 17, 2017 at 11:50 AM, Valentyn Tymofieiev < > > > >>> >> > > [email protected]> wrote: > > > >>> >> > > > > > >>> >> > > > I have verified: SHA & MD5 signatures of Python artifacts > in > > > >>> [2], and > > > >>> >> > > > checked Python side of the validation checklist on Linux. > > > >>> >> > > > > > > >>> >> > > > There is one known issue in UserScore example for Dataflow > > > >>> runner. > > > >>> >> The > > > >>> >> > > > issue has been fixed on master branch and does not > require a > > > >>> >> > cherry-pick > > > >>> >> > > at > > > >>> >> > > > this point. A workaround is to pass --save_main_session > > > pipeline > > > >>> >> > option. > > > >>> >> > > > > > > >>> >> > > > RC4 looks good to me so far. > > > >>> >> > > > > > > >>> >> > > > > > > >>> >> > > > On Fri, Nov 17, 2017 at 11:30 AM, Kenneth Knowles > > > >>> >> > <[email protected] > > > >>> >> > > > > > > >>> >> > > > wrote: > > > >>> >> > > > > > > >>> >> > > > > Hi all, > > > >>> >> > > > > > > > >>> >> > > > > Following up on past discussions and > > > >>> >> > > > > https://issues.apache.org/jira/browse/BEAM-1189 I have > > > >>> prepared a > > > >>> >> > > > > spreadsheet so we can sign up for validation steps that > > > must be > > > >>> >> done > > > >>> >> > > by a > > > >>> >> > > > > human. > > > >>> >> > > > > > > > >>> >> > > > > The spreadsheet for 2.2.0 is at > > > >>> >> > > > > https://s.apache.org/beam-2.2.0-release-validation. > > > Everyone > > > >>> can > > > >>> >> > edit, > > > >>> >> > > > so > > > >>> >> > > > > it is easy to sign up or add new rows. > > > >>> >> > > > > > > > >>> >> > > > > FYI the template is at > > > >>> >> https://s.apache.org/beam-release-validation. > > > >>> >> > > > After > > > >>> >> > > > > this release, I will update the rows in the template to > > > match > > > >>> >> > whatever > > > >>> >> > > we > > > >>> >> > > > > have added to the validation for 2.2.0. > > > >>> >> > > > > > > > >>> >> > > > > Kenn > > > >>> >> > > > > > > > >>> >> > > > > > > > >>> >> > > > > On Thu, Nov 16, 2017 at 10:08 PM, Reuven Lax > > > >>> >> > <[email protected] > > > >>> >> > > > > > > >>> >> > > > > wrote: > > > >>> >> > > > > > > > >>> >> > > > > > Hi everyone, > > > >>> >> > > > > > > > > >>> >> > > > > > Please review and vote on the release candidate #4 for > > the > > > >>> >> version > > > >>> >> > > > 2.2.0, > > > >>> >> > > > > > as follows: > > > >>> >> > > > > > [ ] +1, Approve the release > > > >>> >> > > > > > [ ] -1, Do not approve the release (please provide > > > specific > > > >>> >> > > comments) > > > >>> >> > > > > > > > > >>> >> > > > > > > > > >>> >> > > > > > The complete staging area is available for your > review, > > > which > > > >>> >> > > includes: > > > >>> >> > > > > > * JIRA release notes [1], > > > >>> >> > > > > > * the official Apache source release to be deployed > to > > > >>> >> > > > dist.apache.org > > > >>> >> > > > > > [2], > > > >>> >> > > > > > which is signed with the key with fingerprint B98B7708 > > > [3], > > > >>> >> > > > > > * all artifacts to be deployed to the Maven Central > > > >>> Repository > > > >>> >> > [4], > > > >>> >> > > > > > * source code tag "v2.2.0-RC4" [5], > > > >>> >> > > > > > * website pull request listing the release and > > > publishing > > > >>> the > > > >>> >> API > > > >>> >> > > > > > reference manual [6]. > > > >>> >> > > > > > * Java artifacts were built with Maven 3.5.0 and > > > >>> OpenJDK/Oracle > > > >>> >> > JDK > > > >>> >> > > > > > 1.8.0_144. > > > >>> >> > > > > > * Python artifacts are deployed along with the > source > > > >>> release > > > >>> >> to > > > >>> >> > > the > > > >>> >> > > > > > dist.apache.org [2]. > > > >>> >> > > > > > > > > >>> >> > > > > > The vote will be open for at least 72 hours. It is > > > adopted by > > > >>> >> > > majority > > > >>> >> > > > > > approval, with at least 3 PMC affirmative votes. > > > >>> >> > > > > > > > > >>> >> > > > > > Thanks, > > > >>> >> > > > > > Reuven > > > >>> >> > > > > > > > > >>> >> > > > > > [1] https://issues.apache.org/ > > > jira/secure/ReleaseNote.jspa?p > > > >>> >> > > > > > rojectId=12319527&version=12341044 > > > >>> >> > > > > > [2] https://dist.apache.org/repos/ > dist/dev/beam/2.2.0/ > > > >>> >> > > > > > [3] > > https://dist.apache.org/repos/dist/release/beam/KEYS > > > >>> >> > > > > > [4] https://repository.apache.org/ > content/repositories/ > > > >>> >> > > > > orgapachebeam-1025/ > > > >>> >> > > > > > [5] https://github.com/apache/beam/tree/v2.2.0-RC4 > > > >>> >> > > > > > <https://github.com/apache/beam/tree/v2.2.0-RC4> > > > >>> >> > > > > > [6] https://github.com/apache/beam-site/pull/337 > > > >>> >> > > > > > > > > >>> >> > > > > > > > >>> >> > > > > > > >>> >> > > > > > >>> >> > > > > >>> >> > > > >>> > > > >> > > > >> > > > > > >
