See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/223/display/redirect?page=changes>
Changes: [herohde] Initial sketches of a Go SDK [herohde] Initial version of the direct style w/ direct runner. Incomplete. [herohde] Add Data as UserFn context w/ immediate value. [herohde] Added no-I/O wordcount for profiling. [herohde] Fleshed out possible approach to generic transformations. [herohde] Add “dag” example that use multiplexing and side input. [herohde] Added a more complex DAG example. [herohde] Add yatzy example with more complex construction-time setup [herohde] Add proto for Fn API [herohde] Add beam.Composite helper for the most common pattern to align with java [herohde] Move pipeline-construction time errors into an accumulator [herohde] Add Dataflow job and Fn API clients. Incomplete. [herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete. [herohde] Add tentative runner indirection (default: local). [herohde] Made data flow runner detect user main for cross-compilation. [herohde] Remove error accumulation in favor of panic. [herohde] Improve Dataflow translation of coders, side input and composite names. [herohde] Fix name for AsView. [herohde] Use 2 grpc endpoints in harness [herohde] Add gRPC harness logging [herohde] Flesh out harness and serialization further. [herohde] Made the dataflow runner wait for job termination by default [herohde] beam: [herohde] beam: [herohde] combinefn.go: fix compilation issues [herohde] Improve dataflow serialization and execution. Incomplete. [herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging. [herohde] Move the 30s sleep for logging to the harness instead of in WordCap. [herohde] Post-review updates. [herohde] Doc updates. [herohde] Flesh out coders. Incomplete. [herohde] Added prototype implementation of more coders and the runner source. [herohde] dofn: illustrates how dofns are written. [herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0 [herohde] dofn: timers [herohde] Complete revamp: coders, graph and execution use element-wise [herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is [herohde] Added more godoc comments to graph types. [herohde] Added more comments plus made local GBK use coder equality. [herohde] Added Flatten support and “forest” example that uses it. [herohde] Move bigqueryio to defunct [herohde] Make forest example print less [herohde] Add external struct registry and serialization. [herohde] Updated comments in node.go. [herohde] Replace real type with 'full type' since that's the current term. [herohde] Refactor Fn API dependency. [herohde] Added more comments to the runner/dataflow and runner/beamexec packages [herohde] Fix most go vet issues [herohde] Make core operations panic to cut down on the error propagation [herohde] Add more comments to the graph package. [herohde] Add DoFn wrapper to handle either function or (ptr to) struct [herohde] Fix remaining go vet warnings. [herohde] Code review for beam/graph/coder package. [herohde] Code review of the runtime/graphx package. [herohde] Remove Data options in favor of using a Fn struct [herohde] Code review of the beam/graph/userfn package. [herohde] Code review for beam/graph package. [herohde] godoc for runtime/graphx [herohde] Add support for []T and Combine functions [herohde] Add adapted documentation from the Java SDK to the beam package [herohde] Update snapshot of Fn API. [herohde] Add experiments flag to the Dataflow runner [herohde] Remove context arg from beamexec.Init [herohde] Migration to Runner API. [herohde] Add support for creating DOT graphs. [herohde] Make pretty printing of types and coders more concise [herohde] Add flexible Signature to aid type checking [herohde] Adding unit testability to harness translation. [herohde] Fix crash due to initialization order [herohde] Add CreateValues and Impulse [herohde] Add Runner API support for WindowingStrategy. [herohde] Run goimports on baseline. [herohde] Fix encoding of global window strategy. [herohde] Ensure the windowed value is atomically encoded. [herohde] Limit gRPC messages to max size. [herohde] Developer conveniences for running jobs. [herohde] Fix sends to not close the network channel. [herohde] Add re-iterable side input [herohde] Add per-key Combine [herohde] Add Min [herohde] Reorganize non-user-facing code into core [herohde] Make type register reject unnamed or predeclared types [herohde] Add type specialization tool [herohde] Don't run grpc plugin in generate phase. [herohde] Fix import reference path for runner API proto. [herohde] Revamp runner registration as _ imports [herohde] Add stats.Max and Mean [herohde] Add global pipeline options [herohde] Unify global and per-key combiners [herohde] Add beam convenience wrapper for imports and runner selection [herohde] Add session recording and CPU profiling to harness. [herohde] Add ptest and passert for testing pipelines [herohde] Add GCS and glob support to textio [herohde] Add BigQuery IO and examples [herohde] Adds a session runner for testing. [herohde] Add Partition and dynamic functions [herohde] Adding example that returns 10 words that contain provided search [herohde] Remove duplicate LOG line [herohde] Enable Combine Fns in Dataflow runner by modifying translation. [herohde] Fixing type bug by dropping T and using underlying type of value in [herohde] Adding graph validation at build time. [herohde] Import the Fn API changes. [herohde] Simple changes to support new Fn API coder changes. [herohde] Update translator to work with new Fn API changes. [herohde] Use appropriate equality tests. [herohde] Fix test to not use path of package. [herohde] Renaming directory to match package name. [herohde] Fixing random nits in comments. [herohde] Modify build command to avoid bash. [herohde] Fixing selected golint issues. [herohde] Addressing import review comments. [herohde] Add coder specialization for bytes/strings. [herohde] Adding unit tests to stats. [herohde] Fixing typo. [herohde] Add beam.External [herohde] Fix grpc.Dial calls to block properly. [herohde] Creates a symtab verifier by running Sym2Addr and Addr2Sym in a binary. [herohde] Add spec field to help interpretation of payload. [herohde] Use beam.T alias for typex.T etc with Go 1.9 [herohde] Move shared GCP options to a separate package [herohde] Update portability protos [herohde] Remove old source/sink from beam package [herohde] Add context-aware logging for both pipeline-submission time and runtime [herohde] Fix coder inference for strings. [herohde] Improve tornadoes example [herohde] Fix beam.External to map arguments correctly. [herohde] Added comments to yatzy and forest [herohde] Add comments to tornadoes from the java counterpart [herohde] Rename Pipeline Composite to Scope [herohde] Add 3 progressive wordcount examples [herohde] Clarify comments in wordcount pipelines [herohde] Add apache 2.0 license to files [herohde] Updates to examples. [herohde] Adding more godoc for the main beam package. [herohde] Update to new proto structure [herohde] Split Combine and fields in to global and per-key variants [herohde] Refactor Flatten of a single []T into Explode [herohde] Rename local runner to direct runner [herohde] Fix argument index error in ParDo execution [herohde] Add Apache copyright header to files that need it. [herohde] Made debug.Head not just work per bundle [herohde] Impose a total ordering on Fn parameters. [herohde] Rename Dedup to Distinct for consistency with other SDKs [herohde] Add coder to model coder translation [herohde] Simplify harness coder translation [herohde] Split Pipeline into Pipeline and Scope [herohde] Relocate Go SDK code [herohde] Fix Go SDK maven build [herohde] Move Go SKD to latest version of bigquery [herohde] Add Go SDK container image [herohde] Add Go SDK README [herohde] Update version for Go Dataflow pipelines [herohde] Make Scope a value type [herohde] Add Go graph/pipeline translation [herohde] Stage Go model pipeline for Dataflow [herohde] Use pipeline unmarhaller in runtime harness [herohde] CR: [BEAM-3287] Use model pipelines in Go SDK [herohde] CR: [BEAM-3287] Use model pipelines in Go SDK [herohde] Fix name of syscallx ErrUnsupported [herohde] Allow any named type to be registered and serialized as external [herohde] Add more package comments for core packages [herohde] Make Go SDK External a graph primitive [herohde] Cache Go runtime symbol lookups [wcn] Fix storagePath variable. [wcn] BEAM-3368 fix translation for external [robertwb] [BEAM-3356] Add Go SDK int and varint custom coders (#4276) [lcwik] BEAM-3361 Increase Go gRPC message size [herohde] Go SDK runtime revamp [lcwik] Add a few function call overhead benchmarks [lcwik] Add type-specialized emitters [lcwik] BEAM-3324 improve symtab memory usage [lcwik] BEAM-3324 improve symtab memory usage [lcwik] BEAM-3324 improve symtab memory usage [lcwik] Store objects in pool so they can be reused. [lcwik] Add builtin varint coder [herohde] Type-specialize custom decoders and encoders in Go SDK runtime [herohde] Type-specialize iterators and side input in the Go SDK [herohde] Add warnings if Go runtime registrations are overwritten [herohde] Add reusable element coders in Go SDK runtime [wcn] Updated translater to preserve payload and its URN. [herohde] Initial version of type-specialized general reflective calls [herohde] Add general-purpose untyped callers in Go SDK runtime [herohde] Use fast caller for filter transform predicate [herohde] CR: Clarified comment on caller template [herohde] Fix value encoding for Create [lcwik] BEAM-3473: Fix GroupByKey iterators to be initialized. [lcwik] BEAM-3474 Include stacks in panic messages. [lcwik] BEAM-3299: Add source reporting support. [lcwik] Remove GetId() call from under lock. [lcwik] Add additional comments about concurrency invariants. [lcwik] Add initialization of active plans map. [lcwik] Renamed Go runtime Caller to Func and added name [lcwik] Use reflectx.Func as the fundamental function representation [lcwik] CR: fix DynFn comments [lcwik] CR: fix comments [herohde] Avoid reflect.Value conversions in Go runtime [wcn] Allow grpcx.Dial to support overrides. [rober] Use a typeswitch instead of reflect.Convert when encoding strings or [github] Update coder.go [robert] Replace reflective convert to direct convert. [rober] Fix beam.Combine to combine globally [herohde] Add optional function registration to Go SDK runtime [rober] fixup! Remove reflection from varint codecs [herohde] Changed core GBK to CoGBK [herohde] Add CoGBK support to direct runner and Join example [herohde] [BEAM-3316] Translate bundle descriptors directly to execution plans in [herohde] Translate CoGBK into GBK for Dataflow and model pipeline runners [herohde] CR: [BEAM-3302] Support CoGBK in the Go SDK [herohde] [BEAM-3579] Fix textio.Write [herohde] CR: fix Go SDK textio.Write [wcn] Fixing filename. [ehudm] Integration test for Python HDFS implementation. [wcn] Improve rendering of DOT diagrams. [herohde] Update Go SDK coder constants [1028332163] replace mockito-all [1028332163] replace mockito [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] replace mockito-all and harcrest-all [1028332163] ignoredUnusedDeclaredDependencies [1028332163] ban hamcrest-all and mockito-all [1028332163] ban mockito-all and hamcrest-all [tgroh] Make Impulse#create() visible [tgroh] Update QueryablePipeline Factory Method Name [tgroh] Add JavaReadViaImpulse to core-construction [amyrvold] Beam runner inventory, run as a cron job, not on each CL [amyrvold] [BEAM-3775] Increase timeout in [ehudm] Improve FileBasedSink rename safety. [ehudm] Add missing import statements. [lcwik] [BEAM-3690] ban mockito-all and hamcrest-all ------------------------------------------ [...truncated 13.31 KB...] Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.5.0.dev0) Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) Using cached funcsigs-1.0.2-py2.py3-none-any.whl Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0) Using cached pbr-3.1.1-py2.py3-none-any.whl Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) Using cached rsa-3.4.2-py2.py3-none-any.whl Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0) Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0) Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0) Using cached fasteners-0.14.1-py2.py3-none-any.whl Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0) Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0) Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0) Using cached monotonic-1.4-py2.py3-none-any.whl Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Using cached google_auth-1.4.1-py2.py3-none-any.whl Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Using cached google_gax-0.15.16-py2.py3-none-any.whl Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Using cached cachetools-2.0.1-py2.py3-none-any.whl Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0) Installing collected packages: avro, dill, protobuf, grpcio, hdfs, httplib2, funcsigs, pbr, mock, rsa, pyasn1-modules, oauth2client, pyvcf, typing, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, pyhamcrest, apache-beam Running setup.py develop for apache-beam Successfully installed apache-beam avro-1.8.2 cachetools-2.0.1 dill-0.2.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.10.0 hdfs-2.1.0 httplib2-0.9.2 mock-2.0.0 monotonic-1.4 oauth2client-4.1.2 pbr-3.1.1 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.2 pyasn1-modules-0.2.1 pyhamcrest-1.9.0 pyvcf-0.6.8 rsa-3.4.2 typing-3.6.4 [beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe /tmp/jenkins1680162444616273758.sh + .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.avroioit_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/> --official=true --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/223/]' '--beam_extra_mvn_properties=[filesystem=gcs]' 2018-03-07 00:01:38,273 219817aa MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/219817aa/pkb.log> 2018-03-07 00:01:38,273 219817aa MainThread INFO PerfKitBenchmarker version: v1.12.0-387-g202cd32 2018-03-07 00:01:38,274 219817aa MainThread INFO Flag values: --beam_extra_mvn_properties=[filesystem=gcs] --beam_it_class=org.apache.beam.sdk.io.avro.AvroIOIT --beam_it_timeout=1200 --beam_it_module=sdks/java/io/file-based-io-tests --beam_sdk=java --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/> --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/223/] --beam_prebuilt --project=apache-beam-testing --bigquery_table=beam_performance.avroioit_pkb_results --official --dpb_log_level=INFO --beam_it_profile=io-it --benchmarks=beam_integration_benchmark 2018-03-07 00:01:38,665 219817aa MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo. 2018-03-07 00:01:38,665 219817aa MainThread INFO Initializing the edw service decoder 2018-03-07 00:01:38,809 219817aa MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark 2018-03-07 00:01:38,811 219817aa MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark 2018-03-07 00:01:38,812 219817aa MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git 2018-03-07 00:01:46,030 219817aa MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2018-03-07 00:01:46,037 219817aa MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn -e verify -Dit.test=org.apache.beam.sdk.io.avro.AvroIOIT -DskipITs=false -pl sdks/java/io/file-based-io-tests -Pio-it -Pdataflow-runner -Dfilesystem=gcs -DintegrationTestPipelineOptions=["--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--numberOfRecords=1000000","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/223/","--runner=TestDataflowRunner"] 2018-03-07 00:01:52,333 219817aa MainThread beam_integration_benchmark(1/1) INFO Ran: {/home/jenkins/tools/maven/latest/bin/mvn -e verify -Dit.test=org.apache.beam.sdk.io.avro.AvroIOIT -DskipITs=false -pl sdks/java/io/file-based-io-tests -Pio-it -Pdataflow-runner -Dfilesystem=gcs -DintegrationTestPipelineOptions=["--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--numberOfRecords=1000000","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/223/","--runner=TestDataflowRunner"]} ReturnCode:1, WallTime:0:06.29s, CPU:9.88s, MaxMemory:263396kb STDOUT: [INFO] Error stacktraces are turned on. [INFO] Scanning for projects... [ERROR] [ERROR] Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM for org.apache.beam:beam-sdks-go-container:[unknown-version]: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11 @ [ERROR] The build could not read 1 project -> [Help 1] org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM for org.apache.beam:beam-sdks-go-container:[unknown-version]: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11 at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:382) at org.apache.maven.graph.DefaultGraphBuilder.collectProjects (DefaultGraphBuilder.java:400) at org.apache.maven.graph.DefaultGraphBuilder.getProjectsForMavenReactor (DefaultGraphBuilder.java:391) at org.apache.maven.graph.DefaultGraphBuilder.build (DefaultGraphBuilder.java:78) at org.apache.maven.DefaultMaven.buildGraph (DefaultMaven.java:511) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:221) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) [ERROR] [ERROR] The project org.apache.beam:beam-sdks-go-container:[unknown-version] (<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/219817aa/beam/sdks/go/container/pom.xml)> has 1 error [ERROR] Non-resolvable parent POM for org.apache.beam:beam-sdks-go-container:[unknown-version]: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11 -> [Help 2] org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:197) at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:243) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally (DefaultModelBuilder.java:1051) at org.apache.maven.model.building.DefaultModelBuilder.readParent (DefaultModelBuilder.java:829) at org.apache.maven.model.building.DefaultModelBuilder.build (DefaultModelBuilder.java:331) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:429) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:362) at org.apache.maven.graph.DefaultGraphBuilder.collectProjects (DefaultGraphBuilder.java:400) at org.apache.maven.graph.DefaultGraphBuilder.getProjectsForMavenReactor (DefaultGraphBuilder.java:391) at org.apache.maven.graph.DefaultGraphBuilder.build (DefaultGraphBuilder.java:78) at org.apache.maven.DefaultMaven.buildGraph (DefaultMaven.java:511) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:221) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve (DefaultArtifactResolver.java:422) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts (DefaultArtifactResolver.java:224) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact (DefaultArtifactResolver.java:201) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact (DefaultRepositorySystem.java:260) at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:193) at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:243) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally (DefaultModelBuilder.java:1051) at org.apache.maven.model.building.DefaultModelBuilder.readParent (DefaultModelBuilder.java:829) at org.apache.maven.model.building.DefaultModelBuilder.build (DefaultModelBuilder.java:331) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:429) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:362) at org.apache.maven.graph.DefaultGraphBuilder.collectProjects (DefaultGraphBuilder.java:400) at org.apache.maven.graph.DefaultGraphBuilder.getProjectsForMavenReactor (DefaultGraphBuilder.java:391) at org.apache.maven.graph.DefaultGraphBuilder.build (DefaultGraphBuilder.java:78) at org.apache.maven.DefaultMaven.buildGraph (DefaultMaven.java:511) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:221) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve (DefaultArtifactResolver.java:412) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts (DefaultArtifactResolver.java:224) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact (DefaultArtifactResolver.java:201) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact (DefaultRepositorySystem.java:260) at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:193) at org.apache.maven.project.ProjectModelResolver.resolveModel (ProjectModelResolver.java:243) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally (DefaultModelBuilder.java:1051) at org.apache.maven.model.building.DefaultModelBuilder.readParent (DefaultModelBuilder.java:829) at org.apache.maven.model.building.DefaultModelBuilder.build (DefaultModelBuilder.java:331) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:429) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:515) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:398) at org.apache.maven.project.DefaultProjectBuilder.build (DefaultProjectBuilder.java:362) at org.apache.maven.graph.DefaultGraphBuilder.collectProjects (DefaultGraphBuilder.java:400) at org.apache.maven.graph.DefaultGraphBuilder.getProjectsForMavenReactor (DefaultGraphBuilder.java:391) at org.apache.maven.graph.DefaultGraphBuilder.build (DefaultGraphBuilder.java:78) at org.apache.maven.DefaultMaven.buildGraph (DefaultMaven.java:511) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:221) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException STDERR: 2018-03-07 00:01:52,333 219817aa MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 622, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 525, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-03-07 00:01:52,334 219817aa MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-03-07 00:01:52,336 219817aa MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 732, in RunBenchmarkTask RunBenchmark(spec, collector) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 622, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 525, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-03-07 00:01:52,337 219817aa MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue. 2018-03-07 00:01:52,337 219817aa MainThread beam_integration_benchmark(1/1) INFO Benchmark run statuses: --------------------------------------------------------------------------------- Name UID Status Failed Substatus --------------------------------------------------------------------------------- beam_integration_benchmark beam_integration_benchmark0 FAILED --------------------------------------------------------------------------------- Success rate: 0.00% (0/1) 2018-03-07 00:01:52,337 219817aa MainThread beam_integration_benchmark(1/1) INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/219817aa/pkb.log> 2018-03-07 00:01:52,337 219817aa MainThread beam_integration_benchmark(1/1) INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/219817aa/completion_statuses.json> Build step 'Execute shell' marked build as failure