See <https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/500/display/redirect?page=changes>
Changes: [Etienne Chauchot] [BEAM-12591] Put Spark Structured Streaming runner sources back to main [Etienne Chauchot] [BEAM-12629] As spark DataSourceV2 is only available for spark 2, [Etienne Chauchot] [BEAM-12627] Deal with spark Encoders braking change between spark 2 and [Etienne Chauchot] [BEAM-12591] move SchemaHelpers to correct package [Etienne Chauchot] [BEAM-8470] Disable wait for termination in a streaming pipeline because [Etienne Chauchot] [BEAM-12630] Deal with breaking change in streaming pipelines start by [Etienne Chauchot] [BEAM-12629] Make source tests spark version agnostic and move them back [Etienne Chauchot] [BEAM-12629] Make a spark 3 source impl [Etienne Chauchot] [BEAM-12591] Fix checkstyle and spotless [Etienne Chauchot] [BEAM-12629] Reduce serializable to only needed classes and Fix schema [Etienne Chauchot] [BEAM-12591] Add checkstyle exceptions for version specific classes [Etienne Chauchot] [BEAM-12629] Fix sources javadocs and improve impl [Etienne Chauchot] [BEAM-12591] Add spark 3 to structured streaming validates runner tests [noreply] [BEAM-6516] Fixes race condition in RabbitMqIO causing duplicate acks ------------------------------------------ [...truncated 60.03 KB...] > Task :sdks:go:resolveBuildDependencies Resolving github.com/spf13/cobra: commit='93959269ad99e80983c9ba742a7e01203a4c0e4f', urls=[https://github.com/spf13/cobra.git, [email protected]:spf13/cobra.git] Resolving github.com/spf13/jwalterweatherman: commit='7c0cea34c8ece3fbeb2b27ab9b59511d360fb394', urls=[https://github.com/spf13/jwalterweatherman.git, [email protected]:spf13/jwalterweatherman.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving github.com/spf13/viper: commit='aafc9e6bc7b7bb53ddaa75a5ef49a17d6e654be5', urls=[https://github.com/spf13/viper.git, [email protected]:spf13/viper.git] Resolving github.com/stathat/go: commit='74669b9f388d9d788c97399a0824adbfee78400e', urls=[https://github.com/stathat/go.git, [email protected]:stathat/go.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving github.com/xordataexchange/crypt: commit='b2862e3d0a775f18c7cfe02273500ae307b61218', urls=[https://github.com/xordataexchange/crypt.git, [email protected]:xordataexchange/crypt.git] Resolving go.opencensus.io: commit='3fb168f674736c026e623310bfccb0691e6dec8a', urls=[https://github.com/census-instrumentation/opencensus-go] > Task :runners:google-cloud-dataflow-java:compileJava FROM-CACHE > Task :runners:google-cloud-dataflow-java:classes > Task :examples:java:compileJava FROM-CACHE > Task :examples:java:classes UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:compileTestJava FROM-CACHE > Task :sdks:java:io:google-cloud-platform:testClasses > Task :sdks:java:container:pullLicenses Collecting future<1.0.0,>=0.16.0 Using cached future-0.18.2-py3-none-any.whl Collecting pyyaml<6.0.0,>=3.12 Using cached PyYAML-5.3.1-cp35-cp35m-linux_x86_64.whl Collecting tenacity<6.0,>=5.0.2 Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB) Collecting soupsieve>1.2 Using cached soupsieve-2.1-py3-none-any.whl (32 kB) Collecting six>=1.9.0 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Installing collected packages: soupsieve, six, tenacity, pyyaml, future, beautifulsoup4 Successfully installed beautifulsoup4-4.9.3 future-0.18.2 pyyaml-5.3.1 six-1.16.0 soupsieve-2.1 tenacity-5.1.5 Executing <https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/build/virtualenv/bin/python> <https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py> --license_index=<https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/build/reports/dependency-license/index.json> --output_dir=<https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/build/target/java_third_party_licenses> --dep_url_yaml=<https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/license_scripts/dep_urls_java.yaml> > Task :runners:google-cloud-dataflow-java:jar > Task :examples:java:jar > Task :sdks:java:io:google-cloud-platform:testJar > Task :examples:java:compileTestJava FROM-CACHE > Task :examples:java:testClasses > Task :examples:java:testJar > Task :sdks:go:resolveBuildDependencies Resolving golang.org/x/crypto: commit='d9133f5469342136e669e85192a26056b587f503', urls=[https://go.googlesource.com/crypto] Resolving golang.org/x/debug: commit='95515998a8a4bd7448134b2cb5971dbeb12e0b77', urls=[https://go.googlesource.com/debug] Resolving golang.org/x/net: commit='2fb46b16b8dda405028c50f7c7f0f9dd1fa6bfb1', urls=[https://go.googlesource.com/net] Resolving golang.org/x/oauth2: commit='5d25da1a8d43b66f2898c444f899c7bcfd6a407e', urls=[https://go.googlesource.com/oauth2] Resolving golang.org/x/sync: commit='fd80eb99c8f653c847d294a001bdf2a3a6f768f5', urls=[https://go.googlesource.com/sync] > Task :sdks:java:container:pullLicenses INFO:root:Pulling license for 207 dependencies using 16 threads. INFO:root:pull_licenses_java.py succeed. It took 5.856141 seconds with 16 threads. Copying licenses from <https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/build/target/java_third_party_licenses> to <https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/java/container/build/target/third_party_licenses.> Finished license_scripts.sh > Task :sdks:java:container:java8:copyJavaThirdPartyLicenses NO-SOURCE > Task :sdks:go:resolveBuildDependencies Resolving golang.org/x/sys: commit='fde4db37ae7ad8191b03d30d27f258b5291ae4e3', urls=[https://go.googlesource.com/sys] Resolving golang.org/x/text: commit='23ae387dee1f90d29a23c0e87ee0b46038fbed0e', urls=[https://go.googlesource.com/text] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] > Task :release:go-licenses:java:dockerRun + go-licenses save github.com/apache/beam/sdks/java/container --save_path=/output/licenses + go-licenses csv github.com/apache/beam/sdks/java/container + tee /output/licenses/list.csv github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0 github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0 golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0 google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0 golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause + chmod -R a+w /output/licenses > Task :release:go-licenses:java:createLicenses > Task :sdks:java:container:java8:copyGolangLicenses > Task :sdks:go:resolveBuildDependencies Resolving google.golang.org/api: commit='0324d5e90dc7753607860272666845fad9ceb97e', urls=[https://code.googlesource.com/google-api-go-client] Resolving google.golang.org/genproto: commit='4d944d34d83c502a5f761500a14d8842648415c3', urls=[https://github.com/google/go-genproto] Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go] Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] > Task :sdks:go:installDependencies > Task :sdks:go:buildLinuxAmd64 > Task :sdks:go:goBuild > Task :sdks:java:container:resolveBuildDependencies Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/sdks/go> > Task :sdks:java:container:installDependencies > Task :sdks:java:container:buildLinuxAmd64 > Task :sdks:java:container:goBuild > Task :sdks:java:container:java8:copySdkHarnessLauncher > Task :sdks:java:container:java8:dockerPrepare > Task :sdks:java:container:java8:docker > Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03. As an alternative, use `gcloud auth configure-docker` to configure `docker` to use `gcloud` as a credential helper, then use `docker` as you would for non-GCR registries, e.g. `docker pull gcr.io/project-id/my-image`. Add `--verbosity=error` to silence this warning: `gcloud docker --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java] 0e6119e07a8d: Preparing 11249cf61e4a: Preparing 6a7c0d47e4e3: Preparing 6f1ec4d86684: Preparing 85453810317c: Preparing df55d1a4d5e8: Preparing 6faef509f6a4: Preparing 8abbd882e516: Preparing cda82d775113: Preparing 6506e5906164: Preparing 7f35e8d31259: Preparing 7a9f30f096f0: Preparing ea822a1a4f9d: Preparing fe6a347f93b5: Preparing f3d5b8f65132: Preparing ad83f0aa5c0a: Preparing 5a9a65095453: Preparing 4b0edb23340c: Preparing afa3e488a0ee: Preparing df55d1a4d5e8: Waiting 7f35e8d31259: Waiting 6faef509f6a4: Waiting 8abbd882e516: Waiting cda82d775113: Waiting 7a9f30f096f0: Waiting 6506e5906164: Waiting ea822a1a4f9d: Waiting fe6a347f93b5: Waiting f3d5b8f65132: Waiting ad83f0aa5c0a: Waiting 5a9a65095453: Waiting 4b0edb23340c: Waiting afa3e488a0ee: Waiting 6a7c0d47e4e3: Pushed 11249cf61e4a: Pushed 85453810317c: Pushed 6f1ec4d86684: Pushed 0e6119e07a8d: Pushed df55d1a4d5e8: Pushed cda82d775113: Pushed 8abbd882e516: Pushed ea822a1a4f9d: Layer already exists fe6a347f93b5: Layer already exists f3d5b8f65132: Layer already exists ad83f0aa5c0a: Layer already exists 5a9a65095453: Layer already exists 4b0edb23340c: Layer already exists afa3e488a0ee: Layer already exists 7f35e8d31259: Pushed 7a9f30f096f0: Pushed 6faef509f6a4: Pushed 6506e5906164: Pushed 20210805120100: digest: sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401 size: 4310 > Task :runners:google-cloud-dataflow-java:coreSDKJavaRunnerV2IntegrationTest > NO-SOURCE > Task :runners:google-cloud-dataflow-java:examplesJavaRunnerV2IntegrationTest *********************************************************** *********************************************************** **********************Set Up Pubsub************************ The Pub/Sub topic has been set up for this example: projects/apache-beam-testing/topics/testpipeline-jenkins-0805120748-49c8e94b The Pub/Sub subscription has been set up for this example: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0805120748-49c8e94b ******************Set Up Big Query Table******************* The BigQuery table has been set up for this example: apache-beam-testing:traffic_routes_1628165263170.traffic_routes_table *************************Tear Down************************* The Pub/Sub topic has been deleted: projects/apache-beam-testing/topics/testpipeline-jenkins-0805120748-49c8e94b The Pub/Sub subscription has been deleted: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0805120748-49c8e94b The BigQuery table might contain the example's output, and it is not deleted automatically: apache-beam-testing:traffic_routes_1628165263170.traffic_routes_table Please go to the Developers Console to delete it manually. Otherwise, you may be charged for its usage. *********************************************************** *********************************************************** *********************************************************** *********************************************************** **********************Set Up Pubsub************************ The Pub/Sub topic has been set up for this example: projects/apache-beam-testing/topics/testpipeline-jenkins-0805123635-180792f4 The Pub/Sub subscription has been set up for this example: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0805123635-180792f4 ******************Set Up Big Query Table******************* The BigQuery table has been set up for this example: apache-beam-testing:traffic_max_lane_flow_1628166995094.traffic_max_lane_flow_table *************************Tear Down************************* The Pub/Sub topic has been deleted: projects/apache-beam-testing/topics/testpipeline-jenkins-0805123635-180792f4 The Pub/Sub subscription has been deleted: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0805123635-180792f4 The BigQuery table might contain the example's output, and it is not deleted automatically: apache-beam-testing:traffic_max_lane_flow_1628166995094.traffic_max_lane_flow_table Please go to the Developers Console to delete it manually. Otherwise, you may be charged for its usage. *********************************************************** *********************************************************** > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest org.apache.beam.sdk.io.gcp.pubsub.PubsubReadIT > testReadPubsubMessageId FAILED java.lang.AssertionError at PubsubReadIT.java:88 org.apache.beam.sdk.io.gcp.pubsub.PubsubReadIT > testReadPublicData FAILED java.lang.AssertionError at PubsubReadIT.java:60 37 tests completed, 2 failed, 1 skipped > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest > FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805120100 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805120100] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210805120100] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8311b8bb063388392d2a0c2f42af27506ca903dac136fcf6aaad6d15a602c401]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest'. > There were failing tests. See the report at: > file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/googleCloudPlatformRunnerV2IntegrationTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2h 17m 24s 135 actionable tasks: 93 executed, 40 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/q3q6yddbu6tk4 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
