See <https://ci-beam.apache.org/job/beam_PostCommit_Java/7316/display/redirect?page=changes>
Changes: [Ismaël Mejía] [BEAM-9282] Move structured streaming runner into Spark 2 specific [Ismaël Mejía] [BEAM-9282] Separate modules for Spark 2/3 [Ismaël Mejía] [BEAM-9282] Separate modules for Spark 2/3 job-server [Ismaël Mejía] [BEAM-9282] Separate modules for Spark 2/3 job-server container [Ismaël Mejía] [BEAM-7092] Run PostCommit tests for Spark 3 module too [Ismaël Mejía] [BEAM-7092] Update tests invocation for Spark 2 module [Ismaël Mejía] [BEAM-9283] Add Spark 3 test jobs to the CI (Java 11) [Ismaël Mejía] [BEAM-11654] Publish Spark 2 and 3 specific Job-Server containers [Ismaël Mejía] [BEAM-7092] Add paranamer 2.8 license to container (Spark 3 / Avro) ------------------------------------------ [...truncated 118.47 KB...] Resolving google.golang.org/grpc: commit='5e8f83304c0563d1ba74db05fee83d9c18ab9a58', urls=[https://github.com/grpc/grpc-go] Resolving google.golang.org/protobuf: commit='d165be301fb1e13390ad453281ded24385fd8ebc', urls=[https://go.googlesource.com/protobuf] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] Resolving cached github.com/etcd-io/etcd: commit='11214aa33bf5a47d3d9d8dafe0f6b97237dfe921', urls=[https://github.com/etcd-io/etcd.git, [email protected]:etcd-io/etcd.git] > Task :sdks:go:installDependencies > Task :sdks:go:buildLinuxAmd64 > Task :sdks:go:goBuild > Task :sdks:java:container:resolveBuildDependencies Unable to watch the file system for changes. The inotify watches limit is too low. Resolving ./github.com/apache/beam/sdks/go@<https://ci-beam.apache.org/job/beam_PostCommit_Java/ws/src/sdks/go> > Task :sdks:java:container:installDependencies > Task :sdks:java:io:kafka:kafkaVersion222BatchIT > Task :sdks:java:container:buildLinuxAmd64 > Task :sdks:java:container:goBuild > Task :sdks:java:io:hadoop-format:runHadoopFormatIO321ElasticIT > Task :runners:spark:2:hadoopVersion292Test > Task :sdks:java:container:java8:copySdkHarnessLauncher > Task :sdks:java:container:java8:dockerPrepare > Task :sdks:java:container:java8:docker Removing intermediate container bd711b097276 ---> eb2f25dfae3a Step 4/17 : ARG pull_licenses ---> Running in a54a2f23e3e1 Removing intermediate container a54a2f23e3e1 ---> 667d3534a432 Step 5/17 : ADD target/slf4j-api.jar /opt/apache/beam/jars/ ---> 39b1b6226c23 Step 6/17 : ADD target/slf4j-jdk14.jar /opt/apache/beam/jars/ ---> 09ee94486fdb Step 7/17 : ADD target/beam-sdks-java-harness.jar /opt/apache/beam/jars/ ---> df29ba124d4a Step 8/17 : ADD target/beam-sdks-java-io-kafka.jar /opt/apache/beam/jars/ ---> 6ccb5b65ad21 Step 9/17 : ADD target/kafka-clients.jar /opt/apache/beam/jars/ ---> e4e10496fe4a Step 10/17 : ADD target/linux_amd64/boot /opt/apache/beam/ > Task :sdks:java:io:google-cloud-platform:postCommit > Task :sdks:java:container:java8:docker ---> 918d5c2d1e6d Step 11/17 : COPY target/LICENSE /opt/apache/beam/ ---> cc6dd0852043 Step 12/17 : COPY target/NOTICE /opt/apache/beam/ ---> a1aaf1694d5b Step 13/17 : ADD target/third_party_licenses /opt/apache/beam/third_party_licenses/ ---> 8c99bdedc2fb Step 14/17 : COPY target/LICENSE target/go-licenses/* /opt/apache/beam/third_party_licenses/golang/ ---> 505863d98004 Step 15/17 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE ---> Running in 14758df0f61f Removing intermediate container 14758df0f61f ---> 9f86330067a4 Step 16/17 : RUN if [ "${pull_licenses}" = "false" ] ; then rm -rf /opt/apache/beam/third_party_licenses ; fi ---> Running in 21233f6c6482 Removing intermediate container 21233f6c6482 ---> 6c776b87305a Step 17/17 : ENTRYPOINT ["/opt/apache/beam/boot"] ---> Running in acd15d8d3ac8 Removing intermediate container acd15d8d3ac8 ---> aa9c9f3ccf61 Successfully built aa9c9f3ccf61 Successfully tagged apache/beam_java8_sdk:2.30.0.dev > Task :sdks:java:io:kafka:kafkaVersion222Test > Task :sdks:java:io:hadoop-format:runHadoopFormatIO321IT > Task :sdks:java:io:kafka:kafkaVersion231BatchIT > Task :sdks:java:io:kafka:kafkaVersion231Test > Task :sdks:java:io:hadoop-format:runHadoopFormatIO321Test > Task :runners:spark:2:hadoopVersion321Test > Task :sdks:java:io:kafka:kafkaVersion241BatchIT > Task :sdks:java:io:hadoop-format:hadoopVersionsTest > Task :sdks:java:io:kafka:kafkaVersion241Test > Task :sdks:java:io:kafka:kafkaVersion251BatchIT > Task :runners:spark:2:hadoopVersionsTest > Task :javaHadoopVersionsTest > Task :sdks:java:io:kafka:kafkaVersion251Test > Task :sdks:java:io:kafka:kafkaVersionsCompatibilityTest > Task :sdks:java:extensions:zetasketch:postCommit > Task > :runners:google-cloud-dataflow-java:examplesJavaLegacyWorkerIntegrationTest *********************************************************** *********************************************************** **********************Set Up Pubsub************************ The Pub/Sub topic has been set up for this example: projects/apache-beam-testing/topics/testpipeline-jenkins-0313182417-e87cb173 The Pub/Sub subscription has been set up for this example: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313182417-e87cb173 ******************Set Up Big Query Table******************* The BigQuery table has been set up for this example: apache-beam-testing:traffic_routes_1615659856964.traffic_routes_table *************************Tear Down************************* The Pub/Sub topic has been deleted: projects/apache-beam-testing/topics/testpipeline-jenkins-0313182417-e87cb173 The Pub/Sub subscription has been deleted: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313182417-e87cb173 The BigQuery table might contain the example's output, and it is not deleted automatically: apache-beam-testing:traffic_routes_1615659856964.traffic_routes_table Please go to the Developers Console to delete it manually. Otherwise, you may be charged for its usage. *********************************************************** *********************************************************** > Task > :runners:google-cloud-dataflow-java:coreSDKJavaLegacyWorkerIntegrationTest > NO-SOURCE > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformLegacyWorkerIntegrationTest > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformLegacyWorkerKmsIntegrationTest > Task :runners:google-cloud-dataflow-java:postCommit > Task :runners:google-cloud-dataflow-java:buildAndPushDockerContainer WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03. As an alternative, use `gcloud auth configure-docker` to configure `docker` to use `gcloud` as a credential helper, then use `docker` as you would for non-GCR registries, e.g. `docker pull gcr.io/project-id/my-image`. Add `--verbosity=error` to silence this warning: `gcloud docker --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/java] 2ce3c2e49870: Preparing 92ab45c83a9a: Preparing 1920d8e3a536: Preparing 180498c04bdc: Preparing 154926262bcd: Preparing 40a5c992a44a: Preparing d7eaa9d4db9b: Preparing 2c8ab1da850c: Preparing 0fd9377d1d69: Preparing 43be85361208: Preparing c7c2a51a01c5: Preparing 8a970d4156d3: Preparing 09ac04d65797: Preparing 59f1e8e1ce66: Preparing bde301416dd2: Preparing 81496d8c72c2: Preparing 40a5c992a44a: Waiting d7eaa9d4db9b: Waiting 2c8ab1da850c: Waiting 8a970d4156d3: Waiting c7c2a51a01c5: Waiting 644448d6e877: Preparing 0e41e5bdb921: Preparing 0fd9377d1d69: Waiting 81496d8c72c2: Waiting bde301416dd2: Waiting 43be85361208: Waiting 59f1e8e1ce66: Waiting 644448d6e877: Waiting 0e41e5bdb921: Waiting 92ab45c83a9a: Pushed 154926262bcd: Pushed 180498c04bdc: Pushed 2ce3c2e49870: Pushed 1920d8e3a536: Pushed 2c8ab1da850c: Pushed d7eaa9d4db9b: Pushed 8a970d4156d3: Layer already exists 09ac04d65797: Layer already exists 59f1e8e1ce66: Layer already exists bde301416dd2: Layer already exists 81496d8c72c2: Layer already exists 644448d6e877: Layer already exists 0e41e5bdb921: Layer already exists c7c2a51a01c5: Pushed 43be85361208: Pushed 40a5c992a44a: Pushed 0fd9377d1d69: Pushed 20210313180101: digest: sha256:2299c81664770299bb633771cb1d2ab5c269db0ef17c75ad5c11615eccc51981 size: 4098 > Task :runners:google-cloud-dataflow-java:coreSDKJavaRunnerV2IntegrationTest > NO-SOURCE > Task :runners:google-cloud-dataflow-java:examplesJavaRunnerV2IntegrationTest *********************************************************** *********************************************************** **********************Set Up Pubsub************************ The Pub/Sub topic has been set up for this example: projects/apache-beam-testing/topics/testpipeline-jenkins-0313194018-df6685e0 The Pub/Sub subscription has been set up for this example: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313194018-df6685e0 ******************Set Up Big Query Table******************* The BigQuery table has been set up for this example: apache-beam-testing:traffic_routes_1615664414024.traffic_routes_table *************************Tear Down************************* The Pub/Sub topic has been deleted: projects/apache-beam-testing/topics/testpipeline-jenkins-0313194018-df6685e0 The Pub/Sub subscription has been deleted: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313194018-df6685e0 The BigQuery table might contain the example's output, and it is not deleted automatically: apache-beam-testing:traffic_routes_1615664414024.traffic_routes_table Please go to the Developers Console to delete it manually. Otherwise, you may be charged for its usage. *********************************************************** *********************************************************** *********************************************************** *********************************************************** **********************Set Up Pubsub************************ The Pub/Sub topic has been set up for this example: projects/apache-beam-testing/topics/testpipeline-jenkins-0313200648-376dc552 The Pub/Sub subscription has been set up for this example: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313200648-376dc552 ******************Set Up Big Query Table******************* The BigQuery table has been set up for this example: apache-beam-testing:traffic_max_lane_flow_1615666007806.traffic_max_lane_flow_table *************************Tear Down************************* The Pub/Sub topic has been deleted: projects/apache-beam-testing/topics/testpipeline-jenkins-0313200648-376dc552 The Pub/Sub subscription has been deleted: projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0313200648-376dc552 The BigQuery table might contain the example's output, and it is not deleted automatically: apache-beam-testing:traffic_max_lane_flow_1615666007806.traffic_max_lane_flow_table Please go to the Developers Console to delete it manually. Otherwise, you may be charged for its usage. *********************************************************** *********************************************************** > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest org.apache.beam.sdk.io.gcp.pubsub.PubsubReadIT > testReadPubsubMessageId FAILED java.lang.AssertionError at PubsubReadIT.java:90 org.apache.beam.sdk.io.gcp.pubsub.PubsubReadIT > testReadPublicData FAILED java.lang.AssertionError at PubsubReadIT.java:62 36 tests completed, 2 failed, 1 skipped > Task > :runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest > FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210313180101 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2299c81664770299bb633771cb1d2ab5c269db0ef17c75ad5c11615eccc51981 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2299c81664770299bb633771cb1d2ab5c269db0ef17c75ad5c11615eccc51981 Associated tags: - 20210313180101 Tags: - us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210313180101 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210313180101]. Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2299c81664770299bb633771cb1d2ab5c269db0ef17c75ad5c11615eccc51981]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:googleCloudPlatformRunnerV2IntegrationTest'. > There were failing tests. See the report at: > file://<https://ci-beam.apache.org/job/beam_PostCommit_Java/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/googleCloudPlatformRunnerV2IntegrationTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 23m 3s 251 actionable tasks: 192 executed, 57 from cache, 2 up-to-date Gradle was unable to watch the file system for changes. The inotify watches limit is too low. Publishing build scan... https://gradle.com/s/mpygv57ggt4dw Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
