See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/214/display/redirect?page=changes>
Changes: [david.janicek] [BEAM-10292] LocalFileSystem.matchNewResource method throws [Kamil Wasilewski] [BEAM-10674] Python CoGBK: Remove unnecessary step. Adjust parameters to [Kamil Wasilewski] [BEAM-10674] Python CoGBK: Add streaming job [Kyle Weaver] [BEAM-9118] Increase portable_runner_test timeout. [Kyle Weaver] [BEAM-10770] Remove DataflowPortabilityApiUnsupported annotation. [daniel.o.programmer] [BEAM-3301] Consolidating starcgen method names in graph/fn.go and [daniel.o.programmer] [BEAM-10289] Fixing miscalculated SDF split fraction. [david.janicek] [BEAM-10292] changes after review [noreply] [BEAM-10701] Generate Codecov XML (#12566) [noreply] [BEAM-9979] Fix read index being reported to be reset after any metrics [noreply] [BEAM-9546] DataframeTransform can now consume a schema-aware ------------------------------------------ [...truncated 59.36 KB...] > Task :sdks:java:extensions:join-library:processResources NO-SOURCE > Task :sdks:java:extensions:protobuf:processResources NO-SOURCE > Task :sdks:java:io:parquet:processResources NO-SOURCE > Task :sdks:java:extensions:sql:zetasql:processResources NO-SOURCE > Task :model:job-management:processResources > Task > :sdks:java:extensions:sql:expansion-service:createCheckerFrameworkManifest > Task :runners:flink:1.10:job-server-container:copyLicenses > Task :sdks:java:extensions:sql:expansion-service:processResources NO-SOURCE > Task :runners:flink:1.10:job-server-container:dockerClean UP-TO-DATE > Task :model:fn-execution:processResources > Task :sdks:java:core:generateGrammarSource FROM-CACHE > Task :runners:flink:1.10:copySourceOverrides > Task :runners:flink:1.10:copyTestResourcesOverrides NO-SOURCE > Task :runners:flink:1.10:processResources > Task :sdks:java:core:processResources > Task :sdks:java:build-tools:compileJava FROM-CACHE > Task :sdks:java:build-tools:processResources > Task :sdks:java:build-tools:classes > Task :sdks:java:build-tools:jar > Task :sdks:java:extensions:sql:copyFmppTemplatesFromCalciteCore > Task :sdks:java:extensions:sql:copyFmppTemplatesFromSrc > Task :sdks:java:extensions:sql:generateFmppSources > Task :model:pipeline:extractIncludeProto > Task :model:pipeline:extractProto > Task :model:pipeline:generateProto > Task :sdks:java:extensions:sql:compileJavacc Java Compiler Compiler Version 4.0 (Parser Generator) (type "javacc" with no arguments for help) Reading from file <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/sdks/java/extensions/sql/build/generated/fmpp/javacc/Parser.jj> . . . Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding. Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking. File "TokenMgrError.java" does not exist. Will create one. File "ParseException.java" does not exist. Will create one. File "Token.java" does not exist. Will create one. File "SimpleCharStream.java" does not exist. Will create one. Parser generated with 0 errors and 1 warnings. > Task :sdks:java:extensions:sql:processResources > Task :model:pipeline:compileJava FROM-CACHE > Task :model:pipeline:processResources > Task :model:pipeline:classes > Task :model:pipeline:jar > Task :model:job-management:extractIncludeProto > Task :model:fn-execution:extractIncludeProto > Task :model:job-management:generateProto > Task :model:fn-execution:generateProto > Task :model:job-management:compileJava FROM-CACHE > Task :model:job-management:classes > Task :model:fn-execution:compileJava FROM-CACHE > Task :model:fn-execution:classes > Task :model:pipeline:shadowJar > Task :model:job-management:shadowJar > Task :model:fn-execution:shadowJar > Task :sdks:java:core:compileJava FROM-CACHE > Task :sdks:java:core:classes > Task :sdks:java:core:shadowJar > Task :sdks:java:io:mongodb:compileJava FROM-CACHE > Task :sdks:java:io:mongodb:classes UP-TO-DATE > Task :sdks:java:extensions:join-library:compileJava FROM-CACHE > Task :sdks:java:extensions:join-library:classes UP-TO-DATE > Task :runners:local-java:compileJava FROM-CACHE > Task :runners:local-java:classes UP-TO-DATE > Task :sdks:java:extensions:join-library:jar > Task :sdks:java:io:mongodb:jar > Task :sdks:java:extensions:protobuf:extractIncludeProto > Task :runners:local-java:jar > Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE > Task :runners:core-construction-java:compileJava FROM-CACHE > Task :runners:core-construction-java:classes UP-TO-DATE > Task :vendor:sdks-java-extensions-protobuf:compileJava FROM-CACHE > Task :vendor:sdks-java-extensions-protobuf:classes UP-TO-DATE > Task :sdks:java:extensions:protobuf:compileJava FROM-CACHE > Task :sdks:java:extensions:protobuf:classes UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:compileJava FROM-CACHE > Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE > Task :sdks:java:extensions:protobuf:jar > Task :sdks:java:extensions:google-cloud-platform-core:jar > Task :sdks:java:fn-execution:compileJava FROM-CACHE > Task :sdks:java:fn-execution:classes UP-TO-DATE > Task :runners:core-construction-java:jar > Task :sdks:java:io:hadoop-common:compileJava FROM-CACHE > Task :sdks:java:io:hadoop-common:classes UP-TO-DATE > Task :sdks:java:io:hadoop-common:jar > Task :sdks:java:fn-execution:jar > Task :vendor:sdks-java-extensions-protobuf:shadowJar > Task :sdks:java:io:parquet:compileJava FROM-CACHE > Task :sdks:java:io:parquet:classes UP-TO-DATE > Task :sdks:java:io:parquet:jar > Task :runners:core-java:compileJava FROM-CACHE > Task :runners:core-java:classes UP-TO-DATE > Task :runners:core-java:jar > Task :sdks:java:core:jar > Task :sdks:java:harness:compileJava FROM-CACHE > Task :sdks:java:harness:classes UP-TO-DATE > Task :sdks:java:harness:jar > Task :sdks:java:harness:shadowJar > Task :runners:java-fn-execution:compileJava FROM-CACHE > Task :runners:java-fn-execution:classes UP-TO-DATE > Task :runners:java-fn-execution:jar > Task :sdks:java:expansion-service:compileJava FROM-CACHE > Task :sdks:java:expansion-service:classes UP-TO-DATE > Task :sdks:java:expansion-service:jar > Task :runners:direct-java:compileJava FROM-CACHE > Task :runners:direct-java:classes UP-TO-DATE > Task :runners:java-job-service:compileJava FROM-CACHE > Task :runners:java-job-service:classes UP-TO-DATE > Task :sdks:java:io:kafka:compileJava FROM-CACHE > Task :sdks:java:io:kafka:classes UP-TO-DATE > Task :runners:java-job-service:jar > Task :sdks:java:io:kafka:jar > Task :runners:flink:1.10:compileJava FROM-CACHE > Task :runners:flink:1.10:classes > Task :sdks:java:io:google-cloud-platform:compileJava FROM-CACHE > Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE > Task :runners:flink:1.10:jar > Task :runners:flink:1.10:job-server:compileJava NO-SOURCE > Task :runners:flink:1.10:job-server:classes UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:jar > Task :runners:direct-java:shadowJar > Task :sdks:java:extensions:sql:compileJava FROM-CACHE > Task :sdks:java:extensions:sql:classes > Task :sdks:java:extensions:sql:jar > Task :sdks:java:extensions:sql:zetasql:compileJava FROM-CACHE > Task :sdks:java:extensions:sql:zetasql:classes UP-TO-DATE > Task :sdks:java:extensions:sql:zetasql:jar > Task :sdks:java:extensions:sql:expansion-service:compileJava FROM-CACHE > Task :sdks:java:extensions:sql:expansion-service:classes UP-TO-DATE > Task :sdks:java:extensions:sql:expansion-service:jar > Task :sdks:java:extensions:sql:expansion-service:shadowJar > Task :runners:flink:1.10:job-server:shadowJar > Task :runners:flink:1.10:job-server-container:copyDockerfileDependencies > Task :runners:flink:1.10:job-server-container:dockerPrepare > Task :runners:flink:1.10:job-server-container:docker Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 2m 48s 93 actionable tasks: 64 executed, 28 from cache, 1 up-to-date Publishing build scan... https://gradle.com/s/prfwj3a4rksw2 [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins4291475147857158280.sh + echo 'Tagging image...' Tagging image... [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins5512795796258937022.sh + docker tag gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins7779776700616105562.sh + echo 'Pushing image...' Pushing image... [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins4631316832290009286.sh + docker push gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest The push refers to repository [gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server] a9cc7d840641: Preparing a314b0a79c08: Preparing b3b7b11f88c6: Preparing b1883955ebf8: Preparing 9d6d0aae3934: Preparing fd48b7313a1f: Preparing 16e8bdbf703d: Preparing de9aadc6b492: Preparing e5df62d9b33a: Preparing 7a9460d53218: Preparing b2765ac0333a: Preparing 0ced13fcf944: Preparing fd48b7313a1f: Waiting e5df62d9b33a: Waiting 7a9460d53218: Waiting 16e8bdbf703d: Waiting b2765ac0333a: Waiting 0ced13fcf944: Waiting de9aadc6b492: Waiting a9cc7d840641: Pushed b3b7b11f88c6: Pushed a314b0a79c08: Pushed 16e8bdbf703d: Layer already exists fd48b7313a1f: Layer already exists e5df62d9b33a: Layer already exists de9aadc6b492: Layer already exists 7a9460d53218: Layer already exists b2765ac0333a: Layer already exists 0ced13fcf944: Layer already exists 9d6d0aae3934: Pushed b1883955ebf8: Pushed latest: digest: sha256:f9c98491b181148ae69c3ba0c3343980108cc4181ca88d4b86433285af911544 size: 2841 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content JOB_SERVER_IMAGE=gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest CLUSTER_NAME=beam-postcommit-python-chicago-taxi-flink-214 DETACHED_MODE=true HARNESS_IMAGES_TO_PULL=gcr.io/apache-beam-testing/beam_portability/beam_python2.7_sdk:latest FLINK_NUM_WORKERS=5 FLINK_DOWNLOAD_URL=https://archive.apache.org/dist/flink/flink-1.10.1/flink-1.10.1-bin-scala_2.11.tgz GCS_BUCKET=gs://beam-flink-cluster HADOOP_DOWNLOAD_URL=https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-hadoop-2-uber/2.8.3-9.0/flink-shaded-hadoop-2-uber-2.8.3-9.0.jar FLINK_TASKMANAGER_SLOTS=1 ARTIFACTS_DIR=gs://beam-flink-cluster/beam-postcommit-python-chicago-taxi-flink-214 GCLOUD_ZONE=us-central1-a [EnvInject] - Variables injected successfully. [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins1783297843818933000.sh + echo Setting up flink cluster Setting up flink cluster [beam_PostCommit_Python_Chicago_Taxi_Flink] $ /bin/bash -xe /tmp/jenkins1688410622813308291.sh + cd <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/.test-infra/dataproc> + ./flink_cluster.sh create + GCLOUD_ZONE=us-central1-a + DATAPROC_VERSION=1.2 + MASTER_NAME=beam-postcommit-python-chicago-taxi-flink-214-m + INIT_ACTIONS_FOLDER_NAME=init-actions + FLINK_INIT=gs://beam-flink-cluster/init-actions/flink.sh + BEAM_INIT=gs://beam-flink-cluster/init-actions/beam.sh + DOCKER_INIT=gs://beam-flink-cluster/init-actions/docker.sh + FLINK_LOCAL_PORT=8081 + FLINK_TASKMANAGER_SLOTS=1 + YARN_APPLICATION_MASTER= + create + upload_init_actions + echo 'Uploading initialization actions to GCS bucket: gs://beam-flink-cluster' Uploading initialization actions to GCS bucket: gs://beam-flink-cluster + gsutil cp -r init-actions/beam.sh init-actions/docker.sh init-actions/flink.sh gs://beam-flink-cluster/init-actions Copying file://init-actions/beam.sh [Content-Type=text/x-sh]... / [0 files][ 0.0 B/ 2.3 KiB] / [1 files][ 2.3 KiB/ 2.3 KiB] -Copying file://init-actions/docker.sh [Content-Type=text/x-sh]... - [1 files][ 2.3 KiB/ 6.0 KiB] - [2 files][ 6.0 KiB/ 6.0 KiB] Copying file://init-actions/flink.sh [Content-Type=text/x-sh]... - [2 files][ 6.0 KiB/ 13.7 KiB] - [3 files][ 13.7 KiB/ 13.7 KiB] \ Operation completed over 3 objects/13.7 KiB. + create_cluster + local metadata=flink-snapshot-url=https://archive.apache.org/dist/flink/flink-1.10.1/flink-1.10.1-bin-scala_2.11.tgz, + metadata+=flink-start-yarn-session=true, + metadata+=flink-taskmanager-slots=1, + metadata+=hadoop-jar-url=https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-hadoop-2-uber/2.8.3-9.0/flink-shaded-hadoop-2-uber-2.8.3-9.0.jar + [[ -n gcr.io/apache-beam-testing/beam_portability/beam_python2.7_sdk:latest ]] + metadata+=,beam-sdk-harness-images-to-pull=gcr.io/apache-beam-testing/beam_portability/beam_python2.7_sdk:latest + [[ -n gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest ]] + metadata+=,beam-job-server-image=gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest + local image_version=1.2 + echo 'Starting dataproc cluster. Dataproc version: 1.2' Starting dataproc cluster. Dataproc version: 1.2 + local num_dataproc_workers=6 + gcloud dataproc clusters create beam-postcommit-python-chicago-taxi-flink-214 --region=global --num-workers=6 --initialization-actions gs://beam-flink-cluster/init-actions/docker.sh,gs://beam-flink-cluster/init-actions/beam.sh,gs://beam-flink-cluster/init-actions/flink.sh --metadata flink-snapshot-url=https://archive.apache.org/dist/flink/flink-1.10.1/flink-1.10.1-bin-scala_2.11.tgz,flink-start-yarn-session=true,flink-taskmanager-slots=1,hadoop-jar-url=https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-hadoop-2-uber/2.8.3-9.0/flink-shaded-hadoop-2-uber-2.8.3-9.0.jar,beam-sdk-harness-images-to-pull=gcr.io/apache-beam-testing/beam_portability/beam_python2.7_sdk:latest,beam-job-server-image=gcr.io/apache-beam-testing/beam_portability/beam_flink1.10_job_server:latest, --image-version=1.2 --zone=us-central1-a --quiet Waiting on operation [projects/apache-beam-testing/regions/global/operations/b8061414-e906-3e47-a71f-652cdb7cad84]. Waiting for cluster creation operation... WARNING: For PD-Standard without local SSDs, we strongly recommend provisioning 1TB or larger to ensure consistently high I/O performance. See https://cloud.google.com/compute/docs/disks/performance for information on disk I/O performance. ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done. ERROR: (gcloud.dataproc.clusters.create) Operation [projects/apache-beam-testing/regions/global/operations/b8061414-e906-3e47-a71f-652cdb7cad84] failed: Initialization action timed out. Failed action 'gs://beam-flink-cluster/init-actions/flink.sh', see output in: gs://dataproc-6c5fbcbb-a2de-406e-9cf7-8c1ce0b6a604-us/google-cloud-dataproc-metainfo/ceba732f-9e47-4f4d-b1b5-93c7960d621a/beam-postcommit-python-chicago-taxi-flink-214-w-5/dataproc-initialization-script-2_output. Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
