See
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/3788/display/redirect?page=changes>
Changes:
[chamikaramj] Include coders embedded in CombinePayload when converting x-lang
[kevinsijo] [BEAM-9919] Refining xlang types and wrappers
[Luke Cwik] [BEAM-9979] Support a reset call that happens after the bundle is
done
[noreply] [BEAM-10200] Add optional experiment to enable heap dump through the …
[kevinsijo] [BEAM-9919] xlang transforms wrapped with namespaces
[kevinsijo] [BEAM-9919] cleaning namespace.go
------------------------------------------
[...truncated 248.38 KB...]
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar':
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for
':',5,main]) completed. Took 0.218 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for
':',5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava'
is 1153b98d65572209673d34d303d23816
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date
because:
No history is available.
Custom actions are attached to task
':runners:google-cloud-dataflow-java:compileTestJava'.
Origin for
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@13372fc1:
{executionTime=14984, hostName=apache-ci-beam-jenkins-7,
operatingSystem=Linux, buildInvocationId=hjx6hlnztvcfzldiyiscj5asoi,
creationTime=1597789696015,
identity=:runners:google-cloud-dataflow-java:compileTestJava,
type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution,
userName=jenkins, gradleVersion=5.2.1,
rootPath=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_ParquetIOIT/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava'
from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for
':',5,main]) completed. Took 0.173 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for
':',5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no
actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for
':',5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for
':',5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is
e3df4099d8fa7f3650c10ac69be3c42b
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar':
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for
':',5,main]) completed. Took 0.028 secs.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar
Build cache key for task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is
0ea0d11bd8c0ae485ec6605fbd6d9654
Caching disabled for task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar': Caching has
not been enabled for the task
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not
up-to-date because:
No history is available.
Custom actions are attached to task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.32s [3320ms]
Average Time/Jar: 0.2075s [207.5ms]
*******************
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar
(Thread[Execution **** for ':' Thread 10,5,main]) completed. Took 4.214 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 10,5,main]) started.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is
af8da7566d945b884b9d310157a7c46f
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date
because:
Task.upToDateWhen is false.
Custom actions are attached to task
':sdks:java:io:file-based-io-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests>
Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java
-DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/3788/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar","--region=us-central1";]>
-Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager
-Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US
-Duser.language=en -Duser.variant -ea -cp
/home/jenkins/.gradle/caches/5.2.1/****Main/gradle-****.jar
****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in
[jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.25/110cefe2df103412849d72ef7a67e4e91e4266b4/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Aug 20, 2020 12:22:13 AM
org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory
tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket:
dataflow-staging-us-central1-844138762903
Aug 20, 2020 12:22:13 AM
org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 409, performed 0 retries due to
IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP
framework says request can be retried, (caller responsible for retrying):
https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Aug 20, 2020 12:22:13 AM
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 20, 2020 12:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files
from the classpath: will stage 276 files. Enable logging at DEBUG level to see
which files will be staged.
Aug 20, 2020 12:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 277 files from PipelineOptions.filesToStage to staging
location to prepare for execution.
Aug 20, 2020 12:22:18 AM
org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes
forFileToStage
INFO: Staging custom dataflow-****.jar as
beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT-iD_MKPGpDX1tJAL_aHoX13otTNl49obwh0UTI0wQ7H8.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-common-2.25.0-SNAPSHOT-tests-pcqgN3R4qvPtAPjGhQ37rQvKW7xuLpgXSSw3RYfuEyg.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-direct-java-2.25.0-SNAPSHOT-EoncJL4Y8Lf9QAkWhq1wTVsETIbieu67QkBv_udKJQw.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-tests-_NJd7xjflEp0Cnlw8FSgGeQksbfqF9poIZCdHsbUW9k.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-B7OU8DVpC4GtCqROAIMKQUaPS5x7-89mINe0w0FTrdU.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-fn-execution-2.25.0-SNAPSHOT-OzXCGv22PZN37CE1UpGFoIWKaSZw817qSmv6LIEr1AI.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/libs/beam-sdks-java-io-xml-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-tEkzsoM4P1uqKv5E1oCudshjIvwLbdkZ0koVSmqpH9M.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/libs/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-tests-FwAMuKs_rL4pueG9MyvI14yMlpe4V6lR7tjUS9km36Q.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-tests-GDhqBR-DaspkmTHEZVCnllnwS-2zzlTTitYFORQBIzg.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.25.0-SNAPSHOT-tests-VZkjZqe_GwW9kbQT_DIXlJJad8lPEWD7M8vWpkxH0y8.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/parquet/build/libs/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-tests-aEJXzrZrNY2TnhEO4OUa6Fkuoz-7vnGTUpc3Df-CnLU.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/hadoop-common/build/libs/beam-sdks-java-io-hadoop-common-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-hadoop-common-2.25.0-SNAPSHOT-yOWngANU_87xfIIE_WmdWAhABzwDViI6TM3LrUMoEdg.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-expansion-service-2.25.0-SNAPSHOT-204IVC6x4gv_xaTJfLy1ZZIq5r83SZ7FcMYeo4ZQHMo.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-tests-L1AYUXsq3_pO1Q8UlnoKuutz6t0bzYZNjDLDainGm2I.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-core-construction-java-2.25.0-SNAPSHOT-pW6H9_-gBlKVZsMD6mW9uB0Ox8xXe9WUwIlqQfxp3pc.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-java-fn-execution-2.25.0-SNAPSHOT-kymaVo9w94D5RolWrrcFQw641kOMis8F7yRnea6M8Co.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-common-2.25.0-SNAPSHOT-92mZOQXlvLYl9JDQfnrf2oTiJZT22kbWZxwbaxmvTvA.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.25.0-SNAPSHOT-rmCY3floJ5MpYzHfHAYNPlsWIeWVNWHaKTyZhqHyB7k.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT-0ez84ac9OQg7KRog_9llNF6VPiBKypa5419fOPMoOss.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/parquet/build/libs/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-iyW6IInV2veHmLLt7uess4K7xKxLQYbNQxGa8DJGNT4.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-emTH-kSWzkVE6_WoNWRub_y3gRRF_--dzfbJcnyIwkc.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-Zpc78QIxD-Ds4Lc-p8P9o_Oc9mEx17MTR-nG-IJoaJc.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-tests-7AsrwYGg52utfXgoro9fkpFBdVs4O4NVYhtIxz_nl28.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-j8HQr5hrP84Wcy_y--peN4m98a0Z9nTnF-yhf-DugUs.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading /tmp/test6737569721271523592.zip to
gs://dataflow-staging-us-central1-844138762903/temp/staging/test-BBYvsU3iLhdgfwMHEPQTUOc9-eGb385qrx8ucTNx-F4.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-vendor-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT-1oivZP8Rm02is5ZBKtHxqZq7TgArodELpV2kxJrEttc.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT-iD_MKPGpDX1tJAL_aHoX13otTNl49obwh0UTI0wQ7H8.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-fn-execution-2.25.0-SNAPSHOT-S2ONAmIhUZp-dbxi40Pern16_nd70DZKOmlObz3LX4M.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-pipeline-2.25.0-SNAPSHOT-bBxF_YqROxJian3mFYj1IvlcyEsAuUv97bcBwC_VNrI.jar
Aug 20, 2020 12:22:18 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/job-management/build/libs/beam-model-job-management-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-job-management-2.25.0-SNAPSHOT-cwXGVJvXIEnk5eTJ8_4_6cpewm8xB8RIgEynED9HQUU.jar
Aug 20, 2020 12:22:19 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 248 files cached, 29 files newly uploaded in
1 seconds
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign
as step s5
Aug 20, 2020 12:22:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
as step s11
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
as step s13
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
as step s14
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
as step s15
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize
as step s16
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
as step s17
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s18
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s19
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s20
Aug 20, 2020 12:22:20 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as
step s21
Aug 20, 2020 12:22:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://dataflow-staging-us-central1-844138762903/temp/staging/
Aug 20, 2020 12:22:20 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <165948 bytes, hash
b2132fd60bf7bbe29b7e8e2d3d830521ddc3dd45ab1dca577055e9c53adb7739> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-shMv1gv3u-Kbfo4tPYMFId3D3UWrHcpXcFXpxTrbdzk.pb
Aug 20, 2020 12:22:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.25.0-SNAPSHOT
Aug 20, 2020 12:22:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-08-19_17_22_21-2936144141795722259?project=apache-beam-testing
Aug 20, 2020 12:22:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-08-19_17_22_21-2936144141795722259
Aug 20, 2020 12:22:30 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2020-08-19_17_22_21-2936144141795722259
Aug 20, 2020 12:22:53 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2020-08-20T00:22:50.932Z: Workflow failed. Causes: The Compute
Engine API has not fully initialized. Please try again in a few minutes.,
Unable to get machine type information for machine type n1-standard-1 in zone
us-central1-a because of insufficient permissions. Please refer to
https://cloud.google.com/dataflow/access-control#creating_jobs and make sure
you have sufficient permissions.
Aug 20, 2020 12:22:53 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-20T00:22:51.075Z: Cleaning up.
Aug 20, 2020 12:22:53 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-20T00:22:51.291Z: Worker pool stopped.
Aug 20, 2020 12:22:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-08-20T00:22:54.167Z: Your project already contains 100
Dataflow-created metric descriptors and Stackdriver will not create new
Dataflow custom metrics for this job. Each unique user-defined metric name
(independent of the DoFn in which it is defined) produces a new metric
descriptor. To delete old / unused metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 20, 2020 12:22:58 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-08-19_17_22_21-2936144141795722259 failed with status FAILED.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching
gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/3788/_1597882931809*
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:188)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:109)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:171)
at
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.016 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 10,5,main]) completed. Took 51.419 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 39s
92 actionable tasks: 59 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/mzwk3rgijxhnk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
