See
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/3787/display/redirect?page=changes>
Changes:
[zyichi] Allow Nexmark launcher to publish human-readable events to pubsub.
[irvi.fa] [BEAM-10753] Add Slack link invitation on README
[Luke Cwik] [BEAM-10756] Fix empty pull response to not ack and to not throw
[noreply] Merge pull request #12597: [BEAM-10685] Added integration test for
[ettarapp] clarifying unclear comments
[noreply] Update README.md
[noreply] [BEAM-3301] Adding SDF Go Dataflow translation. (#12629)
[noreply] [BEAM-10752] Use TestPubsubSignal in PubsubToBigqueryIT (#12625)
------------------------------------------
[...truncated 249.26 KB...]
Build cache key for task ':sdks:java:io:google-cloud-platform:testJar' is
204adbb42be3e6241530b240bf36a1ca
Caching disabled for task ':sdks:java:io:google-cloud-platform:testJar':
Caching has not been enabled for the task
Task ':sdks:java:io:google-cloud-platform:testJar' is not up-to-date because:
No history is available.
:sdks:java:io:google-cloud-platform:testJar (Thread[Execution **** for ':'
Thread 8,5,main]) completed. Took 0.245 secs.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for
':' Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:compileTestJava FROM-CACHE
Build cache key for task ':runners:google-cloud-dataflow-java:compileTestJava'
is 1153b98d65572209673d34d303d23816
Task ':runners:google-cloud-dataflow-java:compileTestJava' is not up-to-date
because:
No history is available.
Custom actions are attached to task
':runners:google-cloud-dataflow-java:compileTestJava'.
Origin for
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution@51e310d3:
{executionTime=14984, hostName=apache-ci-beam-jenkins-7,
operatingSystem=Linux, buildInvocationId=hjx6hlnztvcfzldiyiscj5asoi,
creationTime=1597789696015,
identity=:runners:google-cloud-dataflow-java:compileTestJava,
type=org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.TaskExecution,
userName=jenkins, gradleVersion=5.2.1,
rootPath=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_ParquetIOIT/src}
Unpacked trees for task ':runners:google-cloud-dataflow-java:compileTestJava'
from cache.
:runners:google-cloud-dataflow-java:compileTestJava (Thread[Execution **** for
':' Thread 8,5,main]) completed. Took 0.163 secs.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':'
Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testClasses UP-TO-DATE
Skipping task ':runners:google-cloud-dataflow-java:testClasses' as it has no
actions.
:runners:google-cloud-dataflow-java:testClasses (Thread[Execution **** for ':'
Thread 8,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':'
Thread 8,5,main]) started.
> Task :runners:google-cloud-dataflow-java:testJar
Build cache key for task ':runners:google-cloud-dataflow-java:testJar' is
e3df4099d8fa7f3650c10ac69be3c42b
Caching disabled for task ':runners:google-cloud-dataflow-java:testJar':
Caching has not been enabled for the task
Task ':runners:google-cloud-dataflow-java:testJar' is not up-to-date because:
No history is available.
:runners:google-cloud-dataflow-java:testJar (Thread[Execution **** for ':'
Thread 8,5,main]) completed. Took 0.028 secs.
> Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar
Build cache key for task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is
b63068ba1275fd373e53fc16118e0de5
Caching disabled for task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar': Caching has
not been enabled for the task
Task ':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar' is not
up-to-date because:
No history is available.
Custom actions are attached to task
':runners:google-cloud-dataflow-java:****:legacy-****:shadowJar'.
*******************
GRADLE SHADOW STATS
Total Jars: 16 (includes project)
Total Time: 3.498s [3498ms]
Average Time/Jar: 0.218625s [218.625ms]
*******************
:runners:google-cloud-dataflow-java:****:legacy-****:shadowJar
(Thread[Execution **** for ':' Thread 6,5,main]) completed. Took 4.425 secs.
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 6,5,main]) started.
Gradle Test Executor 1 started executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is
60ca5eba3cfa27ea8dd5451159bce98e
Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date
because:
Task.upToDateWhen is false.
Custom actions are attached to task
':sdks:java:io:file-based-io-tests:integrationTest'.
Starting process 'Gradle Test Executor 1'. Working directory:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests>
Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java
-DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/3787/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar","--region=us-central1";]>
-Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager
-Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US
-Duser.language=en -Duser.variant -ea -cp
/home/jenkins/.gradle/caches/5.2.1/****Main/gradle-****.jar
****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]>
SLF4J: Found binding in
[jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.25/bccda40ebc8067491b32a88f49615a747d20082d/slf4j-jdk14-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.25/110cefe2df103412849d72ef7a67e4e91e4266b4/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
Aug 19, 2020 6:22:13 PM
org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory
tryCreateDefaultBucket
INFO: No tempLocation specified, attempting to use default bucket:
dataflow-staging-us-central1-844138762903
Aug 19, 2020 6:22:13 PM
org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 409, performed 0 retries due to
IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP
framework says request can be retried, (caller responsible for retrying):
https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing.
Aug 19, 2020 6:22:13 PM
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Aug 19, 2020 6:22:14 PM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files
from the classpath: will stage 276 files. Enable logging at DEBUG level to see
which files will be staged.
Aug 19, 2020 6:22:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 277 files from PipelineOptions.filesToStage to staging
location to prepare for execution.
Aug 19, 2020 6:22:16 PM
org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes
forFileToStage
INFO: Staging custom dataflow-****.jar as
beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT--yyh0U1ncxA1Svm3Ae_OBFI_CkZ5LoiwpbHDedndR3M.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-tests-RzWswvGOZnlNTrvBLgrIshgLAp1gyhiaeSUXye-KK8Q.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-core-construction-java-2.25.0-SNAPSHOT-bMF57HnFj_g8nMmsHngKmv6fANfaBlpBdu6qADqm3oE.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-leonglRP31-xWx82NkH7lg9v_w6n1UzPxIR3y9WQy6M.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-tests-lkrwwrljyUneXp-Qrb-pPZfrbv2I5gcE450iDinUjP4.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/libs/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-tests-MyhAlunDjAusLGVqvBiNsIu2mKC60D_cC7pZGsolyrU.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/parquet/build/libs/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-tests-SMfLmo5DHIFTcbtfYxyaUUq24TNZHHw0EcF7ty124zg.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading /tmp/test5867569820536657303.zip to
gs://dataflow-staging-us-central1-844138762903/temp/staging/test-PMgU8zFOgxSyPfH6saeChAGinDT4zXQeOWOK5_uZlYs.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/xml/build/libs/beam-sdks-java-io-xml-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-xml-2.25.0-SNAPSHOT-2tD-5XUZnw3__rSasId1AAYHc4MMXrjo9G9DdfiCq_4.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-YiCa6ykt7hb-FWH_XD6CfT8qj6t4ep-OVSjU9ppAomI.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-java-fn-execution-2.25.0-SNAPSHOT-fBZKrGqAOhzThK_6BbuGrkDpTsG0KqzwbH82Y5Aub7E.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/hadoop-common/build/libs/beam-sdks-java-io-hadoop-common-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-hadoop-common-2.25.0-SNAPSHOT-2BxPHto0dp3qq2q0tbPa2ulVsIvWr6Wf1lE3zxU-2zo.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-expansion-service-2.25.0-SNAPSHOT-kuU7l8KNmsSR6-oflaTCvO6ksgbm-jfpgX1fVRzXquA.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-google-cloud-platform-2.25.0-SNAPSHOT-tests-AEMqn_J7ezY1V6gljlgXQOz6Q_J5c2az51lO4ZYqByM.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/parquet/build/libs/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-parquet-2.25.0-SNAPSHOT-SAS1KADBjaE5Q7X1OPafFgmo6QPT2B8J5AyhnXlCFfw.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.25.0-SNAPSHOT-tests-DmZg7V_Eo3BB0iSkpiDL_AromAfZADHWQPomIAh_YYA.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-common-2.25.0-SNAPSHOT-tests-9O1hb_de7DZgPuey9QJQ_9tQtmvR8fkI7erVSjx3FI8.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-test-utils-2.25.0-SNAPSHOT-FYmPnEthKX9mq6ndO3X0OHOaudHLV17hrjSUjGdlO00.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-2.25.0-SNAPSHOT-o22z9vOpGCC_QPV1BnS5TgpbtV-S1srGg2vkv-sDnUg.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.25.0-SNAPSHOT-tests-XHZVrKcBADIB6zn3ZftvU2tkxVf-BkCeb7x-vdY8Vp4.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.25.0-SNAPSHOT-wJfASTQ7wNcsWIZ1bDIXGdb5FcdPnzPFueNwc2stbhQ.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT-kri6e9oGkcXf2TKtVl33DTS5C9-02a1WiJQRHl14cbk.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-common-2.25.0-SNAPSHOT-QhBM2kV-JkNZ4w2zd9Z0a8IKD38NWr6Z5PQDVM2Y-CE.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-direct-java-2.25.0-SNAPSHOT-ta9tMIiTkUeFA7U6vFiQIwf-uhdqpdeDPzzWxBo2dTg.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-vendor-sdks-java-extensions-protobuf-2.25.0-SNAPSHOT-gocY3ha_Co5m-fQxce2bP6E3F0P0wdtmc6E5t22OQYc.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-fn-execution-2.25.0-SNAPSHOT-f0eA1ZgoVmT9ng4k40O_4qsMbDawnFc6KAGMxTMbu9c.jar
Aug 19, 2020 6:22:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/job-management/build/libs/beam-model-job-management-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-job-management-2.25.0-SNAPSHOT-qYhCc2F013W4xygf4Pz66DYIrQA-wfV1VXWIu6DseQg.jar
Aug 19, 2020 6:22:17 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-legacy-****-2.25.0-SNAPSHOT--yyh0U1ncxA1Svm3Ae_OBFI_CkZ5LoiwpbHDedndR3M.jar
Aug 19, 2020 6:22:17 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-fn-execution-2.25.0-SNAPSHOT-wBQLz8311cMK0B0u3AXCyekZ7As37Y-bUM5JHKjj-mI.jar
Aug 19, 2020 6:22:17 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.25.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-pipeline-2.25.0-SNAPSHOT-6mARq3hHoL5ISEyxqFA88h__TryYlzPp-8tWM17jtMw.jar
Aug 19, 2020 6:22:18 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 248 files cached, 29 files newly uploaded in
2 seconds
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign
as step s5
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
as step s11
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
as step s13
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
as step s14
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
as step s15
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize
as step s16
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
as step s17
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s18
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s19
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s20
Aug 19, 2020 6:22:18 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as
step s21
Aug 19, 2020 6:22:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://dataflow-staging-us-central1-844138762903/temp/staging/
Aug 19, 2020 6:22:18 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <165948 bytes, hash
c323d6e075eb507d7eb6201b26f2003c77c73042d6db67db8215893d0b717cc0> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-wyPW4HXrUH1-tiAbJvIAPHfHMELW22fbghWJPQtxfMA.pb
Aug 19, 2020 6:22:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.25.0-SNAPSHOT
Aug 19, 2020 6:22:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-08-19_11_22_19-14795096840512485931?project=apache-beam-testing
Aug 19, 2020 6:22:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-08-19_11_22_19-14795096840512485931
Aug 19, 2020 6:22:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2020-08-19_11_22_19-14795096840512485931
Aug 19, 2020 6:22:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-19T18:22:36.207Z: Worker configuration: n1-standard-1 in
us-central1-c.
Aug 19, 2020 6:22:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2020-08-19T18:22:36.578Z: Workflow failed. Causes: The Compute
Engine API has not fully initialized. Please try again in a few minutes.,
Unable to get information for region us-central1.
Aug 19, 2020 6:22:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-19T18:22:36.617Z: Cleaning up.
Aug 19, 2020 6:22:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-19T18:22:36.672Z: Worker pool stopped.
Aug 19, 2020 6:22:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-08-19T18:22:39.736Z: Your project already contains 100
Dataflow-created metric descriptors and Stackdriver will not create new
Dataflow custom metrics for this job. Each unique user-defined metric name
(independent of the DoFn in which it is defined) produces a new metric
descriptor. To delete old / unused metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Aug 19, 2020 6:22:43 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-08-19_11_22_19-14795096840512485931 failed with status
FAILED.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll FAILED
java.lang.IllegalStateException: Unable to find any files matching
gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/3787/_1597861331982*
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:188)
at org.apache.beam.sdk.io.TFRecordIO$Read.expand(TFRecordIO.java:109)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:171)
at
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT.writeThenReadAll(TFRecordIOIT.java:151)
1 test completed, 1 failed
Finished generating test XML results (0.016 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.024 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 6,5,main]) completed. Took 35.268 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 24s
92 actionable tasks: 59 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/kf4es5ep2wmce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
