See
<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/5888/display/redirect>
Changes:
------------------------------------------
[...truncated 303.36 KB...]
INFO: No stagingLocation provided, falling back to gcpTempLocation
Feb 03, 2022 3:33:53 PM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files
from the classpath: will stage 335 files. Enable logging at DEBUG level to see
which files will be staged.
Feb 03, 2022 3:33:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Feb 03, 2022 3:33:57 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 336 files from PipelineOptions.filesToStage to staging
location to prepare for execution.
Feb 03, 2022 3:33:57 PM
org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes
forFileToStage
INFO: Staging custom dataflow-****.jar as
beam-runners-google-cloud-dataflow-java-legacy-****-2.37.0-SNAPSHOT-pnoPFzsrg0uK36iFTKbiIklaVbWhTEnaObbCgM0jHcI.jar
Feb 03, 2022 3:33:57 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading /tmp/test5895412472035755302.zip to
gs://dataflow-staging-us-central1-844138762903/temp/staging/test-v60g9zLaiU8qOh9KKIc_s6POVp4H_TJmpOwDzYoAzvc.jar
Feb 03, 2022 3:33:58 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 335 files cached, 1 files newly uploaded in 1
seconds
Feb 03, 2022 3:33:58 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to
gs://dataflow-staging-us-central1-844138762903/temp/staging/
Feb 03, 2022 3:33:58 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <219413 bytes, hash
0e8107020628b8459109f3fdf55752d67cf2ed3d3360d2daa2f285595018500b> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-DoEHAgYouEWRCfP99VdS1nzy7T0zYNLaovKFWVAYUAs.pb
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Produce text lines as step s2
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Transform strings to bytes/Map as step s3
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Record time before writing as step s4
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign
as step s5
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
as step s11
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
as step s13
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
as step s14
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
as step s15
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize
as step s16
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
as step s17
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s18
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s19
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s20
Feb 03, 2022 3:34:01 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as
step s21
Feb 03, 2022 3:34:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.37.0-SNAPSHOT
Feb 03, 2022 3:34:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-02-03_07_34_01-4031026346895647625?project=apache-beam-testing
Feb 03, 2022 3:34:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-02-03_07_34_01-4031026346895647625
Feb 03, 2022 3:34:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2022-02-03_07_34_01-4031026346895647625
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:20.479Z: Worker configuration: e2-standard-2 in
us-central1-c.
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.125Z: Expanding CoGroupByKey operations into
optimizable parts.
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.222Z: Expanding GroupByKey operations into
optimizable parts.
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.251Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.382Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.420Z: Unzipping flatten s10 for input
s6.writtenRecords
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.455Z: Fusing unzipped copy of Write content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow),
through flatten Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into
producer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.487Z: Fusing consumer Write content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.521Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.555Z: Fusing consumer Transform strings to
bytes/Map into Produce text lines
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.584Z: Fusing consumer Record time before writing
into Transform strings to bytes/Map
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.609Z: Fusing consumer Write content to
files/Write/RewindowIntoGlobal/Window.Assign into Record time before writing
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.635Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write
content to files/Write/RewindowIntoGlobal/Window.Assign
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.662Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write
content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Feb 03, 2022 3:34:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.701Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write
content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.733Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into
Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.765Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write content
to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.815Z: Fusing consumer Write content to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.847Z: Fusing consumer Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
into Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.883Z: Fusing consumer Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.915Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Finalize into Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.946Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
into Write content to files/Write/FinalizeTempFileBundles/Finalize
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:21.981Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.013Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.047Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.080Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.115Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.151Z: Fusing consumer Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
into Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.595Z: Executing operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:22.680Z: Starting 5 ****s in us-central1-c...
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:23.151Z: Finished operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Feb 03, 2022 3:34:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:23.308Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to
bytes/Map+Record time before writing+Write content to
files/Write/RewindowIntoGlobal/Window.Assign+Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content
to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Feb 03, 2022 3:34:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:34:30.087Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 03, 2022 3:35:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:35:13.594Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Feb 03, 2022 3:35:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:35:38.543Z: Workers have started successfully.
Feb 03, 2022 3:35:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:35:38.580Z: Workers have started successfully.
Feb 03, 2022 3:36:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:24.084Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to
bytes/Map+Record time before writing+Write content to
files/Write/RewindowIntoGlobal/Window.Assign+Write content to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content
to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Feb 03, 2022 3:36:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:24.174Z: Executing operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Feb 03, 2022 3:36:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:24.241Z: Finished operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Feb 03, 2022 3:36:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:24.310Z: Executing operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write content
to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Feb 03, 2022 3:36:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.317Z: Finished operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write content
to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
content to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Feb 03, 2022 3:36:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.448Z: Executing operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
Feb 03, 2022 3:36:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.523Z: Finished operation Write content to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
Feb 03, 2022 3:36:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.680Z: Executing operation Write content to
files/Write/GatherTempFileResults/View.AsList/CreateDataflowView
Feb 03, 2022 3:36:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.753Z: Finished operation Write content to
files/Write/GatherTempFileResults/View.AsList/CreateDataflowView
Feb 03, 2022 3:36:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:26.896Z: Executing operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Feb 03, 2022 3:36:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:27.074Z: Finished operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Feb 03, 2022 3:36:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:27.207Z: Executing operation Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write
content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write
content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write
content to files/Write/FinalizeTempFileBundles/Finalize+Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random
key+Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Feb 03, 2022 3:36:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:29.357Z: Finished operation Write content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write
content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write
content to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write
content to files/Write/FinalizeTempFileBundles/Finalize+Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random
key+Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Feb 03, 2022 3:36:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:29.424Z: Executing operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Feb 03, 2022 3:36:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:29.477Z: Finished operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Feb 03, 2022 3:36:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:29.532Z: Executing operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Feb 03, 2022 3:36:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:30.791Z: Finished operation Write content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
content to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Feb 03, 2022 3:36:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:30.945Z: Cleaning up.
Feb 03, 2022 3:36:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-03T15:36:31.026Z: Stopping **** pool...
Feb 03, 2022 3:37:23 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob lambda$waitUntilFinish$0
WARNING: Job is already running in Google Cloud Platform, Ctrl-C will not
cancel it.
To cancel the job in the cloud, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2022-02-03_07_34_01-4031026346895647625
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll SKIPPED
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 6,5,main]) completed. Took 3 mins 38.262 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 4' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/7.3.2/userguide/java_testing.html#sec:test_execution
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 3m 57s
142 actionable tasks: 84 executed, 56 from cache, 2 up-to-date
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=c8ca7fbf-87d5-442e-9cb8-083924982f13,
currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 4078578
log file: /home/jenkins/.gradle/daemon/7.3.2/daemon-4078578.out.log
----- Last 20 lines from daemon log file - daemon-4078578.out.log -----
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 4' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/7.3.2/userguide/java_testing.html#sec:test_execution
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 3m 57s
142 actionable tasks: 84 executed, 56 from cache, 2 up-to-date
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
