See
<https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT/2645/display/redirect>
Changes:
------------------------------------------
[...truncated 302.80 KB...]
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.374Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.403Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.439Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.477Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.508Z: Unzipping flatten s9-u138 for input
s10.org.apache.beam.sdk.values.PCollection.<init>:400#4663620f501c9270-c136
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.543Z: Fusing unzipped copy of Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign,
through flatten Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1,
into producer Write Parquet files/WriteFiles/GatherTempFileResults/Add void
key/AddKeys/Map
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.578Z: Fusing consumer Write Parquet
files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.616Z: Fusing consumer Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign
into Write Parquet files/WriteFiles/GatherTempFileResults/Add void
key/AddKeys/Map
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.643Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.681Z: Fusing consumer Produce Avro records into
Produce text lines
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.715Z: Fusing consumer Gather write start times
into Produce Avro records
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.751Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into
Gather write start times
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.788Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into
Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.811Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into
Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.848Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
into Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.873Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.897Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.922Z: Fusing consumer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign into Calculate
hashcode/ProduceDefault
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.960Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:38.988Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.027Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.060Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.099Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.491Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.537Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.567Z: Starting 5 workers in us-central1-c...
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.585Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.608Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.624Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.634Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.643Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.670Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.692Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.722Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.734Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.766Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.780Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:39.878Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.016Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.222Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.234Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.236Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.332Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.337Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.699Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Gather write start times+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Mar 09, 2020 12:38:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:38:40.733Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 09, 2020 12:38:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-09T00:38:45.306Z: Your project already contains 100
Dataflow-created metric descriptors and Stackdriver will not create new
Dataflow custom metrics for this job. Each unique user-defined metric name
(independent of the DoFn in which it is defined) produces a new metric
descriptor. To delete old / unused metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 09, 2020 12:39:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:39:04.308Z: Autoscaling: Raised the number of workers
to 5 based on the rate of progress in the currently running step(s).
Mar 09, 2020 12:39:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:39:17.467Z: Workers have started successfully.
Mar 09, 2020 12:39:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:39:17.491Z: Workers have started successfully.
Mar 09, 2020 12:39:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:39:32.817Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 09, 2020 12:42:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:47.420Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Gather write start times+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Mar 09, 2020 12:42:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:47.526Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Mar 09, 2020 12:42:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:47.589Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Mar 09, 2020 12:42:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:47.672Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.035Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.111Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.159Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.223Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write
Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write
Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.897Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write
Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write
Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:48.970Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:49.024Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:49 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:49.100Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write
Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:51.150Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write
Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:51.235Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:51.296Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:51.377Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather
write end times+Get file names/Values/Map+Find files/Match filepatterns+Find
files/Reshuffle.ViaRandomKey/Pair with random key+Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:54.259Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather
write end times+Get file names/Values/Map+Find files/Match filepatterns+Find
files/Reshuffle.ViaRandomKey/Pair with random key+Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Mar 09, 2020 12:42:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:54.488Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:54.568Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Mar 09, 2020 12:42:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:42:54.660Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched
files/ParDo(ToReadableFile)+Gather read start time+Read parquet
files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 09, 2020 12:44:39 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:44:39.384Z: Checking permissions granted to controller
Service Account.
Mar 09, 2020 12:45:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:01.607Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched
files/ParDo(ToReadableFile)+Gather read start time+Read parquet
files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 09, 2020 12:45:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:01.695Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 09, 2020 12:45:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:01.768Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 09, 2020 12:45:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:01.839Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:04.782Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:04.945Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.002Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.159Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.778Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.857Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Mar 09, 2020 12:45:05 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.896Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.917Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:05.952Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.021Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.054Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.452Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.497Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.547Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.597Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.601Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.651Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.669Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:06.729Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:07.056Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 09, 2020 12:45:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:08.851Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Mar 09, 2020 12:45:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:09.000Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Mar 09, 2020 12:45:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:09.053Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Mar 09, 2020 12:45:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:09.197Z: Executing operation Delete test files
Mar 09, 2020 12:45:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:11.134Z: Finished operation Delete test files
Mar 09, 2020 12:45:12 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:11.326Z: Cleaning up.
Mar 09, 2020 12:45:12 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:45:11.390Z: Stopping worker pool...
Mar 09, 2020 12:46:37 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:46:36.029Z: Autoscaling: Resized worker pool from 5 to
0.
Mar 09, 2020 12:46:37 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-09T00:46:36.072Z: Worker pool stopped.
Mar 09, 2020 12:46:42 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-08_17_38_19-15811624444776108820 finished with status
DONE.
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): 39b95fa5-a4ac-4c94-9930-8a0ebe5c8419 and
timestamp: 2020-03-09T00:46:42.236000000Z:
Metric: Value:
run_time 327.991
read_time 126.574
dataset_size 1.08737E9
write_time 199.784
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution worker for
':' Thread 10,5,main]) completed. Took 8 mins 33.293 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> java.io.FileNotFoundException:
> <https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest/binary/output.bin.idx>
> (No such file or directory)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 18s
88 actionable tasks: 56 executed, 32 from cache
Publishing build scan...
https://gradle.com/s/s2y4yq2kt7btq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
