See
<https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/4378/display/redirect?page=changes>
Changes:
[Brian Hulette] Add support std and var in DataFrame and agg
[zhoufek] [BEAM-9487] Raise error in GroupByKey on invalid PCollections
[Brian Hulette] Test std edge cases
[noreply] [BEAM-12199] Run pack_combiners phase by default in FnApiRunner and
[noreply] Merge pull request #14531 from [BEAM-12266] Adds a connector that
------------------------------------------
[...truncated 318.74 KB...]
May 18, 2021 6:27:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.428Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 18, 2021 6:27:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.462Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 18, 2021 6:27:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.498Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 18, 2021 6:27:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.534Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.585Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.603Z: Unzipping flatten s9-u129 for input
s10.org.apache.beam.sdk.values.PCollection.<init>:402#2587af97b4865538-c127
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.629Z: Fusing unzipped copy of Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign,
through flatten Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1,
into producer Write Parquet files/WriteFiles/GatherTempFileResults/Add void
key/AddKeys/Map
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.657Z: Fusing consumer Write Parquet
files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.693Z: Fusing consumer Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign
into Write Parquet files/WriteFiles/GatherTempFileResults/Add void
key/AddKeys/Map
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.720Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.782Z: Fusing consumer Produce Avro records into
Produce text lines
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.816Z: Fusing consumer Gather write start times
into Produce Avro records
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.849Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into
Gather write start times
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.883Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into
Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.926Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into
Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.962Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
into Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.008Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.117Z: Fusing consumer Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.142Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/ProduceDefault
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.179Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.207Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.239Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.278Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.715Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.794Z: Starting 5 ****s in us-central1-f...
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.392Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.581Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.732Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:27:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.909Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Gather write start times+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:27:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:53.778Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2021 6:28:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:23.327Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
May 18, 2021 6:28:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:49.197Z: Workers have started successfully.
May 18, 2021 6:28:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:49.228Z: Workers have started successfully.
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.361Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Gather write start times+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.433Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.591Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.702Z: Executing operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.129Z: Finished operation Write Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
Parquet
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write
Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write
Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.203Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.257Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.342Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.518Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.682Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write
Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write
Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.266Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write
Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write
Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key+Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.352Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.418Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.554Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.762Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.939Z: Executing operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write
Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:02.952Z: Finished operation Write Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write
Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:03.046Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.025Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.113Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.281Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.428Z: Executing operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather
write end times+Get file names/Values/Map+Find files/Match filepatterns+Find
files/Reshuffle.ViaRandomKey/Pair with random key+Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:06.942Z: Finished operation Write Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
Parquet
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather
write end times+Get file names/Values/Map+Find files/Match filepatterns+Find
files/Reshuffle.ViaRandomKey/Pair with random key+Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.053Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.106Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.173Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.326Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:32:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.484Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched
files/ParDo(ToReadableFile)+Gather read start time+Read parquet
files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.637Z: Finished operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched
files/ParDo(ToReadableFile)+Gather read start time+Read parquet
files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.733Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.798Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.949Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.994Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.109Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.138Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.414Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.448Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:46.759Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.803Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.951Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:33:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.985Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:33:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.147Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.865Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.944Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.976Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.019Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.036Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.089Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.125Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.258Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.396Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.932Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.005Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.086Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:33:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.177Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:33:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.006Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:33:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.580Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:33:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.745Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:33:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.791Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:33:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.962Z: Executing operation Delete test files
May 18, 2021 6:33:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.491Z: Finished operation Delete test files
May 18, 2021 6:33:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.660Z: Cleaning up.
May 18, 2021 6:33:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.737Z: Stopping **** pool...
May 18, 2021 6:34:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:34:46.146Z: Autoscaling: Resized **** pool from 5 to 0.
May 18, 2021 6:34:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:34:46.196Z: Worker pool stopped.
May 18, 2021 6:35:02 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-18_11_27_05-7998019210395361861 finished with status DONE.
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): bffb27ab-7ef2-4977-be4f-0f30fdf2f456 and
timestamp: 2021-05-18T18:35:03.235000000Z:
Metric: Value:
read_time 96.592
write_time 181.415
run_time 279.367
dataset_size 1.08737E9
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_ERROR
log4j:WARN No appenders could be found for logger
(org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Finished generating test XML results (0.07 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.083 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
Invalidating in-memory cache of
/home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:file-based-io-tests:integrationTest'
with cache key c6cdf9f1786007343d9f2cd2271eeee2
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main])
completed. Took 10 mins 5.394 secs.
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 11m 41s
120 actionable tasks: 73 executed, 47 from cache
Publishing build scan...
https://gradle.com/s/62zh5kgd5meii
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by
another Gradle instance.
Owner PID: unknown
Our PID: 12980
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
