See
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/1763/display/redirect>
------------------------------------------
[...truncated 343.52 KB...]
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.219Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.272Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key into Write content to
files/WriteFiles/FinalizeTempFileBundles/Finalize
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.319Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.375Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.417Z: Fusing consumer Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.474Z: Unzipping flatten s11 for input
s10.org.apache.beam.sdk.values.PCollection.<init>:402#4663620f501c9270
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.532Z: Fusing unzipped copy of Write content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow),
through flatten Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into
producer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.567Z: Fusing consumer Write content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.630Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
into Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.679Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into
Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.722Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.764Z: Fusing consumer Collect byte count into
Collect write start time
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.803Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into
Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.844Z: Fusing consumer Collect write start time
into Produce text lines
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.887Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.922Z: Fusing consumer Write content to
files/WriteFiles/RewindowIntoGlobal/Window.Assign into Collect element count
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:58.968Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write
content to files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.015Z: Fusing consumer Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into
Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.063Z: Fusing consumer Collect element count into
Collect byte count
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.116Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.165Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 18, 2019 6:28:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.213Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.823Z: Executing operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.874Z: Executing operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.920Z: Executing operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.954Z: Starting 1 workers in us-central1-a...
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.954Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:28:59.987Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.030Z: Finished operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.054Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.063Z: Finished operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.063Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.063Z: Finished operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.093Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.102Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.162Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.169Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.214Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.249Z: Executing operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.310Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.383Z: Finished operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.924Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 18, 2019 6:29:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:29:00.995Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start
time+Collect byte count+Collect element count+Write content to
files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Jun 18, 2019 6:38:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:38:22.662Z: Workers have started successfully.
Jun 18, 2019 6:39:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:39:56.208Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Jun 18, 2019 6:40:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:39:57.615Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 18, 2019 6:40:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:40:26.985Z: Workers have started successfully.
Jun 18, 2019 6:40:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:40:57.551Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start
time+Collect byte count+Collect element count+Write content to
files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Jun 18, 2019 6:40:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:40:57.703Z: Executing operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Jun 18, 2019 6:40:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:40:57.758Z: Finished operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Jun 18, 2019 6:40:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:40:57.964Z: Executing operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write content
to files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write content
to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:00.821Z: Finished operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write content
to files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write content
to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:01.076Z: Executing operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:01.168Z: Finished operation Write content to
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:01.438Z: Executing operation Write content to
files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:01.522Z: Finished operation Write content to
files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Jun 18, 2019 6:41:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:01.810Z: Executing operation Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write
content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write
content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write
content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:05.565Z: Finished operation Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write
content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write
content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write
content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:05.695Z: Executing operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:05.761Z: Finished operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:05.896Z: Executing operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect
write end time+Match all files/Match filepatterns+Match all
files/Reshuffle.ViaRandomKey/Pair with random key+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:12.729Z: Finished operation Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect
write end time+Match all files/Match filepatterns+Match all
files/Reshuffle.ViaRandomKey/Pair with random key+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:12.932Z: Executing operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:13.015Z: Finished operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:13.151Z: Executing operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read
matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split
into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random
key+Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:17.718Z: Finished operation Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map+Read
matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split
into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random
key+Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Jun 18, 2019 6:41:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:17.855Z: Executing operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:17.909Z: Finished operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Jun 18, 2019 6:41:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:18.039Z: Executing operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all
via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via
FileBasedSource/Read ranges+Collect read end time+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jun 18, 2019 6:41:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:28.531Z: Finished operation Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read
all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all
via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via
FileBasedSource/Read ranges+Collect read end time+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jun 18, 2019 6:41:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:28.648Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jun 18, 2019 6:41:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:28.693Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jun 18, 2019 6:41:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:28.816Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Jun 18, 2019 6:41:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:36.628Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Jun 18, 2019 6:41:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:36.908Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Jun 18, 2019 6:41:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:36.977Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Jun 18, 2019 6:41:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:37.220Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 18, 2019 6:41:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:39.820Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 18, 2019 6:41:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:39.914Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Jun 18, 2019 6:41:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:39.970Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Jun 18, 2019 6:41:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:39.974Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Jun 18, 2019 6:41:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:40.037Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Jun 18, 2019 6:41:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:40.127Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Jun 18, 2019 6:41:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:40.186Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 18, 2019 6:41:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:44.221Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Jun 18, 2019 6:41:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:44.432Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Jun 18, 2019 6:41:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:44.497Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Jun 18, 2019 6:41:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:44.614Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Jun 18, 2019 6:41:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:47.544Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 18, 2019 6:41:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:47.671Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Jun 18, 2019 6:41:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:47.727Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Jun 18, 2019 6:41:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:48.059Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jun 18, 2019 6:41:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:52.655Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jun 18, 2019 6:41:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:55.670Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Jun 18, 2019 6:41:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:55.893Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Jun 18, 2019 6:41:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:55.953Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Jun 18, 2019 6:41:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:41:56.214Z: Executing operation Delete test files
Jun 18, 2019 6:42:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:42:00.813Z: Finished operation Delete test files
Jun 18, 2019 6:42:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:42:01.151Z: Cleaning up.
Jun 18, 2019 6:42:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-18T18:42:01.307Z: Stopping worker pool...
STDERR: Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note:
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/src/sdks/java/io/parquet/src/test/java/org/apache/beam/sdk/io/parquet/ParquetIOTest.java>
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note:
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/tfrecord/TFRecordIOIT.java>
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2019-06-18 18:46:11,981 066b4a63 MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-18 18:46:11,984 066b4a63 MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2019-06-18 18:46:11,986 066b4a63 MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/config-beam-performancetests-compressed-textioit-hdfs-1763>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml>
--ignore-not-found
2019-06-18 18:46:44,974 066b4a63 MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 903, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-18 18:46:44,975 066b4a63 MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2019-06-18 18:46:44,976 066b4a63 MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-06-18 18:46:44,976 066b4a63 MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/runs/066b4a63/pkb.log>
2019-06-18 18:46:44,977 066b4a63 MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/ws/runs/066b4a63/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
