See
<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/6523/display/redirect?page=changes>
Changes:
[noreply] Documented supported PyTorch versions (#22974)
[noreply] [Go SDK] Add fake impulse for inputs in Xlang Transform (#23383)
[noreply] Write permissions for issue closer/assigner
[noreply] GA Migration Adding Removal of /.m2/settings.xml (#23481)
------------------------------------------
[...truncated 410.25 KB...]
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.110Z: Elided trivial flatten
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.146Z: Unzipping flatten s10 for input
s6.writtenRecords
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.166Z: Fusing unzipped copy of Write Avro records
to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow),
through flatten Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into
producer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.188Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.221Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.246Z: Fusing consumer Produce Avro records into
Produce text lines
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.274Z: Fusing consumer Collect start time into
Produce Avro records
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.309Z: Fusing consumer Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign into Collect start time
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.345Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write
Avro records to files/Write/RewindowIntoGlobal/Window.Assign
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.369Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.398Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.426Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into
Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.460Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Avro
records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.485Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.516Z: Unzipping flatten s52 for input
s50.org.apache.beam.sdk.values.PCollection.<init>:405#80783fbab681bb5c
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.540Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.571Z: Created new flatten s48-c98 to unzip
producers of s49
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.603Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.635Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.669Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.705Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.737Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.762Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.789Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.820Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.856Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.881Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.913Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.946Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:38.985Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.017Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.040Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.074Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.107Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.132Z: Unzipping flatten s52-u96 for input
s54.org.apache.beam.sdk.values.PCollection.<init>:405#77397181cd44e5f-c94
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.152Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.173Z: Unzipping flatten s48 for input
s43.org.apache.beam.sdk.values.PCollection.<init>:405#fe542dba50f8fd8a
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.210Z: Fusing unzipped copy of
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map,
through flatten Calculate hashcode/Flatten.PCollections, into producer
Calculate hashcode/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.234Z: Unzipping flatten s48-c98 for input
s43.org.apache.beam.sdk.values.PCollection.<init>:405#fe542dba50f8fd8a
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.269Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous), through flatten
Calculate hashcode/Flatten.PCollections, into producer Calculate
hashcode/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.300Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/ProduceDefault
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.342Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.374Z: Fusing consumer Match all files/Match
filepatterns into Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.399Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.434Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.469Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.506Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Finalize into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.528Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
into Write Avro records to files/Write/FinalizeTempFileBundles/Finalize
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.556Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.590Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.616Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.648Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.684Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.720Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.751Z: Fusing consumer Collect middle time into
Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.778Z: Fusing consumer Values/Values/Map into
Collect middle time
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.807Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Pair with random key into Match all files/Match
filepatterns
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.841Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Match
all files/Reshuffle.ViaRandomKey/Pair with random key
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.873Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.907Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.939Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.964Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Oct 04, 2022 11:08:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:39.996Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.028Z: Fusing consumer Read
matches/ParDo(ToReadableFile) into Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.060Z: Fusing consumer Read files/Read all via
FileBasedSource/Split into ranges into Read matches/ParDo(ToReadableFile)
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.096Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Pair with random key into Read files/Read all via
FileBasedSource/Split into ranges
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.119Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read
files/Read all via FileBasedSource/Reshuffle/Pair with random key
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.142Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.169Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.204Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read
files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.232Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.267Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.302Z: Fusing consumer Read files/Read all via
FileBasedSource/Read ranges into Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.324Z: Fusing consumer Collect end time into Read
files/Read all via FileBasedSource/Read ranges
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.358Z: Fusing consumer Parse Avro records to
Strings into Collect end time
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.387Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Parse Avro records to Strings
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.422Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.451Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.479Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.500Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.529Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.555Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.579Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.608Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:40.632Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:41.054Z: Executing operation Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Oct 04, 2022 11:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:41.135Z: Starting 5 ****s in us-central1-a...
Oct 04, 2022 11:08:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:41.461Z: Finished operation Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Oct 04, 2022 11:08:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:08:41.604Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Collect start time+Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign+Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Avro
records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-10-04T23:09:23.644Z: Startup of the **** pool in zone
us-central1-a failed to bring up any of the desired 5 ****s. Please refer to
https://cloud.google.com/dataflow/docs/guides/common-errors#****-pool-failure
for help troubleshooting. QUOTA_EXCEEDED: Instance
'avroioit0writethenreadall-10041608-n9py-harness-wqbt' creation failed: Quota
'IN_USE_ADDRESSES' exceeded. Limit: 1200.0 in region us-central1.
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-10-04T23:09:23.677Z: Workflow failed.
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-10-04T23:09:23.754Z: Unable to delete temp directory:
"gs://dataflow-staging-us-central1-844138762903/temp/dax-tmp-2022-10-04_16_08_14-6863599666211416960-S02-0-eb81a953bf30a695".
Causes: Unable to view metadata for files:
gs://dataflow-staging-us-central1-844138762903/temp/dax-tmp-2022-10-04_16_08_14-6863599666211416960-S02-0-eb81a953bf30a695.
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-10-04T23:09:23.787Z: S02:Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Collect start time+Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign+Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Avro
records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
failed.
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:09:23.825Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Collect start time+Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign+Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Avro
records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:09:23.920Z: Cleaning up.
Oct 04, 2022 11:09:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:09:24.028Z: Stopping **** pool...
Oct 04, 2022 11:09:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-10-04T23:09:50.786Z: Worker pool stopped.
Oct 04, 2022 11:10:25 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-10-04_16_08_14-6863599666211416960 failed with status FAILED.
org.apache.beam.sdk.io.avro.AvroIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): 2ad38c3f-a997-4ec3-90aa-ae257780baa7 and
timestamp: 2022-10-04T23:10:26.018000000Z:
Metric: Value:
write_time 0.0
run_time 0.0
dataset_size 1.08973E9
read_time 0.0
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.avro.AvroIOIT > writeThenReadAll FAILED
java.lang.AssertionError: Values should be different. Actual: FAILED
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failEquals(Assert.java:187)
at org.junit.Assert.assertNotEquals(Assert.java:163)
at org.junit.Assert.assertNotEquals(Assert.java:177)
at
org.apache.beam.sdk.io.avro.AvroIOIT.writeThenReadAll(AvroIOIT.java:165)
1 test completed, 1 failed
Finished generating test XML results (0.024 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.028 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** Thread
5,5,main]) completed. Took 2 mins 33.607 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 5s
153 actionable tasks: 91 executed, 60 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/h3enc6zazorgs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
