See
<https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/4522/display/redirect>
Changes:
------------------------------------------
[...truncated 340.62 KB...]
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.663Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.699Z: Fusing consumer Collect write start time
into Produce text lines
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.738Z: Fusing consumer Write content to
files/WriteFiles/RewindowIntoGlobal/Window.Assign into Collect write start time
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.772Z: Fusing consumer Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey into Write
content to files/WriteFiles/RewindowIntoGlobal/Window.Assign
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.806Z: Fusing consumer Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify into
Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.834Z: Fusing consumer Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write into
Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.866Z: Fusing consumer Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow
into Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.891Z: Fusing consumer Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles into
Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.921Z: Fusing consumer Write content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.946Z: Unzipping flatten s48 for input
s46.org.apache.beam.sdk.values.PCollection.<init>:408#ad6715144982c968
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.973Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:54.998Z: Created new flatten s44-c92 to unzip
producers of s45
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.033Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.066Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.098Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Feb 15, 2022 12:57:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.125Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.151Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.184Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.211Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.244Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.278Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.302Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.335Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.371Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.409Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.459Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.484Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.517Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.561Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.595Z: Unzipping flatten s48-u90 for input
s50.org.apache.beam.sdk.values.PCollection.<init>:408#fdeff999916f420d-c88
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.618Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.654Z: Unzipping flatten s44 for input
s39.org.apache.beam.sdk.values.PCollection.<init>:408#f2015be6c49d0a56
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.683Z: Fusing unzipped copy of
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map,
through flatten Calculate hashcode/Flatten.PCollections, into producer
Calculate hashcode/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.723Z: Unzipping flatten s44-c92 for input
s39.org.apache.beam.sdk.values.PCollection.<init>:408#f2015be6c49d0a56
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.746Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous), through flatten
Calculate hashcode/Flatten.PCollections, into producer Calculate
hashcode/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.780Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/ProduceDefault
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.805Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.833Z: Fusing consumer Match all files/Match
filepatterns into Collect write end time
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.867Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.904Z: Fusing consumer Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
into Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.937Z: Fusing consumer Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.964Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Finalize into Write content to
files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:55.998Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key into Write content to
files/WriteFiles/FinalizeTempFileBundles/Finalize
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.026Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.062Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.098Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.132Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.170Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.210Z: Fusing consumer Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
into Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.232Z: Fusing consumer Values/Values/Map into
Write content to
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.256Z: Fusing consumer Collect write end time into
Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.282Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Pair with random key into Match all files/Match
filepatterns
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.305Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Match
all files/Reshuffle.ViaRandomKey/Pair with random key
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.336Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.382Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.446Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.506Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.546Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.579Z: Fusing consumer Read
matches/ParDo(ToReadableFile) into Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.605Z: Fusing consumer Read files/Read all via
FileBasedSource/Split into ranges into Read matches/ParDo(ToReadableFile)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.636Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Pair with random key into Read files/Read all via
FileBasedSource/Split into ranges
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.658Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read
files/Read all via FileBasedSource/Reshuffle/Pair with random key
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.685Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.708Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.734Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read
files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.760Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.793Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.827Z: Fusing consumer Read files/Read all via
FileBasedSource/Read ranges into Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.860Z: Fusing consumer Collect read end time into
Read files/Read all via FileBasedSource/Read ranges
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.883Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Collect read end time
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.904Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.933Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:56.970Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.005Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.032Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.053Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.086Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.121Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Feb 15, 2022 12:57:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.162Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Feb 15, 2022 12:57:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.563Z: Executing operation Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Feb 15, 2022 12:57:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:57.623Z: Starting 5 ****s in us-central1-b...
Feb 15, 2022 12:57:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:58.201Z: Finished operation Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Feb 15, 2022 12:57:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:57:58.351Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start
time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write
content to
files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content
to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write
content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Feb 15, 2022 12:58:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:58:11.039Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Feb 15, 2022 12:58:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:58:43.285Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Feb 15, 2022 12:59:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:07.712Z: Workers have started successfully.
Feb 15, 2022 12:59:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:07.730Z: Workers have started successfully.
Feb 15, 2022 12:59:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:49.999Z: Finished operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start
time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write
content to
files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content
to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write
content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Feb 15, 2022 12:59:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:50.085Z: Executing operation Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Feb 15, 2022 12:59:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:50.579Z: Finished operation Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Feb 15, 2022 12:59:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-02-15T12:59:50.637Z: Executing operation Write content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write
content to
files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write
content to
files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write
content to
files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
org.apache.beam.sdk.io.text.TextIOIT > writeThenReadAll SKIPPED
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 11,5,main]) completed. Took 2 mins 45.733 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 2' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/7.3.2/userguide/java_testing.html#sec:test_execution
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 58s
147 actionable tasks: 87 executed, 58 from cache, 2 up-to-date
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=3f9c60f3-16ea-40ce-b9bc-81403a036384,
currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 2716654
log file: /home/jenkins/.gradle/daemon/7.3.2/daemon-2716654.out.log
----- Last 20 lines from daemon log file - daemon-2716654.out.log -----
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 2' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/7.3.2/userguide/java_testing.html#sec:test_execution
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 58s
147 actionable tasks: 87 executed, 58 from cache, 2 up-to-date
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
