See
<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT/7359/display/redirect>
Changes:
------------------------------------------
[...truncated 421.26 KB...]
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s60
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
as step s61
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey
as step s62
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
as step s63
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
as step s64
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
as step s65
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly
as step s66
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
as step s67
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView as step s68
Apr 05, 2023 3:35:08 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Delete test files as step s69
Apr 05, 2023 3:35:08 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.47.0-SNAPSHOT
Apr 05, 2023 3:35:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-04_20_35_08-786983365829462218?project=apache-beam-testing
Apr 05, 2023 3:35:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-04_20_35_08-786983365829462218
Apr 05, 2023 3:35:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2023-04-04_20_35_08-786983365829462218
Apr 05, 2023 3:35:28 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:27.731Z: Worker configuration: e2-standard-2 in
us-central1-b.
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:28.987Z: Expanding CoGroupByKey operations into
optimizable parts.
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.298Z: Expanding GroupByKey operations into
optimizable parts.
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.321Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.617Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.644Z: Elided trivial flatten
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.681Z: Unzipping flatten s10 for input
s6.writtenRecords
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.714Z: Fusing unzipped copy of Write Avro records
to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow),
through flatten Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into
producer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.747Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.771Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.797Z: Fusing consumer Produce Avro records into
Produce text lines
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.835Z: Fusing consumer Collect start time into
Produce Avro records
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.872Z: Fusing consumer Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign into Collect start time
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.895Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write
Avro records to files/Write/RewindowIntoGlobal/Window.Assign
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.918Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.955Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:29.995Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into
Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.022Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Avro
records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.055Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.092Z: Unzipping flatten s52 for input
s50.org.apache.beam.sdk.values.PCollection.<init>:405#80783fbab681bb5c
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.127Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.159Z: Created new flatten s48-c98 to unzip
producers of s49
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.191Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.224Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.248Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.283Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.315Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.346Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.381Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.403Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.436Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.458Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.490Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.523Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.556Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.591Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.612Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.635Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.669Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.701Z: Unzipping flatten s52-u96 for input
s54.org.apache.beam.sdk.values.PCollection.<init>:405#77397181cd44e5f-c94
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.736Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.759Z: Unzipping flatten s48 for input
s43.org.apache.beam.sdk.values.PCollection.<init>:405#fe542dba50f8fd8a
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.783Z: Fusing unzipped copy of
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map,
through flatten Calculate hashcode/Flatten.PCollections, into producer
Calculate hashcode/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.808Z: Unzipping flatten s48-c98 for input
s43.org.apache.beam.sdk.values.PCollection.<init>:405#fe542dba50f8fd8a
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.842Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous), through flatten
Calculate hashcode/Flatten.PCollections, into producer Calculate
hashcode/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.866Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/ProduceDefault
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.899Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.931Z: Fusing consumer Match all files/Match
filepatterns into Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.964Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:30.998Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.027Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.055Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Finalize into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.075Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
into Write Avro records to files/Write/FinalizeTempFileBundles/Finalize
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.104Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.133Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.155Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.188Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.223Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.255Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 05, 2023 3:35:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.285Z: Fusing consumer Collect middle time into
Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.318Z: Fusing consumer Values/Values/Map into
Collect middle time
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.341Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Pair with random key into Match all files/Match
filepatterns
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.375Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Match
all files/Reshuffle.ViaRandomKey/Pair with random key
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.398Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.420Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.447Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.484Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.507Z: Fusing consumer Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map into Match all
files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.542Z: Fusing consumer Read
matches/ParDo(ToReadableFile) into Match all
files/Reshuffle.ViaRandomKey/Values/Values/Map
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.564Z: Fusing consumer Read files/Read all via
FileBasedSource/Split into ranges into Read matches/ParDo(ToReadableFile)
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.586Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Pair with random key into Read files/Read all via
FileBasedSource/Split into ranges
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.619Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read
files/Read all via FileBasedSource/Reshuffle/Pair with random key
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.652Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.685Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.707Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read
files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.740Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.764Z: Fusing consumer Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.797Z: Fusing consumer Read files/Read all via
FileBasedSource/Read ranges into Read files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.821Z: Fusing consumer Collect end time into Read
files/Read all via FileBasedSource/Read ranges
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.852Z: Fusing consumer Parse Avro records to
Strings into Collect end time
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.886Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Parse Avro records to Strings
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.910Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.935Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.966Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:31.997Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.029Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.063Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.096Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.119Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.148Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.524Z: Executing operation Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.588Z: Starting 5 ****s in us-central1-b...
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.837Z: Finished operation Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Apr 05, 2023 3:35:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:32.954Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Collect start time+Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign+Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Avro
records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Apr 05, 2023 3:35:36 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-05T03:35:33.466Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-11' is disconnected.
at
hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at com.sun.proxy.$Proxy149.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)
at hudson.Launcher$ProcStarter.join(Launcher.java:524)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:522)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: java.io.IOException: Pipe closed after 0 cycles
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:126)
at
org.apache.sshd.common.channel.ChannelPipedInputStream.read(ChannelPipedInputStream.java:105)
at
hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:94)
at
hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:75)
at
hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:105)
at
hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-11 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
