See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/2341/display/redirect>
Changes: ------------------------------------------ [...truncated 4.38 MB...] INFO: Adding WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s10 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum as step s11 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s12 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key as step s13 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign as step s14 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey as step s15 Dec 29, 2023 6:48:56 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/ExpandIterable as step s16 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Values/Values/Map as step s17 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) as step s18 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/CreateDataflowView as step s19 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/Impulse as step s20 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/MapElements/Map as step s21 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Reify.ReifyView/ParDo(Anonymous) as step s22 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Values/Values/Map as step s23 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/GatherTempFileResults/IterableToList/Map as step s24 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize as step s25 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s26 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s27 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s28 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s29 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s30 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.54.0-SNAPSHOT Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-12-28_22_48_57-10353297680421370277?project=apache-beam-testing Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2023-12-28_22_48_57-10353297680421370277 Dec 29, 2023 6:48:57 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2023-12-28_22_48_57-10353297680421370277 Dec 29, 2023 6:49:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:48:58.489Z: Autoscaling is enabled for job 2023-12-28_22_48_57-10353297680421370277. The number of workers will be between 1 and 4000. Dec 29, 2023 6:49:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:48:58.729Z: Autoscaling was automatically enabled for job 2023-12-28_22_48_57-10353297680421370277. Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:09.490Z: Worker configuration: e2-standard-2 in us-central1-b. Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:10.836Z: Expanding CoGroupByKey operations into optimizable parts. Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:10.972Z: Expanding GroupByKey operations into optimizable parts. Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:10.995Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.143Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.168Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:397#5f2ef1f005ae0b4 Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.189Z: Fusing unzipped copy of WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key, through flatten WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.213Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.235Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Write into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.254Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/GroupByWindow into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Read Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.277Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/ExpandIterable into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/GroupByWindow Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.295Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Values/Values/Map into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/ExpandIterable Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.320Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Values/Values/Map Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.343Z: Unzipping flatten s12-u49 for input s13.org.apache.beam.sdk.values.PCollection.<init>:397#f6e89c42901d787a-c47 Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.367Z: Fusing unzipped copy of WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign, through flatten WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1, into producer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.390Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.412Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign into WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.436Z: Fusing consumer WordCount.CountWords/ParDo(ExtractWords) into ReadLines/Read Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.456Z: Fusing consumer WordCount.CountWords/Count.PerElement/Init/Map into WordCount.CountWords/ParDo(ExtractWords) Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.480Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial into WordCount.CountWords/Count.PerElement/Init/Map Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.505Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.529Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify Dec 29, 2023 6:49:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.553Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Read Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.575Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.595Z: Fusing consumer MapElements/Map into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.617Z: Fusing consumer WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign into MapElements/Map Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.645Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.669Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.692Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.713Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.741Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.767Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.790Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/MapElements/Map into WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/Impulse Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.807Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Reify.ReifyView/ParDo(Anonymous) into WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/MapElements/Map Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.829Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Values/Values/Map into WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Reify.ReifyView/ParDo(Anonymous) Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.850Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/IterableToList/Map into WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Values/Values/Map Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.873Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize into WriteCounts/WriteFiles/GatherTempFileResults/IterableToList/Map Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.897Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.920Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.943Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.968Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:11.993Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:12.016Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:12.045Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:12.353Z: Executing operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Create Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:12.410Z: Starting 1 workers in us-central1-b... Dec 29, 2023 6:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:12.950Z: Finished operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Create Dec 29, 2023 6:49:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:21.571Z: Executing operation ReadLines/Read+WordCount.CountWords/ParDo(ExtractWords)+WordCount.CountWords/Count.PerElement/Init/Map+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write Dec 29, 2023 6:49:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:34.892Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Dec 29, 2023 6:49:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:49:55.970Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s). Dec 29, 2023 6:50:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:29.993Z: Workers have started successfully. Dec 29, 2023 6:50:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:44.813Z: All workers have finished the startup processes and began to receive work requests. Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.622Z: Finished operation ReadLines/Read+WordCount.CountWords/ParDo(ExtractWords)+WordCount.CountWords/Count.PerElement/Init/Map+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.673Z: Executing operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Close Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.715Z: Finished operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Close Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.759Z: Executing operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.882Z: Finished operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:54.988Z: Executing operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Create Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:55.209Z: Finished operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Create Dec 29, 2023 6:50:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:55.311Z: Executing operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Read+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract+MapElements/Map+WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Write+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.224Z: Finished operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Read+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract+MapElements/Map+WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Write+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.279Z: Executing operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.320Z: Finished operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.373Z: Executing operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Write Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.762Z: Finished operation WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Pair with random key+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Write Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.895Z: Executing operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Close Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:58.942Z: Finished operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Close Dec 29, 2023 6:50:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:50:59.053Z: Executing operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Read+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/GroupByWindow+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/ExpandIterable+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Values/Values/Map+WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.434Z: Finished operation WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/Read+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/GroupByKey/GroupByWindow+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Reshuffle/ExpandIterable+WriteCounts/WriteFiles/GatherTempFileResults/Consolidate/Values/Values/Map+WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.543Z: Executing operation WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/CreateDataflowView Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.602Z: Finished operation WriteCounts/WriteFiles/GatherTempFileResults/View.AsIterable/CreateDataflowView Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.697Z: Executing operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.811Z: Finished operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Dec 29, 2023 6:51:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:00.902Z: Executing operation WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/Impulse+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/MapElements/Map+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Reify.ReifyView/ParDo(Anonymous)+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Values/Values/Map+WriteCounts/WriteFiles/GatherTempFileResults/IterableToList/Map+WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:01.619Z: Finished operation WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/Impulse+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Create.Values/MapElements/Map+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Reify.ReifyView/ParDo(Anonymous)+WriteCounts/WriteFiles/GatherTempFileResults/AsPossiblyEmptyList/Values/Values/Map+WriteCounts/WriteFiles/GatherTempFileResults/IterableToList/Map+WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:01.675Z: Executing operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:01.853Z: Finished operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:01.898Z: Executing operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:02.058Z: Finished operation WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:02.153Z: Cleaning up. Dec 29, 2023 6:51:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-12-29T06:51:02.219Z: Stopping worker pool... FATAL: command execution failed java.io.IOException: Backing channel 'apache-beam-jenkins-15' is disconnected. at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285) at com.sun.proxy.$Proxy187.isAlive(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207) at hudson.Launcher$ProcStarter.join(Launcher.java:524) at hudson.plugins.gradle.Gradle.perform(Gradle.java:321) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:164) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526) at hudson.model.Run.execute(Run.java:1900) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44) at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:442) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75) Caused by: java.io.EOFException at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2911) at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3406) at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:932) at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:375) at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) at hudson.remoting.Command.readFrom(Command.java:142) at hudson.remoting.Command.readFrom(Command.java:128) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61) Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure ERROR: apache-beam-jenkins-15 is offline; cannot locate jdk_1.8_latest --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
