See <https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/900/display/redirect>
Changes: ------------------------------------------ [...truncated 3.15 MB...] INFO: Adding ExtractUserScore/MapElements/Map as step s3 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey as step s4 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues as step s5 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/ConvertToRow as step s6 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign as step s7 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s8 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s9 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s10 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum as step s11 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s12 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s13 Feb 23, 2020 12:18:32 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView as step s14 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s15 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s16 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s17 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize as step s18 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s19 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s20 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s21 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s22 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s23 Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/ Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <72064 bytes, hash Ifncvd4Lmg_BtPyli-r6-w> to gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/pipeline-Ifncvd4Lmg_BtPyli-r6-w.pb Feb 23, 2020 12:18:33 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.20.0-SNAPSHOT Feb 23, 2020 12:18:34 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones Feb 23, 2020 12:18:35 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2020-02-23_04_18_34-4156660583558331862?project=apache-beam-testing Feb 23, 2020 12:18:35 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2020-02-23_04_18_34-4156660583558331862 Feb 23, 2020 12:18:35 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2020-02-23_04_18_34-4156660583558331862 Feb 23, 2020 12:18:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:34.370Z: Autoscaling is enabled for job 2020-02-23_04_18_34-4156660583558331862. The number of workers will be between 1 and 1000. Feb 23, 2020 12:18:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:34.370Z: Autoscaling was automatically enabled for job 2020-02-23_04_18_34-4156660583558331862. Feb 23, 2020 12:18:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:37.896Z: Checking permissions granted to controller Service Account. Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:44.629Z: Worker configuration: n1-standard-1 in us-central1-f. Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.285Z: Expanding CoGroupByKey operations into optimizable parts. Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.432Z: Expanding GroupByKey operations into optimizable parts. Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.465Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.640Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.678Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.711Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.744Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.800Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.829Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.863Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.888Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.930Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.956Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:45.998Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.034Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:400#20ff67585e33a8f6 Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.071Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.113Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.137Z: Fusing consumer ParseGameEvent into TextIO.Read/Read Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.172Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.210Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.249Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.286Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.312Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.347Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.380Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.417Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.454Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.487Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.526Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.566Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.602Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow Feb 23, 2020 12:18:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:46.636Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.090Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.121Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.158Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.158Z: Starting 1 workers in us-central1-f... Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.193Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.211Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.211Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Feb 23, 2020 12:18:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:18:47.375Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write Feb 23, 2020 12:18:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2020-02-23T12:18:57.953Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Feb 23, 2020 12:19:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:19:11.646Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Feb 23, 2020 12:19:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:19:44.602Z: Workers have started successfully. Feb 23, 2020 12:19:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:19:44.636Z: Workers have started successfully. Feb 23, 2020 12:20:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:19:59.226Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:20:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:20:06.659Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:20:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:20:17.171Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:20:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:20:27.700Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:20:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:20:39.894Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:21:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:21:10.440Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:21:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:21:32.413Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:22:57.177Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:24:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:24:16.906Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:24:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:24:47.044Z: Checking permissions granted to controller Service Account. Feb 23, 2020 12:27:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:27:14.004Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.reflect.DoFnInvoker.invokeStartBundle(Lorg/apache/beam/sdk/transforms/DoFn$StartBundleContext;)V Feb 23, 2020 12:27:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:27:14.030Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write Feb 23, 2020 12:27:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2020-02-23T12:27:14.140Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers: userscore-jenkins-0223121-02230418-jhah-harness-r20j Root cause: Work item failed., userscore-jenkins-0223121-02230418-jhah-harness-r20j Root cause: Work item failed., userscore-jenkins-0223121-02230418-jhah-harness-r20j Root cause: Work item failed., userscore-jenkins-0223121-02230418-jhah-harness-r20j Root cause: Work item failed. Feb 23, 2020 12:27:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:27:14.267Z: Cleaning up. Feb 23, 2020 12:27:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:27:14.372Z: Stopping worker pool... Feb 23, 2020 12:29:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:29:01.782Z: Autoscaling: Resized worker pool from 1 to 0. Feb 23, 2020 12:29:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-02-23T12:29:01.820Z: Worker pool stopped. Feb 23, 2020 12:29:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2020-02-23_04_18_34-4156660583558331862 failed with status FAILED. gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* [ERROR] Failed command > Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 28m 51s 8 actionable tasks: 7 executed, 1 from cache Publishing build scan... https://gradle.com/s/hh6agnjvu2s5e Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
