See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1341/display/redirect?page=changes>
Changes: [Ismaël Mejía] [BEAM-11712] Make up-to-date build file and codestyle [Ismaël Mejía] [BEAM-11712] Add options for input/output paths, make it run via [Ismaël Mejía] [BEAM-11712] Fix static analysis warnings and typos on TPC-DS module [aromanenko.dev] Fix W201 Trailing whitespace [aromanenko.dev] Remove redundant cast [Kenneth Knowles] Allow manual trigger of wheel build [noreply] [BEAM-12069] Move test_utils.py mock import inside function (#14511) [Boyuan Zhang] Eliminate beam_fn_api from KafkaIO expansion [noreply] [BEAM-11945] Add debug capture to SDK harness (#14197) [Kenneth Knowles] [BEAM-12011] Eliminate WindowFn.getOutputTime method [emilyye] lint ------------------------------------------ [...truncated 4.58 MB...] Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.170Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.193Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.216Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.240Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.625Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:30.680Z: Starting 1 workers in us-central1-f... Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:31.101Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create Apr 14, 2021 11:34:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:31.231Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write Apr 14, 2021 11:34:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:34:56.616Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Apr 14, 2021 11:35:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:16.112Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s). Apr 14, 2021 11:35:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:41.240Z: Workers have started successfully. Apr 14, 2021 11:35:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:41.259Z: Workers have started successfully. Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:58.896Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:58.975Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:59.032Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:59.099Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:59.210Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Apr 14, 2021 11:35:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:35:59.336Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Apr 14, 2021 11:36:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:36:01.810Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:36:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:36:09.055Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:36:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:36:15.362Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:36:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:36:26.927Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:36:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:36:52.028Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:37:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:37:07.722Z: java.lang.NoSuchMethodError: org.apache.beam.sdk.transforms.windowing.WindowFn.getOutputTime(Lorg/joda/time/Instant;Lorg/apache/beam/sdk/transforms/windowing/BoundedWindow;)Lorg/joda/time/Instant; at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:141) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowAndCombineFn.processElement(BatchGroupAlsoByWindowAndCombineFn.java:57) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Apr 14, 2021 11:37:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:37:08.234Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Apr 14, 2021 11:37:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-04-14T11:37:08.305Z: Workflow failed. Causes: S05:ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers: userscore-jenkins-0414113-04140434-5dkv-harness-3vd5 Root cause: Work item failed., userscore-jenkins-0414113-04140434-5dkv-harness-3vd5 Root cause: Work item failed., userscore-jenkins-0414113-04140434-5dkv-harness-3vd5 Root cause: Work item failed., userscore-jenkins-0414113-04140434-5dkv-harness-3vd5 Root cause: Work item failed. Apr 14, 2021 11:37:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:37:08.354Z: Cleaning up. Apr 14, 2021 11:37:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:37:08.448Z: Stopping worker pool... Apr 14, 2021 11:38:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:38:06.368Z: Autoscaling: Resized worker pool from 1 to 0. Apr 14, 2021 11:38:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-04-14T11:38:06.395Z: Worker pool stopped. Apr 14, 2021 11:38:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-04-14_04_34_19-11909457089476779231 failed with status FAILED. gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* [ERROR] Failed command > Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED FAILURE: Build completed with 3 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 3: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 37m 54s 9 actionable tasks: 8 executed, 1 from cache Publishing build scan... https://gradle.com/s/vbil63mnxclwq Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
