See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java17/186/display/redirect?page=changes>
Changes: [Robert Bradshaw] Streaming-related runner fixes. [Robert Bradshaw] Improvements to auto-started services. [Robert Bradshaw] Fix version, asserts for remote execution. [Robert Bradshaw] Add IO dependencies. [Robert Bradshaw] Add several cross-language IOs. [Robert Bradshaw] Disable tests that require new release is required for out-of-the-box [rszper] Correcting the regex for the Dataflow job name. [noreply] Update parameterized requirement in /sdks/python (#21975) [noreply] Merge pull request #21981 from [Playground] Upgrade Flutter linter, fix [noreply] Clean up redundant articles, prepositions, conjunctions appeared [noreply] Fix FlatMap numpy array bug (#22006) [Robert Bradshaw] More strongly typed outputs. ------------------------------------------ [...truncated 115.80 KB...] a037458de4e0: Preparing bafdbe68e4ae: Preparing a13c519c6361: Preparing 2068100e5968: Waiting 4adff6b3b60c: Waiting 4313f1f5eac3: Waiting d67f0c70361d: Waiting 93b9bc47c530: Waiting db3153ed7a91: Waiting 89ae289f499d: Waiting 08fa02ce37eb: Waiting a037458de4e0: Waiting bafdbe68e4ae: Waiting 4a5653ffc16b: Waiting e93827457889: Waiting 3bc383470c05: Waiting a13c519c6361: Waiting fa4e72052357: Waiting 937609003005: Pushed 0131c36c99b4: Pushed 9e22381f8502: Pushed 1374e0922d88: Pushed a7836ced867d: Pushed 4adff6b3b60c: Pushed d67f0c70361d: Pushed 4313f1f5eac3: Pushed db3153ed7a91: Pushed 2068100e5968: Pushed 93b9bc47c530: Pushed 3bc383470c05: Layer already exists e93827457889: Layer already exists 08fa02ce37eb: Layer already exists a037458de4e0: Layer already exists bafdbe68e4ae: Layer already exists 89ae289f499d: Pushed a13c519c6361: Layer already exists 792d689f491f: Pushed fa4e72052357: Pushed 4a5653ffc16b: Pushed 20220623145438: digest: sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6 size: 4729 > Task :sdks:java:testing:load-tests:run Jun 23, 2022 2:55:11 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Jun 23, 2022 2:55:12 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 222 files. Enable logging at DEBUG level to see which files will be staged. Jun 23, 2022 2:55:12 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Window.Into() Jun 23, 2022 2:55:12 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Jun 23, 2022 2:55:15 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 222 files from PipelineOptions.filesToStage to staging location to prepare for execution. Jun 23, 2022 2:55:16 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 222 files cached, 0 files newly uploaded in 1 seconds Jun 23, 2022 2:55:16 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Jun 23, 2022 2:55:16 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <114917 bytes, hash 44d0fdfcae2d6cf75ae698eaeb01cb6fb44b06c957c8a4f8cf224ccbf5fa9f26> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-RND9_K4tbPda5pjq6wHLb7RLBslXyKT4zyJMy_X6nyY.pb Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input as step s1 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s2 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s3 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input as step s4 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s5 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s6 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s9 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s10 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s12 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s13 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s14 Jun 23, 2022 2:55:18 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.41.0-SNAPSHOT Jun 23, 2022 2:55:19 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-06-23_07_55_18-9958187582357096444?project=apache-beam-testing Jun 23, 2022 2:55:19 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2022-06-23_07_55_18-9958187582357096444 Jun 23, 2022 2:55:19 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2022-06-23_07_55_18-9958187582357096444 Jun 23, 2022 2:55:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:55:24.952Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java170dataflow0v20batch0cogbk02-jenkins-062314-k5af. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Jun 23, 2022 2:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:38.964Z: Worker configuration: e2-standard-2 in us-central1-b. Jun 23, 2022 2:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:39.938Z: Expanding SplittableParDo operations into optimizable parts. Jun 23, 2022 2:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:39.967Z: Expanding CollectionToSingleton operations into optimizable parts. Jun 23, 2022 2:55:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.042Z: Expanding CoGroupByKey operations into optimizable parts. Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.111Z: Expanding GroupByKey operations into optimizable parts. Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.182Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.208Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.237Z: Fusing unzipped copy of CoGroupByKey/GBK/Write, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.268Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/Read Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.299Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.334Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.367Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.402Z: Fusing consumer CoGroupByKey/GBK/Write into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.447Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.479Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.502Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.535Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.569Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.601Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.660Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.691Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.714Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.737Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.769Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:40.825Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:41.137Z: Executing operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:41.169Z: Executing operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Jun 23, 2022 2:55:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:41.214Z: Starting 5 ****s in us-central1-b... Jun 23, 2022 2:55:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:55:49.358Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Jun 23, 2022 2:56:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:56:18.684Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s). Jun 23, 2022 2:57:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:57:57.482Z: A **** was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image. Jun 23, 2022 2:58:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:11.191Z: A **** was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image. Jun 23, 2022 2:58:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:11.791Z: A **** was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image. Jun 23, 2022 2:58:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:14.813Z: Autoscaling: Unable to reach resize target for the **** pool in zone us-central1-b. Jun 23, 2022 2:58:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:16.167Z: A **** was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:16.823Z: A **** was unable to start up. Error: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-06-23T14:58:16.871Z: Workflow failed. Causes: Job appears to be stuck. Several ****s have failed to start up in a row, and no **** has successfully started up for this job. Last error reported: Unable to pull container image due to error: image pull request failed with error: Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers). This is likely due to an invalid SDK container image URL. Please verify any provided SDK container image is valid and that Dataflow ****s have permissions to pull image.. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:16.990Z: Unable to delete temp files: "gs://temp-storage-for-perf-tests/loadtests/dax-tmp-2022-06-23_07_55_18-9958187582357096444-S02-0-f8c0c95b010ef76c/[email protected]." Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:17.001Z: Unable to delete temp files: "gs://temp-storage-for-perf-tests/loadtests/dax-tmp-2022-06-23_07_55_18-9958187582357096444-S01-0-1bce29914e70fee3/[email protected]." Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:17.017Z: S02:Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing failed. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-06-23T14:58:17.027Z: S01:Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing failed. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:58:17.052Z: Finished operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:58:17.065Z: Finished operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:58:17.140Z: Cleaning up. Jun 23, 2022 2:58:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T14:58:17.216Z: Stopping **** pool... Jun 23, 2022 3:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T15:01:09.658Z: Autoscaling: Resized **** pool from 1 to 0. Jun 23, 2022 3:01:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-06-23T15:01:09.726Z: Worker pool stopped. Jun 23, 2022 3:01:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2022-06-23_07_55_18-9958187582357096444 failed with status FAILED. Jun 23, 2022 3:01:47 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk Load test results for test (ID): 8c0ae906-ca13-4f9b-b465-c859a3945069 and timestamp: 2022-06-23T14:55:12.493000000Z: Metric: Value: dataflow_v2_java17_runtime_sec 0.0 dataflow_v2_java17_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220623145438 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220623145438] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220623145438] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:45b577167930aa8be88cf81226d474a026489c1b7a3f7bbe45f1f92e8753f6b6]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 7m 18s 105 actionable tasks: 8 executed, 97 up-to-date Publishing build scan... https://gradle.com/s/afmj5gdlo35ba Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
