See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/301/display/redirect?page=changes>
Changes: [Kenneth Knowles] Upgrade to Gradle 7.4 [Kenneth Knowles] Remove Python module dependency on Dataflow worker [noreply] [BEAM-13925] Dont double assign committers if author or other reviewer [noreply] [BEAM-13739] Remove deprecated shallow clone funcs (#17362) [noreply] [BEAM-11104] Pipe Continuation to DataSource level (#17334) [noreply] [BEAM-11105] Basic Watermark Estimation (Wall Clock Observing) (#17267) [noreply] Respect output coder for TextIO. (#17367) [noreply] Merge pull request #17200 from [BEAM-12164]: fix the autoscaling backlog [noreply] [BEAM-17035] Call python3 directly when it is available. (#17366) [noreply] Merge pull request #17375: [BEAM-8691] Declare newer ------------------------------------------ [...truncated 114.56 KB...] 0816d1f73744: Preparing 84f2cb0fc541: Preparing b0dc1a441986: Preparing 7a7698da17f2: Preparing d59769727d80: Preparing 348622fdcc61: Preparing 4ac8bc2cd0be: Preparing 3eeb3f11ebb5: Waiting 2410da0a8c9c: Waiting 6a92eb4f2695: Waiting e8e216b725ed: Waiting 6f3332c56a87: Waiting 029755ddf5cc: Waiting 15225c0e28bb: Waiting 7a7698da17f2: Waiting 4e2cfbb56ecf: Waiting 1d689917f7ab: Waiting cc6c8711c51b: Waiting 4ac8bc2cd0be: Waiting 0816d1f73744: Waiting 67f750f6a75a: Pushed cbea0c958b94: Pushed f6c8bedc9340: Pushed 3fd343da085d: Pushed 6db5a56000b5: Pushed 15225c0e28bb: Pushed cc6c8711c51b: Pushed 3eeb3f11ebb5: Pushed 2410da0a8c9c: Pushed 6f3332c56a87: Pushed e8e216b725ed: Pushed 0816d1f73744: Layer already exists 84f2cb0fc541: Layer already exists 6a92eb4f2695: Pushed b0dc1a441986: Layer already exists 7a7698da17f2: Layer already exists d59769727d80: Layer already exists 348622fdcc61: Layer already exists 4ac8bc2cd0be: Layer already exists 029755ddf5cc: Pushed 4e2cfbb56ecf: Pushed 1d689917f7ab: Pushed 20220415145831: digest: sha256:f5f017cae6bdcbb17b5112ff25a8befc1d3add7a507019a9b620464ac889ce59 size: 4935 > Task :sdks:java:testing:load-tests:run Apr 15, 2022 2:59:01 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Apr 15, 2022 2:59:01 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 208 files. Enable logging at DEBUG level to see which files will be staged. Apr 15, 2022 2:59:02 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Window.Into() Apr 15, 2022 2:59:02 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Apr 15, 2022 2:59:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 208 files from PipelineOptions.filesToStage to staging location to prepare for execution. Apr 15, 2022 2:59:04 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 208 files cached, 0 files newly uploaded in 0 seconds Apr 15, 2022 2:59:04 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Apr 15, 2022 2:59:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <109970 bytes, hash bc19c54c2f7a4aa1b4f44519f2191c70aa2c9a672348fbb07eeba1fc9c233543> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-vBnFTC96SqG09EUZ8hkccKosmmcjSPuwfuuh_JwjNUM.pb Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input as step s1 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s2 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s3 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input as step s4 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s5 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s6 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s9 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s10 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s12 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s13 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s14 Apr 15, 2022 2:59:06 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.39.0-SNAPSHOT Apr 15, 2022 2:59:07 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-04-15_07_59_06-13430055039683891030?project=apache-beam-testing Apr 15, 2022 2:59:07 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2022-04-15_07_59_06-13430055039683891030 Apr 15, 2022 2:59:07 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2022-04-15_07_59_06-13430055039683891030 Apr 15, 2022 2:59:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-04-15T14:59:09.652Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20batch0cogbk02-jenkins-041514-f69f. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Apr 15, 2022 2:59:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:16.898Z: Worker configuration: e2-standard-2 in us-central1-f. Apr 15, 2022 2:59:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:18.099Z: Expanding SplittableParDo operations into optimizable parts. Apr 15, 2022 2:59:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:18.245Z: Expanding CollectionToSingleton operations into optimizable parts. Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:18.536Z: Expanding CoGroupByKey operations into optimizable parts. Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.041Z: Expanding GroupByKey operations into optimizable parts. Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.454Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.598Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.720Z: Fusing unzipped copy of CoGroupByKey/GBK/Write, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.851Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/Read Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:19.935Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.012Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.078Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Apr 15, 2022 2:59:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.129Z: Fusing consumer CoGroupByKey/GBK/Write into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.183Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.218Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.247Z: Fusing consumer Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.284Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.305Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.327Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.374Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.421Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.444Z: Fusing consumer Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.467Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.496Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.532Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:20.939Z: Executing operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:21.046Z: Executing operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Apr 15, 2022 2:59:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:21.190Z: Starting 5 ****s in us-central1-f... Apr 15, 2022 2:59:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T14:59:22.391Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Apr 15, 2022 3:00:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:00:03.706Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s). Apr 15, 2022 3:00:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:00:55.178Z: Workers have started successfully. Apr 15, 2022 3:01:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:07.523Z: Finished operation Read input/Impulse+Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Apr 15, 2022 3:01:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:07.563Z: Finished operation Read co-input/Impulse+Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/PairWithRestriction+Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/SplitWithSizing Apr 15, 2022 3:01:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:07.724Z: Executing operation CoGroupByKey/GBK/Create Apr 15, 2022 3:01:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:10.528Z: Finished operation CoGroupByKey/GBK/Create Apr 15, 2022 3:01:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:10.709Z: Executing operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write Apr 15, 2022 3:01:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:10.744Z: Executing operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write Apr 15, 2022 3:01:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:23.069Z: Finished operation Read-co-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write Apr 15, 2022 3:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:26.250Z: Finished operation Read-input-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Collect start time metrics (input)/ParMultiDo(TimeMonitor)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)+CoGroupByKey/GBK/Write Apr 15, 2022 3:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:26.339Z: Executing operation CoGroupByKey/GBK/Close Apr 15, 2022 3:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:26.403Z: Finished operation CoGroupByKey/GBK/Close Apr 15, 2022 3:01:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:26.482Z: Executing operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor) Apr 15, 2022 3:01:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:49.521Z: Finished operation CoGroupByKey/GBK/Read+CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)+Ungroup and reiterate/ParMultiDo(UngroupAndReiterate)+Collect total bytes/ParMultiDo(ByteMonitor)+Collect end time metrics/ParMultiDo(TimeMonitor) Apr 15, 2022 3:01:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:49.698Z: Cleaning up. Apr 15, 2022 3:01:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:01:49.808Z: Stopping **** pool... Apr 15, 2022 3:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:02:43.227Z: Autoscaling: Resized **** pool from 5 to 0. Apr 15, 2022 3:02:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-04-15T15:02:43.297Z: Worker pool stopped. Apr 15, 2022 3:02:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2022-04-15_07_59_06-13430055039683891030 finished with status DONE. Load test results for test (ID): e11b4c11-cad3-4602-976f-35128ae2e70b and timestamp: 2022-04-15T14:59:01.821000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 36.384 dataflow_v2_java11_total_bytes_count 2.2E9 > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415145831 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5f017cae6bdcbb17b5112ff25a8befc1d3add7a507019a9b620464ac889ce59 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415145831] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5f017cae6bdcbb17b5112ff25a8befc1d3add7a507019a9b620464ac889ce59] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220415145831] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:f5f017cae6bdcbb17b5112ff25a8befc1d3add7a507019a9b620464ac889ce59])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eee12169e8647d26b0d7b6b56f024edd720bc74d035f46fc37835899f8f10bee Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:eee12169e8647d26b0d7b6b56f024edd720bc74d035f46fc37835899f8f10bee ERROR: (gcloud.container.images.delete) Not found: response: {'docker-distribution-api-version': 'registry/2.0', 'content-type': 'application/json', 'date': 'Fri, 15 Apr 2022 15:02:55 GMT', 'server': 'Docker Registry', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status': '404', 'content-length': '168', '-content-encoding': 'gzip'} Failed to compute blob liveness for manifest: 'sha256:eee12169e8647d26b0d7b6b56f024edd720bc74d035f46fc37835899f8f10bee': None > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Batch_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 297 * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'. > Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 4m 26s 104 actionable tasks: 8 executed, 96 up-to-date Publishing build scan... https://gradle.com/s/cpb67rwy34gfi Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
