See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/318/display/redirect?page=changes>
Changes: [yathu] [BEAM-14375] Fix Java Wordcount Dataflow postcommit [Robert Bradshaw] Allow arithmetic between deferred scalars. [noreply] [BEAM-14390] Set user-agent when pulling licenses to avoid 403s (#17521) [noreply] [BEAM-8688] Upgrade GCSIO to 2.2.6 (#17486) [noreply] [BEAM-14253] patch SubscriptionPartitionLoader to work around a dataflow [noreply] Add website link log to notify user of pre-build workflow. (#17498) [noreply] [BEAM-11105] Add timestamp observing watermark estimation (#17476) [noreply] Merge pull request #17487 from Adding user-agent to GCS client in Python [noreply] [BEAM-10265] Display error message if trying to infer recursive schema [noreply] [BEAM-12575] Upgraded ipykernel from v5 to v6 (#17526) [noreply] [BEAM-11105] Add docs + CHANGES.md entry for Go Watermark Estimation [noreply] Merge pull request #17380 from [BEAM-14314][BEAM-9532] Add last_updated ------------------------------------------ [...truncated 51.83 KB...] 0a41459588e0: Layer already exists 30e908a38a18: Layer already exists ea12b6728cfd: Pushed cac2fff6ae3d: Layer already exists 08fa02ce37eb: Layer already exists a037458de4e0: Layer already exists bafdbe68e4ae: Layer already exists a13c519c6361: Layer already exists 7ab7ddbffdd9: Pushed 0ad4f886cb85: Pushed e3817dfcabad: Pushed 20220503124335: digest: sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc size: 4935 > Task :sdks:java:testing:load-tests:run May 03, 2022 12:45:38 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation May 03, 2022 12:45:39 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 221 files. Enable logging at DEBUG level to see which files will be staged. May 03, 2022 12:45:40 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Window.Into() May 03, 2022 12:45:40 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. May 03, 2022 12:45:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 221 files from PipelineOptions.filesToStage to staging location to prepare for execution. May 03, 2022 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 221 files cached, 0 files newly uploaded in 0 seconds May 03, 2022 12:45:47 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ May 03, 2022 12:45:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <119943 bytes, hash 916c5088b3bd03a87ddb8851e5d10a9c1e326c5a3f2545d12c240ea3e199467c> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-kWxQiLO9A6h924hR5dEKnB4ybFo_JUXRLCQOo-GZRnw.pb May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 May 03, 2022 12:45:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@659feb22, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3468ee6e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f4b98f6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@421def93, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58c1da09, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2b2954e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58d6e55a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@751ae8a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d659c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4232b34a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da16263, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f5ce0bb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47e51549, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@101a461c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@360e9c06, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ebffb44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@311ff287, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7377781e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31db34da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@109f8c7e] May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s3 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s4 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5 May 03, 2022 12:45:49 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bf54172, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c9a6717, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b3cde6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6d091cad, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7c663eaf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bb5ceb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e692555, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ba0ae41, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76fe6cdc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2ffb3aec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@786ff1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@46039a21, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@431e86b1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35c4e864, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@32a2a6be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@682af059, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5f36c8e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4da39ca9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6a9344f5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5584d9c6] May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/StripIds as step s6 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s7 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s8 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s11 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s12 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s14 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 May 03, 2022 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.39.0-SNAPSHOT May 03, 2022 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-05-03_05_45_49-12202947964391131369?project=apache-beam-testing May 03, 2022 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2022-05-03_05_45_49-12202947964391131369 May 03, 2022 12:45:50 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2022-05-03_05_45_49-12202947964391131369 May 03, 2022 12:45:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-05-03T12:45:56.412Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-05-tiz5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions May 03, 2022 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:01.823Z: Worker configuration: e2-standard-2 in us-central1-b. May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.649Z: Expanding SplittableParDo operations into optimizable parts. May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.677Z: Expanding CollectionToSingleton operations into optimizable parts. May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.736Z: Expanding CoGroupByKey operations into optimizable parts. May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.813Z: Expanding SplittableProcessKeyed operations into optimizable parts. May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.847Z: Expanding GroupByKey operations into streaming Read/Write steps May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.892Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:02.981Z: Fusing adjacent ParDo, Read, Write, and Flatten operations May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.008Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.035Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.067Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.102Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.124Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.146Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.181Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.206Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.228Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.260Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.296Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.357Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.387Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.419Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.442Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.474Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.507Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.529Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.576Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.611Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.633Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.665Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:03.836Z: Running job using Streaming Engine May 03, 2022 12:46:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:04.175Z: Starting 5 ****s in us-central1-b... May 03, 2022 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:19.910Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete May 03, 2022 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:46:26.469Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. May 03, 2022 12:47:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T12:47:36.901Z: Workers have started successfully. May 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:02.422Z: Cancel request is committed for workflow job: 2022-05-03_05_45_49-12202947964391131369. May 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:02.525Z: Cleaning up. May 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:02.685Z: Stopping **** pool... May 03, 2022 4:01:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:02.742Z: Stopping **** pool... May 03, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:38.352Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. May 03, 2022 4:01:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-05-03T16:01:38.407Z: Worker pool stopped. May 03, 2022 4:01:46 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2022-05-03_05_45_49-12202947964391131369 finished with status CANCELLED. Load test results for test (ID): 3fa53456-e193-4a40-9194-c44fbdaa3b9f and timestamp: 2022-05-03T12:45:39.680000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 11576.192 dataflow_v2_java11_total_bytes_count 1.98093633E10 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220503124335 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc Deleted: sha256:77797acbc43399e9ab657faa73b4cba9807d6c72bb6e46f6edea30eb2fa0fb9f Deleted: sha256:8a55d4f92c0d1cca22085afea7c272e14fcc65d65cf7406939a935b7a5445bc5 Deleted: sha256:43d41362ec0ac58fd8cab074623c8ece1a77c1ad175723c740c5338faf21902e Deleted: sha256:f6a5a984d823a47f549631f558ba6473d3077e90ca0029caa0b2674e29f67cc9 Deleted: sha256:f87d3ea36cabcece3632ff4a62d7091a3703d64e35baa0a84d0713b705a2818b Deleted: sha256:3d871e7fee474850a8251731aea8433c4def3080ffcd2501edb9d7fbe0975c92 Deleted: sha256:0aaad32c70dbaea14fce3a4f010e16593908040df1a844ba4b2e591186f2c4cc Deleted: sha256:f8486df3428b6ae1406bcee567b89d7d3392d33d9988d035e6b4e31cf85b9f24 Deleted: sha256:b27db9e1945e72a938e52ec363f8ad7423e67c448c0aa44e4c5b337514747f12 Deleted: sha256:5176ee6212ed61d21b334e84dfcf43deea67babcf1de8d524a6eb9ab14e4e61e Deleted: sha256:d32951a414779750125b535f6f08d6c3817e7b0d8989b76cb06e1a2e95ebe8e8 Deleted: sha256:ffd241957a7ce08f933a7e75b75fa72d2871e21741655cec76971e05a0d860dc Deleted: sha256:73b82c63a5d99feb4612ca2ee42d43a1fa7beb1eccbff55baeed5d91da116907 Deleted: sha256:1cca4232cf482ab799bbea72eb1b51d44fddef57dc842a8aaf1eae216d8ae5d6 Deleted: sha256:605c3b170015ecbb3c6e1525a7eb5fdb7d27acde7fde0ddc56abae17c2d3a381 Deleted: sha256:97c7c9f7a8d43eb8b113638f65eb10088181a8acfd35ddd2f75f97072903051e Deleted: sha256:b23d222479d8e24cfa92c87f5a5cb2b6a6ca233fbe656481851d82ac00d3986a Deleted: sha256:ead84806d518e2168b678c1a022b902814c83ceeb9c3e5c724f7fbc3392bc04a Deleted: sha256:df04bb64453acf8be71a291c6c632208b2e76244ac906e66e8bc5b1916d4108d Deleted: sha256:55434fb3f2a9bcaaefb3192b477dd1091d9afd5451ccd25e5f4bfef87199644f Deleted: sha256:2281a2de74dba2c958db859fb9ae334fe151280279908fe0fd6ff638712d9acb Deleted: sha256:0c3696bd0f69116745d3c8429911fea097659fe73cc99306aed57d51673e993c Deleted: sha256:befdca89fddbb2471554d829fb8c4bf5948fdfa4e97b92fa835e215acb818b87 Deleted: sha256:2a981d8c87fb6091f2487e4d0966937aba5411c083222b0a4fbf1ff2af2a4774 Deleted: sha256:35cbd9885a01f93cd487130ca3d47f620e32d4fcdb6fd99b35d7a773217f0032 Deleted: sha256:b48baa8cf4619fb4a4e9d456e8f2abfe50ffc386d04ef63ab972a52969aa2a1b Deleted: sha256:995ad8ccb14a171a3dbf1cb759f8b3533aabd55c58c82b47f861ed51a92a5da9 Deleted: sha256:ec39766a8bf313e75267f2410fac32775899dfe9e2f7056bcf6c73212feffb79 Deleted: sha256:050565a41472653c33b0f77947085b0eca61b026b9afb55d3a0e3710616cac33 Deleted: sha256:dbcb31eb4337d3773fa9ee4cf4bf715cc0bb70ba6f4fcc099294c9589420dafe Deleted: sha256:dad918dce7b375c4ba5a7291cf2291565d1f1f871e2921ae19ba953204e3bf69 Deleted: sha256:b00d4e7ce815aac593f4d0e3c3e8f3c3ffca5f59bb1e864887c1342f553a3bea Deleted: sha256:468f275b66fbe4d1ce03751d587303813b73973d4e729f7661580162b4666b63 Deleted: sha256:2b710a722c03b3b01561f12037ed457a612689e160d9953dee9756338c147d5e Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220503124335] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220503124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:c8fd29f18ae9cd4c35079dadd0546583a666dd3bc1827f0935ec365990e097cc]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 3h 18m 45s 109 actionable tasks: 75 executed, 30 from cache, 4 up-to-date Publishing build scan... https://gradle.com/s/5w2pir4tqm5ji Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
