See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/149/display/redirect?page=changes>
Changes: [mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if [mmack] [BEAM-13009] Bugfix: Prevent dataloss in DynamoDBIO write if [aydar.zaynutdinov] [BEAM-13224][Playground] [Alexey Romanenko] [BEAM-4868] Bump com.amazonaws to 1.12.106 [noreply] [BEAM-13147] Fix nullability issues in aws (SDK v1 + v2) for AwsModule [noreply] [BEAM-3293] Add encoding of map/multimap side inputs into proto (#15960) [noreply] [BEAM-13052] Implement ProtoPlusCoder and add it to the default options [noreply] [BEAM-13193] Add elements field in ProcessBundleRequest and ProcessBu… [noreply] [BEAM-3304] Refactor trigger API (#15952) [noreply] [BEAM-3293] Add NewKeyedIterable function to SideInputAdapter interface [noreply] [BEAM-13223] Documents two new features of cross-language transforms ------------------------------------------ [...truncated 48.94 KB...] 36e0782f1159: Waiting 4efe358eb92d: Waiting ba6e5ff31f23: Waiting 9f9f651e9303: Waiting 548ebbd63351: Waiting 5aa55cdcd3b7: Pushed a18f9f86ccf0: Pushed d51569dbd862: Pushed 337e6e031976: Pushed 5f0c26ac26b3: Pushed 548ebbd63351: Pushed 5ceb4016e1e8: Pushed 4efe358eb92d: Pushed 78700b6b35d0: Layer already exists 62a5b8741e83: Layer already exists 45a48cd22bd3: Pushed 36e0782f1159: Layer already exists ba6e5ff31f23: Layer already exists 9f9f651e9303: Layer already exists 0b3c02b5d746: Layer already exists 62a747bf1719: Layer already exists 2b46f4037d09: Pushed f93b95d11f1c: Pushed d85eeaf7a18a: Pushed 20211113124333: digest: sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467 size: 4311 > Task :sdks:java:testing:load-tests:run Nov 13, 2021 12:45:21 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Nov 13, 2021 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 197 files. Enable logging at DEBUG level to see which files will be staged. Nov 13, 2021 12:45:23 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Window.Into() Nov 13, 2021 12:45:23 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Nov 13, 2021 12:45:24 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 197 files from PipelineOptions.filesToStage to staging location to prepare for execution. Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 197 files cached, 0 files newly uploaded in 0 seconds Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Nov 13, 2021 12:45:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <111704 bytes, hash 2967e31a4fd7e7a80b2376cdaad7e25d14e9ea9929b597ef3e988646365190bc> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-KWfjGk_X56gLI3bNqtfiXRTp6pkptZfvPpiGRjZRkLw.pb Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Nov 13, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b676112, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5578be42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4e49ce2b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@136965e3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53c6f96d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@435cc7f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4364712f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1b7a52dd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f93dd4e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ad5be4a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ad85136, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@737d100a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e5da86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6535117e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d1cbd0f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6fa13e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3af7d855, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77049094, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f88bfbe, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59bbe88a] Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s3 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s4 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5 Nov 13, 2021 12:45:27 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ba6ec50, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@642413d4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fb2e3fd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43a09ce2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f183caa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b66322e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@63538bb4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@24534cb0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a50d9fc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@106d77da, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6f9c5048, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5114b7c7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dd71b20, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@767f6ee7, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b6c6e70, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f324455, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a894088, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@370c1968, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2f0bfe17, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3206174f] Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/StripIds as step s6 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s7 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s8 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s11 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s12 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s14 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 Nov 13, 2021 12:45:27 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.35.0-SNAPSHOT Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-11-13_04_45_27-7894454680376627106?project=apache-beam-testing Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-11-13_04_45_27-7894454680376627106 Nov 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-11-13_04_45_27-7894454680376627106 Nov 13, 2021 12:45:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-11-13T12:45:34.552Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-11-8en0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:38.522Z: Worker configuration: e2-standard-2 in us-central1-a. Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.066Z: Expanding SplittableParDo operations into optimizable parts. Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.127Z: Expanding CollectionToSingleton operations into optimizable parts. Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.193Z: Expanding CoGroupByKey operations into optimizable parts. Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.265Z: Expanding SplittableProcessKeyed operations into optimizable parts. Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.293Z: Expanding GroupByKey operations into streaming Read/Write steps Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.355Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.443Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.471Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Nov 13, 2021 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.499Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.562Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.585Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.611Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.644Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.677Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.707Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.744Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.776Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.809Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.840Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.875Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.909Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.955Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:39.982Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.003Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.038Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.070Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.129Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.159Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.185Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Nov 13, 2021 12:45:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:45:40.512Z: Starting 5 ****s in us-central1-a... Nov 13, 2021 12:46:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:46:11.686Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Nov 13, 2021 12:46:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:46:25.510Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Nov 13, 2021 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:47:21.345Z: Workers have started successfully. Nov 13, 2021 12:47:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T12:47:21.366Z: Workers have started successfully. Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:00:26.555Z: Cancel request is committed for workflow job: 2021-11-13_04_45_27-7894454680376627106. Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:00:26.610Z: Cleaning up. Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:00:26.706Z: Stopping **** pool... Nov 13, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:00:26.745Z: Stopping **** pool... Nov 13, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:02:45.063Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Nov 13, 2021 4:02:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-11-13T16:02:45.106Z: Worker pool stopped. Nov 13, 2021 4:02:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-11-13_04_45_27-7894454680376627106 finished with status CANCELLED. Load test results for test (ID): 19cd8509-1910-4f6d-832d-5f0f7276db5c and timestamp: 2021-11-13T12:45:22.673000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 11552.236 dataflow_v2_java11_total_bytes_count 1.95279897E10 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467 Deleted: sha256:42201945660da287e915df98c55289f326627a04547468a0af76161ee7670678 Deleted: sha256:710a039a07511c2feb76251d921c58051ca7e21bf291247993fb7d4a1f393677 Deleted: sha256:b2858822efffb12059a9df456e0dec1f2441cdfb4a6ea1912042188e95d6751b Deleted: sha256:34c907a17ab4b61b18cd76146136e389ba7aafafd590c068cc92c4622744e121 Deleted: sha256:694c40bd299d70813dbdfb275da9927e239be7150004dc1b6a79a9fd9d290b88 Deleted: sha256:1e74f10f4665b395dfccc6caeb5c1110faf7ea49d4f18a88d0903e151bf6899f Deleted: sha256:6e2ee22f59e3e4f1bd3274328beb01792deb409292cb7c7b77da5892460fce6c Deleted: sha256:9b907e93c92256207b20ad7e12b53bc3a105e3952538e6f041b25d81ed3886d0 Deleted: sha256:c0eac6df4091c4d900d1277418f0bfaacf0aab32b9b959f0abd56f49e8da1231 Deleted: sha256:ec839a739e511698749dfe1544b0ea5714dfe33f88e1cc3f673d56bf2f3fd6c9 Deleted: sha256:55dfe866f54c28378b04c355d28d201d8dca52e383f4ca6114af22a138c1652a Deleted: sha256:0828d2e1fe4ec84bdaa0acefb61e879ca7a562277d20fadcdccebfb33af6bab6 Deleted: sha256:e6d178f5ba7b42ab94e0c6258bc340db5ce354df74682e0a72d6c06df23df137 Deleted: sha256:9eb135533d44d58391215a8b1875ba90d7d0b599f5a577b1ee7e4768b3b60a40 Deleted: sha256:58233ef39756774fd183667b75279f6ffc8266397607f016bbfad06d954076c2 Deleted: sha256:dd016ece6bb68c5091a47effbdeb94bfdac8d9816657d79532d22853252a1826 Deleted: sha256:e75b2ae5a24e4e3489969eee381558c028bce5f71f85a44eb5d58f0ebf133b72 Deleted: sha256:7797b5a8932a65bb997dc382eb1ae027e59a6698798a2f1eca0e6a84421cfc6e Deleted: sha256:fd3e42c53163499cf6fa616bf04f45d1816b4caf3f79b8706ffc9a2cc683f094 Deleted: sha256:1a94b04dfaa07fb5cb681b08707c40f3281f791051686f7399db25f00ed91b78 Deleted: sha256:9a0cc664e37a3979c40cabb23f8d53a877f576503e7abe0c99346c32e3355d9c Deleted: sha256:eb6095aad2e18da16b589c46e0e909c7ab8ed3b16442d6f7efd4079b877b628f Deleted: sha256:eb7fcfc703c62e482882d32855789a0f69f7155c2f971a9060401e2e9bb02ed3 Deleted: sha256:50d6d305c87a3a4e2899f696a92980b41b68c43614eecbc2430db39d4c825f8a Deleted: sha256:452df4e79eed2d3fbbbf153b8a741c47af2fd1d6f7ef7f3259160cd2e32def36 Deleted: sha256:8ad51c25810959682656f926083ca543794a8bac715c001818bf890ba369e389 Deleted: sha256:49c144cd09ca614ce300f151b5d81ba4eaab34eddebb736987224602fdbafc22 Deleted: sha256:06fc00fb81fa24768626a757697c07858452ab2ec94d35136c88b3fbd5208d46 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211113124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:bcabd5c18e1479a4df46a226f2b252d16f9ed630894ade61aced83853f334467]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 19m 34s 101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/5275qyqmey4bo Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
