See <https://ci-beam.apache.org/job/beam_LoadTests_Java_Combine_Dataflow_Streaming/848/display/redirect?page=changes>
Changes: [noreply] Merge pull request #14927 from [BEAM-12400] MongoDBIO support for update ------------------------------------------ [...truncated 79.82 KB...] > Task :sdks:java:core:shadowJar UP-TO-DATE > Task :sdks:java:extensions:arrow:compileJava UP-TO-DATE > Task :sdks:java:extensions:arrow:classes UP-TO-DATE > Task :sdks:java:extensions:arrow:jar UP-TO-DATE > Task :sdks:java:extensions:protobuf:extractIncludeProto UP-TO-DATE > Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE > Task :sdks:java:io:synthetic:compileJava UP-TO-DATE > Task :sdks:java:io:synthetic:classes UP-TO-DATE > Task :sdks:java:io:synthetic:jar UP-TO-DATE > Task :sdks:java:testing:test-utils:compileJava UP-TO-DATE > Task :sdks:java:testing:test-utils:classes UP-TO-DATE > Task :sdks:java:testing:test-utils:jar UP-TO-DATE > Task :sdks:java:fn-execution:compileJava UP-TO-DATE > Task :sdks:java:fn-execution:classes UP-TO-DATE > Task :sdks:java:fn-execution:jar UP-TO-DATE > Task :sdks:java:extensions:protobuf:compileJava UP-TO-DATE > Task :sdks:java:extensions:protobuf:classes UP-TO-DATE > Task :sdks:java:extensions:protobuf:jar UP-TO-DATE > Task :sdks:java:io:kinesis:compileJava UP-TO-DATE > Task :sdks:java:io:kinesis:classes UP-TO-DATE > Task :sdks:java:io:kinesis:jar UP-TO-DATE > Task :runners:core-construction-java:compileJava UP-TO-DATE > Task :runners:core-construction-java:classes UP-TO-DATE > Task :runners:core-construction-java:jar UP-TO-DATE > Task :runners:core-java:compileJava UP-TO-DATE > Task :runners:core-java:classes UP-TO-DATE > Task :runners:core-java:jar UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:compileJava UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:jar UP-TO-DATE > Task :sdks:java:harness:compileJava UP-TO-DATE > Task :sdks:java:harness:classes UP-TO-DATE > Task :sdks:java:harness:jar UP-TO-DATE > Task :sdks:java:harness:shadowJar UP-TO-DATE > Task :runners:java-fn-execution:compileJava UP-TO-DATE > Task :runners:java-fn-execution:classes UP-TO-DATE > Task :runners:java-fn-execution:jar UP-TO-DATE > Task :sdks:java:expansion-service:compileJava UP-TO-DATE > Task :sdks:java:expansion-service:classes UP-TO-DATE > Task :sdks:java:expansion-service:jar UP-TO-DATE > Task :sdks:java:io:kafka:compileJava UP-TO-DATE > Task :sdks:java:io:kafka:classes UP-TO-DATE > Task :sdks:java:io:kafka:jar UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE > Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE > Task :sdks:java:testing:load-tests:classes UP-TO-DATE > Task :sdks:java:testing:load-tests:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE > Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava > UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar UP-TO-DATE > Task :sdks:java:testing:load-tests:run Sep 06, 2021 12:31:53 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage. Sep 06, 2021 12:31:53 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Sep 06, 2021 12:31:54 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 193 files. Enable logging at DEBUG level to see which files will be staged. Sep 06, 2021 12:31:55 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Collect end time metric Sep 06, 2021 12:31:55 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Sep 06, 2021 12:31:57 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Sep 06, 2021 12:31:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <132205 bytes, hash d69753451001c771b09151f41a41a6401ee35a9843233775252647adfb3351d8> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-1pdTRRABx3GwkVH0GkGmQB7jWphDIzd1JSZHrfszUdg.pb Sep 06, 2021 12:31:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 194 files from PipelineOptions.filesToStage to staging location to prepare for execution. Sep 06, 2021 12:31:59 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-QwBCBtprx84y_3PbjBTvra8IZ_z5DGx5ADyBhvZp408.jar Sep 06, 2021 12:31:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 194 files cached, 0 files newly uploaded in 0 seconds Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Sep 06, 2021 12:32:00 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 64 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a2929a4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@cda6019, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@797c3c3b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4012d5bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4375b013, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1cf0cacc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4f5b08d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@529c2a9a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c87fdf2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26bbe604, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fe34b86, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c98781a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3f736a16, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4601203a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53abfc07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c8c16c0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@80bfa9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@47c40b56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4b039c6d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7f5b9db, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@507d64aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37045b48, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@60b34931, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4aa21f9d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d7a9786, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7bab5898, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@366d8b97, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@feb98ef, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7654f833, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@571a01f9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d7109be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e0ec41f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@f4a3a8d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@8d8f754, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@d613308, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@25673087, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7b27e8f4, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@348ad293, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@30f74e79, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c88a3e8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68ac9ec5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a50d709, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5ed4bc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6920b0bc] Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metric as step s3 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect metrics as step s4 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s5 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 0/Map as step s6 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 0/GroupByKey as step s7 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 0/Combine.GroupedValues as step s8 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric as step s9 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 1/Map as step s10 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 1/GroupByKey as step s11 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 1/Combine.GroupedValues as step s12 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric2 as step s13 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 2/Map as step s14 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 2/GroupByKey as step s15 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 2/Combine.GroupedValues as step s16 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric3 as step s17 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 3/Map as step s18 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 3/GroupByKey as step s19 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 3/Combine.GroupedValues as step s20 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric4 as step s21 Sep 06, 2021 12:32:00 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.34.0-SNAPSHOT Sep 06, 2021 12:32:01 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-06_05_32_00-13920222753166389013?project=apache-beam-testing Sep 06, 2021 12:32:01 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-09-06_05_32_00-13920222753166389013 Sep 06, 2021 12:32:01 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-09-06_05_32_00-13920222753166389013 Sep 06, 2021 12:32:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-09-06T12:32:05.185Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java0dataflow0streaming0combine04-jenkins-09061-f2yz. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Sep 06, 2021 12:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:07.975Z: Worker configuration: e2-standard-4 in us-central1-a. Sep 06, 2021 12:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:08.735Z: Expanding CoGroupByKey operations into optimizable parts. Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:08.927Z: Expanding SplittableProcessKeyed operations into optimizable parts. Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:08.959Z: Expanding GroupByKey operations into streaming Read/Write steps Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.131Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.320Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.352Z: Fusing consumer Convert to Long: 1/Map into Window.Into()/Window.Assign Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.384Z: Fusing consumer Convert to Long: 2/Map into Window.Into()/Window.Assign Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.420Z: Fusing consumer Convert to Long: 3/Map into Window.Into()/Window.Assign Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.454Z: Fusing consumer Convert to Long: 0/Map into Window.Into()/Window.Assign Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.487Z: Fusing consumer Read input/StripIds into Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.520Z: Fusing consumer Collect start time metric into Read input/StripIds Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.548Z: Fusing consumer Collect metrics into Collect start time metric Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.581Z: Fusing consumer Window.Into()/Window.Assign into Collect metrics Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.615Z: Fusing consumer Combine: 1/GroupByKey/WriteStream into Convert to Long: 1/Map Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.646Z: Fusing consumer Combine: 1/Combine.GroupedValues into Combine: 1/GroupByKey/ReadStream Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.681Z: Fusing consumer Collect end time metric2 into Combine: 1/Combine.GroupedValues Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.718Z: Fusing consumer Combine: 2/GroupByKey/WriteStream into Convert to Long: 2/Map Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.766Z: Fusing consumer Combine: 2/Combine.GroupedValues into Combine: 2/GroupByKey/ReadStream Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.790Z: Fusing consumer Collect end time metric3 into Combine: 2/Combine.GroupedValues Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.824Z: Fusing consumer Combine: 3/GroupByKey/WriteStream into Convert to Long: 3/Map Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.849Z: Fusing consumer Combine: 3/Combine.GroupedValues into Combine: 3/GroupByKey/ReadStream Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.878Z: Fusing consumer Collect end time metric4 into Combine: 3/Combine.GroupedValues Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.907Z: Fusing consumer Combine: 0/GroupByKey/WriteStream into Convert to Long: 0/Map Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.933Z: Fusing consumer Combine: 0/Combine.GroupedValues into Combine: 0/GroupByKey/ReadStream Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:09.964Z: Fusing consumer Collect end time metric into Combine: 0/Combine.GroupedValues Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.311Z: Executing operation Combine: 0/GroupByKey/ReadStream+Combine: 0/Combine.GroupedValues+Collect end time metric Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.346Z: Executing operation Combine: 3/GroupByKey/ReadStream+Combine: 3/Combine.GroupedValues+Collect end time metric4 Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.376Z: Executing operation Combine: 2/GroupByKey/ReadStream+Combine: 2/Combine.GroupedValues+Collect end time metric3 Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.387Z: Starting 16 ****s in us-central1-a... Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.412Z: Executing operation Combine: 1/GroupByKey/ReadStream+Combine: 1/Combine.GroupedValues+Collect end time metric2 Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.444Z: Executing operation Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Read input/StripIds+Collect start time metric+Collect metrics+Window.Into()/Window.Assign+Convert to Long: 1/Map+Convert to Long: 2/Map+Convert to Long: 3/Map+Convert to Long: 0/Map+Combine: 1/GroupByKey/WriteStream+Combine: 2/GroupByKey/WriteStream+Combine: 3/GroupByKey/WriteStream+Combine: 0/GroupByKey/WriteStream Sep 06, 2021 12:32:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:16.623Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Sep 06, 2021 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:50.427Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate. Sep 06, 2021 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:50.449Z: Resized **** pool to 1, though goal was 16. This could be a quota issue. Sep 06, 2021 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:00.911Z: Autoscaling: Raised the number of ****s to 16 so that the pipeline can catch up with its backlog and keep up with its input rate. Sep 06, 2021 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:25.318Z: Workers have started successfully. Sep 06, 2021 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:25.343Z: Workers have started successfully. The message received from the daemon indicates that the daemon has disappeared. Build request sent: Build{id=52d572f7-3eaf-4bc1-bdb0-fa8f29dbf5b2, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_Combine_Dataflow_Streaming/ws/src}> Attempting to read last messages from the daemon log... Daemon pid: 23807 log file: /home/jenkins/.gradle/daemon/6.8.3/daemon-23807.out.log ----- Last 20 lines from daemon log file - daemon-23807.out.log ----- INFO: 2021-09-06T12:32:10.376Z: Executing operation Combine: 2/GroupByKey/ReadStream+Combine: 2/Combine.GroupedValues+Collect end time metric3 Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.387Z: Starting 16 ****s in us-central1-a... Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.412Z: Executing operation Combine: 1/GroupByKey/ReadStream+Combine: 1/Combine.GroupedValues+Collect end time metric2 Sep 06, 2021 12:32:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:10.444Z: Executing operation Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Read input/StripIds+Collect start time metric+Collect metrics+Window.Into()/Window.Assign+Convert to Long: 1/Map+Convert to Long: 2/Map+Convert to Long: 3/Map+Convert to Long: 0/Map+Combine: 1/GroupByKey/WriteStream+Combine: 2/GroupByKey/WriteStream+Combine: 3/GroupByKey/WriteStream+Combine: 0/GroupByKey/WriteStream Sep 06, 2021 12:32:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:16.623Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Sep 06, 2021 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:50.427Z: Autoscaling: Raised the number of ****s to 1 so that the pipeline can catch up with its backlog and keep up with its input rate. Sep 06, 2021 12:32:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:32:50.449Z: Resized **** pool to 1, though goal was 16. This could be a quota issue. Sep 06, 2021 12:33:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:00.911Z: Autoscaling: Raised the number of ****s to 16 so that the pipeline can catch up with its backlog and keep up with its input rate. Sep 06, 2021 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:25.318Z: Workers have started successfully. Sep 06, 2021 12:33:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-06T12:33:25.343Z: Workers have started successfully. Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt. ----- End of the daemon log ----- FAILURE: Build failed with an exception. * What went wrong: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
