See <https://ci-beam.apache.org/job/beam_LoadTests_Java_Combine_Dataflow_Batch/850/display/redirect?page=changes>
Changes: [heejong] [BEAM-12838] Update artifact local path for DataflowRunner Java [kwu] [BEAM-12828] Convert UseTestStream tests to use Long instead of Integer [kwu] Apply SpotlessJava [kwu] Apply SpotlessJava [aydar.zaynutdinov] [BEAM-3385] Add requires about `equals()` and `hashMethod()` to [aydar.zaynutdinov] [BEAM-3385] Changes regarding spotlessApply task [noreply] Update runners/flink/job-server/flink_job_server.gradle [Valentyn Tymofieiev] Disable Kafka perf tests. [heejong] separate into resolveArtifacts method [heejong] add test [aydar.zaynutdinov] [BEAM-3385] wrap up equals() and hashCode() methods into links [Andrew Pilloud] [BEAM-12850] Calcite drops empty Calc now [Andrew Pilloud] [BEAM-12853] VALUES produces a UNION, window can't be set afterwards [Andrew Pilloud] [BEAM-12852] Revert BigTable changes, just cast to bigint [heejong] update [heejong] fix formatting [Andrew Pilloud] [BEAM-12851] Map output table names [Luke Cwik] [BEAM-12802] Define a prefetchable iterator and iterable and utility ------------------------------------------ [...truncated 87.39 KB...] > Task :sdks:java:harness:compileJava UP-TO-DATE > Task :sdks:java:harness:classes UP-TO-DATE > Task :sdks:java:harness:jar UP-TO-DATE > Task :sdks:java:harness:shadowJar UP-TO-DATE > Task :runners:java-fn-execution:compileJava UP-TO-DATE > Task :runners:java-fn-execution:classes UP-TO-DATE > Task :runners:java-fn-execution:jar UP-TO-DATE > Task :sdks:java:expansion-service:compileJava UP-TO-DATE > Task :sdks:java:expansion-service:classes UP-TO-DATE > Task :sdks:java:expansion-service:jar UP-TO-DATE > Task :sdks:java:io:kafka:compileJava UP-TO-DATE > Task :sdks:java:io:kafka:classes UP-TO-DATE > Task :sdks:java:io:kafka:jar UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE > Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE > Task :sdks:java:testing:load-tests:classes UP-TO-DATE > Task :sdks:java:testing:load-tests:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE > Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:compileJava > UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar UP-TO-DATE > Task :sdks:java:testing:load-tests:run Sep 08, 2021 12:29:44 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage. Sep 08, 2021 12:29:44 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Sep 08, 2021 12:29:45 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 193 files. Enable logging at DEBUG level to see which files will be staged. Sep 08, 2021 12:29:46 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Collect end time metric Sep 08, 2021 12:29:46 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Sep 08, 2021 12:29:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 194 files from PipelineOptions.filesToStage to staging location to prepare for execution. Sep 08, 2021 12:29:48 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.34.0-SNAPSHOT-kyLIsdCfiZiuGlhZEOOvAXu4J-2BxQlikUE0jy6y3CM.jar Sep 08, 2021 12:29:49 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 194 files cached, 0 files newly uploaded in 0 seconds Sep 08, 2021 12:29:49 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Sep 08, 2021 12:29:49 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <121675 bytes, hash dc353213646ef201a1f9ae97543f1e707998bbb77cbbc0963c73b56cd718c06a> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-3DUyE2Ru8gGh-a6XVD8ecHmYu7d8u8CWPHO1bNcYwGo.pb Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input as step s1 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metric as step s2 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect metrics as step s3 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s4 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 0/Map as step s5 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 0/GroupByKey as step s6 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 0/Combine.GroupedValues as step s7 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric as step s8 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 1/Map as step s9 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 1/GroupByKey as step s10 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 1/Combine.GroupedValues as step s11 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric2 as step s12 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 2/Map as step s13 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 2/GroupByKey as step s14 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 2/Combine.GroupedValues as step s15 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric3 as step s16 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Convert to Long: 3/Map as step s17 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 3/GroupByKey as step s18 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Combine: 3/Combine.GroupedValues as step s19 Sep 08, 2021 12:29:50 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metric4 as step s20 Sep 08, 2021 12:29:51 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.34.0-SNAPSHOT Sep 08, 2021 12:29:52 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-08_05_29_51-12360855658367078439?project=apache-beam-testing Sep 08, 2021 12:29:52 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-09-08_05_29_51-12360855658367078439 Sep 08, 2021 12:29:52 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-09-08_05_29_51-12360855658367078439 Sep 08, 2021 12:29:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-09-08T12:29:55.718Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java0dataflow0batch0combine04-jenkins-090812294-4315. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Sep 08, 2021 12:29:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:29:59.031Z: Worker configuration: e2-standard-2 in us-central1-a. Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:29:59.804Z: Expanding CoGroupByKey operations into optimizable parts. Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:29:59.944Z: Expanding GroupByKey operations into optimizable parts. Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:29:59.975Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.181Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.208Z: Fusing consumer Convert to Long: 2/Map into Window.Into()/Window.Assign Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.233Z: Fusing consumer Convert to Long: 3/Map into Window.Into()/Window.Assign Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.257Z: Fusing consumer Convert to Long: 0/Map into Window.Into()/Window.Assign Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.279Z: Fusing consumer Convert to Long: 1/Map into Window.Into()/Window.Assign Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.303Z: Fusing consumer Collect start time metric into Read input Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.339Z: Fusing consumer Collect metrics into Collect start time metric Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.364Z: Fusing consumer Window.Into()/Window.Assign into Collect metrics Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.398Z: Fusing consumer Combine: 1/GroupByKey+Combine: 1/Combine.GroupedValues/Partial into Convert to Long: 1/Map Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.431Z: Fusing consumer Combine: 1/GroupByKey/Reify into Combine: 1/GroupByKey+Combine: 1/Combine.GroupedValues/Partial Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.454Z: Fusing consumer Combine: 2/GroupByKey+Combine: 2/Combine.GroupedValues/Partial into Convert to Long: 2/Map Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.487Z: Fusing consumer Combine: 2/GroupByKey/Reify into Combine: 2/GroupByKey+Combine: 2/Combine.GroupedValues/Partial Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.519Z: Fusing consumer Combine: 3/GroupByKey+Combine: 3/Combine.GroupedValues/Partial into Convert to Long: 3/Map Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.552Z: Fusing consumer Combine: 3/GroupByKey/Reify into Combine: 3/GroupByKey+Combine: 3/Combine.GroupedValues/Partial Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.586Z: Fusing consumer Combine: 0/GroupByKey+Combine: 0/Combine.GroupedValues/Partial into Convert to Long: 0/Map Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.610Z: Fusing consumer Combine: 0/GroupByKey/Reify into Combine: 0/GroupByKey+Combine: 0/Combine.GroupedValues/Partial Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.634Z: Fusing consumer Combine: 1/GroupByKey/Write into Combine: 1/GroupByKey/Reify Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.661Z: Fusing consumer Combine: 1/Combine.GroupedValues into Combine: 1/GroupByKey/Read Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.685Z: Fusing consumer Combine: 1/Combine.GroupedValues/Extract into Combine: 1/Combine.GroupedValues Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.708Z: Fusing consumer Collect end time metric2 into Combine: 1/Combine.GroupedValues/Extract Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.767Z: Fusing consumer Combine: 2/GroupByKey/Write into Combine: 2/GroupByKey/Reify Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.796Z: Fusing consumer Combine: 2/Combine.GroupedValues into Combine: 2/GroupByKey/Read Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.826Z: Fusing consumer Combine: 2/Combine.GroupedValues/Extract into Combine: 2/Combine.GroupedValues Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.861Z: Fusing consumer Collect end time metric3 into Combine: 2/Combine.GroupedValues/Extract Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.896Z: Fusing consumer Combine: 3/GroupByKey/Write into Combine: 3/GroupByKey/Reify Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.930Z: Fusing consumer Combine: 3/Combine.GroupedValues into Combine: 3/GroupByKey/Read Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.953Z: Fusing consumer Combine: 3/Combine.GroupedValues/Extract into Combine: 3/Combine.GroupedValues Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:00.978Z: Fusing consumer Collect end time metric4 into Combine: 3/Combine.GroupedValues/Extract Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.002Z: Fusing consumer Combine: 0/GroupByKey/Write into Combine: 0/GroupByKey/Reify Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.034Z: Fusing consumer Combine: 0/Combine.GroupedValues into Combine: 0/GroupByKey/Read Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.060Z: Fusing consumer Combine: 0/Combine.GroupedValues/Extract into Combine: 0/Combine.GroupedValues Sep 08, 2021 12:30:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.088Z: Fusing consumer Collect end time metric into Combine: 0/Combine.GroupedValues/Extract Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.451Z: Executing operation Combine: 1/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.475Z: Executing operation Combine: 2/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.512Z: Executing operation Combine: 3/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.523Z: Starting 16 ****s in us-central1-a... Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.540Z: Executing operation Combine: 0/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.942Z: Finished operation Combine: 3/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.942Z: Finished operation Combine: 1/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.944Z: Finished operation Combine: 0/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:01.962Z: Finished operation Combine: 2/GroupByKey/Create Sep 08, 2021 12:30:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:02.179Z: Executing operation Read input+Collect start time metric+Collect metrics+Window.Into()/Window.Assign+Convert to Long: 2/Map+Convert to Long: 3/Map+Convert to Long: 0/Map+Convert to Long: 1/Map+Combine: 1/GroupByKey+Combine: 1/Combine.GroupedValues/Partial+Combine: 1/GroupByKey/Reify+Combine: 2/GroupByKey+Combine: 2/Combine.GroupedValues/Partial+Combine: 2/GroupByKey/Reify+Combine: 3/GroupByKey+Combine: 3/Combine.GroupedValues/Partial+Combine: 3/GroupByKey/Reify+Combine: 0/GroupByKey+Combine: 0/Combine.GroupedValues/Partial+Combine: 0/GroupByKey/Reify+Combine: 1/GroupByKey/Write+Combine: 2/GroupByKey/Write+Combine: 3/GroupByKey/Write+Combine: 0/GroupByKey/Write Sep 08, 2021 12:30:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-09-08T12:30:04.768Z: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/19411 instances, 2/0 CPUs, 25/277066 disk GB, 0/2397 SSD disk GB, 1/227 instance groups, 1/232 managed instance groups, 1/453 instance templates, 1/640 in-use IP addresses. Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota. Sep 08, 2021 12:30:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:31.881Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Sep 08, 2021 12:30:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-09-08T12:30:42.815Z: Autoscaling: Startup of the **** pool in zone us-central1-a reached 6 ****s, but the goal was 16 ****s. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 2000.0 in region us-central1. Sep 08, 2021 12:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-09-08T12:30:55.705Z: Startup of the **** pool in zone us-central1-a failed to bring up any of the desired 16 ****s. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 2000.0 in region us-central1. Sep 08, 2021 12:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-09-08T12:30:55.738Z: Workflow failed. Sep 08, 2021 12:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:55.830Z: Finished operation Read input+Collect start time metric+Collect metrics+Window.Into()/Window.Assign+Convert to Long: 2/Map+Convert to Long: 3/Map+Convert to Long: 0/Map+Convert to Long: 1/Map+Combine: 1/GroupByKey+Combine: 1/Combine.GroupedValues/Partial+Combine: 1/GroupByKey/Reify+Combine: 2/GroupByKey+Combine: 2/Combine.GroupedValues/Partial+Combine: 2/GroupByKey/Reify+Combine: 3/GroupByKey+Combine: 3/Combine.GroupedValues/Partial+Combine: 3/GroupByKey/Reify+Combine: 0/GroupByKey+Combine: 0/Combine.GroupedValues/Partial+Combine: 0/GroupByKey/Reify+Combine: 1/GroupByKey/Write+Combine: 2/GroupByKey/Write+Combine: 3/GroupByKey/Write+Combine: 0/GroupByKey/Write Sep 08, 2021 12:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:55.909Z: Cleaning up. Sep 08, 2021 12:30:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:30:56.001Z: Stopping **** pool... Sep 08, 2021 12:31:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-08T12:31:20.190Z: Worker pool stopped. Sep 08, 2021 12:31:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-09-08_05_29_51-12360855658367078439 failed with status FAILED. Sep 08, 2021 12:31:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace combine Load test results for test (ID): 060674ee-cff8-45e0-a120-41c6397efa1c and timestamp: 2021-09-08T12:29:45.825000000Z: Metric: Value: dataflow_runtime_sec 0.0 dataflow_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.CombineLoadTest.run(CombineLoadTest.java:66) at org.apache.beam.sdk.loadtests.CombineLoadTest.main(CombineLoadTest.java:172) > Task :sdks:java:testing:load-tests:run FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2m 21s 90 actionable tasks: 1 executed, 89 up-to-date Publishing build scan... https://gradle.com/s/m6sksavmppix2 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
