See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_Batch/1351/display/redirect>
Changes: ------------------------------------------ [...truncated 85.29 KB...] > Task :model:fn-execution:compileJava UP-TO-DATE > Task :model:fn-execution:classes UP-TO-DATE > Task :model:fn-execution:shadowJar UP-TO-DATE > Task :sdks:java:core:compileJava UP-TO-DATE > Task :sdks:java:core:classes UP-TO-DATE > Task :sdks:java:core:shadowJar UP-TO-DATE > Task :sdks:java:extensions:protobuf:extractIncludeProto UP-TO-DATE > Task :sdks:java:extensions:protobuf:generateProto NO-SOURCE > Task :sdks:java:extensions:arrow:compileJava UP-TO-DATE > Task :sdks:java:extensions:arrow:classes UP-TO-DATE > Task :sdks:java:extensions:arrow:jar UP-TO-DATE > Task :sdks:java:io:kinesis:compileJava UP-TO-DATE > Task :sdks:java:io:kinesis:classes UP-TO-DATE > Task :sdks:java:extensions:avro:compileJava UP-TO-DATE > Task :sdks:java:extensions:avro:classes UP-TO-DATE > Task :sdks:java:io:kinesis:jar UP-TO-DATE > Task :sdks:java:extensions:avro:jar UP-TO-DATE > Task :sdks:java:extensions:protobuf:compileJava UP-TO-DATE > Task :sdks:java:extensions:protobuf:classes UP-TO-DATE > Task :sdks:java:testing:test-utils:compileJava UP-TO-DATE > Task :sdks:java:testing:test-utils:classes UP-TO-DATE > Task :sdks:java:io:synthetic:compileJava UP-TO-DATE > Task :sdks:java:io:synthetic:classes UP-TO-DATE > Task :sdks:java:extensions:protobuf:jar UP-TO-DATE > Task :sdks:java:fn-execution:compileJava UP-TO-DATE > Task :sdks:java:fn-execution:classes UP-TO-DATE > Task :sdks:java:testing:test-utils:jar UP-TO-DATE > Task :sdks:java:io:synthetic:jar UP-TO-DATE > Task :sdks:java:fn-execution:jar UP-TO-DATE > Task :runners:core-construction-java:compileJava UP-TO-DATE > Task :runners:core-construction-java:classes UP-TO-DATE > Task :runners:core-construction-java:jar UP-TO-DATE > Task :runners:core-java:compileJava UP-TO-DATE > Task :runners:core-java:classes UP-TO-DATE > Task :runners:core-java:jar UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:compileJava UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:classes UP-TO-DATE > Task :sdks:java:extensions:google-cloud-platform-core:jar UP-TO-DATE > Task :sdks:java:harness:compileJava UP-TO-DATE > Task :sdks:java:harness:classes UP-TO-DATE > Task :sdks:java:harness:jar UP-TO-DATE > Task :runners:java-fn-execution:compileJava UP-TO-DATE > Task :runners:java-fn-execution:classes UP-TO-DATE > Task :runners:java-fn-execution:jar UP-TO-DATE > Task :sdks:java:expansion-service:compileJava UP-TO-DATE > Task :sdks:java:expansion-service:classes UP-TO-DATE > Task :sdks:java:expansion-service:jar UP-TO-DATE > Task :sdks:java:io:kafka:compileJava UP-TO-DATE > Task :sdks:java:io:kafka:classes UP-TO-DATE > Task :sdks:java:io:kafka:jar UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:compileJava UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:classes UP-TO-DATE > Task :sdks:java:io:google-cloud-platform:jar UP-TO-DATE > Task :sdks:java:testing:load-tests:compileJava UP-TO-DATE > Task :sdks:java:testing:load-tests:classes UP-TO-DATE > Task :sdks:java:testing:load-tests:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:compileJava UP-TO-DATE > Task :runners:google-cloud-dataflow-java:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:jar UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:compileJava UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:classes UP-TO-DATE > Task :runners:google-cloud-dataflow-java:****:shadowJar UP-TO-DATE > Task :sdks:java:testing:load-tests:run Apr 23, 2023 2:37:34 PM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage. Apr 23, 2023 2:37:35 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Apr 23, 2023 2:37:36 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 229 files. Enable logging at DEBUG level to see which files will be staged. Apr 23, 2023 2:37:36 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: Window.Into() Apr 23, 2023 2:37:37 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Apr 23, 2023 2:37:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 230 files from PipelineOptions.filesToStage to staging location to prepare for execution. Apr 23, 2023 2:37:39 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.48.0-SNAPSHOT-8PlDRrtDuaU45Vm31wjH7MQknWuljheUNWMuEMUqr78.jar Apr 23, 2023 2:37:39 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 230 files cached, 0 files newly uploaded in 0 seconds Apr 23, 2023 2:37:39 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input as step s1 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s2 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s3 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input as step s4 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s5 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s6 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s7 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s8 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s9 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s10 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s11 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s12 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s13 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s14 Apr 23, 2023 2:37:41 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.48.0-SNAPSHOT Apr 23, 2023 2:37:42 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-23_07_37_41-6309550916619077815?project=apache-beam-testing Apr 23, 2023 2:37:42 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2023-04-23_07_37_41-6309550916619077815 Apr 23, 2023 2:37:42 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2023-04-23_07_37_41-6309550916619077815 Apr 23, 2023 2:37:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2023-04-23T14:37:43.941Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java0dataflow0batch0cogbk02-jenkins-0423143736--q9s7. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:49.523Z: Worker configuration: e2-standard-2 in us-central1-b. Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.394Z: Expanding CoGroupByKey operations into optimizable parts. Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.444Z: Expanding GroupByKey operations into optimizable parts. Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.464Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.535Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.552Z: Unzipping flatten s9 for input s7.org.apache.beam.sdk.values.PCollection.<init>:405#4663620f501c9270 Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.573Z: Fusing unzipped copy of CoGroupByKey/GBK/Reify, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0 Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.594Z: Fusing consumer CoGroupByKey/GBK/GroupByWindow into CoGroupByKey/GBK/Read Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.617Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn into CoGroupByKey/GBK/GroupByWindow Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.636Z: Fusing consumer Ungroup and reiterate into CoGroupByKey/ConstructCoGbkResultFn Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.660Z: Fusing consumer Collect total bytes into Ungroup and reiterate Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.684Z: Fusing consumer Collect end time metrics into Collect total bytes Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.704Z: Unzipping flatten s9-u13 for input s10-reify-value0-c11 Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.726Z: Fusing unzipped copy of CoGroupByKey/GBK/Write, through flatten CoGroupByKey/Flatten/Unzipped-1, into producer CoGroupByKey/GBK/Reify Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.748Z: Fusing consumer Collect start time metrics (input) into Read input Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.772Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input) Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.794Z: Fusing consumer CoGroupByKey/MakeUnionTable0 into Window.Into()/Window.Assign Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.814Z: Fusing consumer CoGroupByKey/GBK/Write into CoGroupByKey/GBK/Reify Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.837Z: Fusing consumer CoGroupByKey/GBK/Reify into CoGroupByKey/MakeUnionTable1 Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.859Z: Fusing consumer Collect start time metrics (co-input) into Read co-input Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.883Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input) Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:50.905Z: Fusing consumer CoGroupByKey/MakeUnionTable1 into Window.Into()2/Window.Assign Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:51.129Z: Executing operation CoGroupByKey/GBK/Create Apr 23, 2023 2:37:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:51.183Z: Starting 5 ****s in us-central1-b... Apr 23, 2023 2:37:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:51.548Z: Finished operation CoGroupByKey/GBK/Create Apr 23, 2023 2:37:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:51.654Z: Executing operation Read co-input+Collect start time metrics (co-input)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1+CoGroupByKey/GBK/Reify+CoGroupByKey/GBK/Write Apr 23, 2023 2:37:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:37:51.676Z: Executing operation Read input+Collect start time metrics (input)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0+CoGroupByKey/GBK/Reify+CoGroupByKey/GBK/Write Apr 23, 2023 2:38:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:10.824Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2023-04-23T14:38:40.736Z: Startup of the **** pool in zone us-central1-b failed to bring up any of the desired 5 ****s. Please refer to https://cloud.google.com/dataflow/docs/guides/common-errors#****-pool-failure for help troubleshooting. ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: Instance 'load0tests0java0dataflow0-04230737-d1l0-harness-z8vk' creation failed: The zone 'projects/apache-beam-testing/zones/us-central1-a' does not have enough resources available to fulfill the request. '(resource type:compute)'. Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2023-04-23T14:38:40.761Z: Workflow failed. Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:40.815Z: Finished operation Read co-input+Collect start time metrics (co-input)+Window.Into()2/Window.Assign+CoGroupByKey/MakeUnionTable1+CoGroupByKey/GBK/Reify+CoGroupByKey/GBK/Write Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:40.815Z: Finished operation Read input+Collect start time metrics (input)+Window.Into()/Window.Assign+CoGroupByKey/MakeUnionTable0+CoGroupByKey/GBK/Reify+CoGroupByKey/GBK/Write Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:40.869Z: Cleaning up. Apr 23, 2023 2:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:40.938Z: Stopping **** pool... Apr 23, 2023 2:38:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2023-04-23T14:38:59.209Z: Worker pool stopped. Apr 23, 2023 2:39:35 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2023-04-23_07_37_41-6309550916619077815 failed with status FAILED. Apr 23, 2023 2:39:35 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk Load test results for test (ID): 473c786a-9197-4a92-a771-e6bb4e5f4005 and timestamp: 2023-04-23T14:37:36.569000000Z: Metric: Value: dataflow_runtime_sec 0.0 dataflow_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2m 25s 103 actionable tasks: 1 executed, 102 up-to-date Build scan background action failed. java.lang.IllegalArgumentException: com.gradle.enterprise.gradleplugin.internal.extension.a is not an interface at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:590) at java.lang.reflect.Proxy$ProxyClassFactory.apply(Proxy.java:557) at java.lang.reflect.WeakCache$Factory.get(WeakCache.java:230) at java.lang.reflect.WeakCache.get(WeakCache.java:127) at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419) at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719) at com.gradle.ProxyFactory$ProxyingInvocationHandler.createLocalProxy(ProxyFactory.java:64) at com.gradle.ProxyFactory$ProxyingInvocationHandler.lambda$adaptActionArg$1(ProxyFactory.java:59) at com.gradle.enterprise.gradleplugin.internal.extension.b$3.run(SourceFile:100) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Publishing build scan... https://gradle.com/s/ywgd57hgr7fsu Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
