See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Batch/848/display/redirect?page=changes>
Changes: [noreply] Added type annotations to some combiners missing it. (#15414) [noreply] [BEAM-12634] JmsIO auto scaling feature (#15464) [noreply] [BEAM-12662] Get Flink version from cluster. (#15223) [noreply] Port changes from Pub/Sub Lite to beam (#15418) [heejong] [BEAM-12805] Fix XLang CombinePerKey test by explicitly assigning the [BenWhitehead] [BEAM-8376] Google Cloud Firestore Connector - Add handling for [noreply] Decreasing peak memory usage for beam.TupleCombineFn (#15494) [noreply] [BEAM-12802] Add support for prefetch through data layers down through [noreply] [BEAM-11097] Add implementation of side input cache (#15483) ------------------------------------------ [...truncated 110.41 KB...] INFO: Adding Step: 107 as step s111 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 108 as step s112 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 109 as step s113 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 110 as step s114 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 111 as step s115 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 112 as step s116 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 113 as step s117 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 114 as step s118 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 115 as step s119 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 116 as step s120 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 117 as step s121 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 118 as step s122 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 119 as step s123 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 120 as step s124 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 121 as step s125 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 122 as step s126 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 123 as step s127 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 124 as step s128 Sep 11, 2021 12:36:41 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 125 as step s129 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 126 as step s130 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 127 as step s131 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 128 as step s132 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 129 as step s133 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 130 as step s134 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 131 as step s135 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 132 as step s136 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 133 as step s137 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 134 as step s138 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 135 as step s139 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 136 as step s140 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 137 as step s141 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 138 as step s142 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 139 as step s143 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 140 as step s144 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 141 as step s145 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 142 as step s146 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 143 as step s147 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 144 as step s148 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 145 as step s149 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 146 as step s150 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 147 as step s151 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 148 as step s152 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 149 as step s153 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 150 as step s154 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 151 as step s155 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 152 as step s156 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 153 as step s157 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 154 as step s158 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 155 as step s159 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 156 as step s160 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 157 as step s161 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 158 as step s162 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 159 as step s163 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 160 as step s164 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 161 as step s165 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 162 as step s166 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 163 as step s167 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 164 as step s168 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 165 as step s169 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 166 as step s170 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 167 as step s171 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 168 as step s172 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 169 as step s173 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 170 as step s174 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 171 as step s175 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 172 as step s176 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 173 as step s177 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 174 as step s178 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 175 as step s179 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 176 as step s180 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 177 as step s181 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 178 as step s182 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 179 as step s183 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 180 as step s184 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 181 as step s185 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 182 as step s186 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 183 as step s187 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 184 as step s188 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 185 as step s189 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 186 as step s190 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 187 as step s191 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 188 as step s192 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 189 as step s193 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 190 as step s194 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 191 as step s195 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 192 as step s196 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 193 as step s197 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 194 as step s198 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 195 as step s199 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 196 as step s200 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 197 as step s201 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 198 as step s202 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 199 as step s203 Sep 11, 2021 12:36:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor)2 as step s204 Sep 11, 2021 12:36:43 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.34.0-SNAPSHOT Sep 11, 2021 12:36:44 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-11_05_36_43-8533884569279126144?project=apache-beam-testing Sep 11, 2021 12:36:44 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-09-11_05_36_43-8533884569279126144 Sep 11, 2021 12:36:44 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-09-11_05_36_43-8533884569279126144 Sep 11, 2021 12:36:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-09-11T12:36:48.407Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java0dataflow0batch0pardo02-jenkins-0911123616--ejrc. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Sep 11, 2021 12:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-11T12:36:53.067Z: Worker configuration: e2-standard-2 in us-central1-c. Sep 11, 2021 12:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-09-11T12:36:53.692Z: Workflow failed. Causes: Project apache-beam-testing has insufficient quota(s) to execute this workflow with 1 instances in region us-central1. Quota summary (required/available): 1/24364 instances, 2/0 CPUs, 25/183491 disk GB, 0/2397 SSD disk GB, 1/217 instance groups, 1/220 managed instance groups, 1/445 instance templates, 1/593 in-use IP addresses. Please see https://cloud.google.com/compute/docs/resource-quotas about requesting more quota. Sep 11, 2021 12:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-11T12:36:53.744Z: Cleaning up. Sep 11, 2021 12:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-11T12:36:53.798Z: Worker pool stopped. Sep 11, 2021 12:36:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-09-11T12:36:55.004Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Sep 11, 2021 12:36:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-09-11_05_36_43-8533884569279126144 failed with status FAILED. Sep 11, 2021 12:36:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace pardo Load test results for test (ID): 52692d46-fac7-4b89-aff1-e9cb27ade83f and timestamp: 2021-09-11T12:36:16.949000000Z: Metric: Value: dataflow_runtime_sec 0.0 dataflow_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: FAILED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53) at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103) > Task :sdks:java:testing:load-tests:run FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1m 15s 90 actionable tasks: 1 executed, 89 up-to-date Publishing build scan... https://gradle.com/s/3sudndo7fo5fy Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
