See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/123/display/redirect>
Changes: ------------------------------------------ [...truncated 56.61 KB...] INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s11 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s12 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s14 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 Oct 18, 2021 12:45:31 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.35.0-SNAPSHOT Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-18_05_45_31-14203049816433255547?project=apache-beam-testing Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-10-18_05_45_31-14203049816433255547 Oct 18, 2021 12:45:32 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-10-18_05_45_31-14203049816433255547 Oct 18, 2021 12:45:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T12:45:38.957Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-sqas. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:43.143Z: Worker configuration: e2-standard-2 in us-central1-a. Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:43.857Z: Expanding SplittableParDo operations into optimizable parts. Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:43.882Z: Expanding CollectionToSingleton operations into optimizable parts. Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:43.949Z: Expanding CoGroupByKey operations into optimizable parts. Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.014Z: Expanding SplittableProcessKeyed operations into optimizable parts. Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.039Z: Expanding GroupByKey operations into streaming Read/Write steps Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.095Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.187Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.208Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.235Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.260Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.292Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.324Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.363Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.390Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.416Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.431Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.501Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.530Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.565Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.597Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.624Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.656Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.689Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.714Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.741Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.785Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.814Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.851Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Oct 18, 2021 12:45:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:44.881Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Oct 18, 2021 12:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:45:45.244Z: Starting 5 ****s in us-central1-a... Oct 18, 2021 12:46:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:46:14.278Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Oct 18, 2021 12:46:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:46:36.203Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Oct 18, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:47:37.482Z: Workers have started successfully. Oct 18, 2021 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T12:47:37.509Z: Workers have started successfully. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:46.425Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.367Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.409Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.454Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.505Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.580Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible. Oct 18, 2021 3:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:47.644Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible. Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:48.974Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible. Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:45:49.550Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible. Oct 18, 2021 3:45:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T15:45:49.590Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 18, 2021 3:48:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T15:48:48.937Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 18, 2021 3:51:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:46.458Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.619Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.659Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.694Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.745Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.792Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible. Oct 18, 2021 3:51:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:47.841Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible. Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:51.684Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible. Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:51:52.173Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible. Oct 18, 2021 3:51:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T15:51:52.196Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 18, 2021 3:54:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T15:54:49.008Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:46.185Z: Staged package checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-qual-3.10.0-pNyILKaqxJbTM4Hl5esGBMRUg7AEvD6sk2jxuxfLJRI.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:46.940Z: Staged package google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-1.39.2-arSWeGCSfwUd1_K1deS3qO8182fcLjQIBzqk3uMo4ic.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:46.987Z: Staged package google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-apache-v2-1.39.2-UtOSCfZav29XfhyrRmrbmfEtHNK6TpjYblAfddf4UM8.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:47.029Z: Staged package google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-appengine-1.39.2-9jINwNsDWv9uqy4odIVg2nGKtGX7w0gbZT6j8nqlysk.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:47.066Z: Staged package google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-gson-1.39.2-z4GUT9Qf4NqoabkTvQqq7w5Nj-Z3ShlVQPRU7NPPULY.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:47.101Z: Staged package google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-jackson2-1.39.2-dKSTp7ol4d-z06Td9xMwbVo3htsAo0idWQYdyM_zIbE.jar' is inaccessible. Oct 18, 2021 3:57:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:47.135Z: Staged package google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-http-client-protobuf-1.39.2-48CRMH_IJ9gY8fZ3-5ANC0qF7rqbijaXD90xuQTMr5s.jar' is inaccessible. Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:48.240Z: Staged package opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-contrib-grpc-util-0.28.0-uRaDRuavZZMwChvCfvdCVKofJAGYhZON2PuFK4d9VfA.jar' is inaccessible. Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-18T15:57:48.720Z: Staged package zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zstd-jni-1.4.5-2-lzQxwUtNCahuI7cYQRb8rC2FUB60p0MPfRhczhr0YFA.jar' is inaccessible. Oct 18, 2021 3:57:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-18T15:57:48.744Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:00:25.846Z: Cancel request is committed for workflow job: 2021-10-18_05_45_31-14203049816433255547. Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:00:25.920Z: Cleaning up. Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:00:26.018Z: Stopping **** pool... Oct 18, 2021 4:00:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:00:26.077Z: Stopping **** pool... Oct 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:02:50.941Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Oct 18, 2021 4:02:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-18T16:02:50.979Z: Worker pool stopped. Oct 18, 2021 4:02:56 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-10-18_05_45_31-14203049816433255547 finished with status CANCELLED. Load test results for test (ID): 43d5a22c-7d4e-467d-b8ba-360457bd8fb0 and timestamp: 2021-10-18T12:45:26.825000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 11541.005 dataflow_v2_java11_total_bytes_count 2.66347914E10 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba Deleted: sha256:a9c8e6a10e40c6458c674ad3df0f71f874203fde2d4f097f76e4ca126c5f5bd7 Deleted: sha256:3dbaaf904543b89a81c99997561f9fc1e1ca04d0465c13a9cbeffb7117a418ed Deleted: sha256:9be7aff646e3ecac7ffe5b5729cb2f10468ec343294a5e6f89a18c0f84538b56 Deleted: sha256:17dc3c4c167c47ec274269d5a7e338afa813d3a86c1cb1ee730a3d4c685dde9c Deleted: sha256:7d7172892a76fa790a898b2b8225479efe3d59f0038cfe69b08af3b3f9e1625b Deleted: sha256:6e94d0611cec08dbe29f947ec16346fff79538800eb712873a0a20b7b7b65283 Deleted: sha256:12a0a42cefa779ad517829f79c3b844598a7378b8c32d699f6eafc70b17f8d62 Deleted: sha256:cd9835914c02a5d47c5c13a47ad0299c215d3b70a986e0ac9f2a7bcc0d05f388 Deleted: sha256:b969294e2397f76eab4d2a31e5c95404627e4133f65a5c39d4cd120471f9bff8 Deleted: sha256:35bc0b3e42753c7df41a4fe00330a9c0c74ef569077002af11b98ca45de41a98 Deleted: sha256:d888356de3ffc6ab1621e26e546f78e96a903e24a948443834a2578d808312f7 Deleted: sha256:1e5f414dd9725a4a7aca61ee9b38e356995bc8d62454f3aacd414deb4c2a9659 Deleted: sha256:0b91fe418ba6384286cab87ebe78460c1adc96cf39684041f0ba53c27d2eef2a Deleted: sha256:ed6dc7e7255bc6a359c7cc86243dcd53f8a67c29168767b9be9fedc5f29ebfff Deleted: sha256:8852fc44a380cf00ad8b3aff3e65f40b368191cdabf6a026e6efdc79cbd30051 Deleted: sha256:9ae3ec8d051530b90a6ea38063bcef8e42b01742c5236195c7c2d71a839db82c Deleted: sha256:2e0b196ed93369c3f6c1713b45347ac214fa5d848e9e2b04245ec7a980884c9f Deleted: sha256:d9127b1356b05acc490eddd987582d0db8b2ac018747b3dd0dc4a09c28c56048 Deleted: sha256:9884a28cd1a12d5a25c815afc824da9c1462284ceee324befebaaa04ec1ebbce Deleted: sha256:581a651601c9058540e65efc34628708043a174c672613ec6e9f701f9aba5769 Deleted: sha256:56abf364cb5b331fe81c2e0ed53998a978e4b867ac646c6884af15cc6039c914 Deleted: sha256:3a9c8ca970db3476d1318117e075652be3dc54a17ff4c6dcbea91b69ba6a809d Deleted: sha256:af1d7e3142536074b5e92da4f36546c65c316d645f471bb64bcf2023b56d6351 Deleted: sha256:394935694891be25b97736cfa134e8122d07066e7158e6d987430f3680a444b4 Deleted: sha256:26496153c8d40746d3ad5ff311ca6020ea4db804a87a59b2f94958136c12f1be Deleted: sha256:f008f4d0af97cd55a2f3e41031a1a25c8b32795803aefeef41bd7aa20ab38867 Deleted: sha256:6f929a762cf92dcaaa08e1a66e1f937f1eff277f7bea37497364820d8e613c1a Deleted: sha256:b114444ee7535e21dec502a8781fa7fb1b96ad41d139c4aa142398911e2f12b1 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211018124331] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4df8ffb0b0f0f953f605731eb08a71de653af8fe1b3e9b93e4f42638fc4a24ba]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 19m 40s 101 actionable tasks: 70 executed, 29 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/p7s2jbh462rrk Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
