See <https://ci-beam.apache.org/job/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java11/51/display/redirect?page=changes>
Changes: [noreply] [BEAM-10212] Integrate caching client (#15214) [noreply] Merge pull request #15249: [BEAM-12690] Fix GroupIntoBatches watermark [noreply] Merge pull request #15246 from [BEAM-12685] Allow managed thread count ------------------------------------------ [...truncated 72.62 KB...] Aug 07, 2021 12:23:06 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-07_05_23_05-7586926925864688707?project=apache-beam-testing Aug 07, 2021 12:23:06 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-08-07_05_23_05-7586926925864688707 Aug 07, 2021 12:23:06 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-08-07_05_23_05-7586926925864688707 Aug 07, 2021 12:23:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T12:23:12.796Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0gbk01-jenkins-0807-6apv. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Aug 07, 2021 12:23:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:16.877Z: Worker configuration: e2-standard-2 in us-central1-a. Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.604Z: Expanding SplittableParDo operations into optimizable parts. Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.629Z: Expanding CollectionToSingleton operations into optimizable parts. Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.697Z: Expanding CoGroupByKey operations into optimizable parts. Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.782Z: Expanding SplittableProcessKeyed operations into optimizable parts. Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.822Z: Expanding GroupByKey operations into streaming Read/Write steps Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:17.904Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.022Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.051Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.087Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.136Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.172Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.204Z: Fusing consumer Collect start time metrics/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.266Z: Fusing consumer Total bytes monitor/ParMultiDo(ByteMonitor) into Collect start time metrics/ParMultiDo(TimeMonitor) Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.328Z: Fusing consumer Window.Into()/Window.Assign into Total bytes monitor/ParMultiDo(ByteMonitor) Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.361Z: Fusing consumer Group by key (0)/WriteStream into Window.Into()/Window.Assign Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.386Z: Fusing consumer Group by key (0)/MergeBuckets into Group by key (0)/ReadStream Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.424Z: Fusing consumer Ungroup and reiterate (0)/ParMultiDo(UngroupAndReiterate) into Group by key (0)/MergeBuckets Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.469Z: Fusing consumer Collect end time metrics (0)/ParMultiDo(TimeMonitor) into Ungroup and reiterate (0)/ParMultiDo(UngroupAndReiterate) Aug 07, 2021 12:23:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:18.823Z: Starting 5 ****s in us-central1-a... Aug 07, 2021 12:23:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:30.869Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Aug 07, 2021 12:24:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:23:58.936Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Aug 07, 2021 12:24:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:24:56.361Z: Workers have started successfully. Aug 07, 2021 12:24:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T12:24:56.389Z: Workers have started successfully. Aug 07, 2021 3:32:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:19.950Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible. Aug 07, 2021 3:32:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:20.104Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible. Aug 07, 2021 3:32:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:20.327Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible. Aug 07, 2021 3:32:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:20.371Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible. Aug 07, 2021 3:32:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:22.237Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible. Aug 07, 2021 3:32:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:22.303Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible. Aug 07, 2021 3:32:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:23.294Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible. Aug 07, 2021 3:32:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:23.344Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible. Aug 07, 2021 3:32:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:32:23.545Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible. Aug 07, 2021 3:32:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:32:23.603Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:35:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:35:22.441Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:38:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:19.609Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible. Aug 07, 2021 3:38:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:19.677Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible. Aug 07, 2021 3:38:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:19.920Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible. Aug 07, 2021 3:38:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:19.965Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:21.906Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:21.965Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:22.731Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:22.776Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:38:22.869Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible. Aug 07, 2021 3:38:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:38:22.919Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:41:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:41:22.810Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:44:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:19.559Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible. Aug 07, 2021 3:44:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:19.616Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible. Aug 07, 2021 3:44:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:19.822Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible. Aug 07, 2021 3:44:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:19.870Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:21.670Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:21.726Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:22.439Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:22.479Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:44:22.561Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible. Aug 07, 2021 3:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:44:22.602Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:47:23.326Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:19.627Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible. Aug 07, 2021 3:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:19.691Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible. Aug 07, 2021 3:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:19.911Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible. Aug 07, 2021 3:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:19.976Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible. Aug 07, 2021 3:50:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:21.904Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible. Aug 07, 2021 3:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:21.962Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible. Aug 07, 2021 3:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:22.751Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible. Aug 07, 2021 3:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:22.790Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible. Aug 07, 2021 3:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:50:22.879Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible. Aug 07, 2021 3:50:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:50:22.909Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:53:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:53:22.699Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:19.609Z: Staged package beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.33.0-SNAPSHOT-l-wlPwuqL3JoShJc8MdSNmXKDFI5maGcuJPHhyiddpA.jar' is inaccessible. Aug 07, 2021 3:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:19.727Z: Staged package beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.33.0-SNAPSHOT-V9yHrJQiSt9kiJZ7EuXMHFTsASG3sfXHAWinjagDbFA.jar' is inaccessible. Aug 07, 2021 3:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:19.941Z: Staged package common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-config-5.3.2-B_HQvj7POWJFHkM1OEwdiS2Y6zStZi7OXoiw-BJN7Nc.jar' is inaccessible. Aug 07, 2021 3:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:19.979Z: Staged package common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/common-utils-5.3.2-4Rm-xdfQw0Yp9EH6ym0CZyPloNIGMAJ4b5DO_bByg9c.jar' is inaccessible. Aug 07, 2021 3:56:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:21.749Z: Staged package kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-avro-serializer-5.3.2-7FAoV2nngjZLbCXD3IPBOus_R_2KrM_zffAaNlVOls4.jar' is inaccessible. Aug 07, 2021 3:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:21.800Z: Staged package kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-schema-registry-client-5.3.2-0mGQmXEF1E2hwRsR-O-mddgC_DwcNIU4Ba9SQcw2hKY.jar' is inaccessible. Aug 07, 2021 3:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:22.550Z: Staged package spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-core-4.3.18.RELEASE-qd-hKocv03STfOrJiihBIfzwVyBqE8q9UnPLPqutu2w.jar' is inaccessible. Aug 07, 2021 3:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:22.599Z: Staged package spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/spring-expression-4.3.18.RELEASE-aB3PS8pe5rtvJwR2BF3WrB6nO_rdEY2KpkFz7bel1_A.jar' is inaccessible. Aug 07, 2021 3:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-07T15:56:22.673Z: Staged package zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/zkclient-0.10-JumIuLuoOMck_YNQszHui1_8WcOpwHTfEVxMOmyEOHg.jar' is inaccessible. Aug 07, 2021 3:56:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:56:22.714Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 3:59:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-07T15:59:23.349Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 07, 2021 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:00:42.122Z: Cancel request is committed for workflow job: 2021-08-07_05_23_05-7586926925864688707. Aug 07, 2021 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:00:42.158Z: Cleaning up. Aug 07, 2021 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:00:42.219Z: Stopping **** pool... Aug 07, 2021 4:00:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:00:42.273Z: Stopping **** pool... Aug 07, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:03:07.013Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Aug 07, 2021 4:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-07T16:03:07.050Z: Worker pool stopped. Aug 07, 2021 4:03:13 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-08-07_05_23_05-7586926925864688707 finished with status CANCELLED. Aug 07, 2021 4:03:13 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace gbk Load test results for test (ID): 8d6ccf92-45f1-43a0-8ba8-97efaca49800 and timestamp: 2021-08-07T12:23:00.244000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 0.0 dataflow_v2_java11_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.GroupByKeyLoadTest.run(GroupByKeyLoadTest.java:57) at org.apache.beam.sdk.loadtests.GroupByKeyLoadTest.main(GroupByKeyLoadTest.java:131) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807121722 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807121722] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210807121722] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a49d0c85df7301a47cb162365b0da4993316b6c863dafe73412a580f220458e5])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:56a43e81ab3da8844f90333234e72125cb593df97a923e9769ab26bc54229cb9 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:56a43e81ab3da8844f90333234e72125cb593df97a923e9769ab26bc54229cb9 ERROR: (gcloud.container.images.delete) Not found: response: {'status': '404', 'content-length': '168', 'x-xss-protection': '0', 'transfer-encoding': 'chunked', 'server': 'Docker Registry', '-content-encoding': 'gzip', 'docker-distribution-api-version': 'registry/2.0', 'cache-control': 'private', 'date': 'Sat, 07 Aug 2021 16:03:16 GMT', 'x-frame-options': 'SAMEORIGIN', 'content-type': 'application/json'} Failed to compute blob liveness for manifest: 'sha256:56a43e81ab3da8844f90333234e72125cb593df97a923e9769ab26bc54229cb9': None > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages FAILED FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Build file '<https://ci-beam.apache.org/job/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 282 * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:cleanUpDockerImages'. > Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 46m 14s 107 actionable tasks: 79 executed, 26 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/gnbrfyqfqlj44 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
