See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/48/display/redirect?page=changes>
Changes: [sayat.satybaldiyev] Fix issue with update schema source format [noreply] [BEAM-12696] Update NewTaggedExternal to not require inputs (#15266) [noreply] [BEAM-11088] Add TestStream package to Go SDK testing capabilities [noreply] Merge pull request #15272:[BEAM-12712] Exclude runners that can't handle ------------------------------------------ [...truncated 72.92 KB...] INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-08-04_05_42_35-6895403065916388477 Aug 04, 2021 12:42:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-04T12:42:45.842Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0pardo01-jenkins-08-ooie. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.164Z: Worker configuration: e2-standard-2 in us-central1-a. Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.691Z: Expanding SplittableParDo operations into optimizable parts. Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.719Z: Expanding CollectionToSingleton operations into optimizable parts. Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.783Z: Expanding CoGroupByKey operations into optimizable parts. Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.848Z: Expanding SplittableProcessKeyed operations into optimizable parts. Aug 04, 2021 12:42:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.880Z: Expanding GroupByKey operations into streaming Read/Write steps Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:50.941Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.020Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.055Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.081Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.114Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.147Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.184Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.222Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.250Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.276Z: Fusing consumer Step: 1/ParMultiDo(CounterOperation) into Step: 0/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.303Z: Fusing consumer Step: 2/ParMultiDo(CounterOperation) into Step: 1/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.339Z: Fusing consumer Step: 3/ParMultiDo(CounterOperation) into Step: 2/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.371Z: Fusing consumer Step: 4/ParMultiDo(CounterOperation) into Step: 3/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.407Z: Fusing consumer Step: 5/ParMultiDo(CounterOperation) into Step: 4/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.443Z: Fusing consumer Step: 6/ParMultiDo(CounterOperation) into Step: 5/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.477Z: Fusing consumer Step: 7/ParMultiDo(CounterOperation) into Step: 6/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.510Z: Fusing consumer Step: 8/ParMultiDo(CounterOperation) into Step: 7/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.536Z: Fusing consumer Step: 9/ParMultiDo(CounterOperation) into Step: 8/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.562Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 9/ParMultiDo(CounterOperation) Aug 04, 2021 12:42:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:42:51.955Z: Starting 5 ****s in us-central1-a... Aug 04, 2021 12:43:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:43:01.707Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Aug 04, 2021 12:43:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:43:36.217Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Aug 04, 2021 12:44:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:44:30.328Z: Workers have started successfully. Aug 04, 2021 12:44:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T12:44:30.348Z: Workers have started successfully. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:52.153Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:52.336Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:52.948Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.001Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.110Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.248Z: Staged package commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar' is inaccessible. Aug 04, 2021 3:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.295Z: Staged package commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.403Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.444Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.503Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:53.579Z: Staged package datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:54.835Z: Staged package gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:54.901Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:54.947Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.065Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.177Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.231Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.297Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible. Aug 04, 2021 3:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.371Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible. Aug 04, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.468Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible. Aug 04, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.526Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible. Aug 04, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:51:55.738Z: Staged package metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar' is inaccessible. Aug 04, 2021 3:51:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-04T15:51:56.694Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 04, 2021 3:54:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-04T15:54:55.950Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 04, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:52.177Z: Staged package annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/annotations-4.1.1.4-unNOHoTAnWFa9qCdMwNLTwRC-Hct7BIO-zdthqVlrhU.jar' is inaccessible. Aug 04, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:52.327Z: Staged package avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/avro-1.8.2-91SggwzmelqfpnpU7BXRA-8V4chQ17Jvr3tkfu3cgtM.jar' is inaccessible. Aug 04, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:52.936Z: Staged package beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-bytebuddy-1_11_0-0.1-SneHjzQgCpxxLkvyN8dugqO_sTzwXJ2995kqFu4l2i4.jar' is inaccessible. Aug 04, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.044Z: Staged package beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-guava-26_0-jre-0.1-8quizfXTrmcvJyOXi3zWDFtP5EofgqjksY5sIbsBqws.jar' is inaccessible. Aug 04, 2021 3:57:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.111Z: Staged package checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/checker-compat-qual-2.5.5-EdE0skXpysxHRRTS1mtbhhj4A5oUZc3FW7wLNOAAi3o.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.271Z: Staged package commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-codec-1.14-oSjk-T-r5Tgd7WTPKHMBngYDC3GOtDzurgsOXRetM-k.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.319Z: Staged package commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-compress-1.20-CutiXJSMaX6nsgUVbhEjY7We1eJVEhLNTkYL23LHwG4.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.403Z: Staged package commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-lang3-3.9-3i4dzc8--ReozoWGYaBnJqmpRPKOM61_ngi-pE3DwjA.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.443Z: Staged package commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-logging-1.2-2t3qHqC-D1aXirMAa4rJKDSv7vvZt-TmMW_KV98PpjY.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.477Z: Staged package commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/commons-math3-3.6.1-HlbXsFjSi2Wr0la4RY44hbZ0wdWI-kPNfRy7nH7yswg.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:53.542Z: Staged package datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/datastore-v1-proto-client-1.6.3-8GhVpKiAAK6wztlpNJ48AH4jyyB1KYo-5qJLXiBC6cw.jar' is inaccessible. Aug 04, 2021 3:57:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:54.925Z: Staged package gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gson-2.8.6-yPtIOQVNKAswM_gA0fWpfeLwKOuLoutFitKH5Tbz8l8.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:54.997Z: Staged package hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-2.1-upOy46ViMiukMvChtTrdzFXLGIJTMZoCDtd_gk5pIFA.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.045Z: Staged package hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/hamcrest-core-2.1-4JEJ5UoonYhQa5v-yYfd0Zn0IXyUZBMmaDUbmk8Avuk.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.173Z: Staged package j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/j2objc-annotations-1.3-Ia8wySJnvWEiwOC00gzMtmQaN-r5VsZUDsRx1YTmSns.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.294Z: Staged package jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-core-asl-1.9.13-RAqctcqVshX5U9OiCmsaENofCbUpqd3qX4pJBd2rT1o.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.362Z: Staged package jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-dataformat-cbor-2.12.1-52d5rqlCfKc9f0B-X6gIoEBVeOwFZlOoZcJuTG8B1Cg.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.401Z: Staged package jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jackson-mapper-asl-1.9.13-dOegenby7breKTEqWi68z6AZEovAIezjhW12GX6b4MI.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.517Z: Staged package javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/javax.annotation-api-1.3.2-4EulGVvNVV3JVlD3zGFNFR5LzVLSmhC4qiGX86uJq5s.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.610Z: Staged package json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/json-20200518-D_MObagvFa-vDna9bplqerVWaFEWaoP5c8ZKmg-UbS0.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.653Z: Staged package jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/jsr305-3.0.2-dmrSoHg_JoeWLIrXTO7MOKKLn3Ki0IXuQ4t4E-ko0Mc.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-04T15:57:55.838Z: Staged package metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/metrics-core-3.2.6-iRF0xSMEduGVCihhaWcgApeFYuvfXIXwGZ7K4b0DD58.jar' is inaccessible. Aug 04, 2021 3:57:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-04T15:57:56.805Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 04, 2021 4:00:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:00:47.229Z: Cancel request is committed for workflow job: 2021-08-04_05_42_35-6895403065916388477. Aug 04, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:00:47.265Z: Cleaning up. Aug 04, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:00:47.345Z: Stopping **** pool... Aug 04, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:00:47.389Z: Stopping **** pool... Aug 04, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:03:04.298Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Aug 04, 2021 4:03:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-04T16:03:04.337Z: Worker pool stopped. Aug 04, 2021 4:03:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-08-04_05_42_35-6895403065916388477 finished with status CANCELLED. Aug 04, 2021 4:03:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace pardo Load test results for test (ID): 97ffde55-bb39-477d-9496-7e6e025c9a5b and timestamp: 2021-08-04T12:42:29.731000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 0.0 dataflow_v2_java11_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53) at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804123738 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff Deleted: sha256:318ffb6bae259d107758e7aca994e64948b49f69a01114537ac01af17b3b4e4e Deleted: sha256:86318eaa1ad8de1b9c297d9d65e19277db0bc1952801e1da540119a6e08ef2d7 Deleted: sha256:80f33e80e1b541b84ba660210b7b0a88830e7f3ab48daccc15d3f7af629df964 Deleted: sha256:0e2efac14b6f0419ef237da5f66383bb819d99b01243a395c3d571561d7adda1 Deleted: sha256:832303596a118044c5af222c93895e8e3523f95dbf60f91dcfe497f27dbed330 Deleted: sha256:71601e0c0fc927270550adebb779aba649637b63770bbbc351985f15ffde406b Deleted: sha256:153aa5a76859c1a84ce9887183a84e86fc44ad04c9d280ea9c7b8f2412733da0 Deleted: sha256:122b8b02e548770dc939a76d384e5b3625273aec4bcf49af846d9aeb70c43ff7 Deleted: sha256:d4139f449973feabe32f76d88e342affa4b3a0909b59ab67330442780ec87249 Deleted: sha256:4b7b09056fbb9cca1109718463270e8852c8589730a347edc3ad98a49aa91f9f Deleted: sha256:72dfa3bf01684d94c19606c7324ca6f9711bbca3b8d54c7d66d870009ef6f9e6 Deleted: sha256:7cd135478c6486395d78a3d1b68e1a2985ba897fca619fa288634c5324e07322 Deleted: sha256:1a8ce3c6bc7a1c1b55a3583f1c53484e4289b09e573dbb013b46018d39f08695 Deleted: sha256:50f7a963e301faee8bab12f028f74397628702ffbf81d7a7a4ef31d7a837d99e Deleted: sha256:508dbcdd588096315fda4983ae2c38db7827a849430be184dbb819f1efe9d108 Deleted: sha256:89bb6f0ae384cf97713211dd8d24f4abc809ca5d2e4aa4e5d576f03af39f7da1 Deleted: sha256:ba66cff2a8d2afeb394a02bc907e2472bb5a4b81efd39e59c72199f2953c87ae Deleted: sha256:0e4718a72b42954b332cc6ac7988563bb38afaf0ee62697eb19f156a73892896 Deleted: sha256:644fe77df0209f178f510fe5141c35e8efa1e4dee9c1f14f9e4a3bc1c0de609c Deleted: sha256:0c28b7b355f8c188495069b6fe49edd2d59826bb962b317496d1853b10ae366d Deleted: sha256:6f09e4ea96fd2ed5b7d397a51ac739c2034a3c869ad1abc4419ee391848dc79f Deleted: sha256:b6e085e90fb0acf738f61d76103771386617faaf39753f823616d6aaaa952d08 Deleted: sha256:2d1d92fd6d77b3377e1211de9b3b53604ae2adc965649ea47052801cb9eb0194 Deleted: sha256:d79421734c96c133d18649cfc2bef5d18f37f0d8d7bede75c320453b9b4b0a2b Deleted: sha256:ec19dd2be1c378992567f8deda6b92e52dcdf1d21ddf61da10af472d076a1698 Deleted: sha256:e78967ea956e497a8ed11f82d282006d58631227e69e15af2957ad58da078759 Deleted: sha256:6a18f8e894e215acd50e3d463f29a446cb0f5242ba874b8a91aa0d7a10356b56 Deleted: sha256:b34f70db453d53d5f677c23c79b53b27ace6c78f911861b82aca63520e2db09c Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804123738] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804123738] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:464e1cacb22be0ad5dba573257a87ea4d324bdf1798b544ab6b76f2108e15aff]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 25m 51s 107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/slz3h4tdwim7y Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
