See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/57/display/redirect?page=changes>
Changes: [bsindhwani] Python Kata - Event Time Triggers [bsindhwani] Python Kata - Event Time Triggers - Added license to task-info.yaml [bsindhwani] Python Kata - Event Time Triggers - refactored the accumulation mode [bsindhwani] Python Kata - Event Time Triggers - task description corrected to [bsindhwani] Python Kata - Event Time Triggers - added streaming options [bsindhwani] Python Kata - Event Time Triggers - add license text in task-info.yaml [bsindhwani] Python Kata - Early Triggers [bsindhwani] Python Kata - Event Time Triggers & Early Triggers - task.md - corrected [bsindhwani] Python Kata - Window Accumulation Mode [bsindhwani] Python Kata - Event Time Triggers - removed the external transform [noreply] Loosen typing extensions requirement [bsindhwani] Python Kata - added license to course-info.yaml [Kyle Weaver] [BEAM-12719] ZetaSQL docs: replace out of date table with link to [Robert Burke] [BEAM-11779] Remove appliance restriction. [ryanthompson591] Add retry_gcs_file_copy when 500 errros happen from server [noreply] [BEAM-6374] Emit PCollection metrics from GoSDK (#15289) [noreply] Merge pull request #15165 from [BEAM-12593] Verify DataFrame API on [noreply] Merge pull request #15265 from [BEAM-12545] Extend FileSink by [noreply] [BEAM-12453]: Add interface to access I/O topic information for a Samza [noreply] [BEAM-10212] Guard state caching with experiment (#15319) ------------------------------------------ [...truncated 64.48 KB...] As an alternative, use `gcloud auth configure-docker` to configure `docker` to use `gcloud` as a credential helper, then use `docker` as you would for non-GCR registries, e.g. `docker pull gcr.io/project-id/my-image`. Add `--verbosity=error` to silence this warning: `gcloud docker --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker > Task :sdks:java:testing:load-tests:run Aug 13, 2021 12:44:13 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Aug 13, 2021 12:44:14 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged. Aug 13, 2021 12:44:15 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor) Aug 13, 2021 12:44:15 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Aug 13, 2021 12:44:17 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Aug 13, 2021 12:44:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <107841 bytes, hash 2eb09da47b15d7cc943d9fba31e7a98a66a17aa7098686f4674910a75a4534a7> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-LrCdpHsV18yUPZ-6MeepimaheqcJhob0Z0kQp1pFNKc.pb Aug 13, 2021 12:44:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution. Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Aug 13, 2021 12:44:19 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6342d610, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dc4a691, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@784abd3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@36c2b646, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@434514d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b1dc20f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4613311f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6540cf1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ec8f4b9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@bc042d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5484117b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@37df14d1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7efb53af, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7724704f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3dfa819, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4ce94d2f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68ab0936, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3cd9aa64, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@42b84286, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@443effcb] Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor) as step s3 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(ByteMonitor) as step s4 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 0 as step s5 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 1 as step s6 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 2 as step s7 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 3 as step s8 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 4 as step s9 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 5 as step s10 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 6 as step s11 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 7 as step s12 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 8 as step s13 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 9 as step s14 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor)2 as step s15 Aug 13, 2021 12:44:19 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.33.0-SNAPSHOT Aug 13, 2021 12:44:20 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-13_05_44_19-1711227200837009647?project=apache-beam-testing Aug 13, 2021 12:44:20 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-08-13_05_44_19-1711227200837009647 Aug 13, 2021 12:44:20 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-08-13_05_44_19-1711227200837009647 Aug 13, 2021 12:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T12:44:29.644Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0pardo01-jenkins-08-ijy5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Aug 13, 2021 12:44:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:34.534Z: Worker configuration: e2-standard-2 in us-central1-a. Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.364Z: Expanding SplittableParDo operations into optimizable parts. Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.391Z: Expanding CollectionToSingleton operations into optimizable parts. Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.449Z: Expanding CoGroupByKey operations into optimizable parts. Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.479Z: Expanding SplittableProcessKeyed operations into optimizable parts. Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.497Z: Expanding GroupByKey operations into streaming Read/Write steps Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.519Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.579Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.608Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.637Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.665Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.691Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.734Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.768Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.793Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.824Z: Fusing consumer Step: 1/ParMultiDo(CounterOperation) into Step: 0/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.853Z: Fusing consumer Step: 2/ParMultiDo(CounterOperation) into Step: 1/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.876Z: Fusing consumer Step: 3/ParMultiDo(CounterOperation) into Step: 2/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.902Z: Fusing consumer Step: 4/ParMultiDo(CounterOperation) into Step: 3/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:35.936Z: Fusing consumer Step: 5/ParMultiDo(CounterOperation) into Step: 4/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.009Z: Fusing consumer Step: 6/ParMultiDo(CounterOperation) into Step: 5/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.036Z: Fusing consumer Step: 7/ParMultiDo(CounterOperation) into Step: 6/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.058Z: Fusing consumer Step: 8/ParMultiDo(CounterOperation) into Step: 7/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.082Z: Fusing consumer Step: 9/ParMultiDo(CounterOperation) into Step: 8/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.115Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 9/ParMultiDo(CounterOperation) Aug 13, 2021 12:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:36.413Z: Starting 5 ****s in us-central1-a... Aug 13, 2021 12:44:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:44:51.600Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Aug 13, 2021 12:45:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:45:26.820Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Aug 13, 2021 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:46:20.488Z: Workers have started successfully. Aug 13, 2021 12:46:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T12:46:20.510Z: Workers have started successfully. Aug 13, 2021 3:14:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:14:39.092Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:14:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:14:39.212Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:14:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:14:40.066Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:17:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:17:40.538Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:20:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:20:40.176Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:20:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:20:40.320Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:20:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:20:41.464Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:23:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:23:40.989Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:26:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:26:39.418Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:26:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:26:39.567Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:26:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:26:40.629Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:29:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:29:41.939Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:32:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:32:39.742Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:32:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:32:39.837Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:32:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:32:40.930Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:35:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:35:41.160Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:38:39.381Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:38:39.468Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:38:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:38:40.401Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:41:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:41:41.174Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:44:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:44:39.559Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:44:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:44:39.658Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:44:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:44:40.641Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:47:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:47:40.650Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:50:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:50:39.519Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:50:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:50:39.807Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:50:40.999Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:53:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:53:40.705Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:56:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:56:39.490Z: Staged package kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/kafka-clients-2.4.1-or4Vrm01S3aXE64MuzGE8iWqFvyYIFseskr9n8KjYNk.jar' is inaccessible. Aug 13, 2021 3:56:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-13T15:56:39.585Z: Staged package lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/lz4-java-1.6.0-0ilUWqKx1SA8h2YUvbz_yswwNpf0-PJvdk4dbB7S5BY.jar' is inaccessible. Aug 13, 2021 3:56:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:56:40.618Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 3:59:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-13T15:59:40.779Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:00:38.241Z: Cancel request is committed for workflow job: 2021-08-13_05_44_19-1711227200837009647. Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:00:38.382Z: Cleaning up. Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:00:38.741Z: Stopping **** pool... Aug 13, 2021 4:00:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:00:38.877Z: Stopping **** pool... Aug 13, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:02:59.920Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Aug 13, 2021 4:03:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-13T16:03:00.019Z: Worker pool stopped. Aug 13, 2021 4:03:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-08-13_05_44_19-1711227200837009647 finished with status CANCELLED. Aug 13, 2021 4:03:07 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace pardo Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53) at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210813123736] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1bc0cf8732771f6f93929932d1c16855bdf5a3d5adf8c4eadb089ab88939c03] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210813123736] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1bc0cf8732771f6f93929932d1c16855bdf5a3d5adf8c4eadb089ab88939c03])]. Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1bc0cf8732771f6f93929932d1c16855bdf5a3d5adf8c4eadb089ab88939c03 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d1bc0cf8732771f6f93929932d1c16855bdf5a3d5adf8c4eadb089ab88939c03]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 25m 50s 107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/adnnir5yixwf6 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
