See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/121/display/redirect?page=changes>
Changes: [noreply] Corrected Join Example [noreply] Minor: Add more links to DataFrame API documentation (#15661) [noreply] [BEAM-11480] Use snippets for DataFrame examples (#15600) [noreply] Allow multiple Python worker processe to share the same VM. (#15642) [noreply] [BEAM-12564] Implement Series.hasnans (#15729) [noreply] Minor: Fix frames_test.py equality check for non-frame outputs (#15734) [noreply] [BEAM-12769] Adds integration tests for Java Class Lookup based [Brian Hulette] Address BEAM-4028 cleanup TODOs [Brian Hulette] Bump Dataflow container to beam-master-20211015 [noreply] [BEAM-13052] Restructure pubsublite folder to move non-user interface ------------------------------------------ [...truncated 58.09 KB...] INFO: Adding Ungroup and reiterate as step s14 Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 Oct 16, 2021 12:45:48 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.35.0-SNAPSHOT Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-10-16_05_45_49-17323798727383694038?project=apache-beam-testing Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-10-16_05_45_49-17323798727383694038 Oct 16, 2021 12:45:49 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-10-16_05_45_49-17323798727383694038 Oct 16, 2021 12:45:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T12:45:54.539Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-10-pft5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Oct 16, 2021 12:46:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:45:59.700Z: Worker configuration: e2-standard-2 in us-central1-a. Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.574Z: Expanding SplittableParDo operations into optimizable parts. Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.622Z: Expanding CollectionToSingleton operations into optimizable parts. Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.694Z: Expanding CoGroupByKey operations into optimizable parts. Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.765Z: Expanding SplittableProcessKeyed operations into optimizable parts. Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.793Z: Expanding GroupByKey operations into streaming Read/Write steps Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.851Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.952Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:00.987Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.019Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.052Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.086Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.110Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.143Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.174Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.209Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.244Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.275Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.318Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.351Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.391Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.427Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.464Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.492Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.518Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.546Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.602Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.629Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.673Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Oct 16, 2021 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:01.714Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Oct 16, 2021 12:46:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:02.125Z: Starting 5 ****s in us-central1-a... Oct 16, 2021 12:46:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:27.138Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Oct 16, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:35.807Z: Autoscaling: Raised the number of ****s to 3 so that the pipeline can catch up with its backlog and keep up with its input rate. Oct 16, 2021 12:46:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:35.837Z: Resized **** pool to 3, though goal was 5. This could be a quota issue. Oct 16, 2021 12:46:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:46:46.060Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Oct 16, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:47:40.119Z: Workers have started successfully. Oct 16, 2021 12:47:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T12:47:40.153Z: Workers have started successfully. Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:19:03.749Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible. Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:19:04.603Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible. Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:19:05.723Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible. Oct 16, 2021 1:19:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:19:06.050Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible. Oct 16, 2021 1:19:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:19:06.698Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible. Oct 16, 2021 1:19:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:19:06.811Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:22:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:22:06.787Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:25:03.744Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible. Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:25:04.634Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible. Oct 16, 2021 1:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:25:05.897Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible. Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:25:06.215Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible. Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:25:06.753Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible. Oct 16, 2021 1:25:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:25:06.889Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:28:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:28:06.865Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:31:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:31:03.590Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible. Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:31:04.668Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible. Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:31:05.756Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible. Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:31:06.063Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible. Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:31:06.693Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible. Oct 16, 2021 1:31:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:31:06.797Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:34:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:34:07.023Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:37:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:37:03.471Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible. Oct 16, 2021 1:37:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:37:04.262Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible. Oct 16, 2021 1:37:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:37:05.367Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible. Oct 16, 2021 1:37:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:37:05.671Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible. Oct 16, 2021 1:37:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:37:06.252Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible. Oct 16, 2021 1:37:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:37:06.366Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:40:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:40:06.274Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:43:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:43:03.533Z: Staged package failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/failureaccess-1.0.1-oXHuTHNN0tqDfksWvp30Zhr6typBra8x64Tf2vk2yiY.jar' is inaccessible. Oct 16, 2021 1:43:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:43:04.274Z: Staged package google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-oauth-client-1.31.0-9fiaR9DCEOJ3VPxypLVvfxhohPwMB0pyoLY-Gc52QtA.jar' is inaccessible. Oct 16, 2021 1:43:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:43:05.544Z: Staged package listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-s3KgN9QjCqV_vv_e8w_WEj-cDC24XQrO0AyRuXTzP5k.jar' is inaccessible. Oct 16, 2021 1:43:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:43:05.963Z: Staged package netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-tcnative-boringssl-static-2.0.33.Final-q3CryMTeke7zEAdYTla2hvWe2cxEwrM38_6C8EAtOH0.jar' is inaccessible. Oct 16, 2021 1:43:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-10-16T13:43:06.816Z: Staged package threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/threetenbp-1.5.1-Q0LuBNhwQPcbCqkYjulgeA7y2nNOMqjUOlIqWAteDzs.jar' is inaccessible. Oct 16, 2021 1:43:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:43:06.953Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 1:46:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-10-16T13:46:06.252Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:00:27.167Z: Cancel request is committed for workflow job: 2021-10-16_05_45_49-17323798727383694038. Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:00:27.227Z: Cleaning up. Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:00:27.374Z: Stopping **** pool... Oct 16, 2021 4:00:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:00:27.445Z: Stopping **** pool... Oct 16, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:02:49.591Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Oct 16, 2021 4:02:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-10-16T16:02:49.638Z: Worker pool stopped. Oct 16, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-10-16_05_45_49-17323798727383694038 finished with status CANCELLED. Load test results for test (ID): 13df6ea1-35df-4668-b89d-87eb2f85798e and timestamp: 2021-10-16T12:45:43.518000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 11528.11 dataflow_v2_java11_total_bytes_count 2.56458986E10 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6 Deleted: sha256:c6608b6371c4637110e361d876b767919e20107a3d6e982557fca038825b21a0 Deleted: sha256:a5692feace293c810ad0f53d96f307288c301534a4aa63c8aa8c3e14959b9233 Deleted: sha256:13d26b6f316d4a433baa86d59e4a468137c6dabf5fb5788bfebe01382496b190 Deleted: sha256:3c9134dbc80125648493f303549a6f88fa321d85e497fd652861b9a2551318af Deleted: sha256:5bdb43ffdccd3275cb844aed35217d28fe43b54bda72c14edc31d8a958a1339c Deleted: sha256:f0137e512f65a6f8b679f5ea8ec98cbc22c94ef0ec57d0b5f2f201279ccedadf Deleted: sha256:efdad8ce1b34f47f2e737402df6a80b86d0a188ded817b79b30ef2483c026a93 Deleted: sha256:c301f0738a9ac93cca8d154a89b7234f75151600b9f76ca734f1c031e090e34b Deleted: sha256:6f115d7dcccc1934213dade5be75ec199935f6c01c09ad418bc9d59f8e21dbc5 Deleted: sha256:3393a42e8a1ee7e56d2a5158862df7dcef6e80493b0aa8b48ae72cc7678f3228 Deleted: sha256:f711be6ab86448ce20c8351c0848aab43f265948989121a7369e53f559edc729 Deleted: sha256:58ff9fb58d37de28f1cba77f31b8431e90df3290df2818e2f1af0bb887bdf192 Deleted: sha256:2e00d8c9ca6557b4be89e4e146a2b127eae08fbd1628653b6c8468062f06c001 Deleted: sha256:46804d896e74d4f1e95531875be0142af5291abe916f04064b27dac263839cd8 Deleted: sha256:34b4caed1a3bdf5be866dbabcd9ca98f7b38033f0dd995de6eb3c622e711672b Deleted: sha256:5908e43a92de689066a54b33fb14e1ad226b96fc554bd164c9a6914c3c619db2 Deleted: sha256:eb932af945883c033fa3fc6b93da740b9b77596e157c423e84a78ba36dbecf62 Deleted: sha256:04ecf2be4164234ff927925174181832e863583b836dbd98647f2958f38f3522 Deleted: sha256:403ab567820fe7129b2d117c8663e5b852e341ff5003ad5d83777384cb9b3abe Deleted: sha256:ddb1bf0bbbbd3f35f354ff392659720ddfd292f1ac80c03931ef3baa1bcaebea Deleted: sha256:8f68508c4657b06b94a02853e848fef202bf2b8ba3eefaaeb7fb5a330743675e Deleted: sha256:2a05e65b8c777c3462ad9574123dc3cd316856740ea67ab92e27d3343bbce2e6 Deleted: sha256:c6021de9fbb391c81380278cf46d13fd09c05247b350549e989deae8b1bad323 Deleted: sha256:2c3032d4a45f19e4e9ab451c4def71962133f19904e230b250acb781fea9366b Deleted: sha256:36691e24c1412315338563b813baa922710ec6e4d385ffe3290c7ce3ab161032 Deleted: sha256:f84fd20473a63ee5f8b1db36aec0cd0987006d32729bd52fe96cdb39bd821fc7 Deleted: sha256:964ec6d0fd593c23dc6dc382addf4d87f9c770583321e430facdde78a8985b3a Deleted: sha256:6ab16c16fe924bef92b502027c8594854970a7e7bb7e6bf5454e73504eaf43c6 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20211016124335] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1139a919877fc8a186c189c0b8d5e7658db9db5dace1af8b78d16323f55be1e6]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 19m 38s 101 actionable tasks: 71 executed, 28 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/khlinekl22gz2 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
