See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/209/display/redirect?page=changes>
Changes: [ningkang0957] [BEAM-13602] Prevented metrics gathering from failing bigtable io [ningkang0957] make the code more pythonic [mmack] [BEAM-13243][BEAM-8374] Add support for missing PublishResponse fields [Robert Bradshaw] Optional args and kwargs for named external transforms. [noreply] [BEAM-13628] Update SideInputCache to use full Transform and [noreply] [BEAM-13432] Skip ExpansionService creation in Job Server (#16222) [noreply] [BEAM-13616] Initial files for vendored gRPC 1.43.2 (#16460) ------------------------------------------ [...truncated 51.90 KB...] Jan 12, 2022 12:45:20 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Jan 12, 2022 12:45:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <113691 bytes, hash 6522d52742e16e054120c6f4efe9c40732f4b02644e43f9cbc0517ce707fc1d5> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-ZSLVJ0LhbgVBIMb07-nEBzL0sCZE5D-cvAUXznB_wdU.pb Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Jan 12, 2022 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@71c17a57, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@640ab13c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1e0a864d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@440e3ce6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3e67f5f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@fd53053, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4527f70a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@707b1a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7132a9dc, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57435801, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2da66a44, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@527fc8e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@61bfc9bf, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2c7106d9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@329bad59, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@862f408, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@178f268a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@33ecbd6c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5c723f2d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@432f521f] Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s3 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s4 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5 Jan 12, 2022 12:45:22 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3ac3f6f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1abebef3, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@18f55704, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@67cefd84, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5fbe155, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6add8e3f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58a2b917, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@48904d5a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12bbfc54, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1491344a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@59b65dce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1386313f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e922647, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@433c6abb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@288f173f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@a22c4d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@45cd7bc5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6504a875, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@35e26d05, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@29fa6b65] Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/StripIds as step s6 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s7 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s8 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s11 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s12 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s14 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 Jan 12, 2022 12:45:22 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.37.0-SNAPSHOT Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-01-12_04_45_22-2112328189684646921?project=apache-beam-testing Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2022-01-12_04_45_22-2112328189684646921 Jan 12, 2022 12:45:26 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2022-01-12_04_45_22-2112328189684646921 Jan 12, 2022 12:45:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T12:45:32.394Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-01-zia0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:37.324Z: Worker configuration: e2-standard-2 in us-central1-b. Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:37.975Z: Expanding SplittableParDo operations into optimizable parts. Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.013Z: Expanding CollectionToSingleton operations into optimizable parts. Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.080Z: Expanding CoGroupByKey operations into optimizable parts. Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.177Z: Expanding SplittableProcessKeyed operations into optimizable parts. Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.205Z: Expanding GroupByKey operations into streaming Read/Write steps Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.270Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.362Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.397Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.429Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.463Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.499Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.533Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.575Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.621Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.651Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.675Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.699Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.735Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.761Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.785Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.823Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.846Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.872Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.909Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.941Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:38.979Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:39.033Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:39.057Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:39.092Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Jan 12, 2022 12:45:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:45:39.442Z: Starting 5 ****s in us-central1-b... Jan 12, 2022 12:46:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:46:02.553Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Jan 12, 2022 12:46:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:46:22.971Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Jan 12, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:47:24.007Z: Workers have started successfully. Jan 12, 2022 12:47:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T12:47:24.043Z: Workers have started successfully. Jan 12, 2022 1:25:54 PM org.apache.beam.sdk.metrics.MetricsEnvironment getCurrentContainer WARNING: Reporting metrics are not supported in the current execution environment. Jan 12, 2022 1:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:51:40.929Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible. Jan 12, 2022 1:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:51:41.206Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible. Jan 12, 2022 1:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:51:43.788Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible. Jan 12, 2022 1:51:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T13:51:43.863Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Jan 12, 2022 1:54:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T13:54:44.130Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Jan 12, 2022 1:57:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:57:41.082Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible. Jan 12, 2022 1:57:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:57:41.267Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible. Jan 12, 2022 1:57:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T13:57:43.441Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible. Jan 12, 2022 1:57:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T13:57:43.533Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Jan 12, 2022 2:00:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T14:00:43.788Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Jan 12, 2022 2:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T14:03:41.562Z: Staged package gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/gcsio-2.2.4-p9i4-xCmDBUow6YptIUTZYGtZZhNat4tocR78u1Q8kQ.jar' is inaccessible. Jan 12, 2022 2:03:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T14:03:41.735Z: Staged package google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/google-api-services-iamcredentials-v1-rev20210326-1.32.1-rzLlTs22sMZOFz3yEOfAB70ZZw2Gl99hr42n_k-f8-M.jar' is inaccessible. Jan 12, 2022 2:03:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2022-01-12T14:03:44.117Z: Staged package util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/util-2.2.4-r-7-okFosTGJSr5au-fNqmC_EgCN_DyG3X-dgI2RZFA.jar' is inaccessible. Jan 12, 2022 2:03:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2022-01-12T14:03:44.199Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:00:37.254Z: Cancel request is committed for workflow job: 2022-01-12_04_45_22-2112328189684646921. Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:00:37.292Z: Cleaning up. Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:00:37.361Z: Stopping **** pool... Jan 12, 2022 4:00:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:00:37.419Z: Stopping **** pool... Jan 12, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:03:08.424Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Jan 12, 2022 4:03:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-01-12T16:03:08.462Z: Worker pool stopped. Jan 12, 2022 4:03:14 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2022-01-12_04_45_22-2112328189684646921 finished with status CANCELLED. Load test results for test (ID): d69e0a5f-b8fc-4db2-b8f3-b7f64275daa1 and timestamp: 2022-01-12T12:45:17.603000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 11556.566 dataflow_v2_java11_total_bytes_count 2.5748142E10 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511 Deleted: sha256:0dfc9854b29ecdf055f1a42d1e398367977085dba482d0c8327d549aff737cf5 Deleted: sha256:a7ce9f3e98d3a30793328e3d86904ebe993029b65db25c864dbc67ab3376f2aa Deleted: sha256:bb793563763b6e29ab48bb341b75591515dbbd697a5c0034333368da152e80d0 Deleted: sha256:53d40fcfaa8360999ac629d6ddc422ddad2fabced1b475953b75d819a6b458f2 Deleted: sha256:3bebf7a0f89af4357ae770d74649c3825cb3f35948685ad3eaa553e8d4f53ba1 Deleted: sha256:3ad6a3a5ded08454a5c93b3829329999ba1d6ed96e2ba1c4972adbb6f142fcae Deleted: sha256:3364957c99fc3e105cd364238d8cf0de55e1310d933b4fb5a10f80396ad5d21b Deleted: sha256:5b4fc224828c6ed98d577a185f18175c3fc506a7b82300e8e0fcd434e531ad63 Deleted: sha256:9b9cb659fcf54a259715307f353c0be84706c7789e34adf1b804d6fa5c097314 Deleted: sha256:b49bb393c62913aae01b4bb1ca8430c039bdb4848d175bf97cc73b638a804081 Deleted: sha256:d0c06bdcd092469d21c5586c75d8b31eb3f4246d3586c126d49e5a9c14a8537a Deleted: sha256:15ff7d38261dbfffac9fcc75bd2754d5a90bf9a465e5756ec3087f24147a8a89 Deleted: sha256:7155b87e21468b2b3cf94b07e4b1786d75d50bfa33878c0edb5fe3cb9b40b16e Deleted: sha256:29c6c85b48dbe99fcee69697fc91fcf888576fe8d9c445b120af1a3369231101 Deleted: sha256:e79aae3f0bfa8cd09685a25fde7fb97d57b38caf01b3546a9a3fc2a3d7e5747c Deleted: sha256:3e8890237274d668fea6f34298ea6f4f31959102dfaa9f0cc237db9bfbab8cf8 Deleted: sha256:59fe8ac607ce86a2e423006de90e0a2de4a526e5d98ececadd19f9f42d7409ac Deleted: sha256:b46fffbf14394c0a3eac04831d609a53ce93da7b0a4510d1a27e879ef4ba0fb6 Deleted: sha256:717cc93369dd7e8dc2e29fd3fc8d65e42b7764bf3cd0c793f3cf080734d24df8 Deleted: sha256:363270ea08d7b1d60c83d3587f8818a07ad30ef25fb4bad2029025e587bf14f0 Deleted: sha256:1a8c546957bd16daecd5308fa3d3433010ae9f208aa1cf606a56163608ec8f6f Deleted: sha256:b81e48de15e461cde6c538a541bdd2cb9458411a81b0a199cab4fdba0b36272b Deleted: sha256:3d8b798767443fcc97c23db81bd5244495b322e51cb9c2fcc94921c999e57689 Deleted: sha256:e1850bd3ab12ef03ed4a58607abf2861a8ee248a88f37747bcf1ae976d846bb7 Deleted: sha256:7791a3541d594230ac548fd9296107cbcc9a9d61f60c596358d3628df622e517 Deleted: sha256:eb685e3b53aa6bf21e9a15625559b33994f5079a423bebbf63c037b8a3ee4b13 Deleted: sha256:e7668c49143ff404a6e4a559adc0a61178ca2e044d809dc2e1bd2c5b08d8d076 Deleted: sha256:cda3b5a3ef13623d08b2489ffcff3bcdc090e0c931d8c190abd5aeb09e6cc8c7 Deleted: sha256:9bb76884f8cfe43892bf80649b821cbf4dc50ad6a3f3d9c880c753028b34fcf6 Deleted: sha256:7b86fc8d765768e6ddf9399618660d19a9ff1315be38d2db2a469e95b21f1c8b Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220112124333] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:644ea5771678e45504dcc8c323ea4f587c10cd548c21ff535811bd26e6e6f511]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 3h 19m 58s 109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date Publishing build scan... https://gradle.com/s/s27airpsigb3w Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
