See <https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/47/display/redirect?page=changes>
Changes: [relax] Ensure timer consistency in Dataflow and portable runners ------------------------------------------ [...truncated 67.24 KB...] INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Aug 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Aug 03, 2021 12:48:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <112699 bytes, hash 5bf4778ae49aeabd7c6f3448b0130ce897d8ae625df6f10057ef59ec354a928f> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-W_R3iuSa6r18bzRIsBMM6JfYrmJd9vEAV-9Z7DVKko8.pb Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution. Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Aug 03, 2021 12:48:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@75de6341, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74170687, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@68f0f72c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d96fa9e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3b545206, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@77bb48d5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@181d8899, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12d5c30e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@b887730, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26586b74, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@52f57666, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6e041285, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@267dc982, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@439b15f2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3aa41da1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74fab04a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3c83468e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@272ce069, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3d7fb838, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1c26273d] Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (input) as step s3 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()/Window.Assign as step s4 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s5 Aug 03, 2021 12:48:29 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ab4a5b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2abe9173, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@235d29d6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1fdca564, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@43f9dd56, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1d12e953, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@57cb70be, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2d4608a6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20d87335, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a8a4e0c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@26c89563, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3bd6ba24, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@58f437b0, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20f6f88c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4277127c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4c7e978c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@354e7004, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@22ebccb9, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5875de6a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@608fe01f] Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read co-input/StripIds as step s6 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect start time metrics (co-input) as step s7 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Window.Into()2/Window.Assign as step s8 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/Flatten as step s11 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/GBK as step s12 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Ungroup and reiterate as step s14 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect total bytes as step s15 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Collect end time metrics as step s16 Aug 03, 2021 12:48:29 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.33.0-SNAPSHOT Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-03_05_48_29-15466374677597675524?project=apache-beam-testing Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-08-03_05_48_29-15466374677597675524 Aug 03, 2021 12:48:30 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-08-03_05_48_29-15466374677597675524 Aug 03, 2021 12:48:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-03T12:48:38.239Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0cogbk01-jenkins-08-30f9. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Aug 03, 2021 12:48:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:42.689Z: Worker configuration: e2-standard-2 in us-central1-a. Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.379Z: Expanding SplittableParDo operations into optimizable parts. Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.398Z: Expanding CollectionToSingleton operations into optimizable parts. Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.467Z: Expanding CoGroupByKey operations into optimizable parts. Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.536Z: Expanding SplittableProcessKeyed operations into optimizable parts. Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.573Z: Expanding GroupByKey operations into streaming Read/Write steps Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.638Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.746Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.775Z: Unzipping flatten CoGroupByKey-Flatten for input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.808Z: Fusing unzipped copy of CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.850Z: Fusing consumer CoGroupByKey/GBK/WriteStream into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.886Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.919Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.951Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:43.977Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.012Z: Fusing consumer Collect start time metrics (input)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.050Z: Fusing consumer Window.Into()/Window.Assign into Collect start time metrics (input)/ParMultiDo(TimeMonitor) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.087Z: Fusing consumer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into Window.Into()/Window.Assign Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.121Z: Fusing consumer Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read co-input/Impulse Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.155Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.182Z: Fusing consumer Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.208Z: Fusing consumer Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.236Z: Fusing consumer Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) into Read co-input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.285Z: Fusing consumer Window.Into()2/Window.Assign into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.315Z: Fusing consumer CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into Window.Into()2/Window.Assign Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.350Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets into CoGroupByKey/GBK/ReadStream Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.395Z: Fusing consumer CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into CoGroupByKey/GBK/MergeBuckets Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.432Z: Fusing consumer Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) into CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.463Z: Fusing consumer Collect total bytes/ParMultiDo(ByteMonitor) into Ungroup and reiterate/ParMultiDo(UngroupAndReiterate) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.499Z: Fusing consumer Collect end time metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor) Aug 03, 2021 12:48:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:44.874Z: Starting 5 ****s in us-central1-a... Aug 03, 2021 12:48:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:48:56.617Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Aug 03, 2021 12:49:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:49:28.554Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Aug 03, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:50:29.747Z: Workers have started successfully. Aug 03, 2021 12:50:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:50:29.771Z: Workers have started successfully. Aug 03, 2021 2:24:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:46.429Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:48.895Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:48.960Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:48.993Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.057Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.101Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.146Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.188Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.226Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.282Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.354Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.416Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.509Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible. Aug 03, 2021 2:24:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.553Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible. Aug 03, 2021 2:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:24:49.964Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible. Aug 03, 2021 2:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-03T14:24:50.113Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:39.148Z: Cancel request is committed for workflow job: 2021-08-03_05_48_29-15466374677597675524. Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:39.192Z: Cleaning up. Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:39.274Z: Stopping **** pool... Aug 03, 2021 4:00:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:39.329Z: Stopping **** pool... Aug 03, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:02:51.136Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Aug 03, 2021 4:02:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:02:51.165Z: Worker pool stopped. Aug 03, 2021 4:02:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-08-03_05_48_29-15466374677597675524 finished with status CANCELLED. Aug 03, 2021 4:02:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace co_gbk Load test results for test (ID): cdc8b56a-074d-4acc-aac6-05c82c5b7db2 and timestamp: 2021-08-03T12:48:24.892000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 0.0 dataflow_v2_java11_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62) at org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c Deleted: sha256:7ffa1617cc5756adca6c4e2a47b03da071e0eb277c7af3a88a83bff52d168a8c Deleted: sha256:7de950114b527f38342b46e86256fb326d7e256892a5c82a25bd86a6295d75d9 Deleted: sha256:f22e3ae1e5a2ea6705a095f5b70272764ed5c78dd66728545fdbeb8130a1f4aa Deleted: sha256:3d11e9cc7091c7111c4fd71ba641afa25361efd0f15fb37274fd6edf05213532 Deleted: sha256:017f298ee13d3d1b35c87dd5021e4079a52c4ac0225d66dda21cb0732a6245ea Deleted: sha256:2c710f291e86fc92e9f5ac23def05533bab3f0c74e3ee189c5531739e37b69af Deleted: sha256:eef629c6850e3774563785be4b0ae55691f7b6d2537f1f3d858e719fb2ee275d Deleted: sha256:5c2a82013053a5b77495d09784ff9bced1b911046b4bb189c743d613aeb0229b Deleted: sha256:f2a252dd325853c3527ce7fb37189a8090b3da5c8707d9b918ae61dad739e4f5 Deleted: sha256:cd47bb2764f6144ae02f6531641aff528d3fa150e61ea89da3e29760ef1daedd Deleted: sha256:a459ef2786bb82f8812e73bd30aa5129a9766c74492f882507ed49bb02a5ce6d Deleted: sha256:afbc2ca52062a1383376d328048aa56ce023d9393cca8178f844641c7e4e630b Deleted: sha256:508709c06bdab7ddde681dbff9b64d95330e2cb61cb570722dd68ea045ae68b6 Deleted: sha256:235d7fcc299cf7b4a61b7bfe72485efada642bd787f698ce4782b8329ce7bef3 Deleted: sha256:9e541c110f768342858c37144b4b634b9b36015c9ab78ba63b62ca6e10e1548b Deleted: sha256:bafff0f6f7c490cdaa3356396f0ebbb496701af271d24547e225845b4dd8fd0a Deleted: sha256:32c190aeeb7870ce156cc4915c65ca1612feccebc1d2bac071403c681fbc196e Deleted: sha256:4618eaaf49cddfee532a8ef759ea61ed0598e76de44781741290d8b02c481ee2 Deleted: sha256:4da66f2e88192cd8158dae2970a5fd60807f5b42494ea2b829d22fed80ac49d1 Deleted: sha256:2d8b113c9c7e410fb469aedf3e654d628aa5c244dd58d6f6c67a3f933cc517a0 Deleted: sha256:7f8c9a6337e852abc3bc18cd1c1dce52927fc5577cac5a6beca6256fe507c250 Deleted: sha256:047e56c88eb512031d6431ffc8986d638806a7640ae44bbb035837a0f5c6bda2 Deleted: sha256:dad0c445a951908af1ec4037312002f0286a2f5b4cda951e2e9cab3725fe2d6f Deleted: sha256:a51cafecec50418e7b9d1b8fdd8f7fe2f5824e12bde026e1b1a53ea0fa048858 Deleted: sha256:d15f72703c91c4514697e888030217189f9ada3212bdf369c04857910612a4b8 Deleted: sha256:4e3ece6ee8d733dd8bcf40d3c39f734c2aa1dc3365a73e339a44fb2e07290b55 Deleted: sha256:44365386bda6993d01da0cb6335e94b1657a32531a0871adaadc866f0f10dbab Deleted: sha256:56b2e344476804ee7dbe1414c05b211a9357569a030dd3e7696d2ab4856c3ead Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803124334] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:62e77f0cbc44bb39fa72bfa484a09f1dac21032330e5e1952cded7950a91540c]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 19m 41s 107 actionable tasks: 76 executed, 29 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/am3rhxrvfxz2c Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
