See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/47/display/redirect?page=changes>
Changes: [relax] Ensure timer consistency in Dataflow and portable runners ------------------------------------------ [...truncated 68.51 KB...] 20210803123734: digest: sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa size: 4310 > Task :sdks:java:testing:load-tests:run Aug 03, 2021 12:42:50 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Aug 03, 2021 12:42:50 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 191 files. Enable logging at DEBUG level to see which files will be staged. Aug 03, 2021 12:42:51 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor) Aug 03, 2021 12:42:51 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Aug 03, 2021 12:42:54 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://temp-storage-for-perf-tests/loadtests/staging/ Aug 03, 2021 12:42:54 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <107856 bytes, hash 4784098c6c7e8e6ed53a30e46f1a8f2c4f501a88f085f4c82ef0d8d215db76ca> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-R4QJjGx-jm7VOjDkbxqPLE9QGojwhfTILvDY0hXbdso.pb Aug 03, 2021 12:42:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 191 files from PipelineOptions.filesToStage to staging location to prepare for execution. Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 191 files cached, 0 files newly uploaded in 0 seconds Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Aug 03, 2021 12:42:56 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@12e0f1cb, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4a163575, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7e642b88, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b350309, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@7ecec90d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@588f63c, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5a6fa56e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1981d861, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@118ffcfd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@53f4c1e6, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@74174a23, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6342d610, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@dc4a691, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@784abd3e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@36c2b646, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@434514d8, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6b1dc20f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@4613311f, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6540cf1d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@ec8f4b9] Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor) as step s3 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(ByteMonitor) as step s4 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 0 as step s5 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 1 as step s6 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 2 as step s7 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 3 as step s8 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 4 as step s9 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 5 as step s10 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 6 as step s11 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 7 as step s12 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 8 as step s13 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 9 as step s14 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor)2 as step s15 Aug 03, 2021 12:42:56 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.33.0-SNAPSHOT Aug 03, 2021 12:42:57 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-08-03_05_42_56-9271568491856649635?project=apache-beam-testing Aug 03, 2021 12:42:57 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2021-08-03_05_42_56-9271568491856649635 Aug 03, 2021 12:42:57 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2021-08-03_05_42_56-9271568491856649635 Aug 03, 2021 12:43:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-03T12:43:04.116Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java110dataflow0v20streaming0pardo01-jenkins-08-791h. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Aug 03, 2021 12:43:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:08.460Z: Worker configuration: e2-standard-2 in us-central1-a. Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:08.974Z: Expanding SplittableParDo operations into optimizable parts. Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.004Z: Expanding CollectionToSingleton operations into optimizable parts. Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.071Z: Expanding CoGroupByKey operations into optimizable parts. Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.114Z: Expanding SplittableProcessKeyed operations into optimizable parts. Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.151Z: Expanding GroupByKey operations into streaming Read/Write steps Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.186Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.286Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.324Z: Fusing consumer Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read input/Impulse Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.359Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.408Z: Fusing consumer Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.440Z: Fusing consumer Read input/ParDo(StripIds)/ParMultiDo(StripIds) into Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.497Z: Fusing consumer ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) into Read input/ParDo(StripIds)/ParMultiDo(StripIds) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.532Z: Fusing consumer ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) into ParDo(TimeMonitor)/ParMultiDo(TimeMonitor) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.566Z: Fusing consumer Step: 0/ParMultiDo(CounterOperation) into ParDo(ByteMonitor)/ParMultiDo(ByteMonitor) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.603Z: Fusing consumer Step: 1/ParMultiDo(CounterOperation) into Step: 0/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.634Z: Fusing consumer Step: 2/ParMultiDo(CounterOperation) into Step: 1/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.665Z: Fusing consumer Step: 3/ParMultiDo(CounterOperation) into Step: 2/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.710Z: Fusing consumer Step: 4/ParMultiDo(CounterOperation) into Step: 3/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.740Z: Fusing consumer Step: 5/ParMultiDo(CounterOperation) into Step: 4/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.772Z: Fusing consumer Step: 6/ParMultiDo(CounterOperation) into Step: 5/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.806Z: Fusing consumer Step: 7/ParMultiDo(CounterOperation) into Step: 6/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.845Z: Fusing consumer Step: 8/ParMultiDo(CounterOperation) into Step: 7/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.875Z: Fusing consumer Step: 9/ParMultiDo(CounterOperation) into Step: 8/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:09.898Z: Fusing consumer ParDo(TimeMonitor)2/ParMultiDo(TimeMonitor) into Step: 9/ParMultiDo(CounterOperation) Aug 03, 2021 12:43:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:10.186Z: Starting 5 ****s in us-central1-a... Aug 03, 2021 12:43:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:18.834Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Aug 03, 2021 12:43:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:43:53.122Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate. Aug 03, 2021 12:44:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:44:51.401Z: Workers have started successfully. Aug 03, 2021 12:44:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T12:44:51.433Z: Workers have started successfully. Aug 03, 2021 2:25:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:10.428Z: Staged package api-common-1.10.1-KgM_JLtiA4PtpECtMHy4B3z-wcfq3GhNZSFhI6G5YTo.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/api-common-1.10.1-KgM_JLtiA4PtpECtMHy4B3z-wcfq3GhNZSFhI6G5YTo.jar' is inaccessible. Aug 03, 2021 2:25:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:11.618Z: Staged package auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/auto-value-annotations-1.8.1-N-wJtH1-01qZ0Tkn21yG_JBx9iD5Q-rV11cURpgxCFI.jar' is inaccessible. Aug 03, 2021 2:25:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:12.704Z: Staged package conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/conscrypt-openjdk-uber-2.5.1-AfnHQstZKhUeLmK9U5eomAYoqWcAH82s1KpHRGeGhfM.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.162Z: Staged package netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-buffer-4.1.52.Final-QcoQNa7m4PgXdZfQp2EKWqLG5vrHRKyB_eTl5Y9jlnU.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.213Z: Staged package netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-4.1.52.Final-PH543M5-NT2GrH26LCUatc3dTrhuqoic_kIpeCuCjlI.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.276Z: Staged package netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http-4.1.52.Final-tdoo0OqHiwfTERrPXk-vQSiO8jArJkFBkP3Rp9lzyAw.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.332Z: Staged package netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-http2-4.1.52.Final-61J6QtIonRLe5tR1PKJxWXije7tBxxkVzk6VOcSfk6E.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.369Z: Staged package netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-codec-socks-4.1.52.Final-_sxXfgu3ch1naM_BhG7hcpEdFquA372ZldllelUdTqc.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.414Z: Staged package netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-common-4.1.52.Final-DR8QHk231TDtBNfKvVfZXAdQ8Fm7az-t_tDIDUc0xxc.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.452Z: Staged package netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-4.1.52.Final-LNwh-_rgSUDIWSkDl3nJP1H2GPXvshYHkxnc_PMqVcE.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.514Z: Staged package netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-handler-proxy-4.1.52.Final-6NB4Y860vUgG4lWh1TLIZQ8POy1NAGrW0hgS6UG0L9w.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.600Z: Staged package netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-resolver-4.1.52.Final-butyojOXnLRbC01tRsWkEzLFfAz4qAOx78YNu-y6CaY.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.667Z: Staged package netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/netty-transport-4.1.52.Final-mj5vjA5V3jY-seoQ_ngXl-yjlOYhht8q4LTrK84LRUE.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.715Z: Staged package opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/opencensus-api-0.28.0-DBcj8_bTBhMjhFzouIs1_dpQCBLgp1uOtfzErYyHGpU.jar' is inaccessible. Aug 03, 2021 2:25:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.785Z: Staged package paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/paranamer-2.8-aIyxGKYCHYGROOhVIIyVYDFoi-S0eiS7YVvsxjrO3wc.jar' is inaccessible. Aug 03, 2021 2:25:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:15.958Z: Staged package perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/perfmark-api-0.23.0-xwW1wQwY_zAyuegXQrwvaw5WB_am38DIrQz_ddSRMEI.jar' is inaccessible. Aug 03, 2021 2:25:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2021-08-03T14:25:16.407Z: Staged package snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar at location 'gs://temp-storage-for-perf-tests/loadtests/staging/snappy-java-1.1.8.4-JMTR_B6J4HgzGrj0AamcrWhZm95KLkUWBCy1SMUbHD4.jar' is inaccessible. Aug 03, 2021 2:25:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2021-08-03T14:25:16.604Z: One or more access checks for temp location or staged files failed. Please refer to other error messages for details. For more information on security and permissions, please see https://cloud.google.com/dataflow/security-and-permissions. Aug 03, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:46.942Z: Cancel request is committed for workflow job: 2021-08-03_05_42_56-9271568491856649635. Aug 03, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:46.989Z: Cleaning up. Aug 03, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:47.068Z: Stopping **** pool... Aug 03, 2021 4:00:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:00:47.145Z: Stopping **** pool... Aug 03, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:03:03.576Z: Autoscaling: Reduced the number of ****s to 0 based on low average **** CPU utilization, and the pipeline having sufficiently low backlog and keeping up with input rate. Aug 03, 2021 4:03:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2021-08-03T16:03:03.659Z: Worker pool stopped. Aug 03, 2021 4:03:17 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2021-08-03_05_42_56-9271568491856649635 finished with status CANCELLED. Aug 03, 2021 4:03:17 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric totalBytes.count, from namespace pardo Load test results for test (ID): 403e8b6c-7e6e-4539-b687-f2b1c9711d0f and timestamp: 2021-08-03T12:42:50.889000000Z: Metric: Value: dataflow_v2_java11_runtime_sec 0.0 dataflow_v2_java11_total_bytes_count -1.0 Exception in thread "main" java.lang.RuntimeException: Invalid job state: CANCELLED. at org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:55) at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:141) at org.apache.beam.sdk.loadtests.ParDoLoadTest.run(ParDoLoadTest.java:53) at org.apache.beam.sdk.loadtests.ParDoLoadTest.main(ParDoLoadTest.java:103) > Task :sdks:java:testing:load-tests:run FAILED > Task :runners:google-cloud-dataflow-java:cleanUpDockerImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803123734 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa Deleted: sha256:b0398a4e7feea97e64d6d0294547164a31fa71702ea084d81d0aa1d4bce76656 Deleted: sha256:db2c3cd146bec7ac2aceca853a50117a2582ffc4d18653a696e3071fee26a54a Deleted: sha256:ac0034fc58af4081b6f1adb637db7ceb14ac7973f1afa4d545b169da7e8a0564 Deleted: sha256:65fe46f21d03771bcba7effe4da30548e7bf024fd85ff8874678be8f2fe9ae43 Deleted: sha256:0db6193cc2d36a148fede854222a929a0ef430a5f47d75f082bdedea840043ff Deleted: sha256:3424ddbe59f6aca33f25ac5b9ffa4ebe9e1c73c185be904548a3d1af021d6a5c Deleted: sha256:2fa314d38ff7cff250752877cbdb90e3d168ccbe5fae79d74ffc2d8db205e609 Deleted: sha256:fdc9c3105b00cf9cb9331a6cedd2fefe32e389189281b6ad743cd1bcc24931e4 Deleted: sha256:0ba037eea5b6dd4ea063ff4c50c6abd5899824725f5191e5bd49924b3a130fc6 Deleted: sha256:96f34b190a26739879492f3b42ec95c4b5a6f0a87551a0257d55abdd6ce379b4 Deleted: sha256:c677e1d674418b371aee35ca8f5cce9ad5219fe1af9555f050ccbdb59f332131 Deleted: sha256:4e0f6384fd82f09288927431d13db62af6cefe98a90ff1d0ff20c8b0f7ec165e Deleted: sha256:241c424ed2d745d072d4518a328cd7c93ed71bae7f091d1120296e8fcbc5f3b0 Deleted: sha256:8cfc2acf0f75f71ba9649a6dd5afea27e8f5241335e926ecd04b1ce77089abcf Deleted: sha256:78157cf2f84e688391935a290b9342c6481a8c63651f47854e1a151e2562ccb7 Deleted: sha256:6ae4c35535adec13f73d90c82acbc32798e1d6068b9d136ca7f2759c8544677b Deleted: sha256:2a451a2015a919e496864101bcea2cb3fe929d0cd9cb017c9658000a3b888f4b Deleted: sha256:f43c1b00839ed5d56596666e2cfe02652d0c21a2c43f5b88635495052e4bd5b5 Deleted: sha256:406b162fde37214d04f4bb7d2636320751932ddd1a4904ed6dab78d2fd22f2f8 Deleted: sha256:21fc2491b17c76e82875ed96d69f4dd3b62a8e022d0eb56d372505427618e448 Deleted: sha256:ead01c0e2db0a1a0818535430dd61dd68a0a50bab89c911c30830a5fa23cfda3 Deleted: sha256:b59bdaa47d9aa46b9c03d7c06335b1ee1e7a3498e8536ec00c3ba0066977460d Deleted: sha256:00eb569b6dc99efd822d0c376ff5119907d76bff130d643cfb3f713c9d48de43 Deleted: sha256:5a8a18a6b12dab602733c144ea44cf852fb8547ef1e052bbe0c880ec821ad37b Deleted: sha256:6b825d27c252a380952867a16962ef3047de30fb9278ce4c81a66bed507a62af Deleted: sha256:9621407ad6da5d085c9d1ee765b1c2604306bba7fa1cef5a00bbc340852fa255 Deleted: sha256:97b9d50b17fcc474037f0f5b804b15a762c16916349b0edb3a9c13327a1f0f07 Deleted: sha256:df1d71e9a34939843f3d742a077a649d5e4d5aca16b025c1f47548130f2b803d Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803123734] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210803123734] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:32136a2c3d28115a489e2f5bab48378c7e7539ef21b403dacc55507be21ae8fa]. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:load-tests:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 3h 26m 1s 107 actionable tasks: 79 executed, 26 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/vk56z26ymfv2g Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
