See
<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/665/display/redirect>
Changes:
------------------------------------------
[...truncated 117.65 KB...]
The push refers to repository
[us.gcr.io/apache-beam-testing/java-postcommit-it/java]
2f24d7efeed3: Preparing
5f70bf18a086: Preparing
fa7e6ce95c6c: Preparing
3595d31b53e1: Preparing
f2539a69d700: Preparing
2dda36c1f10c: Preparing
b47288a945fa: Preparing
82cb217eb369: Preparing
4ca6aa132d2b: Preparing
50e5147fa15f: Preparing
9cadb75d3932: Preparing
de51662a448f: Preparing
0547901d532e: Preparing
eb454feec9c6: Preparing
9e49a10d57bc: Preparing
12438cbb292d: Preparing
baacb5f860f6: Preparing
492642b4821b: Preparing
fb91a4b6eff7: Preparing
256d88da4185: Preparing
2dda36c1f10c: Waiting
0547901d532e: Waiting
baacb5f860f6: Waiting
fb91a4b6eff7: Waiting
eb454feec9c6: Waiting
256d88da4185: Waiting
b47288a945fa: Waiting
492642b4821b: Waiting
9e49a10d57bc: Waiting
82cb217eb369: Waiting
4ca6aa132d2b: Waiting
50e5147fa15f: Waiting
9cadb75d3932: Waiting
5f70bf18a086: Layer already exists
fa7e6ce95c6c: Pushed
3595d31b53e1: Pushed
2dda36c1f10c: Pushed
2f24d7efeed3: Pushed
f2539a69d700: Pushed
b47288a945fa: Pushed
4ca6aa132d2b: Pushed
9cadb75d3932: Pushed
50e5147fa15f: Pushed
82cb217eb369: Pushed
de51662a448f: Pushed
12438cbb292d: Layer already exists
baacb5f860f6: Layer already exists
492642b4821b: Layer already exists
256d88da4185: Layer already exists
fb91a4b6eff7: Layer already exists
0547901d532e: Pushed
eb454feec9c6: Pushed
9e49a10d57bc: Pushed
20231018223548: digest:
sha256:9566ba57d6c4c56e94bdc81a904de2b28a3f71a9bd86ef8017258ef12293d533 size:
4505
> Task :sdks:java:testing:load-tests:run
Oct 18, 2023 10:36:10 PM
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Oct 18, 2023 10:36:10 PM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from
the classpath: will stage 231 files. Enable logging at DEBUG level to see which
files will be staged.
Oct 18, 2023 10:36:11 PM org.apache.beam.sdk.Pipeline validate
WARNING: The following transforms do not have stable unique names: Window.Into()
Oct 18, 2023 10:36:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Oct 18, 2023 10:36:13 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 231 files from PipelineOptions.filesToStage to staging location
to prepare for execution.
Oct 18, 2023 10:36:14 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 231 files cached, 0 files newly uploaded in 0
seconds
Oct 18, 2023 10:36:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to
gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 18, 2023 10:36:14 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <123816 bytes, hash
21c2252675f7adbe96c5b1a26b9867143970588873f5eee4e7f4556e25141da4> to
gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-IcIlJnX3rb6WxbGia5hnFDlwWIhz9e7k5_RVbiUUHaQ.pb
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as
step s1
Oct 18, 2023 10:36:16 PM
org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles: [SyntheticUnboundedSource{startOffset=0,
endOffset=1000000}, SyntheticUnboundedSource{startOffset=1000000,
endOffset=2000000}, SyntheticUnboundedSource{startOffset=2000000,
endOffset=3000000}, SyntheticUnboundedSource{startOffset=3000000,
endOffset=4000000}, SyntheticUnboundedSource{startOffset=4000000,
endOffset=5000000}, SyntheticUnboundedSource{startOffset=5000000,
endOffset=6000000}, SyntheticUnboundedSource{startOffset=6000000,
endOffset=7000000}, SyntheticUnboundedSource{startOffset=7000000,
endOffset=8000000}, SyntheticUnboundedSource{startOffset=8000000,
endOffset=9000000}, SyntheticUnboundedSource{startOffset=9000000,
endOffset=10000000}, SyntheticUnboundedSource{startOffset=10000000,
endOffset=11000000}, SyntheticUnboundedSource{startOffset=11000000,
endOffset=12000000}, SyntheticUnboundedSource{startOffset=12000000,
endOffset=13000000}, SyntheticUnboundedSource{startOffset=13000000,
endOffset=14000000}, SyntheticUnboundedSource{startOffset=14000000,
endOffset=15000000}, SyntheticUnboundedSource{startOffset=15000000,
endOffset=16000000}, SyntheticUnboundedSource{startOffset=16000000,
endOffset=17000000}, SyntheticUnboundedSource{startOffset=17000000,
endOffset=18000000}, SyntheticUnboundedSource{startOffset=18000000,
endOffset=19000000}, SyntheticUnboundedSource{startOffset=19000000,
endOffset=20000000}]
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (input) as step s3
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s4
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as
step s5
Oct 18, 2023 10:36:16 PM
org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 20 bundles: [SyntheticUnboundedSource{startOffset=0,
endOffset=100000}, SyntheticUnboundedSource{startOffset=100000,
endOffset=200000}, SyntheticUnboundedSource{startOffset=200000,
endOffset=300000}, SyntheticUnboundedSource{startOffset=300000,
endOffset=400000}, SyntheticUnboundedSource{startOffset=400000,
endOffset=500000}, SyntheticUnboundedSource{startOffset=500000,
endOffset=600000}, SyntheticUnboundedSource{startOffset=600000,
endOffset=700000}, SyntheticUnboundedSource{startOffset=700000,
endOffset=800000}, SyntheticUnboundedSource{startOffset=800000,
endOffset=900000}, SyntheticUnboundedSource{startOffset=900000,
endOffset=1000000}, SyntheticUnboundedSource{startOffset=1000000,
endOffset=1100000}, SyntheticUnboundedSource{startOffset=1100000,
endOffset=1200000}, SyntheticUnboundedSource{startOffset=1200000,
endOffset=1300000}, SyntheticUnboundedSource{startOffset=1300000,
endOffset=1400000}, SyntheticUnboundedSource{startOffset=1400000,
endOffset=1500000}, SyntheticUnboundedSource{startOffset=1500000,
endOffset=1600000}, SyntheticUnboundedSource{startOffset=1600000,
endOffset=1700000}, SyntheticUnboundedSource{startOffset=1700000,
endOffset=1800000}, SyntheticUnboundedSource{startOffset=1800000,
endOffset=1900000}, SyntheticUnboundedSource{startOffset=1900000,
endOffset=2000000}]
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read co-input/StripIds as step s6
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics (co-input) as step s7
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()2/Window.Assign as step s8
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable0 as step s9
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/MakeUnionTable1 as step s10
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/Flatten as step s11
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/GBK as step s12
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CoGroupByKey/ConstructCoGbkResultFn as step s13
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate as step s14
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect total bytes as step s15
Oct 18, 2023 10:36:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics as step s16
Oct 18, 2023 10:36:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.52.0-SNAPSHOT
Oct 18, 2023 10:36:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2023-10-18_15_36_16-6572009270575700027?project=apache-beam-testing
Oct 18, 2023 10:36:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-10-18_15_36_16-6572009270575700027
Oct 18, 2023 10:36:17 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
> --region=us-central1 2023-10-18_15_36_16-6572009270575700027
Oct 18, 2023 10:36:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-10-18T22:36:22.595Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
load0tests0java170dataflow0v20streaming0cogbk02-jenkins-10-cagq. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 18, 2023 10:36:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:31.172Z: Worker configuration: e2-standard-2 in
us-central1-b.
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.771Z: Expanding SplittableParDo operations into
optimizable parts.
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.801Z: Expanding CollectionToSingleton operations into
optimizable parts.
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.849Z: Expanding CoGroupByKey operations into
optimizable parts.
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.909Z: Expanding SplittableProcessKeyed operations
into optimizable parts.
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.930Z: Expanding GroupByKey operations into streaming
Read/Write steps
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:32.980Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.063Z: Fusing adjacent ParDo, Read, Write, and Flatten
operations
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.095Z: Unzipping flatten CoGroupByKey-Flatten for
input CoGroupByKey-MakeUnionTable0-ParMultiDo-ConstructUnionTable-.output
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.119Z: Fusing unzipped copy of
CoGroupByKey/GBK/WriteStream, through flatten CoGroupByKey/Flatten, into
producer CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.142Z: Fusing consumer CoGroupByKey/GBK/WriteStream
into CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.167Z: Fusing consumer Read
input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read
input/Impulse
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.192Z: Fusing consumer
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.213Z: Fusing consumer
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing
into
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.239Z: Fusing consumer Read
input/ParDo(StripIds)/ParMultiDo(StripIds) into
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.259Z: Fusing consumer Collect start time metrics
(input)/ParMultiDo(TimeMonitor) into Read
input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.278Z: Fusing consumer Window.Into()/Window.Assign
into Collect start time metrics (input)/ParMultiDo(TimeMonitor)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.301Z: Fusing consumer
CoGroupByKey/MakeUnionTable0/ParMultiDo(ConstructUnionTable) into
Window.Into()/Window.Assign
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.329Z: Fusing consumer Read
co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read
co-input/Impulse
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.356Z: Fusing consumer
Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
into Read co-input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.376Z: Fusing consumer
Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing
into
Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.403Z: Fusing consumer Read
co-input/ParDo(StripIds)/ParMultiDo(StripIds) into
Read-co-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.427Z: Fusing consumer Collect start time metrics
(co-input)/ParMultiDo(TimeMonitor) into Read
co-input/ParDo(StripIds)/ParMultiDo(StripIds)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.492Z: Fusing consumer Window.Into()2/Window.Assign
into Collect start time metrics (co-input)/ParMultiDo(TimeMonitor)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.516Z: Fusing consumer
CoGroupByKey/MakeUnionTable1/ParMultiDo(ConstructUnionTable) into
Window.Into()2/Window.Assign
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.540Z: Fusing consumer CoGroupByKey/GBK/MergeBuckets
into CoGroupByKey/GBK/ReadStream
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.560Z: Fusing consumer
CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult) into
CoGroupByKey/GBK/MergeBuckets
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.584Z: Fusing consumer Ungroup and
reiterate/ParMultiDo(UngroupAndReiterate) into
CoGroupByKey/ConstructCoGbkResultFn/ParMultiDo(ConstructCoGbkResult)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.607Z: Fusing consumer Collect total
bytes/ParMultiDo(ByteMonitor) into Ungroup and
reiterate/ParMultiDo(UngroupAndReiterate)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.628Z: Fusing consumer Collect end time
metrics/ParMultiDo(TimeMonitor) into Collect total bytes/ParMultiDo(ByteMonitor)
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:33.797Z: Running job using Streaming Engine
Oct 18, 2023 10:36:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:34.113Z: Starting 5 ****s in us-central1-b...
Oct 18, 2023 10:36:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:36:56.295Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 18, 2023 10:37:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:37:29.154Z: Autoscaling: Raised the number of ****s to 4 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 18, 2023 10:37:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:37:29.176Z: Autoscaling: Resized **** pool to 4, though
goal was 5. This could be a quota issue.
Oct 18, 2023 10:37:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:37:38.591Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Oct 18, 2023 10:38:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:38:15.399Z: Workers have started successfully.
Oct 18, 2023 10:38:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T22:38:15.657Z: All ****s have finished the startup processes
and began to receive work requests.
Oct 18, 2023 11:11:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T23:11:40.956Z: Cleaning up.
Oct 18, 2023 11:11:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T23:11:41.102Z: Stopping **** pool...
Oct 18, 2023 11:11:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T23:11:41.153Z: Stopping **** pool...
Oct 18, 2023 11:14:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T23:14:03.759Z: Autoscaling: Reduced the number of ****s to 0
based on low average **** CPU utilization, and the pipeline having sufficiently
low backlog and keeping up with input rate.
Oct 18, 2023 11:14:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-18T23:14:03.797Z: Worker pool stopped.
Oct 18, 2023 11:14:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
logTerminalState
INFO: Job 2023-10-18_15_36_16-6572009270575700027 finished with status DONE.
Load test results for test (ID): 74e9669d-e230-4950-b34b-909b7628440f and
timestamp: 2023-10-18T22:36:11.098000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 1911.049
dataflow_v2_java17_total_bytes_count 2.199996E9
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20231018223548
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9566ba57d6c4c56e94bdc81a904de2b28a3f71a9bd86ef8017258ef12293d533
ERROR: (gcloud.container.images.untag) Image could not be found:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20231018223548]
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'>
line: 308
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
For more on this, please refer to
https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings
in the Gradle documentation.
BUILD FAILED in 39m 9s
110 actionable tasks: 8 executed, 102 up-to-date
Publishing build scan...
https://ge.apache.org/s/u6ff7l5aqxjvm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
