See
<https://ci-beam.apache.org/job/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java17/267/display/redirect?page=changes>
Changes:
[noreply] [Playground] [Backend] Datastore queries and mappers to get
precompiled
[noreply] Open Allow and test pyarrow 8.x and 9.x (#22997)
[noreply] (BQ Python) Pass project field from options or parameter when writing
[noreply] Update python-machine-learning.md (#23209)
[noreply] Pin the version of cloudpickle to 2.1.x (#23120)
[noreply] Add streaming test for Write API sink (#21903)
[noreply] [Go SDK] Proto changes for timer param (#23216)
[noreply] Bump github.com/testcontainers/testcontainers-go in /sdks (#23201)
[noreply] Update to objsize to 0.5.2 which is under BSD-3 license (fixes #23096)
[noreply] Exclude insignificant whitespace from cloud object (#23217)
[noreply] Trying out property-based tests for Beam python coders (#22233)
[noreply] Publish results of JMH benchmark runs (Java SDK) to InfluxDB (part of
------------------------------------------
[...truncated 288.61 KB...]
a1a8a6946706: Pushed
2bf186b06aee: Pushed
3bc383470c05: Layer already exists
e93827457889: Layer already exists
08fa02ce37eb: Layer already exists
6d7ad32daa5e: Pushed
a037458de4e0: Layer already exists
bafdbe68e4ae: Layer already exists
a13c519c6361: Layer already exists
d8a7160c23c2: Pushed
7846bf1fa23e: Pushed
a7aa5f0bec82: Pushed
20220914143904: digest:
sha256:00f01eda5c938530769d6d7d397a531dfcce1f92c82bfe5c8de339cc7e2dc8f7 size:
4729
> Task :sdks:java:testing:load-tests:run
Sep 14, 2022 2:39:38 PM
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Sep 14, 2022 2:39:39 PM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from
the classpath: will stage 227 files. Enable logging at DEBUG level to see which
files will be staged.
Sep 14, 2022 2:39:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Sep 14, 2022 2:39:42 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 227 files from PipelineOptions.filesToStage to staging location
to prepare for execution.
Sep 14, 2022 2:39:42 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 227 files cached, 0 files newly uploaded in 0
seconds
Sep 14, 2022 2:39:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging portable pipeline proto to
gs://temp-storage-for-perf-tests/loadtests/staging/
Sep 14, 2022 2:39:42 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <132211 bytes, hash
43d9ea7be1ee34d1eb35c4d6c7c632b4da13e11424b61ebb2a9534826d3e3068> to
gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-Q9nqe-HuNNHrNcTWx8YytNoT4RQkth67KpU0gm0-MGg.pb
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as
step s1
Sep 14, 2022 2:39:44 PM
org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split
INFO: Split into 64 bundles: [SyntheticUnboundedSource{startOffset=0,
endOffset=39062}, SyntheticUnboundedSource{startOffset=39062, endOffset=78125},
SyntheticUnboundedSource{startOffset=78125, endOffset=117187},
SyntheticUnboundedSource{startOffset=117187, endOffset=156250},
SyntheticUnboundedSource{startOffset=156250, endOffset=195312},
SyntheticUnboundedSource{startOffset=195312, endOffset=234375},
SyntheticUnboundedSource{startOffset=234375, endOffset=273437},
SyntheticUnboundedSource{startOffset=273437, endOffset=312500},
SyntheticUnboundedSource{startOffset=312500, endOffset=351562},
SyntheticUnboundedSource{startOffset=351562, endOffset=390625},
SyntheticUnboundedSource{startOffset=390625, endOffset=429687},
SyntheticUnboundedSource{startOffset=429687, endOffset=468750},
SyntheticUnboundedSource{startOffset=468750, endOffset=507812},
SyntheticUnboundedSource{startOffset=507812, endOffset=546875},
SyntheticUnboundedSource{startOffset=546875, endOffset=585937},
SyntheticUnboundedSource{startOffset=585937, endOffset=625000},
SyntheticUnboundedSource{startOffset=625000, endOffset=664062},
SyntheticUnboundedSource{startOffset=664062, endOffset=703125},
SyntheticUnboundedSource{startOffset=703125, endOffset=742187},
SyntheticUnboundedSource{startOffset=742187, endOffset=781250},
SyntheticUnboundedSource{startOffset=781250, endOffset=820312},
SyntheticUnboundedSource{startOffset=820312, endOffset=859375},
SyntheticUnboundedSource{startOffset=859375, endOffset=898437},
SyntheticUnboundedSource{startOffset=898437, endOffset=937500},
SyntheticUnboundedSource{startOffset=937500, endOffset=976562},
SyntheticUnboundedSource{startOffset=976562, endOffset=1015625},
SyntheticUnboundedSource{startOffset=1015625, endOffset=1054687},
SyntheticUnboundedSource{startOffset=1054687, endOffset=1093750},
SyntheticUnboundedSource{startOffset=1093750, endOffset=1132812},
SyntheticUnboundedSource{startOffset=1132812, endOffset=1171875},
SyntheticUnboundedSource{startOffset=1171875, endOffset=1210937},
SyntheticUnboundedSource{startOffset=1210937, endOffset=1250000},
SyntheticUnboundedSource{startOffset=1250000, endOffset=1289062},
SyntheticUnboundedSource{startOffset=1289062, endOffset=1328125},
SyntheticUnboundedSource{startOffset=1328125, endOffset=1367187},
SyntheticUnboundedSource{startOffset=1367187, endOffset=1406250},
SyntheticUnboundedSource{startOffset=1406250, endOffset=1445312},
SyntheticUnboundedSource{startOffset=1445312, endOffset=1484375},
SyntheticUnboundedSource{startOffset=1484375, endOffset=1523437},
SyntheticUnboundedSource{startOffset=1523437, endOffset=1562500},
SyntheticUnboundedSource{startOffset=1562500, endOffset=1601562},
SyntheticUnboundedSource{startOffset=1601562, endOffset=1640625},
SyntheticUnboundedSource{startOffset=1640625, endOffset=1679687},
SyntheticUnboundedSource{startOffset=1679687, endOffset=1718750},
SyntheticUnboundedSource{startOffset=1718750, endOffset=1757812},
SyntheticUnboundedSource{startOffset=1757812, endOffset=1796875},
SyntheticUnboundedSource{startOffset=1796875, endOffset=1835937},
SyntheticUnboundedSource{startOffset=1835937, endOffset=1875000},
SyntheticUnboundedSource{startOffset=1875000, endOffset=1914062},
SyntheticUnboundedSource{startOffset=1914062, endOffset=1953125},
SyntheticUnboundedSource{startOffset=1953125, endOffset=1992187},
SyntheticUnboundedSource{startOffset=1992187, endOffset=2031250},
SyntheticUnboundedSource{startOffset=2031250, endOffset=2070312},
SyntheticUnboundedSource{startOffset=2070312, endOffset=2109375},
SyntheticUnboundedSource{startOffset=2109375, endOffset=2148437},
SyntheticUnboundedSource{startOffset=2148437, endOffset=2187500},
SyntheticUnboundedSource{startOffset=2187500, endOffset=2226562},
SyntheticUnboundedSource{startOffset=2226562, endOffset=2265625},
SyntheticUnboundedSource{startOffset=2265625, endOffset=2304687},
SyntheticUnboundedSource{startOffset=2304687, endOffset=2343750},
SyntheticUnboundedSource{startOffset=2343750, endOffset=2382812},
SyntheticUnboundedSource{startOffset=2382812, endOffset=2421875},
SyntheticUnboundedSource{startOffset=2421875, endOffset=2460937},
SyntheticUnboundedSource{startOffset=2460937, endOffset=2500000}]
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read input/StripIds as step s2
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect start time metrics as step s3
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Total bytes monitor as step s4
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Window.Into()/Window.Assign as step s5
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (0) as step s6
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (0) as step s7
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (0) as step s8
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (1) as step s9
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (1) as step s10
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (1) as step s11
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (2) as step s12
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (2) as step s13
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (2) as step s14
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (3) as step s15
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (3) as step s16
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (3) as step s17
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (4) as step s18
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (4) as step s19
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (4) as step s20
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (5) as step s21
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (5) as step s22
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (5) as step s23
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (6) as step s24
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (6) as step s25
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (6) as step s26
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Group by key (7) as step s27
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Ungroup and reiterate (7) as step s28
Sep 14, 2022 2:39:44 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect end time metrics (7) as step s29
Sep 14, 2022 2:39:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.43.0-SNAPSHOT
Sep 14, 2022 2:39:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-09-14_07_39_44-11043569444337819787?project=apache-beam-testing
Sep 14, 2022 2:39:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-09-14_07_39_44-11043569444337819787
Sep 14, 2022 2:39:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
> --region=us-central1 2022-09-14_07_39_44-11043569444337819787
Sep 14, 2022 2:39:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-09-14T14:39:50.494Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
load0tests0java170dataflow0v20streaming0gbk05-jenkins-0914-ijnb. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 14, 2022 2:39:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:56.344Z: Worker configuration: e2-standard-2 in
us-central1-b.
Sep 14, 2022 2:39:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:57.541Z: Expanding SplittableParDo operations into
optimizable parts.
Sep 14, 2022 2:39:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:57.576Z: Expanding CollectionToSingleton operations into
optimizable parts.
Sep 14, 2022 2:39:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:57.649Z: Expanding CoGroupByKey operations into
optimizable parts.
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:57.968Z: Expanding SplittableProcessKeyed operations
into optimizable parts.
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:57.999Z: Expanding GroupByKey operations into streaming
Read/Write steps
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.300Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.688Z: Fusing adjacent ParDo, Read, Write, and Flatten
operations
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.717Z: Fusing consumer Group by key (0)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.740Z: Fusing consumer Group by key (1)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.805Z: Fusing consumer Group by key (2)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.839Z: Fusing consumer Group by key (3)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.875Z: Fusing consumer Group by key (4)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.904Z: Fusing consumer Group by key (5)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.934Z: Fusing consumer Group by key (6)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.969Z: Fusing consumer Group by key (7)/WriteStream
into Window.Into()/Window.Assign
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:58.998Z: Fusing consumer Read
input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource) into Read
input/Impulse
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.028Z: Fusing consumer
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
into Read input/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.062Z: Fusing consumer
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/SplitWithSizing
into
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/PairWithRestriction
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.096Z: Fusing consumer Read
input/ParDo(StripIds)/ParMultiDo(StripIds) into
Read-input-ParDo-UnboundedSourceAsSDFWrapper--ParMultiDo-UnboundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.126Z: Fusing consumer Collect start time
metrics/ParMultiDo(TimeMonitor) into Read
input/ParDo(StripIds)/ParMultiDo(StripIds)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.166Z: Fusing consumer Total bytes
monitor/ParMultiDo(ByteMonitor) into Collect start time
metrics/ParMultiDo(TimeMonitor)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.231Z: Fusing consumer Window.Into()/Window.Assign
into Total bytes monitor/ParMultiDo(ByteMonitor)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.262Z: Fusing consumer Group by key (0)/MergeBuckets
into Group by key (0)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.296Z: Fusing consumer Ungroup and reiterate
(0)/ParMultiDo(UngroupAndReiterate) into Group by key (0)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.329Z: Fusing consumer Collect end time metrics
(0)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(0)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.378Z: Fusing consumer Group by key (1)/MergeBuckets
into Group by key (1)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.416Z: Fusing consumer Ungroup and reiterate
(1)/ParMultiDo(UngroupAndReiterate) into Group by key (1)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.454Z: Fusing consumer Collect end time metrics
(1)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(1)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.516Z: Fusing consumer Group by key (2)/MergeBuckets
into Group by key (2)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.556Z: Fusing consumer Ungroup and reiterate
(2)/ParMultiDo(UngroupAndReiterate) into Group by key (2)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.592Z: Fusing consumer Collect end time metrics
(2)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(2)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.628Z: Fusing consumer Group by key (3)/MergeBuckets
into Group by key (3)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.661Z: Fusing consumer Ungroup and reiterate
(3)/ParMultiDo(UngroupAndReiterate) into Group by key (3)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.690Z: Fusing consumer Collect end time metrics
(3)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(3)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.719Z: Fusing consumer Group by key (4)/MergeBuckets
into Group by key (4)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.756Z: Fusing consumer Ungroup and reiterate
(4)/ParMultiDo(UngroupAndReiterate) into Group by key (4)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.818Z: Fusing consumer Collect end time metrics
(4)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(4)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.846Z: Fusing consumer Group by key (5)/MergeBuckets
into Group by key (5)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.876Z: Fusing consumer Ungroup and reiterate
(5)/ParMultiDo(UngroupAndReiterate) into Group by key (5)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.899Z: Fusing consumer Collect end time metrics
(5)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(5)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.937Z: Fusing consumer Group by key (6)/MergeBuckets
into Group by key (6)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:39:59.977Z: Fusing consumer Ungroup and reiterate
(6)/ParMultiDo(UngroupAndReiterate) into Group by key (6)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.006Z: Fusing consumer Collect end time metrics
(6)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(6)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.043Z: Fusing consumer Group by key (7)/MergeBuckets
into Group by key (7)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.077Z: Fusing consumer Ungroup and reiterate
(7)/ParMultiDo(UngroupAndReiterate) into Group by key (7)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.103Z: Fusing consumer Collect end time metrics
(7)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(7)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.245Z: Running job using Streaming Engine
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.508Z: Starting 16 ****s in us-central1-b...
Sep 14, 2022 2:40:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:04.494Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 14, 2022 2:40:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:42.540Z: Autoscaling: Raised the number of ****s to 16
so that the pipeline can catch up with its backlog and keep up with its input
rate.
Sep 14, 2022 2:41:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:41:32.359Z: Workers have started successfully.
> Task :sdks:java:testing:load-tests:run FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=cf819196-9e15-4110-a9dc-7b09d55675bb,
currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java17/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 3076567
log file: /home/jenkins/.gradle/daemon/7.5.1/daemon-3076567.out.log
----- Last 20 lines from daemon log file - daemon-3076567.out.log -----
INFO: 2022-09-14T14:39:59.977Z: Fusing consumer Ungroup and reiterate
(6)/ParMultiDo(UngroupAndReiterate) into Group by key (6)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.006Z: Fusing consumer Collect end time metrics
(6)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(6)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.043Z: Fusing consumer Group by key (7)/MergeBuckets
into Group by key (7)/ReadStream
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.077Z: Fusing consumer Ungroup and reiterate
(7)/ParMultiDo(UngroupAndReiterate) into Group by key (7)/MergeBuckets
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.103Z: Fusing consumer Collect end time metrics
(7)/ParMultiDo(TimeMonitor) into Ungroup and reiterate
(7)/ParMultiDo(UngroupAndReiterate)
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.245Z: Running job using Streaming Engine
Sep 14, 2022 2:40:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:00.508Z: Starting 16 ****s in us-central1-b...
Sep 14, 2022 2:40:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:04.494Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 14, 2022 2:40:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:40:42.540Z: Autoscaling: Raised the number of ****s to 16
so that the pipeline can catch up with its backlog and keep up with its input
rate.
Sep 14, 2022 2:41:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-09-14T14:41:32.359Z: Workers have started successfully.
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
