See
<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2144/display/redirect>
Changes:
------------------------------------------
[...truncated 314.09 KB...]
INFO: Staging files complete: 227 files cached, 0 files newly uploaded in 0
seconds
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource) as step s1
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/Split as
step s2
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key as step s3
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
as step s4
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey as step s5
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable as step s6
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map as step s7
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/Read as
step s8
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/StripIds as
step s9
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Measure read time as step s10
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Map records to strings/Map as step s11
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s12
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s13
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step
s16
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s18
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as
step s20
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 06, 2021 12:39:00 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 06, 2021 12:39:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.30.0-SNAPSHOT
Apr 06, 2021 12:39:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2021-04-06_05_39_00-10521982016293875080?project=apache-beam-testing
Apr 06, 2021 12:39:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-04-06_05_39_00-10521982016293875080
Apr 06, 2021 12:39:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2021-04-06_05_39_00-10521982016293875080
Apr 06, 2021 12:39:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-04-06T12:39:06.096Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-oasr. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.031Z: Worker configuration: n1-standard-1 in
us-central1-b.
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.638Z: Expanding CoGroupByKey operations into
optimizable parts.
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.737Z: Expanding GroupByKey operations into
optimizable parts.
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.762Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.890Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.925Z: Elided trivial flatten
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.953Z: Unzipping flatten s19 for input
s17.org.apache.beam.sdk.values.PCollection.<init>:402#e63ad18dfc139ac0
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.970Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:15.993Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.018Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.043Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.062Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.083Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.109Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.132Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 06, 2021 12:39:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.151Z: Unzipping flatten s19-u40 for input
s21.org.apache.beam.sdk.values.PCollection.<init>:402#24aa17775bf242ac-c38
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.176Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.223Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.248Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from
bounded Kafka/Read(KafkaUnboundedSource)/Split
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.269Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
into Read from bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with
random key
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.295Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.313Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.340Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.368Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.390Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.418Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Read into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.441Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/StripIds into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Read
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.461Z: Fusing consumer Measure read time into Read
from bounded Kafka/Read(KafkaUnboundedSource)/StripIds
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.486Z: Fusing consumer Map records to strings/Map
into Measure read time
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.508Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.534Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.560Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.631Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.656Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.680Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.705Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.727Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/Values/Values/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.744Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.770Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:16.786Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:17.092Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:17.162Z: Starting 5 ****s in us-central1-b...
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:17.766Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 06, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:17.944Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 06, 2021 12:39:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:33.958Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 06, 2021 12:39:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:48.677Z: Autoscaling: Raised the number of ****s to
2 based on the rate of progress in the currently running stage(s).
Apr 06, 2021 12:39:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:48.707Z: Resized **** pool to 2, though goal was 5.
This could be a quota issue.
Apr 06, 2021 12:40:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:39:59.049Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Apr 06, 2021 12:40:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:22.662Z: Workers have started successfully.
Apr 06, 2021 12:40:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:22.687Z: Workers have started successfully.
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:39.965Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:40.036Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:40.097Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:40.158Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:40.344Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 06, 2021 12:40:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-06T12:40:40.502Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Read+Read from bounded
Kafka/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to
strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
org.apache.beam.sdk.io.kafka.KafkaIOIT >
testKafkaIOReadsAndWritesCorrectlyInBatch SKIPPED
Watching 2490 directories to track changes
Watching 2492 directories to track changes
Watching 2493 directories to track changes
> Task :sdks:java:io:kafka:integrationTest FAILED
:sdks:java:io:kafka:integrationTest (Thread[Daemon ****,5,main]) completed.
Took 12 mins 47.034 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> Process 'Gradle Test Executor 1' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/6.8/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 23s
107 actionable tasks: 64 executed, 43 from cache
Watching 2493 directories to track changes
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=bde89daf-8933-4b09-bd20-9ff84eb6d7c5,
currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 17819
log file: /home/jenkins/.gradle/daemon/6.8/daemon-17819.out.log
----- Last 20 lines from daemon log file - daemon-17819.out.log -----
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> Process 'Gradle Test Executor 1' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/6.8/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 23s
107 actionable tasks: 64 executed, 43 from cache
Watching 2493 directories to track changes
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
