See
<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2152/display/redirect>
Changes:
------------------------------------------
[...truncated 311.31 KB...]
INFO: Uploading <187629 bytes, hash
a8a80be94f67235d1e5491f40d43d5f75576dfaac33f3be498b1c0af98d49840> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-qKgL6U9nI10eVJH0DUPV91V236rDPzvkmLHAr5jUmEA.pb
Apr 08, 2021 12:37:34 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 227 files from PipelineOptions.filesToStage to staging
location to prepare for execution.
Apr 08, 2021 12:37:34 PM
org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes
forFileToStage
INFO: Staging custom dataflow-****.jar as
beam-runners-google-cloud-dataflow-java-legacy-****-2.30.0-SNAPSHOT-r1Q-U6K1C7rXgbO5tW1dJecV9hejJnRIdqvIGLE-UrI.jar
Apr 08, 2021 12:37:35 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 227 files cached, 0 files newly uploaded in 0
seconds
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource) as step s1
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/Split as
step s2
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key as step s3
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
as step s4
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey as step s5
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable as step s6
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map as step s7
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/Read as
step s8
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from bounded Kafka/Read(KafkaUnboundedSource)/StripIds as
step s9
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Measure read time as step s10
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Map records to strings/Map as step s11
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s12
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s13
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step
s16
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s18
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as
step s20
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 08, 2021 12:37:35 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 08, 2021 12:37:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.30.0-SNAPSHOT
Apr 08, 2021 12:37:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2021-04-08_05_37_35-17440113129569674034?project=apache-beam-testing
Apr 08, 2021 12:37:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-04-08_05_37_35-17440113129569674034
Apr 08, 2021 12:37:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2021-04-08_05_37_35-17440113129569674034
Apr 08, 2021 12:37:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-04-08T12:37:39.877Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-b9hh. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:49.016Z: Worker configuration: n1-standard-1 in
us-central1-c.
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:49.807Z: Expanding CoGroupByKey operations into
optimizable parts.
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:49.937Z: Expanding GroupByKey operations into
optimizable parts.
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:49.962Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.160Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.190Z: Elided trivial flatten
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.233Z: Unzipping flatten s19 for input
s17.org.apache.beam.sdk.values.PCollection.<init>:402#e63ad18dfc139ac0
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.269Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.300Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.325Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.356Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.392Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.442Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.478Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.516Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.555Z: Unzipping flatten s19-u40 for input
s21.org.apache.beam.sdk.values.PCollection.<init>:402#24aa17775bf242ac-c38
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.588Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.614Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.644Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from
bounded Kafka/Read(KafkaUnboundedSource)/Split
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.710Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
into Read from bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with
random key
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.777Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.810Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.849Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.881Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.938Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:50.983Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/Read into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.032Z: Fusing consumer Read from bounded
Kafka/Read(KafkaUnboundedSource)/StripIds into Read from bounded
Kafka/Read(KafkaUnboundedSource)/Read
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.066Z: Fusing consumer Measure read time into Read
from bounded Kafka/Read(KafkaUnboundedSource)/StripIds
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.106Z: Fusing consumer Map records to strings/Map
into Measure read time
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.140Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.173Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.208Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.241Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.281Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.322Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.349Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.378Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/Values/Values/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.400Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.432Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 08, 2021 12:37:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.462Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 08, 2021 12:37:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.847Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 08, 2021 12:37:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:51.936Z: Starting 5 ****s in us-central1-c...
Apr 08, 2021 12:37:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:52.255Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 08, 2021 12:37:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:37:52.421Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 08, 2021 12:38:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:38:06.658Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 08, 2021 12:38:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:38:31.888Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Apr 08, 2021 12:39:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:01.497Z: Workers have started successfully.
Apr 08, 2021 12:39:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:01.523Z: Workers have started successfully.
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:17.744Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Split+Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:17.824Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:18.126Z: Finished operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:18.207Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:18.346Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 08, 2021 12:39:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-08T12:39:18.499Z: Executing operation Read from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from
bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read
from bounded
Kafka/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from
bounded Kafka/Read(KafkaUnboundedSource)/Read+Read from bounded
Kafka/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to
strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
org.apache.beam.sdk.io.kafka.KafkaIOIT >
testKafkaIOReadsAndWritesCorrectlyInBatch SKIPPED
Watching 1399 directories to track changes
Watching 1401 directories to track changes
Watching 1402 directories to track changes
> Task :sdks:java:io:kafka:integrationTest FAILED
:sdks:java:io:kafka:integrationTest (Thread[Execution **** for ':' Thread
9,5,main]) completed. Took 13 mins 11.768 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> Process 'Gradle Test Executor 5' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/6.8/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 31s
107 actionable tasks: 64 executed, 43 from cache
Watching 1402 directories to track changes
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=47ff8586-cfab-478c-8fab-4b382944eff6,
currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 5350
log file: /home/jenkins/.gradle/daemon/6.8/daemon-5350.out.log
----- Last 20 lines from daemon log file - daemon-5350.out.log -----
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> Process 'Gradle Test Executor 5' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at
https://docs.gradle.org/6.8/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 31s
107 actionable tasks: 64 executed, 43 from cache
Watching 1402 directories to track changes
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
