See
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/2174/display/redirect>
Changes:
------------------------------------------
[...truncated 304.81 KB...]
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Get values only/Values/Map as step s3
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Values as string as step s4
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s5
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s6
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s7
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s8
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) as step s9
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/View.AsIterable/CreateDataflowView as step
s10
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/CreateVoid/Read(CreateSource) as step s11
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/ProduceDefault as step s12
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Flatten.PCollections as step s13
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step
s14
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s15
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s16
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s17
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as
step s18
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s19
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s20
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s21
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s22
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s23
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s24
Oct 18, 2020 12:23:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s25
Oct 18, 2020 12:23:54 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://dataflow-staging-us-central1-844138762903/temp/staging/
Oct 18, 2020 12:23:54 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <213002 bytes, hash
76ef7e02347cc85c3ebd80f664d439c25f256a7cfbbf0c81bc07ea8a006b6f45> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-du9-AjR8yFw-vYD2ZNQ5wl8lanz7vwyBvAfqigBrb0U.pb
Oct 18, 2020 12:23:55 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.26.0-SNAPSHOT
Oct 18, 2020 12:23:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-10-18_05_23_55-16659901375784079540?project=apache-beam-testing
Oct 18, 2020 12:23:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-10-18_05_23_55-16659901375784079540
Oct 18, 2020 12:23:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2020-10-18_05_23_55-16659901375784079540
Oct 18, 2020 12:24:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-10-18T12:23:58.249Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
hadoopformatioit0writeandreadusinghadoopformat-jenkins-101-iy5z. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:12.836Z: Worker configuration: n1-standard-1 in
us-central1-f.
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.380Z: Expanding CoGroupByKey operations into
optimizable parts.
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.550Z: Expanding GroupByKey operations into
optimizable parts.
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.582Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.715Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.749Z: Elided trivial flatten
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.783Z: Unzipping flatten s17 for input
s15.org.apache.beam.sdk.values.PCollection.<init>:399#d928e938ba0454ea
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.817Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.839Z: Unzipping flatten s13 for input
s12.org.apache.beam.sdk.values.PCollection.<init>:399#805b93b0a398202d
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.874Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous), through flatten
Calculate hashcode/Flatten.PCollections, into producer Calculate
hashcode/ProduceDefault
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.906Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.941Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.968Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:13.990Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.027Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.050Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.072Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.123Z: Unzipping flatten s17-u31 for input
s19.org.apache.beam.sdk.values.PCollection.<init>:399#f0f805bd79288b-c29
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.151Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.186Z: Unzipping flatten s13-u36 for input
s14.org.apache.beam.sdk.values.PCollection.<init>:399#4f24fdb6be16210b-c34
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.221Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables), through flatten Calculate
hashcode/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.255Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.290Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/Values/Values/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.324Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.363Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.396Z: Fusing consumer Collect read time into Read
using Hadoop InputFormat/Read(HadoopInputFormatBoundedSource)
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.428Z: Fusing consumer Get values only/Values/Map
into Collect read time
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.460Z: Fusing consumer Values as string into Get
values only/Values/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.487Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Values as string
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.520Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.543Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.576Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Oct 18, 2020 12:24:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.626Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.662Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.696Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.725Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:14.759Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.216Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Create
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.241Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.296Z: Starting 5 ****s in us-central1-f...
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.347Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Create
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.365Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.488Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:24:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:15.527Z: Executing operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Oct 18, 2020 12:24:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:33.371Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 18, 2020 12:24:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:24:43.439Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Oct 18, 2020 12:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:01.402Z: Workers have started successfully.
Oct 18, 2020 12:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:01.433Z: Workers have started successfully.
Oct 18, 2020 12:25:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:20.988Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:25:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:31.311Z: Finished operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Oct 18, 2020 12:25:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:31.381Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Oct 18, 2020 12:25:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:31.448Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Oct 18, 2020 12:25:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:31.538Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:25:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:40.682Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:25:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:40.825Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Oct 18, 2020 12:25:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:40.878Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Oct 18, 2020 12:25:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:41.025Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:25:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:42.857Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 18, 2020 12:25:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:42.940Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Close
Oct 18, 2020 12:25:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:42.991Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Close
Oct 18, 2020 12:25:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:43.064Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Oct 18, 2020 12:25:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:48.676Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Oct 18, 2020 12:25:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:48.819Z: Cleaning up.
Oct 18, 2020 12:25:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:25:48.896Z: Stopping **** pool...
Oct 18, 2020 12:26:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:26:41.539Z: Autoscaling: Resized **** pool from 5 to 0.
Oct 18, 2020 12:26:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:26:41.578Z: Worker pool stopped.
Oct 18, 2020 12:26:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-10-18_05_23_55-16659901375784079540 finished with status
DONE.
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_OUT
Load test results for test (ID): e17f3aa7-08fe-4623-b475-a46ff25a97c4 and
timestamp: 2020-10-18T12:26:47.723000000Z:
Metric: Value:
read_time 9.254
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:hadoop-format:integrationTest FAILED
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat FAILED
java.lang.NoSuchMethodError:
org.apache.commons.lang3.StringUtils.isNoneBlank([Ljava/lang/CharSequence;)Z
at
org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher.publishWithCheck(InfluxDBPublisher.java:70)
at
org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher.publishWithSettings(InfluxDBPublisher.java:64)
at
org.apache.beam.sdk.testutils.metrics.IOITMetrics.publishToInflux(IOITMetrics.java:87)
at
org.apache.beam.sdk.testutils.metrics.IOITMetrics.publishToInflux(IOITMetrics.java:77)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.collectAndPublishMetrics(HadoopFormatIOIT.java:225)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.writeAndReadUsingHadoopFormat(HadoopFormatIOIT.java:209)
1 test completed, 1 failed
Finished generating test XML results (0.033 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.044 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest>
:sdks:java:io:hadoop-format:integrationTest (Thread[Execution **** for ':'
Thread 3,5,main]) completed. Took 6 mins 42.586 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:hadoop-format:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 45s
95 actionable tasks: 57 executed, 38 from cache
Publishing build scan...
https://gradle.com/s/zvlv5ikbvi6dk
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not
accessed on or after Sun Oct 11 12:19:10 UTC 2020.
Invalidating in-memory cache of
/home/jenkins/.gradle/caches/journal-1/file-access.bin
Cache entries evicted. In-memory cache of
/home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4357}
MaxSize{4500}, CacheStats{hitCount=0, missCount=4357, loadSuccessCount=4358,
loadExceptionCount=0, totalLoadTime=242648689, evictionCount=1}
Performance may suffer from in-memory cache misses. Increase max heap size of
Gradle build process to reduce cache misses.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.325
secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
No JDK named ‘JDK 1.8 (latest)’ found
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
