See
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/4211/display/redirect>
Changes:
------------------------------------------
[...truncated 372.48 KB...]
Mar 14, 2022 7:00:50 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s25
Mar 14, 2022 7:00:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.38.0-SNAPSHOT
Mar 14, 2022 7:00:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-14_12_00_50-10916392612436180620?project=apache-beam-testing
Mar 14, 2022 7:00:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2022-03-14_12_00_50-10916392612436180620
Mar 14, 2022 7:00:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2022-03-14_12_00_50-10916392612436180620
Mar 14, 2022 7:00:54 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2022-03-14T19:00:52.872Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
hadoopformatioit0writeandreadusinghadoopformat-jenkins-031-o49q. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Mar 14, 2022 7:01:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:04.404Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 14, 2022 7:01:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:19.742Z: Worker configuration: e2-standard-2 in
us-central1-b.
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:20.646Z: Expanding CoGroupByKey operations into
optimizable parts.
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:20.799Z: Expanding GroupByKey operations into
optimizable parts.
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:20.832Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:20.999Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.038Z: Elided trivial flatten
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.080Z: Unzipping flatten s17 for input
s15.org.apache.beam.sdk.values.PCollection.<init>:408#4f24fdb6be16210b
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.112Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections, into producer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.142Z: Unzipping flatten s13 for input
s12.org.apache.beam.sdk.values.PCollection.<init>:408#c47715dbb819979d
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.199Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous), through flatten
Calculate hashcode/Flatten.PCollections, into producer Calculate
hashcode/ProduceDefault
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.241Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Write into
PAssert$0/GroupGlobally/GroupByKey/Reify
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.286Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GroupByKey/Read
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.353Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Mar 14, 2022 7:01:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.391Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.439Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.516Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.558Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.591Z: Unzipping flatten s17-u31 for input
s19.org.apache.beam.sdk.values.PCollection.<init>:408#d9ae03fcd5e3bc4a-c29
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.623Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten
PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.668Z: Unzipping flatten s13-u36 for input
s14.org.apache.beam.sdk.values.PCollection.<init>:408#416a1b705666de47-c34
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.728Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables), through flatten Calculate
hashcode/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.780Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.836Z: Fusing consumer
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate
hashcode/Values/Values/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.917Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:21.990Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into
PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.034Z: Fusing consumer Collect read time into Read
using Hadoop InputFormat/Read(HadoopInputFormatBoundedSource)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.078Z: Fusing consumer Get values only/Values/Map
into Collect read time
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.112Z: Fusing consumer Values as string into Get
values only/Values/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.172Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Values as string
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.210Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.251Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.290Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.355Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.385Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.450Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.481Z: Fusing consumer
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.513Z: Fusing consumer
PAssert$0/GroupGlobally/GroupByKey/Reify into
PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.921Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:22.999Z: Starting 5 ****s in us-central1-b...
Mar 14, 2022 7:01:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:23.450Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 14, 2022 7:01:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:23.609Z: Executing operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 14, 2022 7:01:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:59.671Z: Autoscaling: Raised the number of ****s to
1 based on the rate of progress in the currently running stage(s).
Mar 14, 2022 7:01:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:01:59.697Z: Resized **** pool to 1, though goal was 5.
This could be a quota issue.
Mar 14, 2022 7:02:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:02:10.018Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
Mar 14, 2022 7:02:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:02:34.934Z: Workers have started successfully.
Mar 14, 2022 7:02:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:02:34.955Z: Workers have started successfully.
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:21.825Z: Finished operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:21.924Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:21.986Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:22.063Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Create
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:22.251Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Create
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:22.411Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:22.442Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:27.050Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:27.418Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:27.450Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 14, 2022 7:03:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:27.495Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 14, 2022 7:03:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:27.631Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:29.228Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Mar 14, 2022 7:03:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:29.297Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Close
Mar 14, 2022 7:03:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:29.382Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Close
Mar 14, 2022 7:03:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:29.471Z: Executing operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 14, 2022 7:03:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:33.421Z: Finished operation
PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 14, 2022 7:03:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:33.601Z: Cleaning up.
Mar 14, 2022 7:03:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:03:33.679Z: Stopping **** pool...
Mar 14, 2022 7:06:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:06:08.577Z: Autoscaling: Resized **** pool from 5 to 0.
Mar 14, 2022 7:06:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-14T19:06:08.650Z: Worker pool stopped.
Mar 14, 2022 7:06:17 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2022-03-14_12_00_50-10916392612436180620 finished with status
DONE.
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_OUT
Load test results for test (ID): 7b2fb56b-ff92-446f-83c1-9ebd2980a4c5 and
timestamp: 2022-03-14T19:06:17.422000000Z:
Metric: Value:
read_time 6.512
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat FAILED
java.lang.IllegalStateException: Unable to fetch table size
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.lambda$getWriteSuppliers$0(HadoopFormatIOIT.java:260)
at java.util.Optional.orElseThrow(Optional.java:290)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.lambda$getWriteSuppliers$1(HadoopFormatIOIT.java:260)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.lambda$readAll$0(MetricsReader.java:106)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1580)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.readAll(MetricsReader.java:106)
at
org.apache.beam.sdk.testutils.metrics.IOITMetrics.publishToInflux(IOITMetrics.java:56)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.collectAndPublishMetrics(HadoopFormatIOIT.java:246)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.writeAndReadUsingHadoopFormat(HadoopFormatIOIT.java:229)
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT STANDARD_ERROR
Mar 14, 2022 7:06:27 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #1 of 3 failed: Connection to 34.133.211.5:5432 refused.
Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections..
Mar 14, 2022 7:06:27 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Retrying in 2000 ms.
Mar 14, 2022 7:06:29 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #2 of 3 failed: Connection to 34.133.211.5:5432 refused.
Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections..
Mar 14, 2022 7:06:29 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Retrying in 4000 ms.
Mar 14, 2022 7:06:33 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #3 of 3 failed: Connection to 34.133.211.5:5432 refused.
Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections..
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT > classMethod FAILED
org.postgresql.util.PSQLException: Connection to 34.133.211.5:5432 refused.
Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections.
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:303)
at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225)
at org.postgresql.Driver.makeConnection(Driver.java:465)
at org.postgresql.Driver.connect(Driver.java:264)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at
org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:103)
at
org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:87)
at
org.apache.beam.sdk.io.common.DatabaseTestHelper.deleteTable(DatabaseTestHelper.java:105)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.deleteTable(HadoopFormatIOIT.java:190)
at
org.apache.beam.sdk.io.common.IOITHelper.executeWithRetry(IOITHelper.java:86)
at
org.apache.beam.sdk.io.common.IOITHelper.executeWithRetry(IOITHelper.java:66)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.tearDown(HadoopFormatIOIT.java:183)
Caused by:
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at org.postgresql.core.PGStream.createSocket(PGStream.java:231)
at org.postgresql.core.PGStream.<init>(PGStream.java:95)
at
org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:98)
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
... 13 more
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:hadoop-format:integrationTest FAILED
2 tests completed, 2 failed
Finished generating test XML results (0.024 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest>
:sdks:java:io:hadoop-format:integrationTest (Thread[Execution **** for ':'
Thread 5,5,main]) completed. Took 11 mins 29.655 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:hadoop-format:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 11m 59s
150 actionable tasks: 89 executed, 59 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/ariiiblxo7mq2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
