See
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/1283/display/redirect?page=changes>
Changes:
[sunjincheng121] [BEAM-9298] Drop support for Flink 1.7
------------------------------------------
[...truncated 322.15 KB...]
INFO: 2020-03-12T12:45:18.876Z: Checking permissions granted to controller
Service Account.
Mar 12, 2020 12:45:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-03-12T12:45:36.295Z: Your project already contains 100
Dataflow-created metric descriptors and Stackdriver will not create new
Dataflow custom metrics for this job. Each unique user-defined metric name
(independent of the DoFn in which it is defined) produces a new metric
descriptor. To delete old / unused metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:36.958Z: Worker configuration: n1-standard-1 in
us-central1-f.
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:37.624Z: Expanding CoGroupByKey operations into
optimizable parts.
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:37.958Z: Expanding GroupByKey operations into
optimizable parts.
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:37.994Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.231Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.266Z: Elided trivial flatten
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.309Z: Elided trivial flatten
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.357Z: Elided trivial flatten
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.394Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.438Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$0/GroupGlobally/GroupDummyAndContents/Read
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.492Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.526Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.570Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.603Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.636Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.667Z: Unzipping flatten s26 for input
s21.org.apache.beam.sdk.values.PCollection.<init>:400#cdecdb7fa820db14
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.710Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.749Z: Unzipping flatten s13 for input
s12.org.apache.beam.sdk.values.PCollection.<init>:400#64ea862f9c952497
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.787Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Window.Into()/Window.Assign, through flatten Calculate
hashcode/Flatten.PCollections, into producer Calculate hashcode/ProduceDefault
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.822Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.857Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.895Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.932Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:38.978Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.018Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.045Z: Fusing consumer
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.080Z: Fusing consumer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.109Z: Unzipping flatten s13-u44 for input
s14.output-c42
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.154Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous), through
flatten Calculate hashcode/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.193Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.234Z: Fusing consumer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign into Calculate
hashcode/Values/Values/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.271Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.318Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.353Z: Fusing consumer Collect read time into Read
using Hadoop InputFormat/Read(HadoopInputFormatBoundedSource)
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.392Z: Fusing consumer Get values only/Values/Map
into Collect read time
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.425Z: Fusing consumer Values as string into Get
values only/Values/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.465Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Values as string
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.513Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Mar 12, 2020 12:45:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.546Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.584Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.624Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.665Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.703Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.750Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:39.784Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.332Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.369Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.407Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.408Z: Starting 5 workers in us-central1-f...
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.477Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.504Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.504Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.725Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 12, 2020 12:45:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:45:40.780Z: Executing operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 12, 2020 12:46:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:46:09.291Z: Autoscaling: Raised the number of workers
to 5 based on the rate of progress in the currently running step(s).
Mar 12, 2020 12:46:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:46:25.021Z: Workers have started successfully.
Mar 12, 2020 12:46:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:46:25.157Z: Workers have started successfully.
Mar 12, 2020 12:46:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:46:42.266Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 12, 2020 12:47:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:00.000Z: Finished operation Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource)+Collect read time+Get values
only/Values/Map+Values as string+Calculate
hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Mar 12, 2020 12:47:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:00.091Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 12, 2020 12:47:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:00.170Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Mar 12, 2020 12:47:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:00.280Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 12, 2020 12:47:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:08.624Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 12, 2020 12:47:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:09.052Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 12, 2020 12:47:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:09.218Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Mar 12, 2020 12:47:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:09.664Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 12, 2020 12:47:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:11.632Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Mar 12, 2020 12:47:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:11.734Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Mar 12, 2020 12:47:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:11.793Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Mar 12, 2020 12:47:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:11.903Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 12, 2020 12:47:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:13.598Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Mar 12, 2020 12:47:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:13.696Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Mar 12, 2020 12:47:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:13.761Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Mar 12, 2020 12:47:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:13.875Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 12, 2020 12:47:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:15.722Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Mar 12, 2020 12:47:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:16.182Z: Cleaning up.
Mar 12, 2020 12:47:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:47:16.280Z: Stopping worker pool...
Mar 12, 2020 12:48:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:48:23.382Z: Autoscaling: Resized worker pool from 5 to
0.
Mar 12, 2020 12:48:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-03-12T12:48:23.589Z: Worker pool stopped.
Mar 12, 2020 12:48:31 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-03-12_05_45_15-15878392593383556186 finished with status
DONE.
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_OUT
Load test results for test (ID): bcdab88f-3728-4106-87da-bd007fd70aae and
timestamp: 2020-03-12T12:48:31.678000000Z:
Metric: Value:
read_time 8.242
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat FAILED
java.lang.IllegalStateException: Unable to fetch table size
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.lambda$getWriteSuppliers$0(HadoopFormatIOIT.java:231)
at java.util.Optional.orElseThrow(Optional.java:290)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.lambda$getWriteSuppliers$1(HadoopFormatIOIT.java:231)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.lambda$readAll$0(MetricsReader.java:107)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1556)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.readAll(MetricsReader.java:107)
at
org.apache.beam.sdk.testutils.metrics.IOITMetrics.publish(IOITMetrics.java:55)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.collectAndPublishMetrics(HadoopFormatIOIT.java:217)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.writeAndReadUsingHadoopFormat(HadoopFormatIOIT.java:201)
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT STANDARD_ERROR
Mar 12, 2020 12:48:32 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #1 of 3 failed: ERROR: table
"beamtest_hadoopformatioit_2020_03_12_12_41_44_342" does not exist.
Mar 12, 2020 12:48:32 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Retrying in 2000 ms.
Mar 12, 2020 12:48:34 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #2 of 3 failed: ERROR: table
"beamtest_hadoopformatioit_2020_03_12_12_41_44_342" does not exist.
Mar 12, 2020 12:48:34 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Retrying in 4000 ms.
Mar 12, 2020 12:48:38 PM org.apache.beam.sdk.io.common.IOITHelper
executeWithRetry
WARNING: Attempt #3 of 3 failed: ERROR: table
"beamtest_hadoopformatioit_2020_03_12_12_41_44_342" does not exist.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:hadoop-format:integrationTest FAILED
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT > classMethod FAILED
org.postgresql.util.PSQLException: ERROR: table
"beamtest_hadoopformatioit_2020_03_12_12_41_44_342" does not exist
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
at
org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:244)
at
org.apache.beam.sdk.io.common.DatabaseTestHelper.deleteTable(DatabaseTestHelper.java:65)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.deleteTable(HadoopFormatIOIT.java:163)
at
org.apache.beam.sdk.io.common.IOITHelper.executeWithRetry(IOITHelper.java:86)
at
org.apache.beam.sdk.io.common.IOITHelper.executeWithRetry(IOITHelper.java:66)
at
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT.tearDown(HadoopFormatIOIT.java:159)
2 tests completed, 2 failed
Finished generating test XML results (0.026 secs) into:
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.034 secs) into:
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest>
:sdks:java:io:hadoop-format:integrationTest (Thread[Execution worker for ':'
Thread 5,5,main]) completed. Took 6 mins 58.13 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:hadoop-format:integrationTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/io/hadoop-format/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 53s
91 actionable tasks: 62 executed, 29 from cache
Publishing build scan...
https://gradle.com/s/rhxs5wa7qbuci
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
