See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/13/display/redirect?page=changes>
Changes:
[vojtech.janota] BEAM-5933: avoid memory allocation in hashCode call
------------------------------------------
[...truncated 18.32 MB...]
INFO: 2018-11-01T19:19:02.527Z: Expanding CoGroupByKey operations into
optimizable parts.
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:02.824Z: Expanding GroupByKey operations into
optimizable parts.
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:02.923Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:02.965Z: Elided trivial flatten
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.014Z: Elided trivial flatten
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.061Z: Elided trivial flatten
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.113Z: Unzipping flatten s43 for input
s31.org.apache.beam.sdk.values.PCollection.<init>:402#536e37dc9c742e20
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.154Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$27/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.186Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$27/GroupGlobally/GroupDummyAndContents/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.234Z: Fusing consumer PAssert$27/GetPane/Map into
PAssert$27/GroupGlobally/ParDo(Concat)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.280Z: Fusing consumer
PAssert$27/VerifyAssertions/ParDo(DefaultConclude) into PAssert$27/RunChecks
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.327Z: Fusing consumer PAssert$27/RunChecks into
PAssert$27/GetPane/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.374Z: Fusing consumer
PAssert$27/GroupGlobally/Values/Values/Map into
PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.414Z: Fusing consumer
PAssert$27/GroupGlobally/ParDo(Concat) into
PAssert$27/GroupGlobally/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.461Z: Unzipping flatten s43-u80 for input
s45-reify-value58-c78
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.511Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80,
into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.557Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.601Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Write into
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.646Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.685Z: Fusing consumer
PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.733Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.782Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.829Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.870Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.914Z: Fusing consumer
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:03.960Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.011Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.057Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.096Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.145Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.194Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.237Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.279Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.325Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.367Z: Fusing consumer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.411Z: Fusing consumer
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.451Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.498Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.519Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.556Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.595Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.645Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.690Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$27/GroupGlobally/Window.Into()/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.737Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.785Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.825Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.884Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create47/Read(CreateSource)/Impulse
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.936Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:04.982Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.023Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.059Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.108Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create123/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.149Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.205Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.242Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.288Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.336Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.385Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.429Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.473Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.512Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.559Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create123/Read(CreateSource)/Impulse
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.594Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.641Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.690Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.737Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create47/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.779Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.827Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:05.875Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.409Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.458Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.504Z: Starting 1 workers in us-central1-b...
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.505Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.541Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.576Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.625Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:06.673Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:07.042Z: Executing operation
Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:07.090Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 01, 2018 7:19:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:07.131Z: Executing operation
Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 01, 2018 7:19:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:19:17.639Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Nov 01, 2018 7:20:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:20:40.540Z: Workers have started successfully.
Nov 01, 2018 7:20:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:20:51.853Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Nov 01, 2018 7:20:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:20:51.888Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 01, 2018 7:21:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:00.503Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 01, 2018 7:21:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:02.759Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 01, 2018 7:21:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:02.857Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Nov 01, 2018 7:21:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:09.973Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Nov 01, 2018 7:21:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:10.057Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
Nov 01, 2018 7:21:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:11.603Z: Workers have started successfully.
Nov 01, 2018 7:21:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:16.725Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close
Nov 01, 2018 7:21:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:16.812Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
Nov 01, 2018 7:21:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:20.995Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 01, 2018 7:21:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:21.093Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 01, 2018 7:21:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:33.778Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize
Nov 01, 2018 7:21:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:33.997Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Nov 01, 2018 7:21:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:38.481Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Nov 01, 2018 7:21:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:38.557Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 01, 2018 7:21:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:43.225Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Close
Nov 01, 2018 7:21:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:43.330Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude)
Nov 01, 2018 7:21:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:49.062Z: Cleaning up.
Nov 01, 2018 7:21:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:21:49.167Z: Stopping worker pool...
Nov 01, 2018 7:25:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:25:15.399Z: Autoscaling: Resized worker pool from 1 to
0.
Nov 01, 2018 7:25:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:25:15.478Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 01, 2018 7:25:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-01T19:25:15.528Z: Worker pool stopped.
Nov 01, 2018 7:25:22 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-11-01_12_18_54-414101965935432291 finished with status DONE.
Nov 01, 2018 7:25:22 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-11-01_12_18_54-414101965935432291. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 92 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
159 tests completed, 1 failed, 2 skipped
Finished generating test XML results (0.562 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest>
Generating HTML test report...
Finished generating test html results (0.832 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest>
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
> FAILED
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
(Thread[Task worker for ':' Thread 22,5,main]) completed. Took 4 hrs 5 mins
18.927 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4h 9m 27s
75 actionable tasks: 70 executed, 4 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/kwnjznre5iydu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
