See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/259/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-4150] Use unwindowed coder in FnApiRunner optimization phases.
[robertwb] [BEAM-6186] Optimization cleanup: move phase utilities out of local
[robertwb] [BEAM-6167] Add class ReadFromTextWithFilename (Python) (#7193)
------------------------------------------
[...truncated 17.94 MB...]
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.103Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80,
into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.140Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.176Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Write into
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.211Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.246Z: Fusing consumer
PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.280Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.319Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.357Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.391Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.424Z: Fusing consumer
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.463Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.502Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.533Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.571Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.610Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.641Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.673Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.712Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.748Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.787Z: Fusing consumer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.821Z: Fusing consumer
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.856Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.894Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.920Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.958Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:19.992Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.035Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.065Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$27/GroupGlobally/Window.Into()/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.100Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.137Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.169Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.211Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create47/Read(CreateSource)/Impulse
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.251Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.278Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.314Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.351Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.386Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create123/Read(CreateSource)/ParDo(SplitBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.423Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.459Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.494Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.532Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.573Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.605Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.643Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.679Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.716Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.751Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create123/Read(CreateSource)/Impulse
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.788Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.825Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.870Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.907Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create47/Read(CreateSource)/ParDo(SplitBoundedSource)
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.940Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:20.977Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.013Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.434Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.474Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.509Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.523Z: Starting 1 workers in us-central1-b...
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.546Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.592Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.630Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:21.670Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:22.041Z: Executing operation
Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:22.110Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Dec 10, 2018 10:34:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:22.154Z: Executing operation
Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Dec 10, 2018 10:34:42 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:34:41.910Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Dec 10, 2018 10:35:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:35:16.935Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Dec 10, 2018 10:35:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:35:16.975Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Dec 10, 2018 10:35:39 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:35:39.208Z: Workers have started successfully.
Dec 10, 2018 10:35:39 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:35:39.243Z: Workers have started successfully.
Dec 10, 2018 10:36:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:02.263Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Dec 10, 2018 10:36:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:02.346Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Dec 10, 2018 10:36:12 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:10.166Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Dec 10, 2018 10:36:12 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:11.619Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Dec 10, 2018 10:36:12 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:11.705Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
Dec 10, 2018 10:36:19 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:19.025Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close
Dec 10, 2018 10:36:19 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:19.103Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
Dec 10, 2018 10:36:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:21.489Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Dec 10, 2018 10:36:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:21.564Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Dec 10, 2018 10:36:35 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:33.416Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize
Dec 10, 2018 10:36:35 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:33.614Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Dec 10, 2018 10:36:42 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:41.216Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Dec 10, 2018 10:36:42 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:41.317Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Dec 10, 2018 10:36:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:46.727Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Close
Dec 10, 2018 10:36:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:46.832Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude)
Dec 10, 2018 10:36:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:50.280Z: Cleaning up.
Dec 10, 2018 10:36:52 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:36:50.366Z: Stopping worker pool...
Dec 10, 2018 10:38:46 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:38:45.101Z: Autoscaling: Resized worker pool from 1 to
0.
Dec 10, 2018 10:38:46 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:38:45.140Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Dec 10, 2018 10:38:46 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-10T10:38:45.186Z: Worker pool stopped.
Dec 10, 2018 10:38:51 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-12-10_02_34_08-5918730384675089512 finished with status DONE.
Dec 10, 2018 10:38:52 AM
org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-12-10_02_34_08-5918730384675089512. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 93 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
> FAILED
159 tests completed, 1 failed, 2 skipped
Finished generating test XML results (0.382 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest>
Generating HTML test report...
Finished generating test html results (0.569 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest>
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
(Thread[Daemon worker,5,main]) completed. Took 2 hrs 13 mins 16.345 secs.
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Daemon
worker,5,main]) started.
> Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages
Caching disabled for task
':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not
been enabled for the task
Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not
up-to-date because:
Task has not declared any outputs despite executing actions.
Starting process 'command 'docker''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: docker rmi
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181210082256
Successfully started process 'command 'docker''
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181210082256
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5685faeb17ad86a15979612dcd8dddd5e282bcd23a51fb5daef133f8dd5322af
Deleted: sha256:e8477975ab2cbb9e1682d6417e988afd5df7adde061a8175c89dca07c908a5c9
Deleted: sha256:611f71ec2cb031196ea7ea88b93cbff1a2d9e43df73a346769168bbbf5c64e3d
Deleted: sha256:db782996f72940a93e62d826e6e8caaa800e1787bfad291890d2b847a8802d33
Deleted: sha256:516b27925b08a9bc1bcd435659befcb096149cc3dcd3a1ba5f3c314b08f8345d
Deleted: sha256:225b40aee9cec413809c2a520b43dc6701f7422a0f5b0b776532dd0294bb5caf
Deleted: sha256:a2e0e85c12de283c80918e766af46bf6d1c58570cbf02ea425455e9f947b85b1
Deleted: sha256:fe0c22356415946e63083c7afcc249ff5bd5758cf69da033ac39a67aa3c3347b
Deleted: sha256:a561be77fdf111a95a71bb484324b69105a49329d6117740efe7b371ba3863f8
Deleted: sha256:dc5eab6aef8c3cf659d21f71194f2a68eff6491feb20f59a9083e82f82df8efc
Deleted: sha256:e5fa0ba3beec039c37aecbf32b58ef0a6c8940a00e66adbf0065c8e7fd9258e6
Starting process 'command 'gcloud''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: gcloud --quiet container images delete --force-delete-tags
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181210082256
Successfully started process 'command 'gcloud''
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5685faeb17ad86a15979612dcd8dddd5e282bcd23a51fb5daef133f8dd5322af
Associated tags:
- 20181210082256
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181210082256
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181210082256].
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5685faeb17ad86a15979612dcd8dddd5e282bcd23a51fb5daef133f8dd5322af].
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Daemon
worker,5,main]) completed. Took 2.354 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 16m 12s
75 actionable tasks: 69 executed, 5 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/hziie3cong35g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
