See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/58/display/redirect?page=changes>
Changes:
[lcwik] Tolerate duplicate counters from SDK in StreamingDataflowWorker. (#6951)
[lcwik] Dataflow Worker: Add worker_id to windmill request headers (#6663)
------------------------------------------
[...truncated 18.31 MB...]
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.506Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80,
into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.541Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.581Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Write into
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.621Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.655Z: Fusing consumer
PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.692Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.732Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.766Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.809Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.843Z: Fusing consumer
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.874Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.916Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.956Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:11.989Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.020Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.051Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.082Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.119Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.154Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.191Z: Fusing consumer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.232Z: Fusing consumer
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.263Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.305Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.338Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.377Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.411Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.449Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.488Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$27/GroupGlobally/Window.Into()/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.531Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.566Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.604Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.641Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create47/Read(CreateSource)/Impulse
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.689Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.730Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.767Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.803Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.842Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create123/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.879Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.921Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.947Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:12.981Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.015Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.059Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.099Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.141Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.178Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.211Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create123/Read(CreateSource)/Impulse
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.250Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.283Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.323Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.364Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create47/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.390Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.431Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.468Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.931Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:13.967Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.005Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.028Z: Starting 1 workers in us-central1-b...
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.040Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.078Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.115Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.142Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.451Z: Executing operation
Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.486Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 08, 2018 11:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:14.516Z: Executing operation
Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 08, 2018 11:14:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:14:26.048Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Nov 08, 2018 11:15:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:15:44.489Z: Workers have started successfully.
Nov 08, 2018 11:15:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:15:49.976Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Nov 08, 2018 11:15:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:15:50.014Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 08, 2018 11:16:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:05.119Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 08, 2018 11:16:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:07.408Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 08, 2018 11:16:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:07.496Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 08, 2018 11:16:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:14.581Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 08, 2018 11:16:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:14.669Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Nov 08, 2018 11:16:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:15.563Z: Workers have started successfully.
Nov 08, 2018 11:16:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:23.806Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Nov 08, 2018 11:16:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:23.900Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
Nov 08, 2018 11:16:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:29.356Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close
Nov 08, 2018 11:16:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:29.432Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
Nov 08, 2018 11:16:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:37.755Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize
Nov 08, 2018 11:16:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:37.949Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Nov 08, 2018 11:16:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:44.388Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Nov 08, 2018 11:16:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:44.472Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 08, 2018 11:16:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:49.118Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Close
Nov 08, 2018 11:16:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:49.213Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude)
Nov 08, 2018 11:16:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:52.943Z: Cleaning up.
Nov 08, 2018 11:16:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:16:53.050Z: Stopping worker pool...
Nov 08, 2018 11:20:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:20:40.947Z: Autoscaling: Resized worker pool from 1 to
0.
Nov 08, 2018 11:20:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:20:40.976Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 08, 2018 11:20:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-08T23:20:41.030Z: Worker pool stopped.
Nov 08, 2018 11:20:46 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-11-08_15_14_03-18213082558887547496 finished with status
DONE.
Nov 08, 2018 11:20:46 PM
org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-11-08_15_14_03-18213082558887547496. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 94 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
> FAILED
159 tests completed, 1 failed, 2 skipped
Finished generating test XML results (0.41 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest>
Generating HTML test report...
Finished generating test html results (0.404 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest>
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
(Thread[Task worker for ':' Thread 66,5,main]) completed. Took 3 hrs 11 mins
2.602 secs.
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 66,5,main]) started.
> Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages
Caching disabled for task
':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not
been enabled for the task
Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not
up-to-date because:
Task has not declared any outputs despite executing actions.
Starting process 'command 'docker''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: docker rmi
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181108200636
Successfully started process 'command 'docker''
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181108200636
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8cf757dcba7588f8f899d42251d99da43429adfde7c9ec9587ce346906a99518
Deleted: sha256:5059bdb152fe8547cd6f85d52f255e97a55bbf5e57491cb7f49259febd8d3942
Deleted: sha256:70c58f09ff2a15670e56dc214a1dcadfd4237deb39d91c19905c3751c501c707
Deleted: sha256:480bfe7f66f044b759cac63cb44c139b7b956cdf3e370241c4ca42884abfd2a1
Deleted: sha256:2d1eb3962645c956d6c2647b00308807befe0436cb276cac136c292d6937efc5
Deleted: sha256:8405fa05fc862780060c1d20c52085f4faf65945984b91baad57b5d35fa6ebdf
Deleted: sha256:a9f20cfc06fca9791ad041e1473b3206764e25480897a8ce4fbb2ef01db6d5e4
Deleted: sha256:752b1e8c6bb60a8160da8ff426aaae6927f6f15d80848010dfbd89a4e54d2251
Deleted: sha256:f5b25d56bdf635d37b6361fbee2331f4089d83271f3dfbc15c2aa329899bf5a3
Deleted: sha256:f08e94e59e22e8347ace69739e708b50206239b88f69895a8c309c141481012e
Deleted: sha256:2d41c140fdc5b56768613d043c5305059aee99cd8ef4cabfabef3c99b2fad656
Starting process 'command 'gcloud''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: gcloud --quiet container images delete --force-delete-tags
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181108200636
Successfully started process 'command 'gcloud''
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8cf757dcba7588f8f899d42251d99da43429adfde7c9ec9587ce346906a99518
Associated tags:
- 20181108200636
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181108200636
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181108200636].
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8cf757dcba7588f8f899d42251d99da43429adfde7c9ec9587ce346906a99518].
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 66,5,main]) completed. Took 2.527 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 14m 24s
76 actionable tasks: 71 executed, 4 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/lor6hl7gnkddg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org
For additional commands, e-mail: builds-h...@beam.apache.org
