See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/108/display/redirect?page=changes>
Changes:
[github] Fix possible internal dependency conflicts
[foegler] [BEAM-6049] Add option to load job to GCS in Dataflow Runner
[scott] [BEAM-6080] Rollback PR/7049 for failing java post-commits
[robert] [BEAM-6073] Remove use of assert pacakge
[github] [BEAM-6073] Update README with lockfile & gradle instructions
[github] Add 2.7.0 LTS information to the docs. (#7069)
------------------------------------------
[...truncated 18.22 MB...]
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.780Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80,
into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.814Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.852Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Write into
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.890Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.927Z: Fusing consumer
PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.962Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:49.998Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.035Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.071Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.107Z: Fusing consumer
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.147Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.185Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.227Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.265Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.300Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.332Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.367Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.408Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.432Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.460Z: Fusing consumer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.496Z: Fusing consumer
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.533Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.564Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.592Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.627Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.664Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.699Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.724Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$27/GroupGlobally/Window.Into()/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.762Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.802Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.834Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.872Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create47/Read(CreateSource)/Impulse
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.906Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.941Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:50.978Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.015Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.045Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create123/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.079Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.114Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.152Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.188Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.225Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.254Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.289Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.326Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.365Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.387Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create123/Read(CreateSource)/Impulse
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.426Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.473Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.502Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.537Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create47/Read(CreateSource)/ParDo(SplitBoundedSource)
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.564Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.599Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:51.636Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.011Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.049Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.086Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.098Z: Starting 1 workers in us-central1-b...
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.121Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.157Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.196Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.221Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.485Z: Executing operation
Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.508Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 17, 2018 2:26:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:25:52.549Z: Executing operation
Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Nov 17, 2018 2:26:14 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:26:13.363Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:26:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:26:47.400Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:26:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:26:47.441Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 17, 2018 2:27:15 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:14.375Z: Workers have started successfully.
Nov 17, 2018 2:27:15 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:14.408Z: Workers have started successfully.
Nov 17, 2018 2:27:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:32.316Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 17, 2018 2:27:33 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:32.386Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Nov 17, 2018 2:27:39 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:37.385Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 17, 2018 2:27:39 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:37.486Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 17, 2018 2:27:46 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:46.445Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Nov 17, 2018 2:27:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:50.869Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Nov 17, 2018 2:27:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:50.944Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
Nov 17, 2018 2:27:56 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.461Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close
Nov 17, 2018 2:27:56 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.524Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
Nov 17, 2018 2:28:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:03.857Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize
Nov 17, 2018 2:28:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:04.046Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Nov 17, 2018 2:28:10 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:09.433Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Nov 17, 2018 2:28:10 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:09.499Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Nov 17, 2018 2:28:16 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:14.949Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Close
Nov 17, 2018 2:28:16 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:15.025Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude)
Nov 17, 2018 2:28:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:20.695Z: Cleaning up.
Nov 17, 2018 2:28:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:20.780Z: Stopping worker pool...
Nov 17, 2018 2:30:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:54.120Z: Autoscaling: Resized worker pool from 1 to
0.
Nov 17, 2018 2:30:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:54.157Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Nov 17, 2018 2:30:55 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:54.210Z: Worker pool stopped.
Nov 17, 2018 2:31:02 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-11-16_18_25_41-1347732189060196629 finished with status DONE.
Nov 17, 2018 2:31:02 AM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-11-16_18_25_41-1347732189060196629. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 96 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
> FAILED
159 tests completed, 2 failed, 2 skipped
Finished generating test XML results (0.495 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest>
Generating HTML test report...
Finished generating test html results (0.78 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest>
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
(Thread[Task worker for ':' Thread 116,5,main]) completed. Took 2 hrs 19 mins
20.354 secs.
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 116,5,main]) started.
> Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages
Caching disabled for task
':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not
been enabled for the task
Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not
up-to-date because:
Task has not declared any outputs despite executing actions.
Starting process 'command 'docker''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: docker rmi
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181117000833
Successfully started process 'command 'docker''
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181117000833
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3045b331dba5993c8d7cf1ccffd51a63e9af6cbcfa7503e6b84a8e723c42c23
Deleted: sha256:795c629734e97bd32eae11cce47f0d9de8f581d04aad7cd7326c51f6d9458d53
Deleted: sha256:835cef3a1d014a9eedd2aa7c88966bcf177997557f4e177967e8dea5d9a49c2a
Deleted: sha256:883e9748df7f6bc34d2f26b313dce57d5bccbc43aa7b8a48851b30c1af678498
Deleted: sha256:44f6393cfeaedb372ebb62fa1b502886bda4900178ab03699a74cebd8a68c139
Deleted: sha256:6a5ac0afed57a51f9a1008b4169e5b394341694f70bbce4d2cdcc3d29f7eceba
Deleted: sha256:415c586a8b145c123db36c6edd9225344df54b70e13dc1fe7eacfc4f20607c4e
Deleted: sha256:6fedc1b75632be61595a636bdd828d46f3502350425f8fa1664dfbbfbaf393b6
Deleted: sha256:ee5ed25df2c8fc50f001cae2f6bc567e8e2496cd56ba5b073e40dfe8ff5bd74d
Deleted: sha256:fe88d731fd1ffe34d04c2cef7bbce8c901222c266473e32ea1c947fdea815073
Deleted: sha256:e9d0628812573a2ee106f0882a66a6be23164f75dcfd1a9ea267cd942b214c98
Starting process 'command 'gcloud''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: gcloud --quiet container images delete --force-delete-tags
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181117000833
Successfully started process 'command 'gcloud''
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3045b331dba5993c8d7cf1ccffd51a63e9af6cbcfa7503e6b84a8e723c42c23
Associated tags:
- 20181117000833
Tags:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181117000833
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181117000833].
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:a3045b331dba5993c8d7cf1ccffd51a63e9af6cbcfa7503e6b84a8e723c42c23].
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 116,5,main]) completed. Took 2.289 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 22m 45s
76 actionable tasks: 70 executed, 5 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/or3o7r2b5obrq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
