See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/95/display/redirect?page=changes>
Changes: [scott] Roll-forwawrd "Merge pull request #6979: Add helper task to print [scott] Share docker image name along with shared task [mxm] ParDoTest: Check expected errors inline instead of after pipeline [altay] fix lint error un userstate.py [mxm] Add output collection id to timer specs. [mxm] Move doFn wrapper initialization to open() [mxm] [BEAM-6009] Add note about default mode for PortableValidatesRunner [mxm] [BEAM-4681] Add support for portable timers in Flink streaming mode [altay] Convert Top combiner to a full PTransform ------------------------------------------ [...truncated 18.21 MB...] Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.353Z: Fusing consumer PAssert$27/VerifyAssertions/ParDo(DefaultConclude) into PAssert$27/RunChecks Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.377Z: Fusing consumer PAssert$27/RunChecks into PAssert$27/GetPane/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.413Z: Fusing consumer PAssert$27/GroupGlobally/Values/Values/Map into PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.448Z: Fusing consumer PAssert$27/GroupGlobally/ParDo(Concat) into PAssert$27/GroupGlobally/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.482Z: Unzipping flatten s43-u80 for input s45-reify-value58-c78 Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.515Z: Fusing unzipped copy of PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80, into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.550Z: Fusing consumer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.584Z: Fusing consumer PAssert$27/GroupGlobally/GroupDummyAndContents/Write into PAssert$27/GroupGlobally/GroupDummyAndContents/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.610Z: Fusing consumer Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.632Z: Fusing consumer PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.667Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.701Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.737Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.762Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.796Z: Fusing consumer PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.825Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.886Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.942Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:41.988Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.031Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.071Z: Fusing consumer OutputSideInputs into Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.102Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.126Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.154Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.186Z: Fusing consumer PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into PAssert$27/GroupGlobally/RewindowActuals/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.216Z: Fusing consumer PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.243Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.277Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.312Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.341Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.378Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.413Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.449Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into PAssert$27/GroupGlobally/Window.Into()/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.483Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.520Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.553Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.589Z: Fusing consumer Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into Create47/Read(CreateSource)/Impulse Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.624Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.651Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.686Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.714Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.748Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create123/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.769Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.806Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.826Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.859Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.881Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.918Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.954Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:42.981Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.008Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.047Z: Fusing consumer Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into Create123/Read(CreateSource)/Impulse Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.082Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.118Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey into View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.150Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.186Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create47/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.220Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.252Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.286Z: Fusing consumer Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.637Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.666Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.685Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.703Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.718Z: Starting 1 workers in us-central1-b... Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.729Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.751Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.777Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:43.978Z: Executing operation Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:44.007Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 14, 2018 11:49:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:49:44.038Z: Executing operation Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 14, 2018 11:50:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:50:02.280Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Nov 14, 2018 11:50:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:50:48.364Z: Workers have started successfully. Nov 14, 2018 11:50:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:50:48.886Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Nov 14, 2018 11:50:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:50:48.916Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 14, 2018 11:51:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:14.026Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 14, 2018 11:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:16.265Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 14, 2018 11:51:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:16.380Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 14, 2018 11:51:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:19.416Z: Workers have started successfully. Nov 14, 2018 11:51:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:23.646Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 14, 2018 11:51:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:23.729Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write Nov 14, 2018 11:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:30.594Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close Nov 14, 2018 11:51:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:30.674Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write Nov 14, 2018 11:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:37.327Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close Nov 14, 2018 11:51:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:37.418Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap Nov 14, 2018 11:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:46.190Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize Nov 14, 2018 11:51:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:46.624Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write Nov 14, 2018 11:51:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:50.353Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close Nov 14, 2018 11:51:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:50.430Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 14, 2018 11:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:55.755Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Close Nov 14, 2018 11:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:51:55.833Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude) Nov 14, 2018 11:52:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:52:02.517Z: Cleaning up. Nov 14, 2018 11:52:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:52:02.617Z: Stopping worker pool... Nov 14, 2018 11:54:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:54:21.648Z: Autoscaling: Resized worker pool from 1 to 0. Nov 14, 2018 11:54:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:54:21.676Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 14, 2018 11:54:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-14T23:54:21.721Z: Worker pool stopped. Nov 14, 2018 11:54:27 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-11-14_15_49_32-10586514527041571379 finished with status DONE. Nov 14, 2018 11:54:27 PM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-11-14_15_49_32-10586514527041571379. Found 0 success, 0 failures out of 0 expected assertions. Gradle Test Executor 95 finished executing tests. > Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest > FAILED 159 tests completed, 1 failed, 2 skipped Finished generating test XML results (0.356 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest> Generating HTML test report... Finished generating test html results (0.243 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest> :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest (Thread[Task worker for ':',5,main]) completed. Took 2 hrs 30 mins 23.61 secs. :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task worker for ':',5,main]) started. > Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages Caching disabled for task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not been enabled for the task Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not up-to-date because: Task has not declared any outputs despite executing actions. Starting process 'command 'docker''. Working directory: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java> Command: docker rmi us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181114212056 Successfully started process 'command 'docker'' Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181114212056 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4cea6b16ceeff260b06c89e0bc8638b3430a40a053d7e11ed49d42d261da6200 Starting process 'command 'gcloud''. Working directory: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java> Command: gcloud --quiet container images delete --force-delete-tags us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181114212056 Successfully started process 'command 'gcloud'' Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4cea6b16ceeff260b06c89e0bc8638b3430a40a053d7e11ed49d42d261da6200 Associated tags: - 20181114212056 Tags: - us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181114212056 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181114212056]. Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4cea6b16ceeff260b06c89e0bc8638b3430a40a053d7e11ed49d42d261da6200]. :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task worker for ':',5,main]) completed. Took 2.194 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'. > There were failing tests. See the report at: > file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2h 33m 44s 76 actionable tasks: 70 executed, 5 from cache, 1 up-to-date Publishing build scan... https://gradle.com/s/j3lpnjdldphkk Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org