See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/82/display/redirect?page=changes>
Changes: [github] [BEAM-5446] SplittableDoFn: Remove "internal" methods for public API [lcwik] [BEAM-6037] Make Spark runner pipeline translation based on URNs (#7005) [huangry] Update worker container version to most recent release. [lcwik] [BEAM-3608] Vendor guava 20.0 (#6809) ------------------------------------------ [...truncated 17.72 MB...] INFO: 2018-11-13T02:47:37.624Z: Expanding CollectionToSingleton operations into optimizable parts. Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:37.727Z: Expanding CoGroupByKey operations into optimizable parts. Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.028Z: Expanding GroupByKey operations into optimizable parts. Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.139Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.184Z: Elided trivial flatten Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.221Z: Elided trivial flatten Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.270Z: Elided trivial flatten Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.321Z: Unzipping flatten s43 for input s31.org.apache.beam.sdk.values.PCollection.<init>:402#536e37dc9c742e20 Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.363Z: Fusing unzipped copy of PAssert$27/GroupGlobally/GroupDummyAndContents/Reify, through flatten PAssert$27/GroupGlobally/FlattenDummyAndContents, into producer PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.395Z: Fusing consumer PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow into PAssert$27/GroupGlobally/GroupDummyAndContents/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.435Z: Fusing consumer PAssert$27/GetPane/Map into PAssert$27/GroupGlobally/ParDo(Concat) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.473Z: Fusing consumer PAssert$27/VerifyAssertions/ParDo(DefaultConclude) into PAssert$27/RunChecks Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.521Z: Fusing consumer PAssert$27/RunChecks into PAssert$27/GetPane/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.596Z: Fusing consumer PAssert$27/GroupGlobally/Values/Values/Map into PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.647Z: Fusing consumer PAssert$27/GroupGlobally/ParDo(Concat) into PAssert$27/GroupGlobally/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.696Z: Unzipping flatten s43-u80 for input s45-reify-value58-c78 Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.738Z: Fusing unzipped copy of PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80, into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.788Z: Fusing consumer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.837Z: Fusing consumer PAssert$27/GroupGlobally/GroupDummyAndContents/Write into PAssert$27/GroupGlobally/GroupDummyAndContents/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.887Z: Fusing consumer Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.929Z: Fusing consumer PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:38.981Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.026Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.060Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.109Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.158Z: Fusing consumer PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.198Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.239Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.272Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.319Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.369Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.420Z: Fusing consumer OutputSideInputs into Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.474Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.513Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.554Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.601Z: Fusing consumer PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into PAssert$27/GroupGlobally/RewindowActuals/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.647Z: Fusing consumer PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.687Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.728Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.775Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.824Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.873Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.924Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:39.970Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into PAssert$27/GroupGlobally/Window.Into()/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.015Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.051Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.095Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.144Z: Fusing consumer Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into Create47/Read(CreateSource)/Impulse Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.183Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.227Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.271Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.316Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.354Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create123/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.396Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.431Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.471Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.514Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.555Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.578Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.625Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.671Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.719Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.771Z: Fusing consumer Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into Create123/Read(CreateSource)/Impulse Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.818Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.863Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey into View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.907Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.949Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create47/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:40.996Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.045Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.093Z: Fusing consumer Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.630Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.659Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.711Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.712Z: Starting 1 workers in us-central1-b... Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.751Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.790Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.825Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Create Nov 13, 2018 2:47:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:41.861Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 13, 2018 2:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:42.244Z: Executing operation Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 13, 2018 2:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:42.282Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 13, 2018 2:47:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:42.324Z: Executing operation Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 13, 2018 2:47:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:47:55.835Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Nov 13, 2018 2:48:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:48:28.726Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Nov 13, 2018 2:48:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:48:28.765Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 13, 2018 2:48:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:48:46.899Z: Workers have started successfully. Nov 13, 2018 2:48:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:48:46.938Z: Workers have started successfully. Nov 13, 2018 2:49:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:06.838Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 13, 2018 2:49:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:06.912Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 13, 2018 2:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:11.823Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 13, 2018 2:49:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:11.904Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write Nov 13, 2018 2:49:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:21.051Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 13, 2018 2:49:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:22.285Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close Nov 13, 2018 2:49:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:22.346Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write Nov 13, 2018 2:49:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:29.990Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close Nov 13, 2018 2:49:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:30.070Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap Nov 13, 2018 2:49:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:34.570Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize Nov 13, 2018 2:49:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:34.751Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write Nov 13, 2018 2:49:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:40.281Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close Nov 13, 2018 2:49:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:40.366Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 13, 2018 2:49:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:44.113Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Close Nov 13, 2018 2:49:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:44.188Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude) Nov 13, 2018 2:49:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:48.000Z: Cleaning up. Nov 13, 2018 2:49:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:49:48.095Z: Stopping worker pool... Nov 13, 2018 2:51:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:51:49.141Z: Autoscaling: Resized worker pool from 1 to 0. Nov 13, 2018 2:51:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:51:49.182Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 13, 2018 2:51:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:51:49.231Z: Worker pool stopped. Nov 13, 2018 2:51:57 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-11-12_18_47_29-17581202294640307301 finished with status DONE. Nov 13, 2018 2:51:57 AM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-11-12_18_47_29-17581202294640307301. Found 0 success, 0 failures out of 0 expected assertions. Gradle Test Executor 96 finished executing tests. > Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest > FAILED 159 tests completed, 4 failed, 2 skipped Finished generating test XML results (0.437 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest> Generating HTML test report... Finished generating test html results (0.453 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest> :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest (Thread[Task worker for ':' Thread 37,5,main]) completed. Took 2 hrs 16 mins 20.448 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'. > There were failing tests. See the report at: > file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2h 19m 55s 75 actionable tasks: 69 executed, 5 from cache, 1 up-to-date Publishing build scan... https://gradle.com/s/7uyjaguoggk74 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org