See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/3/display/redirect?page=changes>
Changes:
[migryz] Add script to sync data for Beam GitHub community metrics.
[migryz] Address PR comments.
[swegner] Add GitHub data to source freshness dashboard
[swegner] Set y axis limits to 0-100%
[migryz] Fix unittests
[aaltay] [BEAM-5617] Use bytes consistently for pcollection ids. (#6844)
[github] Merge pull request #6883: [BEAM-4461] Resubmit PR to switch SQL over to
[migryz] Address PR changes
[amyrvold] [BEAM-5735] Contributor guide improvements
[lcwik] [BEAM-5299] Define max timestamp for global window in proto (#6381)
[lcwik] Remove unintended comment in BeamFnLoggingClient (#6796)
[migryz] Add Code Velocity dashboard json
------------------------------------------
[...truncated 18.16 MB...]
INFO: 2018-10-31T03:07:57.261Z: Expanding CoGroupByKey operations into
optimizable parts.
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.597Z: Expanding GroupByKey operations into
optimizable parts.
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.726Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.783Z: Elided trivial flatten
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.832Z: Elided trivial flatten
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.879Z: Elided trivial flatten
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.930Z: Unzipping flatten s43 for input
s31.org.apache.beam.sdk.values.PCollection.<init>:402#536e37dc9c742e20
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:57.977Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$27/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.028Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$27/GroupGlobally/GroupDummyAndContents/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.069Z: Fusing consumer PAssert$27/GetPane/Map into
PAssert$27/GroupGlobally/ParDo(Concat)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.113Z: Fusing consumer
PAssert$27/VerifyAssertions/ParDo(DefaultConclude) into PAssert$27/RunChecks
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.159Z: Fusing consumer PAssert$27/RunChecks into
PAssert$27/GetPane/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.208Z: Fusing consumer
PAssert$27/GroupGlobally/Values/Values/Map into
PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.255Z: Fusing consumer
PAssert$27/GroupGlobally/ParDo(Concat) into
PAssert$27/GroupGlobally/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.301Z: Unzipping flatten s43-u80 for input
s45-reify-value58-c78
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.352Z: Fusing unzipped copy of
PAssert$27/GroupGlobally/GroupDummyAndContents/Write, through flatten s43-u80,
into producer PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.404Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.446Z: Fusing consumer
PAssert$27/GroupGlobally/GroupDummyAndContents/Write into
PAssert$27/GroupGlobally/GroupDummyAndContents/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.495Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.543Z: Fusing consumer
PAssert$27/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.592Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.642Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.683Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.730Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.779Z: Fusing consumer
PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.827Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.886Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.936Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:58.986Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.037Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.086Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.134Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.182Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.227Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.277Z: Fusing consumer
PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.327Z: Fusing consumer
PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.379Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.438Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.488Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.536Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.584Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.633Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.680Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$27/GroupGlobally/Window.Into()/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.729Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.775Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.825Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.872Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create47/Read(CreateSource)/Impulse
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.923Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:07:59.972Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.022Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.072Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.125Z: Fusing consumer
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create123/Read(CreateSource)/ParDo(SplitBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.171Z: Fusing consumer
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.221Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.270Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.326Z: Fusing consumer
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.397Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.447Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.496Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.550Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.597Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.645Z: Fusing consumer
Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into
Create123/Read(CreateSource)/Impulse
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.697Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.740Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey
into
View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.787Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.837Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into
Create47/Read(CreateSource)/ParDo(SplitBoundedSource)
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.884Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.935Z: Fusing consumer
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:00.986Z: Fusing consumer
Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.556Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.607Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.659Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.676Z: Starting 1 workers in us-central1-b...
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.708Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.761Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.808Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:01.869Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:02.273Z: Executing operation
Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:02.318Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 31, 2018 3:08:03 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:02.362Z: Executing operation
Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 31, 2018 3:08:14 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:13.998Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Oct 31, 2018 3:08:56 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:08:55.181Z: Workers have started successfully.
Oct 31, 2018 3:09:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:00.250Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Oct 31, 2018 3:09:00 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:00.299Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Oct 31, 2018 3:09:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:19.202Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 31, 2018 3:09:23 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:21.528Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 31, 2018 3:09:23 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:21.620Z: Executing operation
PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Oct 31, 2018 3:09:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:26.268Z: Workers have started successfully.
Oct 31, 2018 3:09:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:31.171Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 31, 2018 3:09:31 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:31.294Z: Executing operation
Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Oct 31, 2018 3:09:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:37.480Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Oct 31, 2018 3:09:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:37.596Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write
Oct 31, 2018 3:09:42 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:42.201Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close
Oct 31, 2018 3:09:42 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:42.312Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap
Oct 31, 2018 3:09:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:50.112Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize
Oct 31, 2018 3:09:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:50.378Z: Executing operation
Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Oct 31, 2018 3:09:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:56.906Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Oct 31, 2018 3:09:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:09:57.015Z: Executing operation
PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write
Oct 31, 2018 3:10:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:10:01.486Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Close
Oct 31, 2018 3:10:02 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:10:01.599Z: Executing operation
PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude)
Oct 31, 2018 3:10:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:10:04.695Z: Cleaning up.
Oct 31, 2018 3:10:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:10:04.828Z: Stopping worker pool...
Oct 31, 2018 3:14:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:14:06.080Z: Autoscaling: Resized worker pool from 1 to
0.
Oct 31, 2018 3:14:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:14:06.148Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Oct 31, 2018 3:14:06 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-31T03:14:06.205Z: Worker pool stopped.
Oct 31, 2018 3:14:11 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-10-30_20_07_48-2773160727477103622 finished with status DONE.
Oct 31, 2018 3:14:12 AM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-10-30_20_07_48-2773160727477103622. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 119 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
159 tests completed, 11 failed, 2 skipped
Finished generating test XML results (0.53 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest>
Generating HTML test report...
Finished generating test html results (0.658 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest>
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
> FAILED
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest
(Thread[Task worker for ':' Thread 4,5,main]) completed. Took 2 hrs 58 mins
3.323 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3h 3m 16s
75 actionable tasks: 69 executed, 5 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/6mpgr6wqnpvyq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
