See
<https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/22/display/redirect?page=changes>
Changes:
[aromanenko.dev] [BEAM-2852] Added Kafka source of events and sink of results
[aromanenko.dev] [BEAM-2852] Fix the code that uses Long key whereas there is
no key in
[aromanenko.dev] [BEAM-2852] Fix minor code style issues
------------------------------------------
[...truncated 17.92 MB...]
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.051Z: Fusing consumer
PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous) into
PAssert$1/CreateActual/FilterActuals/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.083Z: Fusing consumer Count rows/ProduceDefault
into Count rows/CreateVoid/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.096Z: Fusing consumer Count
rows/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Count
rows/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.129Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey/Reify into Count
rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.155Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial into Count
rows/WithKeys/AddKeys/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.187Z: Fusing consumer Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract into Count
rows/Combine.perKey(Count)/Combine.GroupedValues
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.214Z: Fusing consumer Count
rows/WithKeys/AddKeys/Map into SpannerIO.Read/Execute query/Read from Cloud
Spanner/Read from Partitions
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.236Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.268Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Values/Values/Map into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/ExpandIterable
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.294Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/ExpandIterable into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.327Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Read from Partitions into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.377Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.408Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign into
SpannerIO.Read/Execute query/Reshuffle/Pair with random key
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.436Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Write
into SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.466Z: Fusing consumer Count
rows/Combine.perKey(Count)/Combine.GroupedValues into Count
rows/Combine.perKey(Count)/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.489Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Reify
into SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.520Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Pair with random key
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.546Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Generate Partitions into SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.569Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Pair with random key into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Generate Partitions
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.591Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Pair with random key into
SpannerIO.Read/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.616Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Write into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.642Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.673Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.699Z: Fusing consumer Count
rows/Values/Values/Map into Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.722Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.744Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey/Write into Count
rows/Combine.perKey(Count)/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.766Z: Fusing consumer
PAssert$1/WindowToken/Window.Assign into
PAssert$1/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.786Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.811Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.834Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.859Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.891Z: Fusing consumer PAssert$1/RunChecks into
PAssert$1/WindowToken/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.922Z: Fusing consumer
PAssert$1/VerifyAssertions/ParDo(DefaultConclude) into PAssert$1/RunChecks
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.953Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.978Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into SpannerIO.CreateTransaction/Create transaction
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.002Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.023Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.035Z: Fusing consumer
SpannerIO.CreateTransaction/Create transaction into
SpannerIO.CreateTransaction/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.057Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.089Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.457Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.491Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.520Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.530Z: Starting 1 workers in us-central1-f...
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.545Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.567Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.591Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.615Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.648Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.739Z: Executing operation
SpannerIO.Read/Create.Values/Read(CreateSource)+SpannerIO.Read/Execute
query/Reshuffle/Pair with random key+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Write
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:08:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:33.097Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:08:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:34.196Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Apr 10, 2018 3:08:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:34.682Z: Executing operation
SpannerIO.CreateTransaction/Create.Values/Read(CreateSource)+SpannerIO.CreateTransaction/Create
transaction+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Apr 10, 2018 3:08:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:39.810Z: Autoscaling: Resizing worker pool from 1 to
8.
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:08:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:43.833Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:08:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:46.709Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Apr 10, 2018 3:08:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:46.737Z: Resized worker pool to 1, though goal was
8. This could be a quota issue.
Apr 10, 2018 3:08:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:57.619Z: Autoscaling: Raised the number of workers
to 8 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:09:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:07.965Z: Workers have started successfully.
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:09:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:07.753Z: Workers have started successfully.
Apr 10, 2018 3:09:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:30.224Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Close
Apr 10, 2018 3:09:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:31.533Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Apr 10, 2018 3:09:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:31.619Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Apr 10, 2018 3:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.264Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Apr 10, 2018 3:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.345Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:09:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.779Z: Cleaning up.
Apr 10, 2018 3:09:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.839Z: Stopping worker pool...
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:09:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:42.983Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/CreateDataflowView
Apr 10, 2018 3:09:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:43.158Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Read+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable+SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map+SpannerIO.Read/Execute query/Read from Cloud
Spanner/Generate Partitions+SpannerIO.Read/Execute query/Read from Cloud
Spanner/Shuffle partitions/Pair with random key+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Reify+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Write
Apr 10, 2018 3:09:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:46.859Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Close
Apr 10, 2018 3:09:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:46.941Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Read+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow+SpannerIO.Read/Execute query/Read
from Cloud Spanner/Shuffle
partitions/Reshuffle/ExpandIterable+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle partitions/Values/Values/Map+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Read from Partitions+Count
rows/WithKeys/AddKeys/Map+Count rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial+Count
rows/Combine.perKey(Count)/GroupByKey/Reify+Count
rows/Combine.perKey(Count)/GroupByKey/Write
Apr 10, 2018 3:09:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:56.534Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Close
Apr 10, 2018 3:09:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:56.625Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Read+Count
rows/Combine.perKey(Count)/Combine.GroupedValues+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract+Count
rows/Values/Values/Map+PAssert$1/CreateActual/FilterActuals/Window.Assign+PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous)+PAssert$1/CreateActual/GatherPanes/WithKeys/AddKeys/Map+PAssert$1/CreateActual/GatherPanes/Window.Into()/Window.Assign+PAssert$1/CreateActual/GatherPanes/GroupByKey/Reify+PAssert$1/CreateActual/GatherPanes/GroupByKey/Write+Count
rows/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Apr 10, 2018 3:10:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:01.704Z: Executing operation Count
rows/View.AsIterable/CreateDataflowView
Apr 10, 2018 3:10:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:01.899Z: Executing operation Count
rows/CreateVoid/Read(CreateSource)+Count
rows/ProduceDefault+PAssert$1/CreateActual/FilterActuals/Window.Assign+PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous)+PAssert$1/CreateActual/GatherPanes/WithKeys/AddKeys/Map+PAssert$1/CreateActual/GatherPanes/Window.Into()/Window.Assign+PAssert$1/CreateActual/GatherPanes/GroupByKey/Reify+PAssert$1/CreateActual/GatherPanes/GroupByKey/Write
Apr 10, 2018 3:10:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:03.781Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Close
Apr 10, 2018 3:10:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:03.866Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Read+PAssert$1/CreateActual/GatherPanes/GroupByKey/GroupByWindow+PAssert$1/CreateActual/GatherPanes/Values/Values/Map+PAssert$1/CreateActual/ExtractPane/Map+PAssert$1/CreateActual/Flatten.Iterables/FlattenIterables/FlatMap+PAssert$1/CreateActual/RewindowActuals/Window.Assign+PAssert$1/CreateActual/ParDo(Anonymous)+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:05.723Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:05.818Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:07.194Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:07.265Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Apr 10, 2018 3:10:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:11.346Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Apr 10, 2018 3:10:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:11.521Z: Executing operation
PAssert$1/Create.Values/Read(CreateSource)+PAssert$1/WindowToken/Window.Assign+PAssert$1/RunChecks+PAssert$1/VerifyAssertions/ParDo(DefaultConclude)
Apr 10, 2018 3:10:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:13.670Z: Cleaning up.
Apr 10, 2018 3:10:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:13.851Z: Stopping worker pool...
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:11:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:02.083Z: Autoscaling: Resized worker pool from 1 to
0.
Apr 10, 2018 3:11:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:02.115Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Apr 10, 2018 3:11:10 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-04-10_08_08_16-4270479033920952683 finished with status DONE.
Apr 10, 2018 3:11:10 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-04-10_08_08_16-4270479033920952683. Found 0 success, 0 failures out of 0
expected assertions.
Apr 10, 2018 3:11:11 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Writing batch of 500 entities
Apr 10, 2018 3:11:12 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Successfully wrote 500 entities
Apr 10, 2018 3:11:12 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Writing batch of 500 entities
Apr 10, 2018 3:11:13 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Successfully wrote 500 entities
Apr 10, 2018 3:11:13 PM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil
deleteAllEntities
INFO: Successfully deleted 1000 entities
Gradle Test Executor 126 finished executing tests.
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.562Z: Autoscaling: Resized worker pool from 8 to
0.
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.586Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.625Z: Worker pool stopped.
Apr 10, 2018 3:12:06 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-04-10_08_08_18-17222637697064452262 finished with status
DONE.
Apr 10, 2018 3:12:06 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-04-10_08_08_18-17222637697064452262. Found 1 success, 0 failures out of 1
expected assertions.
Gradle Test Executor 127 finished executing tests.
10 tests completed, 1 failed
:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest
FAILED
:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest
(Thread[Task worker for ':' Thread 2,5,main]) completed. Took 14 mins 11.772
secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest'.
> Process 'Gradle Test Executor 128' finished with non-zero exit value 137
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
See
https://docs.gradle.org/4.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 14s
760 actionable tasks: 760 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Recording test results
Not sending mail to unregistered user eh...@google.com
Not sending mail to unregistered user c...@google.com
Not sending mail to unregistered user daniel.o.program...@gmail.com
Not sending mail to unregistered user hero...@google.com
Not sending mail to unregistered user aromanenko....@gmail.com
