See
<https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/22/display/redirect?page=changes>
Changes:
[aromanenko.dev] [BEAM-2852] Added Kafka source of events and sink of results
[aromanenko.dev] [BEAM-2852] Fix the code that uses Long key whereas there is
no key in
[aromanenko.dev] [BEAM-2852] Fix minor code style issues
------------------------------------------
[...truncated 17.92 MB...]
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.051Z: Fusing consumer
PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous) into
PAssert$1/CreateActual/FilterActuals/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.083Z: Fusing consumer Count rows/ProduceDefault
into Count rows/CreateVoid/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.096Z: Fusing consumer Count
rows/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Count
rows/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.129Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey/Reify into Count
rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.155Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial into Count
rows/WithKeys/AddKeys/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.187Z: Fusing consumer Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract into Count
rows/Combine.perKey(Count)/Combine.GroupedValues
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.214Z: Fusing consumer Count
rows/WithKeys/AddKeys/Map into SpannerIO.Read/Execute query/Read from Cloud
Spanner/Read from Partitions
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.236Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.268Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Values/Values/Map into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/ExpandIterable
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.294Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/ExpandIterable into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.327Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Read from Partitions into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.377Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.408Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign into
SpannerIO.Read/Execute query/Reshuffle/Pair with random key
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.436Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Write
into SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.466Z: Fusing consumer Count
rows/Combine.perKey(Count)/Combine.GroupedValues into Count
rows/Combine.perKey(Count)/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.489Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Reify
into SpannerIO.Read/Execute query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.520Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Pair with random key
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.546Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Generate Partitions into SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.569Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Pair with random key into
SpannerIO.Read/Execute query/Read from Cloud Spanner/Generate Partitions
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.591Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Pair with random key into
SpannerIO.Read/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.616Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Write into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.642Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.673Z: Fusing consumer SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable into SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.699Z: Fusing consumer Count
rows/Values/Values/Map into Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.722Z: Fusing consumer SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow into SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.744Z: Fusing consumer Count
rows/Combine.perKey(Count)/GroupByKey/Write into Count
rows/Combine.perKey(Count)/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.766Z: Fusing consumer
PAssert$1/WindowToken/Window.Assign into
PAssert$1/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.786Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.811Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.834Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.859Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.891Z: Fusing consumer PAssert$1/RunChecks into
PAssert$1/WindowToken/Window.Assign
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.922Z: Fusing consumer
PAssert$1/VerifyAssertions/ParDo(DefaultConclude) into PAssert$1/RunChecks
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.953Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:25.978Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into SpannerIO.CreateTransaction/Create transaction
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.002Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.023Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.035Z: Fusing consumer
SpannerIO.CreateTransaction/Create transaction into
SpannerIO.CreateTransaction/Create.Values/Read(CreateSource)
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.057Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.089Z: Fusing consumer
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.457Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.491Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.520Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.530Z: Starting 1 workers in us-central1-f...
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.545Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.567Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.591Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.615Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.648Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Apr 10, 2018 3:08:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:26.739Z: Executing operation
SpannerIO.Read/Create.Values/Read(CreateSource)+SpannerIO.Read/Execute
query/Reshuffle/Pair with random key+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/Window.Into()/Window.Assign+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Reify+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Write
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:08:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:33.097Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:08:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:34.196Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Apr 10, 2018 3:08:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:34.682Z: Executing operation
SpannerIO.CreateTransaction/Create.Values/Read(CreateSource)+SpannerIO.CreateTransaction/Create
transaction+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Apr 10, 2018 3:08:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:39.810Z: Autoscaling: Resizing worker pool from 1 to
8.
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:08:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:43.833Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:08:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:46.709Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Apr 10, 2018 3:08:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:46.737Z: Resized worker pool to 1, though goal was
8. This could be a quota issue.
Apr 10, 2018 3:08:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:08:57.619Z: Autoscaling: Raised the number of workers
to 8 based on the rate of progress in the currently running step(s).
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:09:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:07.965Z: Workers have started successfully.
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:09:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:07.753Z: Workers have started successfully.
Apr 10, 2018 3:09:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:30.224Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Close
Apr 10, 2018 3:09:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:31.533Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Apr 10, 2018 3:09:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:31.619Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Apr 10, 2018 3:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.264Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Apr 10, 2018 3:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.345Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:09:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.779Z: Cleaning up.
Apr 10, 2018 3:09:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:35.839Z: Stopping worker pool...
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:09:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:42.983Z: Executing operation
SpannerIO.CreateTransaction/As
PCollectionView/Combine.GloballyAsSingletonView/CreateDataflowView
Apr 10, 2018 3:09:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:43.158Z: Executing operation SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/Read+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+SpannerIO.Read/Execute
query/Reshuffle/Reshuffle/ExpandIterable+SpannerIO.Read/Execute
query/Reshuffle/Values/Values/Map+SpannerIO.Read/Execute query/Read from Cloud
Spanner/Generate Partitions+SpannerIO.Read/Execute query/Read from Cloud
Spanner/Shuffle partitions/Pair with random key+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/Window.Into()/Window.Assign+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Reify+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Write
Apr 10, 2018 3:09:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:46.859Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle partitions/Reshuffle/GroupByKey/Close
Apr 10, 2018 3:09:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:46.941Z: Executing operation SpannerIO.Read/Execute
query/Read from Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/Read+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle
partitions/Reshuffle/GroupByKey/GroupByWindow+SpannerIO.Read/Execute query/Read
from Cloud Spanner/Shuffle
partitions/Reshuffle/ExpandIterable+SpannerIO.Read/Execute query/Read from
Cloud Spanner/Shuffle partitions/Values/Values/Map+SpannerIO.Read/Execute
query/Read from Cloud Spanner/Read from Partitions+Count
rows/WithKeys/AddKeys/Map+Count rows/Combine.perKey(Count)/GroupByKey+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Partial+Count
rows/Combine.perKey(Count)/GroupByKey/Reify+Count
rows/Combine.perKey(Count)/GroupByKey/Write
Apr 10, 2018 3:09:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:56.534Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Close
Apr 10, 2018 3:09:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:09:56.625Z: Executing operation Count
rows/Combine.perKey(Count)/GroupByKey/Read+Count
rows/Combine.perKey(Count)/Combine.GroupedValues+Count
rows/Combine.perKey(Count)/Combine.GroupedValues/Extract+Count
rows/Values/Values/Map+PAssert$1/CreateActual/FilterActuals/Window.Assign+PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous)+PAssert$1/CreateActual/GatherPanes/WithKeys/AddKeys/Map+PAssert$1/CreateActual/GatherPanes/Window.Into()/Window.Assign+PAssert$1/CreateActual/GatherPanes/GroupByKey/Reify+PAssert$1/CreateActual/GatherPanes/GroupByKey/Write+Count
rows/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Apr 10, 2018 3:10:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:01.704Z: Executing operation Count
rows/View.AsIterable/CreateDataflowView
Apr 10, 2018 3:10:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:01.899Z: Executing operation Count
rows/CreateVoid/Read(CreateSource)+Count
rows/ProduceDefault+PAssert$1/CreateActual/FilterActuals/Window.Assign+PAssert$1/CreateActual/GatherPanes/Reify.Window/ParDo(Anonymous)+PAssert$1/CreateActual/GatherPanes/WithKeys/AddKeys/Map+PAssert$1/CreateActual/GatherPanes/Window.Into()/Window.Assign+PAssert$1/CreateActual/GatherPanes/GroupByKey/Reify+PAssert$1/CreateActual/GatherPanes/GroupByKey/Write
Apr 10, 2018 3:10:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:03.781Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Close
Apr 10, 2018 3:10:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:03.866Z: Executing operation
PAssert$1/CreateActual/GatherPanes/GroupByKey/Read+PAssert$1/CreateActual/GatherPanes/GroupByKey/GroupByWindow+PAssert$1/CreateActual/GatherPanes/Values/Values/Map+PAssert$1/CreateActual/ExtractPane/Map+PAssert$1/CreateActual/Flatten.Iterables/FlattenIterables/FlatMap+PAssert$1/CreateActual/RewindowActuals/Window.Assign+PAssert$1/CreateActual/ParDo(Anonymous)+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:05.723Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:05.818Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:07.194Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Apr 10, 2018 3:10:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:07.265Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Apr 10, 2018 3:10:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:11.346Z: Executing operation
PAssert$1/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Apr 10, 2018 3:10:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:11.521Z: Executing operation
PAssert$1/Create.Values/Read(CreateSource)+PAssert$1/WindowToken/Window.Assign+PAssert$1/RunChecks+PAssert$1/VerifyAssertions/ParDo(DefaultConclude)
Apr 10, 2018 3:10:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:13.670Z: Cleaning up.
Apr 10, 2018 3:10:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:10:13.851Z: Stopping worker pool...
org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR
Apr 10, 2018 3:11:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:02.083Z: Autoscaling: Resized worker pool from 1 to
0.
Apr 10, 2018 3:11:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:02.115Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Apr 10, 2018 3:11:10 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-04-10_08_08_16-4270479033920952683 finished with status DONE.
Apr 10, 2018 3:11:10 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-04-10_08_08_16-4270479033920952683. Found 0 success, 0 failures out of 0
expected assertions.
Apr 10, 2018 3:11:11 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Writing batch of 500 entities
Apr 10, 2018 3:11:12 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Successfully wrote 500 entities
Apr 10, 2018 3:11:12 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Writing batch of 500 entities
Apr 10, 2018 3:11:13 PM
org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch
INFO: Successfully wrote 500 entities
Apr 10, 2018 3:11:13 PM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil
deleteAllEntities
INFO: Successfully deleted 1000 entities
Gradle Test Executor 126 finished executing tests.
org.apache.beam.sdk.io.gcp.spanner.SpannerReadIT > testRead STANDARD_ERROR
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.562Z: Autoscaling: Resized worker pool from 8 to
0.
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.586Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Apr 10, 2018 3:11:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-04-10T15:11:58.625Z: Worker pool stopped.
Apr 10, 2018 3:12:06 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-04-10_08_08_18-17222637697064452262 finished with status
DONE.
Apr 10, 2018 3:12:06 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-04-10_08_08_18-17222637697064452262. Found 1 success, 0 failures out of 1
expected assertions.
Gradle Test Executor 127 finished executing tests.
10 tests completed, 1 failed
:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest
FAILED
:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest
(Thread[Task worker for ':' Thread 2,5,main]) completed. Took 14 mins 11.772
secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest'.
> Process 'Gradle Test Executor 128' finished with non-zero exit value 137
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
See
https://docs.gradle.org/4.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 52m 14s
760 actionable tasks: 760 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Recording test results
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
