See
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/3401/display/redirect?page=changes>
Changes:
[github] [BEAM-9634] Add natural language analysis transform (#11611)
------------------------------------------
[...truncated 333.20 KB...]
INFO: 2020-05-14T18:51:16.562Z: Fusing consumer
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.591Z: Fusing consumer
Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.630Z: Fusing consumer
Reshuffle.ViaRandomKey/Values/Values/Map into
Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.670Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.707Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.743Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.778Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.813Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.850Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.887Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.940Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:16.996Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.045Z: Fusing consumer
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map
May 14, 2020 6:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.074Z: Fusing consumer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.116Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$0/GroupGlobally/GroupDummyAndContents/Read
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.153Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.182Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.221Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.262Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.301Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.329Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.369Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.414Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.447Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.481Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.511Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.549Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.591Z: Unzipping flatten s12-u72 for input
s13.org.apache.beam.sdk.values.PCollection.<init>:400#2dd2283a4fb68066-c70
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.629Z: Fusing unzipped copy of
Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign, through flatten
Calculate hashcode/Flatten.PCollections/Unzipped-1, into producer
Reshuffle.ViaRandomKey/Pair with random key
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.670Z: Fusing consumer
Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into
Reshuffle.ViaRandomKey/Pair with random key
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.706Z: Unzipping flatten s30-u77 for input
s32-reify-value36-c75
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.743Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Write, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents/Unzipped-1, into producer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.784Z: Fusing consumer Record time after reading
into TFRecordIO.Read/Read
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.824Z: Fusing consumer Transform bytes to
strings/Map into Record time after reading
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.862Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Transform bytes to strings/Map
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.903Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.939Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:17.981Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.020Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.058Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.095Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.126Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.165Z: Fusing consumer Delete test files into
Create.Values/Read(CreateSource)
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.195Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
May 14, 2020 6:51:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.229Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.259Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.683Z: Executing operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.721Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.754Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.767Z: Starting 5 ****s in us-central1-a...
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.792Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.817Z: Finished operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.829Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.838Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.838Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.858Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.867Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.893Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:18.927Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:19.133Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
May 14, 2020 6:51:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:19.169Z: Executing operation
TFRecordIO.Read/Read+Record time after reading+Transform bytes to
strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 14, 2020 6:51:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-05-14T18:51:36.157Z: Your project already contains 100
Dataflow-created metric descriptors and Stackdriver will not create new
Dataflow custom metrics for this job. Each unique user-defined metric name
(independent of the DoFn in which it is defined) produces a new metric
descriptor. To delete old / unused metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 14, 2020 6:51:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:51:50.570Z: Autoscaling: Raised the number of ****s to
5 based on the rate of progress in the currently running stage(s).
May 14, 2020 6:52:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:08.123Z: Workers have started successfully.
May 14, 2020 6:52:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:08.146Z: Workers have started successfully.
May 14, 2020 6:52:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:38.210Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
May 14, 2020 6:52:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:44.089Z: Finished operation
TFRecordIO.Read/Read+Record time after reading+Transform bytes to
strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 14, 2020 6:52:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:44.158Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 14, 2020 6:52:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:44.218Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 14, 2020 6:52:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:44.293Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+Reshuffle.ViaRandomKey/Pair
with random
key+Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:49.280Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+Reshuffle.ViaRandomKey/Pair
with random
key+Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:49.421Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:49.474Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:49.613Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+Reshuffle.ViaRandomKey/Pair with random
key+Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:51.383Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+Reshuffle.ViaRandomKey/Pair with random
key+Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:51.469Z: Executing operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:51.528Z: Finished operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 14, 2020 6:52:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:51.614Z: Executing operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.558Z: Finished operation
Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.633Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.668Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.691Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.724Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.762Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 14, 2020 6:52:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:54.800Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.240Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.310Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.366Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.434Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.445Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.510Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.559Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 14, 2020 6:52:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:57.630Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 14, 2020 6:53:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:59.229Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 14, 2020 6:53:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:59.807Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 14, 2020 6:53:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:52:59.950Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 14, 2020 6:53:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:53:00.018Z: Finished operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 14, 2020 6:53:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:53:00.164Z: Executing operation
Create.Values/Read(CreateSource)+Delete test files
May 14, 2020 6:53:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:53:00.728Z: Finished operation
Create.Values/Read(CreateSource)+Delete test files
May 14, 2020 6:53:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:53:00.936Z: Cleaning up.
May 14, 2020 6:53:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:53:01.027Z: Stopping **** pool...
May 14, 2020 6:54:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:54:47.295Z: Autoscaling: Resized **** pool from 5 to 0.
May 14, 2020 6:54:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-05-14T18:54:47.344Z: Worker pool stopped.
May 14, 2020 6:54:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-05-14_11_50_59-15333464591642818253 finished with status
DONE.
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): dfc5c3c2-5354-4571-a4e7-cc01c8b5ca08 and
timestamp: 2020-05-14T18:54:54.877000000Z:
Metric: Value:
dataset_size 1.01938E9
write_time 30.381
read_time 20.122
run_time 267.37
org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR
log4j:WARN No appenders could be found for logger
(org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 7,5,main]) completed. Took 8 mins 9.987 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> java.io.IOException: No space left on device
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 57s
90 actionable tasks: 57 executed, 33 from cache
Publishing build scan...
https://gradle.com/s/ktvzp6dhjmqbi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
