See
<https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/930/display/redirect?page=changes>
Changes:
[lukasz.gajowy] [BEAM-5356] Move metrics related functionality to common place
[lukasz.gajowy] [BEAM-5356] Move step & source common code to one place
[lukasz.gajowy] [BEAM-5356] Provide common load test pipeline options
[lukasz.gajowy] [BEAM-5356] Add CoGroupByKey Load Test
------------------------------------------
[...truncated 541.23 KB...]
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly
as step s20
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
as step s21
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step
s22
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Select as step s23
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Forget as step s24
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s25
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s26
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s27
Nov 06, 2018 9:09:16 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s28
Nov 06, 2018 9:09:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 06, 2018 9:09:17 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <96595 bytes, hash emlRRFgzYADY5CxgskLLkw> to
gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-emlRRFgzYADY5CxgskLLkw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 06, 2018 9:09:19 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 114 files cached, 27 files newly uploaded
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4.ReadBounded as step s1
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Monitor as step s2
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Snoop as step s3
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Filter/ParDo(Anonymous) as step s4
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/Window/Window.Assign as step s5
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
Query4/Query4.WinningBids/justNewAuctions/IsNewAuction/ParDo(Anonymous) as step
s6
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/justNewAuctions/AsAuction as step s7
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/AuctionById: as step s8
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/justBids/IsBid/ParDo(Anonymous) as step
s9
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/justBids/AsBid as step s10
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/BidByAuction as step s11
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/CoGroupByKey/MakeUnionTable0 as step s12
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/CoGroupByKey/MakeUnionTable1 as step s13
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/CoGroupByKey/Flatten as step s14
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/CoGroupByKey/GBK as step s15
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/CoGroupByKey/ConstructCoGbkResultFn as
step s16
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBids/Query4.WinningBids.Join as step s17
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.WinningBidsMonitor as step s18
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Rekey as step s19
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Window.Into()/Window.Assign as step s20
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/AddNonce as step s21
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/PreCombineHot/GroupByKey as
step s22
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
Query4/Combine.perKeyWithFanout(Mean)/PreCombineHot/Combine.GroupedValues as
step s23
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/StripNonce/Map as step s24
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/Window.Remerge/Identity/Map
as step s25
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/PrepareCold/Map as step s26
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/Flatten.PCollections as step
s27
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Combine.perKeyWithFanout(Mean)/PostCombine/GroupByKey as
step s28
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
Query4/Combine.perKeyWithFanout(Mean)/PostCombine/Combine.GroupedValues as step
s29
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Project as step s30
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Forget as step s31
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Debug as step s32
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4/Query4.Stamp as step s33
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4.Format as step s34
Nov 06, 2018 9:09:19 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query4.DevNull as step s35
Nov 06, 2018 9:09:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 06, 2018 9:09:19 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <86612 bytes, hash KfR3Q2PXBTwjnCXDF-RBoQ> to
gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-KfR3Q2PXBTwjnCXDF-RBoQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-06T09:09:11.576Z: The requested max number of workers (4) is
ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:14.202Z: Checking permissions granted to controller
Service Account.
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:21.773Z: Worker configuration: n1-standard-1 in
us-central1-b.
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.245Z: Expanding CoGroupByKey operations into
optimizable parts.
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.387Z: Expanding GroupByKey operations into
optimizable parts.
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.425Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.508Z: Fusing adjacent ParDo, Read, Write, and Flatten
operations
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.536Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor
into JoinToFiles.ReadBounded
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.572Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop
into JoinToFiles/JoinToFiles.Monitor
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.611Z: Fusing consumer
JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.648Z: Fusing consumer JoinToFiles/justBids/AsBid into
JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.686Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp
into JoinToFiles/JoinToFiles.Debug
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.723Z: Fusing consumer JoinToFiles.DevNull into
JoinToFiles.Format
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.758Z: Fusing consumer JoinToFiles.Format into
JoinToFiles/JoinToFiles.Stamp
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.787Z: Fusing consumer JoinToFiles/JoinToFiles.Debug
into JoinToFiles/JoinToFiles.Forget
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.825Z: Fusing consumer
JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/justBids/AsBid
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.855Z: Fusing consumer JoinToFiles/JoinToFiles.Forget
into JoinToFiles/JoinToFiles.JoinToFiles
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.891Z: Fusing consumer
JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into
JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.919Z: Fusing consumer
JoinToFiles/View.AsMap/GBKaSVForSize/Write into
JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.957Z: Fusing consumer
JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into
JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:22.995Z: Fusing consumer
JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.030Z: Fusing consumer
NexmarkUtils.GenerateSideInputData/MapElements/Map into
NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.068Z: Fusing consumer
JoinToFiles/View.AsMap/GBKaSVForKeys/Write into
JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.105Z: Fusing consumer
JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into
JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.143Z: Fusing consumer
JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.538Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.578Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.616Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.627Z: Starting 4 workers in us-central1-b...
Nov 06, 2018 9:09:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:23.855Z: Executing operation
NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 06, 2018 9:09:36 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:09:33.855Z: Autoscaling: Raised the number of workers to 0
based on the rate of progress in the currently running step(s).
Nov 06, 2018 9:10:04 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:10:02.389Z: Workers have started successfully.
Nov 06, 2018 9:10:36 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:10:35.612Z: Autoscaling: Raised the number of workers to 4
based on the rate of progress in the currently running step(s).
Nov 06, 2018 9:10:38 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:10:37.712Z: Workers have started successfully.
Nov 06, 2018 9:11:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:22.649Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 06, 2018 9:11:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:22.729Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 06, 2018 9:11:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:30.255Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 06, 2018 9:11:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:30.303Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 06, 2018 9:11:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:30.356Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 06, 2018 9:11:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:30.404Z: Executing operation
JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 06, 2018 9:11:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:38.952Z: Executing operation
JoinToFiles/View.AsMap/Flatten.PCollections
Nov 06, 2018 9:11:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:39.166Z: Executing operation
JoinToFiles/View.AsMap/CreateDataflowView
Nov 06, 2018 9:11:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:11:39.368Z: Executing operation
JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Forget+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 06, 2018 9:12:44 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:12:42.177Z: Cleaning up.
Nov 06, 2018 9:12:44 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-06T09:12:42.590Z: Stopping worker pool...
Nov 06, 2018 9:13:12 AM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-06T09:13:12.553Z Waiting for main pipeline to 'finish'
2018-11-06T09:13:12.593Z RUNNING JoinToFiles
2018-11-06T09:13:13.619Z new perf
{"snapshots":null,"runtimeSec":60.553,"numEvents":10000000,"eventsPerSec":165144.58408336499,"eventBytesPerSec":2.1197379105907224E7,"numResults":9200000,"resultsPerSec":151933.01735669578,"resultBytesPerSec":5415615.989298631,"startupDelaySec":-91.874,"processingDelaySec":0.835,"shutdownDelaySec":31.321,"timeDilation":16.513632685416084,"errors":null,"jobId":null}
2018-11-06T09:13:28.670Z RUNNING JoinToFiles
2018-11-06T09:13:29.699Z no activity
2018-11-06T09:13:44.763Z RUNNING JoinToFiles
2018-11-06T09:13:45.756Z no activity
2018-11-06T09:14:00.801Z RUNNING JoinToFiles
2018-11-06T09:14:01.759Z no activity
2018-11-06T09:14:16.805Z RUNNING JoinToFiles
2018-11-06T09:14:17.738Z no activity
2018-11-06T09:14:32.797Z RUNNING JoinToFiles
2018-11-06T09:14:33.813Z no activity
2018-11-06T09:14:48.846Z RUNNING JoinToFiles
2018-11-06T09:14:50.041Z no activity
2018-11-06T09:15:05.107Z RUNNING JoinToFiles
2018-11-06T09:15:06.111Z no activity
2018-11-06T09:15:21.156Z RUNNING JoinToFiles
2018-11-06T09:15:22.278Z no activity
2018-11-06T09:15:37.485Z RUNNING JoinToFiles
2018-11-06T09:15:38.418Z no activity
2018-11-06T09:15:53.452Z RUNNING JoinToFiles
2018-11-06T09:15:54.380Z no activity
2018-11-06T09:16:09.424Z RUNNING JoinToFiles
2018-11-06T09:16:10.462Z no activity
2018-11-06T09:16:25.506Z RUNNING JoinToFiles
2018-11-06T09:16:26.470Z no activity
2018-11-06T09:16:41.510Z RUNNING JoinToFiles
2018-11-06T09:16:42.436Z no activity
2018-11-06T09:16:57.479Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main])
completed. Took 8 mins 22.867 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java''
> finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 48s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/jkcuqv4agyslw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org
For additional commands, e-mail: builds-h...@beam.apache.org
