See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/368/display/redirect>
------------------------------------------ [...truncated 1.12 MB...] Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s37 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s38 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s39 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s40 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s41 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s42 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/CreateDataflowView as step s43 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s44 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s45 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/ParDo(Anonymous) as step s46 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WritePartitionUntriggered as step s47 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/Window.Into()/Window.Assign as step s48 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey as step s49 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable as step s50 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables) as step s51 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map as step s52 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign as step s53 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey as step s54 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/Values/Values/Map as step s55 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/ParDo(GarbageCollectTemporaryFiles) as step s56 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s57 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s58 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s59 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s60 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s61 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s62 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/CreateDataflowView as step s63 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s64 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s65 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/ParDo(Anonymous) as step s66 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WriteRenameUntriggered as step s67 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/Window.Into()/Window.Assign as step s68 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/GroupByKey as step s69 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/ExpandIterable as step s70 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/ParMultiDo(WriteTables) as step s71 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/WithKeys/AddKeys/Map as step s72 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/Window.Into()/Window.Assign as step s73 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/GroupByKey as step s74 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/Values/Values/Map as step s75 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/ParDo(GarbageCollectTemporaryFiles) as step s76 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/CreateEmptyFailedInserts/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s77 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 0) cannot estimate its size, skips the initial splits. Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/CreateEmptyFailedInserts/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s78 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding DropInputs as step s79 Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/ Sep 12, 2018 6:55:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <265581 bytes, hash 4OdKPNxguWyIjcZ2r7JQQw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4OdKPNxguWyIjcZ2r7JQQw.pb Dataflow SDK version: 2.8.0-SNAPSHOT Submitted job: 2018-09-11_23_55_22-6385441353809393036 Sep 12, 2018 6:55:24 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-09-11_23_55_22-6385441353809393036?project=apache-beam-testing Sep 12, 2018 6:55:24 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2018-09-11_23_55_22-6385441353809393036 ========================================================================================== Run started 2018-09-12T06:10:21.558Z and ran for PT2703.003S Default configuration: {"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1} Configurations: Conf Description 0000 query:0; exportSummaryToBigQuery:true; numEvents:10000000 0001 query:1; exportSummaryToBigQuery:true; numEvents:10000000 0002 query:2; exportSummaryToBigQuery:true; numEvents:10000000 0003 query:3; exportSummaryToBigQuery:true; numEvents:10000000 0004 query:4; exportSummaryToBigQuery:true; numEvents:1000000 0005 query:5; exportSummaryToBigQuery:true; numEvents:10000000 0006 query:6; exportSummaryToBigQuery:true; numEvents:1000000 0007 query:7; exportSummaryToBigQuery:true; numEvents:10000000 0008 query:8; exportSummaryToBigQuery:true; numEvents:10000000 0009 query:9; exportSummaryToBigQuery:true; numEvents:1000000 0010 query:10; exportSummaryToBigQuery:true; numEvents:10000000 0011 query:11; exportSummaryToBigQuery:true; numEvents:10000000 0012 query:12; exportSummaryToBigQuery:true; numEvents:10000000 Performance: Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) 0000 39.6 252544.4 10000000 0001 19.8 505254.6 9200000 0002 11.6 865576.0 83527 0003 58.7 170279.4 59378 0004 72.2 13841.4 140 0005 125.9 79428.7 4567 0006 50.3 19896.9 14072 0007 126.0 79355.6 100 0008 55.9 178782.1 92773 0009 57.8 17296.3 32529 0010 104.0 96174.2 100 0011 2018.9 66667.2 1594055 *** Streaming query was stuck for 10 min *** *** Streaming query was stuck for 10 min *** *** Streaming query was stuck for 10 min *** *** Streaming query was stuck for 10 min *** *** Streaming query was stuck for 11 min *** *** Streaming query was stuck for 11 min *** *** Streaming query was stuck for 11 min *** *** Streaming query was stuck for 12 min *** *** Streaming query was stuck for 12 min *** *** Streaming query was stuck for 12 min *** *** Streaming query was stuck for 12 min *** *** Streaming query was stuck for 13 min *** *** Streaming query was stuck for 13 min *** *** Streaming query was stuck for 13 min *** *** Streaming query was stuck for 13 min *** *** Streaming query was stuck for 14 min *** *** Streaming query was stuck for 14 min *** *** Streaming query was stuck for 14 min *** *** Streaming query was stuck for 14 min *** *** Streaming query was stuck for 15 min *** *** Streaming query was stuck for 15 min *** *** Streaming query was stuck for 15 min *** *** Streaming query was stuck for 15 min *** *** Streaming query was stuck for 16 min *** *** Streaming query was stuck for 16 min *** *** Streaming query was stuck for 16 min *** *** Streaming query was stuck for 16 min *** *** Streaming query was stuck for 17 min *** *** Streaming query was stuck for 17 min *** *** Streaming query was stuck for 17 min *** Exception in thread "main" *** Streaming query was stuck for 18 min *** *** Streaming query was stuck for 18 min *** *** Streaming query was stuck for 18 min *** *** Streaming query was stuck for 18 min *** *** Streaming query was stuck for 19 min *** *** Streaming query was stuck for 19 min *** *** Streaming query was stuck for 19 min *** *** Streaming query was stuck for 19 min *** *** Streaming query was stuck for 20 min *** *** Streaming query was stuck for 20 min *** *** Streaming query was stuck for 20 min *** *** Streaming query was stuck for 20 min *** *** Streaming query was stuck for 21 min *** *** Streaming query was stuck for 21 min *** *** Streaming query was stuck for 21 min *** *** Streaming query was stuck for 21 min *** *** Streaming query was stuck for 22 min *** *** Streaming query was stuck for 22 min *** *** Streaming query was stuck for 22 min *** *** Streaming query was stuck for 22 min *** *** Streaming query was stuck for 23 min *** *** Streaming query was stuck for 23 min *** *** Streaming query was stuck for 23 min *** *** Streaming query was stuck for 24 min *** *** Streaming query was stuck for 24 min *** *** Streaming query was stuck for 24 min *** *** Streaming query was stuck for 24 min *** *** Streaming query was stuck for 25 min *** *** Streaming query was stuck for 25 min *** *** Streaming query was stuck for 25 min *** *** Streaming query was stuck for 25 min *** *** Streaming query was stuck for 26 min *** *** Streaming query was stuck for 26 min *** *** Streaming query was stuck for 26 min *** *** Streaming query was stuck for 26 min *** java.lang.RuntimeException: Execution was not successful at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:174) *** Streaming query was stuck for 27 min *** *** Streaming query was stuck for 27 min *** at org.apache.beam.sdk.nexmark.Main.main(Main.java:477) *** Streaming query was stuck for 27 min *** *** Streaming query was stuck for 27 min *** *** Streaming query was stuck for 28 min *** *** Streaming query was stuck for 28 min *** *** Streaming query was stuck for 28 min *** *** Streaming query was stuck for 29 min *** *** Streaming query was stuck for 29 min *** *** Streaming query was stuck for 29 min *** *** Streaming query was stuck for 29 min *** *** Streaming query was stuck for 30 min *** 0012 208.2 48027.0 667761 ========================================================================================== > Task :beam-sdks-java-nexmark:run FAILED :beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 45 mins 3.889 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-sdks-java-nexmark:run'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 46m 52s 65 actionable tasks: 15 executed, 50 up-to-date Publishing build scan... https://gradle.com/s/v2lnto224luzk Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure
