See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/845/display/redirect>
------------------------------------------ [...truncated 1.15 MB...] Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query1/Query1.Debug as step s9 Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query1/Query1.Stamp as step s10 Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query1.Format as step s11 Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query1.DevNull as step s12 Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/ Oct 28, 2018 12:14:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <39613 bytes, hash VLaxU3WcCKEo7UpRgChWfg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-VLaxU3WcCKEo7UpRgChWfg.pb Dataflow SDK version: 2.9.0-SNAPSHOT Oct 28, 2018 12:14:54 AM org.apache.beam.runners.dataflow.DataflowRunner run Submitted job: 2018-10-27_17_14_53-884993445729491414 INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-27_17_14_53-884993445729491414?project=apache-beam-testing Oct 28, 2018 12:14:54 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2018-10-27_17_14_53-884993445729491414 Oct 28, 2018 12:14:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:14:55.199Z: Workers have started successfully. Oct 28, 2018 12:14:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:14:56.479Z: Workers have started successfully. Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:14:56.732Z: Checking permissions granted to controller Service Account. Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.093Z: Worker configuration: n1-standard-4 in us-central1-b. Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.561Z: Expanding CoGroupByKey operations into optimizable parts. Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.567Z: Expanding SplittableProcessKeyed operations into optimizable parts. Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.570Z: Expanding GroupByKey operations into streaming Read/Write steps Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.573Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.593Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.596Z: Fusing consumer Query1.ReadUnbounded/StripIds into Query1.ReadUnbounded/DataflowRunner.StreamingUnboundedRead.ReadWithIds Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.598Z: Fusing consumer Query1/Query1.Snoop into Query1/Query1.Monitor Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.601Z: Fusing consumer Query1/Query1.Forget into Query1/Query1.ToEuros Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.603Z: Fusing consumer Query1/Query1.ToEuros into Query1/justBids/AsBid Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.606Z: Fusing consumer Query1.DevNull into Query1.Format Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.608Z: Fusing consumer Query1.Format into Query1/Query1.Stamp Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.611Z: Fusing consumer Query1/Query1.Stamp into Query1/Query1.Debug Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.614Z: Fusing consumer Query1/Query1.Debug into Query1/Query1.Forget Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.616Z: Fusing consumer Query1/Query1.Monitor into Query1.ReadUnbounded/StripIds Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.618Z: Fusing consumer Query1/justBids/IsBid/ParDo(Anonymous) into Query1/Query1.Snoop Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.623Z: Fusing consumer Query1/justBids/AsBid into Query1/justBids/IsBid/ParDo(Anonymous) Oct 28, 2018 12:15:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:00.849Z: Starting 4 workers... Oct 28, 2018 12:15:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:06.965Z: Executing operation Query1.ReadUnbounded/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Query1.ReadUnbounded/StripIds+Query1/Query1.Monitor+Query1/Query1.Snoop+Query1/justBids/IsBid/ParDo(Anonymous)+Query1/justBids/AsBid+Query1/Query1.ToEuros+Query1/Query1.Forget+Query1/Query1.Debug+Query1/Query1.Stamp+Query1.Format+Query1.DevNull Oct 28, 2018 12:15:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:15:47.453Z: Workers have started successfully. Oct 28, 2018 12:16:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:33.792Z: Cleaning up. Oct 28, 2018 12:16:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:33.814Z: Worker pool stopped. Oct 28, 2018 12:16:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:33.826Z: Stopping worker pool... Oct 28, 2018 12:16:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:34.847Z: Cleaning up. Oct 28, 2018 12:16:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:34.878Z: Worker pool stopped. Oct 28, 2018 12:16:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:34.894Z: Stopping worker pool... Oct 28, 2018 12:16:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:37.328Z: Cleaning up. Oct 28, 2018 12:16:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:37.352Z: Worker pool stopped. Oct 28, 2018 12:16:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:37.363Z: Stopping worker pool... Oct 28, 2018 12:16:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:46.460Z: Cleaning up. Oct 28, 2018 12:16:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:46.482Z: Worker pool stopped. Oct 28, 2018 12:16:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:16:46.494Z: Stopping worker pool... Oct 28, 2018 12:17:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:17:06.861Z: Cleaning up. Oct 28, 2018 12:17:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:17:06.978Z: Worker pool stopped. Oct 28, 2018 12:17:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:17:07.002Z: Stopping worker pool... Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:17:27.373Z Waiting for main pipeline to 'finish' WARNING: No terminal state was returned. State value RUNNING Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.375Z Waiting for main pipeline to 'finish' Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.376Z Waiting for main pipeline to 'finish' Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.392Z Waiting for main pipeline to 'finish' Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.395Z Waiting for main pipeline to 'finish' 2018-10-28T00:17:27.401Z RUNNING Query9 2018-10-28T00:17:27.404Z RUNNING Query5 2018-10-28T00:17:27.405Z RUNNING Query11 2018-10-28T00:17:27.418Z RUNNING Query6 2018-10-28T00:17:27.420Z RUNNING Query2 Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:17:27.439Z Waiting for main pipeline to 'finish' WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.468Z RUNNING Query10 Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.479Z Waiting for main pipeline to 'finish' 2018-10-28T00:17:27.509Z RUNNING Query3 2018-10-28T00:17:27.537Z Waiting for main pipeline to 'finish' Oct 28, 2018 12:17:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:27.567Z RUNNING Query7 ========================================================================================== Run started 2018-10-28T00:13:16.646Z and ran for PT251.152S Default configuration: {"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1} Configurations: Conf Description 0000 query:0; exportSummaryToBigQuery:true; numEvents:10000000 Exception in thread "main" 0001 query:1; exportSummaryToBigQuery:true; numEvents:10000000 0002 query:2; exportSummaryToBigQuery:true; numEvents:10000000 java.lang.RuntimeException: java.lang.IllegalStateException: More than one metric matches name: result.elements in namespace Query10.Results. 0003 query:3; exportSummaryToBigQuery:true; numEvents:10000000 at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:144) 0004 query:4; exportSummaryToBigQuery:true; numEvents:1000000 at org.apache.beam.sdk.nexmark.Main.main(Main.java:477) 0005 query:5; exportSummaryToBigQuery:true; numEvents:10000000 0006 query:6; exportSummaryToBigQuery:true; numEvents:1000000 0007 query:7; exportSummaryToBigQuery:true; numEvents:10000000 0008 query:8; exportSummaryToBigQuery:true; numEvents:10000000 0009 query:9; exportSummaryToBigQuery:true; numEvents:1000000 0010 query:10; exportSummaryToBigQuery:true; numEvents:10000000 0011 query:11; exportSummaryToBigQuery:true; numEvents:10000000 0012 query:12; exportSummaryToBigQuery:true; numEvents:10000000 Performance: Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) 0000 *** not run *** 0001 *** not run *** 0002 *** not run *** 0003 *** not run *** 0004 *** not run *** 0005 *** not run *** 0006 *** not run *** 0007 *** not run *** 0008 *** not run *** 0009 *** not run *** 0010 *** not run *** 0011 *** not run *** 0012 *** not run *** ========================================================================================== Caused by: java.lang.IllegalStateException: More than one metric matches name: result.elements in namespace Query10.Results. at org.apache.beam.repackaged.beam_sdks_java_test_utils.com.google.common.base.Preconditions.checkState(Preconditions.java:444) at org.apache.beam.sdk.testutils.metrics.MetricsReader.checkIfMetricResultIsUnique(MetricsReader.java:127) at org.apache.beam.sdk.testutils.metrics.MetricsReader.getCounterMetric(MetricsReader.java:65) at org.apache.beam.sdk.nexmark.NexmarkLauncher.currentPerf(NexmarkLauncher.java:257) at org.apache.beam.sdk.nexmark.NexmarkLauncher.monitor(NexmarkLauncher.java:435) at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1156) at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:108) at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:96) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-10-28T00:17:28.106Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.175Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.180Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.217Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.247Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.248Z 1 samples not enough to calculate steady-state event rate 2018-10-28T00:17:28.327Z 1 samples not enough to calculate steady-state event rate Oct 28, 2018 12:17:33 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:17:33.589Z Waiting for main pipeline to 'finish' WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:33.615Z RUNNING Query0 2018-10-28T00:17:34.234Z 1 samples not enough to calculate steady-state event rate Oct 28, 2018 12:17:38 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:17:38.680Z Waiting for main pipeline to 'finish' WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:38.704Z RUNNING Query8 2018-10-28T00:17:39.391Z 1 samples not enough to calculate steady-state event rate Oct 28, 2018 12:17:39 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:17:39.606Z WARNING: No terminal state was returned. State value RUNNING Waiting for main pipeline to 'finish' 2018-10-28T00:17:39.636Z RUNNING Query12 2018-10-28T00:17:40.315Z 1 samples not enough to calculate steady-state event rate Oct 28, 2018 12:17:44 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:17:44.669Z Waiting for main pipeline to 'finish' 2018-10-28T00:17:44.702Z RUNNING Query4 Oct 28, 2018 12:18:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:18:22.233Z: Cleaning up. Oct 28, 2018 12:18:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:18:22.295Z: Worker pool stopped. Oct 28, 2018 12:18:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-10-28T00:18:22.331Z: Stopping worker pool... Oct 28, 2018 12:18:54 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish 2018-10-28T00:18:54.901Z Waiting for main pipeline to 'finish' WARNING: No terminal state was returned. State value RUNNING 2018-10-28T00:18:54.930Z RUNNING Query1 2018-10-28T00:18:55.592Z 1 samples not enough to calculate steady-state event rate > Task :beam-sdks-java-nexmark:run FAILED :beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5 mins 39.752 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-sdks-java-nexmark:run'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_172/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 7m 12s 68 actionable tasks: 15 executed, 53 up-to-date Publishing build scan... https://gradle.com/s/2roau4bt3jqro Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
