See
<https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/844/display/redirect?page=changes>
Changes:
[lukasz.gajowy] [BEAM-5875] Add missing dependency in test-utils module
------------------------------------------
[...truncated 1.02 MB...]
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.726Z: Fusing consumer Query0/Query0.Snoop into
Query0/Query0.Monitor
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.728Z: Fusing consumer Query0/Query0.Stamp into
Query0/Query0.Debug
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.730Z: Fusing consumer Query0/Query0.Debug into
Query0/Query0.Forget
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.732Z: Fusing consumer Query0.DevNull into
Query0.Format
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.734Z: Fusing consumer Query0.Format into
Query0/Query0.Stamp
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.736Z: Fusing consumer Query0/Query0.Monitor into
Query0.ReadUnbounded/StripIds
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.738Z: Fusing consumer Query0/Query0.Serialize into
Query0/Query0.Snoop
Oct 27, 2018 9:49:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.740Z: Fusing consumer Query0/Query0.Forget into
Query0/Query0.Serialize
Oct 27, 2018 9:49:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:52.959Z: Starting 4 workers...
Oct 27, 2018 9:50:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:49:58.841Z: Executing operation
Query0.ReadUnbounded/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Query0.ReadUnbounded/StripIds+Query0/Query0.Monitor+Query0/Query0.Snoop+Query0/Query0.Serialize+Query0/Query0.Forget+Query0/Query0.Debug+Query0/Query0.Stamp+Query0.Format+Query0.DevNull
Oct 27, 2018 9:50:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:14.876Z: Workers have started successfully.
Oct 27, 2018 9:50:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:14.673Z: Workers have started successfully.
Oct 27, 2018 9:50:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:14.069Z: Workers have started successfully.
Oct 27, 2018 9:50:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:15.635Z: Workers have started successfully.
Oct 27, 2018 9:50:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:15.180Z: Workers have started successfully.
Oct 27, 2018 9:50:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:17.009Z: Workers have started successfully.
Oct 27, 2018 9:50:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:18.454Z: Workers have started successfully.
Oct 27, 2018 9:50:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:19.576Z: Workers have started successfully.
Oct 27, 2018 9:50:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:21.383Z: Workers have started successfully.
Oct 27, 2018 9:50:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:22.213Z: Workers have started successfully.
Oct 27, 2018 9:50:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:32.103Z: Workers have started successfully.
Oct 27, 2018 9:50:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:33.806Z: Workers have started successfully.
Oct 27, 2018 9:50:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:50:35.019Z: Workers have started successfully.
Oct 27, 2018 9:52:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:05.826Z: Cleaning up.
Oct 27, 2018 9:52:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:05.863Z: Worker pool stopped.
Oct 27, 2018 9:52:06 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:05.877Z: Stopping worker pool...
Oct 27, 2018 9:52:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:19.834Z: Cleaning up.
Oct 27, 2018 9:52:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:19.854Z: Worker pool stopped.
Oct 27, 2018 9:52:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:19.865Z: Stopping worker pool...
Oct 27, 2018 9:52:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:23.765Z: Cleaning up.
Oct 27, 2018 9:52:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:23.804Z: Worker pool stopped.
Oct 27, 2018 9:52:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:23.822Z: Stopping worker pool...
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:24.305Z: Cleaning up.
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:24.332Z: Worker pool stopped.
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:24.344Z: Stopping worker pool...
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:26.353Z: Cleaning up.
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:26.382Z: Worker pool stopped.
Oct 27, 2018 9:52:26 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:26.394Z: Stopping worker pool...
Oct 27, 2018 9:52:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.049Z: Cleaning up.
Oct 27, 2018 9:52:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.078Z: Worker pool stopped.
Oct 27, 2018 9:52:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.092Z: Stopping worker pool...
Oct 27, 2018 9:52:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.763Z: Cleaning up.
Oct 27, 2018 9:52:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.795Z: Worker pool stopped.
Oct 27, 2018 9:52:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:52:34.811Z: Stopping worker pool...
Oct 27, 2018 9:53:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:01.784Z: Cleaning up.
Oct 27, 2018 9:53:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:01.812Z: Worker pool stopped.
Oct 27, 2018 9:53:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:01.825Z: Stopping worker pool...
Oct 27, 2018 9:53:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:27.904Z: Cleaning up.
Oct 27, 2018 9:53:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:27.936Z: Worker pool stopped.
Oct 27, 2018 9:53:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-10-27T21:53:27.949Z: Stopping worker pool...
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:32.751Z Waiting for main pipeline to 'finish'
2018-10-27T21:53:32.781Z RUNNING Query8
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:32.788Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:32.818Z RUNNING Query11
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:32.870Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:32.894Z RUNNING Query4
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:32.943Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:32.944Z Waiting for main pipeline to 'finish'
2018-10-27T21:53:32.967Z RUNNING Query10
2018-10-27T21:53:32.967Z RUNNING Query1
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:32.989Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
Oct 27, 2018 9:53:32 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:32.995Z Waiting for main pipeline to 'finish'
2018-10-27T21:53:33.023Z RUNNING Query5
2018-10-27T21:53:33.026Z RUNNING Query9
Oct 27, 2018 9:53:33 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:33.049Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:33.077Z RUNNING Query3
2018-10-27T21:53:33.563Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.584Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.592Z 1 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2018-10-27T21:49:22.179Z and ran for PT251.450S
Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:0; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:1; exportSummaryToBigQuery:true; numEvents:10000000
0002 query:2; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:3; exportSummaryToBigQuery:true; numEvents:10000000
Exception in thread "main" 0004 query:4; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:5; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:6; exportSummaryToBigQuery:true; numEvents:1000000
0007 query:7; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:8; exportSummaryToBigQuery:true; numEvents:10000000
0009 query:9; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:10; exportSummaryToBigQuery:true; numEvents:10000000
java.lang.RuntimeException: java.lang.IllegalStateException: More than one
metric matches name: fatal in namespace Query11.
0011 query:11; exportSummaryToBigQuery:true; numEvents:10000000
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:144)
0012 query:12; exportSummaryToBigQuery:true; numEvents:10000000
at org.apache.beam.sdk.nexmark.Main.main(Main.java:477)
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 *** not run ***
0002 *** not run ***
0003 *** not run ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
==========================================================================================
Caused by: java.lang.IllegalStateException: More than one metric matches name:
fatal in namespace Query11.
at
org.apache.beam.repackaged.beam_sdks_java_test_utils.com.google.common.base.Preconditions.checkState(Preconditions.java:444)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.checkIfMetricResultIsUnique(MetricsReader.java:127)
at
org.apache.beam.sdk.testutils.metrics.MetricsReader.getCounterMetric(MetricsReader.java:65)
at
org.apache.beam.sdk.nexmark.NexmarkLauncher.monitor(NexmarkLauncher.java:447)
at
org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1156)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:108)
at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:96)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-10-27T21:53:33.645Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.689Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.717Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.766Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:33.788Z 1 samples not enough to calculate steady-state event
rate
Oct 27, 2018 9:53:34 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:34.507Z Waiting for main pipeline to 'finish'
2018-10-27T21:53:34.539Z RUNNING Query7
2018-10-27T21:53:35.259Z 1 samples not enough to calculate steady-state event
rate
Oct 27, 2018 9:53:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:36.474Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:36.502Z RUNNING Query6
2018-10-27T21:53:37.221Z 1 samples not enough to calculate steady-state event
rate
2018-10-27T21:53:39.776Z Waiting for main pipeline to 'finish'
Oct 27, 2018 9:53:39 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:39.800Z RUNNING Query12
2018-10-27T21:53:40.488Z 1 samples not enough to calculate steady-state event
rate
Oct 27, 2018 9:53:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:40.567Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:40.589Z RUNNING Query2
2018-10-27T21:53:41.159Z 1 samples not enough to calculate steady-state event
rate
Oct 27, 2018 9:53:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2018-10-27T21:53:47.371Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-10-27T21:53:47.400Z RUNNING Query0
2018-10-27T21:53:48.184Z 1 samples not enough to calculate steady-state event
rate
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 9,5,main])
completed. Took 4 mins 26.796 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_172/bin/java''
> finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 2s
68 actionable tasks: 15 executed, 53 up-to-date
Publishing build scan...
https://gradle.com/s/fwmfgjfi7vxh4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
