See
<https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/2532/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-7110] Add Spark master option to SparkJobServerDriver
[kcweaver] [BEAM-7133] make Spark job server Gradle task pass along system
[pabloem] Add some pydoc for runner classes in Python
------------------------------------------
[...truncated 966.70 KB...]
Apr 26, 2019 10:39:53 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:39:53 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:39:53 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:39:53 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:39:54.172Z no activity
2019-04-26T22:40:09.222Z RUNNING Query0
Apr 26, 2019 10:40:09 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:40:09 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:40:09 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:40:09 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:40:09.939Z no activity
2019-04-26T22:40:24.984Z RUNNING Query0
Apr 26, 2019 10:40:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:40:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:40:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:40:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:40:25.696Z no activity
2019-04-26T22:40:40.735Z RUNNING Query0
Apr 26, 2019 10:40:40 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:40:40 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:40:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:40:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:40:41.408Z no activity
2019-04-26T22:40:56.445Z RUNNING Query0
Apr 26, 2019 10:40:56 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:40:56 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:40:56 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:40:56 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:40:57.070Z no activity
2019-04-26T22:41:12.104Z RUNNING Query0
Apr 26, 2019 10:41:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:41:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:41:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:41:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:41:12.879Z no activity
2019-04-26T22:41:27.956Z RUNNING Query0
Apr 26, 2019 10:41:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:41:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:41:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:41:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:41:28.642Z no activity
2019-04-26T22:41:43.705Z RUNNING Query0
Apr 26, 2019 10:41:43 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:41:43 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:41:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:41:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:41:44.468Z no activity
2019-04-26T22:41:59.519Z RUNNING Query0
Apr 26, 2019 10:41:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:41:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:41:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:41:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:42:00.234Z no activity
2019-04-26T22:42:15.265Z RUNNING Query0
Apr 26, 2019 10:42:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:42:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:42:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:42:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:42:16.002Z no activity
2019-04-26T22:42:31.054Z RUNNING Query0
Apr 26, 2019 10:42:31 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:42:31 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:42:31 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:42:31 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:42:31.730Z no activity
2019-04-26T22:42:46.762Z RUNNING Query0
Apr 26, 2019 10:42:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Apr 26, 2019 10:42:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:42:47 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Apr 26, 2019 10:42:47 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2019-04-26T22:42:47.416Z no activity
2019-04-26T22:43:02.463Z FAILED Query0
Apr 26, 2019 10:43:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
==========================================================================================
Run started 2019-04-26T21:13:25.222Z and ran for PT5377.277S
Apr 26, 2019 10:43:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Apr 26, 2019 10:43:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Default configuration:
Apr 26, 2019 10:43:02 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardMinutes":10,"standardDays":0,"standardHours":0,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 65.0 153957.5 9200000
0002 44.4 225093.4 83527
0003 80.9 123615.5 59507
0004 56.3 17757.9 140
0005 77.8 128505.0 4567
0006 59.6 16774.3 8902
0007 113.8 87891.2 100
0008 53.6 186410.7 92833
0009 51.4 19440.9 32548
0010 87.6 114092.7 -1
*** Job was unexpectedly updated ***
0011 75.0 133320.9 1839657
0012 48.8 204876.0 199929
0013 62.0 161415.3 9200000
0014 130.8 76434.5 9200000
==========================================================================================
==========================================================================================
Run started 2019-04-26T21:13:25.222Z and ran for PT5381.108S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardMinutes":10,"standardDays":0,"standardHours":0,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 65.0 153957.5 9200000
0002 44.4 225093.4 83527
0003 80.9 123615.5 59507
0004 56.3 17757.9 140
Exception in thread "main" 0005 77.8 128505.0
4567
0006 59.6 16774.3 8902
0007 113.8 87891.2 100
0008 53.6 186410.7 92833
0009 51.4 19440.9 32548
0010 87.6 114092.7 -1
*** Job was unexpectedly updated ***
0011 75.0 133320.9 1839657
0012 48.8 204876.0 199929
0013 62.0 161415.3 9200000
0014 130.8 76434.5 9200000
==========================================================================================
java.lang.RuntimeException: Execution was not successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:170)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:415)
> Task :beam-sdks-java-nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 31m 11s
76 actionable tasks: 52 executed, 24 from cache
Publishing build scan...
https://gradle.com/s/rtb3fvxw3fn2q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
