See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/6193/display/redirect?page=changes>
Changes:
[noreply] [BEAM-10715] Update Jet Runner validates runner testing (#12567)
[noreply] [BEAM-10557] Implemented SchemaIOProvider for DataStoreV1, Refactored
------------------------------------------
[...truncated 628.06 KB...]
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 62.1 160929.5 10000000
0001 49.4 202355.4 9200000
0002 35.5 281531.5 83527
0003 42.6 234807.9 59507
0004 54.5 18360.4 140
0005 64.4 155397.7 4567
0006 48.1 20785.7 8902
0007 *** not run ***
0008 50.9 196398.1 92833
0009 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0010 *** not run ***
0011 53.2 188058.3 1839657
0012 38.9 257056.2 199929
0013 46.3 216034.0 9200000
0014 *** not run ***
==========================================================================================
2020-08-18T18:26:44.811Z Waiting for main pipeline to 'finish'
2020-08-18T18:26:44.812Z DONE Query7
Aug 18, 2020 6:26:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
logTerminalState
INFO: Job 2020-08-18_11_23_08-13881938584485392310 finished with status DONE.
==========================================================================================
Run started 2020-08-18T18:19:19.042Z and ran for PT446.129S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 62.1 160929.5 10000000
0001 49.4 202355.4 9200000
0002 35.5 281531.5 83527
0003 42.6 234807.9 59507
0004 54.5 18360.4 140
0005 64.4 155397.7 4567
0006 48.1 20785.7 8902
0007 83.5 119763.3 100
0008 50.9 196398.1 92833
0009 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0010 *** not run ***
0011 53.2 188058.3 1839657
0012 38.9 257056.2 199929
0013 46.3 216034.0 9200000
0014 *** not run ***
==========================================================================================
Aug 18, 2020 6:26:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-18T18:26:47.251Z: Autoscaling: Resized **** pool from 4 to 0.
Aug 18, 2020 6:26:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-08-18T18:26:47.281Z: Worker pool stopped.
Aug 18, 2020 6:26:55 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
logTerminalState
INFO: Job 2020-08-18_11_23_07-5598144254563377286 finished with status DONE.
2020-08-18T18:26:55.370Z Waiting for main pipeline to 'finish'
2020-08-18T18:26:55.371Z DONE Query10
==========================================================================================
Run started 2020-08-18T18:19:19.042Z and ran for PT456.792S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 62.1 160929.5 10000000
0001 49.4 202355.4 9200000
0002 35.5 281531.5 83527
0003 42.6 234807.9 59507
0004 54.5 18360.4 140
0005 64.4 155397.7 4567
0006 48.1 20785.7 8902
0007 83.5 119763.3 100
0008 50.9 196398.1 92833
0009 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0010 91.7 109070.3 100
0011 53.2 188058.3 1839657
0012 38.9 257056.2 199929
0013 46.3 216034.0 9200000
0014 *** not run ***
==========================================================================================
2020-08-18T18:27:12.860Z Waiting for main pipeline to 'finish'
Aug 18, 2020 6:27:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2020-08-18T18:27:12.902Z RUNNING BoundedSideInputJoin
2020-08-18T18:27:14.318Z new perf
{"snapshots":null,"runtimeSec":122.644,"numEvents":10000000,"eventsPerSec":81536.80571409935,"eventBytesPerSec":1.0465778162812693E7,"numResults":9200000,"resultsPerSec":75013.86125697139,"resultBytesPerSec":6424544.168487655,"startupDelaySec":-156.736,"processingDelaySec":60.755,"shutdownDelaySec":34.092,"timeDilation":8.153109813769936,"errors":null,"jobId":null}
2020-08-18T18:27:29.394Z RUNNING BoundedSideInputJoin
2020-08-18T18:27:30.955Z no activity
2020-08-18T18:27:46.037Z DONE BoundedSideInputJoin
==========================================================================================
Run started 2020-08-18T18:19:19.042Z and ran for PT507.074S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 62.1 160929.5 10000000
0001 49.4 202355.4 9200000
0002 35.5 281531.5 83527
0003 42.6 234807.9 59507
0004 54.5 18360.4 140
0005 64.4 155397.7 4567
0006 48.1 20785.7 8902
0007 83.5 119763.3 100
0008 50.9 196398.1 92833
0009 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0010 91.7 109070.3 100
0011 53.2 188058.3 1839657
0012 38.9 257056.2 199929
0013 46.3 216034.0 9200000
0014 122.6 81536.8 9200000
==========================================================================================
==========================================================================================
Run started 2020-08-18T18:19:19.042Z and ran for PT514.095S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 62.1 160929.5 10000000
0001 49.4 202355.4 9200000
0002 35.5 281531.5 83527
0003 42.6 234807.9 59507
0004 54.5 18360.4 140
0005 64.4 155397.7 4567
0006 48.1 20785.7 8902
0007 83.5 119763.3 100
0008 50.9 196398.1 92833
0009 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0010 91.7 109070.3 100
0011 53.2 188058.3 1839657
0012 38.9 257056.2 199929
0013 46.3 216034.0 9200000
0014 122.6 81536.8 9200000
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:490)
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 53s
89 actionable tasks: 81 executed, 8 from cache
Publishing build scan...
https://gradle.com/s/agrgmbsbciwqy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
