See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/6198/display/redirect?page=changes>
Changes:
[chamikaramj] Include coders embedded in CombinePayload when converting x-lang
[kevinsijo] [BEAM-9919] Refining xlang types and wrappers
[Luke Cwik] [BEAM-9979] Support a reset call that happens after the bundle is
done
[noreply] [BEAM-10200] Add optional experiment to enable heap dump through the …
[kevinsijo] [BEAM-9919] xlang transforms wrapped with namespaces
[kevinsijo] [BEAM-9919] cleaning namespace.go
------------------------------------------
[...truncated 1.64 MB...]
2020-08-20T01:09:05.693Z RUNNING SqlQuery0
2020-08-20T01:09:06.424Z RUNNING SqlQuery2
2020-08-20T01:09:06.541Z new perf
{"snapshots":null,"runtimeSec":1275.342,"numEvents":8800000,"eventsPerSec":6900.109931296859,"eventBytesPerSec":885667.5919086802,"numResults":8096000,"resultsPerSec":6348.1011367931105,"resultBytesPerSec":637983.6788877022,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":8.413,"timeDilation":0.7840641961136698,"errors":null,"jobId":null}
2020-08-20T01:09:07.666Z no activity
2020-08-20T01:09:21.592Z RUNNING SqlQuery0
2020-08-20T01:09:22.342Z no activity
2020-08-20T01:09:22.717Z RUNNING SqlQuery2
2020-08-20T01:09:23.593Z no activity
2020-08-20T01:09:37.407Z RUNNING SqlQuery0
2020-08-20T01:09:38.356Z no activity
2020-08-20T01:09:38.770Z RUNNING SqlQuery2
2020-08-20T01:09:39.907Z no activity
2020-08-20T01:09:53.507Z RUNNING SqlQuery0
2020-08-20T01:09:54.327Z new perf
{"snapshots":null,"runtimeSec":1321.577,"numEvents":9000000,"eventsPerSec":6810.045877009058,"eventBytesPerSec":874106.2352023378,"numResults":8280000,"resultsPerSec":6265.242206848334,"resultBytesPerSec":629654.6406300957,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":9.943,"timeDilation":0.756633930523912,"errors":null,"jobId":null}
2020-08-20T01:09:54.947Z DONE SqlQuery2
==========================================================================================
Run started 2020-08-20T00:46:08.000Z and ran for PT1426.987S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 *** not run ***
0002 1262.7 7919.7 83527
0003 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 *** not run ***
==========================================================================================
2020-08-20T01:10:09.376Z RUNNING SqlQuery0
2020-08-20T01:10:10.285Z new perf
{"snapshots":null,"runtimeSec":1331.037,"numEvents":9200000,"eventsPerSec":6911.904026710001,"eventBytesPerSec":887181.6515994671,"numResults":8464000,"resultsPerSec":6358.951704573201,"resultBytesPerSec":639071.4405384674,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":16.454,"timeDilation":0.7512563512509419,"errors":null,"jobId":null}
2020-08-20T01:10:25.352Z RUNNING SqlQuery0
2020-08-20T01:10:26.218Z no activity
2020-08-20T01:10:41.302Z RUNNING SqlQuery0
2020-08-20T01:10:42.153Z new perf
{"snapshots":null,"runtimeSec":1370.363,"numEvents":9300000,"eventsPerSec":6786.522986974984,"eventBytesPerSec":871089.2303718065,"numResults":8556000,"resultsPerSec":6243.6011480169855,"resultBytesPerSec":627479.4167676739,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":9.02,"timeDilation":0.7296971678307135,"errors":null,"jobId":null}
2020-08-20T01:10:57.201Z RUNNING SqlQuery0
2020-08-20T01:10:58.174Z new perf
{"snapshots":null,"runtimeSec":1386.409,"numEvents":9600000,"eventsPerSec":6924.363589676639,"eventBytesPerSec":888784.9977892525,"numResults":8832000,"resultsPerSec":6370.414502502508,"resultBytesPerSec":640222.0001457001,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":8.909,"timeDilation":0.7212518095309537,"errors":null,"jobId":null}
2020-08-20T01:11:13.271Z RUNNING SqlQuery0
2020-08-20T01:11:14.196Z no activity
2020-08-20T01:11:29.273Z RUNNING SqlQuery0
2020-08-20T01:11:30.160Z no activity
2020-08-20T01:11:45.224Z RUNNING SqlQuery0
2020-08-20T01:11:46.185Z new perf
{"snapshots":null,"runtimeSec":1426.939,"numEvents":9700000,"eventsPerSec":6797.767809275659,"eventBytesPerSec":872535.9724557251,"numResults":8924000,"resultsPerSec":6253.946384533606,"resultBytesPerSec":628516.6752047564,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":16.385,"timeDilation":0.7007657650397109,"errors":null,"jobId":null}
2020-08-20T01:12:01.287Z RUNNING SqlQuery0
2020-08-20T01:12:02.200Z new perf
{"snapshots":null,"runtimeSec":1442.76,"numEvents":10000000,"eventsPerSec":6931.159721644625,"eventBytesPerSec":889659.3314203332,"numResults":9200000,"resultsPerSec":6376.666943913056,"resultBytesPerSec":640851.2476087499,"startupDelaySec":-160.46,"processingDelaySec":0.611,"shutdownDelaySec":16.589,"timeDilation":0.6930813163658543,"errors":null,"jobId":null}
2020-08-20T01:12:17.296Z RUNNING SqlQuery0
2020-08-20T01:12:18.134Z no activity
2020-08-20T01:12:33.192Z RUNNING SqlQuery0
2020-08-20T01:12:34.204Z no activity
2020-08-20T01:12:49.294Z RUNNING SqlQuery0
2020-08-20T01:12:50.111Z no activity
2020-08-20T01:13:05.178Z RUNNING SqlQuery0
2020-08-20T01:13:06.064Z no activity
2020-08-20T01:13:21.142Z RUNNING SqlQuery0
2020-08-20T01:13:21.998Z no activity
2020-08-20T01:13:37.069Z RUNNING SqlQuery0
2020-08-20T01:13:37.951Z no activity
2020-08-20T01:13:53.001Z RUNNING SqlQuery0
2020-08-20T01:13:53.833Z no activity
2020-08-20T01:14:08.890Z RUNNING SqlQuery0
2020-08-20T01:14:09.729Z no activity
2020-08-20T01:14:24.781Z RUNNING SqlQuery0
2020-08-20T01:14:25.653Z no activity
2020-08-20T01:14:40.716Z RUNNING SqlQuery0
2020-08-20T01:14:41.480Z no activity
2020-08-20T01:14:56.554Z RUNNING SqlQuery0
2020-08-20T01:14:57.349Z no activity
2020-08-20T01:15:12.534Z RUNNING SqlQuery0
2020-08-20T01:15:13.390Z no activity
2020-08-20T01:15:28.449Z RUNNING SqlQuery0
2020-08-20T01:15:29.276Z no activity
2020-08-20T01:15:44.347Z RUNNING SqlQuery0
2020-08-20T01:15:45.158Z no activity
2020-08-20T01:16:00.232Z RUNNING SqlQuery0
2020-08-20T01:16:01.096Z no activity
2020-08-20T01:16:16.179Z RUNNING SqlQuery0
2020-08-20T01:16:16.940Z no activity
2020-08-20T01:16:32.006Z RUNNING SqlQuery0
2020-08-20T01:16:32.813Z no activity
2020-08-20T01:16:47.888Z RUNNING SqlQuery0
2020-08-20T01:16:48.682Z no activity
2020-08-20T01:17:03.753Z RUNNING SqlQuery0
2020-08-20T01:17:04.895Z no activity
2020-08-20T01:17:20.002Z RUNNING SqlQuery0
2020-08-20T01:17:20.788Z no activity
2020-08-20T01:17:35.881Z RUNNING SqlQuery0
2020-08-20T01:17:36.655Z no activity
2020-08-20T01:17:51.721Z RUNNING SqlQuery0
2020-08-20T01:17:52.508Z no activity
2020-08-20T01:18:07.564Z RUNNING SqlQuery0
2020-08-20T01:18:08.411Z no activity
2020-08-20T01:18:23.479Z RUNNING SqlQuery0
2020-08-20T01:18:24.283Z no activity
2020-08-20T01:18:39.334Z RUNNING SqlQuery0
2020-08-20T01:18:40.174Z no activity
2020-08-20T01:18:55.274Z DONE SqlQuery0
==========================================================================================
Run started 2020-08-20T00:46:08.000Z and ran for PT1967.321S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 1442.8 6931.2 9200000
0001 *** not run ***
0002 1262.7 7919.7 83527
0003 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 *** not run ***
==========================================================================================
==========================================================================================
Run started 2020-08-20T00:46:08.000Z and ran for PT1968.820S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 1442.8 6931.2 9200000
0001 *** not run ***
0002 1262.7 7919.7 83527
0003 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 *** not run ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:490)
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 32m 56s
89 actionable tasks: 2 executed, 87 up-to-date
Publishing build scan...
https://gradle.com/s/2ack5gx3m3wto
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
