See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/6194/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-9956] Remove unbalanced code markup.
[zyichi] Bump up advanceProcessingTime duration in ParDoTest
[tobiasz.kedzierski] [BEAM-10623] Add workflow to run python tests on
Linux/Windows/Mac
[tobiasz.kedzierski] [BEAM-10624] dtype explicit for the numpy arrays
[Steve Niemitz] [BEAM-10523] Add support for custom DatumWriters to AvroIO.Write
[Luke Cwik] [BEAM-10040, BEAM-6804] Increase wait times to reduce flakiness.
[noreply] [BEAM-10752] add use_deprecated_read experiment for testSimpleInsert,
------------------------------------------
[...truncated 1017.10 KB...]
2020-08-19T00:13:46.956Z RUNNING Query5
2020-08-19T00:13:47.425Z 1 samples not enough to calculate steady-state event
rate
Aug 19, 2020 12:13:47 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2020-08-19T00:13:47.482Z streaming query appears to have finished waiting for
completion.
SEVERE: Failed to get metric fatal, from namespace Query12
2020-08-19T00:13:47.482Z no activity
2020-08-19T00:13:47.542Z 1 samples not enough to calculate steady-state event
rate
Aug 19, 2020 12:13:47 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace BoundedSideInputJoin
2020-08-19T00:13:47.609Z streaming query appears to have finished waiting for
completion.
2020-08-19T00:13:47.609Z no activity
2020-08-19T00:13:47.858Z 2 samples not enough to calculate steady-state event
rate
Aug 19, 2020 12:13:47 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2020-08-19T00:13:47.926Z no activity
SEVERE: Failed to get metric fatal, from namespace Query5
2020-08-19T00:14:02.554Z RUNNING Query12 (waiting for shutdown)
2020-08-19T00:14:02.715Z RUNNING BoundedSideInputJoin (waiting for shutdown)
2020-08-19T00:14:03.024Z DONE Query5
2020-08-19T00:14:03.102Z 2 samples not enough to calculate steady-state event
rate
Aug 19, 2020 12:14:03 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2020-08-19T00:14:03.102Z streaming query appears to have finished waiting for
completion.
SEVERE: Failed to get metric fatal, from namespace Query5
==========================================================================================
Run started 2020-08-19T00:08:09.283Z and ran for PT353.819S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 22.7 441228.4 10000000
0001 17.2 580282.0 9200000
0002 13.3 752558.7 83527
0003 49.5 202224.5 59439
0004 55.5 18024.8 140
0005 141.2 70811.0 4567
0006 55.2 18119.2 15062
0007 106.3 94040.6 100
0008 49.1 203682.6 92833
0009 39.7 25205.4 32548
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 *** not run ***
0013 26.1 383068.4 9200000
0014 *** not run ***
==========================================================================================
2020-08-19T00:14:03.407Z 1 samples not enough to calculate steady-state event
rate
2020-08-19T00:14:03.407Z no activity
2020-08-19T00:14:03.477Z 1 samples not enough to calculate steady-state event
rate
2020-08-19T00:14:03.477Z no activity
2020-08-19T00:14:18.529Z RUNNING Query12 (waiting for shutdown)
2020-08-19T00:14:18.659Z DONE BoundedSideInputJoin (waiting for shutdown)
2020-08-19T00:14:18.705Z 1 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2020-08-19T00:08:09.283Z and ran for PT369.422S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 22.7 441228.4 10000000
0001 17.2 580282.0 9200000
0002 13.3 752558.7 83527
0003 49.5 202224.5 59439
0004 55.5 18024.8 140
0005 141.2 70811.0 4567
0006 55.2 18119.2 15062
0007 106.3 94040.6 100
0008 49.1 203682.6 92833
0009 39.7 25205.4 32548
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 *** not run ***
0013 26.1 383068.4 9200000
0014 151.0 66229.1 9200000
==========================================================================================
2020-08-19T00:14:19.332Z 1 samples not enough to calculate steady-state event
rate
2020-08-19T00:14:19.332Z no activity
2020-08-19T00:14:39.197Z RUNNING Query12 (waiting for shutdown)
2020-08-19T00:14:39.388Z 1 samples not enough to calculate steady-state event
rate
2020-08-19T00:14:39.388Z no activity
2020-08-19T00:14:54.388Z DONE Query12 (waiting for shutdown)
2020-08-19T00:14:54.389Z 1 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2020-08-19T00:08:09.283Z and ran for PT405.106S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 22.7 441228.4 10000000
0001 17.2 580282.0 9200000
0002 13.3 752558.7 83527
0003 49.5 202224.5 59439
0004 55.5 18024.8 140
0005 141.2 70811.0 4567
0006 55.2 18119.2 15062
0007 106.3 94040.6 100
0008 49.1 203682.6 92833
0009 39.7 25205.4 32548
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 94.2 106196.6 615395
0013 26.1 383068.4 9200000
0014 151.0 66229.1 9200000
==========================================================================================
==========================================================================================
Run started 2020-08-19T00:08:09.283Z and ran for PT408.437S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true;
numEvents:10000000
0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true;
numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:1000000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:1000000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true;
numEvents:10000000
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true;
numEvents:10000000
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true;
numEvents:10000000
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true;
numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 22.7 441228.4 10000000
0001 17.2 580282.0 9200000
0002 13.3 752558.7 83527
0003 49.5 202224.5 59439
0004 55.5 18024.8 140
0005 141.2 70811.0 4567
0006 55.2 18119.2 15062
0007 106.3 94040.6 100
0008 49.1 203682.6 92833
0009 39.7 25205.4 32548
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 94.2 106196.6 615395
0013 26.1 383068.4 9200000
0014 151.0 66229.1 9200000
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:182)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:490)
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 3s
89 actionable tasks: 2 executed, 87 up-to-date
Publishing build scan...
https://gradle.com/s/ddmxki7ea2dio
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
