See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/6166/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-10432] Improve error message for missing pipeline options.
[noreply] Refactor ZetaSQL planner code and support setting default timezone
------------------------------------------
[...truncated 233.37 KB...]
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 25.6 3900.8 100000
0001 15.2 6595.9 92000
0002 8.0 12503.1 351
0003 14.1 7111.9 444
0004 3.7 2720.3 40
0005 30.1 3322.1 12
0006 2.4 4110.2 401
0007 1070.4 93.4 1
0008 6.5 15415.4 6000
0009 1.6 6293.3 298
0010 56.0 1786.8 1
0011 10.4 9628.3 1919
0012 8.7 11492.9 1919
0013 *** not run ***
0014 *** not run ***
==========================================================================================
2020-07-10T00:25:18.869Z Generating 100000 events in streaming mode
2020-07-10T00:25:30.068Z Waiting for main pipeline to 'finish'
2020-07-10T00:25:30.068Z DONE BoundedSideInputJoin
2020-07-10T00:25:30.069Z 1 samples not enough to calculate steady-state event
rate
Jul 10, 2020 12:25:30 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace BoundedSideInputJoin
2020-07-10T00:25:30.069Z Running query:SESSION_SIDE_INPUT_JOIN;
exportSummaryToBigQuery:true
==========================================================================================
Run started 2020-07-10T00:04:20.595Z and ran for PT1269.474S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 25.6 3900.8 100000
0001 15.2 6595.9 92000
0002 8.0 12503.1 351
0003 14.1 7111.9 444
0004 3.7 2720.3 40
0005 30.1 3322.1 12
0006 2.4 4110.2 401
0007 1070.4 93.4 1
0008 6.5 15415.4 6000
0009 1.6 6293.3 298
0010 56.0 1786.8 1
0011 10.4 9628.3 1919
0012 8.7 11492.9 1919
0013 11.1 9038.3 92000
0014 *** not run ***
==========================================================================================
2020-07-10T00:25:30.072Z Generating 100000 events in streaming mode
2020-07-10T00:25:43.394Z Waiting for main pipeline to 'finish'
2020-07-10T00:25:43.394Z DONE BoundedSideInputJoin
2020-07-10T00:25:43.395Z 1 samples not enough to calculate steady-state event
rate
Jul 10, 2020 12:25:43 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace BoundedSideInputJoin
==========================================================================================
Run started 2020-07-10T00:04:20.595Z and ran for PT1282.800S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 25.6 3900.8 100000
0001 15.2 6595.9 92000
0002 8.0 12503.1 351
0003 14.1 7111.9 444
0004 3.7 2720.3 40
0005 30.1 3322.1 12
0006 2.4 4110.2 401
0007 1070.4 93.4 1
0008 6.5 15415.4 6000
0009 1.6 6293.3 298
0010 56.0 1786.8 1
0011 10.4 9628.3 1919
0012 8.7 11492.9 1919
0013 11.1 9038.3 92000
0014 12.1 8231.8 92000
==========================================================================================
==========================================================================================
Run started 2020-07-10T00:04:20.595Z and ran for PT1305.729S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":{"standardDays":0,"standardHours":0,"standardMinutes":10,"standardSeconds":600,"millis":600000},"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
Exception in thread "main" 0007 query:HIGHEST_BID;
exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0014 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 25.6 3900.8 100000
0001 15.2 6595.9 92000
0002 8.0 12503.1 351
0003 14.1 7111.9 444
0004 3.7 2720.3 40
0005 30.1 3322.1 12
0006 2.4 4110.2 401
0007 1070.4 93.4 1
0008 6.5 15415.4 6000
0009 1.6 6293.3 298
0010 56.0 1786.8 1
0011 10.4 9628.3 1919
0012 8.7 11492.9 1919
0013 11.1 9038.3 92000
0014 12.1 8231.8 92000
==========================================================================================
com.google.cloud.bigquery.BigQueryException: connect timed out
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:106)
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:461)
at
com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:983)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:123)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertRow(BigQueryClient.java:98)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertRow(BigQueryClient.java:89)
at
org.apache.beam.sdk.testutils.publishing.BigQueryResultsPublisher.publish(BigQueryResultsPublisher.java:47)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:199)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:160)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:491)
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:681)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1340)
at
sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1315)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:264)
at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:113)
at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:542)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:459)
... 8 more
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 21m 56s
76 actionable tasks: 2 executed, 74 up-to-date
Publishing build scan...
https://gradle.com/s/66appmry5pcoi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
