See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/7314/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12194] Implement CREATE AGGREGATE FUNCTION.
[Daniel Kulp] [BEAM-12210] Use formatting string for checkArgument to avoid
excess
[Daniel Kulp] Fix spotless
[noreply] [BEAM-8691] Upgrading bigtable-client-core to 1.19.1
[suztomo] [BEAM-8684] Upgrading com.google.apis:google-api-services-bigquery
[suztomo] [BEAM-8750] Upgrading
com.google.apis:google-api-services-clouddebugger
[suztomo] [BEAM-8751] Upgrading
[suztomo] [BEAM-8752] Upgrading com.google.apis:google-api-services-dataflow
[suztomo] [BEAM-10349] Upgrading com.google.apis:google-api-services-healthcare
[suztomo] [BEAM-8753] Upgrading com.google.apis:google-api-services-pubsub
[noreply] Plumb worker pool from properties to capabilities in Dataflow java
------------------------------------------
[...truncated 110.15 KB...]
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:PORTABILITY_BATCH; exportSummaryToBigQuery:true
0014 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0015 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 2.4 42498.9 100000
0001 1.3 76452.6 92000
0002 0.9 109890.1 351
0003 2.1 46554.9 580
0004 1.5 6566.0 40
0005 6.3 15964.2 12
0006 1.0 10121.5 401
0007 3.1 32754.7 1
0008 1.5 68681.3 6000
0009 0.7 14367.8 298
0010 2.6 38925.7 1
0011 2.4 41911.1 1919
0012 1.5 64557.8 1919
0013 20.7 4841.7 100000
0014 *** not run ***
0015 *** not run ***
==========================================================================================
2021-04-23T00:11:57.349Z Generating 100000 events in batch mode
2021-04-23T00:11:59.293Z Waiting for main pipeline to 'finish'
2021-04-23T00:11:59.293Z DONE BoundedSideInputJoin
2021-04-23T00:11:59.293Z Running query:SESSION_SIDE_INPUT_JOIN;
exportSummaryToBigQuery:true
==========================================================================================
Run started 2021-04-23T00:10:49.713Z and ran for PT69.580S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:PORTABILITY_BATCH; exportSummaryToBigQuery:true
0014 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0015 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 2.4 42498.9 100000
0001 1.3 76452.6 92000
0002 0.9 109890.1 351
0003 2.1 46554.9 580
0004 1.5 6566.0 40
0005 6.3 15964.2 12
0006 1.0 10121.5 401
0007 3.1 32754.7 1
0008 1.5 68681.3 6000
0009 0.7 14367.8 298
0010 2.6 38925.7 1
0011 2.4 41911.1 1919
0012 1.5 64557.8 1919
0013 20.7 4841.7 100000
0014 1.5 67567.6 92000
0015 *** not run ***
==========================================================================================
2021-04-23T00:11:59.294Z Generating 100000 events in batch mode
2021-04-23T00:12:02.586Z Waiting for main pipeline to 'finish'
2021-04-23T00:12:02.586Z DONE SessionSideInputJoin
==========================================================================================
Run started 2021-04-23T00:10:49.713Z and ran for PT72.873S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:PORTABILITY_BATCH; exportSummaryToBigQuery:true
0014 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0015 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 2.4 42498.9 100000
0001 1.3 76452.6 92000
0002 0.9 109890.1 351
0003 2.1 46554.9 580
0004 1.5 6566.0 40
0005 6.3 15964.2 12
0006 1.0 10121.5 401
0007 3.1 32754.7 1
0008 1.5 68681.3 6000
0009 0.7 14367.8 298
0010 2.6 38925.7 1
0011 2.4 41911.1 1919
0012 1.5 64557.8 1919
0013 20.7 4841.7 100000
0014 1.5 67567.6 92000
0015 2.4 41050.9 92000
==========================================================================================
==========================================================================================
Run started 2021-04-23T00:10:49.713Z and ran for PT94.383S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; exportSummaryToBigQuery:true
0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true
0002 query:SELECTION; exportSummaryToBigQuery:true
0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true
0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true;
numEvents:10000
0005 query:HOT_ITEMS; exportSummaryToBigQuery:true
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true;
numEvents:10000
0007 query:HIGHEST_BID; exportSummaryToBigQuery:true
0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true
0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:10000
0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true
0011 query:USER_SESSIONS; exportSummaryToBigQuery:true
0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true
0013 query:PORTABILITY_BATCH; exportSummaryToBigQuery:true
0014 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
0015 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 2.4 42498.9 100000
0001 1.3 76452.6 92000
0002 0.9 109890.1 351
0003 2.1 46554.9 580
0004 1.5 6566.0 40
0005 6.3 15964.2 12
0006 1.0 10121.5 401
0007 3.1 32754.7 1
0008 1.5 68681.3 6000
0009 0.7 14367.8 298
0010 2.6 38925.7 1
0011 2.4 41911.1 1919
0012 1.5 64557.8 1919
Exception in thread "main" 0013 20.7 4841.7
100000
0014 1.5 67567.6 92000
0015 2.4 41050.9 92000
==========================================================================================
com.google.cloud.bigquery.BigQueryException: Read timed out
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at
com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:123)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertRow(BigQueryClient.java:98)
at
org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertRow(BigQueryClient.java:89)
at
org.apache.beam.sdk.testutils.publishing.BigQueryResultsPublisher.publish(BigQueryResultsPublisher.java:47)
at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:203)
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:164)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:494)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at
sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:457)
at
sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
at
sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1095)
at sun.security.ssl.SSLSocketImpl.access$200(SSLSocketImpl.java:72)
at
sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:815)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at
com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:36)
at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:149)
at
com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
at
com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:492)
... 8 more
> Task :sdks:java:testing:nexmark:run FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 12m 4s
100 actionable tasks: 99 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/xtw5jhgwkefdk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
