See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2/266/display/redirect?page=changes>
Changes:
[ajamato] [BEAM-11994] Update BigQueryStorageStreamSource and
BigQueryServicesImpl
[ajamato] [BEAM-12670] Fix regression, properly report API call metric properly
on
[noreply] [BEAM-12609] Enable projection pushdown in SchemaIO. (#15216)
------------------------------------------
[...truncated 2.91 MB...]
Jul 30, 2021 1:29:40 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query11.Results
2021-07-30T01:29:40.844Z 0 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2021-07-30T00:22:15.584Z and ran for PT4045.260S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
2021-07-30T01:29:54.997Z CANCELLED Query0 (waiting for shutdown)
Jul 30, 2021 1:29:55 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query0.Events
Jul 30, 2021 1:29:55 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query0.Events
Jul 30, 2021 1:29:55 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query0.Results
Jul 30, 2021 1:29:55 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query0.Results
2021-07-30T01:29:55.018Z 0 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2021-07-30T00:22:15.584Z and ran for PT4059.434S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0001 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
==========================================================================================
Run started 2021-07-30T00:22:15.584Z and ran for PT4059.657S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0001 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:186)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:495)
> Task :sdks:java:testing:nexmark:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730002102
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0
Deleted: sha256:c1f550e460bfc01ed1cc02f7abc9414f4de31eeb51abd5ecf61d3118f29812d6
Deleted: sha256:cd371ba9582bd3bad846cd897f678f7e8211754a38835410cec3fafaa3174ed2
Deleted: sha256:eb6c34d76da67b0885ee0e195fd5bd7d2b700150fb8074f97e63e5cf6eb761fa
Deleted: sha256:75bcbe04fd463e51d08d4903810bef8e55b09db18f49fb7fa62ef419cc2f3cfc
Deleted: sha256:a46c9b57aea8529864e135a4cad65600a35020fa5cfcb6d9afb09ebc7c1e1584
Deleted: sha256:b6f0ef5695088848c207c104249c010b2ac03b887ebd8d24d00a18c659dfc2fb
Deleted: sha256:2800489bd2c93f0152de4711d29a71cf2edd4a92ff53ae27a43a619509b4f0da
Deleted: sha256:2fe86a548c6037e71e48ce0a1b92f8e8f760d72c209fb74411ac78852673c8d9
Deleted: sha256:47a58abc8e0daec88d60e6f2b511e3e1214ba8c1c3f341ce02a4f9a3b2e24425
Deleted: sha256:42c2d52e46601fa749a9fe894575f7d69bbacfa57717118e87a87f58fac7c16f
Deleted: sha256:c306a5ae0d197e7e25fc7077ba00dd81298f905dd4b8cb05ec6c0bebcb8a9944
Deleted: sha256:8f625c63e2bdd0b734be0ab03b256d27d8a825bc30a644e10d8b2e8cbcc7e786
Deleted: sha256:a3cc1aa9b767b1a1c0d950c928c44a1f11513d666df8c825f693e2dc02331594
Deleted: sha256:ff45a76de01b26aebd675b9268a2cc73f02dce7b971b070ae0917510511a41fa
Deleted: sha256:88de4bd77172521f02d2505ed564d3bfe729958c2817c84c8940fdd772d81fae
Deleted: sha256:fd6d5c53f13dd7337ce4ed4d5c7019cca875d7c056706f170cd7137778504488
Deleted: sha256:62af625a1f9debd93a828b451de621c47a19b1e0455d19f3a922efae862d64f3
Deleted: sha256:7e213c4f37131e4d2f3cfd97aaa677bfdcba85e7097201db6e37ba37cd93efae
Deleted: sha256:5cb4745c28acad4d82d434994c627bedab5f4055ca697e555db063063e726928
Deleted: sha256:4602d828dd0c7711897fb1a33e0cfec4032c56bf74cd92aff2107dea4afaf808
Deleted: sha256:54ba1af02c37dadbca485aeb6ec42bebe3744a8814f9d667ca2b666f7924c9d4
Deleted: sha256:624dc7323faf672dafa6c2360d1180201b6330f94c29190700f76d70ec030f07
Deleted: sha256:e0751ac00ac529dd9608fd17a007f7656f1a16117a9ca643ddd597308da170e6
Deleted: sha256:147f374d1ab183d2c497054b6a3985dbcdc4727b40fee59f90955dd579f8190f
Deleted: sha256:ffed1b0936bca66382384c0dd85c8646a50836902d012d090357a06d0abe0123
Deleted: sha256:5a4584d6fa898ea7935af8942d93c5418763bdefd3bcce237756e4e940932245
Deleted: sha256:2eaacdcface82ed6046f4a4854c7b374c55f6623229e43e9d4843af43841041f
Deleted: sha256:23c4b30742d7289d344b234aeb7bfb5006386045412cee5fff1cd347fa1be63f
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730002102]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210730002102]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:5f72a9b3d490df7980fbacd4c401079163332143a1ecc48500aa62a40d37a9e0].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 9m 1s
129 actionable tasks: 15 executed, 114 up-to-date
Publishing build scan...
https://gradle.com/s/woqad624tyq2a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
