See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2/289/display/redirect?page=changes>
Changes:
[dpires] [BEAM-12715] Use shard number specified by user in SnowflakeIO batch
[relax] fix GroupIntoBatches
[noreply] [BEAM-12703] Fix universal metrics. (#15260)
[noreply] [BEAM-12702] Pull step unique names from pipeline for metrics.
(#15261)
[noreply] [BEAM-12678] Add dependency of java jars when running go VR on
portable
------------------------------------------
[...truncated 1.66 MB...]
Aug 04, 2021 7:29:04 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query3.Results
2021-08-04T19:29:04.763Z 0 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2021-08-04T18:21:54.007Z and ran for PT4030.756S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 *** not run ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
2021-08-04T19:29:10.208Z CANCELLED Query1 (waiting for shutdown)
Aug 04, 2021 7:29:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.elements, from namespace Query1.Events
Aug 04, 2021 7:29:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric event.bytes, from namespace Query1.Events
Aug 04, 2021 7:29:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query1.Results
Aug 04, 2021 7:29:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query1.Results
2021-08-04T19:29:10.232Z 0 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2021-08-04T18:21:54.007Z and ran for PT4036.226S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
==========================================================================================
Run started 2021-08-04T18:21:54.007Z and ran for PT4036.462S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0015 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0002 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0003 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0008 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Cancelling streaming job since it appeared stuck for 60 min. ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:186)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:495)
> Task :sdks:java:testing:nexmark:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804182104
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d
Deleted: sha256:44a4166f11898097d6e1738debd397773ebff7f8c8ec054a96dd617863949741
Deleted: sha256:1e738c53c7bf06b34ec05d17264aeed834326bc1692199172292b425967ac7da
Deleted: sha256:15970d6bc5002c2c889528b4d357fef27ef40e5f70a3a0f8a225f3543eb1dd03
Deleted: sha256:f7656aa259098a6f8d34358e64763d2defb5f02d8fc4cfe6ebeaa9536299b40a
Deleted: sha256:847faed470667256aa2d81ad282a36e94fb6f0d04f6b10bb1308082995a6b80a
Deleted: sha256:f61aab2d594fa6fa5f390ae63a4485933134f33a4b719684c77bd3acebadeff7
Deleted: sha256:0b077cd4512d59546e441cb860625d5800c0b45cd5cb045173819b597553b4cb
Deleted: sha256:cec9b74a11b90cb02cb7d65243ca04e7c5d7866c62603c1256c9bd23f3b35cf6
Deleted: sha256:0ae45d4df99641155fa130c5b895f03684bae27cb67a936ddb62fc80ba115a7a
Deleted: sha256:4e752b6d88f6d0645807209c359df97bbbc02dbc5104d3981f5d3542caad7126
Deleted: sha256:8127c1bbb88549e91a385123de263c6baba4ec0ce166eb828b293e02f10367a8
Deleted: sha256:58a196e1145249b0adf420c92d0357284cb4aba3aee8053af358ceca4f6eed57
Deleted: sha256:820d22ee4c70b699de3abdee02014bd29c230d8ac7c3da075359ec439475027c
Deleted: sha256:e75ceeaab2775a0eab1e1c1bb13f52da817b6af3c36fe99cddc3af6d89769377
Deleted: sha256:ab7fc6b6044e74ac35d64bb81554c23fcbe250d9ad8ec51351aeb5d0a365ff9e
Deleted: sha256:a27bb273c903c84d47e602436eb0e50174467fe1d43877c1c84c6180a7807a8d
Deleted: sha256:cd9568be7fd7a35176f4ff0a55e0c71f19a5c2b6e592e65e166fbf7ac23da4ad
Deleted: sha256:6e99c840cdb0c542eea4d218a4a9c06144efaac3816fd7a7dc12bcaac84f68d2
Deleted: sha256:a7dc0da3825ec7e7cc8ab6fd17d4341af40afaaade7de0239377881f4cd7c7b2
Deleted: sha256:39872daca9430797b41b455baa280ecf73cef3b5a4fc16b512581b3bf4a3f1ef
Deleted: sha256:f97749adc7e21fb9af3630218db1e1b3175d0eeab525a626a660f2ed30a3b30f
Deleted: sha256:9a0b7fede4e930559ffcf849138d97119bf50145fd2d7f475da4a58547e37d21
Deleted: sha256:bdd8adaf7a08bd7b5091faed7190a65197e84eda234e2b09530a0a6f79b7f3dc
Deleted: sha256:9de551bc8af94f24d900f29f68732074e57e2073ce2043e564f3aaf0a6a5abff
Deleted: sha256:aa59dd64c045740e7ce897129eb05043813421c013fb82a4cd13e37ed87dc7ee
Deleted: sha256:186d983e8d9713743cc050c6461919682b1b4a3de343a58201e0a40817879d8d
Deleted: sha256:fc01a1ab228a02351f99cc5e64c15c6dca84f084e17c02ceba6854fe0bd79f45
Deleted: sha256:2044a028fc7257c19792870048c3fd9f26df12a8bcf63053795442fa6491e3bb
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804182104]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20210804182104]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:8d0c60216a6442cc40aa91b731d5fdd1192d2ff7810dcd81af5368a7d7f86f6d].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 8m 12s
129 actionable tasks: 15 executed, 114 up-to-date
Publishing build scan...
https://gradle.com/s/y35snc3gt522c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
