See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java17/1801/display/redirect>
Changes:
------------------------------------------
[...truncated 836.01 KB...]
2023-03-17T10:35:51.602Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:35:51 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:35:51.681Z new perf
{"snapshots":null,"runtimeSec":18.057,"numEvents":2515255,"eventsPerSec":139295.28714625907,"eventBytesPerSec":1.7882957246497206E7,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":3488.687,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null}
2023-03-17T10:36:06.719Z RUNNING Query7
Mar 17, 2023 10:36:07 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Mar 17, 2023 10:36:07 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
2023-03-17T10:36:07.472Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:36:07 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:36:07.545Z new perf
{"snapshots":null,"runtimeSec":29.412,"numEvents":5831234,"eventsPerSec":198260.36991704066,"eventBytesPerSec":2.5449320379436966E7,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":3488.687,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null}
2023-03-17T10:36:22.590Z RUNNING Query7
Mar 17, 2023 10:36:22 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Mar 17, 2023 10:36:23 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
2023-03-17T10:36:23.279Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:36:23 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:36:23.370Z no activity
2023-03-17T10:36:38.424Z RUNNING Query7
Mar 17, 2023 10:36:38 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Mar 17, 2023 10:36:38 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
2023-03-17T10:36:39.038Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:36:39 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:36:39.095Z new perf
{"snapshots":null,"runtimeSec":64.894,"numEvents":9654142,"eventsPerSec":148767.8675994699,"eventBytesPerSec":1.9095710435479395E7,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":3488.687,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null}
2023-03-17T10:36:54.139Z RUNNING Query7
Mar 17, 2023 10:36:54 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Mar 17, 2023 10:36:54 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
2023-03-17T10:36:54.771Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:36:54 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-03-17T10:36:54.830Z no activity
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:37:09.874Z RUNNING Query7
Mar 17, 2023 10:37:10 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Mar 17, 2023 10:37:10 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
2023-03-17T10:37:10.532Z 0 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:37:10 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:37:10.588Z new perf
{"snapshots":null,"runtimeSec":87.832,"numEvents":10000000,"eventsPerSec":113853.72073959377,"eventBytesPerSec":1.4613863933418345E7,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":3488.687,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null}
2023-03-17T10:37:25.628Z RUNNING Query7
2023-03-17T10:37:26.388Z 1 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:37:26 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:37:26.463Z new perf
{"snapshots":null,"runtimeSec":99.886,"numEvents":10000000,"eventsPerSec":100114.13010832349,"eventBytesPerSec":1.2850298310073484E7,"numResults":26,"resultsPerSec":0.26029673828164107,"resultBytesPerSec":26.03968524117494,"startupDelaySec":3488.687,"processingDelaySec":95.006,"shutdownDelaySec":19.743,"timeDilation":2.5135654646296777,"errors":null,"jobId":null}
2023-03-17T10:37:41.508Z RUNNING Query7
2023-03-17T10:37:42.244Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:37:42.309Z new perf
{"snapshots":null,"runtimeSec":120.834,"numEvents":10000000,"eventsPerSec":82758.16409288776,"eventBytesPerSec":1.0622547436979657E7,"numResults":100,"resultsPerSec":0.8275816409288776,"resultBytesPerSec":83.61057318304451,"startupDelaySec":3488.687,"processingDelaySec":95.006,"shutdownDelaySec":14.669,"timeDilation":8.138437856894582,"errors":null,"jobId":null}
Mar 17, 2023 10:37:42 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:37:57.355Z RUNNING Query7
2023-03-17T10:37:58.046Z 2 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:37:58 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-03-17T10:37:58.113Z no activity
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:38:13.149Z RUNNING Query7
2023-03-17T10:38:13.785Z 2 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:38:13 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:38:13.842Z no activity
2023-03-17T10:38:28.877Z RUNNING Query7
2023-03-17T10:38:29.511Z 2 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:38:29 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-03-17T10:38:29.612Z no activity
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:38:44.653Z RUNNING Query7
2023-03-17T10:38:45.323Z 2 samples not enough to calculate steady-state event
rate
Mar 17, 2023 10:38:45 AM org.apache.beam.sdk.testutils.metrics.MetricsReader
getCounterMetric
2023-03-17T10:38:45.383Z streaming query appears to have finished waiting for
completion.
SEVERE: Failed to get metric fatal, from namespace Query7
2023-03-17T10:38:45.383Z no activity
2023-03-17T10:39:00.421Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:39:01.126Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:39:01.126Z no activity
2023-03-17T10:39:16.165Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:39:16.849Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:39:16.850Z no activity
2023-03-17T10:39:31.889Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:39:32.559Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:39:32.560Z no activity
2023-03-17T10:39:47.607Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:39:48.324Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:39:48.324Z no activity
2023-03-17T10:40:03.365Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:40:04.111Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:40:04.111Z no activity
2023-03-17T10:40:19.183Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:40:19.890Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:40:19.890Z no activity
2023-03-17T10:40:34.926Z RUNNING Query7 (waiting for shutdown)
2023-03-17T10:40:35.598Z 2 samples not enough to calculate steady-state event
rate
2023-03-17T10:40:35.599Z no activity
2023-03-17T10:40:50.637Z DONE Query7 (waiting for shutdown)
2023-03-17T10:40:50.681Z 2 samples not enough to calculate steady-state event
rate
==========================================================================================
Run started 2023-03-17T09:33:02.906Z and ran for PT4067.775S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 51.3 194791.3 10000000
0001 24.1 415783.1 9200000
0002 26.7 375009.4 83527
0003 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0004 *** not run ***
0005 210.3 47550.9 4567
0006 *** not run ***
0007 120.8 82758.2 100
0008 65.7 152132.9 92833
0009 *** not run ***
0010 120.1 83251.5 100
0011 203.9 49044.6 1839657
0012 92.5 108073.1 316273
0013 *** not run ***
0014 99.9 100097.1 10000276
0015 *** not run ***
0016 *** not run ***
==========================================================================================
==========================================================================================
Run started 2023-03-17T09:33:02.906Z and ran for PT4068.023S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 51.3 194791.3 10000000
0001 24.1 415783.1 9200000
0002 26.7 375009.4 83527
0003 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0004 *** not run ***
0005 210.3 47550.9 4567
0006 *** not run ***
0007 120.8 82758.2 100
0008 65.7 152132.9 92833
0009 *** not run ***
0010 120.1 83251.5 100
0011 203.9 49044.6 1839657
0012 92.5 108073.1 316273
0013 *** not run ***
0014 99.9 100097.1 10000276
0015 *** not run ***
0016 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:169)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:441)
> Task :sdks:java:testing:nexmark:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230317093236
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230317093236]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230317093236]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:0de27cf0d7b0b70c6aea7a37203b7870cffd66db9981f535919f43368784e83d].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1h 8m 33s
138 actionable tasks: 10 executed, 128 up-to-date
Publishing build scan...
https://gradle.com/s/e4zpsv77qc3me
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
