See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java11/3515/display/redirect>
Changes:
------------------------------------------
[...truncated 232.56 KB...]
Oct 24, 2023 4:28:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:26.328Z: Finished operation
Query14-ReadBounded-ParDo-BoundedSourceAsSDFWrapper--ParMultiDo-BoundedSourceAsSDFWrapper-/ProcessElementAndRestrictionWithSizing+Query14/Query14.Monitor/ParMultiDo(Monitor)+Query14/Query14.Snoop/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Pair
with random
key/ParMultiDo(AssignShard)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/ReifyOriginalTimestamps/ParDo(Anonymous)/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 24, 2023 4:28:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:26.375Z: Executing operation
Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 24, 2023 4:28:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:26.421Z: Finished operation
Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 24, 2023 4:28:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:26.474Z: Executing operation
Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/RestoreOriginalTimestamps/ReifyTimestamps.RemoveWildcard/ParDo(Anonymous)/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/RestoreOriginalTimestamps/Reify.ExtractTimestampsFromValues/ParDo(Anonymous)/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Values/Values/Map/ParMultiDo(Anonymous)+Query14/Query14.Debug/ParMultiDo(Monitor)+Query14/Query14.Stamp/ParMultiDo(Anonymous)+Query14.Format/ParMultiDo(Anonymous)+Query14.DevNull/ParMultiDo(Anonymous)
Oct 24, 2023 4:28:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:56.643Z: Finished operation
Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/RestoreOriginalTimestamps/ReifyTimestamps.RemoveWildcard/ParDo(Anonymous)/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Reshuffle/RestoreOriginalTimestamps/Reify.ExtractTimestampsFromValues/ParDo(Anonymous)/ParMultiDo(Anonymous)+Query14/Query14/Reshuffle.ViaRandomKey/Values/Values/Map/ParMultiDo(Anonymous)+Query14/Query14.Debug/ParMultiDo(Monitor)+Query14/Query14.Stamp/ParMultiDo(Anonymous)+Query14.Format/ParMultiDo(Anonymous)+Query14.DevNull/ParMultiDo(Anonymous)
Oct 24, 2023 4:28:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:56.747Z: Cleaning up.
Oct 24, 2023 4:28:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-10-24T04:28:56.805Z: Stopping **** pool...
Oct 24, 2023 4:29:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
2023-10-24T04:29:49.444Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2023-10-24T04:29:49.470Z RUNNING Query2
Oct 24, 2023 4:29:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2023-10-24T04:29:49.543Z Waiting for main pipeline to 'finish'
2023-10-24T04:29:49.568Z RUNNING Query14
2023-10-24T04:29:50.263Z new perf
{"snapshots":null,"runtimeSec":58.276,"numEvents":10000000,"eventsPerSec":171597.22698881186,"eventBytesPerSec":2.202561769853799E7,"numResults":10000000,"resultsPerSec":171597.22698881186,"resultBytesPerSec":2.202561769853799E7,"startupDelaySec":-111.284,"processingDelaySec":29.1,"shutdownDelaySec":53.008,"timeDilation":17.159551101654195,"errors":null,"jobId":null}
2023-10-24T04:29:50.326Z new perf
{"snapshots":null,"runtimeSec":13.332,"numEvents":10000000,"eventsPerSec":750075.00750075,"eventBytesPerSec":9.627699497449744E7,"numResults":83527,"resultsPerSec":6265.151515151515,"resultBytesPerSec":100242.42424242424,"startupDelaySec":-125.566,"processingDelaySec":0.359,"shutdownDelaySec":112.235,"timeDilation":75.00300030003001,"errors":null,"jobId":null}
2023-10-24T04:30:05.318Z RUNNING Query14
2023-10-24T04:30:05.369Z RUNNING Query2
2023-10-24T04:30:05.957Z no activity
2023-10-24T04:30:05.995Z no activity
2023-10-24T04:30:21.006Z RUNNING Query14
2023-10-24T04:30:21.043Z RUNNING Query2
2023-10-24T04:30:21.782Z no activity
2023-10-24T04:30:21.783Z no activity
2023-10-24T04:30:36.822Z RUNNING Query14
2023-10-24T04:30:36.833Z DONE Query2
==========================================================================================
Run started 2023-10-24T04:25:33.058Z and ran for PT303.798S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0002 13.3 750075.0 83527
0003 *** not run ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 *** not run ***
0008 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2023-10-24T04:30:37.411Z no activity
2023-10-24T04:30:52.454Z RUNNING Query14
2023-10-24T04:30:53.331Z no activity
2023-10-24T04:31:08.374Z RUNNING Query14
2023-10-24T04:31:09.061Z no activity
2023-10-24T04:31:24.116Z DONE Query14
==========================================================================================
Run started 2023-10-24T04:25:33.058Z and ran for PT351.085S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0002 13.3 750075.0 83527
0003 *** not run ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 *** not run ***
0008 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 58.3 171597.2 10000000
0015 *** not run ***
0016 *** not run ***
==========================================================================================
==========================================================================================
Run started 2023-10-24T04:25:33.058Z and ran for PT351.294S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0001 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0002 13.3 750075.0 83527
0003 *** not run ***
0004 *** not run ***
0005 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0006 *** not run ***
0007 *** not run ***
0008 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0009 *** not run ***
0010 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0011 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0012 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0013 -1.0 -1.0 -1
*** Job was unexpectedly updated ***
0014 58.3 171597.2 10000000
0015 *** not run ***
0016 *** not run ***
==========================================================================================
Exception in thread "main" java.lang.RuntimeException: Execution was not
successful
at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:169)
at org.apache.beam.sdk.nexmark.Main.main(Main.java:441)
> Task :sdks:java:testing:nexmark:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20231024042444
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9af57e87f0cd07eaa25e97e8f930f66cd6471d034a2ff651d2e469d9c854797c
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20231024042444]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9af57e87f0cd07eaa25e97e8f930f66cd6471d034a2ff651d2e469d9c854797c]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20231024042444]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9af57e87f0cd07eaa25e97e8f930f66cd6471d034a2ff651d2e469d9c854797c])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9af57e87f0cd07eaa25e97e8f930f66cd6471d034a2ff651d2e469d9c854797c
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:nexmark:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
For more on this, please refer to
https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings
in the Gradle documentation.
BUILD FAILED in 7m
143 actionable tasks: 99 executed, 42 from cache, 2 up-to-date
Publishing build scan...
https://ge.apache.org/s/3pyysce47c4u6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
