See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java17/218/display/redirect?page=changes>
Changes:
[Carl Yeksigian] Cache bucket matcher regex in GcsPath
[noreply] [BEAM-4032]Support staging binary distributions of dependency packages
------------------------------------------
[...truncated 252.26 KB...]
2022-02-11T18:58:41.696Z RUNNING Query0
2022-02-11T18:58:41.728Z RUNNING Query14
Feb 11, 2022 6:58:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2022-02-11T18:58:41.844Z Waiting for main pipeline to 'finish'
2022-02-11T18:58:41.890Z new perf
{"snapshots":null,"runtimeSec":35.872,"numEvents":10000000,"eventsPerSec":278768.95628902764,"eventBytesPerSec":3.578180466659233E7,"numResults":1839657,"resultsPerSec":51283.92618198037,"resultBytesPerSec":820542.818911686,"startupDelaySec":-99.196,"processingDelaySec":27.445,"shutdownDelaySec":63.324,"timeDilation":27.875501784121322,"errors":null,"jobId":null}
2022-02-11T18:58:41.906Z RUNNING Query12
Feb 11, 2022 6:58:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2022-02-11T18:58:41.910Z Waiting for main pipeline to 'finish'
2022-02-11T18:58:41.938Z RUNNING Query1
Feb 11, 2022 6:58:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2022-02-11T18:58:41.940Z Waiting for main pipeline to 'finish'
Feb 11, 2022 6:58:41 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value
RUNNING
2022-02-11T18:58:41.964Z Waiting for main pipeline to 'finish'
2022-02-11T18:58:41.980Z RUNNING Query2
2022-02-11T18:58:41.992Z RUNNING Query5
2022-02-11T18:58:42.200Z new perf
{"snapshots":null,"runtimeSec":20.789,"numEvents":10000000,"eventsPerSec":481023.61825965653,"eventBytesPerSec":6.174250310260233E7,"numResults":92833,"resultsPerSec":4465.486555389869,"resultBytesPerSec":127192.02462840924,"startupDelaySec":-105.187,"processingDelaySec":17.863,"shutdownDelaySec":84.398,"timeDilation":47.621338207705996,"errors":null,"jobId":null}
2022-02-11T18:58:42.286Z new perf
{"snapshots":null,"runtimeSec":24.14,"numEvents":10000000,"eventsPerSec":414250.20712510357,"eventBytesPerSec":5.317170244407622E7,"numResults":10000000,"resultsPerSec":414250.20712510357,"resultBytesPerSec":5.317170244407622E7,"startupDelaySec":-102.479,"processingDelaySec":0.153,"shutdownDelaySec":78.339,"timeDilation":41.42460646230323,"errors":null,"jobId":null}
2022-02-11T18:58:42.548Z new perf
{"snapshots":null,"runtimeSec":53.857,"numEvents":10000000,"eventsPerSec":185676.88508457583,"eventBytesPerSec":2.383283318788644E7,"numResults":10000000,"resultsPerSec":185676.88508457583,"resultBytesPerSec":2.383283318788644E7,"startupDelaySec":-95.459,"processingDelaySec":23.379,"shutdownDelaySec":41.602,"timeDilation":18.567502831572497,"errors":null,"jobId":null}
2022-02-11T18:58:42.590Z new perf
{"snapshots":null,"runtimeSec":10.183,"numEvents":10000000,"eventsPerSec":982028.8716488265,"eventBytesPerSec":1.2604977874889521E8,"numResults":199929,"resultsPerSec":19633.60502798782,"resultBytesPerSec":314137.68044780515,"startupDelaySec":-116.34,"processingDelaySec":9.335,"shutdownDelaySec":106.157,"timeDilation":-1.0,"errors":null,"jobId":null}
2022-02-11T18:58:42.626Z new perf
{"snapshots":null,"runtimeSec":18.178,"numEvents":10000000,"eventsPerSec":550115.5242600946,"eventBytesPerSec":7.061089762350093E7,"numResults":9200000,"resultsPerSec":506106.282319287,"resultBytesPerSec":5.0863381340081416E7,"startupDelaySec":-92.781,"processingDelaySec":0.304,"shutdownDelaySec":74.603,"timeDilation":55.00880184838816,"errors":null,"jobId":null}
2022-02-11T18:58:42.628Z new perf
{"snapshots":null,"runtimeSec":47.671,"numEvents":10000000,"eventsPerSec":209771.1396866019,"eventBytesPerSec":2.692548713054058E7,"numResults":100,"resultsPerSec":2.097711396866019,"resultBytesPerSec":96.49472425583689,"startupDelaySec":-82.82,"processingDelaySec":47.51,"shutdownDelaySec":35.149,"timeDilation":20.76734282897359,"errors":null,"jobId":null}
2022-02-11T18:58:42.692Z new perf
{"snapshots":null,"runtimeSec":29.474,"numEvents":10000000,"eventsPerSec":339282.07912058086,"eventBytesPerSec":4.354905669403542E7,"numResults":4567,"resultsPerSec":154.95012553436928,"resultBytesPerSec":2479.2020085499084,"startupDelaySec":-94.583,"processingDelaySec":29.235,"shutdownDelaySec":65.109,"timeDilation":33.928207912058085,"errors":null,"jobId":null}
2022-02-11T18:58:42.701Z new perf
{"snapshots":null,"runtimeSec":14.568,"numEvents":10000000,"eventsPerSec":686436.024162548,"eventBytesPerSec":8.810851846512905E7,"numResults":83527,"resultsPerSec":5733.594179022515,"resultBytesPerSec":91737.50686436024,"startupDelaySec":-111.952,"processingDelaySec":0.393,"shutdownDelaySec":97.384,"timeDilation":68.63948380010983,"errors":null,"jobId":null}
2022-02-11T18:58:56.592Z RUNNING Query13
2022-02-11T18:58:56.914Z RUNNING Query11
2022-02-11T18:58:57.245Z RUNNING Query8
2022-02-11T18:58:57.254Z no activity
2022-02-11T18:58:57.317Z RUNNING Query0
2022-02-11T18:58:57.532Z no activity
2022-02-11T18:58:57.593Z RUNNING Query14
2022-02-11T18:58:57.619Z RUNNING Query12
2022-02-11T18:58:57.656Z RUNNING Query1
2022-02-11T18:58:57.670Z RUNNING Query10
2022-02-11T18:58:57.720Z RUNNING Query5
2022-02-11T18:58:57.733Z RUNNING Query2
2022-02-11T18:58:57.815Z no activity
2022-02-11T18:58:57.860Z no activity
2022-02-11T18:58:58.209Z no activity
2022-02-11T18:58:58.236Z no activity
2022-02-11T18:58:58.293Z no activity
2022-02-11T18:58:58.321Z no activity
2022-02-11T18:58:58.372Z no activity
2022-02-11T18:58:58.553Z no activity
2022-02-11T18:59:12.312Z RUNNING Query13
2022-02-11T18:59:12.567Z RUNNING Query11
2022-02-11T18:59:12.843Z RUNNING Query0
2022-02-11T18:59:12.854Z no activity
2022-02-11T18:59:12.898Z RUNNING Query8
2022-02-11T18:59:13.143Z no activity
2022-02-11T18:59:13.257Z RUNNING Query14
2022-02-11T18:59:13.270Z RUNNING Query12
2022-02-11T18:59:13.323Z RUNNING Query1
2022-02-11T18:59:13.347Z RUNNING Query10
2022-02-11T18:59:13.401Z RUNNING Query2
2022-02-11T18:59:13.415Z no activity
2022-02-11T18:59:13.599Z RUNNING Query5
2022-02-11T18:59:13.629Z no activity
2022-02-11T18:59:13.776Z no activity
2022-02-11T18:59:13.804Z no activity
2022-02-11T18:59:13.893Z no activity
2022-02-11T18:59:14.008Z no activity
2022-02-11T18:59:14.597Z no activity
2022-02-11T18:59:14.686Z no activity
2022-02-11T18:59:27.891Z RUNNING Query13
2022-02-11T18:59:28.219Z RUNNING Query11
2022-02-11T18:59:28.447Z RUNNING Query0
2022-02-11T18:59:28.669Z RUNNING Query8
2022-02-11T18:59:28.740Z no activity
2022-02-11T18:59:28.799Z RUNNING Query12
2022-02-11T18:59:28.834Z RUNNING Query1
2022-02-11T18:59:28.904Z no activity
2022-02-11T18:59:28.917Z RUNNING Query14
2022-02-11T18:59:29.033Z RUNNING Query2
2022-02-11T18:59:29.079Z no activity
2022-02-11T18:59:29.337Z no activity
2022-02-11T18:59:29.490Z no activity
2022-02-11T18:59:29.610Z no activity
2022-02-11T18:59:29.630Z RUNNING Query10
2022-02-11T18:59:29.630Z no activity
2022-02-11T18:59:29.751Z RUNNING Query5
2022-02-11T18:59:29.761Z no activity
2022-02-11T18:59:30.670Z no activity
2022-02-11T18:59:30.689Z no activity
2022-02-11T18:59:43.773Z RUNNING Query13
2022-02-11T18:59:43.938Z RUNNING Query11
2022-02-11T18:59:44.113Z RUNNING Query0
2022-02-11T18:59:44.370Z RUNNING Query8
2022-02-11T18:59:44.550Z no activity
2022-02-11T18:59:44.551Z RUNNING Query1
2022-02-11T18:59:44.606Z no activity
2022-02-11T18:59:44.646Z RUNNING Query14
2022-02-11T18:59:44.656Z DONE Query12
==========================================================================================
Run started 2022-02-11T18:54:23.584Z and ran for PT321.098S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 *** not run ***
0002 *** not run ***
0003 *** not run ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 10.2 982028.9 199929
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2022-02-11T18:59:44.716Z no activity
2022-02-11T18:59:44.787Z DONE Query2
==========================================================================================
Run started 2022-02-11T18:54:23.584Z and ran for PT321.229S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 *** not run ***
0001 *** not run ***
0002 14.6 686436.0 83527
0003 *** not run ***
0004 *** not run ***
0005 *** not run ***
0006 *** not run ***
0007 *** not run ***
0008 *** not run ***
0009 *** not run ***
0010 *** not run ***
0011 *** not run ***
0012 10.2 982028.9 199929
0013 *** not run ***
0014 *** not run ***
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2022-02-11T18:59:45.061Z no activity
2022-02-11T18:59:45.259Z no activity
2022-02-11T18:59:45.274Z no activity
2022-02-11T18:59:45.695Z RUNNING Query10
2022-02-11T18:59:45.724Z RUNNING Query5
2022-02-11T18:59:46.221Z no activity
2022-02-11T18:59:46.605Z no activity
> Task :sdks:java:testing:nexmark:run FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=339722fd-464e-4f27-8b30-2fcfa25a8b77,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java17/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 2152883
log file: /home/jenkins/.gradle/daemon/7.3.2/daemon-2152883.out.log
----- Last 20 lines from daemon log file - daemon-2152883.out.log -----
2022-02-11T18:59:46.605Z no activity
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220211185228
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2e6c06a0690d9063db82f6f9a0f80d763aae963b5e1905f2b127d0c94117077e
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
Remove shutdown hook failed
java.lang.IllegalStateException: Shutdown in progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at
org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38)
at
org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at
org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at
org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at
org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
at
org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
at java.lang.Thread.run(Thread.java:748)
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220211185228
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:2e6c06a0690d9063db82f6f9a0f80d763aae963b5e1905f2b127d0c94117077e
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
