See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java11/2549/display/redirect>
Changes:
------------------------------------------
[...truncated 286.12 KB...]
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 24.5 408113.3 10000000
0001 20.7 482020.6 9200000
0002 12.0 830082.2 83527
0003 *** not run ***
0004 *** not run ***
0005 37.4 267029.8 4567
0006 *** not run ***
0007 *** not run ***
0008 17.8 562113.5 92833
0009 *** not run ***
0010 *** not run ***
0011 34.1 293668.5 1839657
0012 20.9 479019.0 199929
0013 43.1 232088.6 10000000
0014 51.3 195057.2 10000000
0015 *** not run ***
0016 *** not run ***
==========================================================================================
2023-02-24T10:34:19.714Z RUNNING Query10
2023-02-24T10:34:20.391Z no activity
2023-02-24T10:34:35.463Z DONE Query10
==========================================================================================
Run started 2023-02-24T10:28:10.150Z and ran for PT385.345S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 24.5 408113.3 10000000
0001 20.7 482020.6 9200000
0002 12.0 830082.2 83527
0003 *** not run ***
0004 *** not run ***
0005 37.4 267029.8 4567
0006 *** not run ***
0007 *** not run ***
0008 17.8 562113.5 92833
0009 *** not run ***
0010 58.3 171388.4 100
0011 34.1 293668.5 1839657
0012 20.9 479019.0 199929
0013 43.1 232088.6 10000000
0014 51.3 195057.2 10000000
0015 *** not run ***
0016 *** not run ***
==========================================================================================
==========================================================================================
Run started 2023-02-24T10:28:10.150Z and ran for PT385.742S
Default configuration:
{"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0}
Configurations:
Conf Description
0000 query:PASSTHROUGH; numEvents:10000000
0001 query:CURRENCY_CONVERSION; numEvents:10000000
0002 query:SELECTION; numEvents:10000000
0003 query:LOCAL_ITEM_SUGGESTION; numEvents:10000000
0004 query:AVERAGE_PRICE_FOR_CATEGORY; numEvents:1000000
0005 query:HOT_ITEMS; numEvents:10000000
0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; numEvents:1000000
0007 query:HIGHEST_BID; numEvents:10000000
0008 query:MONITOR_NEW_USERS; numEvents:10000000
0009 query:WINNING_BIDS; numEvents:1000000
0010 query:LOG_TO_SHARDED_FILES; numEvents:10000000
0011 query:USER_SESSIONS; numEvents:10000000
0012 query:PROCESSING_TIME_WINDOWS; numEvents:10000000
0013 query:PORTABILITY_BATCH; numEvents:10000000
0014 query:RESHUFFLE; numEvents:10000000
0015 query:BOUNDED_SIDE_INPUT_JOIN; numEvents:10000000
0016 query:SESSION_SIDE_INPUT_JOIN; numEvents:10000000
Performance:
Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results
(Baseline)
0000 24.5 408113.3 10000000
0001 20.7 482020.6 9200000
0002 12.0 830082.2 83527
0003 *** not run ***
0004 *** not run ***
0005 37.4 267029.8 4567
0006 *** not run ***
0007 *** not run ***
0008 17.8 562113.5 92833
0009 *** not run ***
0010 58.3 171388.4 100
0011 34.1 293668.5 1839657
0012 20.9 479019.0 199929
0013 43.1 232088.6 10000000
0014 51.3 195057.2 10000000
0015 *** not run ***
0016 *** not run ***
==========================================================================================
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230224102513
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230224102513]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230224102513]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f42c6df4b2602911a7e54db12b48e27a9d0ad80f995341ecc989b2cd08a71eb].
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD SUCCESSFUL in 10m 6s
144 actionable tasks: 97 executed, 43 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/imdm7yqtfwura
Build step 'Invoke Gradle script' changed build result to SUCCESS
[beam_PostCommit_Java_Nexmark_Dataflow_V2_Java11] $ /bin/bash -xe
/tmp/jenkins8744538030842246214.sh
+ echo '*** RUN NEXMARK IN STREAMING MODE USING DATAFLOW RUNNER with Java 11***'
*** RUN NEXMARK IN STREAMING MODE USING DATAFLOW RUNNER with Java 11***
[Gradle] - Launching build.
[src] $
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java11/ws/src/gradlew>
--continue --max-****s=8 -Dorg.gradle.jvmargs=-Xms2g
-Dorg.gradle.jvmargs=-Xmx6g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses
-PcompileAndRunTestsWithJava11 -Pjava11Home=/usr/lib/jvm/java-11-openjdk-amd64
-Pnexmark.runner=:runners:google-cloud-dataflow-java
'-Pnexmark.args=--manageResources=false --monitorJobs=true
--bigQueryTable=nexmark --bigQueryDataset=nexmark --project=apache-beam-testing
--resourceNameMode=QUERY_RUNNER_AND_MODE --exportSummaryToBigQuery=false
--tempLocation=gs://temp-storage-for-perf-tests/nexmark
--influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086
--baseInfluxMeasurement=nexmark --exportSummaryToInfluxDB=true
--influxRetentionPolicy=forever
--influxTags={"runnerVersion":"V2","javaVersion":"11"} --region=us-central1
--suite=STRESS --numWorkers=4 --maxNumWorkers=4 --autoscalingAlgorithm=NONE
--nexmarkParallel=16 --enforceEncodability=true --enforceImmutability=true
--runner=DataflowRunner --streaming=true' -Pnexmark.runner.version=V2
:sdks:java:testing:nexmark:run
To honour the JVM settings for this build a single-use Daemon process will be
forked. See
https://docs.gradle.org/7.5.1/userguide/gradle_daemon.html#sec:disabling_the_daemon.
FAILURE: Build failed with an exception.
* What went wrong:
Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.
Please refer to the User Manual chapter on the daemon at
https://docs.gradle.org/7.5.1/userguide/gradle_daemon.html
Process command line: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xmx6g
-Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp
/home/jenkins/.gradle/wrapper/dists/gradle-7.5.1-bin/7jzzequgds1hbszbhq3npc5ng/gradle-7.5.1/lib/gradle-launcher-7.5.1.jar
org.gradle.launcher.daemon.bootstrap.GradleDaemon 7.5.1
Please read the following process output to find out more:
-----------------------
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by
another Gradle instance.
Owner PID: unknown
Our PID: 3573411
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/7.5.1/registry.bin.lock
* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Exception is:
org.gradle.cache.LockTimeoutException: Timeout waiting to lock daemon addresses
registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 3573411
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/7.5.1/registry.bin.lock
at
org.gradle.cache.internal.DefaultFileLockManager$DefaultFileLock.timeoutException(DefaultFileLockManager.java:344)
at
org.gradle.cache.internal.DefaultFileLockManager$DefaultFileLock.lock(DefaultFileLockManager.java:305)
at
org.gradle.cache.internal.DefaultFileLockManager$DefaultFileLock.<init>(DefaultFileLockManager.java:164)
at
org.gradle.cache.internal.DefaultFileLockManager.lock(DefaultFileLockManager.java:110)
at
org.gradle.cache.internal.DefaultFileLockManager.lock(DefaultFileLockManager.java:96)
at
org.gradle.cache.internal.DefaultFileLockManager.lock(DefaultFileLockManager.java:91)
at
org.gradle.cache.internal.OnDemandFileAccess.updateFile(OnDemandFileAccess.java:51)
at
org.gradle.cache.internal.SimpleStateCache.update(SimpleStateCache.java:87)
at
org.gradle.cache.internal.FileIntegrityViolationSuppressingPersistentStateCacheDecorator$1.create(FileIntegrityViolationSuppressingPersistentStateCacheDecorator.java:50)
at
org.gradle.cache.internal.FileIntegrityViolationSuppressingPersistentStateCacheDecorator.doUpdate(FileIntegrityViolationSuppressingPersistentStateCacheDecorator.java:67)
at
org.gradle.cache.internal.FileIntegrityViolationSuppressingPersistentStateCacheDecorator.update(FileIntegrityViolationSuppressingPersistentStateCacheDecorator.java:47)
at
org.gradle.launcher.daemon.registry.PersistentDaemonRegistry.store(PersistentDaemonRegistry.java:232)
at
org.gradle.launcher.daemon.server.DaemonRegistryUpdater.onStart(DaemonRegistryUpdater.java:80)
at org.gradle.launcher.daemon.server.Daemon.start(Daemon.java:171)
at
org.gradle.launcher.daemon.bootstrap.DaemonMain.doAction(DaemonMain.java:125)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:60)
at
org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:37)
at
org.gradle.launcher.daemon.bootstrap.GradleDaemon.main(GradleDaemon.java:22)
* Get more help at https://help.gradle.org
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
