garydgregory commented on pull request #812:
URL: https://github.com/apache/commons-lang/pull/812#issuecomment-972928104
This is what I see on Microsoft Windows [Version 10.0.19042.1288], Java 8,
and 11:
### StringJoinerBenchmark.stringJoinerPrimitiveInt on Java 8
```
8:29:24.04 C:\temp\apache-commons-lang-jmh\string-joiner>java -jar
target/benchmarks.jar
# JMH version: 1.33
# VM version: JDK 1.8.0_312, OpenJDK 64-Bit Server VM, 25.312-b07
# VM invoker: C:\Program Files\Eclipse
Adoptium\jdk-8.0.312.7-hotspot\jre\bin\java.exe
# VM options: <none>
# Blackhole mode: full + dont-inline hint (default, use
-Djmh.blackhole.autoDetect=true to auto-detect)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark:
com.github.hubertwo.acljmh.StringJoinerBenchmark.stringJoinerPrimitiveInt
# Run progress: 0.00% complete, ETA 00:08:20
# Fork: 1 of 5
# Warmup Iteration 1: 4028049.411 ops/s
# Warmup Iteration 2: 4760495.147 ops/s
# Warmup Iteration 3: 5272759.983 ops/s
# Warmup Iteration 4: 5217695.792 ops/s
# Warmup Iteration 5: 5281258.927 ops/s
Iteration 1: 5223688.819 ops/s
Iteration 2: 5818978.183 ops/s
Iteration 3: 5632986.395 ops/s
Iteration 4: 5972299.312 ops/s
Iteration 5: 5729786.303 ops/s
# Run progress: 20.00% complete, ETA 00:06:43
# Fork: 2 of 5
# Warmup Iteration 1: 5754531.770 ops/s
# Warmup Iteration 2: 6204029.918 ops/s
# Warmup Iteration 3: 5997509.994 ops/s
# Warmup Iteration 4: 6068735.288 ops/s
# Warmup Iteration 5: 6259803.017 ops/s
Iteration 1: 6097840.957 ops/s
Iteration 2: 5893229.379 ops/s
Iteration 3: 6310146.092 ops/s
Iteration 4: 6137171.976 ops/s
Iteration 5: 5792914.747 ops/s
# Run progress: 40.00% complete, ETA 00:05:02
# Fork: 3 of 5
# Warmup Iteration 1: 6154276.895 ops/s
# Warmup Iteration 2: 6103484.534 ops/s
# Warmup Iteration 3: 6218572.312 ops/s
# Warmup Iteration 4: 6004638.320 ops/s
# Warmup Iteration 5: 6268515.946 ops/s
Iteration 1: 6133699.105 ops/s
Iteration 2: 5754235.565 ops/s
Iteration 3: 6157983.152 ops/s
Iteration 4: 6049810.564 ops/s
Iteration 5: 6281478.040 ops/s
# Run progress: 60.00% complete, ETA 00:03:21
# Fork: 4 of 5
# Warmup Iteration 1: 6142365.003 ops/s
# Warmup Iteration 2: 6397104.178 ops/s
# Warmup Iteration 3: 6243445.936 ops/s
# Warmup Iteration 4: 6309976.325 ops/s
# Warmup Iteration 5: 6190286.103 ops/s
Iteration 1: 6503690.619 ops/s
Iteration 2: 6381886.335 ops/s
Iteration 3: 6218666.379 ops/s
Iteration 4: 6429726.336 ops/s
Iteration 5: 6123348.392 ops/s
# Run progress: 80.00% complete, ETA 00:01:40
# Fork: 5 of 5
# Warmup Iteration 1: 6219247.906 ops/s
# Warmup Iteration 2: 6613614.639 ops/s
# Warmup Iteration 3: 6578974.636 ops/s
# Warmup Iteration 4: 6380279.170 ops/s
# Warmup Iteration 5: 6555631.361 ops/s
Iteration 1: 6459206.567 ops/s
Iteration 2: 6399048.111 ops/s
Iteration 3: 6573701.097 ops/s
Iteration 4: 6561843.207 ops/s
Iteration 5: 6490106.995 ops/s
Result
"com.github.hubertwo.acljmh.StringJoinerBenchmark.stringJoinerPrimitiveInt":
6125098.905 ±(99.9%) 252123.433 ops/s [Average]
(min, avg, max) = (5223688.819, 6125098.905, 6573701.097), stdev =
336577.570
CI (99.9%): [5872975.472, 6377222.338] (assumes normal distribution)
# Run complete. Total time: 00:08:23
REMEMBER: The numbers below are just data. To gain reusable insights, you
need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof),
design factorial
experiments, perform baseline and negative tests that provide experimental
control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews
from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score
Error Units
StringJoinerBenchmark.stringJoinerPrimitiveInt thrpt 25 6125098.905 ±
252123.433 ops/s
```
### StringBuilderBenchmark.stringBuilderPrimitiveInt on Java 8
```
8:41:58.46 C:\temp\apache-commons-lang-jmh\string-builder>java -jar
target/benchmarks.jar
# JMH version: 1.33
# VM version: JDK 1.8.0_312, OpenJDK 64-Bit Server VM, 25.312-b07
# VM invoker: C:\Program Files\Eclipse
Adoptium\jdk-8.0.312.7-hotspot\jre\bin\java.exe
# VM options: <none>
# Blackhole mode: full + dont-inline hint (default, use
-Djmh.blackhole.autoDetect=true to auto-detect)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark:
com.github.hubertwo.acljmh.StringBuilderBenchmark.stringBuilderPrimitiveInt
# Run progress: 0.00% complete, ETA 00:08:20
# Fork: 1 of 5
# Warmup Iteration 1: 5638828.105 ops/s
# Warmup Iteration 2: 6012329.374 ops/s
# Warmup Iteration 3: 6028870.456 ops/s
# Warmup Iteration 4: 6378532.707 ops/s
# Warmup Iteration 5: 6415691.661 ops/s
Iteration 1: 6402935.472 ops/s
Iteration 2: 6403059.626 ops/s
Iteration 3: 6281448.263 ops/s
Iteration 4: 6485849.516 ops/s
Iteration 5: 6413346.541 ops/s
# Run progress: 20.00% complete, ETA 00:06:42
# Fork: 2 of 5
# Warmup Iteration 1: 5940727.783 ops/s
# Warmup Iteration 2: 6298039.536 ops/s
# Warmup Iteration 3: 6128525.034 ops/s
# Warmup Iteration 4: 6255026.777 ops/s
# Warmup Iteration 5: 5969974.958 ops/s
Iteration 1: 6240497.474 ops/s
Iteration 2: 6339066.113 ops/s
Iteration 3: 6246787.751 ops/s
Iteration 4: 6191611.372 ops/s
Iteration 5: 5393780.839 ops/s
# Run progress: 40.00% complete, ETA 00:05:02
# Fork: 3 of 5
# Warmup Iteration 1: 6292573.853 ops/s
# Warmup Iteration 2: 6550340.320 ops/s
# Warmup Iteration 3: 6586276.662 ops/s
# Warmup Iteration 4: 6475453.557 ops/s
# Warmup Iteration 5: 6445092.592 ops/s
Iteration 1: 6551636.499 ops/s
Iteration 2: 6494809.307 ops/s
Iteration 3: 6443369.421 ops/s
Iteration 4: 6437411.692 ops/s
Iteration 5: 6419118.459 ops/s
# Run progress: 60.00% complete, ETA 00:03:21
# Fork: 4 of 5
# Warmup Iteration 1: 5838425.211 ops/s
# Warmup Iteration 2: 6051767.339 ops/s
# Warmup Iteration 3: 6295104.534 ops/s
# Warmup Iteration 4: 6222992.137 ops/s
# Warmup Iteration 5: 6173991.713 ops/s
Iteration 1: 6236300.996 ops/s
Iteration 2: 6275138.569 ops/s
Iteration 3: 5890347.878 ops/s
Iteration 4: 6263455.340 ops/s
Iteration 5: 6335664.159 ops/s
# Run progress: 80.00% complete, ETA 00:01:40
# Fork: 5 of 5
# Warmup Iteration 1: 6141810.962 ops/s
# Warmup Iteration 2: 6396032.327 ops/s
# Warmup Iteration 3: 6360946.084 ops/s
# Warmup Iteration 4: 6324751.367 ops/s
# Warmup Iteration 5: 6273036.646 ops/s
Iteration 1: 6421965.689 ops/s
Iteration 2: 6508246.906 ops/s
Iteration 3: 6402817.891 ops/s
Iteration 4: 6146288.108 ops/s
Iteration 5: 6358559.337 ops/s
Result
"com.github.hubertwo.acljmh.StringBuilderBenchmark.stringBuilderPrimitiveInt":
6303340.529 ±(99.9%) 176697.569 ops/s [Average]
(min, avg, max) = (5393780.839, 6303340.529, 6551636.499), stdev =
235886.199
CI (99.9%): [6126642.960, 6480038.097] (assumes normal distribution)
# Run complete. Total time: 00:08:23
REMEMBER: The numbers below are just data. To gain reusable insights, you
need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof),
design factorial
experiments, perform baseline and negative tests that provide experimental
control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews
from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score
Error Units
StringBuilderBenchmark.stringBuilderPrimitiveInt thrpt 25 6303340.529 ±
176697.569 ops/s
```
To summarize Java 8:
```
Benchmark Mode Cnt Score
Error Units
StringJoinerBenchmark.stringJoinerPrimitiveInt thrpt 25 6125098.905 ±
252123.433 ops/s
StringBuilderBenchmark.stringBuilderPrimitiveInt thrpt 25 6303340.529 ±
176697.569 ops/s
```
StringJoinerBenchmark.stringJoinerPrimitiveInt on Java 11
```
9:01:02.35 C:\temp\apache-commons-lang-jmh\string-joiner>java -jar
target/benchmarks.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils
(file:/C:/temp/apache-commons-lang-jmh/string-joiner/target/benchmarks.jar) to
method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of
org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.33
# VM version: JDK 11.0.13, OpenJDK 64-Bit Server VM, 11.0.13+8
# VM invoker: C:\Program Files\Eclipse
Adoptium\jdk-11.0.13.8-hotspot\bin\java.exe
# VM options: <none>
# Blackhole mode: full + dont-inline hint (default, use
-Djmh.blackhole.autoDetect=true to auto-detect)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark:
com.github.hubertwo.acljmh.StringJoinerBenchmark.stringJoinerPrimitiveInt
# Run progress: 0.00% complete, ETA 00:08:20
# Fork: 1 of 5
# Warmup Iteration 1: 6211311.992 ops/s
# Warmup Iteration 2: 6747262.431 ops/s
# Warmup Iteration 3: 6558912.817 ops/s
# Warmup Iteration 4: 6771742.789 ops/s
# Warmup Iteration 5: 6763107.701 ops/s
Iteration 1: 6798738.867 ops/s
Iteration 2: 6777925.203 ops/s
Iteration 3: 6647225.301 ops/s
Iteration 4: 6614494.453 ops/s
Iteration 5: 6737617.471 ops/s
# Run progress: 20.00% complete, ETA 00:06:43
# Fork: 2 of 5
# Warmup Iteration 1: 6459452.620 ops/s
# Warmup Iteration 2: 6382410.985 ops/s
# Warmup Iteration 3: 6489995.342 ops/s
# Warmup Iteration 4: 6393524.885 ops/s
# Warmup Iteration 5: 5875825.717 ops/s
Iteration 1: 5822396.839 ops/s
Iteration 2: 6413539.391 ops/s
Iteration 3: 6401168.928 ops/s
Iteration 4: 6473265.494 ops/s
Iteration 5: 6546418.900 ops/s
# Run progress: 40.00% complete, ETA 00:05:02
# Fork: 3 of 5
# Warmup Iteration 1: 6374389.170 ops/s
# Warmup Iteration 2: 6450623.105 ops/s
# Warmup Iteration 3: 6700384.526 ops/s
# Warmup Iteration 4: 5818958.017 ops/s
# Warmup Iteration 5: 5864596.372 ops/s
Iteration 1: 6615271.944 ops/s
Iteration 2: 6571880.374 ops/s
Iteration 3: 6654752.846 ops/s
Iteration 4: 6549549.943 ops/s
Iteration 5: 6599145.093 ops/s
# Run progress: 60.00% complete, ETA 00:03:21
# Fork: 4 of 5
# Warmup Iteration 1: 6473242.760 ops/s
# Warmup Iteration 2: 6379801.578 ops/s
# Warmup Iteration 3: 6252289.638 ops/s
# Warmup Iteration 4: 6505810.812 ops/s
# Warmup Iteration 5: 6505187.390 ops/s
Iteration 1: 6525028.117 ops/s
Iteration 2: 6489421.325 ops/s
Iteration 3: 6175421.960 ops/s
Iteration 4: 6322478.938 ops/s
Iteration 5: 6407571.523 ops/s
# Run progress: 80.00% complete, ETA 00:01:40
# Fork: 5 of 5
# Warmup Iteration 1: 6368889.086 ops/s
# Warmup Iteration 2: 6545990.921 ops/s
# Warmup Iteration 3: 6492715.633 ops/s
# Warmup Iteration 4: 6610368.781 ops/s
# Warmup Iteration 5: 6143078.704 ops/s
Iteration 1: 6742870.379 ops/s
Iteration 2: 6709030.111 ops/s
Iteration 3: 6703347.856 ops/s
Iteration 4: 6603775.783 ops/s
Iteration 5: 6656342.851 ops/s
Result
"com.github.hubertwo.acljmh.StringJoinerBenchmark.stringJoinerPrimitiveInt":
6542347.196 ±(99.9%) 158331.625 ops/s [Average]
(min, avg, max) = (5822396.839, 6542347.196, 6798738.867), stdev =
211368.189
CI (99.9%): [6384015.571, 6700678.820] (assumes normal distribution)
# Run complete. Total time: 00:08:23
REMEMBER: The numbers below are just data. To gain reusable insights, you
need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof),
design factorial
experiments, perform baseline and negative tests that provide experimental
control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews
from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score
Error Units
StringJoinerBenchmark.stringJoinerPrimitiveInt thrpt 25 6542347.196 ±
158331.625 ops/s
```
### StringBuilderBenchmark.stringBuilderPrimitiveInt on Java 11
```
9:09:36.63 C:\temp\apache-commons-lang-jmh\string-joiner>cd
..\string-builder
9:10:05.99 C:\temp\apache-commons-lang-jmh\string-builder>java -jar
target/benchmarks.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils
(file:/C:/temp/apache-commons-lang-jmh/string-builder/target/benchmarks.jar) to
method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of
org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.33
# VM version: JDK 11.0.13, OpenJDK 64-Bit Server VM, 11.0.13+8
# VM invoker: C:\Program Files\Eclipse
Adoptium\jdk-11.0.13.8-hotspot\bin\java.exe
# VM options: <none>
# Blackhole mode: full + dont-inline hint (default, use
-Djmh.blackhole.autoDetect=true to auto-detect)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark:
com.github.hubertwo.acljmh.StringBuilderBenchmark.stringBuilderPrimitiveInt
# Run progress: 0.00% complete, ETA 00:08:20
# Fork: 1 of 5
# Warmup Iteration 1: 6124265.805 ops/s
# Warmup Iteration 2: 6280599.406 ops/s
# Warmup Iteration 3: 5537584.025 ops/s
# Warmup Iteration 4: 5442440.575 ops/s
# Warmup Iteration 5: 6005466.091 ops/s
Iteration 1: 6303651.443 ops/s
Iteration 2: 6351930.132 ops/s
Iteration 3: 6462915.675 ops/s
Iteration 4: 6261262.701 ops/s
Iteration 5: 4764485.626 ops/s
# Run progress: 20.00% complete, ETA 00:06:43
# Fork: 2 of 5
# Warmup Iteration 1: 5610674.138 ops/s
# Warmup Iteration 2: 5716158.382 ops/s
# Warmup Iteration 3: 6387316.364 ops/s
# Warmup Iteration 4: 6389239.939 ops/s
# Warmup Iteration 5: 6230817.570 ops/s
Iteration 1: 6294865.152 ops/s
Iteration 2: 6487501.018 ops/s
Iteration 3: 6385049.433 ops/s
Iteration 4: 6332667.452 ops/s
Iteration 5: 6395522.434 ops/s
# Run progress: 40.00% complete, ETA 00:05:02
# Fork: 3 of 5
# Warmup Iteration 1: 6501740.764 ops/s
# Warmup Iteration 2: 6245764.152 ops/s
# Warmup Iteration 3: 6325628.565 ops/s
# Warmup Iteration 4: 6362356.744 ops/s
# Warmup Iteration 5: 6445932.539 ops/s
Iteration 1: 6317627.507 ops/s
Iteration 2: 6415113.085 ops/s
Iteration 3: 6368157.935 ops/s
Iteration 4: 6132464.782 ops/s
Iteration 5: 6418124.475 ops/s
# Run progress: 60.00% complete, ETA 00:03:21
# Fork: 4 of 5
# Warmup Iteration 1: 5816353.649 ops/s
# Warmup Iteration 2: 6271228.168 ops/s
# Warmup Iteration 3: 5020208.787 ops/s
# Warmup Iteration 4: 5472176.864 ops/s
# Warmup Iteration 5: 6339575.933 ops/s
Iteration 1: 6448960.953 ops/s
Iteration 2: 6415139.122 ops/s
Iteration 3: 6352952.242 ops/s
Iteration 4: 6206598.587 ops/s
Iteration 5: 6316547.353 ops/s
# Run progress: 80.00% complete, ETA 00:01:40
# Fork: 5 of 5
# Warmup Iteration 1: 6017145.794 ops/s
# Warmup Iteration 2: 5962584.069 ops/s
# Warmup Iteration 3: 5923748.905 ops/s
# Warmup Iteration 4: 5431951.901 ops/s
# Warmup Iteration 5: 5939084.630 ops/s
Iteration 1: 5953082.874 ops/s
Iteration 2: 6310576.360 ops/s
Iteration 3: 6299812.394 ops/s
Iteration 4: 6253949.139 ops/s
Iteration 5: 6603114.825 ops/s
Result
"com.github.hubertwo.acljmh.StringBuilderBenchmark.stringBuilderPrimitiveInt":
6274082.908 ±(99.9%) 253394.583 ops/s [Average]
(min, avg, max) = (4764485.626, 6274082.908, 6603114.825), stdev =
338274.518
CI (99.9%): [6020688.325, 6527477.491] (assumes normal distribution)
# Run complete. Total time: 00:08:23
REMEMBER: The numbers below are just data. To gain reusable insights, you
need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof),
design factorial
experiments, perform baseline and negative tests that provide experimental
control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews
from the domain experts.
Do not assume the numbers tell you what you want them to tell.
Benchmark Mode Cnt Score
Error Units
StringBuilderBenchmark.stringBuilderPrimitiveInt thrpt 25 6274082.908 ±
253394.583 ops/s
```
To summarize Java 11:
```
Benchmark Mode Cnt Score
Error Units
StringJoinerBenchmark.stringJoinerPrimitiveInt thrpt 25 6542347.196 ±
158331.625 ops/s
StringBuilderBenchmark.stringBuilderPrimitiveInt thrpt 25 6274082.908 ±
253394.583 ops/s
```
To summarize Java 8 and 11:
```
Benchmark Mode Cnt Score
Error Units
StringJoinerBenchmark.stringJoinerPrimitiveInt Java 8 thrpt 25
6125098.905 ± 252123.433 ops/s
StringJoinerBenchmark.stringJoinerPrimitiveInt Java 11 thrpt 25
6542347.196 ± 158331.625 ops/s
StringBuilderBenchmark.stringBuilderPrimitiveInt Java 8 thrpt 25
6303340.529 ± 176697.569 ops/s
StringBuilderBenchmark.stringBuilderPrimitiveInt Java 11 thrpt 25
6274082.908 ± 253394.583 ops/s
```
These results look mixed to me: The PR implementation overlaps the current
code accounting for the margin of errors.
Can anyone reproduce the results? Or any result?
I did not run Java 17 since I won't be able to use it in production for a
while.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
