yuanchaodu commented on PR #1515:
URL: https://github.com/apache/commons-lang/pull/1515#issuecomment-3619686456
> Hello @yuanchaodu -1 until you provide a JMH benchmark to back up your
claim.
``` java
package org.apache;
import org.openjdk.jmh.annotations.*;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.ThreadLocalRandom;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@State(Scope.Thread)
public class StringUtilsGetDigitsBenchmark {
@Param({"10", "100", "1000", "10000"})
public int length;
private String testStr;
@Setup(Level.Invocation)
public void setup() {
StringBuilder sb = new StringBuilder(length);
for (int i = 0; i < length; i++) {
int code = ThreadLocalRandom.current().nextInt(33, 127);
sb.append((char) code);
}
testStr = sb.toString();
}
@Benchmark
public String testGetDigitsWithBuilder() {
return getDigitsWithBuilder(testStr);
}
@Benchmark
public String testGetDigitsManually() {
return getDigitsManually(testStr);
}
public static String getDigitsWithBuilder(final String str) {
final int sz = str.length();
final StringBuilder strDigits = new StringBuilder(sz);
for (int i = 0; i < sz; i++) {
final char tempChar = str.charAt(i);
if (Character.isDigit(tempChar)) {
strDigits.append(tempChar);
}
}
return strDigits.toString();
}
public static String getDigitsManually(final String str) {
final int len = str.length();
final char[] buffer = new char[len];
int count = 0;
for (int i = 0; i < len; i++) {
final char tempChar = str.charAt(i);
if (Character.isDigit(tempChar)) {
buffer[count++] = tempChar;
}
}
return new String(buffer, 0, count);
}
}
package org.apache;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
public class StringUtilsBenchmarkRunner {
public static void main(String[] args) throws Exception {
Options opt = new OptionsBuilder()
.include(StringUtilsGetDigitsBenchmark.class.getSimpleName())
.warmupIterations(5)
.measurementIterations(5)
.forks(1)
.build();
new Runner(opt).run();
}
}
```
JMH Results
``` bash
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
-javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8 -classpath
/Users/elonmusk/Workspace/Code/JMHTest/target/classes:/Users/elonmusk/.m2/repository/org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar:/Users/elonmusk/.m2/repository/net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.jar:/Users/elonmusk/.m2/repository/org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar
org.apache.StringUtilsBenchmarkRunner
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually
# Parameters: (length = 10)
# Run progress: 0.00% complete, ETA 00:13:20
# Fork: 1 of 1
# Warmup Iteration 1: 52.196 ns/op
# Warmup Iteration 2: 48.994 ns/op
# Warmup Iteration 3: 48.280 ns/op
# Warmup Iteration 4: 47.547 ns/op
# Warmup Iteration 5: 47.343 ns/op
Iteration 1: 48.120 ns/op
Iteration 2: 47.759 ns/op
Iteration 3: 48.558 ns/op
Iteration 4: 47.209 ns/op
Iteration 5: 47.972 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually":
47.924 ±(99.9%) 1.907 ns/op [Average]
(min, avg, max) = (47.209, 47.924, 48.558), stdev = 0.495
CI (99.9%): [46.017, 49.831] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually
# Parameters: (length = 100)
# Run progress: 12.50% complete, ETA 00:11:45
# Fork: 1 of 1
# Warmup Iteration 1: 160.338 ns/op
# Warmup Iteration 2: 160.424 ns/op
# Warmup Iteration 3: 159.402 ns/op
# Warmup Iteration 4: 160.299 ns/op
# Warmup Iteration 5: 159.486 ns/op
Iteration 1: 157.911 ns/op
Iteration 2: 161.919 ns/op
Iteration 3: 156.989 ns/op
Iteration 4: 160.349 ns/op
Iteration 5: 158.000 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually":
159.034 ±(99.9%) 7.837 ns/op [Average]
(min, avg, max) = (156.989, 159.034, 161.919), stdev = 2.035
CI (99.9%): [151.197, 166.871] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually
# Parameters: (length = 1000)
# Run progress: 25.00% complete, ETA 00:10:04
# Fork: 1 of 1
# Warmup Iteration 1: 1205.523 ns/op
# Warmup Iteration 2: 1155.145 ns/op
# Warmup Iteration 3: 1143.332 ns/op
# Warmup Iteration 4: 1170.217 ns/op
# Warmup Iteration 5: 1150.013 ns/op
Iteration 1: 1148.069 ns/op
Iteration 2: 1155.677 ns/op
Iteration 3: 1177.489 ns/op
Iteration 4: 1139.463 ns/op
Iteration 5: 1150.758 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually":
1154.291 ±(99.9%) 54.833 ns/op [Average]
(min, avg, max) = (1139.463, 1154.291, 1177.489), stdev = 14.240
CI (99.9%): [1099.458, 1209.124] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually
# Parameters: (length = 10000)
# Run progress: 37.50% complete, ETA 00:08:23
# Fork: 1 of 1
# Warmup Iteration 1: 12147.021 ns/op
# Warmup Iteration 2: 12273.438 ns/op
# Warmup Iteration 3: 12212.595 ns/op
# Warmup Iteration 4: 12056.969 ns/op
# Warmup Iteration 5: 12192.355 ns/op
Iteration 1: 12172.471 ns/op
Iteration 2: 12346.400 ns/op
Iteration 3: 12156.589 ns/op
Iteration 4: 12986.955 ns/op
Iteration 5: 11815.254 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsManually":
12295.534 ±(99.9%) 1662.669 ns/op [Average]
(min, avg, max) = (11815.254, 12295.534, 12986.955), stdev = 431.790
CI (99.9%): [10632.865, 13958.202] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark:
org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder
# Parameters: (length = 10)
# Run progress: 50.00% complete, ETA 00:06:43
# Fork: 1 of 1
# Warmup Iteration 1: 59.569 ns/op
# Warmup Iteration 2: 65.313 ns/op
# Warmup Iteration 3: 58.708 ns/op
# Warmup Iteration 4: 57.362 ns/op
# Warmup Iteration 5: 57.799 ns/op
Iteration 1: 58.263 ns/op
Iteration 2: 58.120 ns/op
Iteration 3: 57.970 ns/op
Iteration 4: 57.540 ns/op
Iteration 5: 56.544 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder":
57.687 ±(99.9%) 2.672 ns/op [Average]
(min, avg, max) = (56.544, 57.687, 58.263), stdev = 0.694
CI (99.9%): [55.015, 60.359] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark:
org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder
# Parameters: (length = 100)
# Run progress: 62.50% complete, ETA 00:05:02
# Fork: 1 of 1
# Warmup Iteration 1: 237.017 ns/op
# Warmup Iteration 2: 238.549 ns/op
# Warmup Iteration 3: 233.113 ns/op
# Warmup Iteration 4: 233.338 ns/op
# Warmup Iteration 5: 234.521 ns/op
Iteration 1: 235.478 ns/op
Iteration 2: 232.369 ns/op
Iteration 3: 236.248 ns/op
Iteration 4: 233.509 ns/op
Iteration 5: 234.388 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder":
234.398 ±(99.9%) 5.933 ns/op [Average]
(min, avg, max) = (232.369, 234.398, 236.248), stdev = 1.541
CI (99.9%): [228.465, 240.332] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark:
org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder
# Parameters: (length = 1000)
# Run progress: 75.00% complete, ETA 00:03:21
# Fork: 1 of 1
# Warmup Iteration 1: 2061.928 ns/op
# Warmup Iteration 2: 2017.468 ns/op
# Warmup Iteration 3: 2076.288 ns/op
# Warmup Iteration 4: 2054.085 ns/op
# Warmup Iteration 5: 2049.848 ns/op
Iteration 1: 2042.963 ns/op
Iteration 2: 2034.011 ns/op
Iteration 3: 2034.458 ns/op
Iteration 4: 2036.371 ns/op
Iteration 5: 2036.183 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder":
2036.797 ±(99.9%) 13.859 ns/op [Average]
(min, avg, max) = (2034.011, 2036.797, 2042.963), stdev = 3.599
CI (99.9%): [2022.938, 2050.656] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 22, Java HotSpot(TM) 64-Bit Server VM, 22+36-2370
# VM invoker:
/Library/Java/JavaVirtualMachines/jdk-22.jdk/Contents/Home/bin/java
# VM options: -javaagent:/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar=57527:/Applications/IntelliJ
IDEA.app/Contents/bin -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8
-Dsun.stderr.encoding=UTF-8
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark:
org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder
# Parameters: (length = 10000)
# Run progress: 87.50% complete, ETA 00:01:40
# Fork: 1 of 1
# Warmup Iteration 1: 19436.556 ns/op
# Warmup Iteration 2: 18318.262 ns/op
# Warmup Iteration 3: 18441.518 ns/op
# Warmup Iteration 4: 18065.338 ns/op
# Warmup Iteration 5: 18540.429 ns/op
Iteration 1: 18253.569 ns/op
Iteration 2: 18468.867 ns/op
Iteration 3: 18790.053 ns/op
Iteration 4: 18093.047 ns/op
Iteration 5: 18736.861 ns/op
Result "org.apache.StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder":
18468.480 ±(99.9%) 1159.316 ns/op [Average]
(min, avg, max) = (18093.047, 18468.480, 18790.053), stdev = 301.071
CI (99.9%): [17309.163, 19627.796] (assumes normal distribution)
# Run complete. Total time: 00:13:26
REMEMBER: The numbers below are just data. To gain reusable insights, you
need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof),
design factorial
experiments, perform baseline and negative tests that provide experimental
control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews
from the domain experts.
Do not assume the numbers tell you what you want them to tell.
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are
in use. Please exercise
extra caution when trusting the results, look into the generated code to
check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while
comparisons between
different JVMs are already problematic, the performance difference caused by
different Blackhole
modes can be very significant. Please make sure you use the consistent
Blackhole mode for comparisons.
Benchmark (length) Mode Cnt
Score Error Units
StringUtilsGetDigitsBenchmark.testGetDigitsManually 10 avgt 5
47.924 ± 1.907 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsManually 100 avgt 5
159.034 ± 7.837 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsManually 1000 avgt 5
1154.291 ± 54.833 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsManually 10000 avgt 5
12295.534 ± 1662.669 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder 10 avgt 5
57.687 ± 2.672 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder 100 avgt 5
234.398 ± 5.933 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder 1000 avgt 5
2036.797 ± 13.859 ns/op
StringUtilsGetDigitsBenchmark.testGetDigitsWithBuilder 10000 avgt 5
18468.480 ± 1159.316 ns/op
Process finished with exit code 0
```
@garydgregory Please review. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
