NarekDW commented on PR #40040:
URL: https://github.com/apache/spark/pull/40040#issuecomment-1445466115
Also, I'd like to share some performance measurements from my local machine,
using JMH:
code example:
```java
...
@Benchmark
public void convert_branch_master(Blackhole bh) {
// Convert to unsigned
for (int i = -10_000; i < 10_000; i++) {
UTF8String convert = NumberConverter
.convert(UTF8String.fromString(String.valueOf(i)).getBytes(), 10, 16);
bh.consume(convert);
}
// Convert to signed
for (int i = -10_000; i < 10_000; i++) {
UTF8String convert = NumberConverter
.convert(UTF8String.fromString(String.valueOf(i)).getBytes(), 10, -16);
bh.consume(convert);
}
}
...
```
With Java 8, current PR even speeds up the performance:
```java
# JMH version: 1.36
# VM version: JDK 1.8.0_362, OpenJDK 64-Bit Server VM, 25.362-b00
# VM invoker:
/usr/local/Cellar/openjdk@8/1.8.0+362/libexec/openjdk.jdk/Contents/Home/jre/bin/java
# Blackhole mode: full + dont-inline hint (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
Benchmark Mode Cnt Score
Error Units
NumberConverterBenchmark.convert_branch_master avgt 10 30.458 ±
1.638 ms/op
NumberConverterBenchmark.convert_branch_spark42399 avgt 10 22.857 ±
0.421 ms/op
```
With Java 11, implementation from master branch works more then 2 times
faster than the same implementation with Java 8!
But there is not a big gap in performance difference between master branch
implementation and implementation from current branch.
```java
# JMH version: 1.36
# VM version: JDK 11.0.16.1, OpenJDK 64-Bit Server VM, 11.0.16.1+0
# VM invoker:
/usr/local/Cellar/openjdk@11/11.0.16.1/libexec/openjdk.jdk/Contents/Home/bin/java
# Blackhole mode: full + dont-inline hint (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
Benchmark Mode Cnt Score
Error Units
NumberConverterBenchmark.convert_branch_master avgt 10 14.453 ±
1.082 ms/op
NumberConverterBenchmark.convert_branch_spark42399 avgt 10 17.956 ±
0.489 ms/op
```
With Java 17, implementation from master branch works more then 3 times
faster than the same implementation with Java 8 and ~ 2 times faster then the
same implementation with Java 11!
And here is a significant difference between master branch implementation
and current branch (master branch is ~2x times faster...).
```java
# JMH version: 1.36
# VM version: JDK 17.0.4.1, OpenJDK 64-Bit Server VM, 17.0.4.1+1
# VM invoker:
/usr/local/Cellar/openjdk@17/17.0.4.1_1/libexec/openjdk.jdk/Contents/Home/bin/java
# Blackhole mode: compiler (auto-detected, use
-Djmh.blackhole.autoDetect=false to disable)
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
Benchmark Mode Cnt Score
Error Units
NumberConverterBenchmark.convert_branch_master avgt 10 8.410 ±
0.161 ms/op
NumberConverterBenchmark.convert_branch_spark42399 avgt 10 18.162 ±
0.036 ms/op
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
