LuciferYang commented on PR #37339:
URL: https://github.com/apache/spark/pull/37339#issuecomment-1199053334
The result of `IteratorSizeBenchmark ` using Scala 2.13 as follows,
`Utils.getIteratorSize` method slightly slower than `Iterator.size`
```
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Range iterator size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 0 1
0 259.9 3.8 1.0X
Use Utils.getIteratorSize 1 2
0 72.8 13.7 0.3X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Range iterator size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 2 2
0 55.4 18.0 1.0X
Use Utils.getIteratorSize 9 9
0 11.6 86.5 0.2X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Range iterator size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 2 2
0 54.4 18.4 1.0X
Use Utils.getIteratorSize 83 83
0 1.2 827.0 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Range iterator size 10000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 2 2
0 54.0 18.5 1.0X
Use Utils.getIteratorSize 818 820
2 0.1 8181.3 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Range iterator size 30000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 2 2
0 57.6 17.4 1.0X
Use Utils.getIteratorSize 2437 2448
16 0.0 24367.8 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Seq iterator size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 3 4
0 30.0 33.4 1.0X
Use Utils.getIteratorSize 9 10
1 10.8 92.2 0.4X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Seq iterator size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 3 4
0 28.8 34.7 1.0X
Use Utils.getIteratorSize 80 82
1 1.3 798.8 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Seq iterator size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 4 4
1 25.1 39.8 1.0X
Use Utils.getIteratorSize 751 753
2 0.1 7513.1 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Seq iterator size 10000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 4 4
0 25.3 39.5 1.0X
Use Utils.getIteratorSize 7659 7676
24 0.0 76589.3 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Seq iterator size 30000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 4 4
0 24.8 40.4 1.0X
Use Utils.getIteratorSize 23008 23025
24 0.0 230080.0 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Array iterator size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 1 1
0 166.5 6.0 1.0X
Use Utils.getIteratorSize 6 7
0 15.5 64.7 0.1X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Array iterator size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 0 0
0 315.6 3.2 1.0X
Use Utils.getIteratorSize 56 56
0 1.8 559.3 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Array iterator size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 0 0
0 309.1 3.2 1.0X
Use Utils.getIteratorSize 613 617
2 0.2 6134.1 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Array iterator size 10000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 0 0
0 306.4 3.3 1.0X
Use Utils.getIteratorSize 6220 6237
25 0.0 62196.7 0.0X
OpenJDK 64-Bit Server VM 1.8.0_342-b07 on Linux 5.15.0-1014-azure
Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Test Array iterator size 30000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Use Iterator.size 0 0
0 316.8 3.2 1.0X
Use Utils.getIteratorSize 18712 18739
39 0.0 187118.4 0.0X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
