LuciferYang commented on PR #36484:
URL: https://github.com/apache/spark/pull/36484#issuecomment-1123327781
change `COUNT` to 102400 and add `-XX:+PrintGC -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps` to observe GC behavior:
**Before**
```
OpenJDK 64-Bit Server VM (25.312-b07) for linux-amd64 JRE (Zulu
8.58.0.13-CA-linux64) (1.8.0_312-b07), built on Oct 9 2021 03:04:03 by
"zulu_re" with gcc 4.4.7 20120313 (Red Hat 4.4.7-23)
Memory: 4k page, physical 527627268k(39637496k free), swap 0k(0k free)
CommandLine flags: -XX:+IgnoreUnrecognizedVMOptions
-XX:InitialHeapSize=2147483648 -XX:MaxHeapSize=4294967296
-XX:MaxMetaspaceSize=2147483648 -XX:+PrintGC -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=134217728
-XX:ThreadStackSize=4096 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops
-XX:+UseParallelGC
4.229: [GC (Allocation Failure) [PSYoungGen: 524800K->25093K(611840K)]
524800K->25109K(2010112K), 0.1434697 secs] [Times: user=5.47 sys=0.23,
real=0.15 secs]
6.878: [GC (Allocation Failure) [PSYoungGen: 549893K->30226K(1136640K)]
549909K->30250K(2534912K), 0.1407982 secs] [Times: user=4.57 sys=0.03,
real=0.14 secs]
10.916: [GC (Allocation Failure) [PSYoungGen: 1079826K->56841K(1136640K)]
1079850K->56873K(2534912K), 0.5514541 secs] [Times: user=22.90 sys=0.54,
real=0.55 secs]
14.788: [GC (Allocation Failure) [PSYoungGen: 1106441K->39200K(1310720K)]
1106473K->39240K(2708992K), 0.4692478 secs] [Times: user=22.50 sys=0.28,
real=0.47 secs]
20.524: [GC (Allocation Failure) [PSYoungGen: 1262880K->51227K(1275392K)]
1262920K->51275K(2673664K), 0.7022824 secs] [Times: user=28.76 sys=0.38,
real=0.70 secs]
25.734: [GC (Allocation Failure) [PSYoungGen: 1274907K->56794K(1279488K)]
1274955K->56850K(2677760K), 0.3269271 secs] [Times: user=10.53 sys=0.05,
real=0.33 secs]
31.718: [GC (Allocation Failure) [PSYoungGen: 1274842K->56160K(1285120K)]
1274898K->58980K(2683392K), 0.3651417 secs] [Times: user=12.49 sys=0.11,
real=0.37 secs]
Heap
PSYoungGen total 1285120K, used 518037K [0x000000076ab00000,
0x00000007c0000000, 0x00000007c0000000)
eden space 1218048K, 37% used
[0x000000076ab00000,0x0000000786e0d198,0x00000007b5080000)
from space 67072K, 83% used
[0x00000007b5080000,0x00000007b8758368,0x00000007b9200000)
to space 70144K, 0% used
[0x00000007bbb80000,0x00000007bbb80000,0x00000007c0000000)
ParOldGen total 1398272K, used 2819K [0x00000006c0000000,
0x0000000715580000, 0x000000076ab00000)
object space 1398272K, 0% used
[0x00000006c0000000,0x00000006c02c0fc0,0x0000000715580000)
Metaspace used 19352K, capacity 19716K, committed 19840K, reserved
1067008K
class space used 2325K, capacity 2422K, committed 2432K, reserved
1048576K
```
**After**
```
OpenJDK 64-Bit Server VM (25.312-b07) for linux-amd64 JRE (Zulu
8.58.0.13-CA-linux64) (1.8.0_312-b07), built on Oct 9 2021 03:04:03 by
"zulu_re" with gcc 4.4.7 20120313 (Red Hat 4.4.7-23)
Memory: 4k page, physical 527627268k(39516972k free), swap 0k(0k free)
CommandLine flags: -XX:+IgnoreUnrecognizedVMOptions
-XX:InitialHeapSize=2147483648 -XX:MaxHeapSize=4294967296
-XX:MaxMetaspaceSize=2147483648 -XX:+PrintGC -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=134217728
-XX:ThreadStackSize=4096 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops
-XX:+UseParallelGC
4.080: [GC (Allocation Failure) [PSYoungGen: 524800K->24653K(611840K)]
524800K->24669K(2010112K), 0.2088060 secs] [Times: user=8.93 sys=0.14,
real=0.21 secs]
6.767: [GC (Allocation Failure) [PSYoungGen: 549453K->29881K(1136640K)]
549469K->29905K(2534912K), 0.1403327 secs] [Times: user=4.42 sys=0.02,
real=0.14 secs]
10.577: [GC (Allocation Failure) [PSYoungGen: 1079481K->55196K(1136640K)]
1079505K->55228K(2534912K), 0.3690663 secs] [Times: user=12.57 sys=0.04,
real=0.37 secs]
14.225: [GC (Allocation Failure) [PSYoungGen: 1104796K->40043K(1310720K)]
1104828K->40083K(2708992K), 0.4300225 secs] [Times: user=20.10 sys=0.25,
real=0.43 secs]
20.011: [GC (Allocation Failure) [PSYoungGen: 1263723K->51450K(1275392K)]
1263763K->51498K(2673664K), 0.3348333 secs] [Times: user=11.46 sys=0.09,
real=0.34 secs]
25.481: [GC (Allocation Failure) [PSYoungGen: 1275130K->56619K(1278976K)]
1275178K->56675K(2677248K), 0.4491326 secs] [Times: user=16.92 sys=0.12,
real=0.45 secs]
31.391: [GC (Allocation Failure) [PSYoungGen: 1274155K->56192K(1285120K)]
1274211K->58965K(2683392K), 0.4748197 secs] [Times: user=19.71 sys=0.14,
real=0.48 secs]
Heap
PSYoungGen total 1285120K, used 505049K [0x000000076ab00000,
0x00000007c0000000, 0x00000007c0000000)
eden space 1217536K, 36% used
[0x000000076ab00000,0x00000007861563e8,0x00000007b5000000)
from space 67584K, 83% used
[0x00000007b5000000,0x00000007b86e0368,0x00000007b9200000)
to space 70656K, 0% used
[0x00000007bbb00000,0x00000007bbb00000,0x00000007c0000000)
ParOldGen total 1398272K, used 2772K [0x00000006c0000000,
0x0000000715580000, 0x000000076ab00000)
object space 1398272K, 0% used
[0x00000006c0000000,0x00000006c02b53c0,0x0000000715580000)
Metaspace used 19373K, capacity 19786K, committed 19840K, reserved
1067008K
class space used 2337K, capacity 2423K, committed 2432K, reserved
1048576K
```
Before `2.71s (0.15 + 0.14 + 0.55 + 0.47 + 0.70 + 0.33 + 0.37)` , after
`2.42s (0.21 + 0.14 + 0.37 + 0.43 + 0.34 + 0.45 + 0.48)`, no significant
change in GC behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
