[
https://issues.apache.org/jira/browse/HDFS-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796824#comment-15796824
]
Gang Xie commented on HDFS-7784:
--------------------------------
About the GC activities, the following is the gstat output. And it do caused
some long-time GC. But comparing it to the one used in full block report, it
looks OK.
jstat -gcutil 10885 5000 1000
S0 S1 E O P YGC YGCT FGC FGCT GCT
0.00 100.00 67.94 89.32 69.63 313 188.870 3 3.130 192.000
0.00 100.00 67.94 89.32 69.63 313 188.870 3 3.130 192.000
0.00 100.00 67.95 89.32 69.63 313 188.870 3 3.130 192.000
0.00 100.00 81.32 89.32 70.61 313 188.870 3 3.130 192.000
100.00 0.00 19.44 89.68 70.62 314 192.495 3 3.130 195.626
0.00 64.43 60.41 90.04 70.62 315 192.938 3 3.130 196.068
56.75 7.26 100.00 90.27 70.62 317 193.167 3 3.130 196.297
2.27 0.00 43.16 90.38 70.62 318 193.653 3 3.130 196.783
0.00 0.68 91.15 90.38 70.62 319 193.729 3 3.130 196.859
0.00 0.05 38.53 90.38 70.62 321 193.875 3 3.130 197.005
0.01 0.00 82.04 90.38 70.62 322 193.951 3 3.130 197.081
0.00 0.00 19.95 90.38 70.62 324 194.084 3 3.130 197.214
0.00 0.00 0.00 90.38 70.62 326 194.235 4 3.130 197.365
0.00 0.00 98.27 90.33 70.62 326 194.235 4 5.240 199.475
0.00 0.00 40.11 90.27 70.62 328 194.372 4 5.240 199.612
0.00 0.00 90.25 90.20 70.62 329 194.449 4 5.240 199.689
0.00 0.00 30.08 90.13 70.62 331 194.605 4 5.240 199.845
0.00 0.00 74.21 90.05 70.62 332 194.676 4 5.240 199.916
0.00 0.00 14.04 89.95 70.62 334 194.819 4 5.240 200.059
0.00 0.00 62.17 89.85 70.62 335 194.894 4 5.240 200.134
0.00 0.00 4.01 89.79 70.62 337 195.042 4 5.240 200.282
0.00 0.00 48.13 89.74 60.00 338 195.116 4 5.240 200.356
0.00 0.00 80.22 89.74 60.00 339 195.192 5 5.241 200.433
0.00 0.00 4.01 89.74 60.00 341 195.349 5 5.241 200.590
0.00 0.00 24.07 89.74 60.00 342 195.423 5 5.241 200.664
0.00 0.00 50.14 89.74 60.00 343 195.498 5 5.241 200.739
0.00 0.00 96.27 89.74 60.00 344 195.571 5 5.241 200.813
0.00 0.00 38.11 89.74 60.00 346 195.708 5 5.241 200.949
0.00 0.00 86.24 89.74 60.00 347 195.785 5 5.241 201.026
Total time for which application threads were stopped: 1.6167710 seconds
Total time for which application threads were stopped: 9.6578530 seconds
Total time for which application threads were stopped: 1.0820690 seconds
Total time for which application threads were stopped: 1.1189530 seconds
Total time for which application threads were stopped: 1.2096840 seconds
Total time for which application threads were stopped: 8.6128080 seconds
Total time for which application threads were stopped: 7.5763860 seconds
Total time for which application threads were stopped: 2.1393520 seconds
Total time for which application threads were stopped: 1.9607400 seconds
Total time for which application threads were stopped: 3.0785030 seconds
Total time for which application threads were stopped: 2.7774960 seconds
Total time for which application threads were stopped: 4.5180250 seconds
Total time for which application threads were stopped: 1.9637590 seconds
Total time for which application threads were stopped: 1.8422970 seconds
Total time for which application threads were stopped: 1.9868880 seconds
Total time for which application threads were stopped: 2.2927440 seconds
Total time for which application threads were stopped: 2.7141160 seconds
Total time for which application threads were stopped: 2.9030460 seconds
Total time for which application threads were stopped: 5.2282350 seconds
Total time for which application threads were stopped: 3.6261510 seconds
Total time for which application threads were stopped: 2.1100760 seconds
> load fsimage in parallel
> ------------------------
>
> Key: HDFS-7784
> URL: https://issues.apache.org/jira/browse/HDFS-7784
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Walter Su
> Assignee: Walter Su
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HDFS-7784.001.patch, test-20150213.pdf
>
>
> When single Namenode has huge amount of files, without using federation, the
> startup/restart speed is slow. The fsimage loading step takes the most of the
> time. fsimage loading can seperate to two parts, deserialization and object
> construction(mostly map insertion). Deserialization takes the most of CPU
> time. So we can do deserialization in parallel, and add to hashmap in serial.
> It will significantly reduce the NN start time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
