[
https://issues.apache.org/jira/browse/HDDS-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712615#comment-17712615
]
ChenXi edited comment on HDDS-8427 at 4/15/23 8:42 AM:
-------------------------------------------------------
There is an `OmMultipartKeyInfo` field (TreeMap type) which will hold the
partKeyInfo of all previous partkeys and itself partKeyInfo,
For example, the 10000th key will save 9999 (previous) + 1 (itself) in the
`OmMultipartKeyInfo` this will be cost a lot of memory
!screenshot-1.png!
If we have 10000 If we have 10,000 parts when multiupload, there will create (1
+ 10000) * 10000 / 2 = 50005000 TreeMap.Entry
Test of multiupload uploading 10000 "part key", From the results, ParOldGen
will increase about 2 GB, and most of them are TreeMap$Entry
{code:bash}
[root@centos /root/ozone]% jcmd `jps | grep OzoneManagerStarter | awk '{ print
$1 }'` GC.heap_info
33896:
PSYoungGen total 691712K, used 448768K [0x000000076ab00000,
0x000000079df80000, 0x00000007c0000000)
eden space 566784K, 78% used
[0x000000076ab00000,0x0000000785faef78,0x000000078d480000)
from space 124928K, 1% used
[0x0000000796580000,0x0000000796711278,0x000000079df80000)
to space 136704K, 0% used
[0x000000078d480000,0x000000078d480000,0x0000000795a00000)
ParOldGen total 2796544K, used 2115856K [0x00000006c0000000,
0x000000076ab00000, 0x000000076ab00000)
object space 2796544K, 75% used
[0x00000006c0000000,0x0000000741244250,0x000000076ab00000)
Metaspace used 57872K, capacity 60290K, committed 60800K, reserved
1103872K
class space used 6552K, capacity 6955K, committed 7040K, reserved 1048576K
[pony.chen@C02GKEF9MD6M /Users/pony.chen/project/ozone]%
[root@centos /root/ozone]% jmap -histo:live `jps | grep OzoneManagerStarter |
awk '\{ print $1 }
'` | head -n 20
num #instances #bytes class name
----------------------------------------------
1: 46497817 1859912680 java.util.TreeMap$Entry
1: 13112075 419586400 java.util.HashMap$Node
2: 558238 78747928 [B
3: 444650 34326104 [C
4: 268746 14390832 [Ljava.lang.Object;
5: 260101 12484848 org.apache.hadoop.hdds.protocol.proto.HddsProtos$Port
6: 486330 11671920 com.google.protobuf.LiteralByteString
7: 443954 10654896 java.lang.String
8: 330385 10572320 org.apache.hadoop.ozone.OzoneAcl
9: 330434 7932848 [J
10: 330388 7929312 java.util.BitSet
11: 294524 7068576 java.util.ArrayList
12: 104041 5826296 org.apache.hadoop.hdds.protocol.proto.HddsProtos$UUID
13: 22023 5814072
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMResponse
14: 72411 5639624 [Ljava.util.HashMap$Node;
15: 220320 5287680 org.apache.hadoop.hdds.protocol.DatanodeDetails$Port
16: 101153 4855344 java.util.HashMap
17: 148143 4740576 java.util.UUID
{code}
was (Author: JIRAUSER294158):
There is an `OmMultipartKeyInfo` field (TreeMap type) which will hold the
partKeyInfo of all previous partkeys and itself partKeyInfo,
For example, the 10000th key will save 9999 (previous) + 1 (itself) in the
`OmMultipartKeyInfo` this will be cost a lot of memory
!screenshot-1.png!
If we have 10000 If we have 10,000 parts when multiupload, there will create (1
+ 10000) * 10000 / 2 = 50005000 TreeMap.Entry
Test of multiupload uploading 10000 "part key", From the results, ParOldGen
will increase about 2 GB, and most of them are TreeMap$Entry
{code:bash}
[root@centos /root/ozone]% jcmd `jps | grep OzoneManagerStarter | awk '{ print
$1 }'` GC.heap_info
33896:
PSYoungGen total 691712K, used 448768K [0x000000076ab00000,
0x000000079df80000, 0x00000007c0000000)
eden space 566784K, 78% used
[0x000000076ab00000,0x0000000785faef78,0x000000078d480000)
from space 124928K, 1% used
[0x0000000796580000,0x0000000796711278,0x000000079df80000)
to space 136704K, 0% used
[0x000000078d480000,0x000000078d480000,0x0000000795a00000)
ParOldGen total 2796544K, used 2115856K [0x00000006c0000000,
0x000000076ab00000, 0x000000076ab00000)
object space 2796544K, 75% used
[0x00000006c0000000,0x0000000741244250,0x000000076ab00000)
Metaspace used 57872K, capacity 60290K, committed 60800K, reserved
1103872K
class space used 6552K, capacity 6955K, committed 7040K, reserved 1048576K
[pony.chen@C02GKEF9MD6M /Users/pony.chen/project/ozone]%
[root@centos /root/ozone]% jmap -histo:live `jps | grep OzoneManagerStarter |
awk '\{ print $1 }
'` | head -n 20
num #instances #bytes class name
----------------------------------------------
1: 46497817 1859912680 java.util.TreeMap$Entry
1: 13112075 419586400 java.util.HashMap$Node
2: 558238 78747928 [B
3: 444650 34326104 [C
4: 268746 14390832 [Ljava.lang.Object;
5: 260101 12484848 org.apache.hadoop.hdds.protocol.proto.HddsProtos$Port
6: 486330 11671920 com.google.protobuf.LiteralByteString
7: 443954 10654896 java.lang.String
8: 330385 10572320 org.apache.hadoop.ozone.OzoneAcl
9: 330434 7932848 [J
10: 330388 7929312 java.util.BitSet
11: 294524 7068576 java.util.ArrayList
12: 104041 5826296 org.apache.hadoop.hdds.protocol.proto.HddsProtos$UUID
13: 22023 5814072
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMResponse
14: 72411 5639624 [Ljava.util.HashMap$Node;
15: 220320 5287680 org.apache.hadoop.hdds.protocol.DatanodeDetails$Port
16: 101153 4855344 java.util.HashMap
17: 148143 4740576 java.util.UUID
{code}
> Optimize multipartUpload memory usage
> -------------------------------------
>
> Key: HDDS-8427
> URL: https://issues.apache.org/jira/browse/HDDS-8427
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: ChenXi
> Assignee: ChenXi
> Priority: Major
> Attachments: screenshot-1.png
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
