xichen01 commented on PR #4572:
URL: https://github.com/apache/ozone/pull/4572#issuecomment-1537847736
> Sorry, the `IntSparseMap` does not look like a good solution to the O(n^2)
problem. It makes the code complicated and does not help much. Also, we will
need to remove it when we reduce O(n^2) to O(n) later on.
In my tests, this does reduce memory usage.
I found that it is not the `PartKeyInfo` that consumes the most memory, but
the `Map.entry` where the PartKeyInfo is stored.
```shell
[root@centos /root/]% jmap -histo:live `jps | grep OzoneManagerStarter | awk
'{ print $1 }'` | head -n 20
num #instances #bytes class name
----------------------------------------------
1: 46497817 1859912680 java.util.TreeMap$Entry
//...
```
`PartkeyInfo` is a shallow copy, so it doesn't consume a lot of memory (So
even if the size of `ParKeyinfo` is reduced, it is difficult to reduce the
memory consumption). The `Map.entry` itself consumes a lot of memory.
```java
static final class Entry<K,V> implements Map.Entry<K,V> {
K key;
V value;
Entry<K,V> left;
Entry<K,V> right;
Entry<K,V> parent;
boolean color = BLACK;
//...
}
```
The idea of `IntSparseMap` is to store `ParkeyInfo` using a data structure
that consumes very little memory itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
