mridulm edited a comment on pull request #32790:
URL: https://github.com/apache/spark/pull/32790#issuecomment-865516802
I am referring to the size of the Map's themselves @attilapiros.
Essentially, sum of sizes of each value `JHashMap` : even though empty, java
hashmap does not release memory - so end up with the same underlying array.
(Edit):
For example:
```
> val map = new java.util.HashMap[Int, Int]()
> for (i <- 0 until 1024 * 1024) map.put(i, i)
> val field = map.getClass.getDeclaredField("table")
> field.setAccessible(true)
> map.size
res1: Int = 1048576
> field.get(map).asInstanceOf[Array[AnyRef]].length
res2: Int = 2097152
> map.clear()
> map.size
res3: Int = 0
> field.get(map).asInstanceOf[Array[AnyRef]].length
res4: Int = 2097152
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
