dongjoon-hyun commented on a change in pull request #35559:
URL: https://github.com/apache/spark/pull/35559#discussion_r814358891
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java
##########
@@ -46,8 +50,9 @@ public ShuffleIndexInformation(File indexFile) throws
IOException {
* Size of the index file
* @return size
*/
- public int getSize() {
- return size;
+ public int getRetainedMemorySize() {
+ // SPARK-33206: here the offsets' capacity is multiplied by 8 as offsets
stores long values.
+ return (offsets.capacity() << 3) + INSTANCE_MEMORY_FOOTPRINT;
Review comment:
If it's not feasible, it would be great if we add the explanation here
explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
