dongjoon-hyun commented on a change in pull request #35559:
URL: https://github.com/apache/spark/pull/35559#discussion_r814358375
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java
##########
@@ -46,8 +50,9 @@ public ShuffleIndexInformation(File indexFile) throws
IOException {
* Size of the index file
* @return size
*/
- public int getSize() {
- return size;
+ public int getRetainedMemorySize() {
+ // SPARK-33206: here the offsets' capacity is multiplied by 8 as offsets
stores long values.
+ return (offsets.capacity() << 3) + INSTANCE_MEMORY_FOOTPRINT;
Review comment:
Is this safe from `overflow`? Since the range of `offsets.capacity()` is
`int` and the return value of `getRetainedMemorySize` is `int`, this formula
may hit overflow issue theoretically. I'm wondering if there is a chance for
this function to return a negative value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
