attilapiros commented on pull request #35559: URL: https://github.com/apache/spark/pull/35559#issuecomment-1048123958
I have extended the description: > But we can go further and get rid of java.io.File completely and store the ShuffleIndexInformation for the file path. This way not only the cache size estimate is improved but the its size is decreased as well. Here the path size is not counted into the cache size as that string is interned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
