[GitHub] spark pull request #18940: YSPARK-734 Change CacheLoader to limit entries ba...

2017-08-14 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18940#discussion_r133077743 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -104,15 +105,22 @@ public

[GitHub] spark pull request #18940: YSPARK-734 Change CacheLoader to limit entries ba...

2017-08-14 Thread redsanket
GitHub user redsanket opened a pull request: https://github.com/apache/spark/pull/18940 YSPARK-734 Change CacheLoader to limit entries based on memory footprint Right now the spark shuffle service has a cache for index files. It is based on a # of files cached