spark git commit: [SPARK-4480] Avoid many small spills in external data structures (1.1)

2014-11-19 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 e22a75923 - 16bf5f3d1 [SPARK-4480] Avoid many small spills in external data structures (1.1) This is the branch-1.1 version of #3353. This requires a separate PR because the code in master has been refactored a little to eliminate

spark git commit: [SPARK-4480] Avoid many small spills in external data structures

2014-11-19 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 f21e550e3 - 4a5c3d21b [SPARK-4480] Avoid many small spills in external data structures **Summary.** Currently, we may spill many small files in `ExternalAppendOnlyMap` and `ExternalSorter`. The underlying root cause of this is

spark git commit: [SPARK-4480] Avoid many small spills in external data structures

2014-11-19 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 73fedf5a6 - 0eb4a7fb0 [SPARK-4480] Avoid many small spills in external data structures **Summary.** Currently, we may spill many small files in `ExternalAppendOnlyMap` and `ExternalSorter`. The underlying root cause of this is summarized