spark git commit: [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk…

2016-05-27 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 5bdbedf22 -> ce756daa4 [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk… ## What changes were proposed in this pull request? Profiling a Spark job spilling large amount of intermediate data we found that significant

spark git commit: [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk…

2016-05-27 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 89fdb6972 -> 30e87b55b [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk… ## What changes were proposed in this pull request? Profiling a Spark job spilling large amount of intermediate data we found that signifi