[GitHub] spark pull request: Patch for SPARK-942

2014-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/50 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: Patch for SPARK-942

2014-03-06 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36947949 I've created SPARK-1201 (https://spark-project.atlassian.net/browse/SPARK-1201) to cover optimizations in cases other than DISK_ONLY. --- If your project is set up for i

[GitHub] spark pull request: Patch for SPARK-942

2014-03-06 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36947762 Thanks @kellrott for this patch - sorry it took us a long time to review it. I'm going to merge this now. --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: Patch for SPARK-942

2014-03-06 Thread kellrott
Github user kellrott commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36931153 I think I've covered all the formatting requests. Any other issues? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Patch for SPARK-942

2014-03-04 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10245880 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -71,10 +71,21 @@ private[spark] class CacheManager(blockManager: BlockManager) exten

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36598072 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36598073 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12985/ --- If your project i

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36595760 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36595762 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10243269 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -71,10 +71,21 @@ private[spark] class CacheManager(blockManager: BlockManager) extends L

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36582785 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36582786 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12975/ --- If your pr

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36582197 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36582195 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10238895 --- Diff: core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala --- @@ -23,9 +23,27 @@ import java.nio.ByteBuffer import org.apache.spark

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10238901 --- Diff: core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala --- @@ -23,9 +23,27 @@ import java.nio.ByteBuffer import org.apache.spark

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10238870 --- Diff: core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala --- @@ -23,9 +23,27 @@ import java.nio.ByteBuffer import org.apache.spark

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10238815 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -71,10 +71,21 @@ private[spark] class CacheManager(blockManager: BlockManager) exten

[GitHub] spark pull request: Patch for SPARK-942

2014-03-03 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36580259 Alright, sounds good. Looking forward to it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread kellrott
Github user kellrott commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36483912 Thank you for the notes. I'll start working on fixing things. I'd like to keep this patch 'simple', and limit the scope to DISK_ONLY, and get it accepted before thinki

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201307 --- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala --- @@ -59,24 +59,45 @@ private class MemoryStore(blockManager: BlockManager, maxMemory:

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201291 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -52,11 +52,21 @@ private class DiskStore(blockManager: BlockManager, diskManager: D

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201264 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -549,34 +555,43 @@ private[spark] class BlockManager( var marke

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201268 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -549,34 +555,43 @@ private[spark] class BlockManager( var marke

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36483263 Hey Kyle, thanks for bringing this to the new repo. I looked through it and made a few comments. Another concern though is that it would be good to make this work for MEMORY

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201195 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -71,10 +71,21 @@ private[spark] class CacheManager(blockManager: BlockManager) extends L

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201182 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -534,8 +539,9 @@ private[spark] class BlockManager( // If we're s

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201181 --- Diff: core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala --- @@ -23,9 +23,27 @@ import java.nio.ByteBuffer import org.apache.spark.Spa

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201177 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -71,10 +71,21 @@ private[spark] class CacheManager(blockManager: BlockManager) extends L

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201097 --- Diff: core/src/test/scala/org/apache/spark/storage/FlatmapIteratorSuite.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201080 --- Diff: core/src/test/scala/org/apache/spark/storage/FlatmapIteratorSuite.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Patch for SPARK-942

2014-03-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/50#discussion_r10201076 --- Diff: core/src/test/scala/org/apache/spark/storage/FlatmapIteratorSuite.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36410331 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36410332 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12935/ --- If your project i

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36408974 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36408973 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36408986 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread kellrott
GitHub user kellrott opened a pull request: https://github.com/apache/spark/pull/50 Patch for SPARK-942 This is a port of a pull request original targeted at incubator-spark: https://github.com/apache/incubator-spark/pull/180 Essentially if a user returns a generative itera