[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8831 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141748157 Thanks - I'm going to merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141723442 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141723775 [Test build #1777 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1777/consoleFull) for PR 8831 at commit

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141741093 [Test build #1777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1777/console) for PR 8831 at commit

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141635017 LGTM - only comment is maybe we should warn in SparkConf for the core settings. But I'm ok with merging this as is (provided that tests pass either on Jenkins or locally).

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8831#discussion_r39917683 --- Diff: core/src/main/scala/org/apache/spark/shuffle/hash/HashShuffleManager.scala --- @@ -24,7 +24,13 @@ import org.apache.spark.shuffle._ * A

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141601051 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141601028 [Test build #42698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42698/console) for PR 8831 at commit

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141601050 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/8831#discussion_r39907246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/sort.scala --- @@ -31,38 +31,12 @@ import org.apache.spark.{SparkEnv, InternalAccumulator,

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141584039 [Test build #42698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42698/consoleFull) for PR 8831 at commit

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/8831 [SPARK-10710] Remove ability to disable spilling in core and SQL It does not make much sense to set `spark.shuffle.spill` or `spark.sql.planner.externalSort` to false: I believe that these

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141583145 /cc @rxin, @marmbrus, and @davies for review and sign-off. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141583377 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10710] Remove ability to disable spilli...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8831#issuecomment-141583355 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not