[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53727924 thanks @ankurdave . I'd also like to recommend @srowen as a reviewer since he made a few contributions to the GraphGenerator in the past. I've made the

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2168#discussion_r16842842 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/util/GraphGenerators.scala --- @@ -48,33 +48,36 @@ object GraphGenerators { * Ilan Horn, Naty

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53729304 Thanks, @srowen ! I forgot to update the docs as I was making changes. :( Fixed now! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53729660 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19401/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53730337 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19402/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53730908 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19403/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53730920 QA tests have started for PR 2137. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19406/consoleFull ---

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53730896 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19404/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53730898 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19405/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53731054 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19403/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53731069 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19404/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53731050 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19405/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-2435] Add shutdown hook to pyspark

2014-08-28 Thread mattf
GitHub user mattf opened a pull request: https://github.com/apache/spark/pull/2183 [SPARK-2435] Add shutdown hook to pyspark You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattf/spark SPARK-2435 Alternatively you can review

[GitHub] spark pull request: [SPARK-3187] [yarn] Cleanup allocator code.

2014-08-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2169#issuecomment-53745219 @tgravescs I think they're complementary. This change merges a lot of the logic between the alpha and stable allocators. @sryza's change cleans up the Yarn API usage in

[GitHub] spark pull request: [SPARK-3272][MLLib]Calculate prediction for no...

2014-08-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2180#issuecomment-53746339 @chouqin Thanks for observing that we can sometimes avoid calculating the prediction and/or the info gain. I'm worried that this won't really change the scaling of

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53746828 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19408/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-2435] Add shutdown hook to pyspark

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2183#issuecomment-53746821 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19407/consoleFull) for PR 2183 at commit

[GitHub] spark pull request: [SPARK-2435] Add shutdown hook to pyspark

2014-08-28 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/2183#issuecomment-53752758 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53753067 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1750#issuecomment-53753420 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19409/consoleFull) for PR 1750 at commit

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53753489 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1750#issuecomment-53753579 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19409/consoleFull) for PR 1750 at commit

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread willb
Github user willb commented on the pull request: https://github.com/apache/spark/pull/1750#issuecomment-53753423 @marmbrus I've rebased this atop the current master and added support for casting string-valued SQRT arguments to double and a test case in SQLQuerySuite for the same.

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53754354 QA tests have started for PR 2137. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19410/consoleFull ---

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53754520 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-3265 Allow using custom ipython executab...

2014-08-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2167#issuecomment-53755162 Yeah, that's a good idea. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53754981 QA tests have started for PR 2137. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19411/consoleFull ---

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1750#issuecomment-53757176 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19412/consoleFull) for PR 1750 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53757353 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19408/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-2435] Add shutdown hook to pyspark

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2183#issuecomment-53758559 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19407/consoleFull) for PR 2183 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53760885 The code changes look good to me (and were badly need). Thanks for fixing it! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53761843 I fixed comparison of Edge objects. I hadn't been using testOnly properly before so I didn't catch this. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53761979 Thanks, @jegonzal ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3150] Fix NullPointerException in in Sp...

2014-08-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2062#issuecomment-53762494 This looks good to me, too. I'm merging this into `master`, `branch-1.1`, and `branch-1.0`. Thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53762525 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19413/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3150] Fix NullPointerException in in Sp...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2062 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53763573 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread byF
Github user byF commented on the pull request: https://github.com/apache/spark/pull/2084#issuecomment-53763658 @marmbrus so the test fails in jenkins build, however it passes okay on my machine (in Intellij). Any idea what's the reason for that? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53764429 @mattf good point. Will get some data posted to the JIRA ticket. It's mainly a list of Spark SQL, MLlib, and spark-perf tests. --- If your project is set up for it, you

[GitHub] spark pull request: Use user defined $SPARK_HOME in spark-submit i...

2014-08-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1969#issuecomment-53764382 Hi @iven, `spark-shell` actually goes through `spark-submit`. As @liancheng mentioned, you can set `spark.home` to control the executor side Spark location. This is

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53764594 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19414/consoleFull) for PR 2171 at commit

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53764890 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53763548 Build failure was caused by PySpark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53768026 QA results for PR 2137:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-3264] Allow users to set executor Spark...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2166 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2608][Core] Fixed command line option p...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2161 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2608][Core] Fixed command line option p...

2014-08-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2161#issuecomment-53768783 Ok, @rxin merged it. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53769129 QA results for PR 2137:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2155#discussion_r16857315 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/package.scala --- @@ -37,4 +37,15 @@ package object trees extends Logging {

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53772862 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19413/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1750#issuecomment-53773223 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19412/consoleFull) for PR 1750 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53774037 These test failures are occurring in PySpark and seem unrelated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Add role and checkpoint support for Mesos back...

2014-08-28 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/60#discussion_r16859321 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -70,8 +71,16 @@ private[spark] class

[GitHub] spark pull request: SPARK-3069 [DOCS] Build instructions in README...

2014-08-28 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2014#issuecomment-53775856 Is the net conclusion that README.md should use Maven if anything? Not sure. It sounds like Maven is indeed in the official standard for building Spark, but we

[GitHub] spark pull request: Improved GraphX PageRank Test Coverage

2014-08-28 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/1228#issuecomment-53776044 Yes. This is an extension of the unit tests to catch a class of bugs addressed in PR #1217 (which has not been merged). I believe @ankurdave was working on a merge of

[GitHub] spark pull request: Add role and checkpoint support for Mesos back...

2014-08-28 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/60#discussion_r16860333 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -70,8 +71,16 @@ private[spark] class

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-53777614 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19416/consoleFull) for PR 1217 at commit

[GitHub] spark pull request: Introducing an Improved Pregel API

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1217#issuecomment-5394 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19416/consoleFull) for PR 1217 at commit

[GitHub] spark pull request: Add role and checkpoint support for Mesos back...

2014-08-28 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/60#issuecomment-53777908 I think you need to rebase on master, the mesos version you specified (0.16) is way out of date, and spark already is updated to 0.18.1. --- If your project is set up for

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53778867 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19414/consoleFull) for PR 2171 at commit

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53781358 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19417/consoleFull) for PR 2181 at commit

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2142#discussion_r16861865 --- Diff: python/pyspark/rdd.py --- @@ -1993,11 +1993,38 @@ def meanApprox(self, timeout, confidence=0.95): (rdd.meanApprox(1000) - r) / r 0.05

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53782111 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19418/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-3139] Akka timeouts from ContextCleaner...

2014-08-28 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2056#issuecomment-53782725 @andrewor14 and I looked into and increasing timeout does solve SPARK-3015. Well the real reason is still unknown. All we know is that once in a while, when the executor

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53782951 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53783057 LGTM. sum is also much faster than fold. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3094] [PySpark] compatitable with PyPy

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2144#issuecomment-53783568 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19419/consoleFull) for PR 2144 at commit

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53783575 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19420/consoleFull) for PR 2182 at commit

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53783574 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19421/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53785845 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19422/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53790095 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19418/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/2179#issuecomment-53790150 Good catch, wonder why we did it the long way before. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16864683 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala --- @@ -65,36 +66,39 @@ class DecisionTree (private val strategy: Strategy)

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16864832 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53790794 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19417/consoleFull) for PR 2181 at commit

[GitHub] spark pull request: [SPARK-2435] Add shutdown hook to pyspark

2014-08-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2183#issuecomment-53791070 What's the problem without this patch? I remember that the JVM will shutdown itself after shell exited. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16865085 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16865423 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16865586 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16865673 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53792618 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19421/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16865813 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16866042 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impl/DecisionTreeMetadata.scala --- @@ -70,32 +83,48 @@ private[tree] object

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53793408 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19420/consoleFull) for PR 2182 at commit

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53793682 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19422/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53794570 This looks good to me. I think it makes sense to perform this change here in its own PR rather than as part of the subsequent BlockTransferService patch, since that

[GitHub] spark pull request: [SPARK-2871] [PySpark] add countApproxDistinct...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2142#issuecomment-53795341 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19423/consoleFull) for PR 2142 at commit

[GitHub] spark pull request: [SPARK-3273]The spark version in the welcome m...

2014-08-28 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/2175#issuecomment-53795465 Do we need to make any matching changes to the [PySpark shell](https://github.com/apache/spark/blob/master/python/pyspark/shell.py) as part of this PR? As an

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53795847 Build failure caused by unrelated streaming test suites again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53795864 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16866883 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala --- @@ -1395,96 +1027,24 @@ object DecisionTree extends Serializable with

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16867125 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala --- @@ -65,36 +66,39 @@ class DecisionTree (private val strategy: Strategy)

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53796788 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19424/consoleFull) for PR 2171 at commit

[GitHub] spark pull request: [SPARK-3216] [SPARK-3232] Spark-shell is broke...

2014-08-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2136#issuecomment-53797942 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3086] [SPARK-3043] [SPARK-3156] [mllib]...

2014-08-28 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/2125#discussion_r16867878 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/DecisionTree.scala --- @@ -619,661 +662,258 @@ object DecisionTree extends Serializable with

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53799386 Thanks. Merging this. I will submit following PRs to standardize the API and add Netty back. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53799453 Thanks. Merging this in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2182 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2181 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [Spark QA] only check code files for new class...

2014-08-28 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/2184 [Spark QA] only check code files for new classes Look only at code files (`.py`, `.java`, and `.scala`) for new classes. Should get rid of false alarms like [the one reported

<    1   2   3   4   >