[GitHub] spark pull request: [SPARK-3273]The spark version in the welcome m...

2014-08-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2175#issuecomment-53676807 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [HOTFIX] Wait for EOF only for the PySpark she...

2014-08-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2170#issuecomment-53677017 Okay thanks - I'll merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread lirui-intel
GitHub user lirui-intel opened a pull request: https://github.com/apache/spark/pull/2176 SPARK-2636: no where to get job identifier while submit spark job through spark API This PR adds the async actions to the Java API. User can call these async actions to get the FutureAction

[GitHub] spark pull request: [HOTFIX] Wait for EOF only for the PySpark she...

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [HOTFIX][SQL] Remove cleaning of UDFs

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2174 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2947] DAGScheduler resubmit the stage i...

2014-08-28 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1877#issuecomment-53677259 [SPARK-3224](https://issues.apache.org/jira/browse/SPARK-3224) is the same problem. This PR adds some boundary judgments and removed some redundant code --- If your

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53677401 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19377/consoleFull) for PR 2176 at commit

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53677618 @lirui-intel - JobWaiter is an internal API that's never designed to be public. I would not expose it simply because you need the job id. There are lots of ways to get the

[GitHub] spark pull request: [SPARK-3273]The spark version in the welcome m...

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2175#discussion_r16822707 --- Diff: repl/src/main/scala/org/apache/spark/repl/SparkILoopInit.scala --- @@ -26,7 +26,7 @@ trait SparkILoopInit { __

[GitHub] spark pull request: [SPARK-3273]The spark version in the welcome m...

2014-08-28 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/2175#discussion_r16822912 --- Diff: repl/src/main/scala/org/apache/spark/repl/SparkILoopInit.scala --- @@ -26,7 +26,7 @@ trait SparkILoopInit { __

[GitHub] spark pull request: [SPARK-2947] DAGScheduler resubmit the stage i...

2014-08-28 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/1877#discussion_r16823367 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -472,6 +472,44 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53679584 Hi @rxin, thanks for the review! I can add interface to SimpleFutureAction to get the job id if we shouldn't expose JobWaiter to users. Hive on spark currently

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53679729 If foreachAsync is the only one you need right now, why don't you just add foreachAsync (and remove the rest), and add jobId to SimpleFutureAction? --- If your project is

[GitHub] spark pull request: [SPARK-2947] DAGScheduler resubmit the stage i...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1877#issuecomment-53679879 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19375/consoleFull) for PR 1877 at commit

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53680289 I thought these async actions are missing in the java API so I added all of them from AsyncRDDActions. But sure, let me just add foreachAsync. --- If your project

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53680380 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19377/consoleFull) for PR 2176 at commit

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53680555 Yea let's not add all of them since they are highly experimental. I'm not even sure if those are the APIs we want to commit to in the long run. --- If your project is set

[GitHub] spark pull request: [SQL] Fixed 2 comment typos in SQLConf

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2172 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3230][SQL] Fix udfs that return structs

2014-08-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2133 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3181][MLLIB]: Add Robust Regression Alg...

2014-08-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2096#issuecomment-53682676 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2636: no where to get job identifier whi...

2014-08-28 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/2176#issuecomment-53682639 @rxin I've updated the patch. Yes I see these APIs are experimental. We can make hive use it as a workaround and change it when we have a better solution. ---

[GitHub] spark pull request: [SPARK-3181][MLLIB]: Add Robust Regression Alg...

2014-08-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2096#issuecomment-53682689 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3181][MLLIB]: Add Robust Regression Alg...

2014-08-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2096#issuecomment-53682783 @fjiang6 Could you try LBFGS instead of SGD? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53682909 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Added Sql to mima checks

2014-08-28 Thread ScrapCodes
Github user ScrapCodes closed the pull request at: https://github.com/apache/spark/pull/1342 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53683388 QA tests have started for PR 2137. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19382/consoleFull ---

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/2177 [SPARK-3279] Remove useless field variable in ApplicationMaster You can merge this pull request into a Git repository by running: $ git pull https://github.com/sarutak/spark SPARK-3279

[GitHub] spark pull request: [SPARK-1477]: Add the lifecycle interface

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/991#issuecomment-53683774 @witgo I'm going to take a look at this later for 1.2. I think it's a good idea to have a Service abstraction for service that we can start/stop. The current API is slightly

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53684027 The assumption is usually unrealistic. For logistic regression, it is common to have the predictions be something like 0.9 or 0.01, and they cannot be interpreted

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53684178 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19383/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/2178 [SPARK-3280] Made sort-based shuffle the default implementation Sort-based shuffle has lower memory usage and seems to outperform hash-based in almost all of our testing. You can merge this pull

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53684899 Hopefully I caught all the cases --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-28 Thread colorant
Github user colorant commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16825748 --- Diff: core/src/main/scala/org/apache/spark/shuffle/FileShuffleBlockManager.scala --- @@ -181,17 +171,30 @@ class ShuffleBlockManager(blockManager:

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/2179 [SPARK-1912] Lazily initialize buffers for local shuffle blocks. This is a simplified fix for SPARK-1912. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2179#issuecomment-53685306 Note that this was previously fixed by @cloud-fan in #860. cc @cloud-fan @ash211 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [WIP][SPARK-2816][SQL] Type-safe SQL Queries

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1759#issuecomment-53685952 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19387/consoleFull) for PR 1759 at commit

[GitHub] spark pull request: [WIP][SPARK-2816][SQL] Type-safe SQL Queries

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1759#issuecomment-53685965 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19387/consoleFull) for PR 1759 at commit

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-28 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/1241#issuecomment-53686699 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Dt predict

2014-08-28 Thread chouqin
GitHub user chouqin opened a pull request: https://github.com/apache/spark/pull/2180 Dt predict In current implementation, prediction for a node is calculated along with calculation of information gain stats for each possible splits. The value to predict for a specific node is

[GitHub] spark pull request: Dt predict

2014-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2180#issuecomment-53687434 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/2181 [SPARK-3281] Remove Netty specific code in BlockManager. Netty functionality will be added back in subsequent PRs by using the BlockTransferService interface. You can merge this pull request into a

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53688268 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19383/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: Dt predict

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2180#discussion_r16827232 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -47,9 +47,9 @@ class DecisionTreeSuite extends FunSuite with

[GitHub] spark pull request: Dt predict

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2180#discussion_r16827247 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -885,7 +887,7 @@ object DecisionTreeSuite { }

[GitHub] spark pull request: Dt predict

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2180#issuecomment-53688602 I can not say anything about the usefulness of the patch. But we follow the spark style guide across our code base.

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53688701 QA results for PR 2137:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2155#issuecomment-53688875 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19390/consoleFull) for PR 2155 at commit

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53688872 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19389/consoleFull) for PR 2181 at commit

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-28 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2155#issuecomment-53688906 Thank you @marmbrus , you're right. I've updated the code by providing a new class called `TreeNodeRef` which is a wrapper simply re-implement the `equals` and

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-28 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-53689679 @marmbrus @yhuai Can you review this for me? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-3265 Allow using custom ipython executab...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2167#issuecomment-53689831 Shouldn't we update the documentation to include this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: Use user defined $SPARK_HOME in spark-submit i...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1969#issuecomment-53689928 Actually you can just set `spark.home` in `spark-defaults.conf` for this use case. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-2813: [SQL] Implement SQRT() directly in...

2014-08-28 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1750#discussion_r16827967 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -935,6 +936,7 @@ private[hive] object HiveQl { case Token(DIV(),

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2179#issuecomment-53690190 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53690186 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53690286 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2179#issuecomment-53690416 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19391/consoleFull) for PR 2179 at commit

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53690413 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19392/consoleFull) for PR 2178 at commit

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread byF
Github user byF commented on the pull request: https://github.com/apache/spark/pull/2084#issuecomment-53690368 @SparkQA says the test fails https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19336/consoleFull I was running the test from Intellij where it's

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread byF
Github user byF commented on a diff in the pull request: https://github.com/apache/spark/pull/2084#discussion_r16828221 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -218,11 +218,18 @@ trait HiveTypeCoercion {

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53690908 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19393/consoleFull) for PR 2171 at commit

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread byF
Github user byF commented on a diff in the pull request: https://github.com/apache/spark/pull/2084#discussion_r16828700 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -218,11 +218,18 @@ trait HiveTypeCoercion {

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2084#issuecomment-53693340 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19394/consoleFull) for PR 2084 at commit

[GitHub] spark pull request: [SPARK-1912] Lazily initialize buffers for loc...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2179#issuecomment-53697805 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19391/consoleFull) for PR 2179 at commit

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53698044 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19392/consoleFull) for PR 2178 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53698415 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19395/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53699785 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19396/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53699939 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19396/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53700053 @mengxr This patch removed the `accumulable` operation . repair formula errors in `dropOneDistSampler ` method and some of the performance optimization. About how I

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2155#issuecomment-53700780 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19390/consoleFull) for PR 2155 at commit

[GitHub] spark pull request: [SPARK-3269][SQL] Decreases initial buffer siz...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2171#issuecomment-53701899 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19393/consoleFull) for PR 2171 at commit

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread BigCrunsh
Github user BigCrunsh commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53702702 @mengxr, might it be that you mistake logistic regression for Naive Bayes? Logistic regression typically predicts well-calibrated probabilities, see e.g. [1]; it might

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53702994 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3173][SQL] Timestamp support in the par...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2084#issuecomment-53703006 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19394/consoleFull) for PR 2084 at commit

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53703196 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19397/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: [SPARK-3281] Remove Netty specific code in Blo...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2181#issuecomment-53704032 **Tests timed out** after a configured wait of `120m`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53704442 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19398/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53704512 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19395/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [SPARK-3000][CORE] drop old blocks to disk in ...

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2134#discussion_r16832936 --- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala --- @@ -200,81 +248,118 @@ private[spark] class MemoryStore(blockManager:

[GitHub] spark pull request: [SPARK-3000][CORE] drop old blocks to disk in ...

2014-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2134#discussion_r16832952 --- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala --- @@ -291,54 +376,71 @@ private[spark] class MemoryStore(blockManager:

[GitHub] spark pull request: [SPARK-3272][MLLib]Calculate prediction for no...

2014-08-28 Thread chouqin
Github user chouqin commented on a diff in the pull request: https://github.com/apache/spark/pull/2180#discussion_r16832992 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -34,9 +34,9 @@ import

[GitHub] spark pull request: [SPARK-3272][MLLib]Calculate prediction for no...

2014-08-28 Thread chouqin
Github user chouqin commented on the pull request: https://github.com/apache/spark/pull/2180#issuecomment-53706696 @ScrapCodes thanks for you comments, I have changed indentation to meet the spark style guide just now. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53707200 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19397/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: Using values.sum is easier to understand than ...

2014-08-28 Thread watermen
GitHub user watermen opened a pull request: https://github.com/apache/spark/pull/2182 Using values.sum is easier to understand than using values.foldLeft(0)(_ + _) def sum[B : A](implicit num: Numeric[B]): B = foldLeft(num.zero)(num.plus) Using values.sum is easier to

[GitHub] spark pull request: Using values.sum is easier to understand than ...

2014-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53707923 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Using values.sum is easier to understand than ...

2014-08-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53708729 +1 -- you should open a JIRA though. Although there's reluctance to do cross-cutting code polish PRs, this looks targeted, restricted to example/test code, and is also

[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB]Collapsed Gibbs sampli...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1983#issuecomment-53709038 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19398/consoleFull) for PR 1983 at commit

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53711128 QA tests have started for PR 2137. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19399/consoleFull ---

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53713170 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53713444 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19400/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: [SPARK-3187] [yarn] Cleanup allocator code.

2014-08-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2169#issuecomment-53716263 @vanzin is this purely moving things around again or does it also subsume https://github.com/apache/spark/pull/655/? --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-3069 [DOCS] Build instructions in README...

2014-08-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2014#issuecomment-53717258 @nchammas @pwendell Is the net conclusion that `README.md` should use Maven if anything? I'd be happy to move the wiki into `CONTRIBUTING.md` but then I can't remove

[GitHub] spark pull request: [SPARK-3251][MLLIB]: Clarify learning interfac...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2137#issuecomment-53718901 QA results for PR 2137:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-3285] [examples] Using values.sum is ea...

2014-08-28 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2182#issuecomment-53719188 +1 nice catch, the simpler the examples the easier they'll be to consume by their intended audience: folks who aren't experts yet --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-3279] Remove useless field variable in ...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2177#issuecomment-53719526 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19400/consoleFull) for PR 2177 at commit

[GitHub] spark pull request: [SPARK-3280] Made sort-based shuffle the defau...

2014-08-28 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2178#issuecomment-53719576 is the testing captured somewhere so this change can be evaluated in the future, maybe against other strategies? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-3273]The spark version in the welcome m...

2014-08-28 Thread mattf
Github user mattf commented on a diff in the pull request: https://github.com/apache/spark/pull/2175#discussion_r16838727 --- Diff: repl/src/main/scala/org/apache/spark/repl/SparkILoopInit.scala --- @@ -26,9 +28,9 @@ trait SparkILoopInit { __

[GitHub] spark pull request: [SPARK-3264] Allow users to set executor Spark...

2014-08-28 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2166#issuecomment-53720190 lgtm, nice idea i've been using rpm installed spark, which provides a single version and location on all nodes. however, this will make for a clear path to running

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53721328 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19401/consoleFull) for PR 2168 at commit

[GitHub] spark pull request: [SPARK-3263][GraphX] Fix changes made to Graph...

2014-08-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2168#issuecomment-53722074 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19402/consoleFull) for PR 2168 at commit

  1   2   3   4   >