[GitHub] spark issue #9524: [SPARK-10387][ML] Add code gen for gbt

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9524 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-09-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13950 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #9524: [SPARK-10387][ML] Add code gen for gbt

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65122/ Test PASSed. ---

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9207 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65123/ Test PASSed. ---

[GitHub] spark issue #9524: [SPARK-10387][ML] Add code gen for gbt

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9524 **[Test build #65122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65122/consoleFull)** for PR 9524 at commit

[GitHub] spark issue #15019: [SPARK-17387][PYSPARK] Allow passing of args to gateway ...

2016-09-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15019 > that won't work in the PySpark shell either though, even if you restart the SparkContext That's a separate issue which should also be fixed (stopping a python SparkContext does not stop

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9207 **[Test build #65123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65123/consoleFull)** for PR 9207 at commit

[GitHub] spark issue #15019: [SPARK-17387][PYSPARK] Allow passing of args to gateway ...

2016-09-08 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15019 Hmmm, that won't work in the PySpark shell either though, even if you restart the SparkContext. If that's what you want though, @zjffdu 's PR addresses that. --- If your project is set up

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-09-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/13950 LGTM. Merging to master! Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14887: [SPARK-17321][YARN] YARN shuffle service should u...

2016-09-08 Thread zhaoyunjiong
Github user zhaoyunjiong closed the pull request at: https://github.com/apache/spark/pull/14887 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65119/ Test PASSed. ---

[GitHub] spark issue #14887: [SPARK-17321][YARN] YARN shuffle service should use good...

2016-09-08 Thread zhaoyunjiong
Github user zhaoyunjiong commented on the issue: https://github.com/apache/spark/pull/14887 @jerryshao, In the original patch I made a mistake, it should return _recoveryPath in the last instead of return null. After that fix, I don't think it will introduce a situation where

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #65119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65119/consoleFull)** for PR 14467 at commit

[GitHub] spark issue #15019: [SPARK-17387][PYSPARK] Allow passing of args to gateway ...

2016-09-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15019 @BryanCutler sorry, this is not what I meant. Guess I was confused about your comment on the bug. See my original code on the bug: ``` >>> from pyspark import SparkContext

[GitHub] spark issue #15019: [SPARK-17387][PYSPARK] Allow passing of args to gateway ...

2016-09-08 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15019 @vanzin and @zjffdu , this is my take on a simplified solution that will allow configuring the JVM from the command line. You still won't be able to use these confs in the shell when creating

[GitHub] spark issue #15019: [SPARK-17387][PYSPARK] Allow passing of args to gateway ...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15019 **[Test build #65125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65125/consoleFull)** for PR 15019 at commit

[GitHub] spark issue #14887: [SPARK-17321][YARN] YARN shuffle service should use good...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14887 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14887: [SPARK-17321][YARN] YARN shuffle service should u...

2016-09-08 Thread zhaoyunjiong
GitHub user zhaoyunjiong reopened a pull request: https://github.com/apache/spark/pull/14887 [SPARK-17321][YARN] YARN shuffle service should use good disk from yarn.nodemanager.local-dirs ## What changes were proposed in this pull request? Use good disk from

[GitHub] spark pull request #15019: [SPARK-17387][PYSPARK] Allow passing of args to g...

2016-09-08 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/15019 [SPARK-17387][PYSPARK] Allow passing of args to gateway to configure JVM ## What changes were proposed in this pull request? When not using spark-submit, user configuration for the JVM is

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78111319 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -370,49 +420,102 @@ class LogisticRegression

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78111049 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -370,49 +420,102 @@ class LogisticRegression

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78110797 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -333,22 +387,18 @@ class LogisticRegression

[GitHub] spark issue #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend should...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15013 **[Test build #65124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65124/consoleFull)** for PR 15013 at commit

[GitHub] spark pull request #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend...

2016-09-08 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15013#discussion_r78109991 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -199,6 +199,9 @@ private[spark] class BlockManager(

[GitHub] spark pull request #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend...

2016-09-08 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15013#discussion_r78109911 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -148,12 +149,32 @@ private[spark] class

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14691 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65118/ Test PASSed. ---

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65116/ Test PASSed. ---

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14691 **[Test build #65118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65118/consoleFull)** for PR 14691 at commit

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #65116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65116/consoleFull)** for PR 14750 at commit

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78109655 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -333,22 +387,18 @@ class LogisticRegression

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65120/ Test FAILed. ---

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13571 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13571 **[Test build #65120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65120/consoleFull)** for PR 13571 at commit

[GitHub] spark pull request #15016: [SPARK-17405] RowBasedKeyValueBatch should use de...

2016-09-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15016: [SPARK-17405] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15016 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15017 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65115/ Test PASSed. ---

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15017 **[Test build #65115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65115/consoleFull)** for PR 15017 at commit

[GitHub] spark pull request #14750: [SPARK-17183][SQL] put hive serde table schema to...

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14750#discussion_r78106864 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -446,29 +449,46 @@ private[spark] class

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r78106170 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78105821 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -827,67 +1743,117 @@ class LogisticRegressionSuite

[GitHub] spark issue #15016: [SPARK-17405] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15016 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15016: [SPARK-17405] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15016 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65114/ Test PASSed. ---

[GitHub] spark issue #15016: [SPARK-17405] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15016 **[Test build #65114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65114/consoleFull)** for PR 15016 at commit

[GitHub] spark pull request #14959: [SPARK-17387][PYSPARK] Creating SparkContext() fr...

2016-09-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14959#discussion_r78105215 --- Diff: python/pyspark/java_gateway.py --- @@ -51,13 +51,16 @@ def launch_gateway(): on_windows = platform.system() == "Windows"

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78104785 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -232,6 +363,47 @@ class LogisticRegressionSuite

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65117/ Test PASSed. ---

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15018 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15018 **[Test build #65117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65117/consoleFull)** for PR 15018 at commit

[GitHub] spark pull request #14750: [SPARK-17183][SQL] put hive serde table schema to...

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14750#discussion_r78104317 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -408,7 +411,9 @@ private[spark] class

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/15015 how about adding type check? `expect_equal(typeof(summary$weights), "list")` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15015 since the format of the output is changing it would be great to check more than just the length in test --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend should...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15013 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65113/ Test PASSed. ---

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78101965 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -333,22 +387,18 @@ class LogisticRegression

[GitHub] spark issue #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend should...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15013 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend should...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15013 **[Test build #65113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65113/consoleFull)** for PR 15013 at commit

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13950 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65112/ Test PASSed. ---

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/15015 There is already a test `expect_equal(length(summary$weights), 64)` https://github.com/apache/spark/blob/master/R/pkg/inst/tests/testthat/test_mllib.R#L371 is it enough or we should

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13950 **[Test build #65112 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65112/consoleFull)** for PR 13950 at commit

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78101327 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -311,8 +348,25 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark issue #9524: [SPARK-10387][ML] Add code gen for gbt

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9524 **[Test build #65122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65122/consoleFull)** for PR 9524 at commit

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9207 **[Test build #65123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65123/consoleFull)** for PR 9207 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #65121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65121/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65121/ Test FAILed. ---

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15018 **[Test build #65117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65117/consoleFull)** for PR 15018 at commit

[GitHub] spark issue #9524: [SPARK-10387][ML] Add code gen for gbt

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/9524 ping @dbtsai if you have some cycles :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #65119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65119/consoleFull)** for PR 14467 at commit

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14691 **[Test build #65118 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65118/consoleFull)** for PR 14691 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #65121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65121/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13571 **[Test build #65120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65120/consoleFull)** for PR 13571 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11105 Back from vacation if anyone has review bandwidth would be really cool to try and get this in :) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13571: [SPARK-15369][WIP][RFC][PySpark][SQL] Expose potential t...

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13571 Ping @yanboliang & @davies for thoughts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15018: [SPARK-17455][MLlib] Improve PAVA implementation ...

2016-09-08 Thread neggert
GitHub user neggert opened a pull request: https://github.com/apache/spark/pull/15018 [SPARK-17455][MLlib] Improve PAVA implementation in IsotonicRegression ## What changes were proposed in this pull request? New implementation of the Pool Adjacent Violators Algorithm

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14467 Back from vacation if anyone has review bandwidth I'd love to get this cleanup in :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15017 Would it make sense to replace some of the hacks with this util at the same time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #65116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65116/consoleFull)** for PR 14750 at commit

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14750 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14691 @zsxwing I'm not sure I understand - the current API allows users to specify the class name and uses reflection to create it - this allows users to specify a specific instance of the class but the

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-09-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/14691 In general, we don't want to use any source or sink classes to an API of DataStreamWriter because they are not as stable as DataStreamWriter. This is similar to DataFrameWriter. --- If your

[GitHub] spark pull request #15015: [SPARK-16445][MLlib][SparkR] Fix @return descript...

2016-09-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15015#discussion_r78093981 --- Diff: R/pkg/R/mllib.R --- @@ -732,7 +733,6 @@ setMethod("summary", signature(object = "MultilayerPerceptronClassificationModel

[GitHub] spark issue #15015: [SPARK-16445][MLlib][SparkR] Fix @return description for...

2016-09-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15015 cool thanks, just one comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78093424 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -50,6 +50,8 @@ private[classification] trait

[GitHub] spark pull request #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend...

2016-09-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15013#discussion_r78093133 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -199,6 +199,9 @@ private[spark] class BlockManager(

[GitHub] spark pull request #15013: [SPARK-17451] [CORE] CoarseGrainedExecutorBackend...

2016-09-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15013#discussion_r78092817 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -148,12 +149,32 @@ private[spark] class

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-08 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r78092320 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #14834: [SPARK-17163][ML][WIP] Unified LogisticRegression interf...

2016-09-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14834 @sethah For sparse MLOR problems with L1, the models will be sparse in row. As a result, in the sparse, we need to store the models in CSR format, and CSR models can be used for model prediction

[GitHub] spark issue #14998: [SPARK-11496][GRAPHX] Parallel implementation of persona...

2016-09-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14998 Some minor issues, and LGTM. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15016: [SPARK-17405] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread ooq
Github user ooq commented on the issue: https://github.com/apache/spark/pull/15016 LGTM. This is good. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14998: [SPARK-11496][GRAPHX] Parallel implementation of ...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14998#discussion_r78090880 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala --- @@ -163,6 +166,85 @@ object PageRank extends Logging { }

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15017 **[Test build #65115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65115/consoleFull)** for PR 15017 at commit

[GitHub] spark issue #15017: [SPARK-17456][CORE] Utility for parsing Spark versions

2016-09-08 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15017 CC: @yanboliang @MLnick Does one of you have time to review this? It should be very quick. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #15017: [SPARK-17456][CORE] Utility for parsing Spark ver...

2016-09-08 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/15017 [SPARK-17456][CORE] Utility for parsing Spark versions ## What changes were proposed in this pull request? This patch adds methods for extracting major and minor versions as Int types

[GitHub] spark issue #15016: [SPARK-16525] RowBasedKeyValueBatch should use default p...

2016-09-08 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15016 LGTM pending tests. I think this is definitely the right fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #14998: [SPARK-11496][GRAPHX] Parallel implementation of ...

2016-09-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14998#discussion_r78090050 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala --- @@ -163,6 +166,85 @@ object PageRank extends Logging { }

<    1   2   3   4   5   >