[GitHub] spark pull request: [Spark-14976][Streaming] make StreamingContext...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12752#issuecomment-215498291 **[Test build #57262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57262/consoleFull)** for PR 12752 at commit [`93d9c62`](https://gi

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12757#issuecomment-215495384 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215494778 **[Test build #2900 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2900/consoleFull)** for PR 12758 at commit [`7355387`](https://g

[GitHub] spark pull request: [SPARK-14931][ML][PySpark] Mismatched default ...

2016-04-28 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/12738#issuecomment-215496215 So this still leaves the difference between the two, but just doesn't impact model persistence. If we do end up doing this fix - might it make sense to leave open anoth

[GitHub] spark pull request: [SPARK-3767] [CORE] Support wildcard in Spark ...

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12753#issuecomment-215495717 I think this is a good idea in general, but I'm less sure about @. Is @ used in windows? Any other places? --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-14555] Second cut of Python API for Str...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12673#discussion_r61464196 --- Diff: python/pyspark/sql/streaming.py --- @@ -51,22 +51,22 @@ def isActive(self): return self._jcq.isActive() @since(2.0)

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215495292 **[Test build #2901 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2901/consoleFull)** for PR 12758 at commit [`7355387`](https://g

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215495090 @zhengruifeng Thanks. While the double space isn't ideal, it isn't wrong either. Merging this right before the code freeze can cause a lot of merge conflicts with other pu

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215494655 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [Spark-14976][Streaming] make StreamingContext...

2016-04-28 Thread mwws
Github user mwws commented on a diff in the pull request: https://github.com/apache/spark/pull/12752#discussion_r61463812 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -195,10 +195,13 @@ class FileInputDStream[K, V, F <: NewInp

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215494155 **[Test build #57261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57261/consoleFull)** for PR 12756 at commit [`61546b8`](https://gi

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215493848 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12720#issuecomment-215492506 **[Test build #57260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57260/consoleFull)** for PR 12720 at commit [`7cf58ef`](https://gi

[GitHub] spark pull request: [Spark-14976][Streaming] make StreamingContext...

2016-04-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/12752#discussion_r61462767 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -195,10 +195,13 @@ class FileInputDStream[K, V, F <: NewI

[GitHub] spark pull request: [SPARK-13257] [Improvement] Scala naive Bayes ...

2016-04-28 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/11192#issuecomment-215492111 Thanks for taking on this issue & fixing it across multiple languages. A super minor thing is generally we try and have separate JIRAs for separate PRs - so in

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215491931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215491928 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215491698 **[Test build #57255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57255/consoleFull)** for PR 12756 at commit [`61546b8`](https://g

[GitHub] spark pull request: [SPARK-14661] [MLlib] trim PCAModel by require...

2016-04-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/12419#discussion_r61462095 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala --- @@ -379,15 +379,21 @@ class RowMatrix @Since("1.0.0") (

[GitHub] spark pull request: [SPARK-13257] [Improvement] Scala naive Bayes ...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/11192#discussion_r61461958 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/NaiveBayesExample.scala --- @@ -46,10 +59,18 @@ object NaiveBayesExample {

[GitHub] spark pull request: [SPARK-13257] [Improvement] Scala naive Bayes ...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/11192#discussion_r61461835 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/NaiveBayesExample.scala --- @@ -23,6 +33,9 @@ import org.apache.spark.{SparkConf, SparkCon

[GitHub] spark pull request: [SPARK-13257] [Improvement] Scala naive Bayes ...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/11192#discussion_r61461459 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/NaiveBayesExample.scala --- @@ -15,6 +15,16 @@ * limitations under the License.

[GitHub] spark pull request: [SPARK-14293] Improve shuffle load balancing a...

2016-04-28 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/12085#issuecomment-215488874 This is interesting, there are some minor formatting things that might make sense to fix while waiting for review - but since this is designed to improve performance it

[GitHub] spark pull request: [SPARK-12235][SPARKR] Enhance mutate() to supp...

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12235][SPARKR] Enhance mutate() to supp...

2016-04-28 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10220#issuecomment-215487241 LGTM. Thanks @sun-rui - Merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-12235][SPARKR] Enhance mutate() to supp...

2016-04-28 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/10220#discussion_r61459532 --- Diff: R/pkg/R/DataFrame.R --- @@ -1451,17 +1451,54 @@ setMethod("mutate", function(.data, ...) { x <- .data

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215486837 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215486833 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215486575 **[Test build #57254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57254/consoleFull)** for PR 12755 at commit [`5167b97`](https://g

[GitHub] spark pull request: [Spark-14976][Streaming] make StreamingContext...

2016-04-28 Thread mwws
Github user mwws commented on a diff in the pull request: https://github.com/apache/spark/pull/12752#discussion_r61457451 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -195,10 +195,13 @@ class FileInputDStream[K, V, F <: NewInp

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215481926 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread freastro
GitHub user freastro opened a pull request: https://github.com/apache/spark/pull/12758 [SPARK-14965] [SQL] Indicate an exception is thrown for a missing struct field ## What changes were proposed in this pull request? Fix to ScalaDoc for StructType. ## How was

[GitHub] spark pull request: [SPARK-14661] [MLlib] trim PCAModel by require...

2016-04-28 Thread psuszyns
Github user psuszyns commented on a diff in the pull request: https://github.com/apache/spark/pull/12419#discussion_r61456369 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala --- @@ -379,15 +379,21 @@ class RowMatrix @Since("1.0.0") (

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r61455519 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11601#discussion_r61455460 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215476737 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215476740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215476580 **[Test build #57256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57256/consoleFull)** for PR 12698 at commit [`8f279a8`](https://g

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215476234 **[Test build #57259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57259/consoleFull)** for PR 12416 at commit [`cc220fc`](https://gi

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215473555 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215471995 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215473565 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215473222 **[Test build #57258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57258/consoleFull)** for PR 12755 at commit [`5167b97`](https://gi

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215473255 **[Test build #57253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57253/consoleFull)** for PR 12640 at commit [`c6c3584`](https://g

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12757#issuecomment-215469182 **[Test build #57257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57257/consoleFull)** for PR 12757 at commit [`14aa401`](https://g

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12757#issuecomment-215469306 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12757#issuecomment-215469308 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215465792 **[Test build #57252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57252/consoleFull)** for PR 12755 at commit [`95b7ca6`](https://g

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12757#issuecomment-215466500 **[Test build #57257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57257/consoleFull)** for PR 12757 at commit [`14aa401`](https://gi

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215466200 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215466204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/12757 [SPARK-14882] [DOCS] Clarify that Spark can be cross-built for other Scala versions ## What changes were proposed in this pull request? Add simple clarification that Spark can be cross-buil

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215463941 **[Test build #57256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57256/consoleFull)** for PR 12698 at commit [`8f279a8`](https://gi

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215463160 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-28 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-215462428 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-14576] [Web UI] Spark console should di...

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12341 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215461379 **[Test build #57255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57255/consoleFull)** for PR 12756 at commit [`61546b8`](https://gi

[GitHub] spark pull request: [SPARK-14576] [Web UI] Spark console should di...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12341#issuecomment-215461027 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215459923 We are close on this, thank you @pravingadakh . I think we need a rebase, and to undo the Python changes to "test_tags" since it actually wasn't the same "test tags" bei

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/12756 [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt throw java.lang.ArrayIndexOutOfBoundsException ## What changes were proposed in this pull request? Handle case where number of predictions is les

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes in Dataset.scala

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215458848 **[Test build #57254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57254/consoleFull)** for PR 12755 at commit [`5167b97`](https://gi

[GitHub] spark pull request: [SPARK-14955] [SQL] avoid stride value equals ...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12739#issuecomment-215452604 I don't feel qualified to evaluate the logic change, but the code style itself looks OK --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-14955] [SQL] avoid stride value equals ...

2016-04-28 Thread bomeng
Github user bomeng commented on the pull request: https://github.com/apache/spark/pull/12739#issuecomment-215450711 @srowen Please review again. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-7424] [ML] ML ClassificationModel shoul...

2016-04-28 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12066#issuecomment-215450008 @jkbradley This is ready for another pass. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-7424] [ML] ML ClassificationModel shoul...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12066#issuecomment-215448778 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-7424] [ML] ML ClassificationModel shoul...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12066#issuecomment-215448783 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-7424] [ML] ML ClassificationModel shoul...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12066#issuecomment-215448622 **[Test build #57251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57251/consoleFull)** for PR 12066 at commit [`486d5ab`](https://g

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-28 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12683#discussion_r61437879 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -71,7 +71,25 @@ test_that("glm and predict", { data = iris, family = poisson(link =

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-28 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12683#discussion_r61437435 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -71,7 +71,25 @@ test_that("glm and predict", { data = iris, family = poisson(link =

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-28 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-215444282 retest it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215441061 **[Test build #57253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57253/consoleFull)** for PR 12640 at commit [`c6c3584`](https://gi

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12754#issuecomment-215440275 cc @mengxr @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215439643 I saw you put ```save/load``` tests at tests.py rather than in doctest. I think the doctest is not only unit tests but also used for illustrating how to use this ```

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes in Dataset.scala

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215439210 OK, that's pretty trivial. Can you take a pass over other related files for typos and formatting problems? --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215437825 @jkbradley @mengxr @jaceklaskowski The new `Dataset.as` API is appled to ML in this PR. --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request: [SPARK-14852][ML] refactored GLM summary into ...

2016-04-28 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12624#issuecomment-215436444 LGTM except the last minor issue. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes in Dataset.scala

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12755#issuecomment-215436193 **[Test build #57252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57252/consoleFull)** for PR 12755 at commit [`95b7ca6`](https://gi

[GitHub] spark pull request: [SPARK-14957][Yarn] Adopt healthy dir to store...

2016-04-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/12735#issuecomment-215436163 Actually after looking a bit more, Spark does fail fast if the shuffle service isn't there because very soon after start up the BlockManager registers with the shuffl

[GitHub] spark pull request: [Minor][DOC] Minor typo fixes in Dataset.scala

2016-04-28 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/12755 [Minor][DOC] Minor typo fixes in Dataset.scala ## What changes were proposed in this pull request? Minor typo fixes ## How was this patch tested? local build You can merge thi

[GitHub] spark pull request: [SPARK-14852][ML] refactored GLM summary into ...

2016-04-28 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12624#discussion_r61433313 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -834,36 +836,55 @@ object GeneralizedLinearRegress

[GitHub] spark pull request: [SPARK-7424] [ML] ML ClassificationModel shoul...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12066#issuecomment-215434643 **[Test build #57251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57251/consoleFull)** for PR 12066 at commit [`486d5ab`](https://gi

[GitHub] spark pull request: [SPARK-14957][Yarn] Adopt healthy dir to store...

2016-04-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/12735#issuecomment-215431514 If the disk is bad or missing there is nothing else you can do then create a new db since as you say deleting wouldn't work. Note I think all it does is log

[GitHub] spark pull request: [SPARK-14914] Normalize Paths/URIs for windows...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12695#issuecomment-215425968 Yes, I'm assuming all of the inputs need to be parseable as a URI in order to keep this sensible. At least, that seems less problematic than trying to accept inputs like

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215425031 @viirya Thanks. I updated this PR following your example. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12754#issuecomment-215414469 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12754#issuecomment-215414472 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12754#issuecomment-215414343 **[Test build #57250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57250/consoleFull)** for PR 12754 at commit [`0b0d969`](https://g

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12754#issuecomment-215412571 **[Test build #57250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57250/consoleFull)** for PR 12754 at commit [`0b0d969`](https://gi

[GitHub] spark pull request: [SPARK-14979] [ML] [PySpark] Add examples for ...

2016-04-28 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/12754 [SPARK-14979] [ML] [PySpark] Add examples for GeneralizedLinearRegression ## What changes were proposed in this pull request? Add Scala/Java/Python examples for ```GeneralizedLinearRegression

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61419435 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-14942][SQL][Streaming] Reduce delay bet...

2016-04-28 Thread lw-lin
Github user lw-lin commented on the pull request: https://github.com/apache/spark/pull/12725#issuecomment-215405018 Sure, I'll add a manual timed executor and some dedicated tests as well. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Tr...

2016-04-28 Thread vruusmann
Github user vruusmann commented on the pull request: https://github.com/apache/spark/pull/9207#issuecomment-215404038 I've been experimenting with a standalone Spark ML Pipelines to PMML converter in recent days. The goal is to cover basic transformers (eg. `StringIndexer`, `OneHotEnc

[GitHub] spark pull request: [SPARK-14914] Normalize Paths/URIs for windows...

2016-04-28 Thread taoli91
Github user taoli91 commented on the pull request: https://github.com/apache/spark/pull/12695#issuecomment-215403420 @srowen `new FIle("D:\test").toURI.toString` can return `file://D/test` but `new URI("D:\test")` won't. But we should make sure that it is a local file before convertin

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61415652 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -866,71 +867,189 @@ class Analyzer( * Note: CT

[GitHub] spark pull request: [SPARK-12810][PySpark] PySpark CrossValidatorM...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12464#issuecomment-215401350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12810][PySpark] PySpark CrossValidatorM...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12464#issuecomment-215401347 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12810][PySpark] PySpark CrossValidatorM...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12464#issuecomment-215401253 **[Test build #57249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57249/consoleFull)** for PR 12464 at commit [`51b412f`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400798 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400722 **[Test build #57247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57247/consoleFull)** for PR 12718 at commit [`9ef9a51`](https://g

<    2   3   4   5   6   7   8   9   >