[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66521/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66518/ Test FAILed. ---

[GitHub] spark issue #13656: [SPARK-15938]Adding "support" property to MLlib Associat...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13656 I'm not sure if this is something that would still be considered since we aren't doing new development for MLlib anymore. It might make more sense to work on

[GitHub] spark pull request #15371: [SPARK-17816] [Core] Fix ConcurrentModificationEx...

2016-10-07 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15371#discussion_r82467368 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -344,15 +338,20 @@ private[spark] class BlockStatusesAccumulator

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15366 **[Test build #66518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66518/consoleFull)** for PR 15366 at commit

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15089 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/14006 @holdenk I'll update a version ASAP --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13794: [SPARK-15574][ML][PySpark] Python meta-algorithms in Sca...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13794 @yinxusen - is this something you are still interested in? If so updating it to master would be good as well as making sure the unit tests pass in jenkins (a lot of reviwers just skip PRs which are

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2016-10-07 Thread zasdfgbnm
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14198 Something to mention is, there is still one problem that I'm not sure whether I solve it correctly: in python's array, unsigned types are supported, but unsigned types are not supported in JVM.

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15394 cc @yanboliang @dbtsai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/9 @dbtsai @sethah I updated the code. Now we check the equivalence of K when setting initialModel if K is set previously. We also check the equivalence when fitting a model. --- If your project

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15089 LGTM - we should double check whether the synchronized block is sufficient, but we can fix that in a follow-up pr if needed. Merging in master. --- If your project is set up for it, you

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66515/ Test PASSed. ---

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #66529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66529/consoleFull)** for PR 15394 at commit

[GitHub] spark issue #12691: [Spark-14761][SQL][WIP] Reject invalid join methods when...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12691 Thanks for working on this @bkpathak :) Are you still interested in working on this? If so can you update this to the latest master and then we can try and find a committer to take a more thorough

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #66515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66515/consoleFull)** for PR 14426 at commit

[GitHub] spark pull request #15394: [SPARK-17749][ML] One pass solver for Weighted Le...

2016-10-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r82466110 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala --- @@ -85,73 +101,193 @@ private[ml] class WeightedLeastSquares(

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66524/ Test PASSed. ---

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #66524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66524/consoleFull)** for PR 9 at commit

[GitHub] spark pull request #15394: [SPARK-17749][ML] One pass solver for Weighted Le...

2016-10-07 Thread sethah
GitHub user sethah opened a pull request: https://github.com/apache/spark/pull/15394 [SPARK-17749][ML] One pass solver for Weighted Least Squares with ElasticNet ## What changes were proposed in this pull request? 1. Make a pluggable solver interface for

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15371 @seyfe I'm taking my words back. Yea, `BlockStatusesAccumulator.merge` will be called in driver. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #3299 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3299/consoleFull)** for PR 13762 at commit

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66528/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15370 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66514/ Test PASSed. ---

[GitHub] spark pull request #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses i...

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/13762#discussion_r82465493 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -196,8 +196,13 @@ final class OneVsRestModel private[ml] (

[GitHub] spark issue #8318: [SPARK-1267][PYSPARK] Adds pip installer for pyspark

2016-10-07 Thread mateiz
Github user mateiz commented on the issue: https://github.com/apache/spark/pull/8318 BTW the other change now is that we don't make an assembly JAR by default anymore, though we could build one for this. We just need a build script for this that's solid, produces a

[GitHub] spark pull request #14375: [SPARK-15194] [ML] Add Python ML API for Multivar...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14375#discussion_r82465422 --- Diff: python/pyspark/ml/stat/__init__.py --- @@ -0,0 +1,27 @@ + --- End diff -- Super minor but probably don't need this blank line.

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15370 **[Test build #66514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66514/consoleFull)** for PR 15370 at commit

[GitHub] spark issue #14375: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14375 Ok so you are blocked on reviewers, but still active on the PR then yes? You might want to try pinging (by using @[username] some of the reviewers from your previous PR

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2016-10-07 Thread zasdfgbnm
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14198 I'd love to help --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #8318: [SPARK-1267][PYSPARK] Adds pip installer for pyspark

2016-10-07 Thread mateiz
Github user mateiz commented on the issue: https://github.com/apache/spark/pull/8318 Something like this would be great IMO. A few questions though: * How will it work if users want to run a different version of PySpark from a different version of Spark (maybe something they

[GitHub] spark issue #14375: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-10-07 Thread praveendareddy21
Github user praveendareddy21 commented on the issue: https://github.com/apache/spark/pull/14375 @holdenk The work on this feature is done. This PR is reopen of PR on 2.0 branch. refer https://github.com/apache/spark/pull/13248 I waiting for admins to verify and merge.

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14006 Hi @yinxusen thanks for working on this - can you update it against the latest master? I'm all for switching our examples to be more include based and the improved error messages during doc builds

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14198 Oh interesting - thanks for working on this @zasdfgbnm and sorry its sort of fallen through the cracks. Is this something you are still working on? For PRs to get in you generally need some form of

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15249 The question to me comes down to how many and how often do you expect temporary resource issues. At some point if its just from that much skew you should probably fix your configs and it would be

[GitHub] spark issue #14375: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14375 Is this something you are still working on? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14394: [SPARK-16786] [Python] [WIP] LDA topic distributi...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14394#discussion_r82463086 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -777,6 +777,10 @@ class DistributedLDAModel private[clustering] (

[GitHub] spark pull request #15375: [SPARK-17790] Support for parallelizing R data.fr...

2016-10-07 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/15375#discussion_r82462875 --- Diff: R/pkg/R/context.R --- @@ -126,13 +126,13 @@ parallelize <- function(sc, coll, numSlices = 1) { if (numSlices > length(coll))

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15384 @tdas since this one has not yet been merged, I just updated the PR to remove the topic cleanup. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15390 LGTM, pending the new test run. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66527/consoleFull)** for PR 15375 at commit

[GitHub] spark pull request #15074: [SPARK-17520] Implement a better __eq__ for Spars...

2016-10-07 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/15074#discussion_r82461977 --- Diff: python/pyspark/mllib/linalg/__init__.py --- @@ -1296,9 +1296,19 @@ def asML(self): return newlinalg.SparseMatrix(self.numRows,

[GitHub] spark issue #15392: [SPARK-17830] Annotate spark.sql package with InterfaceS...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15392 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66511/ Test PASSed. ---

[GitHub] spark issue #15392: [SPARK-17830] Annotate spark.sql package with InterfaceS...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15392 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15392: [SPARK-17830] Annotate spark.sql package with InterfaceS...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15392 **[Test build #66511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66511/consoleFull)** for PR 15392 at commit

[GitHub] spark issue #8318: [SPARK-1267][PYSPARK] Adds pip installer for pyspark

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/8318 cc @mateiz are you interested in seeing something like this move forward? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66526/ Test FAILed. ---

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #66526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66526/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15391: [MINOR][ML]:remove redundant comment in LogisticRegressi...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66512/ Test PASSed. ---

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r82459788 --- Diff: python/pyspark/ml/clustering.py --- @@ -201,7 +203,74 @@ def computeCost(self, dataset): """ return

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r82460090 --- Diff: python/pyspark/ml/clustering.py --- @@ -201,7 +203,74 @@ def computeCost(self, dataset): """ return

[GitHub] spark issue #15391: [MINOR][ML]:remove redundant comment in LogisticRegressi...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15391 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r82459893 --- Diff: python/pyspark/ml/clustering.py --- @@ -201,7 +203,74 @@ def computeCost(self, dataset): """ return

[GitHub] spark issue #15391: [MINOR][ML]:remove redundant comment in LogisticRegressi...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15391 **[Test build #66512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66512/consoleFull)** for PR 15391 at commit

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13440 Is this something your still working on? If so it would be good to merge in the latest master. We can also check with @jkbradley to see if he has some review bandwidth. --- If your project is set

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13780 so just following up because I know there is some other loosely blocked on this. Do @rxin @marmbrus @davies @gatorsmile have any comments? --- If your project is set up for it, you can reply to

[GitHub] spark issue #13036: [SPARK-15243][ML][SQL][PYSPARK] Param methods should use...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13036 Just a quick ping @sethah - I know your pretty busy but I'm assuming this is still active. One minor note is it seems there is another new addition to types.py which maybe should also be changed.

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #66526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66526/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15393 Yep. I knew. What I'm asking is we don't need to explain 'contains' usage here in this PR. I prefer not to advertise `contains` for `Option[Boolean]`. ``` scala> val o:

[GitHub] spark issue #12398: [SPARK-5929][PYSPARK] Context addPyPackage and addPyRequ...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12398 @buckhx Just following up to see if this is something you are still interested in working on? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15393 @dongjoon-hyun I already did :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15393 BTW, could you update the PR description, @HyukjinKwon ? Maybe, removing `exists` part? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15113: [SPARK-17508][PYSPARK][ML] PySpark treat Param values No...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15113 @yanboliang : I've certainly seen even Spark developers be tripped up with the handling of `None` by the Py4J bridge so I wouldn't be surprised if this is a more common problem than we might

[GitHub] spark issue #14215: [SPARK-16544][SQL][WIP] Support for conversion from comp...

2016-10-07 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/spark/pull/14215 @HyukjinKwon yep, keep each PR as small as possible is a good idea. BTW, may I know the target version of your non-vectorize fix? Our production job is in need of this fix. Separating

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #66525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66525/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #13771: [SPARK-13748][PYSPARK][DOC] Add the description for expl...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13771 I think this is a pretty reasonable simple docstirng improvement - maybe @srowen or @davies can take a look? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66516/ Test PASSed. ---

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11601 **[Test build #66516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66516/consoleFull)** for PR 11601 at commit

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15390 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66510/ Test PASSed. ---

[GitHub] spark issue #15159: [SPARK-17605][SPARK_SUBMIT] Add option spark.usePython a...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15159 @zjffdu so is the intent of this to allow people to use PySpark from Scala Spark applications? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15390 **[Test build #66510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66510/consoleFull)** for PR 15390 at commit

[GitHub] spark issue #15159: [SPARK-17605][SPARK_SUBMIT] Add option spark.usePython a...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15159 Just a heads up this has conflicts with master so it might be good to update (since I know a lot of reviewers use the spark-pr dashboard and maybe skip PRs which aren't mergable). --- If your

[GitHub] spark issue #15209: replace function type with function isinstance

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15209 Thanks for looking to get involved @frankfqchen :) I think this PR definitely needs a JIRA and maybe some more description about what the intent of the change is rather than just the

[GitHub] spark issue #15074: [SPARK-17520] Implement a better __eq__ for SparseMatrix

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15074 Maybe it would be good to add some more tests for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #15074: [SPARK-17520] Implement a better __eq__ for Spars...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15074#discussion_r82455939 --- Diff: python/pyspark/mllib/linalg/__init__.py --- @@ -1296,9 +1296,19 @@ def asML(self): return newlinalg.SparseMatrix(self.numRows,

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #66524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66524/consoleFull)** for PR 9 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #66523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66523/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66520/ Test PASSed. ---

[GitHub] spark pull request #15074: [SPARK-17520] Implement a better __eq__ for Spars...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15074#discussion_r82454768 --- Diff: python/pyspark/mllib/linalg/__init__.py --- @@ -1296,9 +1296,19 @@ def asML(self): return newlinalg.SparseMatrix(self.numRows,

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15384 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66520/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread mtustin-handy
Github user mtustin-handy commented on the issue: https://github.com/apache/spark/pull/12004 I don't see any downsides to this. At present working with s3 isn't super painful, but I do see why one would want support to be better and smoother. --- If your project is set up for it,

[GitHub] spark issue #14547: [SPARK-16718][MLlib] gbm-style treeboost

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14547 Just a heads up there is a merge conflict with the excludes you might want to update for so that jenkins can run its tests on this PR :) --- If your project is set up for it, you can reply to this

[GitHub] spark issue #10307: [SPARK-12334][SQL][PYSPARK] Support read from multiple i...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/10307 @Donderia it seems like some of the files have changed since 1.6 so this won't apply cleanly against 1.6 @zjffdu if your still working on this can you update this against the latest master?

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15390 **[Test build #66522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66522/consoleFull)** for PR 15390 at commit

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15393 I apologize and will definitely be careful as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14861: [SPARK-17287] [PYSPARK] Add recursive kwarg to Python Sp...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14861 @jpiper are you still working on this PR? If so you can merge in the latest version of master so we can continue to review? (If not interested that's ok just let us know :)). --- If your project

[GitHub] spark issue #14233: [SPARK-16490] [Examples] added a python example for chis...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14233 Thanks for adding it to the documentation files, looks reasonable pending @sethah's suggestions and jenkins :) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15393 Actually, I hit this before. Sorry, I did not catch it when I merging the PR. Next time, I will be more careful about it. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-10-07 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/9766 +1 to this functionality, but also to the request to add more tests and documentation. It would also to be good to comment on the idea of using SQL as a more general way to implement this. ---

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15393 LGTM pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66506/ Test PASSed. ---

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    1   2   3   4   5   6   7   >