[GitHub] spark issue #8318: [SPARK-1267][PYSPARK] Adds pip installer for pyspark

2016-10-07 Thread mateiz
Github user mateiz commented on the issue: https://github.com/apache/spark/pull/8318 Something like this would be great IMO. A few questions though: * How will it work if users want to run a different version of PySpark from a different version of Spark (maybe something they

[GitHub] spark issue #14375: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-10-07 Thread praveendareddy21
Github user praveendareddy21 commented on the issue: https://github.com/apache/spark/pull/14375 @holdenk The work on this feature is done. This PR is reopen of PR on 2.0 branch. refer https://github.com/apache/spark/pull/13248 I waiting for admins to verify and merge.

[GitHub] spark issue #14375: [SPARK-15194] [ML] Add Python ML API for MultivariateGau...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14375 Ok so you are blocked on reviewers, but still active on the PR then yes? You might want to try pinging (by using @[username] some of the reviewers from your previous PR

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15089 LGTM - we should double check whether the synchronized block is sufficient, but we can fix that in a follow-up pr if needed. Merging in master. --- If your project is set up for it, you

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66521/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66529/ Test FAILed. ---

[GitHub] spark issue #15333: [SPARK-17761][SQL] Remove MutableRow

2016-10-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15333 Merging to master. Thanks for commenting. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #10162: [SPARK-11250] [SQL] Generate different alias for columns...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/10162 Just pinging @davies or @marmbrus to see if this is something you might have bandwith to review (in which case we can ask the author if they have the time to update to the latest master). --- If

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15393 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66519/ Test PASSed. ---

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15395 **[Test build #66531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66531/consoleFull)** for PR 15395 at commit

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15393 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15371: [SPARK-17816] [Core] Fix ConcurrentModificationEx...

2016-10-07 Thread seyfe
Github user seyfe commented on a diff in the pull request: https://github.com/apache/spark/pull/15371#discussion_r82470766 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -344,15 +338,20 @@ private[spark] class BlockStatusesAccumulator

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15394 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14006 **[Test build #66533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66533/consoleFull)** for PR 14006 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15371 **[Test build #66534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66534/consoleFull)** for PR 15371 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15371 **[Test build #66535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66535/consoleFull)** for PR 15371 at commit

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15366 **[Test build #66536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66536/consoleFull)** for PR 15366 at commit

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66533/ Test PASSed. ---

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14006 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15370: [SPARK-17417][Core] Fix # of partitions for Relia...

2016-10-07 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15370#discussion_r82387014 --- Diff: core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala --- @@ -96,16 +96,15 @@ private[spark] class ReliableCheckpointRDD[T:

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14087 **[Test build #66498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66498/consoleFull)** for PR 14087 at commit

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66498/ Test PASSed. ---

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15370 If you still believe this to complex can you please clarify why so we can discuss. This is basically 5 lines of code (yes lines of code aren't good complexity indicator but..), 3 of them to

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66497/ Test PASSed. ---

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15346 **[Test build #66497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66497/consoleFull)** for PR 15346 at commit

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15370 The central problem with the formatter is just that it doesn't output padding, and I don't see a reason to make the lexical ordering no longer match numerical, as it does now. You can make it output

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15375 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66519/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66520/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66507/ Test PASSed. ---

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-10-07 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/9766 +1 to this functionality, but also to the request to add more tests and documentation. It would also to be good to comment on the idea of using SQL as a more general way to implement this. ---

[GitHub] spark pull request #15394: [SPARK-17749][ML] One pass solver for Weighted Le...

2016-10-07 Thread sethah
GitHub user sethah opened a pull request: https://github.com/apache/spark/pull/15394 [SPARK-17749][ML] One pass solver for Weighted Least Squares with ElasticNet ## What changes were proposed in this pull request? 1. Make a pluggable solver interface for

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #66524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66524/consoleFull)** for PR 9 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15371 @seyfe I'm taking my words back. Yea, `BlockStatusesAccumulator.merge` will be called in driver. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #12691: [Spark-14761][SQL][WIP] Reject invalid join methods when...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12691 Thanks for working on this @bkpathak :) Are you still interested in working on this? If so can you update this to the latest master and then we can try and find a committer to take a more thorough

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #66515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66515/consoleFull)** for PR 14426 at commit

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15089 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66530/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15393 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15393 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12968: [SPARK-15064][ML] Locale support in StopWordsRemover

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12968 @burakkose is this something you are still working on? If so can you update it to master and look at @mengxr's comments - if not interested in working on it anymore no worries. --- If your

[GitHub] spark issue #15387: [SPARK-17782][STREAMING][KAFKA] eliminate race condition...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15387 > but polling ordinarily consumes messages and adjusts position. Even if `enable.auto.commit` is `false`? In the doc, it says `automatically set as the last committed offset`, so I guess

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15371 **[Test build #66534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66534/consoleFull)** for PR 15371 at commit

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #3299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3299/consoleFull)** for PR 13762 at commit

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66522/ Test PASSed. ---

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15390 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15390 **[Test build #66522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66522/consoleFull)** for PR 15390 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15371 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15366 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15387: [SPARK-17782][STREAMING][KAFKA] eliminate race condition...

2016-10-07 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15387 I set auto commit to false, and still recreated the test failure. That makes sense to me, consumer position should still be getting updated in memory even if it isn't saved to storage

[GitHub] spark issue #15391: [MINOR][ML]:remove redundant comment in LogisticRegressi...

2016-10-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15391 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15353: [SPARK-17724][WebUI][Streaming] Unevaluated new lines in...

2016-10-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15353 it looks like the "after" image still has a \n in socket text stream? https://cloud.githubusercontent.com/assets/3925641/19097318/a8edc9de-8a58-11e6-8c37-30271b761284.png --- If your

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15375 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66527/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15371: [SPARK-17816] [Core] Fix ConcurrentModificationException...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15371 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66537/ Test FAILed. ---

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14006 Hi @yinxusen thanks for working on this - can you update it against the latest master? I'm all for switching our examples to be more include based and the improved error messages during doc builds

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2016-10-07 Thread zasdfgbnm
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14198 I'd love to help --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2016-10-07 Thread zasdfgbnm
Github user zasdfgbnm commented on the issue: https://github.com/apache/spark/pull/14198 Something to mention is, there is still one problem that I'm not sure whether I solve it correctly: in python's array, unsigned types are supported, but unsigned types are not supported in JVM.

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15394 cc @yanboliang @dbtsai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-07 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/9 @dbtsai @sethah I updated the code. Now we check the equivalence of K when setting initialModel if K is set previously. We also check the equivalence when fitting a model. --- If your project

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66518/ Test FAILed. ---

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13656: [SPARK-15938]Adding "support" property to MLlib Associat...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13656 I'm not sure if this is something that would still be considered since we aren't doing new development for MLlib anymore. It might make more sense to work on

[GitHub] spark pull request #15371: [SPARK-17816] [Core] Fix ConcurrentModificationEx...

2016-10-07 Thread seyfe
Github user seyfe commented on a diff in the pull request: https://github.com/apache/spark/pull/15371#discussion_r82468147 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -344,15 +338,20 @@ private[spark] class BlockStatusesAccumulator

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-07 Thread dusenberrymw
Github user dusenberrymw commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82468103 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2030,11 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3,

[GitHub] spark issue #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as values in...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/8384 I think its safe to say that this isn't something that there is committer interest on bringing into Spark. Especially now that we can use DataFrames to write out to other formats from Python its

[GitHub] spark issue #7963: [SPARK-6227] [MLlib] [PySpark] Implement PySpark wrappers...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/7963 Now that its past the 2.0 release should we maybe take another look @MLNick / @davies? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #12888: [SPARK-14772][ML,PySpark]Python ML Params.copy treats ui...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12888 So to trigger the test we will need one of the committers - e.g. @davies is one of the more active Python committers (although he has been busy lately) so we can also check with @MLnick . It would

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #66532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66532/consoleFull)** for PR 15394 at commit

[GitHub] spark pull request #7963: [SPARK-6227] [MLlib] [PySpark] Implement PySpark w...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/7963#discussion_r82472226 --- Diff: python/pyspark/mllib/linalg/distributed.py --- @@ -303,6 +303,121 @@ def tallSkinnyQR(self, computeQ=False): R = decomp.call("R")

[GitHub] spark pull request #7963: [SPARK-6227] [MLlib] [PySpark] Implement PySpark w...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/7963#discussion_r82471915 --- Diff: docs/mllib-dimensionality-reduction.md --- @@ -84,6 +84,25 @@ quick-start guide. Be sure to also include *spark-mllib* to your build file as a

[GitHub] spark pull request #7963: [SPARK-6227] [MLlib] [PySpark] Implement PySpark w...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/7963#discussion_r82472791 --- Diff: python/pyspark/mllib/linalg/distributed.py --- @@ -303,6 +303,121 @@ def tallSkinnyQR(self, computeQ=False): R = decomp.call("R")

[GitHub] spark issue #10162: [SPARK-11250] [SQL] Generate different alias for columns...

2016-10-07 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/10162 I'd be happy to update to the latest master if we want to review this now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13794: [SPARK-15574][ML][PySpark] Python meta-algorithms in Sca...

2016-10-07 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/13794 Thanks @holdenk Yes, I am still interested in this. @jkbradley Do we still need the PR to support meta-algorithms in PySpark? --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15366 It's failed twice now on the same issue but it's completely unrelated to my change. Trying one more time, but is this failure a known issue? --- If your project is set up for it, you can reply

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-07 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15249 @tgravescs @mridulm To avoid being stuck in analysis paralysis for this feature, I'd propose the following: (1) We merge this PR. I think we're mostly in agreement that the behavior

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-07 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15395 @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66514/ Test PASSed. ---

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15370 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66528/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #3299 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3299/consoleFull)** for PR 13762 at commit

[GitHub] spark pull request #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses i...

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/13762#discussion_r82465493 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -196,8 +196,13 @@ final class OneVsRestModel private[ml] (

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #66529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66529/consoleFull)** for PR 15394 at commit

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66515/ Test PASSed. ---

[GitHub] spark issue #13794: [SPARK-15574][ML][PySpark] Python meta-algorithms in Sca...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13794 @yinxusen - is this something you are still interested in? If so updating it to master would be good as well as making sure the unit tests pass in jenkins (a lot of reviwers just skip PRs which are

[GitHub] spark issue #14006: [SPARK-13015][MLlib][DOC] Replace example code in mllib-...

2016-10-07 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/14006 @holdenk I'll update a version ASAP --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #12675: [SPARK-14894][PySpark] Add result summary api to ...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12675#discussion_r82468682 --- Diff: python/pyspark/ml/clustering.py --- @@ -56,8 +56,83 @@ def gaussiansDF(self): """ return self._call_java("gaussiansDF")

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-07 Thread dusenberrymw
Github user dusenberrymw commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82468575 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2030,11 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3,

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15390 **[Test build #3300 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3300/consoleFull)** for PR 15390 at commit

[GitHub] spark pull request #12675: [SPARK-14894][PySpark] Add result summary api to ...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12675#discussion_r82468590 --- Diff: python/pyspark/ml/tests.py --- @@ -1070,6 +1070,21 @@ def test_logistic_regression_summary(self): sameSummary = model.evaluate(df)

[GitHub] spark issue #15394: [SPARK-17749][ML] One pass solver for Weighted Least Squ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #66529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66529/consoleFull)** for PR 15394 at commit

[GitHub] spark pull request #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnType...

2016-10-07 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/15395 [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite ## What changes were proposed in this pull request? The default buffer size is not big enough for randomly generated MapType.

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66519/consoleFull)** for PR 15393 at commit

[GitHub] spark pull request #15333: [SPARK-17761][SQL] Remove MutableRow

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15333 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #12337: [SPARK-15566] Expose null checking function to Python la...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12337 So I'm curious what is the intention of this versus the existing isnull function? It seems like this might not be needed in which case we should consider closing the PR & JR (of course if there is

  1   2   3   4   5   6   7   >