[GitHub] spark pull request: [SPARK-14369][SQL] Locality support for FileSc...

2016-04-21 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12527#issuecomment-213037505 Just want to add a note. For that test case, we have a join that only shuffle one side of the input, so we have both preferred locations of original input files as well a

[GitHub] spark pull request: [SPARK-14483][WEBUI] Display user name for eac...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12257#issuecomment-213037359 **[Test build #56561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56561/consoleFull)** for PR 12257 at commit [`c09e17c`](https://gi

[GitHub] spark pull request: [SPARK-10001][Core] Interrupt tasks in repl wi...

2016-04-21 Thread jodersky
Github user jodersky commented on a diff in the pull request: https://github.com/apache/spark/pull/12557#discussion_r60625828 --- Diff: repl/src/main/scala/org/apache/spark/repl/Signaling.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-213036192 **[Test build #56560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56560/consoleFull)** for PR 9441 at commit [`9e05eba`](https://git

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-213036211 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-213036217 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9441#issuecomment-213035265 **[Test build #56560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56560/consoleFull)** for PR 9441 at commit [`9e05eba`](https://gith

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60624962 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -182,7 +182,7 @@ private[spark] class BlockManager( val shuffleC

[GitHub] spark pull request: [SPARK-14369][SQL] Locality support for FileSc...

2016-04-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12527#issuecomment-213034425 The root cause of the deadlock has been found. Essentially, we should prevent "localhost" to be returned as `FileScanRDD` preferred locations. Here's a detailed descr

[GitHub] spark pull request: [SPARK-14783] Preserve full exception stacktra...

2016-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12548 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-213032974 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-21 Thread xwu0226
GitHub user xwu0226 opened a pull request: https://github.com/apache/spark/pull/12579 [SPARK-14346][SQL] Show Create Table (Native) This is a rebased version of [#12132](https://github.com/apache/spark/pull/12132) and [#12406](https://github.com/apache/spark/pull/12406) ##

[GitHub] spark pull request: [SPARK-14783] Preserve full exception stacktra...

2016-04-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12548#issuecomment-213032298 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60623283 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -182,7 +182,7 @@ private[spark] class BlockManager( val shuffleConfi

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60623144 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java --- @@ -186,7 +186,12 @@ public void tes

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread dusenberrymw
Github user dusenberrymw commented on a diff in the pull request: https://github.com/apache/spark/pull/9441#discussion_r60622845 --- Diff: python/pyspark/mllib/linalg/distributed.py --- @@ -151,6 +153,151 @@ def numCols(self): """ return self._java_matrix_w

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60622806 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java --- @@ -49,7 +49,7 @@ public class

[GitHub] spark pull request: [SPARK-14799][SQL] Remove MetastoreRelation de...

2016-04-21 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12566#issuecomment-213029923 The current changes look good! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60622768 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleCleanupSuite.java --- @@ -37,19 +37,20 @@ // Same-threa

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60622524 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -149,16 +149,23 @@ public void regis

[GitHub] spark pull request: [SPARK-10496][SQL] Add DataFrame cumulative su...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12578#issuecomment-213026193 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9656] [MLlib] [Python] Add missing meth...

2016-04-21 Thread dusenberrymw
Github user dusenberrymw commented on a diff in the pull request: https://github.com/apache/spark/pull/9441#discussion_r60621403 --- Diff: python/pyspark/mllib/linalg/distributed.py --- @@ -297,6 +444,20 @@ def numCols(self): """ return self._java_matrix_wr

[GitHub] spark pull request: [SPARK-14790] Always run scalastyle on sbt com...

2016-04-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12555#issuecomment-213026077 I'm not very against it (though Maven should ideally change to check style as well on 'pre-compile'). I remember someone arguing that they didn't want to have to pass al

[GitHub] spark pull request: [SPARK-10496][SQL] Add DataFrame cumulative su...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12578#issuecomment-213026200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14744][examples] Clean up examples pack...

2016-04-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12544#discussion_r60621196 --- Diff: examples/pom.xml --- @@ -35,6 +35,10 @@ examples none package +provided --- End diff -- Yeah, these d

[GitHub] spark pull request: [SPARK-14369][SQL] Locality support for FileSc...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12527#issuecomment-213025638 **[Test build #56559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56559/consoleFull)** for PR 12527 at commit [`e0bfa3e`](https://gi

[GitHub] spark pull request: [SPARK-14744][examples] Clean up examples pack...

2016-04-21 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/12544#issuecomment-213025682 > Is there any documentation that needs to be updated There isn't any explicit documentation about how to run streaming examples (that I can find). The pyspark b

[GitHub] spark pull request: [SPARK-10496][SQL] Add DataFrame cumulative su...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12578#issuecomment-213025805 **[Test build #56551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56551/consoleFull)** for PR 12578 at commit [`97af903`](https://g

[GitHub] spark pull request: [SPARK-14790] Always run scalastyle on sbt com...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12555#issuecomment-213025184 Yea I think the main thing is that this will be more integrated and more automated. Many pull requests when they were first submitted fail our style checker, both from lon

[GitHub] spark pull request: [SPARK-14793][SQL] Code generation for large c...

2016-04-21 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12559#issuecomment-213025223 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-213024675 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-213024678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-213024453 **[Test build #56553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56553/consoleFull)** for PR 12500 at commit [`ec54d74`](https://g

[GitHub] spark pull request: Investigate flaky test

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12565#issuecomment-213024169 **[Test build #56558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56558/consoleFull)** for PR 12565 at commit [`64447a4`](https://gi

[GitHub] spark pull request: [SPARK-14790] Always run scalastyle on sbt com...

2016-04-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12555#issuecomment-213023963 (Can it be done without all this code though?) You can run style checks before submitting of course, and they do get run by the PR builder. Does it change much except

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-21 Thread wangmiao1981
Github user wangmiao1981 commented on the pull request: https://github.com/apache/spark/pull/12560#issuecomment-213023729 Thanks all for your comments! Let me figure out how to collect the information without slowing the algorithm. @MLnick The names are passed to the log. For example,

[GitHub] spark pull request: [SPARK-14796][SQL] Add spark.sql.optimizer.min...

2016-04-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12562#discussion_r60620112 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeInSuite.scala --- @@ -128,4 +131,21 @@ class OptimizeInSuite extends Plan

[GitHub] spark pull request: [SPARK-14699][Core]Stop endpoints before closi...

2016-04-21 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/12481#issuecomment-213022550 > @BryanCutler Fixed it in this PR. Outbox should not close the client since it will be reused by others. I confirmed this fix takes care of the error in the app

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-213022286 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-213022233 **[Test build #56554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56554/consoleFull)** for PR 12498 at commit [`5081e2c`](https://g

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-213022288 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14796][SQL] Add spark.sql.optimizer.min...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12562#issuecomment-213021952 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14796][SQL] Add spark.sql.optimizer.min...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12562#issuecomment-213021953 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14790] Always run scalastyle on sbt com...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12555#issuecomment-213021785 I was concerned about this increasing the time but last night I had 3 prs that failed style checkers and now do think it would've been great if the checking was more autom

[GitHub] spark pull request: [SPARK-10001][Core] Interrupt tasks in repl wi...

2016-04-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12557#discussion_r60619566 --- Diff: repl/src/main/scala/org/apache/spark/repl/Signaling.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14796][SQL] Add spark.sql.optimizer.min...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12562#issuecomment-213021683 **[Test build #56550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56550/consoleFull)** for PR 12562 at commit [`9df35bb`](https://g

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/12536#issuecomment-213021639 Oh sorry, I had not realized that @jkbradley had asked you to remove the parameter. @jkbradley there is no performance issue doing that? --- If your project is set u

[GitHub] spark pull request: [SPARK-14793][SQL] Code generation for large c...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12559#issuecomment-213021325 @ueshin in the pr description can you elaborate the approach taken in this pr? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-14793][SQL] Code generation for large c...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12559#issuecomment-213021087 cc @davies and @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/12536#issuecomment-213020180 @BenFradet thanks for the PR. I have one comment about performance. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12536#discussion_r60618838 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -610,7 +604,9 @@ private[spark] object RandomForest extends Logging

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12536#discussion_r60618857 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -668,6 +664,7 @@ private[spark] object RandomForest extends Logging

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12536#discussion_r60618828 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -80,6 +80,7 @@ private[spark] object RandomForest extends Logging {

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12536#discussion_r60618750 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -94,17 +94,33 @@ final class DecisionTreeClassifier

[GitHub] spark pull request: [SPARK-14570][ML] Log instrumentation in Rando...

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12536#discussion_r60618432 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -94,17 +94,33 @@ final class DecisionTreeClassifier

[GitHub] spark pull request: [SPARK-14799][SQL] Remove MetastoreRelation de...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12566#issuecomment-213018745 cc @yhuai This one doesn't yet remove the dependency, but puts everything in one line. I think the next step is to make MetastoreRelation also a CatalogRelation.

[GitHub] spark pull request: [SPARK-4452][Core]Shuffle data structures can ...

2016-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10024 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14798][SQL] Move native command and scr...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12564#issuecomment-213017489 **[Test build #56557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56557/consoleFull)** for PR 12564 at commit [`a38eaf1`](https://gi

[GitHub] spark pull request: [SPARK-4452][Core]Shuffle data structures can ...

2016-04-21 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/10024#issuecomment-213017267 Merging this into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-14797] [BUILD] Spark SQL POM should not...

2016-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12563 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213016020 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213016016 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: Investigate flaky test

2016-04-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12565#discussion_r60617642 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala --- @@ -140,7 +140,7 @@ class HiveSqlAstBuilder(conf: SQLConf) extend

[GitHub] spark pull request: [SPARK-14763] [SQL] fix subquery resolution

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12539#issuecomment-213015476 **[Test build #56556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56556/consoleFull)** for PR 12539 at commit [`fd8c75c`](https://gi

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213015581 **[Test build #56547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56547/consoleFull)** for PR 12568 at commit [`1b9eaa8`](https://g

[GitHub] spark pull request: [SPARK-14762][SQL] TPCDS Q90 fails to parse

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12537#issuecomment-213015363 **[Test build #2845 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2845/consoleFull)** for PR 12537 at commit [`d48b19e`](https://g

[GitHub] spark pull request: [SPARK-14795][SQL] Remove the use of Hive's va...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12561#issuecomment-213015461 **[Test build #56555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56555/consoleFull)** for PR 12561 at commit [`f74b381`](https://gi

[GitHub] spark pull request: [SPARK-14801][SQL] Move MetastoreRelation to i...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12567#issuecomment-213015412 **[Test build #2846 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2846/consoleFull)** for PR 12567 at commit [`0a6bf1a`](https://g

[GitHub] spark pull request: [SPARK-14797] [BUILD] Spark SQL POM should not...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12563#issuecomment-213015390 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-10496][SQL] Add DataFrame cumulative su...

2016-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12578#issuecomment-213013699 Thanks for contributing, but isn't this similar to just window function? --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213011318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213011309 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213010394 **[Test build #56546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56546/consoleFull)** for PR 12568 at commit [`9a01db7`](https://g

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-21 Thread thunterdb
Github user thunterdb commented on the pull request: https://github.com/apache/spark/pull/12560#issuecomment-213009688 @wangmiao1981 thanks for the PR. Can you see if you could make it work without having to evaluate the input data? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12560#discussion_r60616196 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -395,13 +395,21 @@ class ALS(@Since("1.4.0") override val uid: String)

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-213008407 **[Test build #56554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56554/consoleFull)** for PR 12498 at commit [`5081e2c`](https://gi

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-21 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12498#discussion_r60615930 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/HashingTF.scala --- @@ -66,8 +83,9 @@ class HashingTF(val numFeatures: Int) extends Seriali

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-21 Thread thunterdb
Github user thunterdb commented on a diff in the pull request: https://github.com/apache/spark/pull/12560#discussion_r60615798 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -395,13 +395,21 @@ class ALS(@Since("1.4.0") override val uid: String)

[GitHub] spark pull request: [SPARK-14312] [ML] [SparkR] NaiveBayes model p...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12573#issuecomment-213006740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14312] [ML] [SparkR] NaiveBayes model p...

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12573#issuecomment-213006737 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14312] [ML] [SparkR] NaiveBayes model p...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12573#issuecomment-213006612 **[Test build #56552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56552/consoleFull)** for PR 12573 at commit [`c1df7d2`](https://g

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-213006241 **[Test build #56553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56553/consoleFull)** for PR 12500 at commit [`ec54d74`](https://gi

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-213003134 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-213003138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [MINOR][DOCS] Minor typo fixes

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12469#issuecomment-213002570 **[Test build #56548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56548/consoleFull)** for PR 12469 at commit [`8b64a1d`](https://g

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60613929 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -183,12 +190,7 @@ public Managed

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-213000250 Thanks for picking this up, @lianhuiwang I didn't get a chance to work on this yesterday, so I am very glad you are picked this up, thanks! Left some comments. ---

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60613362 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java --- @@ -186,7 +186,12 @@ public void

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-21 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/12568#discussion_r60613340 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java --- @@ -186,7 +186,12 @@ public void

[GitHub] spark pull request: [SPARK-14459] [SQL] Detect relation partitioni...

2016-04-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/12239#discussion_r60611716 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -259,4 +261,78 @@ class InsertIntoHiveTableSuite extends Qu

[GitHub] spark pull request: [SPARK-14459] [SQL] Detect relation partitioni...

2016-04-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/12239#discussion_r60611545 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -414,8 +414,42 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-14459] [SQL] Detect relation partitioni...

2016-04-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/12239#discussion_r60612218 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -414,8 +414,42 @@ class Analyzer( }

[GitHub] spark pull request: [SPARK-14558][CORE] In ClosureCleaner, clean t...

2016-04-21 Thread a-roberts
Github user a-roberts commented on the pull request: https://github.com/apache/spark/pull/12327#issuecomment-212991789 @cloud-fan I've had a closer look at this and think a more robust method would be to use weak references to identify when an object is out of scope, with IBM Java we

[GitHub] spark pull request: [SPARK-14433][PySpark][ML]:PySpark ml Gaussian...

2016-04-21 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12402#discussion_r60611229 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/GaussianMixture.scala --- @@ -105,6 +108,15 @@ class GaussianMixtureModel private[ml] (

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread sethah
Github user sethah commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-212990210 A couple minor syntax comments, other than that LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-212990133 @MLnick @sethah thanks for the reviews, will fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/12500#discussion_r60610289 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -92,21 +111,28 @@ object MLTestingUtils extends SparkFunSuite { t

[GitHub] spark pull request: [SPARK-14542][CORE] PipeRDD should allow confi...

2016-04-21 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/12309#discussion_r60610117 --- Diff: core/src/main/scala/org/apache/spark/rdd/PipedRDD.scala --- @@ -17,10 +17,7 @@ package org.apache.spark.rdd -import java.io.

[GitHub] spark pull request: [SPARK-14312] [ML] [SparkR] NaiveBayes model p...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12573#issuecomment-212988976 **[Test build #56552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56552/consoleFull)** for PR 12573 at commit [`c1df7d2`](https://gi

[GitHub] spark pull request: [SPARK-10496][SQL] Add DataFrame cumulative su...

2016-04-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12578#issuecomment-212987557 **[Test build #56551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56551/consoleFull)** for PR 12578 at commit [`97af903`](https://gi

<    4   5   6   7   8   9   10   11   12   13   >