[GitHub] spark issue #17561: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread shaolinliu
Github user shaolinliu commented on the issue: https://github.com/apache/spark/pull/17561 @ueshin please take a look at this pr, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17557 **[Test build #75593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75593/testReport)** for PR 17557 at commit [`30949a1`](https://github.com/apache/spark/commit/30

[GitHub] spark issue #17561: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17561 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter ...

2017-04-06 Thread shaolinliu
Github user shaolinliu closed the pull request at: https://github.com/apache/spark/pull/17560 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread shaolinliu
Github user shaolinliu commented on the issue: https://github.com/apache/spark/pull/17560 @ueshin i resubmit the pr, please close this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request #17561: [SPARK-20248][ SQL]Spark SQL add limit parameter ...

2017-04-06 Thread shaolinliu
GitHub user shaolinliu opened a pull request: https://github.com/apache/spark/pull/17561 [SPARK-20248][ SQL]Spark SQL add limit parameter to enhance the reliability. ## What changes were proposed in this pull request? Add a parameter "spark.sql.thriftServer.retainedResu

[GitHub] spark issue #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread shaolinliu
Github user shaolinliu commented on the issue: https://github.com/apache/spark/pull/17560 yes, i am fixing, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17560 @shaolinliu Can you fix conflicts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter to enha...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17560 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17552 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75587/ Test PASSed. ---

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17552 **[Test build #75587 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75587/testReport)** for PR 17552 at commit [`0fbd4a6`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #17560: [SPARK-20248][ SQL]Spark SQL add limit parameter ...

2017-04-06 Thread shaolinliu
GitHub user shaolinliu opened a pull request: https://github.com/apache/spark/pull/17560 [SPARK-20248][ SQL]Spark SQL add limit parameter to enhance the reliability. ## What changes were proposed in this pull request? Add a parameter "spark.sql.thriftServer.retainedResults" with

[GitHub] spark issue #17559: [SPARK-20246][SQL] Don't pushdown non-deterministic expr...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17559 **[Test build #75592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75592/testReport)** for PR 17559 at commit [`77896e9`](https://github.com/apache/spark/commit/77

[GitHub] spark pull request #17559: [SPARK-20246][SQL] Don't pushdown non-determinist...

2017-04-06 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/17559 [SPARK-20246][SQL] Don't pushdown non-deterministic expression through Aggregate ## What changes were proposed in this pull request? import org.apache.spark.sql.functions._ val

[GitHub] spark issue #17558: [SPARK-20247][CORE] Add jar but this jar is missing late...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17558 @wangyum what if the task requires that jar? From your fix what I got is that you catch the exception and make it warning log instead, but what if that task requires the jar, will you fix suppress

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317549 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -328,7 +329,7 @@ object PartitioningUtils {

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110298557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala --- @@ -396,7 +397,7 @@ object Partitioni

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -222,7 +225,7 @@ case class PreprocessTableCreation(spa

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317441 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -128,7 +128,8 @@ object PartitioningUtils {

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110314669 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringKeyHashMap.scala --- @@ -25,7 +27,7 @@ object StringKeyHashMap {

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110315394 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -82,8 +84,8 @@ case class OptimizeMetadataOnlyQ

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110314541 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CaseInsensitiveMap.scala --- @@ -26,11 +28,12 @@ package org.apache.spark.sql.

[GitHub] spark pull request #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java S...

2017-04-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17527#discussion_r110317272 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala --- @@ -52,7 +54,11 @@ case class HadoopFsRelation(

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -134,7 +132,7 @@ case class CostBased

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318621 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(wei

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/17516 Don't we also need the skip if cran statement ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17516 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17516 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75589/ Test PASSed. ---

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17516 **[Test build #75589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75589/testReport)** for PR 17516 at commit [`a3e8b35`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17557 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75588/ Test PASSed. ---

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17557 **[Test build #75588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75588/testReport)** for PR 17557 at commit [`27e94fd`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110318101 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorder

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17222 I'll try and follow up this weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17494 Thanks @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17494 LGTM as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75591/testReport)** for PR 17222 at commit [`4da2994`](https://github.com/apache/spark/commit/4d

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/17222 @viirya Thanks for careful review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17494 Thanks @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17222 LGTM, see if @marmbrus or @holdenk have any more comments about this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJav...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17222#discussion_r110316824 --- Diff: python/pyspark/sql/tests.py --- @@ -436,6 +436,20 @@ def test_udf_with_order_by_and_limit(self): res.explain(True) self.as

[GitHub] spark issue #17558: [SPARK-20247][CORE] Add jar but this jar is missing late...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17558 **[Test build #75590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75590/testReport)** for PR 17558 at commit [`de5b5fe`](https://github.com/apache/spark/commit/de

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17546 This looks pretty good over all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110316465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight =>

[GitHub] spark pull request #17558: [SPARK-20247][CORE] Add jar but this jar is missi...

2017-04-06 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/17558 [SPARK-20247][CORE] Add jar but this jar is missing later shouldn't affect jobs that doesn't use this jar ## What changes were proposed in this pull request? Catch exception when jar is mi

[GitHub] spark issue #17516: [SPARK-20197][SPARKR] CRAN check fail with package insta...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17516 **[Test build #75589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75589/testReport)** for PR 17516 at commit [`a3e8b35`](https://github.com/apache/spark/commit/a3

[GitHub] spark issue #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17557 **[Test build #75588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75588/testReport)** for PR 17557 at commit [`27e94fd`](https://github.com/apache/spark/commit/27

[GitHub] spark pull request #17557: [SPARK-20208][WIP][R][DOCS] Document R fpGrowth s...

2017-04-06 Thread zero323
GitHub user zero323 opened a pull request: https://github.com/apache/spark/pull/17557 [SPARK-20208][WIP][R][DOCS] Document R fpGrowth support ## What changes were proposed in this pull request? Document fpGrowth in: - vignettes - programming guide - code ex

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15770 Any update on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR...

2017-04-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17553#discussion_r110315204 --- Diff: examples/src/main/r/ml/glm.R --- @@ -56,6 +56,15 @@ summary(binomialGLM) # Prediction binomialPredictions <- predict(binomialGLM, bin

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314839 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* +

[GitHub] spark issue #17556: [SPARK-16957][MLlib] Use weighted midpoints for split va...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17556 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314588 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* +

[GitHub] spark pull request #17556: [SPARK-16957][MLlib] Use weighted midpoints for s...

2017-04-06 Thread facaiy
GitHub user facaiy opened a pull request: https://github.com/apache/spark/pull/17556 [SPARK-16957][MLlib] Use weighted midpoints for split values. ## What changes were proposed in this pull request? Use weighted midpoints for split values. ## How was this patch test

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110314369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(wei

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -54,14 +54,12 @@ case class CostBasedJoi

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313661 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -134,7 +132,7 @@ case class CostBasedJoi

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110313349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110312633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package org.apache.spark.sql.

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17552 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17552 **[Test build #75587 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75587/testReport)** for PR 17552 at commit [`0fbd4a6`](https://github.com/apache/spark/commit/0f

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110311641 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package org.apache.spark.sql.e

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110309359 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* + * Lice

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110309073 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/* + * Lice

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110308327 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight =>

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307898 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exte

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exte

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110307666 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exten

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110306486 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exte

[GitHub] spark pull request #17555: [SPARK-19495][SQL] Make SQLConf slightly more ext...

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17555 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17555 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110305903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exten

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75586/ Test PASSed. ---

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17555 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17555 **[Test build #75586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75586/testReport)** for PR 17555 at commit [`6084d95`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #17495: [SPARK-20172][Core] Add file permission check when listi...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17495 Ping @vanzin @tgravescs again. Sorry to bother you and really appreciate your time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 I see. The current code leverages `SparkListenerBlockUpdated` event to calculate memory usage, let me try to investigate the feasibility of using `taskEnd.taskMetrics.updatedBlocks`, to see if it

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14617 yeah, we definitely don't want to start logging more events. But it seems like this info is already available -- taskEnd.taskMetrics.updatedBlocks already has everything, doesn't it? --- If your p

[GitHub] spark pull request #17534: [SPARK-20218]'/applications/[app-id]/stages' in R...

2017-04-06 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/17534#discussion_r110303522 --- Diff: docs/monitoring.md --- @@ -299,12 +299,12 @@ can be identified by their `[attempt-id]`. In the API listed below, when running

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110303409 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exte

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110302895 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exten

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110300420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object JoinReorderDP exte

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 Thanks @squito . Regarding showing memory usage in history server. My major concern is that putting so many block update event into event log will significantly increase the file size and

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17551 @barnardb only in Spark standalone mode HistoryServer is embedded into Master process for convenience IIRC. You can always start a standalone HistoryServer process. Also `FsHistoryProvid

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17553 Could you add [SPARKR] to the PR title please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75585/ Test PASSed. ---

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16648 **[Test build #75585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75585/testReport)** for PR 16648 at commit [`320db91`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75584/ Test PASSed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #75584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75584/testReport)** for PR 15009 at commit [`0cfd4a7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #17555: [SPARK-19495][SQL] Make SQLConf slightly more extensible...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17555 **[Test build #75586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75586/testReport)** for PR 17555 at commit [`6084d95`](https://github.com/apache/spark/commit/60

[GitHub] spark pull request #17555: [SPARK-19495][SQL] Make SQLConf slightly more ext...

2017-04-06 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/17555 [SPARK-19495][SQL] Make SQLConf slightly more extensible - addendum ## What changes were proposed in this pull request? This is a tiny addendum to SPARK-19495 to remove the private visibility for

[GitHub] spark pull request #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17554 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17554: [MINOR][DOCS] Fix typo in Hive Examples

2017-04-06 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17554 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17552 LGTM except only one comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110287008 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -18,39 +18,21 @@ package org.apache.spark.sql.

[GitHub] spark issue #17092: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-04-06 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/17092 Ping. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

  1   2   3   >