[GitHub] spark issue #14093: SPARK-16420: Ensure compression streams are closed.

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14093 **[Test build #61916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61916/consoleFull)** for PR 14093 at commit

[GitHub] spark issue #14081: [SPARK-16403][Examples] Cleanup to remove unused imports...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14081 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14081: [SPARK-16403][Examples] Cleanup to remove unused imports...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61922/ Test PASSed. ---

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61919/ Test PASSed. ---

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11956 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14092: [SPARK-16419][SQL] EnsureRequirements adds extra Sort to...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14092 **[Test build #3169 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3169/consoleFull)** for PR 14092 at commit

[GitHub] spark issue #14092: [SPARK-16419][SQL] EnsureRequirements adds extra Sort to...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14092 **[Test build #3168 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3168/consoleFull)** for PR 14092 at commit

[GitHub] spark pull request #14090: [SPARK-16112][SparkR] Programming guide for gappl...

2016-07-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14090#discussion_r69955401 --- Diff: docs/sparkr.md --- @@ -306,6 +306,64 @@ head(ldf, 3) {% endhighlight %} + Run a given function on a large dataset

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61917/ Test FAILed. ---

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13680 **[Test build #61917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61917/consoleFull)** for PR 13680 at commit

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13894 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13894 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61921/ Test PASSed. ---

[GitHub] spark pull request #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` ...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13765#discussion_r69963613 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala --- @@ -370,8 +370,11 @@ package object dsl { case

[GitHub] spark issue #14083: [SPARK-16406][SQL] Improve performance of LogicalPlan.re...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14083 **[Test build #61923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61923/consoleFull)** for PR 14083 at commit

[GitHub] spark issue #14092: [SPARK-16419][SQL] EnsureRequirements adds extra Sort to...

2016-07-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14092 Ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13894 **[Test build #61921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61921/consoleFull)** for PR 13894 at commit

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14030 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11956 **[Test build #61919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61919/consoleFull)** for PR 11956 at commit

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61920/consoleFull)** for PR 14004 at commit

[GitHub] spark issue #14022: [SPARK-16272][core] Allow config values to reference con...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14022 **[Test build #61924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61924/consoleFull)** for PR 14022 at commit

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14030#discussion_r69855673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -155,7 +155,7 @@ private[sql] object Dataset { class Dataset[T] private[sql](

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14030#discussion_r69855749 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ForeachSinkSuite.scala --- @@ -35,35 +35,109 @@ class ForeachSinkSuite extends

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14030#discussion_r69855731 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ForeachSinkSuite.scala --- @@ -35,35 +35,109 @@ class ForeachSinkSuite extends

[GitHub] spark issue #14083: [SPARK-16406][SQL] Improve performance of LogicalPlan.re...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14083 I've got another idea (orthogonal to this). Thank you, @hvanhovell . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-07 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/13969 JavaReflectMethod isn't the best, but calling it Reflect is pretty bad because there are many "Reflect" classes in various libraries -- just within Spark dependencies there are 3 classes called

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13876 Thank you for review, @cloud-fan . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69856801 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -652,6 +654,145 @@ case class

[GitHub] spark issue #14065: [SPARK-14743][YARN][WIP] Add a configurable token manage...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #61896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61896/consoleFull)** for PR 14065 at commit

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61897/consoleFull)** for PR 13876 at commit

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-07 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r69856576 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JavaMethodReflectSuite.scala --- @@ -0,0 +1,101 @@ +/* + *

[GitHub] spark pull request #14004: [SPARK-16285][SQL] Implement sentences SQL functi...

2016-07-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14004#discussion_r69857049 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -198,6 +203,66 @@ case class

[GitHub] spark pull request #14004: [SPARK-16285][SQL] Implement sentences SQL functi...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14004#discussion_r69857442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -198,6 +203,66 @@ case class

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-07 Thread janplus
Github user janplus commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69857849 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -652,6 +654,160 @@ case class

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r69854777 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -820,16 +820,24 @@ object ConstantFolding

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r69855334 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -820,16 +820,24 @@ object ConstantFolding

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-07 Thread janplus
Github user janplus commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69855302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -652,6 +654,160 @@ case class

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer...

2016-07-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r69855189 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -820,16 +820,24 @@ object ConstantFolding extends

[GitHub] spark issue #14030: [SPARK-16350][SQL] Fix support for incremental planning ...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14030 **[Test build #61893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61893/consoleFull)** for PR 14030 at commit

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-07 Thread janplus
Github user janplus commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69855494 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -652,6 +654,145 @@ case class

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13876 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-07 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r69857166 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/JavaMethodReflect.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-07 Thread janplus
Github user janplus commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69857153 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -652,6 +654,145 @@ case class

[GitHub] spark issue #14030: [SPARK-16350][SQL] Fix support for incremental planning ...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14030 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14030#discussion_r69855694 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ForeachSink.scala --- @@ -30,7 +32,42 @@ import org.apache.spark.sql.{DataFrame,

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61898/consoleFull)** for PR 13876 at commit

[GitHub] spark issue #14030: [SPARK-16350][SQL] Fix support for incremental planning ...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61893/ Test PASSed. ---

[GitHub] spark pull request #14030: [SPARK-16350][SQL] Fix support for incremental pl...

2016-07-07 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14030#discussion_r69856084 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ForeachSink.scala --- @@ -30,7 +32,42 @@ import org.apache.spark.sql.{DataFrame,

[GitHub] spark issue #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creating Vie...

2016-07-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14047 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14084: [SPARK-16021][test-maven] Fix the maven build

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14084 **[Test build #61894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61894/consoleFull)** for PR 14084 at commit

[GitHub] spark issue #14085: [SPARK-16408][SQL] SparkSQL Added file get Exception: is...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14085 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14085: [SPARK-16408][SQL] SparkSQL Added file get Except...

2016-07-07 Thread zenglinxi0615
GitHub user zenglinxi0615 opened a pull request: https://github.com/apache/spark/pull/14085 [SPARK-16408][SQL] SparkSQL Added file get Exception: is a directory … ## What changes were proposed in this pull request? This PR is for adding an parameter (spark.input.dir.recursive)

[GitHub] spark pull request #14081: [SPARK-16403][Examples] Cleanup to remove unused ...

2016-07-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14081#discussion_r69862983 --- Diff: examples/src/main/python/ml/gaussian_mixture_example.py --- @@ -31,7 +31,7 @@ if __name__ == "__main__": spark = SparkSession\

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61902/consoleFull)** for PR 14045 at commit

[GitHub] spark issue #13778: [SPARK-16062][SPARK-15989][SQL] Fix two bugs of Python-o...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13778 **[Test build #61903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61903/consoleFull)** for PR 13778 at commit

[GitHub] spark issue #14068: enhanced simulate multiply

2016-07-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14068 I don't think this is trivial. You also need to explain the change in more detail. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14030: [SPARK-16350][SQL] Fix support for incremental planning ...

2016-07-07 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14030 Updated. @zsxwing could you take another look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14065: [SPARK-14743][YARN][WIP] Add a configurable token manage...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61896/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN][WIP] Add a configurable token manage...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13494 **[Test build #61906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61906/consoleFull)** for PR 13494 at commit

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-07 Thread lianhuiwang
Github user lianhuiwang commented on the issue: https://github.com/apache/spark/pull/13494 @cloud-fan thanks. I have rebased it to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14086: [SPARK-16410][SQL] Support `truncate` option in O...

2016-07-07 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/14086 [SPARK-16410][SQL] Support `truncate` option in Overwrite mode for JDBC DataFrameWriter ## What changes were proposed in this pull request? This PR adds a boolean option,

[GitHub] spark issue #14086: [SPARK-16410][SQL] Support `truncate` option in Overwrit...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14086 **[Test build #61907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61907/consoleFull)** for PR 14086 at commit

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61899/ Test PASSed. ---

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61899/consoleFull)** for PR 14004 at commit

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-07 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69860765 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -1689,4 +1689,76 @@ class SQLQuerySuite extends

[GitHub] spark pull request #13778: [SPARK-16062][SPARK-15989][SQL] Fix two bugs of P...

2016-07-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13778#discussion_r69863023 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -374,13 +407,15 @@ object MapObjects { *

[GitHub] spark pull request #14081: [SPARK-16403][Examples] Cleanup to remove unused ...

2016-07-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14081#discussion_r69863695 --- Diff: examples/src/main/java/org/apache/spark/examples/ml/JavaPipelineExample.java --- @@ -1,88 +0,0 @@ -/* - * Licensed to the Apache Software

[GitHub] spark pull request #14085: [SPARK-16408][SQL] SparkSQL Added file get Except...

2016-07-07 Thread zenglinxi0615
Github user zenglinxi0615 commented on a diff in the pull request: https://github.com/apache/spark/pull/14085#discussion_r69864435 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/commands.scala --- @@ -113,8 +113,9 @@ case class AddFile(path: String) extends

[GitHub] spark issue #13701: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-07-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13701 > It sounds like the invalid rows will still be filtered out during column assembly even if we are unable to prune any row group. If so, counting number of rows does not help us determine how many

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61897/ Test PASSed. ---

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61897/consoleFull)** for PR 13876 at commit

[GitHub] spark issue #14065: [SPARK-14743][YARN][WIP] Add a configurable token manage...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #61896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61896/consoleFull)** for PR 14065 at commit

[GitHub] spark issue #13861: [SPARK-15885] [Web UI] Provide links to executor logs fr...

2016-07-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13861 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61898/consoleFull)** for PR 13876 at commit

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61898/ Test FAILed. ---

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13876 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13876 There is one failure in `HiveSparkSubmitSuite`. It seems to be irrelevant. I'll retry the test. ``` - SPARK-8020: set sql conf in spark conf *** FAILED *** (31 seconds, 981

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61900/consoleFull)** for PR 14045 at commit

[GitHub] spark issue #14084: [SPARK-16021][test-maven] Fix the maven build

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14084 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve `OptimizeIn` optimizer to rem...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61901/consoleFull)** for PR 13876 at commit

[GitHub] spark issue #14084: [SPARK-16021][test-maven] Fix the maven build

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61894/ Test PASSed. ---

[GitHub] spark issue #13701: [SPARK-15639][SQL] Try to push down filter at RowGroups ...

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13701 **[Test build #61904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61904/consoleFull)** for PR 13701 at commit

[GitHub] spark pull request #14085: [SPARK-16408][SQL] SparkSQL Added file get Except...

2016-07-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14085#discussion_r69863303 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/commands.scala --- @@ -113,8 +113,9 @@ case class AddFile(path: String) extends

[GitHub] spark pull request #14085: [SPARK-16408][SQL] SparkSQL Added file get Except...

2016-07-07 Thread zenglinxi0615
Github user zenglinxi0615 commented on a diff in the pull request: https://github.com/apache/spark/pull/14085#discussion_r69865365 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/commands.scala --- @@ -113,8 +113,9 @@ case class AddFile(path: String) extends

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14004 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61895/ Test PASSed. ---

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-07 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/13969 Alright - I've updated it. The expression is now called Reflect, and it prints different messages depending on whether it is a class not found or a method not found. --- If your project is

[GitHub] spark pull request #13861: [SPARK-15885] [Web UI] Provide links to executor ...

2016-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13861 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14080: [SPARK-16405] [ShuffleService] [Metrics] Add metrics and...

2016-07-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14080 Can you update to the title to just say "[SPARK-16405] Add metrics and source for external shuffle service"? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creat...

2016-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14047 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61895/consoleFull)** for PR 14004 at commit

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61905/ Test FAILed. ---

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61899/consoleFull)** for PR 14004 at commit

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14088 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14089: [SPARK-16415][SQL] fix catalog string error

2016-07-07 Thread adrian-wang
GitHub user adrian-wang opened a pull request: https://github.com/apache/spark/pull/14089 [SPARK-16415][SQL] fix catalog string error ## What changes were proposed in this pull request? In #13537 we truncate `simpleString` if it is a long `StructType`. But sometimes we

[GitHub] spark pull request #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Vali...

2016-07-07 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/13894#discussion_r69892610 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -56,7 +56,10 @@ private[ml] trait CrossValidatorParams extends

<    1   2   3   4   5   6   >