[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...

2015-12-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10488#discussion_r48479722 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -57,9 +57,10 @@ case class Md5(child: Expression

[GitHub] spark pull request: [SPARK-8641][SPARK-12455][SQL] Native Spark Wi...

2015-12-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10402#discussion_r48495956 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -327,9 +327,9 @@ abstract class

[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...

2015-12-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10488#discussion_r48500579 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -57,9 +57,10 @@ case class Md5(child: Expression

[GitHub] spark pull request: [SPARK-8641][SPARK-12455][SQL] Native Spark Wi...

2015-12-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10402#discussion_r48496502 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -327,9 +327,9 @@ abstract class

[GitHub] spark pull request: [SPARK-12010][SQL] Spark JDBC requires support...

2015-12-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10380#discussion_r48135557 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -60,20 +60,6 @@ object JdbcUtils extends

[GitHub] spark pull request: [SPARK-12421][SQL] Fixed copy() method of Gene...

2015-12-22 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10374#issuecomment-166556640 @Apo1 Could you add a unit test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-12-20 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/9819#issuecomment-166118773 @yhuai & @davies thanks for the reviews! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-12-20 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10402#issuecomment-166127579 jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-12-20 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/10402 [SPARK-8641][SQL] Native Spark Window functions - Follow-up (docs & tests) This PR is a follow-up for PR https://github.com/apache/spark/pull/9819. It adds documentation for the wi

[GitHub] spark pull request: [SPARK-12437][SQL] Encapsulate the table and c...

2015-12-20 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10403#issuecomment-166148352 @naveenminchu I tried triggering a build, but it seems that I do not have sufficient rights for this (I do have them for my own PRs). --- If your project is set up

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-29 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-167947021 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/10525 [SPARK-12362][SQL][WIP] Inline Hive Parser This PR inlines the Hive SQL parser in Spark SQL. The previous (merged) incarnation of this PR passed all tests, but had and still has

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-167984272 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10521#issuecomment-168003040 @maropu you might need to merge the latest master. My PR (https://github.com/apache/spark/pull/10509) non-deterministically broke the build, and has since been

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-168004557 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-168010422 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-168012049 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-168013255 last one promise :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-30 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10525#issuecomment-168013202 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-12374][SPARK-12150][SQL] Adding logical...

2015-12-17 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/10335#discussion_r47952489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala --- @@ -126,6 +127,69 @@ case class Sample

[GitHub] spark pull request: [SPARK-12362][SQL][WIP] Inline Hive Parser

2015-12-29 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/10509 [SPARK-12362][SQL][WIP] Inline Hive Parser This is a WIP. The PR has been taken over from @nongli (see https://github.com/apache/spark/pull/10420). I have remove some additional deadcode

[GitHub] spark pull request: [SPARK-12024][SQL] More efficient multi-column...

2015-11-27 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/10015 [SPARK-12024][SQL] More efficient multi-column counting. In https://github.com/apache/spark/pull/9409 we enabled multi-column counting. The approach taken in that PR introduces a bit

[GitHub] spark pull request: [SPARK-7286] [SQL] Deprecate !== in favour of ...

2015-11-24 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/9925#issuecomment-159366507 Why not use the more common ```<>``` symbol instead ```=!=```? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-11990][SQL] Don't collapse projections ...

2015-11-26 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/9993#issuecomment-159868802 I wonder if this is an actuall improvement. Doing back to back projections also incur a codegen and a runtime cost; which can easily be higher than the cost

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45270291 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -187,7 +184,7 @@ sealed abstract class

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45270377 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -328,3 +280,208 @@ object

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45271240 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -25,7 +25,7 @@ import scala.collection.mutable.ArrayBuffer

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-18 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/9819 [SPARK-8641][SQL] Native Spark Window functions This PR removes Hive windows functions from Spark and replaces them with (native) Spark ones. The PR is on par with Hive in terms of features

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-23 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45659284 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -328,3 +281,223 @@ object

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-23 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45658962 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -866,26 +874,37 @@ class Analyzer

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-23 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/9819#discussion_r45659112 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -328,3 +281,223 @@ object

[GitHub] spark pull request: [SPARK-12745] [SQL] Hive Parser: Limit is not ...

2016-01-11 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10689#issuecomment-170473951 @gatorsmile the fix looks good. @rxin / @marmbrus / @gatorsmile I am not sure if we should support this at all. Using a limit in SELECT's connected

[GitHub] spark pull request #13472: [SPARK-15735] Allow specifying min time to run in...

2016-06-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13472#discussion_r65619857 --- Diff: core/src/main/scala/org/apache/spark/util/Benchmark.scala --- @@ -97,6 +111,39 @@ private[spark] class Benchmark( println

[GitHub] spark pull request #13472: [SPARK-15735] Allow specifying min time to run in...

2016-06-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13472#discussion_r65620010 --- Diff: core/src/main/scala/org/apache/spark/util/Benchmark.scala --- @@ -97,6 +111,39 @@ private[spark] class Benchmark( println

[GitHub] spark issue #13447: [SPARK-15706] [SQL] Fix Wrong Answer when using IF NOT E...

2016-06-02 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13447 @gatorsmile the overall approach seems good to me. We are currently fixing this for the SQL codepath. I was wondering if are other codepaths that can cause this unwanted behavior

[GitHub] spark issue #13418: [SPARK-15677][SQL] Query with scalar sub-query in the SE...

2016-06-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13418 whoops triggered a build on this one... sorry about that --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13498 Seems to be related to the sequence in which tests are executed... Perhaps `src` is changed during tests. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #13566: [SPARK-15678] Add support to REFRESH data source paths

2016-06-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13566 Looks pretty good. Left one comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...

2016-06-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 @ioana-delaney great catch! The overall PR seems pretty solid. I left one smallish code organization related comment. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #13472: [SPARK-15735] Allow specifying min time to run in microb...

2016-06-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13472 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13472: [SPARK-15735] Allow specifying min time to run in microb...

2016-06-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13472 Merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate s...

2016-06-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13570#discussion_r66364271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1715,31 +1715,68 @@ object

[GitHub] spark issue #13550: SPARK-15755

2016-06-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13550 @marymwu this has been fixed in https://github.com/apache/spark/commit/09b3c56c91831b3e8d909521b8f3ffbce4eb0395. Could you close this PR? --- If your project is set up for it, you can

[GitHub] spark pull request #13534: [SPARK-15789][SQL] Allow reserved keywords in mos...

2016-06-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13534#discussion_r66161676 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -594,16 +594,13 @@ qualifiedName : identifier

[GitHub] spark issue #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE US...

2016-06-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13414 Thanks! Merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13534: [SPARK-15789][SQL] Allow reserved keywords in mos...

2016-06-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13534#discussion_r66067691 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -594,16 +594,13 @@ qualifiedName : identifier

[GitHub] spark pull request #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY T...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13414#discussion_r65966463 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -90,6 +90,8 @@ statement identifierCommentList

[GitHub] spark issue #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE US...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13414 @clockfly this looks pretty good. I have left some (minor) comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #13534: [SPARK-15789][SQL] Allow reserved keywords in mos...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13534#discussion_r65975203 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -106,9 +106,9 @@ statement | SHOW FUNCTIONS (LIKE

[GitHub] spark issue #13534: [SPARK-15789][SQL] Allow reserved keywords in most place...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13534 cc @yhuai @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #13534: [SPARK-15789][SQL] Allow reserved keywords in mos...

2016-06-06 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13534 [SPARK-15789][SQL] Allow reserved keywords in most places ## What changes were proposed in this pull request? The parser currently does not allow the use of some SQL keywords as table

[GitHub] spark pull request #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY T...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13414#discussion_r65967765 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -375,9 +375,12 @@ private[sql] abstract class

[GitHub] spark pull request #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY T...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13414#discussion_r65967943 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -115,6 +115,10 @@ class DDLSuite extends QueryTest

[GitHub] spark pull request #13534: [SPARK-15789][SQL] Allow reserved keywords in mos...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13534#discussion_r65975425 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -453,8 +453,8 @@ expression

[GitHub] spark pull request #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY T...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13414#discussion_r65966717 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -344,6 +344,19 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark pull request #13414: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY T...

2016-06-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13414#discussion_r65966588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -344,6 +344,19 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark issue #10706: [SPARK-12543] [SPARK-4226] [SQL] Subquery in expression

2016-06-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/10706 @kamalcoursera you could use a predicate scalar subquery here, i.e.: ```sql select runon as runon case when (select max(true) from sqltesttable b where b.key

[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...

2016-06-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 LGTM - merging to master/2.0 thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScalarSubque...

2016-06-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13155 LGTM - merging to master/2.0 (will do a small follow-up) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13626: [SPARK-15370][SQL] Revert PR "Update RewriteCorre...

2016-06-12 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13626 [SPARK-15370][SQL] Revert PR "Update RewriteCorrelatedSuquery rule" This reverts commit 9770f6ee60f6834e4e1200234109120427a5cc0d. You can merge this pull request into a Git

[GitHub] spark pull request #13629: [SPARK-15370][SQL] Fix count bug

2016-06-12 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13629 [SPARK-15370][SQL] Fix count bug # What changes were proposed in this pull request? This pull request fixes the COUNT bug in the `RewriteCorrelatedScalarSubquery` rule. After

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66680913 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66680885 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66680599 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66680557 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66682052 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66682369 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark issue #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScalarSubque...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13155 @frreiss PR looks good. It does need a little bit of work in the style department. I'll leave comments. Lemme know if you have time to address these; else I'll take over and do that chore

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66680828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66681029 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66681873 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1696,205 @@ object

[GitHub] spark pull request #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate s...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13570#discussion_r66697790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1715,31 +1715,52 @@ object

[GitHub] spark pull request #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate s...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13570#discussion_r66697830 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1715,31 +1715,52 @@ object

[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 @ioana-delaney no worries. I think the approach you have taken is the correct one. I have left one smallish comment. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #13581: [SPARK-14321][SQL] Reduce date format cost and string-to...

2016-06-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13581 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13581: [SPARK-14321][SQL] Reduce date format cost and st...

2016-06-09 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13581 [SPARK-14321][SQL] Reduce date format cost and string-to-date cost in date functions ## What changes were proposed in this pull request? The current implementations of `UnixTime

[GitHub] spark pull request #13155: [SPARK-15370] [SQL] Update RewriteCorrelatedScala...

2016-06-09 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13155#discussion_r66540105 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1695,16 +1695,176 @@ object

[GitHub] spark pull request #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/...

2016-06-09 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13589 [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/Invalid results ## What changes were proposed in this pull request? I'll add desc later ## How was this patch tested? TBD

[GitHub] spark issue #13582: [SPARK-15850][SQL] Remove function grouping in SparkSess...

2016-06-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13582 LGTM - merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13572: [SPARK-15838] [SQL] CACHE TABLE AS SELECT should not rep...

2016-06-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13572 LGTM @liancheng anything to add? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13498 Ok I suspect that not all tables are created in the same way. Added some code to dump the table metadata. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRe...

2016-06-03 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13498 [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations' in hive StatisticsSuite [WIP] ## What changes were proposed in this pull request? This test re-enables the `analyze

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13498 I'll trigger a bunch of tests and see what happens. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-15436][SQL] Remove DescribeFunction and...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13292#issuecomment-221644166 merging to master & 2.0 thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT p...

2016-05-25 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13299 [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT plugin ## What changes were proposed in this pull request? The ANTLR4 SBT plugin has been moved from its own repo to one on bintray. The version

[GitHub] spark pull request: [SPARK-15534][SPARK-15535][SQL] Truncate table...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13302#discussion_r64652459 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -288,9 +288,10 @@ case class TruncateTableCommand

[GitHub] spark pull request: [SPARK-15534][SPARK-15535][SQL] Truncate table...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13302#issuecomment-221706431 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-15525][SQL][BUILD] Upgrade ANTLR4 SBT p...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13299#issuecomment-221651230 cc @rxin @MLnick @vanzin (could you take a look at the build) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-15538][SQL] Adding error check for trun...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13305#issuecomment-221724543 @sureshthalamati Yeah you are right. Sorry for jumping the gun. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-15538][SQL] Adding error check for trun...

2016-05-25 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/13305#issuecomment-221719109 @sureshthalamati I think https://github.com/apache/spark/pull/13302 already takes care of this. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #13589: [SPARK-15822][SPARK-15825][SQL] Fix SMJ Segfault/...

2016-06-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13589#discussion_r66615434 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -490,6 +490,7 @@ class CodegenContext

[GitHub] spark issue #13553: [SPARK-15814][SQL] Aggregator can return null result

2016-06-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13553 LGTM - merging to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13524: [SPARK-15776][SQL] Type coercion incorrect

2016-06-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13524#discussion_r66834271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -52,10 +52,10 @@ object TypeCoercion

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13498 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13666: [SPARK-15934] [SQL] Return binary mode in ThriftServer

2016-06-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13666 @epahomov in this case you can just open against master (the branches haven't diverged much yet). --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #13678: [SPARK-15824][SQL] Execute WITH .... INSERT ... s...

2016-06-14 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13678 [SPARK-15824][SQL] Execute WITH INSERT ... statements immediately ## What changes were proposed in this pull request? We currently immediately execute `INSERT` commands when

[GitHub] spark issue #13678: [SPARK-15824][SQL] Execute WITH .... INSERT ... statemen...

2016-06-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13678 cc @Sephiroth-Lin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13678: [SPARK-15824][SQL] Execute WITH .... INSERT ... s...

2016-06-15 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13678#discussion_r67104025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -179,15 +179,16 @@ class Dataset[T] private[sql]( case _ => fa

[GitHub] spark pull request #13681: [SPARK-15960][SQL] Rename `spark.sql.enableFallBa...

2016-06-15 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13681 [SPARK-15960][SQL] Rename `spark.sql.enableFallBackToHdfsForStats` config ## What changes were proposed in this pull request? Since we are probably going to add more statistics related

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66880904 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left

<    1   2   3   4   5   6   7   8   9   10   >