spark git commit: [SPARK-15735] Allow specifying min time to run in microbenchmarks

ode defaultNumIters now override minNumIters. cc hvanhovell Author: Eric Liang <e...@databricks.com> Author: Eric Liang <ekhli...@gmail.com> Closes #13472 from ericl/spark-15735. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/a

spark git commit: [SPARK-15735] Allow specifying min time to run in microbenchmarks

sly overrode defaultNumIters now override minNumIters. cc hvanhovell Author: Eric Liang <e...@databricks.com> Author: Eric Liang <ekhli...@gmail.com> Closes #13472 from ericl/spark-15735. (cherry picked from commit 4e8ac6edd5808ca8245b39d804c6d4f5ea9d0d36) Signed-off-by: Herman van

spark git commit: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE USING...", uses "CREAT TEMPORARY VIEW USING..." instead

Repository: spark Updated Branches: refs/heads/master 200f01c8f -> 890baaca5 [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE USING...", uses "CREAT TEMPORARY VIEW USING..." instead ## What changes were proposed in this pull request? The current implementation of "CREATE TEMPORARY

spark git commit: [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE USING...", uses "CREAT TEMPORARY VIEW USING..." instead

Repository: spark Updated Branches: refs/heads/branch-2.0 7a7ec0547 -> 9e16f23e7 [SPARK-15674][SQL] Deprecates "CREATE TEMPORARY TABLE USING...", uses "CREAT TEMPORARY VIEW USING..." instead ## What changes were proposed in this pull request? The current implementation of "CREATE TEMPORARY

spark git commit: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery throws TreeNodeException

Repository: spark Updated Branches: refs/heads/master 9770f6ee6 -> 0ff8a68b9 [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery throws TreeNodeException ## What changes were proposed in this pull request? Queries with embedded existential sub-query predicates throws exception when

spark git commit: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery throws TreeNodeException

Repository: spark Updated Branches: refs/heads/branch-2.0 8e371670f -> b699a7b81 [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery throws TreeNodeException ## What changes were proposed in this pull request? Queries with embedded existential sub-query predicates throws exception when

spark git commit: [SPARK-15370][SQL] Revert PR "Update RewriteCorrelatedSuquery rule"

Closes #13626 from hvanhovell/SPARK-15370-revert. (cherry picked from commit 20b8f2c32af696c3856221c4c4fcd12c3f068af2) Signed-off-by: Herman van Hovell <hvanhov...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/c

spark git commit: [SPARK-15370][SQL] Update RewriteCorrelatedScalarSubquery rule to fix COUNT bug

Repository: spark Updated Branches: refs/heads/master 0a6f09083 -> 9770f6ee6 [SPARK-15370][SQL] Update RewriteCorrelatedScalarSubquery rule to fix COUNT bug ## What changes were proposed in this pull request? This pull request fixes the COUNT bug in the `RewriteCorrelatedScalarSubquery`

spark git commit: [SPARK-15370][SQL] Revert PR "Update RewriteCorrelatedSuquery rule"

Repository: spark Updated Branches: refs/heads/master e3554605b -> 20b8f2c32 [SPARK-15370][SQL] Revert PR "Update RewriteCorrelatedSuquery rule" This reverts commit 9770f6ee60f6834e4e1200234109120427a5cc0d. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #13

spark git commit: [SPARK-15850][SQL] Remove function grouping in SparkSession

Repository: spark Updated Branches: refs/heads/master 4d9d9cc58 -> 16df133d7 [SPARK-15850][SQL] Remove function grouping in SparkSession ## What changes were proposed in this pull request? SparkSession does not have that many functions due to better namespacing, and as a result we probably

spark git commit: [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions

Repository: spark Updated Branches: refs/heads/branch-2.0 1dad1a891 -> f63ba2210 [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions ## What changes were proposed in this pull request? This patch removes the last two commands defined in the catalyst module: DescribeFunction and

spark git commit: [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions

Repository: spark Updated Branches: refs/heads/master 9082b7968 -> 4f27b8dd5 [SPARK-15436][SQL] Remove DescribeFunction and ShowFunctions ## What changes were proposed in this pull request? This patch removes the last two commands defined in the catalyst module: DescribeFunction and

spark git commit: [SPARK-15814][SQL] Aggregator can return null result

Repository: spark Updated Branches: refs/heads/master d681742b2 -> cd47e2337 [SPARK-15814][SQL] Aggregator can return null result ## What changes were proposed in this pull request? It's similar to the bug fixed in https://github.com/apache/spark/pull/13425, we should consider null object

spark git commit: [SPARK-15814][SQL] Aggregator can return null result

Repository: spark Updated Branches: refs/heads/branch-2.0 41f309bfb -> d64cb1994 [SPARK-15814][SQL] Aggregator can return null result ## What changes were proposed in this pull request? It's similar to the bug fixed in https://github.com/apache/spark/pull/13425, we should consider null

spark git commit: [SPARK-15956][SQL] Revert "[] When unwrapping ORC avoid pattern matching…

: Herman van Hovell <hvanhov...@databricks.com> Closes #13853 from hvanhovell/SPARK-15956-revert. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/472d611a Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/472d611a Di

spark git commit: [SPARK-15977][SQL] Fix TRUNCATE TABLE for Spark specific datasource tables

csv, ...). This PR correctly sets the location for these datasources which allows them to be truncated. ## How was this patch tested? Extended the datasources `TRUNCATE TABLE` tests in `DDLSuite`. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #13697 from hvanhovell/SPARK-15977.

spark git commit: [SPARK-15977][SQL] Fix TRUNCATE TABLE for Spark specific datasource tables

csv, ...). This PR correctly sets the location for these datasources which allows them to be truncated. ## How was this patch tested? Extended the datasources `TRUNCATE TABLE` tests in `DDLSuite`. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #13697 from hvanhovell/SP

spark git commit: [SPARK-15956][SQL] When unwrapping ORC avoid pattern matching at runtime

~15%. Author: Brian Cho <b...@fb.com> Author: Herman van Hovell <hvanhov...@databricks.com> Closes #13854 from hvanhovell/SPARK-15956-scala210. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4f869f88 Tree:

spark git commit: [SPARK-16078][SQL] from_utc_timestamp/to_utc_timestamp should not depends on local timezone

Repository: spark Updated Branches: refs/heads/master 43b04b7ec -> 20d411bc5 [SPARK-16078][SQL] from_utc_timestamp/to_utc_timestamp should not depends on local timezone ## What changes were proposed in this pull request? Currently, we use local timezone to parse or format a timestamp

spark git commit: [SPARK-16078][SQL] from_utc_timestamp/to_utc_timestamp should not depends on local timezone

Repository: spark Updated Branches: refs/heads/branch-2.0 299f427b7 -> 282a3cd02 [SPARK-16078][SQL] from_utc_timestamp/to_utc_timestamp should not depends on local timezone ## What changes were proposed in this pull request? Currently, we use local timezone to parse or format a timestamp

spark git commit: Remove non-obvious conf settings from TPCDS benchmark

Repository: spark Updated Branches: refs/heads/master ef43b4ed8 -> 34d6c4cd1 Remove non-obvious conf settings from TPCDS benchmark ## What changes were proposed in this pull request? My fault -- these 2 conf entries are mysteriously hidden inside the benchmark code and makes it non-obvious

spark git commit: [SPARK-16123] Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader

Repository: spark Updated Branches: refs/heads/branch-2.0 e8d1bf60a -> edd5dd24a [SPARK-16123] Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader ## What changes were proposed in this pull request? This patch fixes an overflow bug in vectorized

spark git commit: [SPARK-16123] Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader

Repository: spark Updated Branches: refs/heads/master 264bc6362 -> cc71d4fa3 [SPARK-16123] Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader ## What changes were proposed in this pull request? This patch fixes an overflow bug in vectorized

spark git commit: [SQL][MINOR] ParserUtils.operationNotAllowed should throw exception directly

Repository: spark Updated Branches: refs/heads/branch-2.0 edd5dd24a -> a6edec2c5 [SQL][MINOR] ParserUtils.operationNotAllowed should throw exception directly ## What changes were proposed in this pull request? It's weird that `ParserUtils.operationNotAllowed` returns an exception and the

spark git commit: [SQL][MINOR] ParserUtils.operationNotAllowed should throw exception directly

Repository: spark Updated Branches: refs/heads/master cc71d4fa3 -> 6a3c6276f [SQL][MINOR] ParserUtils.operationNotAllowed should throw exception directly ## What changes were proposed in this pull request? It's weird that `ParserUtils.operationNotAllowed` returns an exception and the caller

spark git commit: [SPARK-16220][SQL] Revert Change to Bring Back SHOW FUNCTIONS Functionality

Repository: spark Updated Branches: refs/heads/master 3e4e868c8 -> c48c8ebc0 [SPARK-16220][SQL] Revert Change to Bring Back SHOW FUNCTIONS Functionality ## What changes were proposed in this pull request? - Fix tests regarding show functions functionality - Revert `catalog.ListFunctions` and

spark git commit: [SPARK-16220][SQL] Revert Change to Bring Back SHOW FUNCTIONS Functionality

Repository: spark Updated Branches: refs/heads/branch-2.0 22fe336c8 -> f2017c59b [SPARK-16220][SQL] Revert Change to Bring Back SHOW FUNCTIONS Functionality ## What changes were proposed in this pull request? - Fix tests regarding show functions functionality - Revert `catalog.ListFunctions`

spark git commit: [SPARK-16195][SQL] Allow users to specify empty over clause in window expressions through dataset API

Repository: spark Updated Branches: refs/heads/branch-2.0 9de095513 -> 9e2384845 [SPARK-16195][SQL] Allow users to specify empty over clause in window expressions through dataset API ## What changes were proposed in this pull request? Allow to specify empty over clause in window expressions

spark git commit: [SPARK-16195][SQL] Allow users to specify empty over clause in window expressions through dataset API

Repository: spark Updated Branches: refs/heads/master e5d0928e2 -> 9053054c7 [SPARK-16195][SQL] Allow users to specify empty over clause in window expressions through dataset API ## What changes were proposed in this pull request? Allow to specify empty over clause in window expressions

spark git commit: [SPARK-16192][SQL] Add type checks in CollectSet

Repository: spark Updated Branches: refs/heads/master 9053054c7 -> d2e44d7db [SPARK-16192][SQL] Add type checks in CollectSet ## What changes were proposed in this pull request? `CollectSet` cannot have map-typed data because MapTypeData does not implement `equals`. So, this pr is to add

spark git commit: [SPARK-16192][SQL] Add type checks in CollectSet

Repository: spark Updated Branches: refs/heads/branch-2.0 9e2384845 -> d079b5de7 [SPARK-16192][SQL] Add type checks in CollectSet ## What changes were proposed in this pull request? `CollectSet` cannot have map-typed data because MapTypeData does not implement `equals`. So, this pr is to add

spark git commit: [SPARK-13235][SQL] Removed an Extra Distinct from the Plan when Using Union in SQL

Repository: spark Updated Branches: refs/heads/master 1842c55d8 -> e88bff127 [SPARK-13235][SQL] Removed an Extra Distinct from the Plan when Using Union in SQL Currently, the parser added two `Distinct` operators in the plan if we are using `Union` or `Union Distinct` in the SQL. This PR is

spark git commit: [SPARK-12982][SQL] Add table name validation in temp table registration

Repository: spark Updated Branches: refs/heads/master e31c80737 -> 0d50a2208 [SPARK-12982][SQL] Add table name validation in temp table registration Add the table name validation at the temp table creation Author: jayadevanmurali Closes #11051 from

spark git commit: [SPARK-13277][SQL] ANTLR ignores other rule using the USING keyword

Repository: spark Updated Branches: refs/heads/master 219a74a7c -> e31c80737 [SPARK-13277][SQL] ANTLR ignores other rule using the USING keyword JIRA: https://issues.apache.org/jira/browse/SPARK-13277 There is an ANTLR warning during compilation: warning(200):

spark git commit: [SPARK-14396][BUILD][HOT] Fix compilation against Scala 2.10

Repository: spark Updated Branches: refs/heads/master 78c1076d0 -> 25a4c8e0c [SPARK-14396][BUILD][HOT] Fix compilation against Scala 2.10 What changes were proposed in this pull request? This PR is to fix the compilation errors in Scala 2.10 build, as shown in the link:

spark git commit: [SPARK-13325][SQL] Create a 64-bit hashcode expression

f.show() val ns = System.nanoTime() - t // Before ns: Long = 5821524302 // After ns: Long = 2836418963 cc cloud-fan (you have been working on hashcodes) / rxin Author: Herman van Hovell <hvanhov...@questtec.nl> Closes #11209 from hvanhovell/xxHash. Project: http://git-wi

spark git commit: [SPARK-13535][SQL] Fix Analysis Exceptions when Using Backticks in Transform Clause

Repository: spark Updated Branches: refs/heads/master d6969ffc0 -> 9e01fe2ed [SPARK-13535][SQL] Fix Analysis Exceptions when Using Backticks in Transform Clause What changes were proposed in this pull request? ```SQL FROM (FROM test SELECT TRANSFORM(key, value) USING 'cat' AS (`thing1`

spark git commit: [SPARK-14184][SQL] Support native execution of SHOW DATABASE command and fix SHOW TABLE to use table identifier pattern

Repository: spark Updated Branches: refs/heads/master 3715ecdf4 -> 0b04f8fdf [SPARK-14184][SQL] Support native execution of SHOW DATABASE command and fix SHOW TABLE to use table identifier pattern ## What changes were proposed in this pull request? This PR addresses the following 1.

spark git commit: [SPARK-14350][SQL] EXPLAIN output should be in a single cell

Repository: spark Updated Branches: refs/heads/master 2262a9335 -> 1f0c5dceb [SPARK-14350][SQL] EXPLAIN output should be in a single cell ## What changes were proposed in this pull request? EXPLAIN output should be in a single cell. **Before** ``` scala> sql("explain select 1").collect()

spark git commit: [SPARK-14341][SQL] Throw exception on unsupported create / drop macro ddl

Repository: spark Updated Branches: refs/heads/master 1f0c5dceb -> c238cd074 [SPARK-14341][SQL] Throw exception on unsupported create / drop macro ddl ## What changes were proposed in this pull request? We throw an AnalysisException that looks like this: ``` scala> sqlContext.sql("CREATE

spark git commit: [SPARK-14360][SQL] QueryExecution.debug.codegen() to dump codegen

Repository: spark Updated Branches: refs/heads/master 76f3c735a -> 0340b3d27 [SPARK-14360][SQL] QueryExecution.debug.codegen() to dump codegen ## What changes were proposed in this pull request? We recently added the ability to dump the generated code for a given query. However, the method

spark git commit: [SPARK-14348][SQL] Support native execution of SHOW TBLPROPERTIES command

Repository: spark Updated Branches: refs/heads/master 064623014 -> 2715bc68b [SPARK-14348][SQL] Support native execution of SHOW TBLPROPERTIES command ## What changes were proposed in this pull request? This PR adds Native execution of SHOW TBLPROPERTIES command. Command Syntax: ``` SQL

spark git commit: [SPARK-14349][SQL] Issue Error Messages for Unsupported Operators/DML/DDL in SQL Context.

orm` in SQL Context, we got the message: ``` assertion failed: No plan for ScriptTransformation [key#9,value#10], cat, [tKey#155,tValue#156], null +- LogicalRDD [key#9,value#10], MapPartitionsRDD[3] at beforeAll at BeforeAndAfterAll.scala:187 ``` Updates: Based on the investigation from hvanhov

spark git commit: [SPARK-14691][SQL] Simplify and Unify Error Generation for Unsupported Alter Table DDL

Repository: spark Updated Branches: refs/heads/master 8df8a8182 -> 337289d71 [SPARK-14691][SQL] Simplify and Unify Error Generation for Unsupported Alter Table DDL What changes were proposed in this pull request? So far, we are capturing each unsupported Alter Table in separate visit

spark git commit: [SPARK-13739][SQL] Push Predicate Through Window

Repository: spark Updated Branches: refs/heads/master 3c5e65c33 -> 0c47e274a [SPARK-13739][SQL] Push Predicate Through Window What changes were proposed in this pull request? For performance, predicates can be pushed through Window if and only if the following conditions are satisfied:

spark git commit: [SPARK-14950][SQL] Fix BroadcastHashJoin's unique key Anti-Joins

emi Join results . This PR fixes this bug. ### How was this patch tested? Added tests cases to `ExistenceJoinSuite`. cc davies gatorsmile Author: Herman van Hovell <hvanhov...@questtec.nl> Closes #12730 from hvanhovell/SPARK-14950. Project: http://git-wip-us.apache.org/repos/asf/spark/re

spark git commit: [SPARK-14664][SQL] Implement DecimalAggregates optimization for Window queries

( ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)#47] +- Exchange SinglePartition, None +- Generate explode([1.0,2.0]), false, false, [a#45] +- Scan OneRowRelation[] ``` In this PR, *SUM over window* pattern matching is based on the code of hvanhovell ; he should be cre

spark git commit: [SPARK-14910][SQL] Native DDL Command Support for Describe Function in Non-identifier Format

Repository: spark Updated Branches: refs/heads/master b208229ba -> 162cf02ef [SPARK-14910][SQL] Native DDL Command Support for Describe Function in Non-identifier Format What changes were proposed in this pull request? The existing `Describe Function` only support the function name in

spark git commit: [SPARK-14763][SQL] fix subquery resolution

Repository: spark Updated Branches: refs/heads/master d060da098 -> c417cec06 [SPARK-14763][SQL] fix subquery resolution ## What changes were proposed in this pull request? Currently, a column could be resolved wrongly if there are columns from both outer table and subquery have the same

spark git commit: [SPARK-15056][SQL] Parse Unsupported Sampling Syntax and Issue Better Exceptions

Repository: spark Updated Branches: refs/heads/branch-2.0 f21eddb29 -> 70c5ca220 [SPARK-15056][SQL] Parse Unsupported Sampling Syntax and Issue Better Exceptions What changes were proposed in this pull request? Compared with the current Spark parser, there are two extra syntax are

spark git commit: [SPARK-15215][SQL] Fix Explain Parsing and Output

Repository: spark Updated Branches: refs/heads/master f45379173 -> 570647267 [SPARK-15215][SQL] Fix Explain Parsing and Output What changes were proposed in this pull request? This PR is to address a few existing issues in `EXPLAIN`: - The `EXPLAIN` options `LOGICAL | FORMATTED |

spark git commit: [SPARK-15215][SQL] Fix Explain Parsing and Output

Repository: spark Updated Branches: refs/heads/branch-2.0 27bb51ca4 -> 58f77421b [SPARK-15215][SQL] Fix Explain Parsing and Output What changes were proposed in this pull request? This PR is to address a few existing issues in `EXPLAIN`: - The `EXPLAIN` options `LOGICAL | FORMATTED |

spark git commit: [SPARK-14656][CORE] Fix Benchmark.getPorcessorName() always return "Unknown processor" on Linux

Repository: spark Updated Branches: refs/heads/master 5cb2e3360 -> 0b8369d85 [SPARK-14656][CORE] Fix Benchmark.getPorcessorName() always return "Unknown processor" on Linux ## What changes were proposed in this pull request? This PR returns correct processor name in ```/proc/cpuinfo``` on

spark git commit: [SPARK-14398][SQL] Audit non-reserved keyword list in ANTLR4 parser

Repository: spark Updated Branches: refs/heads/master d4b94ead9 -> 74fe235ab [SPARK-14398][SQL] Audit non-reserved keyword list in ANTLR4 parser ## What changes were proposed in this pull request? I have compared non-reserved list in Antlr3 and Antlr4 one by one as well as all the existing

spark git commit: [HOTFIX] Remove wrong DDL tests

Repository: spark Updated Branches: refs/heads/master d53a51c1e -> 4ac6e75cd [HOTFIX] Remove wrong DDL tests ## What changes were proposed in this pull request? As we moved most parsing rules to `SparkSqlParser`, some tests expected to throw exception are not correct anymore. ## How was

spark git commit: [SPARK-16530][SQL][TRIVIAL] Wrong Parser Keyword in ALTER TABLE CHANGE COLUMN

ect. Thus, I put the issue as a Trivial level. cc hvanhovell How was this patch tested? Existing test cases Author: gatorsmile <gatorsm...@gmail.com> Closes #14186 from gatorsmile/changeColumns. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.a

spark git commit: [SPARK-16543][SQL] Rename the columns of `SHOW PARTITION/COLUMNS` commands

Repository: spark Updated Branches: refs/heads/master 1b5c9e52a -> 56183b84f [SPARK-16543][SQL] Rename the columns of `SHOW PARTITION/COLUMNS` commands ## What changes were proposed in this pull request? This PR changes the name of columns returned by `SHOW PARTITION` and `SHOW COLUMNS`

spark git commit: [SPARK-16726][SQL] Improve `Union/Intersect/Except` error messages on incompatible types

Repository: spark Updated Branches: refs/heads/master 579fbcf3b -> 64d8f37c7 [SPARK-16726][SQL] Improve `Union/Intersect/Except` error messages on incompatible types ## What changes were proposed in this pull request? Currently, `UNION` queries on incompatible types show misleading error

spark git commit: [SPARK-10601][SQL] Support `MINUS` set operator

Repository: spark Updated Branches: refs/heads/master bdd537164 -> 41a7dbdd3 [SPARK-10601][SQL] Support `MINUS` set operator ## What changes were proposed in this pull request? This PR adds `MINUS` set operator which is equivalent `EXCEPT DISTINCT`. This will slightly improve the

spark git commit: [SPARK-16771][SQL] WITH clause should not fall into infinite loop.

Repository: spark Updated Branches: refs/heads/master bbae20ade -> 2a105134e [SPARK-16771][SQL] WITH clause should not fall into infinite loop. ## What changes were proposed in this pull request? This PR changes the CTE resolving rule to use only **forward-declared** tables in order to

spark git commit: [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables

Repository: spark Updated Branches: refs/heads/master ffcb6e055 -> 840853ed0 [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables ## What changes were proposed in this pull request? Temporary tables are used frequently, but `spark.catalog.listColumns` does not

spark git commit: [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables

Repository: spark Updated Branches: refs/heads/branch-2.0 72cf74324 -> aea33bf05 [SPARK-16458][SQL] SessionCatalog should support `listColumns` for temporary tables ## What changes were proposed in this pull request? Temporary tables are used frequently, but `spark.catalog.listColumns` does

spark git commit: [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators.

Repository: spark Updated Branches: refs/heads/master 6cb75db9a -> 5ad68ba5c [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators. ## What changes were proposed in this pull request? when query only use metadata

spark git commit: [SPARK-14839][SQL] Support for other types for `tableProperty` rule in SQL syntax

Repository: spark Updated Branches: refs/heads/master 44c7c62bc -> 34283de16 [SPARK-14839][SQL] Support for other types for `tableProperty` rule in SQL syntax ## What changes were proposed in this pull request? Currently, Scala API supports to take options with the types, `String`, `Long`,

spark git commit: [SPARK-16459][SQL] Prevent dropping current database

Repository: spark Updated Branches: refs/heads/master 9cb1eb7af -> 7ac79da0e [SPARK-16459][SQL] Prevent dropping current database ## What changes were proposed in this pull request? This PR prevents dropping the current database to avoid errors like the followings. ```scala scala>

spark git commit: [SPARK-16459][SQL] Prevent dropping current database

Repository: spark Updated Branches: refs/heads/branch-2.0 7e4ba66d9 -> f97dd8a8f [SPARK-16459][SQL] Prevent dropping current database This PR prevents dropping the current database to avoid errors like the followings. ```scala scala> sql("create database delete_db") scala> sql("use

spark git commit: [SPARK-16879][SQL] unify logical plans for CREATE TABLE and CTAS

Repository: spark Updated Branches: refs/heads/master faaefab26 -> 5effc016c [SPARK-16879][SQL] unify logical plans for CREATE TABLE and CTAS ## What changes were proposed in this pull request? we have various logical plans for CREATE TABLE and CTAS: `CreateTableUsing`,

spark git commit: [SPARK-16804][SQL] Correlated subqueries containing non-deterministic operations return incorrect results

Repository: spark Updated Branches: refs/heads/master e10ca8de4 -> 06f5dc841 [SPARK-16804][SQL] Correlated subqueries containing non-deterministic operations return incorrect results ## What changes were proposed in this pull request? This patch fixes the incorrect results in the rule

spark git commit: [SPARK-17032][SQL] Add test cases for methods in ParserUtils.

Repository: spark Updated Branches: refs/heads/master 0f6aa8afa -> 4d0cc84af [SPARK-17032][SQL] Add test cases for methods in ParserUtils. ## What changes were proposed in this pull request? Currently methods in `ParserUtils` are tested indirectly, we should add test cases in

spark git commit: [SPARK-19118][SQL] Percentile support for frequency distribution table

Repository: spark Updated Branches: refs/heads/master 3d314d08c -> e99e34d0f [SPARK-19118][SQL] Percentile support for frequency distribution table ## What changes were proposed in this pull request? I have a frequency distribution table with following entries Age,No of person 21, 10 22,

spark git commit: [SPARK-16609] Add to_date/to_timestamp with format functions

Repository: spark Updated Branches: refs/heads/master 6ed285c68 -> 7a7ce272f [SPARK-16609] Add to_date/to_timestamp with format functions ## What changes were proposed in this pull request? This pull request adds two new user facing functions: - `to_date` which accepts an expression and a

spark git commit: [SPARK-18609][SPARK-18841][SQL] Fix redundant Alias removal in the optimizer

sts to the `SQLQueryTestSuite.union` and `SQLQueryTestSuite.cte` test cases. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #16757 from hvanhovell/SPARK-18609. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit

spark git commit: [SPARK-18863][SQL] Output non-aggregate expressions without GROUP BY in a subquery does not yield an error

Repository: spark Updated Branches: refs/heads/master 0e821ec6f -> f1ddca5fc [SPARK-18863][SQL] Output non-aggregate expressions without GROUP BY in a subquery does not yield an error ## What changes were proposed in this pull request? This PR will report proper error messages when a

spark git commit: [SPARK-18863][SQL] Output non-aggregate expressions without GROUP BY in a subquery does not yield an error

Repository: spark Updated Branches: refs/heads/branch-2.1 f391ad2c8 -> af9545538 [SPARK-18863][SQL] Output non-aggregate expressions without GROUP BY in a subquery does not yield an error ## What changes were proposed in this pull request? This PR will report proper error messages when a

spark git commit: [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join

Repository: spark Updated Branches: refs/heads/master 552e5f088 -> 9b7a03f15 [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join ## What changes were proposed in this pull request? PythonUDF is unevaluable, which can not be used inside a join condition, currently

spark git commit: [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join

Repository: spark Updated Branches: refs/heads/branch-2.1 482d361c3 -> 4d286c903 [SPARK-18589][SQL] Fix Python UDF accessing attributes from both side of join PythonUDF is unevaluable, which can not be used inside a join condition, currently the optimizer will push a PythonUDF which

[2/2] spark git commit: [SPARK-18936][SQL] Infrastructure for session local timezone support.

[SPARK-18936][SQL] Infrastructure for session local timezone support. ## What changes were proposed in this pull request? As of Spark 2.1, Spark SQL assumes the machine timezone for datetime manipulation, which is bad if users are not in the same timezones as the machines, or if different

[1/2] spark git commit: [SPARK-18936][SQL] Infrastructure for session local timezone support.

Repository: spark Updated Branches: refs/heads/master 7045b8b35 -> 2969fb437 http://git-wip-us.apache.org/repos/asf/spark/blob/2969fb43/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala

spark git commit: [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results

Repository: spark Updated Branches: refs/heads/master 8f3f73abc -> cdb691eb4 [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results ## What changes were proposed in this pull request? This PR fixes the code in Optimizer phase where the NULL-aware

spark git commit: [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results

Repository: spark Updated Branches: refs/heads/branch-2.1 d128b6a39 -> b94fb284b [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results ## What changes were proposed in this pull request? This PR fixes the code in Optimizer phase where the NULL-aware

spark git commit: [SPARK-19334][SQL] Fix the code injection vulnerability related to Generator functions.

Repository: spark Updated Branches: refs/heads/master cdb691eb4 -> 15ef3740d [SPARK-19334][SQL] Fix the code injection vulnerability related to Generator functions. ## What changes were proposed in this pull request? Similar to SPARK-15165, codegen is in danger of arbitrary code injection.

spark git commit: [SPARK-19691][SQL] Fix ClassCastException when calculating percentile of decimal column

Repository: spark Updated Branches: refs/heads/master 769aa0f1d -> 93aa42715 [SPARK-19691][SQL] Fix ClassCastException when calculating percentile of decimal column ## What changes were proposed in this pull request? This pr fixed a class-cast exception below; ``` scala>

spark git commit: [SPARK-13721][SQL] Make GeneratorOuter unresolved.

Repository: spark Updated Branches: refs/heads/master bf7bb4977 -> 10c566cc3 [SPARK-13721][SQL] Make GeneratorOuter unresolved. ## What changes were proposed in this pull request? This is a small change to make GeneratorOuter always unresolved. It is mostly no-op change but makes it more

spark git commit: [SPARK-19509][SQL] Grouping Sets do not respect nullable grouping columns

ach. Closes https://github.com/apache/spark/pull/16874 ## How was this patch tested? Added a regression test to `SQLQueryTestSuite.grouping_set`. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #16873 from hvanhovell/SPARK-19509. Project: http://git-wip-us.apache.org/repos/asf/s

[1/2] spark git commit: [SPARK-19589][SQL] Removal of SQLGEN files

Repository: spark Updated Branches: refs/heads/master 9b5e460a9 -> 457850e6f http://git-wip-us.apache.org/repos/asf/spark/blob/457850e6/sql/hive/src/test/resources/sqlgen/rollup_cube_7_3.sql -- diff --git

[2/2] spark git commit: [SPARK-19589][SQL] Removal of SQLGEN files

[SPARK-19589][SQL] Removal of SQLGEN files ### What changes were proposed in this pull request? SQLGen is removed. Thus, the generated files should be removed too. ### How was this patch tested? N/A Author: Xiao Li Closes #16921 from gatorsmile/removeSQLGenFiles.

spark git commit: [SPARK-19544][SQL] Improve error message when some column types are compatible and others are not in set operations

Repository: spark Updated Branches: refs/heads/master 04ad82253 -> 4321ff9ed [SPARK-19544][SQL] Improve error message when some column types are compatible and others are not in set operations ## What changes were proposed in this pull request? This PR proposes to fix the error message when

spark git commit: [SPARK-19496][SQL] to_date udf to return null when input date is invalid

Repository: spark Updated Branches: refs/heads/master 8f03ad547 -> 04ad82253 [SPARK-19496][SQL] to_date udf to return null when input date is invalid ## What changes were proposed in this pull request? Currently the udf `to_date` has different return value with an invalid date input. ```

spark git commit: [SPARK-19543] from_json fails when the input row is empty

Repository: spark Updated Branches: refs/heads/branch-2.1 ff5818b8c -> 7b5ea000e [SPARK-19543] from_json fails when the input row is empty ## What changes were proposed in this pull request? Using from_json on a column with an empty string results in: java.util.NoSuchElementException: head

spark git commit: [SPARK-19543] from_json fails when the input row is empty

Repository: spark Updated Branches: refs/heads/master fd6c3a0b1 -> d5593f7f5 [SPARK-19543] from_json fails when the input row is empty ## What changes were proposed in this pull request? Using from_json on a column with an empty string results in: java.util.NoSuchElementException: head of

spark git commit: [SPARK-19025][SQL] Remove SQL builder for operators

Repository: spark Updated Branches: refs/heads/master 1af0dee41 -> af63c52fd [SPARK-19025][SQL] Remove SQL builder for operators ## What changes were proposed in this pull request? With the new approach of view resolution, we can get rid of SQL generation on view creation, so let's remove

spark git commit: [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN subquery 2nd batch

Repository: spark Updated Branches: refs/heads/master 601b9c3e6 -> d22db6278 [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN subquery 2nd batch ## What changes were proposed in this pull request? This is 2nd batch of test case for IN/NOT IN subquery. In this PR, it has these test

spark git commit: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subquery (Joins + CTE)

Repository: spark Updated Branches: refs/heads/master 5ad10c531 -> a8a139820 [SPARK-18872][SQL][TESTS] New test cases for EXISTS subquery (Joins + CTE) ## What changes were proposed in this pull request? This PR adds the third and final set of tests for EXISTS subquery. File name

spark git commit: [SPARK-16475][SQL] broadcast hint for SQL queries - follow up

Repository: spark Updated Branches: refs/heads/master b55563c17 -> 733c59ec1 [SPARK-16475][SQL] broadcast hint for SQL queries - follow up ## What changes were proposed in this pull request? A small update to https://github.com/apache/spark/pull/16925 1. Rename SubstituteHints ->

spark git commit: [SPARK-18873][SQL][TEST] New test cases for scalar subquery (part 2 of 2) - scalar subquery in predicate context

Repository: spark Updated Branches: refs/heads/master d22db6278 -> 5ad10c531 [SPARK-18873][SQL][TEST] New test cases for scalar subquery (part 2 of 2) - scalar subquery in predicate context ## What changes were proposed in this pull request? This PR adds new test cases for scalar subquery in

spark git commit: [SPARK-19512][SQL] codegen for compare structs fails

Repository: spark Updated Branches: refs/heads/master 4064574d0 -> 1af0dee41 [SPARK-19512][SQL] codegen for compare structs fails ## What changes were proposed in this pull request? Set currentVars to null in GenerateOrdering.genComparisons before genCode is called. genCode ignores

spark git commit: [SPARK-13721][SQL] Support outer generators in DataFrame API

Repository: spark Updated Branches: refs/heads/master 83dff87de -> 2992a0e79 [SPARK-13721][SQL] Support outer generators in DataFrame API ## What changes were proposed in this pull request? Added outer_explode, outer_posexplode, outer_inline functions and expressions. Some bug fixing in

spark git commit: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to handle `COUNT(NULL) OVER` correctly

Repository: spark Updated Branches: refs/heads/branch-2.0 029789611 -> e62b29f29 [SPARK-17098][SQL] Fix `NullPropagation` optimizer to handle `COUNT(NULL) OVER` correctly ## What changes were proposed in this pull request? Currently, `NullPropagation` optimizer replaces `COUNT` on null

spark git commit: [SPARK-17098][SQL] Fix `NullPropagation` optimizer to handle `COUNT(NULL) OVER` correctly

Repository: spark Updated Branches: refs/heads/master ab7143463 -> 91c239768 [SPARK-17098][SQL] Fix `NullPropagation` optimizer to handle `COUNT(NULL) OVER` correctly ## What changes were proposed in this pull request? Currently, `NullPropagation` optimizer replaces `COUNT` on null literals

spark git commit: [SPARK-19766][SQL][BRANCH-2.0] Constant alias columns in INNER JOIN should not be folded by FoldablePropagation rule

Repository: spark Updated Branches: refs/heads/branch-2.0 c9c45d97b -> e30fe1c6a [SPARK-19766][SQL][BRANCH-2.0] Constant alias columns in INNER JOIN should not be folded by FoldablePropagation rule This PR fix for branch-2.0 Refer #17099 gatorsmile Author: Stan Zhai

  1   2   3   4   5   6   7   8   9   >