GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/11314
[SPARK-13723] [SQL] Comprehensive Verification and Fixing of SQL Generation
Support for Expressions
#### What changes were proposed in this pull request?
Ensure that all built-in expressions can be mapped to its SQL
representation if there is one (e.g. ScalaUDF doesn't have a SQL
representation). The function lists are from the expression list in
`FunctionRegistry`.
window functions, grouping sets functions (`cube`, `rollup`, `grouping`,
`grouping_id`), generator functions (`explode` and `json_tuple`) are covered by
separate JIRA and PRs. Thus, this PR does not cover them. Except these
functions, all the built-in expressions are covered. For details, see the list
in `ExpressionToSQLSuite`.
Fixed a few issues. For example, the `prettyName` of
`approx_count_distinct` is not right. The `sql` of `hash` function is not
right, since the `hash` function does not accept `seed`.
Additionally, also correct the order of expressions in `FunctionRegistry`
so that people are easier to find which functions are missing.
cc @liancheng
#### How was the this patch tested?
Added two test cases in LogicalPlanToSQLSuite for covering `not like` and
`not in`.
Added a new test suite `ExpressionToSQLSuite` to cover the functions:
1. misc non-aggregate functions + complex type creators + null expressions
2. math functions
3. aggregate functions
4. string functions
5. date time functions + calendar interval
6. collection functions
7. misc functions
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark expressionToSQL
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11314.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11314
----
commit 2fc93649b2e7294d79ab7c7c5c591b77d46d9c54
Author: gatorsmile <[email protected]>
Date: 2016-02-22T22:58:05Z
Comprehensive SQL generation support for expressions
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
