GitHub user tanejagagan opened a pull request:
https://github.com/apache/spark/pull/17174
[SPARK-19145][SQL] Timestamp to String casting is slowing the query sâ¦
â¦ignificantly
If BinaryComparison has expression with timestamp and string datatype then
cast string to timestamp if string type expression is foldable. This results in
order of magnitude performance improvement in query execution
## What changes were proposed in this pull request?
If BinaryComparison has expression with timestamp and string datatype then
cast string to timestamp if string type expression is foldable. This results in
order of magnitude performance improvement in query execution
## How was this patch tested?
Added new unit tests to conver the functionality
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tanejagagan/spark branch-19145
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17174.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17174
----
commit 746b8926747ce85d4d5ecd78a9291a13b15e52d5
Author: gagan taneja <[email protected]>
Date: 2017-03-06T07:26:39Z
[SPARK-19145][SQL] Timestamp to String casting is slowing the query
significantly
If BinaryComparison has expression with timestamp and string datatype then
cast string to timestamp if string type expression is foldable. This results in
order of magnitude performance improvement in query execution
Added new unit tests to conver the functionality
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
