GitHub user anabranch opened a pull request:
https://github.com/apache/spark/pull/16138
[WIP][Spark-16609] Add to_date with format function.
## What changes were proposed in this pull request?
This pull request adds a user facing `to_date` function that allows for a
format to be expressed by the user.
Given a date in format: `2016-21-05`.
### Date Function
- [X] Add Function to Scala
- [ ] Add Function to Python
- [ ] Add Function to SQL
*Previously*
```
to_date(unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp"))
```
*Current*
```
to_date(lit("2016-21-05"), "yyyy-dd-MM")
```
### Timestamp Function
- [X] Add Function to Scala (not sure how to finalize a cast)
- [ ] Add Function to Python
- [ ] Add Function to SQL
*Previously*
```
unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp")
```
*Current*
```
to_timestamp(lit("2016-21-05"), "yyyy-dd-MM")
```
## How was this patch tested?
- [ ] `ToDate` Expression Tests (DateExpressionsSuite)
- [ ] DataFrame Function Tests (TODO)
- [ ] SQL Function Tests (TODO)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/anabranch/spark SPARK-16609
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16138.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16138
----
commit 087feaec0a1907010b2457d7169b5f2d6c5c1df6
Author: anabranch <[email protected]>
Date: 2016-11-16T03:18:15Z
prepping for adding the function
commit bef59b10025d685dae33f5572e35c0474c79a05e
Author: anabranch <[email protected]>
Date: 2016-11-16T05:18:51Z
added base structure
commit 9db65b809d0efa27e5805394c258a2391d397e3e
Author: anabranch <[email protected]>
Date: 2016-11-17T05:50:21Z
time to start fixing tests
commit feafa3d9314ed2521a9e9e141942f6357fbc1937
Author: anabranch <[email protected]>
Date: 2016-11-17T05:54:05Z
base function
commit 562df343d18c68ca4622d65e1cc8a2dcd46ca9e2
Author: anabranch <[email protected]>
Date: 2016-12-01T03:26:42Z
Merge branch 'master' of https://github.com/apache/spark into SPARK-18424
commit c4d997b4ff04991beb183164d316fa01f8dd36da
Author: anabranch <[email protected]>
Date: 2016-12-01T04:39:18Z
improved comment
commit 65d668fe469387519048edb39a9fcc50ca64be37
Author: anabranch <[email protected]>
Date: 2016-12-01T04:59:00Z
Merge branch 'master' of https://github.com/apache/spark into SPARK-16609
commit fcf73ec7d10c5b8da7dd803c886179c650f071c5
Author: anabranch <[email protected]>
Date: 2016-12-01T05:00:49Z
first pass at code gen
commit 27e4abe01a03edcc4c80a5ab92d4d84f3dbace1c
Author: anabranch <[email protected]>
Date: 2016-12-04T20:12:36Z
ready to start a discussion
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
