Bill Chambers created SPARK-18424:
-------------------------------------
Summary: Cumbersome Date Manipulation
Key: SPARK-18424
URL: https://issues.apache.org/jira/browse/SPARK-18424
Project: Spark
Issue Type: Improvement
Reporter: Bill Chambers
Priority: Minor
I've found it quite cumbersome to work with dates thus far in Spark, it can be
hard to reason about the timeformat and what type you're working with, for
instance:
say that I have a date in the format
{code}
2017-20-12
// Y-D-M
{code}
In order to parse that into a Date, I have to perform several conversions.
{code}
to_date(
unix_timestamp(col("date"), dateFormat)
.cast("timestamp"))
.alias("date")
{code}
I propose simplifying this by adding a to_date function (exists) but adding one
that accepts a format for that date. so that you can avoid entirely the above
conversion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
