Bill Chambers created SPARK-18424: ------------------------------------- Summary: Cumbersome Date Manipulation Key: SPARK-18424 URL: https://issues.apache.org/jira/browse/SPARK-18424 Project: Spark Issue Type: Improvement Reporter: Bill Chambers Priority: Minor
I've found it quite cumbersome to work with dates thus far in Spark, it can be hard to reason about the timeformat and what type you're working with, for instance: say that I have a date in the format {code} 2017-20-12 // Y-D-M {code} In order to parse that into a Date, I have to perform several conversions. {code} to_date( unix_timestamp(col("date"), dateFormat) .cast("timestamp")) .alias("date") {code} I propose simplifying this by adding a to_date function (exists) but adding one that accepts a format for that date. so that you can avoid entirely the above conversion. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org