Maxim Gekk created SPARK-31212: ---------------------------------- Summary: Failure of casting the '1000-02-29' string to the date type Key: SPARK-31212 URL: https://issues.apache.org/jira/browse/SPARK-31212 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.5 Reporter: Maxim Gekk
The '1000-02-29' is valid date in the Julian calendar used in Spark 2.4.5 for dates before 1582-10-15 but casting the string to the date type fails: {code:scala} scala> val df = Seq("1000-02-29").toDF("dateS").select($"dateS".cast("date").as("date")) df: org.apache.spark.sql.DataFrame = [date: date] scala> df.show +----+ |date| +----+ |null| +----+ {code} Creating a dataset from java.sql.Date w/ the same input string works correctly: {code:scala} scala> val df2 = Seq(java.sql.Date.valueOf("1000-02-29")).toDF("dateS").select($"dateS".as("date")) df2: org.apache.spark.sql.DataFrame = [date: date] scala> df2.show +----------+ | date| +----------+ |1000-02-29| +----------+ {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org