SaurabhChawla100 commented on a change in pull request #32558:
URL: https://github.com/apache/spark/pull/32558#discussion_r643734467
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVOptions.scala
##########
@@ -206,6 +206,12 @@ class CSVOptions(
sep
}
+ /**
+ * option to infer date Type in the schema
Review comment:
This option inferDateType is added to -> This is to keep in sync with
older version of spark, If someone wants to use the dateType, they can enable
it in the option. This is just to prevent any migration issue to spark-3.2.0
from the older version. if they don't enable this option inferDateType, It will
infer it as StringType.
Where as on other hand inferSchema is to enable the inferring of schema.
If inferSchema is enabled and inferDateType option is enabled in that case
on reading the schema is this will infer at data type as DateType format
instead of StringType
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
