Hi,
you can use the customSchema(for DateType) and specify dateFormat in .option().
or
at spark dataframe side, you can convert the timestamp to date using cast to
the column.
Thanks and regards,
Anand Viswanathan
> On Oct 26, 2016, at 8:07 PM, Koert Kuipers <ko...@tresata.com> wrote:
>
> hey,
> i create a file called test.csv with contents:
> date
> 2015-01-01
> 2016-03-05
>
> next i run this code in spark 2.0.1:
> spark.read
> .format("csv")
> .option("header", true)
> .option("inferSchema", true)
> .load("test.csv")
> .printSchema
>
> the result is:
> root
> |-- date: timestamp (nullable = true)
>
>
> On Wed, Oct 26, 2016 at 7:35 PM, Hyukjin Kwon <gurwls...@gmail.com
> <mailto:gurwls...@gmail.com>> wrote:
> There are now timestampFormat for TimestampType and dateFormat for DateType.
>
> Do you mind if I ask to share your codes?
>
>
> On 27 Oct 2016 2:16 a.m., "Koert Kuipers" <ko...@tresata.com
> <mailto:ko...@tresata.com>> wrote:
> is there a reason a column with dates in format yyyy-mm-dd in a csv file is
> inferred to be TimestampType and not DateType?
>
> thanks! koert
>
