Hi,
I think Spark UDF can only handle `java.sql.Date`.
So, you need to change the return type in you UDF.
// maropu
On Tue, Jan 24, 2017 at 8:18 AM, Marco Mistroni <mmistr...@gmail.com> wrote:
> HI all
> i am trying to convert a string column, in a Dataframe , to a
> java.util.Date but i am getting this exception
>
> [dispatcher-event-loop-0] INFO org.apache.spark.storage.BlockManagerInfo
> - Removed broadcast_0_piece0 on 169.254.2.140:53468 in memory (size: 14.3
> KB, free: 767.4 MB)
> Exception in thread "main" java.lang.UnsupportedOperationException:
> Schema for type java.util.Date is not supported
>
> here's my code
>
> val tstampToDateFunc:(java.sql.Timestamp => java.util.Date) = ts => new
> java.util.Date(ts.getTime)
> val tsampConversionFunc = udf(tstampToDateFunc)
>
> sharesDf.withColumn("price", col("_c2").cast("double"))
> .withColumn("creationTime",
> tsampConversionFunc(col("_c1")))
>
> Are there any workarounds?
> i m trying to import data into mongoDB via Spark. The source is a csv file
> where
> i have 1 timestamp column and a bunch of strings..... i will need to
> convert that
> to something compatible with a mongo's ISODate
>
> kr
> marco
>
>
--
---
Takeshi Yamamuro
