Re: Easily creating custom encoders

2017-03-21 Thread Koert Kuipers 
see:
https://issues.apache.org/jira/browse/SPARK-18122

On Tue, Mar 21, 2017 at 1:13 PM, Ashic Mahtab  wrote:

> I'm trying to easily create custom encoders for case classes having
> "unfriendly" fields. I could just kryo the whole thing, but would like to
> at least have a few fields in the schema instead of one binary blob. For
> example,
>
>
> case class MyClass(id: UUID, items: Map[String, Double], name: String)
>
>
> Is there a way to create an Encoder[MyClass] by kryo-ing the things that
> don't work, and not the ones that do, while retaining distinct columns? In
> the previous example, I'd at least want (binary, binary, String).
>
>
> Is this possible?
>
>
> -Ashic.
>

Easily creating custom encoders

2017-03-21 Thread Ashic Mahtab 
I'm trying to easily create custom encoders for case classes having 
"unfriendly" fields. I could just kryo the whole thing, but would like to at 
least have a few fields in the schema instead of one binary blob. For example,


case class MyClass(id: UUID, items: Map[String, Double], name: String)


Is there a way to create an Encoder[MyClass] by kryo-ing the things that don't 
work, and not the ones that do, while retaining distinct columns? In the 
previous example, I'd at least want (binary, binary, String).


Is this possible?


-Ashic.