Re: [Spark SQL]: Java Spark Classes With Attributes of Type Set In Datasets

2018-09-25 Thread Dillon Dukek
Actually, it appears walking through it in a debug terminal that the deserializer can properly transform the data on read to an ArrayType, but the serializer doesn't know what to do when we try to go back out from the internal spark representation. tags, if (isnull(lambdavariable(MapObjects_loopVa

[Spark SQL]: Java Spark Classes With Attributes of Type Set In Datasets

2018-09-25 Thread ddukek
I'm trying to use a data model that has a instance variable that is a Set. If I leave the type as the Abstract Set class I get an error thrown because Set is an interface so it cannot be instantiated. If I then try and make the variable a concrete implementation of Set I get an analysis exception