Greetings, I have recently started using Spark SQL and ran up against two rather odd limitations related to UserDefinedTypes.
The first is that there appears to be no way to register a UserDefinefType other than by adding the @SQLUserDefinedType annotation to the class being mapped. This makes it impossible to define mappings for classes that the developer does not control. In my case, I wanted to be able to map the Instant class from JSR-310 (more specifically, the Java 7 backport) as a replacement for java.sql.Timestamp. To my dismay, I discovered that Spark SQL provides no way to do this, as far as I can tell. Secondly, it seems that Spark SQL absolutely refuses to allow ordering to be applied to UserDefinedTypes, even when the types are mapped to builtin types that are themselves ordered. At first I thought that perhaps I had Kay overlooked the way to do this, but when I looked in the source code I found, to my despair, that the code that implements ordering operations specially checks that the type extends AtomicType, and bails out with an exception if it does not. It seems like it should be relatively straightforward to implement both of these features. Are there currently any plans to do so? Note that I am using Spark 1.4.0, but I haven't seen any indication that either of these are addressed by 1.4.1 or 1.5.0. Thanks. Rich Eggert
