Spark SQL has proven to be quite useful in applying a partial schema to large JSON logs and being able to write plain SQL to perform a wide variety of operations over this data. However, one small thing that keeps coming back to haunt me is the lack of support for recursive data types, whereby a member of a complex/struct value can be of the same type as the complex/struct value itself.
I am hoping someone may be able to point me in the right direction of where to start to build out such capabilities, as I'd be happy to contribute, but am very new to this particular component of the Spark project.
