On Tue, Jul 8, 2014 at 12:43 PM, Pierre B < pierre.borckm...@realimpactanalytics.com> wrote: > > 1/ Is there a way to convert a SchemaRDD (for instance loaded from a > parquet > file) back to a RDD of a given case class? >
There may be someday, but doing so will either require a lot of reflection or a bunch of macro magic. So while I think this would be cool, it will probably be a while before we can implement it, and it'll likely be experimental. > 2/ Even better, is there a way to get the schema information from a > SchemaRDD ? I am trying to figure out how to properly get the various > fields > of the Rows of a SchemaRDD. Knowing the schema (in the form of a Map?), I > guess I could nicely use getInt, getString, ..., on each row. > We are actively working on this (SPARK-2179 <https://issues.apache.org/jira/browse/SPARK-2179>). Hopefully there will be a PR soon, and we are targeting the 1.1 release.
