Re: Attaching schema to RDD created from Parquet file

2014-10-20 Thread Michael Armbrust
This has to be done manually: rdd2.map(row => Person(row.getString(0), row.getInt(1))) On Fri, Oct 17, 2014 at 9:30 AM, Akshat Aranya wrote: > Hi, > > How can I convert an RDD loaded from a Parquet file into its original type: > > case class Person(name: String, age: Int) > > val rdd: RDD[Perso

Attaching schema to RDD created from Parquet file

2014-10-17 Thread Akshat Aranya
Hi, How can I convert an RDD loaded from a Parquet file into its original type: case class Person(name: String, age: Int) val rdd: RDD[Person] = ... rdd.saveAsParquetFile("people") val rdd2: sqlContext.parquetFile("people") How can I map rdd2 back into an RDD[Person]? All of the examples just