Can I get schema information from SchemaRDD?
For example,
*case class Person(name:String, Age:Int, Gender:String, Birth:String)
val peopleRDD = sc.textFile("/sample/sample.csv").map(_.split(",")).map(p =>
Person(p(0).toString, p(1).toInt, p(2).toString, p(3).toString))
peopleRDD.saveAsParquetFile("people.parquet")*
(few days later...)
*val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._
val loadedPeopleRDD = sqlContext.parquetFile("people.parquet")
loadedPeopleRDD.registerAsTable("peopleTable")*
Someone who doesn't know Person class can't know what columns and types this
table have.
Maybe they want to get schema information from loadedPeopleRDD.
How can I do this?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/get-schema-from-SchemaRDD-tp7830.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
