Ian O Connell created SPARK-2449:
------------------------------------
Summary: Spark sql reflection code requires a constructor taking
all the fields for the table
Key: SPARK-2449
URL: https://issues.apache.org/jira/browse/SPARK-2449
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.0.0
Reporter: Ian O Connell
The reflection code does a lookup for the fields passed to the constructor to
make the types for the table. Specifically the code:
val params = t.member(nme.CONSTRUCTOR).asMethod.paramss
in ScalaReflection.scala
Simple repo case from the spark shell:
trait PersonTrait extends Product
case class Person(a: Int) extends PersonTrait
val l: List[PersonTrait] = List(1, 2, 3, 4).map(Person(_))
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._
sc.parallelize(l).registerAsTable("people")
scala> sc.parallelize(l).registerAsTable("people")
scala.ScalaReflectionException: <none> is not a method
at scala.reflect.api.Symbols$SymbolApi$class.asMethod(Symbols.scala:279)
at
scala.reflect.internal.Symbols$SymbolContextApiImpl.asMethod(Symbols.scala:73)
at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:52)
at
--
This message was sent by Atlassian JIRA
(v6.2#6252)
