Currently the design of Encoder implies the possibility that encoders might
be customized, or at least that there are other internal alternatives to
ExpressionEncoder. However there are both implicit and explicit
restrictions in spark-sql, such that ExpressionEncoder is the only
functional option, including but not limited to:
def encoderFor[A : Encoder]: ExpressionEncoder[A] =
implicitly[Encoder[A]] match {
case e: ExpressionEncoder[A] =>
e.assertUnresolved()
e
case _ => sys.error(s"Only expression encoders are supported today")
}
My impression, based on recent work around the SQL code, is that:
1) ExpressionEncoder does support the space of plausible data types and
objects, including custom ones.
2) Supporting other subclasses of Encoder in actuality would be nontrivial
Should spark SQL more explicitly commit to an "ExpressionEncoder only"
position?
