hvanhovell commented on a change in pull request #28223: [SPARK-31450][SQL]
Make ExpressionEncoder thread-safe
URL: https://github.com/apache/spark/pull/28223#discussion_r409055761
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
##########
@@ -332,30 +344,67 @@ case class ExpressionEncoder[T](
}
/**
- * Returns an encoded version of `t` as a Spark SQL row. Note that multiple
calls to
- * toRow are allowed to return the same actual [[InternalRow]] object.
Thus, the caller should
- * copy the result before making another call if required.
+ * Create a serializer that can convert an object of type `T` to a Spark SQL
Row.
+ *
+ * Note that the returned [[Serializer]] is not thread safe. Multiple calls
to
+ * `serializer.apply(..)` are allowed to return the same actual
[[InternalRow]] object. Thus,
+ * the caller should copy the result before making another call if required.
*/
- def toRow(t: T): InternalRow = try {
- inputRow(0) = t
- extractProjection(inputRow)
- } catch {
- case e: Exception =>
- throw new RuntimeException(s"Error while encoding: $e\n" +
+ def createSerializer(): Serializer[T] = new Serializer[T] {
Review comment:
This currently relies on use serializing the enclosing encoder as well. We
technically don't need the entire encoder but only a couple fields. I could
move this class into the companion object and just use the fields I need.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
