Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/7840#discussion_r36043875
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala
---
@@ -137,19 +136,27 @@ object UnsafeProjection {
/**
* A projection that could turn UnsafeRow into GenericInternalRow
*/
-case class FromUnsafeProjection(fields: Seq[DataType]) extends Projection {
+object FromUnsafeProjection {
- def this(schema: StructType) = this(schema.fields.map(_.dataType))
-
- private[this] val expressions = fields.zipWithIndex.map { case (dt, idx)
=>
- new BoundReference(idx, dt, true)
+ /**
+ * Returns an Projection for given StructType.
+ */
+ def apply(schema: StructType): Projection = {
+ apply(schema.fields.map(_.dataType))
}
- @transient private[this] lazy val generatedProj =
- GenerateMutableProjection.generate(expressions)()
+ /**
+ * Returns an UnsafeProjection for given Array of DataTypes.
+ */
+ def apply(fields: Seq[DataType]): Projection = {
+ create(fields.zipWithIndex.map(x => new BoundReference(x._2, x._1,
true)))
+ }
- override def apply(input: InternalRow): InternalRow = {
- generatedProj(input)
+ /**
+ * Returns an Projection for given sequence of Expressions (bounded).
+ */
+ private def create(exprs: Seq[Expression]): Projection = {
+ GenerateSafeProjection.generate(exprs)
--- End diff --
I was thinking about remove this unsafe->safe conversion after get rid of
`InternalRow`'s generic getter and toSeq, but now looks like we do need this
conversion if we must do deep copy of unsafe data like UTF8String, unsafe array
and map, etc.
cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
