Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12087#discussion_r58289975
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala
---
@@ -91,6 +91,32 @@ case class MapPartitions(
override def deserializers: Seq[(Expression, Seq[Attribute])] =
Seq(deserializer -> child.output)
}
+object MapElements {
+ def apply[T : Encoder, U : Encoder](
+ func: T => U,
+ child: LogicalPlan): MapElements = {
+ MapElements(
+ func.asInstanceOf[Any => Any],
+ encoderFor[T].deserializer,
+ encoderFor[U].namedExpressions,
+ child)
+ }
+}
+
+/**
+ * A relation produced by applying `func` to each element of the `child`.
+ *
+ * @param deserializer used to extract the input to `func` from an input
row.
+ * @param serializer use to serialize the output of `func`.
+ */
+case class MapElements(
+ func: Any => Any,
--- End diff --
this should just be AnyRef? then you can use codegen to actually call the
java version without wrapping the scala one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
