vicennial commented on code in PR #41959:
URL: https://github.com/apache/spark/pull/41959#discussion_r1261765371
##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala:
##########
@@ -92,17 +92,21 @@ sealed abstract class UserDefinedFunction {
/**
* Holder class for a scalar user-defined function and it's input/output
encoder(s).
*/
-case class ScalarUserDefinedFunction(
+case class ScalarUserDefinedFunction private (
function: AnyRef,
inputEncoders: Seq[AgnosticEncoder[_]],
outputEncoder: AgnosticEncoder[_],
name: Option[String],
override val nullable: Boolean,
- override val deterministic: Boolean)
+ override val deterministic: Boolean,
Review Comment:
Thanks for the suggestion! It is indeed easier to work with it this way.
I also just realized the previous solution is flawed since `nullable` is
part of the proto.ScalarScalaUDF so we can't override it in the case of
`asNonNullable` update.
In the new approach, I add the `serializedUdfPacket` + the input/output
proto types so that we can reconstruct the proto.ScalarScalaUDF message without
hitting the ser issue while also supporting the nullable update. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
