grundprinzip commented on PR #39585: URL: https://github.com/apache/spark/pull/39585#issuecomment-1385376135
> As for the message `PythonFunction`, it was a placeholder for all the information required to construct a PySpark SimplePythonFunction, as shown below. > > ``` > private[spark] case class SimplePythonFunction( > command: Seq[Byte], > envVars: JMap[String, String], > pythonIncludes: JList[String], > pythonExec: String, > pythonVer: String, > broadcastVars: JList[Broadcast[PythonBroadcast]], > accumulator: PythonAccumulatorV2) > ``` Thats great to know. I'm wondering where you would draw the line between `PythonUDF` and `PythonFunction` in terms of which members go where? Given that we now have the one level higher abstraction on the `ScalarInlineUserDefinedFunction` I'm not sure we need an additional level of indirection. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
