ueshin commented on code in PR #53837:
URL: https://github.com/apache/spark/pull/53837#discussion_r2710222981
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonUDTFExec.scala:
##########
Review Comment:
`child` or `child.output` should also be captured in advance?
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonUDTFExec.scala:
##########
Review Comment:
`evaluate` belongs to `EvalPythonUDTFExec`, so I guess we can't avoid
capturing `this`?
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonUDTFExec.scala:
##########
@@ -65,26 +88,6 @@ trait EvalPythonUDTFExec extends UnaryExecNode {
queue.close()
}
- // flatten all the arguments
- val allInputs = new ArrayBuffer[Expression]
- val dataTypes = new ArrayBuffer[DataType]
- val argMetas = udtf.children.zip(
- udtf.tableArguments.getOrElse(Seq.fill(udtf.children.length)(false))
- ).map { case (e: Expression, isTableArg: Boolean) =>
- val (key, value) = e match {
- case NamedArgumentExpression(key, value) =>
- (Some(key), value)
- case _ =>
- (None, e)
- }
- if (allInputs.exists(_.semanticEquals(value))) {
- ArgumentMetadata(allInputs.indexWhere(_.semanticEquals(value)), key,
isTableArg)
- } else {
- allInputs += value
- dataTypes += value.dataType
- ArgumentMetadata(allInputs.length - 1, key, isTableArg)
- }
- }.toArray
val projection = MutableProjection.create(allInputs.toSeq, child.output)
Review Comment:
ditto for `child.output`?
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonUDTFExec.scala:
##########
Review Comment:
ditto for `udtf` or `udtf.elementSchema`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
