Re: [PR] [WIP][SPARK-47032][Python] Prototype for adding pass-through columns to Python UDTF API [spark]
dtenedor commented on PR #45142: URL: https://github.com/apache/spark/pull/45142#issuecomment-1955609505 Note this is a work-in-progress, I'm a bit busy over the next day or so but will add testing and push a commit then :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [WIP][SPARK-47032][Python] Prototype for adding pass-through columns to Python UDTF API [spark]
dongjoon-hyun commented on code in PR #45142: URL: https://github.com/apache/spark/pull/45142#discussion_r1493041669 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/PythonUDF.scala: ## @@ -195,13 +198,13 @@ case class PythonUDTF( override protected def withNewChildrenInternal(newChildren: IndexedSeq[Expression]): PythonUDTF = copy(children = newChildren) -} Review Comment: This removal looks like a mistake. Could you make CIs happy? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[PR] [WIP][SPARK-47032][Python] Prototype for adding pass-through columns to Python UDTF API [spark]
dtenedor opened a new pull request, #45142: URL: https://github.com/apache/spark/pull/45142 ### What changes were proposed in this pull request? [WIP] This is a prototype for adding pass-through columns to Python UDTF API. We'll develop it more before sending out for formal review. ### Why are the changes needed? We can use this API to forward column values from the UDTF input table to the output table directly, bypassing JVM/Python interchange and improving performance. ### Does this PR introduce _any_ user-facing change? Yes, see above. ### How was this patch tested? This PR adds test coverage. ### Was this patch authored or co-authored using generative AI tooling? NO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
