dtenedor commented on code in PR #45142:
URL: https://github.com/apache/spark/pull/45142#discussion_r1507971562
##########
python/pyspark/sql/udtf.py:
##########
@@ -123,10 +123,17 @@ class SelectedColumn:
alias : str, default ''
If non-empty, this is the alias for the column or expression as
visible from the UDTF's
'eval' method. This is required if the expression is not a simple
column reference.
+ forwardToOutputTable : bool, default False
+ If true, the UDTF is specifying to Catalyst a metadata property
wherein the function call
+ will copy the result of evaluating this column or expression from the
most recent input row
+ through to the output table, to a column with the same name specified
in the 'alias' field
+ (or the name of the simple column reference otherwise). This is useful
because it lets the
+ optimizer push filters or other operations down through the UDTF call
to the input table.
"""
name: str
alias: str = ""
+ forwardToOutputTable: bool = False
Review Comment:
sure, that sounds good. Let me try to hack that up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
