[
https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844467#comment-17844467
]
Daniel commented on SPARK-48180:
--------------------------------
The bug is that parentheses are required around the two arguments in {{ORDER BY
device_id, data_ds.}} Otherwise the SQL analyzer cannot tell the difference
between ordering by an additional table column vs. another expression argument
to the TVF.
It could help to improve the error message here to make it more explicit.
> Analyzer bug with multiple ORDER BY items for input table argument
> ------------------------------------------------------------------
>
> Key: SPARK-48180
> URL: https://issues.apache.org/jira/browse/SPARK-48180
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.5.0, 4.0.0, 3.5.1
> Reporter: Daniel
> Priority: Major
>
> Steps to reproduce:
>
> {{from pyspark.sql.functions import udtf}}
> {{@udtf(returnType="a: int, b: int")}}
> {{class tvf:}}
> {{ def eval(self, *args):}}
> {{ yield 1, 2}}
>
> {{SELECT * FROM tvf(}}
> {{ TABLE(}}
> {{ SELECT 1 AS device_id, 2 AS data_ds}}
> {{ )}}
> {{ WITH SINGLE PARTITION}}
> {{ ORDER BY device_id, data_ds}}
> {{ )}}
> {{[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_TABLE_ARGUMENT]
> Unsupported subquery expression: Table arguments are used in a function where
> they are not supported:}}
> {{'UnresolvedTableValuedFunction [tvf], [table-argument#338 [], 'data_ds],
> false}}
> {{ +- Project [1 AS device_id#336, 2 AS data_ds#337]}}
> {{ +- OneRowRelation}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
