[ 
https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Majid Hajiheidari updated SPARK-46349:
--------------------------------------
    Affects Version/s: 4.3.0

> Fail analysis with a clear error for nested sort orders
> -------------------------------------------------------
>
>                 Key: SPARK-46349
>                 URL: https://issues.apache.org/jira/browse/SPARK-46349
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark, SQL
>    Affects Versions: 4.0.0, 4.3.0
>            Reporter: Majid Hajiheidari
>            Priority: Minor
>              Labels: pull-request-available
>
> Applying a sort direction to an expression that is already a sort expression 
> creates a nested SortOrder. Analysis accepts it, and the query only fails at 
> execution time with an internal error that does not point at the user's code.
> Scala:
> {code:scala}
> spark.range(10).orderBy($"id".desc.asc).show()
> {code}
> Python:
> {code:python}
> spark.range(10).orderBy(f.desc("id"), ascending=False).show()
> {code}
> Both fail with:
> {noformat}
> org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot generate code for 
> expression: input[0, bigint, false] DESC NULLS LAST SQLSTATE: XX000
> {noformat}
> Analysis should reject a nested sort order with a clear error instead.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to