[
https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Majid Hajiheidari updated SPARK-46349:
--------------------------------------
Affects Version/s: 4.3.0
> Fail analysis with a clear error for nested sort orders
> -------------------------------------------------------
>
> Key: SPARK-46349
> URL: https://issues.apache.org/jira/browse/SPARK-46349
> Project: Spark
> Issue Type: Bug
> Components: PySpark, SQL
> Affects Versions: 4.0.0, 4.3.0
> Reporter: Majid Hajiheidari
> Priority: Minor
> Labels: pull-request-available
>
> Applying a sort direction to an expression that is already a sort expression
> creates a nested SortOrder. Analysis accepts it, and the query only fails at
> execution time with an internal error that does not point at the user's code.
> Scala:
> {code:scala}
> spark.range(10).orderBy($"id".desc.asc).show()
> {code}
> Python:
> {code:python}
> spark.range(10).orderBy(f.desc("id"), ascending=False).show()
> {code}
> Both fail with:
> {noformat}
> org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot generate code for
> expression: input[0, bigint, false] DESC NULLS LAST SQLSTATE: XX000
> {noformat}
> Analysis should reject a nested sort order with a clear error instead.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]