XiDuo You created SPARK-38578:
---------------------------------
Summary: Avoid unnecessary sort in FileFormatWriter if user has
specified sort
Key: SPARK-38578
URL: https://issues.apache.org/jira/browse/SPARK-38578
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.0
Reporter: XiDuo You
FileFormatWriter will check and add an implicit sort for dynamic partition
columns or bucket columns according to the input physical plan. The check
became always failure since AQE AdaptiveSparkPlanExec has no outputOrdering.
That casues a redundant sort if user has specified a sort which satisfies the
required ordering (dynamic partition and bucket columns).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
