XiDuo You created SPARK-38578: --------------------------------- Summary: Avoid unnecessary sort in FileFormatWriter if user has specified sort Key: SPARK-38578 URL: https://issues.apache.org/jira/browse/SPARK-38578 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.3.0 Reporter: XiDuo You
FileFormatWriter will check and add an implicit sort for dynamic partition columns or bucket columns according to the input physical plan. The check became always failure since AQE AdaptiveSparkPlanExec has no outputOrdering. That casues a redundant sort if user has specified a sort which satisfies the required ordering (dynamic partition and bucket columns). -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org