[ https://issues.apache.org/jira/browse/SPARK-41368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wan Kun updated SPARK-41368: ---------------------------- Description: We can reorder window partition expressions by the distinct values stats. Sorting with high cardinality will be faster. Benchmark result: {code:java} Window: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ---------------------------------------------------------------------------------------------------------------------------- Disable reorder window partition expressions 13338 14035 1138 0.0 1333756.2 1.0X Enable reorder window partition expressions 5981 6030 77 0.0 598102.1 2.2X {code} was: We can reorder window partition expressions by the distinct values stats. Sorting with high cardinality will be faster. Benchmark result: {code:java} Window: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ---------------------------------------------------------------------------------------------------------------------------- Without reorder window partition expressions 13338 14035 1138 0.0 1333756.2 1.0X With reorder window partition expressions 5981 6030 77 0.0 598102.1 2.2X {code} > Reorder the window partition expressions by expression stats > ------------------------------------------------------------ > > Key: SPARK-41368 > URL: https://issues.apache.org/jira/browse/SPARK-41368 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.4.0 > Reporter: Wan Kun > Priority: Major > > We can reorder window partition expressions by the distinct values stats. > Sorting with high cardinality will be faster. > Benchmark result: > {code:java} > Window: Best Time(ms) Avg Time(ms) > Stdev(ms) Rate(M/s) Per Row(ns) Relative > ---------------------------------------------------------------------------------------------------------------------------- > Disable reorder window partition expressions 13338 14035 > 1138 0.0 1333756.2 1.0X > Enable reorder window partition expressions 5981 6030 > 77 0.0 598102.1 2.2X > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org