Wan Kun created SPARK-41368:
-------------------------------
Summary: Reorder the window partition expressions by expression
stats
Key: SPARK-41368
URL: https://issues.apache.org/jira/browse/SPARK-41368
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.4.0
Reporter: Wan Kun
We can reorder window partition expressions by the distinct values stats.
Sorting with high cardinality will be faster.
Benchmark result:
{code:java}
Window: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Without reorder window partition expressions 13338 14035
1138 0.0 1333756.2 1.0X
With reorder window partition expressions 5981 6030
77 0.0 598102.1 2.2X
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
