From 294d80904065298b1643831738bacc2c0af9fca9 Mon Sep 17 00:00:00 2001 From: zhenglong li Date: Sat, 15 Aug 2026 13:55:00 +0800 Subject: [PATCH v2 2/2] Consider window functions when reordering GROUP BY items preprocess_groupclause() reorders the GROUP BY items to match the query's ORDER BY clause so that a single sort operation can serve both the grouping step and the final ordering. However, when the query contains window functions, the sort performed directly above the grouping step is the one required by the first window clause, not the one for ORDER BY, which is only performed above the WindowAgg nodes. Unless the window's sort requirements happened to coincide with a prefix of the ORDER BY clause, an additional sort was needed to provide the first WindowAgg with correctly ordered input. Teach preprocess_groupclause() to instead match the GROUP BY items against the first active window's PARTITION BY and ORDER BY keys when there are window functions, then against the query's ORDER BY clause for any remaining items. We match the first window only because it alone can share a sort with the grouping step: select_active_windows() has already fixed the windows' evaluation order by this point, placing the window with the strongest sort requirements first, directly above the grouping step. Satisfying its requirements may satisfy subsequent windows too, since windows whose requirements form a prefix of another's are evaluated later. For the same reason, if the first window requires no sort at all then no active window does, and matching the ORDER BY clause remains as useful as it is without window functions. Appending the query's ORDER BY keys after the window's keys orders the otherwise-arbitrary tail of the GROUP BY items. Since WindowAgg nodes preserve their input ordering, this can additionally save the final sort, and it retains the previous behavior for queries where the window's sort requirements form a prefix of the ORDER BY clause, which the ORDER BY matching alone already handled well. For example, the query SELECT a, b, count(*) OVER (PARTITION BY b) FROM t GROUP BY a, b; previously sorted twice: once by (a, b) for the grouping step and again by (b) for the WindowAgg. Reordering the GROUP BY items to (b, a) allows a single sort to serve both. With an ORDER BY that does not lead with the window's sort keys, such as adding "ORDER BY a" to the query above, the sort count similarly drops from three to two. Matching on the ordering alone is not enough, though, because the GROUP BY items and the window's sort clauses get their sort directions independently. The parser makes a GROUP BY item take on the sort operator and NULLS ordering of a matching ORDER BY item, and a PARTITION BY item those of a matching item in its window's ORDER BY, but nothing relates a GROUP BY item to the window's ORDER BY. So with SELECT a, b, c, count(*) OVER (PARTITION BY b ORDER BY c DESC) FROM t GROUP BY a, b, c; the GROUP BY item c sorts ascending and would not match the window's c DESC, leaving the WindowAgg to re-sort its input incrementally. Since grouping only requires equal values to be adjacent, the sort direction of a GROUP BY item is as insignificant as its position in the list. Therefore, match a GROUP BY item to a sort clause whenever both refer to the same targetlist entry and use the same equality operator, and make the item take on the sort clause's sort operator and NULLS ordering (using a modified copy, so as not to scribble on the parse tree). In the query above, a single sort by (b, c DESC, a) then serves both steps. The equality operator must match because it determines what the groups are; e.g., an ORDER BY ... USING clause may have given the GROUP BY item a different one, in which case we stop matching as before. Likewise, the sort operator is only adopted from the btree operator family of the GROUP BY item's own: another family sharing the equality operator need not order consistently with it. For instance, under a nondeterministic collation text_pattern_ops' ~<~ compares bytes, so sorting by it might not bring equal values together. With sort clauses no longer required to be identical to match, the list to match against may refer to the same targetlist entry more than once, e.g. when the window's ORDER BY and the query's ORDER BY sort the same column in different directions. A later reference using the same equality operator is redundant, as the earlier sort key already brought the equal values together, so we skip it and continue matching instead of stopping. This also affects queries without window functions, where an ORDER BY can contain such a reference through a USING clause, as in "ORDER BY x USING ~<~, x, y", allowing the GROUP BY items after it to be matched too. Adopting the window's sort direction changes the GROUP BY pathkeys, so an input path sorted in the original direction, such as an index scan, could no longer be used to group without a sort. The preceding commit made the reordering of GROUP BY keys to match the input path insensitive to sort direction, which keeps such plans available. Note that, like the long-standing ORDER BY matching, this heuristic is insensitive to column cardinalities, as is the sort cost model. Moving a low-cardinality window key to the front of a large grouping sort defeats the leading-key specializations in tuplesort and can cost more than the eliminated sort saved when the number of groups is small relative to the input. This is mitigated in practice because a small number of groups is also when hash aggregation tends to win, whereas sorted aggregation tends to be chosen when groups are numerous, which is exactly when the eliminated sort is large. Choosing the ordering based on cost is left for future work in get_useful_group_keys_orderings(). The preprocessing of a plain GROUP BY clause is moved until after select_active_windows() has determined the evaluation order of the windows. This is safe since nothing in between examines root->processed_groupClause. Suggested-by: Henson Choi --- src/backend/optimizer/path/pathkeys.c | 5 +- src/backend/optimizer/plan/planner.c | 161 +++++++-- .../regress/expected/collate.icu.utf8.out | 30 ++ src/test/regress/expected/window.out | 330 ++++++++++++++++++ src/test/regress/sql/collate.icu.utf8.sql | 12 + src/test/regress/sql/window.sql | 149 ++++++++ 6 files changed, 653 insertions(+), 34 deletions(-) diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c index d0c3ff86ff5..71ad905ef77 100644 --- a/src/backend/optimizer/path/pathkeys.c +++ b/src/backend/optimizer/path/pathkeys.c @@ -548,8 +548,9 @@ group_keys_reorder_by_pathkeys(List *pathkeys, List **group_pathkeys, * * The function considers (and keeps) following GROUP BY orderings: * - * - GROUP BY keys as ordered by preprocess_groupclause() to match target - * ORDER BY clause (as much as possible), + * - GROUP BY keys as ordered and directed by preprocess_groupclause() to + * match the target ORDER BY clause or the first window's sort order (as + * much as possible), * - GROUP BY keys reordered to match 'path' ordering (as much as possible), * taking on the sort direction and NULLS ordering of 'path' where they * differ from those of the GROUP BY keys. diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index edaa42d713a..220bbce49e6 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -170,7 +170,8 @@ static void preprocess_rowmarks(PlannerInfo *root); static double preprocess_limit(PlannerInfo *root, double tuple_fraction, int64 *offset_est, int64 *count_est); -static List *preprocess_groupclause(PlannerInfo *root, List *force); +static List *preprocess_groupclause(PlannerInfo *root, List *force, + List *activeWindows); static List *extract_rollup_sets(List *groupingSets); static List *reorder_grouping_sets(List *groupingSets, List *sortclause); static void standard_qp_callback(PlannerInfo *root, void *extra); @@ -1854,16 +1855,9 @@ grouping_planner(PlannerInfo *root, double tuple_fraction, /* A recursive query should always have setOperations */ Assert(!root->hasRecursion); - /* Preprocess grouping sets and GROUP BY clause, if any */ + /* Preprocess grouping sets, if any */ if (parse->groupingSets) - { gset_data = preprocess_grouping_sets(root); - } - else if (parse->groupClause) - { - /* Preprocess regular GROUP BY clause, if any */ - root->processed_groupClause = preprocess_groupclause(root, NIL); - } /* * Preprocess targetlist. Note that much of the remaining planning @@ -1916,6 +1910,17 @@ grouping_planner(PlannerInfo *root, double tuple_fraction, parse->hasWindowFuncs = false; } + /* + * Preprocess a plain GROUP BY clause, if any. We do this after + * selecting the active windows so that preprocess_groupclause can try + * to match the ordering of the GROUP BY elements to the sort order + * required by the first window, if there is one. (Grouping sets were + * already handled by preprocess_grouping_sets, above.) + */ + if (parse->groupClause && !parse->groupingSets) + root->processed_groupClause = preprocess_groupclause(root, NIL, + activeWindows); + /* * Preprocess MIN/MAX aggregates, if any. Note: be careful about * adding logic between here and the query_planner() call. Anything @@ -2613,7 +2618,7 @@ preprocess_grouping_sets(PlannerInfo *root) * The groupClauses for hashed grouping sets are built later on.) */ if (gs->set) - rollup->groupClause = preprocess_groupclause(root, gs->set); + rollup->groupClause = preprocess_groupclause(root, gs->set, NIL); else rollup->groupClause = NIL; @@ -3114,6 +3119,22 @@ limit_needed(Query *parse) * We also consider partial match between GROUP BY and ORDER BY elements, * which could allow to implement ORDER BY using the incremental sort. * + * When the query contains window functions, the sort performed directly + * above the grouping step is the one required by the first window clause + * rather than the one for ORDER BY (which is performed above the window + * functions). In that case we try to match the sort order required by the + * first of the 'activeWindows' (ie its PARTITION BY keys followed by its + * ORDER BY keys) first, and then the query's ORDER BY clause, since the + * WindowAgg steps may preserve the grouping step's output ordering up to + * the final sort. Callers passing a non-NIL 'force' list need not bother + * passing activeWindows. + * + * The GROUP BY elements also take on the sort direction and NULLS ordering + * of the sort clauses they are matched to, as long as the sort operator + * belongs to the same btree opfamily, which is semantically insignificant as + * well, since grouping depends only on the equality operator. The parser + * has already done that for ORDER BY, but not for the window clauses. + * * We also consider other orderings of the GROUP BY elements, which could * match the sort ordering of other possible plans (eg an indexscan) and * thereby reduce cost. This is implemented during the generation of grouping @@ -3122,9 +3143,11 @@ limit_needed(Query *parse) * Note: we need no comparable processing of the distinctClause because * the parser already enforced that that matches ORDER BY. * - * Note: we return a fresh List, but its elements are the same - * SortGroupClauses appearing in parse->groupClause. This is important - * because later processing may modify the processed_groupClause list. + * Note: we return a fresh List, whose elements are the SortGroupClauses + * appearing in parse->groupClause, except that a GROUP BY item whose sort + * ordering we change to match a window's sort clause is replaced by a + * modified copy. Returning a fresh List is important because later + * processing may modify the processed_groupClause list. * * For grouping sets, the order of items is instead forced to agree with that * of the grouping set (and items not in the grouping set are skipped). The @@ -3132,10 +3155,12 @@ limit_needed(Query *parse) * possible is done elsewhere. */ static List * -preprocess_groupclause(PlannerInfo *root, List *force) +preprocess_groupclause(PlannerInfo *root, List *force, List *activeWindows) { Query *parse = root->parse; List *new_groupclause = NIL; + List *matchClause; + Bitmapset *matched_refs = NULL; ListCell *sl; ListCell *gl; @@ -3153,32 +3178,104 @@ preprocess_groupclause(PlannerInfo *root, List *force) return new_groupclause; } - /* If no ORDER BY, nothing useful to do here */ - if (parse->sortClause == NIL) + /* + * Select the list of sort clauses that we'll try to match the GROUP BY + * elements to. Without window functions, this is simply the ORDER BY + * clause. + * + * If there are any active windows, the sort directly above the grouping + * step is the one for the first window, so match its PARTITION BY keys + * followed by its ORDER BY keys first. + * + * We then append the query's ORDER BY clause. The ordering of any GROUP + * BY items beyond the first window's sort requirements is otherwise + * arbitrary, and since the WindowAgg steps preserve their input ordering + * (as long as the remaining windows require no additional sorts, which is + * the case whenever their sort requirements are prefixes of the first + * window's, per select_active_windows()), making the tail match the ORDER + * BY can save the final sort too. This also covers the case where no + * active window requires any sort at all. + */ + if (activeWindows != NIL) + { + WindowClause *wc = linitial_node(WindowClause, activeWindows); + + matchClause = list_concat_copy(wc->partitionClause, wc->orderClause); + matchClause = list_concat(matchClause, parse->sortClause); + } + else + matchClause = parse->sortClause; + + /* If nothing to match against, nothing useful to do here */ + if (matchClause == NIL) return list_copy(parse->groupClause); /* - * Scan the ORDER BY clause and construct a list of matching GROUP BY + * Scan the clauses to match and construct a list of matching GROUP BY * items, but only as far as we can make a matching prefix. * - * This code assumes that the sortClause contains no duplicate items. + * A GROUP BY item matches a sort clause if both refer to the same + * targetlist entry and use the same equality operator. The sort + * direction doesn't matter for grouping, which only needs equal values to + * be adjacent. The parser already makes GROUP BY items take on the sort + * ordering of matching ORDER BY items, but the window clauses get theirs + * independently: PARTITION BY items follow the window's ORDER BY, while + * the window's ORDER BY items are as the user wrote them. So when the + * ordering differs, we make the GROUP BY item adopt the ordering of the + * sort clause we match it to. We must not scribble on the parse tree's + * SortGroupClause, so use a modified copy in that case. + * + * We only adopt a sort operator from the same btree opfamily, though. The + * opfamily's sort operators are consistent with its equality operator, + * but a sort operator from another opfamily need not be, even when the + * two opfamilies share the equality operator: under a nondeterministic + * collation, text_pattern_ops' ~<~ compares bytes whereas = does not, so + * ~<~ wouldn't bring all equal values together. + * + * The list may refer to the same targetlist entry more than once, for + * instance when the window's ORDER BY and the query's ORDER BY sort the + * same column in different directions. A later reference using the same + * equality operator is redundant, as the earlier sort key already brought + * its equal values together, so just skip over it. */ - foreach(sl, parse->sortClause) + foreach(sl, matchClause) { SortGroupClause *sc = lfirst_node(SortGroupClause, sl); + SortGroupClause *gc; + + gc = get_sortgroupref_clause_noerr(sc->tleSortGroupRef, + parse->groupClause); + if (gc == NULL || gc->eqop != sc->eqop || !OidIsValid(sc->sortop)) + break; /* no match, so stop scanning */ + + if (bms_is_member(gc->tleSortGroupRef, matched_refs)) + continue; /* redundant with an earlier match */ - foreach(gl, parse->groupClause) + if (gc->sortop != sc->sortop) { - SortGroupClause *gc = lfirst_node(SortGroupClause, gl); + Oid gc_opfamily; + Oid sc_opfamily; + Oid opcintype; + CompareType cmptype; + + if (!get_ordering_op_properties(gc->sortop, &gc_opfamily, + &opcintype, &cmptype) || + !get_ordering_op_properties(sc->sortop, &sc_opfamily, + &opcintype, &cmptype) || + gc_opfamily != sc_opfamily) + break; /* different opfamily, so stop scanning */ + } - if (equal(gc, sc)) - { - new_groupclause = lappend(new_groupclause, gc); - break; - } + if (gc->sortop != sc->sortop || gc->nulls_first != sc->nulls_first) + { + gc = copyObject(gc); + gc->sortop = sc->sortop; + gc->reverse_sort = sc->reverse_sort; + gc->nulls_first = sc->nulls_first; } - if (gl == NULL) - break; /* no match, so stop scanning */ + + new_groupclause = lappend(new_groupclause, gc); + matched_refs = bms_add_member(matched_refs, gc->tleSortGroupRef); } @@ -3196,8 +3293,8 @@ preprocess_groupclause(PlannerInfo *root, List *force) { SortGroupClause *gc = lfirst_node(SortGroupClause, gl); - if (list_member_ptr(new_groupclause, gc)) - continue; /* it matched an ORDER BY item */ + if (bms_is_member(gc->tleSortGroupRef, matched_refs)) + continue; /* it matched a sort clause item */ if (!OidIsValid(gc->sortop)) /* give up, GROUP BY can't be sorted */ return list_copy(parse->groupClause); new_groupclause = lappend(new_groupclause, gc); @@ -4589,7 +4686,7 @@ consider_groupingsets_paths(PlannerInfo *root, { rollup = makeNode(RollupData); - rollup->groupClause = preprocess_groupclause(root, gset); + rollup->groupClause = preprocess_groupclause(root, gset, NIL); rollup->gsets_data = list_make1(gs); rollup->gsets = remap_to_groupclause_idx(rollup->groupClause, rollup->gsets_data, @@ -4778,7 +4875,7 @@ consider_groupingsets_paths(PlannerInfo *root, Assert(gs->set != NIL); - rollup->groupClause = preprocess_groupclause(root, gs->set); + rollup->groupClause = preprocess_groupclause(root, gs->set, NIL); rollup->gsets_data = list_make1(gs); rollup->gsets = remap_to_groupclause_idx(rollup->groupClause, rollup->gsets_data, diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out index 69b2f5109ad..33da1d8bef9 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8.out @@ -2046,6 +2046,36 @@ SELECT x FROM test1ci INTERSECT SELECT x FROM test2ci ORDER BY x; ghi (2 rows) +-- GROUP BY must not adopt a window's sort operator from another opfamily: +-- text_pattern_ops' ~<~ sorts 'A' and 'a' apart although they are equal +SET enable_hashagg TO off; +EXPLAIN (COSTS OFF) +SELECT x, count(*), count(*) OVER (ORDER BY x USING ~<~) +FROM (VALUES ('a' COLLATE case_insensitive), ('B'), ('A'), ('b'), ('a')) v(x) +GROUP BY x; + QUERY PLAN +--------------------------------------------------------------------------- + WindowAgg + Window: w1 AS (ORDER BY "*VALUES*".column1) + -> Sort + Sort Key: "*VALUES*".column1 COLLATE case_insensitive USING ~<~ + -> GroupAggregate + Group Key: "*VALUES*".column1 + -> Sort + Sort Key: "*VALUES*".column1 COLLATE case_insensitive + -> Values Scan on "*VALUES*" +(9 rows) + +SELECT x, count(*), count(*) OVER (ORDER BY x USING ~<~) +FROM (VALUES ('a' COLLATE case_insensitive), ('B'), ('A'), ('b'), ('a')) v(x) +GROUP BY x; + x | count | count +---+-------+------- + B | 2 | 1 + a | 3 | 2 +(2 rows) + +RESET enable_hashagg; SELECT x FROM test2ci INTERSECT SELECT x FROM test1ci ORDER BY x; x ----- diff --git a/src/test/regress/expected/window.out b/src/test/regress/expected/window.out index c0bde1c5eec..c5a0fe899b6 100644 --- a/src/test/regress/expected/window.out +++ b/src/test/regress/expected/window.out @@ -4839,6 +4839,336 @@ WHERE first_emp = 1 OR last_emp = 1; sales | 4 | 4800 | 08-08-2007 | 3 | 1 (6 rows) +-- Test reordering of the GROUP BY keys to match the first window's sort +-- order +SET enable_hashagg TO off; +-- Ensure the GROUP BY keys are reordered to match the sort order of the +-- window's PARTITION BY clause, allowing the sort for the grouping step to +-- also provide the sorted input required by the WindowAgg. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, depname; + QUERY PLAN +----------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY depname) + -> GroupAggregate + Group Key: depname, empno + -> Sort + Sort Key: depname, empno + -> Seq Scan on empsalary +(7 rows) + +-- As above, but include the window's ORDER BY keys too. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno) c +FROM empsalary +GROUP BY empno, depname; + QUERY PLAN +------------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY depname ORDER BY empno) + -> Group + Group Key: depname, empno + -> Sort + Sort Key: depname, empno + -> Seq Scan on empsalary +(7 rows) + +-- Ensure the GROUP BY keys are matched to the window's sort order in +-- preference to the query's ORDER BY, since the latter sort is performed +-- above the WindowAgg. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, depname +ORDER BY empno; + QUERY PLAN +----------------------------------------------- + Sort + Sort Key: empno + -> WindowAgg + Window: w1 AS (PARTITION BY depname) + -> GroupAggregate + Group Key: depname, empno + -> Sort + Sort Key: depname, empno + -> Seq Scan on empsalary +(9 rows) + +-- Ensure the GROUP BY keys remaining after matching the window's sort order +-- are ordered to match the query's ORDER BY, allowing the final sort to be +-- avoided when the WindowAgg preserves the grouping step's output ordering. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY depname, enroll_date; + QUERY PLAN +----------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY depname) + -> GroupAggregate + Group Key: depname, enroll_date, empno + -> Sort + Sort Key: depname, enroll_date, empno + -> Seq Scan on empsalary +(7 rows) + +-- As above, but with an ORDER BY that does not lead with the window's sort +-- keys. Ensure the remaining GROUP BY keys still follow the query's ORDER +-- BY even though the final sort cannot be avoided. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY enroll_date, empno; + QUERY PLAN +----------------------------------------------------------- + Sort + Sort Key: enroll_date, empno + -> WindowAgg + Window: w1 AS (PARTITION BY depname) + -> GroupAggregate + Group Key: depname, enroll_date, empno + -> Sort + Sort Key: depname, enroll_date, empno + -> Seq Scan on empsalary +(9 rows) + +-- Ensure the GROUP BY keys are still matched to the query's ORDER BY when +-- the window imposes no sort order of its own. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER () salarysum +FROM empsalary +GROUP BY depname, empno +ORDER BY empno, depname; + QUERY PLAN +----------------------------------------- + WindowAgg + Window: w1 AS () + -> GroupAggregate + Group Key: empno, depname + -> Sort + Sort Key: empno, depname + -> Seq Scan on empsalary +(7 rows) + +-- Ensure the GROUP BY keys adopt the sort direction of the window's ORDER BY +-- keys, so that a single sort still serves both steps. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno DESC) c +FROM empsalary +GROUP BY empno, depname; + QUERY PLAN +------------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY depname ORDER BY empno) + -> Group + Group Key: depname, empno + -> Sort + Sort Key: depname, empno DESC + -> Seq Scan on empsalary +(7 rows) + +-- As above, but with NULLS FIRST in the window's ORDER BY. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno NULLS FIRST) c +FROM empsalary +GROUP BY empno, depname; + QUERY PLAN +------------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY depname ORDER BY empno) + -> Group + Group Key: depname, empno + -> Sort + Sort Key: depname, empno NULLS FIRST + -> Seq Scan on empsalary +(7 rows) + +-- Ensure the window's sort direction is preferred even when the query's +-- ORDER BY has given the GROUP BY key a different one. The final sort can +-- then be done incrementally. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno DESC) c +FROM empsalary +GROUP BY empno, depname +ORDER BY depname, empno; + QUERY PLAN +------------------------------------------------------------- + Incremental Sort + Sort Key: depname, empno + Presorted Key: depname + -> WindowAgg + Window: w1 AS (PARTITION BY depname ORDER BY empno) + -> Group + Group Key: depname, empno + -> Sort + Sort Key: depname, empno DESC + -> Seq Scan on empsalary +(10 rows) + +-- Likewise for a PARTITION BY key whose direction differs from that given +-- to the GROUP BY key by the query's ORDER BY. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno) c +FROM empsalary +GROUP BY empno, depname +ORDER BY depname DESC; + QUERY PLAN +------------------------------------------------------------- + Sort + Sort Key: depname DESC + -> WindowAgg + Window: w1 AS (PARTITION BY depname ORDER BY empno) + -> Group + Group Key: depname, empno + -> Sort + Sort Key: depname, empno + -> Seq Scan on empsalary +(9 rows) + +-- Ensure the GROUP BY key does not adopt the window's sort ordering when +-- the query's ORDER BY has given it a different equality operator. +EXPLAIN (COSTS OFF) +SELECT r, count(*) OVER (ORDER BY r DESC) c +FROM (SELECT row(depname, empno) AS r FROM empsalary) s +GROUP BY r +ORDER BY r USING *<; + QUERY PLAN +---------------------------------------------------------------------------------------- + Sort + Sort Key: (ROW(empsalary.depname, empsalary.empno)) USING *< + -> WindowAgg + Window: w1 AS (ORDER BY (ROW(empsalary.depname, empsalary.empno))) + -> Sort + Sort Key: (ROW(empsalary.depname, empsalary.empno)) DESC + -> Group + Group Key: (ROW(empsalary.depname, empsalary.empno)) + -> Sort + Sort Key: (ROW(empsalary.depname, empsalary.empno)) USING *< + -> Seq Scan on empsalary +(11 rows) + +-- Ensure the GROUP BY keys do not adopt a window's ORDER BY that uses a +-- different operator family, even with the same equality operator. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY empno ORDER BY depname USING ~<~) c +FROM empsalary +GROUP BY depname, empno; + QUERY PLAN +------------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY empno ORDER BY depname) + -> Incremental Sort + Sort Key: empno, depname USING ~<~ + Presorted Key: empno + -> Group + Group Key: empno, depname + -> Sort + Sort Key: empno, depname + -> Seq Scan on empsalary +(10 rows) + +-- Ensure a later sort clause referencing an already-matched GROUP BY key +-- with the same equality operator is skipped as redundant rather than +-- ending the match, even without window functions. +EXPLAIN (COSTS OFF) +SELECT empno, enroll_date, depname +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY depname USING ~<~, depname, enroll_date, empno; + QUERY PLAN +--------------------------------------------------------- + Group + Group Key: depname, enroll_date, empno + -> Sort + Sort Key: depname USING ~<~, enroll_date, empno + -> Seq Scan on empsalary +(5 rows) + +CREATE TABLE groupwin (x int, y int, z int); +INSERT INTO groupwin + SELECT i % 4, i % 5, CASE WHEN i % 7 = 0 THEN NULL ELSE i % 3 END + FROM generate_series(1, 1000) i; +CREATE INDEX groupwin_x_y_idx ON groupwin (x, y); +ANALYZE groupwin; +-- Ensure the GROUP BY keys adopting the window's sort direction doesn't +-- prevent grouping directly off a presorted input, then sorting just the +-- groups for the window. +EXPLAIN (COSTS OFF) +SELECT x, y, count(*), count(*) OVER (PARTITION BY x ORDER BY y DESC) c +FROM groupwin +GROUP BY x, y; + QUERY PLAN +---------------------------------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY x ORDER BY y) + -> Incremental Sort + Sort Key: x, y DESC + Presorted Key: x + -> GroupAggregate + Group Key: x, y + -> Index Only Scan using groupwin_x_y_idx on groupwin +(8 rows) + +-- Check the results with a descending window ORDER BY over NULLs. +EXPLAIN (COSTS OFF) +SELECT x, z, count(*), + sum(count(*)) OVER (PARTITION BY x ORDER BY z DESC NULLS LAST) s +FROM groupwin +GROUP BY x, z; + QUERY PLAN +---------------------------------------------- + WindowAgg + Window: w1 AS (PARTITION BY x ORDER BY z) + -> GroupAggregate + Group Key: x, z + -> Sort + Sort Key: x, z DESC NULLS LAST + -> Seq Scan on groupwin +(7 rows) + +SELECT x, z, count(*), + sum(count(*)) OVER (PARTITION BY x ORDER BY z DESC NULLS LAST) s +FROM groupwin +GROUP BY x, z +ORDER BY x, z DESC NULLS LAST; + x | z | count | s +---+---+-------+----- + 0 | 2 | 71 | 71 + 0 | 1 | 72 | 143 + 0 | 0 | 72 | 215 + 0 | | 35 | 250 + 1 | 2 | 72 | 72 + 1 | 1 | 72 | 144 + 1 | 0 | 71 | 215 + 1 | | 35 | 250 + 2 | 2 | 72 | 72 + 2 | 1 | 71 | 143 + 2 | 0 | 71 | 214 + 2 | | 36 | 250 + 3 | 2 | 71 | 71 + 3 | 1 | 71 | 142 + 3 | 0 | 72 | 214 + 3 | | 36 | 250 +(16 rows) + +DROP TABLE groupwin; +RESET enable_hashagg; CREATE INDEX empsalary_salary_empno_idx ON empsalary (salary, empno); SET enable_seqscan = 0; -- Ensure no sorting is done and that the IndexScan maintains all pathkeys diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql index 87cea263780..eed796871f7 100644 --- a/src/test/regress/sql/collate.icu.utf8.sql +++ b/src/test/regress/sql/collate.icu.utf8.sql @@ -738,6 +738,18 @@ SELECT x FROM test3ci WHERE x ~ 'a'; SELECT x FROM test1ci UNION SELECT x FROM test2ci ORDER BY x; SELECT x FROM test2ci UNION SELECT x FROM test1ci ORDER BY x; SELECT x FROM test1ci INTERSECT SELECT x FROM test2ci ORDER BY x; + +-- GROUP BY must not adopt a window's sort operator from another opfamily: +-- text_pattern_ops' ~<~ sorts 'A' and 'a' apart although they are equal +SET enable_hashagg TO off; +EXPLAIN (COSTS OFF) +SELECT x, count(*), count(*) OVER (ORDER BY x USING ~<~) +FROM (VALUES ('a' COLLATE case_insensitive), ('B'), ('A'), ('b'), ('a')) v(x) +GROUP BY x; +SELECT x, count(*), count(*) OVER (ORDER BY x USING ~<~) +FROM (VALUES ('a' COLLATE case_insensitive), ('B'), ('A'), ('b'), ('a')) v(x) +GROUP BY x; +RESET enable_hashagg; SELECT x FROM test2ci INTERSECT SELECT x FROM test1ci ORDER BY x; SELECT x FROM test1ci EXCEPT SELECT x FROM test2ci; SELECT x FROM test2ci EXCEPT SELECT x FROM test1ci; diff --git a/src/test/regress/sql/window.sql b/src/test/regress/sql/window.sql index 8e6f92d94c7..707ce2069f0 100644 --- a/src/test/regress/sql/window.sql +++ b/src/test/regress/sql/window.sql @@ -1659,6 +1659,155 @@ SELECT * FROM FROM empsalary) emp WHERE first_emp = 1 OR last_emp = 1; +-- Test reordering of the GROUP BY keys to match the first window's sort +-- order +SET enable_hashagg TO off; + +-- Ensure the GROUP BY keys are reordered to match the sort order of the +-- window's PARTITION BY clause, allowing the sort for the grouping step to +-- also provide the sorted input required by the WindowAgg. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, depname; + +-- As above, but include the window's ORDER BY keys too. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno) c +FROM empsalary +GROUP BY empno, depname; + +-- Ensure the GROUP BY keys are matched to the window's sort order in +-- preference to the query's ORDER BY, since the latter sort is performed +-- above the WindowAgg. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, depname +ORDER BY empno; + +-- Ensure the GROUP BY keys remaining after matching the window's sort order +-- are ordered to match the query's ORDER BY, allowing the final sort to be +-- avoided when the WindowAgg preserves the grouping step's output ordering. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY depname, enroll_date; + +-- As above, but with an ORDER BY that does not lead with the window's sort +-- keys. Ensure the remaining GROUP BY keys still follow the query's ORDER +-- BY even though the final sort cannot be avoided. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER (PARTITION BY depname) depminsum +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY enroll_date, empno; + +-- Ensure the GROUP BY keys are still matched to the query's ORDER BY when +-- the window imposes no sort order of its own. +EXPLAIN (COSTS OFF) +SELECT empno, depname, min(salary) minsalary, + sum(min(salary)) OVER () salarysum +FROM empsalary +GROUP BY depname, empno +ORDER BY empno, depname; + +-- Ensure the GROUP BY keys adopt the sort direction of the window's ORDER BY +-- keys, so that a single sort still serves both steps. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno DESC) c +FROM empsalary +GROUP BY empno, depname; + +-- As above, but with NULLS FIRST in the window's ORDER BY. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno NULLS FIRST) c +FROM empsalary +GROUP BY empno, depname; + +-- Ensure the window's sort direction is preferred even when the query's +-- ORDER BY has given the GROUP BY key a different one. The final sort can +-- then be done incrementally. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno DESC) c +FROM empsalary +GROUP BY empno, depname +ORDER BY depname, empno; + +-- Likewise for a PARTITION BY key whose direction differs from that given +-- to the GROUP BY key by the query's ORDER BY. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY depname ORDER BY empno) c +FROM empsalary +GROUP BY empno, depname +ORDER BY depname DESC; + +-- Ensure the GROUP BY key does not adopt the window's sort ordering when +-- the query's ORDER BY has given it a different equality operator. +EXPLAIN (COSTS OFF) +SELECT r, count(*) OVER (ORDER BY r DESC) c +FROM (SELECT row(depname, empno) AS r FROM empsalary) s +GROUP BY r +ORDER BY r USING *<; + +-- Ensure the GROUP BY keys do not adopt a window's ORDER BY that uses a +-- different operator family, even with the same equality operator. +EXPLAIN (COSTS OFF) +SELECT empno, depname, + count(*) OVER (PARTITION BY empno ORDER BY depname USING ~<~) c +FROM empsalary +GROUP BY depname, empno; + +-- Ensure a later sort clause referencing an already-matched GROUP BY key +-- with the same equality operator is skipped as redundant rather than +-- ending the match, even without window functions. +EXPLAIN (COSTS OFF) +SELECT empno, enroll_date, depname +FROM empsalary +GROUP BY empno, enroll_date, depname +ORDER BY depname USING ~<~, depname, enroll_date, empno; + +CREATE TABLE groupwin (x int, y int, z int); +INSERT INTO groupwin + SELECT i % 4, i % 5, CASE WHEN i % 7 = 0 THEN NULL ELSE i % 3 END + FROM generate_series(1, 1000) i; +CREATE INDEX groupwin_x_y_idx ON groupwin (x, y); +ANALYZE groupwin; + +-- Ensure the GROUP BY keys adopting the window's sort direction doesn't +-- prevent grouping directly off a presorted input, then sorting just the +-- groups for the window. +EXPLAIN (COSTS OFF) +SELECT x, y, count(*), count(*) OVER (PARTITION BY x ORDER BY y DESC) c +FROM groupwin +GROUP BY x, y; + +-- Check the results with a descending window ORDER BY over NULLs. +EXPLAIN (COSTS OFF) +SELECT x, z, count(*), + sum(count(*)) OVER (PARTITION BY x ORDER BY z DESC NULLS LAST) s +FROM groupwin +GROUP BY x, z; +SELECT x, z, count(*), + sum(count(*)) OVER (PARTITION BY x ORDER BY z DESC NULLS LAST) s +FROM groupwin +GROUP BY x, z +ORDER BY x, z DESC NULLS LAST; + +DROP TABLE groupwin; + +RESET enable_hashagg; + CREATE INDEX empsalary_salary_empno_idx ON empsalary (salary, empno); SET enable_seqscan = 0; -- 2.50.1 (Apple Git-155)