Le vendredi 16 juillet 2021, 17:37:15 CEST James Coleman a écrit :
> Thanks for hacking on this; as you're not surprised given I made the
> original suggestion, I'm particularly interested in this for
> incremental sort benefits, but I find the other examples you gave
> compelling also.
>
> Of course I haven't seen code yet, but my first intuition is to try to
> avoid adding extra nodes and teach the (hopefully few) relevant nodes
> to remove the resjunk entries themselves. Presumably in this case that
> would mostly be the sort nodes (including gather merge).
>
> One thing to pay attention to here is that we can't necessarily remove
> resjunk entries every time in a sort node since, for example, in
> parallel mode the gather merge node above it will need those entries
> to complete the sort.
Yes that is actually a concern, especially as the merge node is already
handled specially when applying a projection.
>
> I'm interested to see what you're working on with a patch.
I am posting this proof-of-concept, for the record, but I don't think the
numerous problems can be solved easily. I tried to teach Sort to use a limited
sort of projection, but it brings its own slate of problems...
Quick list of problems with the current implementation, leaving aside the fact
that it's quite hacky in a few places:
* result nodes are added for numerous types of non-projection-capable paths,
since the above (final) target includes resjunk columns which should be
eliminated.
* handling of appendrel seems difficult, as both ordered and unordered appends
are generated at the same time against the same target
* I'm having trouble understanding the usefulness of a building physical
tlists for SubqueryScans
The second patch is a very hacky way to try to eliminate some generated result
nodes. The idea is to bypass the whole interpreter when using a "simple"
projection which is just a reduction of the number of columns, and teach Sort
and Result to perform it. To do this, I added a parameter to
is_projection_capable_path to make the test depend on the actual asked target:
for a sort node, only a "simple" projection.
The implementation uses a junkfilter which assumes nothing else than Const and
outer var will be present.
I don't feel like this is going anywhere, but at least it's here for
discussion and posterity, if someone is interested.
--
Ronan Dunklau
commit 31193a9040801965409ff608e222fa5b899d9721
Author: Ronan Dunklau <ronan.dunk...@aiven.io>
Date: Tue Jul 20 11:03:27 2021 +0200
Tag and remove resjunk added for SortGroupClause.
We can provide better plans if we remove the resjunk columns from
the final target. For example, index scan may not need to fetch /
compute the key column.
To achieve that without intruding everywhere, we tag the resjunk
TargetEntries with why they were added, and keep them in the PlannerInfo
processed_tlist until it's time to compute the plan's finaltarget.
This allows us to properly differentiate sort_input_target and the
finaltarget, only adding the needed column for nodes which actually need
them to perform the sort (explicit sorts, aggregates, gathermerge...)
One drawback is that this forces us to add many projection plans on top
of the sort node to finally get the final target.
This can be alleviated in a later patch by teaching indivual
non-projection-capable (for example, sort) to perform a limited form of
projection using the JunkFilter infrastructure.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e81b990092..e89e2ac120 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2323,8 +2323,7 @@ static void
show_sort_keys(SortState *sortstate, List *ancestors, ExplainState *es)
{
Sort *plan = (Sort *) sortstate->ss.ps.plan;
-
- show_sort_group_keys((PlanState *) sortstate, "Sort Key",
+ show_sort_group_keys((PlanState *) outerPlanState(sortstate), "Sort Key",
plan->numCols, 0, plan->sortColIdx,
plan->sortOperators, plan->collations,
plan->nullsFirst,
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 01c110cd2f..87274193be 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -238,7 +238,7 @@ TargetEntry *
makeTargetEntry(Expr *expr,
AttrNumber resno,
char *resname,
- bool resjunk)
+ uint16 resjunk)
{
TargetEntry *tle = makeNode(TargetEntry);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index c13da7a879..2722f9e35b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -2119,6 +2119,11 @@ create_sort_plan(PlannerInfo *root, SortPath *best_path, int flags)
best_path->path.parent->relids : NULL);
copy_generic_path_info(&plan->plan, (Path *) best_path);
+ /*
+ * XXX: This should probably be passed as an arg to make_sort_from_pathkeys
+ * ?
+ */
+ plan->plan.targetlist = make_tlist_from_pathtarget(best_path->path.pathtarget);
return plan;
}
@@ -4233,13 +4238,13 @@ create_nestloop_plan(PlannerInfo *root,
Relids saveOuterRels = root->curOuterRels;
/* NestLoop can project, so no need to be picky about child tlists */
- outer_plan = create_plan_recurse(root, best_path->outerjoinpath, 0);
+ outer_plan = create_plan_recurse(root, best_path->outerjoinpath, CP_EXACT_TLIST);
/* For a nestloop, include outer relids in curOuterRels for inner side */
root->curOuterRels = bms_union(root->curOuterRels,
best_path->outerjoinpath->parent->relids);
- inner_plan = create_plan_recurse(root, best_path->innerjoinpath, 0);
+ inner_plan = create_plan_recurse(root, best_path->innerjoinpath, CP_EXACT_TLIST);
/* Restore curOuterRels */
bms_free(root->curOuterRels);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1868c4eff4..bf0ccf155c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -195,19 +195,23 @@ static RelOptInfo *create_ordered_paths(PlannerInfo *root,
bool target_parallel_safe,
double limit_tuples);
static PathTarget *make_group_input_target(PlannerInfo *root,
- PathTarget *final_target);
+ PathTarget *final_target,
+ List *sortGroupTargetEntries);
static PathTarget *make_partial_grouping_target(PlannerInfo *root,
PathTarget *grouping_target,
+ PathTarget *input_target,
Node *havingQual);
static List *postprocess_setop_tlist(List *new_tlist, List *orig_tlist);
static List *select_active_windows(PlannerInfo *root, WindowFuncLists *wflists);
static PathTarget *make_window_input_target(PlannerInfo *root,
PathTarget *final_target,
+ List *sortGroupTargetEntries,
List *activeWindows);
static List *make_pathkeys_for_window(PlannerInfo *root, WindowClause *wc,
List *tlist);
static PathTarget *make_sort_input_target(PlannerInfo *root,
PathTarget *final_target,
+ List *sortGroupTargetEntries,
bool *have_postponed_srfs);
static void adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
List *targets, List *targets_contain_srfs);
@@ -1345,6 +1349,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ List *filteredFinalTlist = NIL;
+ List *sortGroupTargetEntries = NIL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -1449,9 +1455,19 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* because the target width estimates can use per-Var width numbers
* that were obtained within query_planner().
*/
- final_target = create_pathtarget(root, root->processed_tlist);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *tle = castNode(TargetEntry, lfirst(lc));
+
+ if (!TargetEntryIsSortGroupClauseResjunk(tle))
+ filteredFinalTlist = lappend(filteredFinalTlist, tle);
+ else
+ sortGroupTargetEntries = lappend(sortGroupTargetEntries, tle);
+ }
+ final_target = create_pathtarget(root, filteredFinalTlist);
final_target_parallel_safe =
is_parallel_safe(root, (Node *) final_target->exprs);
+ root->processed_tlist = filteredFinalTlist;
/*
* If ORDER BY was given, consider whether we should use a post-sort
@@ -1462,6 +1478,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
sort_input_target = make_sort_input_target(root,
final_target,
+ sortGroupTargetEntries,
&have_postponed_srfs);
sort_input_target_parallel_safe =
is_parallel_safe(root, (Node *) sort_input_target->exprs);
@@ -1481,6 +1498,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
grouping_target = make_window_input_target(root,
final_target,
+ sortGroupTargetEntries,
activeWindows);
grouping_target_parallel_safe =
is_parallel_safe(root, (Node *) grouping_target->exprs);
@@ -1500,7 +1518,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
parse->hasAggs || root->hasHavingQual);
if (have_grouping)
{
- scanjoin_target = make_group_input_target(root, final_target);
+ scanjoin_target = make_group_input_target(root, final_target,
+ sortGroupTargetEntries);
scanjoin_target_parallel_safe =
is_parallel_safe(root, (Node *) scanjoin_target->exprs);
}
@@ -4119,6 +4138,7 @@ create_one_window_path(PlannerInfo *root,
List *activeWindows)
{
PathTarget *window_target;
+ List *targets;
ListCell *l;
/*
@@ -4137,6 +4157,7 @@ create_one_window_path(PlannerInfo *root,
* for the WindowFuncs computed at each level.
*/
window_target = input_target;
+ targets = make_tlist_from_pathtarget(input_target);
foreach(l, activeWindows)
{
@@ -4147,7 +4168,7 @@ create_one_window_path(PlannerInfo *root,
window_pathkeys = make_pathkeys_for_window(root,
wc,
- root->processed_tlist);
+ targets);
is_sorted = pathkeys_count_contained_in(window_pathkeys,
path->pathkeys,
@@ -4701,7 +4722,7 @@ create_ordered_paths(PlannerInfo *root,
* query_planner().
*/
static PathTarget *
-make_group_input_target(PlannerInfo *root, PathTarget *final_target)
+make_group_input_target(PlannerInfo *root, PathTarget *final_target, List *sortGroupTargetEntries)
{
Query *parse = root->parse;
PathTarget *input_target;
@@ -4743,6 +4764,17 @@ make_group_input_target(PlannerInfo *root, PathTarget *final_target)
i++;
}
+ /* Look if we have to add more entries from the resjunk columns */
+ foreach(lc, sortGroupTargetEntries)
+ {
+ TargetEntry *entry = castNode(TargetEntry, lfirst(lc));
+
+ if (entry->ressortgroupref && get_sortgroupref_clause_noerr(entry->ressortgroupref, parse->groupClause) != NULL)
+ {
+ add_column_to_pathtarget(input_target, entry->expr, entry->ressortgroupref);
+ }
+ }
+
/*
* If there's a HAVING clause, we'll need the Vars it uses, too.
*/
@@ -4784,12 +4816,16 @@ make_group_input_target(PlannerInfo *root, PathTarget *final_target)
* used outside of Aggrefs in the aggregation tlist and HAVING. (Presumably,
* these would be Vars that are grouped by or used in grouping expressions.)
*
+ * We also need to preseve every group column not already there, similar to
+ * make_group_input_target.
+ *
* grouping_target is the tlist to be emitted by the topmost aggregation step.
* havingQual represents the HAVING clause.
*/
static PathTarget *
make_partial_grouping_target(PlannerInfo *root,
PathTarget *grouping_target,
+ PathTarget *input_target,
Node *havingQual)
{
Query *parse = root->parse;
@@ -4829,6 +4865,28 @@ make_partial_grouping_target(PlannerInfo *root,
i++;
}
+ /*
+ * Now scan the input target for grouping columns. Here, we don't care
+ * about non-grouping columns because they will already be in
+ * grouping_target if they are needed.
+ */
+ i = 0;
+ foreach(lc, input_target->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Index sgref = get_pathtarget_sortgroupref(input_target, i);
+
+ if (sgref && parse->groupClause &&
+ get_sortgroupref_clause_noerr(sgref, parse->groupClause) != NULL)
+ {
+ /*
+ * It's a grouping column, so add it to the partial_target as-is.
+ * (This allows the upper agg step to repeat the grouping calcs.)
+ */
+ add_column_to_pathtarget(partial_target, expr, sgref);
+ }
+ }
+
/*
* If there's a HAVING clause, we'll need the Vars/Aggrefs it uses, too.
*/
@@ -5110,6 +5168,7 @@ common_prefix_cmp(const void *a, const void *b)
static PathTarget *
make_window_input_target(PlannerInfo *root,
PathTarget *final_target,
+ List *sortGroupTargetEntries,
List *activeWindows)
{
Query *parse = root->parse;
@@ -5192,6 +5251,21 @@ make_window_input_target(PlannerInfo *root,
i++;
}
+ /*
+ * Now look into targetentries that are not really part of the final rel
+ */
+ foreach(lc, sortGroupTargetEntries)
+ {
+ TargetEntry *entry = castNode(TargetEntry, lfirst(lc));
+ Index sgref = entry->ressortgroupref;
+
+ if (sgref != 0 && bms_is_member(sgref, sgrefs))
+ {
+ add_column_to_pathtarget(input_target, entry->expr, sgref);
+ }
+ }
+
+
/*
* Pull out all the Vars and Aggrefs mentioned in flattenable columns, and
* add them to the input target if not already present. (Some might be
@@ -5322,6 +5396,7 @@ make_pathkeys_for_window(PlannerInfo *root, WindowClause *wc,
static PathTarget *
make_sort_input_target(PlannerInfo *root,
PathTarget *final_target,
+ List *sortGroupTargetEntries,
bool *have_postponed_srfs)
{
Query *parse = root->parse;
@@ -5427,7 +5502,8 @@ make_sort_input_target(PlannerInfo *root,
*/
if (!(postpone_srfs || have_volatile ||
(have_expensive &&
- (parse->limitCount || root->tuple_fraction > 0))))
+ (parse->limitCount || root->tuple_fraction > 0)) ||
+ (list_length(sortGroupTargetEntries) > 0)))
return final_target;
/*
@@ -5460,6 +5536,14 @@ make_sort_input_target(PlannerInfo *root,
i++;
}
+ /* Now also add the missing sort columns */
+ foreach(lc, sortGroupTargetEntries)
+ {
+ TargetEntry *entry = castNode(TargetEntry, lfirst(lc));
+
+ add_column_to_pathtarget(input_target, entry->expr, entry->ressortgroupref);
+ }
+
/*
* Pull out all the Vars, Aggrefs, and WindowFuncs mentioned in
* postponable columns, and add them to the sort-input target if not
@@ -6396,6 +6480,7 @@ create_partial_grouping_paths(PlannerInfo *root,
*/
partially_grouped_rel->reltarget =
make_partial_grouping_target(root, grouped_rel->reltarget,
+ input_rel->reltarget,
extra->havingQual);
if (!extra->partial_costs_set)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 9ce5f95e3b..f864364edb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2711,6 +2711,7 @@ create_projection_path(PlannerInfo *root,
return pathnode;
}
+
/*
* apply_projection_to_path
* Add a projection step, or just apply the target directly to given path.
@@ -2791,12 +2792,27 @@ apply_projection_to_path(PlannerInfo *root,
else
{
GatherMergePath *gmpath = (GatherMergePath *) path;
-
+ ListCell *lc;
+ List *sp_tlist = make_tlist_from_pathtarget(target);
+ List *old_tlist = make_tlist_from_pathtarget(gmpath->subpath->pathtarget);
+ PathTarget *sp_target;
+ /* In the GatherMerge path, we must be careful to not discard the
+ * pathkeys from the subpath.
+ * XXX: maybe this should be computed from the pathkeys directly
+ * instead of relying on ressortgroupref ?
+ */
+ foreach(lc, old_tlist)
+ {
+ TargetEntry * tle = lfirst_node(TargetEntry, lc);
+ if (tle->ressortgroupref && !tlist_member(tle->expr, sp_tlist))
+ sp_tlist = lappend(sp_tlist, tle);
+ }
+ sp_target = make_pathtarget_from_tlist(sp_tlist);
gmpath->subpath = (Path *)
create_projection_path(root,
gmpath->subpath->parent,
gmpath->subpath,
- target);
+ sp_target);
}
}
else if (path->parallel_safe &&
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index c5194fdbbf..eb08802939 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1695,16 +1695,19 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
TargetEntry *tle = (TargetEntry *) lfirst(l);
/*
- * A resjunk column of the subquery can be reflected as
- * resjunk in the physical tlist; we need not punt.
+ * We exclude resjunk columns, as we don't want
+ * to have to deal with them in upper nodes.
*/
- var = makeVarFromTargetEntry(varno, tle);
+ if (!TargetEntryIsResjunk(tle))
+ {
+ var = makeVarFromTargetEntry(varno, tle);
- tlist = lappend(tlist,
- makeTargetEntry((Expr *) var,
- tle->resno,
- NULL,
- tle->resjunk));
+ tlist = lappend(tlist,
+ makeTargetEntry((Expr *) var,
+ tle->resno,
+ NULL,
+ tle->resjunk));
+ }
}
break;
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 311579d059..178316b388 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -662,6 +662,37 @@ copy_pathtarget(PathTarget *src)
return dst;
}
+/*
+ * Check wether a tlist is a subset of another tlist.
+ * That subset can be in a different order, or not.
+ */
+bool
+pathtarget_is_subset(PathTarget *target1, PathTarget *target2)
+{
+ ListCell *lc1;
+
+ foreach(lc1, target1->exprs)
+ {
+ Expr *expr1 = (Expr*) lfirst(lc1);
+ ListCell *lc2;
+ bool found = false;
+
+ foreach(lc2, target2->exprs)
+ {
+ Expr *expr2 = (Expr*) lfirst(lc2);
+
+ if (equal(expr1, expr2))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ return false;
+ }
+ return true;
+}
+
/*
* create_empty_pathtarget
* Create an empty (zero columns, zero cost) PathTarget.
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index 71c360bea5..18b422e22c 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -2096,7 +2096,7 @@ findTargetlistEntrySQL99(ParseState *pstate, Node *node, List **tlist,
* will not be projected into the final tuple.
*/
target_result = transformTargetEntry(pstate, node, expr, exprKind,
- NULL, true);
+ NULL, TARGET_ENTRY_SORTGROUP_CLAUSE);
*tlist = lappend(*tlist, target_result);
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 6e8fbc4780..08fbc07f53 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -79,7 +79,7 @@ transformTargetEntry(ParseState *pstate,
Node *expr,
ParseExprKind exprKind,
char *colname,
- bool resjunk)
+ uint16 resjunk)
{
/* Transform the node if caller didn't do it already */
if (expr == NULL)
@@ -1964,3 +1964,34 @@ FigureColnameInternal(Node *node, char **name)
return strength;
}
+
+/*
+ * Check wether a tlist is a subset of another tlist.
+ * That subset can be in a different order, or not.
+ */
+bool
+tlist_is_subset(List *tlist1, List *tlist2)
+{
+ ListCell *lc1;
+
+ foreach(lc1, tlist1)
+ {
+ TargetEntry *tle1 = lfirst_node(TargetEntry, lc1);
+ ListCell *lc2;
+ bool found = false;
+
+ foreach(lc2, tlist2)
+ {
+ TargetEntry *tle2 = lfirst_node(TargetEntry, lc2);
+
+ if (equal(tle1->expr, tle2->expr))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ return false;
+ }
+ return true;
+}
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 48a7ebfe45..8b32b00914 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -42,7 +42,7 @@ extern Var *makeWholeRowVar(RangeTblEntry *rte,
extern TargetEntry *makeTargetEntry(Expr *expr,
AttrNumber resno,
char *resname,
- bool resjunk);
+ uint16 resjunk);
extern TargetEntry *flatCopyTargetEntry(TargetEntry *src_tle);
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 996c3e4016..3ad4b8a946 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1441,13 +1441,18 @@ typedef struct InferenceElem
* simple reference to a column of a base table (or view). If it is not
* a simple reference, these fields are zeroes.
*
- * If resjunk is true then the column is a working column (such as a sort key)
- * that should be removed from the final output of the query. Resjunk columns
- * must have resnos that cannot duplicate any regular column's resno. Also
- * note that there are places that assume resjunk columns come after non-junk
+ * If resjunk is different from 0 then the column is a working column
+ * (such as a sort key) that should be removed from the final output of the query.
+ * Resjunk columns must have resnos that cannot duplicate any regular column's resno.
+ * When different from zero the value is a flag indicating the resjunk's source
+ * Also note that there are places that assume resjunk columns come after non-junk
* columns.
*--------------------
*/
+
+#define TARGET_ENTRY_REGULAR 0x0000
+#define TARGET_ENTRY_SORTGROUP_CLAUSE 0x0002
+
typedef struct TargetEntry
{
Expr xpr;
@@ -1458,10 +1463,24 @@ typedef struct TargetEntry
* clause */
Oid resorigtbl; /* OID of column's source table */
AttrNumber resorigcol; /* column's number in source table */
- bool resjunk; /* set to true to eliminate the attribute from
- * final target list */
+ uint16 resjunk; /* Flag indicating the resjunk source, 0 if not a resjunk */
} TargetEntry;
+#define TargetEntryIsResjunk(tle) \
+( \
+ (tle)->resjunk != TARGET_ENTRY_REGULAR \
+)
+
+#define TargetEntryIsSortGroupClauseResjunk(tle) \
+( \
+ ((tle)->resjunk & TARGET_ENTRY_SORTGROUP_CLAUSE) != 0 \
+)
+
+#define TargetEntrySetSortGroupClauseResjunk(tle) \
+( \
+ (tle)->resjunk |= TARGET_ENTRY_SORTGROUP_CLAUSE \
+)
+
/* ----------------------------------------------------------------
* node types for join trees
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index d62a09665a..42f27a4a7d 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -24,6 +24,7 @@ extern List *add_to_flat_tlist(List *tlist, List *exprs);
extern List *get_tlist_exprs(List *tlist, bool includeJunk);
extern bool tlist_same_exprs(List *tlist1, List *tlist2);
+extern bool tlist_is_subset(List* tlist1, List *tlist2);
extern bool tlist_same_datatypes(List *tlist, List *colTypes, bool junkOK);
extern bool tlist_same_collations(List *tlist, List *colCollations, bool junkOK);
@@ -39,6 +40,7 @@ extern bool grouping_is_hashable(List *groupClause);
extern PathTarget *make_pathtarget_from_tlist(List *tlist);
extern List *make_tlist_from_pathtarget(PathTarget *target);
extern PathTarget *copy_pathtarget(PathTarget *src);
+extern bool pathtarget_is_subset(PathTarget *target1, PathTarget *target2);
extern PathTarget *create_empty_pathtarget(void);
extern void add_column_to_pathtarget(PathTarget *target,
Expr *expr, Index sortgroupref);
diff --git a/src/include/parser/parse_target.h b/src/include/parser/parse_target.h
index 1a7b1a9277..fa3047032f 100644
--- a/src/include/parser/parse_target.h
+++ b/src/include/parser/parse_target.h
@@ -25,7 +25,7 @@ extern void resolveTargetListUnknowns(ParseState *pstate, List *targetlist);
extern void markTargetListOrigins(ParseState *pstate, List *targetlist);
extern TargetEntry *transformTargetEntry(ParseState *pstate,
Node *node, Node *expr, ParseExprKind exprKind,
- char *colname, bool resjunk);
+ char *colname, uint16 resjunk);
extern Expr *transformAssignedExpr(ParseState *pstate, Expr *expr,
ParseExprKind exprKind,
const char *colname,
diff --git a/src/include/parser/parsetree.h b/src/include/parser/parsetree.h
index 6e1058f1c2..ffe5f34e08 100644
--- a/src/include/parser/parsetree.h
+++ b/src/include/parser/parsetree.h
@@ -50,6 +50,7 @@ extern bool get_rte_attribute_is_dropped(RangeTblEntry *rte,
*/
extern TargetEntry *get_tle_by_resno(List *tlist, AttrNumber resno);
+extern bool tlist_is_subset(List* tlist1, List *tlist2);
/* ----------------
* FOR UPDATE/SHARE info
commit 9ecde9d256d5f865f2cebe1c602bedbd04564da7
Author: Ronan Dunklau <ronan.dunk...@aiven.io>
Date: Tue Jul 20 11:17:40 2021 +0200
Teach sort and result nodes to perform a simplified version of a
projection.
Whenever we just need to remove some columns, we can use the resjunk
infrastructure instead.
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index b099368809..72c9a20aa9 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2323,7 +2323,9 @@ static void
show_sort_keys(SortState *sortstate, List *ancestors, ExplainState *es)
{
Sort *plan = (Sort *) sortstate->ss.ps.plan;
- show_sort_group_keys((PlanState *) outerPlanState(sortstate), "Sort Key",
+ /* Sort target list does not necessarily contain the keys, use them from the
+ * outer plan*/
+ show_sort_group_keys(outerPlanState(sortstate), "Sort Key",
plan->numCols, 0, plan->sortColIdx,
plan->sortOperators, plan->collations,
plan->nullsFirst,
diff --git a/src/backend/executor/execJunk.c b/src/backend/executor/execJunk.c
index 9741897e83..92308b739b 100644
--- a/src/backend/executor/execJunk.c
+++ b/src/backend/executor/execJunk.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "executor/executor.h"
+#include "optimizer/tlist.h"
/*-------------------------------------------------------------------------
* XXX this stuff should be rewritten to take advantage
@@ -200,6 +201,64 @@ ExecInitJunkFilterConversion(List *targetList,
return junkfilter;
}
+JunkFilter *
+ExecInitDiffJunkFilter(List *inputTargetList, List *outputTargetList, TupleTableSlot *slot)
+{
+ JunkFilter * junkfilter;
+ int cleanLength;
+ AttrNumber *cleanMap;
+ ListCell *t;
+ int i;
+ TupleDesc cleanTupType = ExecTypeFromTL(outputTargetList);
+
+ if (equal(inputTargetList, outputTargetList))
+ return NULL;
+
+ /*
+ * Use the given slot, or make a new slot if we weren't given one.
+ */
+ if (!slot)
+ slot = MakeSingleTupleTableSlot(cleanTupType, &TTSOpsVirtual);
+
+ cleanLength = cleanTupType->natts;
+ if (cleanLength > 0)
+ {
+ cleanMap = (AttrNumber *) palloc0(cleanLength * sizeof(AttrNumber));
+ i = 0;
+ foreach(t, outputTargetList)
+ {
+ int j;
+ TargetEntry *outTle = lfirst_node(TargetEntry, t);
+ TargetEntry * inTle;
+ switch (outTle->expr->type)
+ {
+ /* Const are kept as-is, so look it up underneath */
+ case T_Const:
+ inTle = tlist_member(outTle->expr, inputTargetList);
+ j = inTle->resno;
+ break;
+ case T_Var:
+ j = ((Var*) (outTle->expr))->varattno;
+ break;
+ default:
+ elog(ERROR, "Unsupported node type: %d", outTle->expr->type);
+ }
+ cleanMap[i] = j;
+ i++;
+ }
+ } else
+ cleanMap = NULL;
+ junkfilter = makeNode(JunkFilter);
+
+ junkfilter->jf_targetList = outputTargetList;
+ junkfilter->jf_cleanTupType = cleanTupType;
+ junkfilter->jf_cleanMap = cleanMap;
+ junkfilter->jf_resultSlot = slot;
+
+ return junkfilter;
+
+}
+
/*
* ExecFindJunkAttribute
*
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index ad11392b99..d2ca6e3acc 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -522,6 +522,22 @@ ExecGetResultSlotOps(PlanState *planstate, bool *isfixed)
}
+bool
+ExecCanUseJunkFilter(PlanState *planstate)
+{
+ Plan * outerPlan= outerPlan(planstate->plan);
+ return outerPlan && tlist_is_subset(planstate->plan->targetlist, outerPlan->targetlist);
+}
+
+void
+ExecAssignJunkProjection(PlanState *planstate)
+{
+ /* The caller should have checked it first */
+ planstate->ps_junkFilter = ExecInitDiffJunkFilter(outerPlan(planstate->plan)->targetlist,
+ planstate->plan->targetlist,
+ planstate->ps_ResultTupleSlot);
+}
+
/* ----------------
* ExecAssignProjectionInfo
*
diff --git a/src/backend/executor/nodeResult.c b/src/backend/executor/nodeResult.c
index 0946af0a54..2a517637c2 100644
--- a/src/backend/executor/nodeResult.c
+++ b/src/backend/executor/nodeResult.c
@@ -132,8 +132,15 @@ ExecResult(PlanState *pstate)
node->rs_done = true;
}
- /* form the result tuple using ExecProject(), and return it */
- return ExecProject(node->ps.ps_ProjInfo);
+ /* form the result tuple using ExecProject() or ExecFilterJunk(), and return it
+ * Only one of ps_junkFilter or ps_ProjInfo must be set at a time
+ */
+ Assert((node->ps.ps_junkFilter == NULL && node->ps.ps_ProjInfo != NULL) ||
+ (node->ps.ps_junkFilter != NULL && node->ps.ps_ProjInfo == NULL));
+ if (node->ps.ps_junkFilter != NULL)
+ return ExecFilterJunk(node->ps.ps_junkFilter, econtext->ecxt_outertuple);
+ else
+ return ExecProject(node->ps.ps_ProjInfo);
}
return NULL;
@@ -218,7 +225,14 @@ ExecInitResult(Result *node, EState *estate, int eflags)
* Initialize result slot, type and projection.
*/
ExecInitResultTupleSlotTL(&resstate->ps, &TTSOpsVirtual);
- ExecAssignProjectionInfo(&resstate->ps, NULL);
+ /*
+ * If a real projection is needed, assign a projection info.
+ * If we only need a subset of the column, assign a resjunkfilter
+ */
+ if (ExecCanUseJunkFilter(&resstate->ps))
+ ExecAssignJunkProjection(&resstate->ps);
+ else
+ ExecAssignProjectionInfo(&resstate->ps, NULL);
/*
* initialize child expressions
diff --git a/src/backend/executor/nodeSort.c b/src/backend/executor/nodeSort.c
index b99027e0d7..9c98928f70 100644
--- a/src/backend/executor/nodeSort.c
+++ b/src/backend/executor/nodeSort.c
@@ -149,10 +149,12 @@ ExecSort(PlanState *pstate)
* tuples. Note that we only rely on slot tuple remaining valid until the
* next fetch from the tuplesort.
*/
- slot = node->ss.ps.ps_ResultTupleSlot;
+ slot = node->outsortTupleSlot;
(void) tuplesort_gettupleslot(tuplesortstate,
ScanDirectionIsForward(dir),
false, slot, NULL);
+ if (!TTS_EMPTY(slot) && node->ss.ps.ps_junkFilter)
+ slot = ExecFilterJunk(node->ss.ps.ps_junkFilter, slot);
return slot;
}
@@ -221,6 +223,18 @@ ExecInitSort(Sort *node, EState *estate, int eflags)
ExecInitResultTupleSlotTL(&sortstate->ss.ps, &TTSOpsMinimalTuple);
sortstate->ss.ps.ps_ProjInfo = NULL;
+ /* Initialize the junkfilter if any is needed.
+ */
+ sortstate->ss.ps.ps_junkFilter = ExecInitDiffJunkFilter(outerPlan(sortstate->ss.ps.plan)->targetlist,
+ sortstate->ss.ps.plan->targetlist,
+ sortstate->ss.ps.ps_ResultTupleSlot);
+ if (sortstate->ss.ps.ps_junkFilter)
+ sortstate->outsortTupleSlot = ExecInitExtraTupleSlot(estate, sortstate->ss.ss_ScanTupleSlot->tts_tupleDescriptor, &TTSOpsMinimalTuple);
+ else
+ sortstate->outsortTupleSlot = sortstate->ss.ps.ps_ResultTupleSlot;
+
+
+
SO1_printf("ExecInitSort: %s\n",
"sort node initialized");
@@ -243,6 +257,7 @@ ExecEndSort(SortState *node)
ExecClearTuple(node->ss.ss_ScanTupleSlot);
/* must drop pointer to sort result tuple */
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot);
+ ExecClearTuple(node->outsortTupleSlot);
/*
* Release tuplesort resources
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 90f430834f..e8a65d332c 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -1963,7 +1963,7 @@ create_projection_plan(PlannerInfo *root, ProjectionPath *best_path, int flags)
apply_pathtarget_labeling_to_tlist(tlist,
best_path->path.pathtarget);
}
- else if (is_projection_capable_path(best_path->subpath))
+ else if (is_projection_capable_path(best_path->subpath, NULL))
{
/*
* Our caller requires that we return the exact tlist, but no separate
@@ -7028,17 +7028,20 @@ make_modifytable(PlannerInfo *root, Plan *subplan,
/*
* is_projection_capable_path
* Check whether a given Path node is able to do projection.
+ * If the target argument is NULL, the node must be able to perform any
+ * projection, otherwise we check that the given projection is possible.
*/
bool
-is_projection_capable_path(Path *path)
+is_projection_capable_path(Path *path, PathTarget *target)
{
/* Most plan types can project, so just list the ones that can't */
switch (path->pathtype)
{
+ case T_Sort:
+ return target && pathtarget_is_subset(target, path->pathtarget);
case T_Hash:
case T_Material:
case T_Memoize:
- case T_Sort:
case T_IncrementalSort:
case T_Unique:
case T_SetOp:
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bf0ccf155c..de6c8232df 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4519,8 +4519,15 @@ create_ordered_paths(PlannerInfo *root,
limit_tuples);
/* Add projection step if needed */
if (sorted_path->pathtarget != target)
+ {
+ /* In our case, we should raise the target cost by the
+ * previous cost, since all previous columns are also
+ * required before the sort.
+ */
+ target->cost.per_tuple += sorted_path->pathtarget->cost.per_tuple;
sorted_path = apply_projection_to_path(root, ordered_rel,
sorted_path, target);
+ }
add_path(ordered_rel, sorted_path);
}
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index b145c5f45f..46d130cabe 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -770,6 +770,8 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
case T_Material:
case T_Sort:
+ set_upper_references(root, plan, rtoffset);
+ break;
case T_IncrementalSort:
case T_Unique:
case T_SetOp:
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 96440c01ef..2a78a974ab 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2673,7 +2673,7 @@ create_projection_path(PlannerInfo *root,
* conclusion; see comments therein.
*/
oldtarget = subpath->pathtarget;
- if (is_projection_capable_path(subpath) ||
+ if (is_projection_capable_path(subpath, target) ||
equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
@@ -2744,7 +2744,7 @@ apply_projection_to_path(PlannerInfo *root,
* If given path can't project, we might need a Result node, so make a
* separate ProjectionPath.
*/
- if (!is_projection_capable_path(path))
+ if (!is_projection_capable_path(path, target))
return (Path *) create_projection_path(root, rel, path, target);
/*
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 3dc03c913e..7c447e8544 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -159,6 +159,9 @@ extern JunkFilter *ExecInitJunkFilter(List *targetList,
extern JunkFilter *ExecInitJunkFilterConversion(List *targetList,
TupleDesc cleanTupType,
TupleTableSlot *slot);
+extern JunkFilter *ExecInitDiffJunkFilter(List *inputTargetList,
+ List *outputTargetList,
+ TupleTableSlot *slot);
extern AttrNumber ExecFindJunkAttribute(JunkFilter *junkfilter,
const char *attrName);
extern AttrNumber ExecFindJunkAttributeInTlist(List *targetlist,
@@ -551,6 +554,8 @@ extern const TupleTableSlotOps *ExecGetResultSlotOps(PlanState *planstate,
bool *isfixed);
extern void ExecAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc);
+extern bool ExecCanUseJunkFilter(PlanState *planstate);
+extern void ExecAssignJunkProjection(PlanState *planstate);
extern void ExecConditionalAssignProjectionInfo(PlanState *planstate,
TupleDesc inputDesc, Index varno);
extern void ExecFreeExprContext(PlanState *planstate);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 105180764e..3674c8f4f0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1004,6 +1004,7 @@ typedef struct PlanState
TupleTableSlot *ps_ResultTupleSlot; /* slot for my result tuples */
ExprContext *ps_ExprContext; /* node's expression-evaluation context */
ProjectionInfo *ps_ProjInfo; /* info for doing tuple projection */
+ JunkFilter *ps_junkFilter;
bool async_capable; /* true if node is async-capable */
@@ -2151,6 +2152,7 @@ typedef struct SortState
int64 bound_Done; /* value of bound we did the sort with */
void *tuplesortstate; /* private state of tuplesort.c */
bool am_worker; /* are we a worker? */
+ TupleTableSlot *outsortTupleSlot;
SharedSortInfo *shared_info; /* one entry per worker */
} SortState;
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index bf1adfc52a..ac5d769e4b 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -46,7 +46,7 @@ extern ForeignScan *make_foreignscan(List *qptlist, List *qpqual,
extern Plan *change_plan_targetlist(Plan *subplan, List *tlist,
bool tlist_parallel_safe);
extern Plan *materialize_finished_plan(Plan *subplan);
-extern bool is_projection_capable_path(Path *path);
+extern bool is_projection_capable_path(Path *path, PathTarget *target);
extern bool is_projection_capable_plan(Plan *plan);
/* External use of these functions is deprecated: */
