On Mon, Mar 26, 2018 at 5:24 PM, Ashutosh Bapat <
ashutosh.ba...@enterprisedb.com> wrote:
> On Fri, Mar 23, 2018 at 4:35 PM, Jeevan Chalke
> <jeevan.cha...@enterprisedb.com> wrote:
> >
> > Changes related to postgres_fdw which allows pushing aggregate on the
> > foreign server is not yet committed. Due to this, we will end up getting
> an
> > error when we have foreign partitions + aggregation.
> >
> > Attached 0001 patch here (0006 from my earlier patch-set) which adds
> support
> > for this and thus will not have any error.
>
I have observed that, target member in GroupPathExtraData is not needed as
we store the target in grouped_rel itself.
Attached 0001 patch to remove that.
>
>
> else if (IS_UPPER_REL(foreignrel))
> {
> PgFdwRelationInfo *ofpinfo;
> - PathTarget *ptarget = root->upper_targets[UPPERREL_
> GROUP_AGG];
> + PathTarget *ptarget = fpinfo->grouped_target;
>
> I think we need an assert there to make sure that the upper relation is a
> grouping relation. That way any future push down will notice it.
>
I am not sure on what we should Assetrt here. Note that we end-up here only
when doing grouping, and thus I don't think we need any Assert here.
Let me know if I missed anything.
>
> - get_agg_clause_costs(root, (Node *)
> root->parse->havingQual,
> + get_agg_clause_costs(root, fpinfo->havingQual,
> AGGSPLIT_SIMPLE, &aggcosts);
> }
> Should we pass agg costs as well through GroupPathExtraData to avoid
> calculating it again in this function?
>
Adding an extra member in GroupPathExtraData just for FDW does not look
good to me.
But yes, if we do that, then we can save this calculation.
Let me know if its OK to have an extra member for just FDW use, will
prepare a separate patch for that.
>
> /*
> + /*
> + * Store passed-in target and havingQual in fpinfo. If its a foreign
> + * partition, then path target and HAVING quals fetched from the root
> are
> + * not correct as Vars within it won't match with this child relation.
> + * However, server passed them through extra and thus fetch from it.
> + */
> + if (extra)
> + {
> + /* Partial aggregates are not supported. */
> + Assert(extra->patype != PARTITIONWISE_AGGREGATE_PARTIAL);
> +
> + fpinfo->grouped_target = extra->target;
> + fpinfo->havingQual = extra->havingQual;
> + }
> + else
> + {
> + fpinfo->grouped_target = root->upper_targets[UPPERREL_GROUP_AGG];
> + fpinfo->havingQual = parse->havingQual;
> + }
> I think both these cases, extra should always be present whether a child
> relation or a parent relation. Just pick from extra always.
>
Yes.
Done.
>
> /* Grouping information */
> List *grouped_tlist;
> + PathTarget *grouped_target;
>
> We should use the target stored in the grouped rel directly.
>
Yep.
>
> + Node *havingQual;
> I am wondering whether we could use remote_conds member for storing this.
>
This havingQual is later checked for shippability and classified into
pushable and non-pushable quals and stored in remote_conds and local_conds
respectively.
Storing it directly in remote_conds and then splitting it does not look
good to me.
Also, remote_conds is list of RestrictInfo nodes whereas havingQual is not.
So using that for storing havingQual does not make sense. So better to have
a separate member in PgFdwRelationInfo.
> /*
> * Likewise, copy the relids that are represented by this foreign
> scan. An
> - * upper rel doesn't have relids set, but it covers all the base
> relations
> - * participating in the underlying scan, so use root's all_baserels.
> + * upper rel (but not the other upper rel) doesn't have relids set,
> but it
> + * covers all the base relations participating in the underlying
> scan, so
> + * use root's all_baserels.
> */
>
> This is correct only for "other" grouping relations. We are yet to
> decide what to do
> for the other upper relations.
>
I have removed this comment change as existing comments look good after
doing following changes:
>
> - if (IS_UPPER_REL(rel))
> + if (IS_UPPER_REL(rel) && !IS_OTHER_REL(rel))
> I guess, this condition boils down to rel->reloptkind == RELOPT_UPPER_REL.
> Use
> it that way?
>
Done.
Attached 0002 for this.
Thanks
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>
--
Jeevan Chalke
Technical Architect, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
From dabe9c41402cdc8dc4afd3bb01202de654e43bf4 Mon Sep 17 00:00:00 2001
From: Jeevan Chalke <jeevan.cha...@enterprisedb.com>
Date: Tue, 27 Mar 2018 14:23:00 +0530
Subject: [PATCH 1/2] Remove target from GroupPathExtraData, instead fetch it
from grouped_rel.
---
src/backend/optimizer/plan/planner.c | 4 +---
src/include/nodes/relation.h | 2 --
2 files changed, 1 insertion(+), 5 deletions(-)
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 52c21e6..3bd63f3 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -3731,7 +3731,6 @@ create_grouping_paths(PlannerInfo *root,
flags |= GROUPING_CAN_PARTIAL_AGG;
extra.flags = flags;
- extra.target = target;
extra.target_parallel_safe = target_parallel_safe;
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
@@ -6906,7 +6905,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
int cnt_parts;
List *grouped_live_children = NIL;
List *partially_grouped_live_children = NIL;
- PathTarget *target = extra->target;
+ PathTarget *target = grouped_rel->reltarget;
Assert(patype != PARTITIONWISE_AGGREGATE_NONE);
Assert(patype != PARTITIONWISE_AGGREGATE_PARTIAL ||
@@ -6940,7 +6939,6 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
adjust_appendrel_attrs(root,
(Node *) target->exprs,
nappinfos, appinfos);
- child_extra.target = child_target;
/* Translate havingQual and targetList. */
child_extra.havingQual = (Node *)
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index abbbda9..2b4f773 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -2340,7 +2340,6 @@ typedef enum
* have been initialized.
* agg_partial_costs gives partial aggregation costs.
* agg_final_costs gives finalization costs.
- * target is the PathTarget to be used while creating paths.
* target_parallel_safe is true if target is parallel safe.
* havingQual gives list of quals to be applied after aggregation.
* targetList gives list of columns to be projected.
@@ -2355,7 +2354,6 @@ typedef struct
AggClauseCosts agg_final_costs;
/* Data which may differ across partitions. */
- PathTarget *target;
bool target_parallel_safe;
Node *havingQual;
List *targetList;
--
1.8.3.1
From 00f1f87d48ff1aebeb6e16305375faaa96b34684 Mon Sep 17 00:00:00 2001
From: Jeevan Chalke <jeevan.cha...@enterprisedb.com>
Date: Tue, 27 Mar 2018 14:23:34 +0530
Subject: [PATCH 2/2] Teach postgres_fdw to push aggregates for child relations
too.
GetForeignUpperPaths() now takes an extra void parameter which will
be used to pass any additional details required to create an upper
path at the remote server. However, we support only grouping over
remote server today and thus it passes grouping specific details i.e.
GroupPathExtraData, NULL otherwise.
Since we don't know how to get a partially aggregated result from a
remote server, only full aggregation is pushed on the remote server.
---
contrib/postgres_fdw/expected/postgres_fdw.out | 132 +++++++++++++++++++++++++
contrib/postgres_fdw/postgres_fdw.c | 37 ++++---
contrib/postgres_fdw/postgres_fdw.h | 1 +
contrib/postgres_fdw/sql/postgres_fdw.sql | 51 ++++++++++
doc/src/sgml/fdwhandler.sgml | 8 +-
src/backend/optimizer/plan/createplan.c | 2 +-
src/backend/optimizer/plan/planner.c | 29 +++---
src/backend/optimizer/prep/prepunion.c | 2 +-
src/include/foreign/fdwapi.h | 3 +-
src/include/optimizer/planner.h | 3 +-
10 files changed, 238 insertions(+), 30 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 2d6e387..a211aa9 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -7852,3 +7852,135 @@ SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE
(14 rows)
RESET enable_partitionwise_join;
+-- ===================================================================
+-- test partitionwise aggregates
+-- ===================================================================
+CREATE TABLE pagg_tab (a int, b int, c text) PARTITION BY RANGE(a);
+CREATE TABLE pagg_tab_p1 (LIKE pagg_tab);
+CREATE TABLE pagg_tab_p2 (LIKE pagg_tab);
+CREATE TABLE pagg_tab_p3 (LIKE pagg_tab);
+INSERT INTO pagg_tab_p1 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 10;
+INSERT INTO pagg_tab_p2 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 20 and (i % 30) >= 10;
+INSERT INTO pagg_tab_p3 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 30 and (i % 30) >= 20;
+-- Create foreign partitions
+CREATE FOREIGN TABLE fpagg_tab_p1 PARTITION OF pagg_tab FOR VALUES FROM (0) TO (10) SERVER loopback OPTIONS (table_name 'pagg_tab_p1');
+CREATE FOREIGN TABLE fpagg_tab_p2 PARTITION OF pagg_tab FOR VALUES FROM (10) TO (20) SERVER loopback OPTIONS (table_name 'pagg_tab_p2');;
+CREATE FOREIGN TABLE fpagg_tab_p3 PARTITION OF pagg_tab FOR VALUES FROM (20) TO (30) SERVER loopback OPTIONS (table_name 'pagg_tab_p3');;
+ANALYZE pagg_tab;
+ANALYZE fpagg_tab_p1;
+ANALYZE fpagg_tab_p2;
+ANALYZE fpagg_tab_p3;
+-- When GROUP BY clause matches with PARTITION KEY.
+-- Plan with partitionwise aggregates is disabled
+SET enable_partitionwise_aggregate TO false;
+EXPLAIN (COSTS OFF)
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+ QUERY PLAN
+-------------------------------------------------------
+ Sort
+ Sort Key: fpagg_tab_p1.a
+ -> HashAggregate
+ Group Key: fpagg_tab_p1.a
+ Filter: (avg(fpagg_tab_p1.b) < '22'::numeric)
+ -> Append
+ -> Foreign Scan on fpagg_tab_p1
+ -> Foreign Scan on fpagg_tab_p2
+ -> Foreign Scan on fpagg_tab_p3
+(9 rows)
+
+-- Plan with partitionwise aggregates is enabled
+SET enable_partitionwise_aggregate TO true;
+EXPLAIN (COSTS OFF)
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Sort
+ Sort Key: fpagg_tab_p1.a
+ -> Append
+ -> Foreign Scan
+ Relations: Aggregate on (public.fpagg_tab_p1 pagg_tab)
+ -> Foreign Scan
+ Relations: Aggregate on (public.fpagg_tab_p2 pagg_tab)
+ -> Foreign Scan
+ Relations: Aggregate on (public.fpagg_tab_p3 pagg_tab)
+(9 rows)
+
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+ a | sum | min | count
+----+------+-----+-------
+ 0 | 2000 | 0 | 100
+ 1 | 2100 | 1 | 100
+ 10 | 2000 | 0 | 100
+ 11 | 2100 | 1 | 100
+ 20 | 2000 | 0 | 100
+ 21 | 2100 | 1 | 100
+(6 rows)
+
+-- Check with whole-row reference
+-- Should have all the columns in the target list for the given relation
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT a, count(t1) FROM pagg_tab t1 GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+ QUERY PLAN
+------------------------------------------------------------------------
+ Sort
+ Output: t1.a, (count(((t1.*)::pagg_tab)))
+ Sort Key: t1.a
+ -> Append
+ -> HashAggregate
+ Output: t1.a, count(((t1.*)::pagg_tab))
+ Group Key: t1.a
+ Filter: (avg(t1.b) < '22'::numeric)
+ -> Foreign Scan on public.fpagg_tab_p1 t1
+ Output: t1.a, t1.*, t1.b
+ Remote SQL: SELECT a, b, c FROM public.pagg_tab_p1
+ -> HashAggregate
+ Output: t1_1.a, count(((t1_1.*)::pagg_tab))
+ Group Key: t1_1.a
+ Filter: (avg(t1_1.b) < '22'::numeric)
+ -> Foreign Scan on public.fpagg_tab_p2 t1_1
+ Output: t1_1.a, t1_1.*, t1_1.b
+ Remote SQL: SELECT a, b, c FROM public.pagg_tab_p2
+ -> HashAggregate
+ Output: t1_2.a, count(((t1_2.*)::pagg_tab))
+ Group Key: t1_2.a
+ Filter: (avg(t1_2.b) < '22'::numeric)
+ -> Foreign Scan on public.fpagg_tab_p3 t1_2
+ Output: t1_2.a, t1_2.*, t1_2.b
+ Remote SQL: SELECT a, b, c FROM public.pagg_tab_p3
+(25 rows)
+
+SELECT a, count(t1) FROM pagg_tab t1 GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+ a | count
+----+-------
+ 0 | 100
+ 1 | 100
+ 10 | 100
+ 11 | 100
+ 20 | 100
+ 21 | 100
+(6 rows)
+
+-- When GROUP BY clause does not match with PARTITION KEY.
+EXPLAIN (COSTS OFF)
+SELECT b, avg(a), max(a), count(*) FROM pagg_tab GROUP BY b HAVING sum(a) < 700 ORDER BY 1;
+ QUERY PLAN
+------------------------------------------------------
+ Sort
+ Sort Key: fpagg_tab_p1.b
+ -> Finalize HashAggregate
+ Group Key: fpagg_tab_p1.b
+ Filter: (sum(fpagg_tab_p1.a) < 700)
+ -> Append
+ -> Partial HashAggregate
+ Group Key: fpagg_tab_p1.b
+ -> Foreign Scan on fpagg_tab_p1
+ -> Partial HashAggregate
+ Group Key: fpagg_tab_p2.b
+ -> Foreign Scan on fpagg_tab_p2
+ -> Partial HashAggregate
+ Group Key: fpagg_tab_p3.b
+ -> Foreign Scan on fpagg_tab_p3
+(15 rows)
+
+-- Clean-up
+RESET enable_partitionwise_aggregate;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index e8a0d54..ec4416b 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -352,7 +352,8 @@ static bool postgresRecheckForeignScan(ForeignScanState *node,
static void postgresGetForeignUpperPaths(PlannerInfo *root,
UpperRelationKind stage,
RelOptInfo *input_rel,
- RelOptInfo *output_rel);
+ RelOptInfo *output_rel,
+ void *extra);
/*
* Helper functions
@@ -427,7 +428,8 @@ static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
Path *epq_path);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
- RelOptInfo *grouped_rel);
+ RelOptInfo *grouped_rel,
+ GroupPathExtraData *extra);
static void apply_server_options(PgFdwRelationInfo *fpinfo);
static void apply_table_options(PgFdwRelationInfo *fpinfo);
static void merge_fdw_options(PgFdwRelationInfo *fpinfo,
@@ -2775,7 +2777,7 @@ estimate_path_cost_size(PlannerInfo *root,
else if (IS_UPPER_REL(foreignrel))
{
PgFdwRelationInfo *ofpinfo;
- PathTarget *ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
+ PathTarget *ptarget = foreignrel->reltarget;
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
@@ -2805,7 +2807,7 @@ estimate_path_cost_size(PlannerInfo *root,
{
get_agg_clause_costs(root, (Node *) fpinfo->grouped_tlist,
AGGSPLIT_SIMPLE, &aggcosts);
- get_agg_clause_costs(root, (Node *) root->parse->havingQual,
+ get_agg_clause_costs(root, fpinfo->havingQual,
AGGSPLIT_SIMPLE, &aggcosts);
}
@@ -5020,8 +5022,8 @@ static bool
foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel)
{
Query *query = root->parse;
- PathTarget *grouping_target = root->upper_targets[UPPERREL_GROUP_AGG];
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
+ PathTarget *grouping_target = grouped_rel->reltarget;
PgFdwRelationInfo *ofpinfo;
List *aggvars;
ListCell *lc;
@@ -5131,11 +5133,11 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel)
* Classify the pushable and non-pushable HAVING clauses and save them in
* remote_conds and local_conds of the grouped rel's fpinfo.
*/
- if (root->hasHavingQual && query->havingQual)
+ if (root->hasHavingQual && fpinfo->havingQual)
{
ListCell *lc;
- foreach(lc, (List *) query->havingQual)
+ foreach(lc, (List *) fpinfo->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
RestrictInfo *rinfo;
@@ -5232,7 +5234,8 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel)
*/
static void
postgresGetForeignUpperPaths(PlannerInfo *root, UpperRelationKind stage,
- RelOptInfo *input_rel, RelOptInfo *output_rel)
+ RelOptInfo *input_rel, RelOptInfo *output_rel,
+ void *extra)
{
PgFdwRelationInfo *fpinfo;
@@ -5252,7 +5255,8 @@ postgresGetForeignUpperPaths(PlannerInfo *root, UpperRelationKind stage,
fpinfo->pushdown_safe = false;
output_rel->fdw_private = fpinfo;
- add_foreign_grouping_paths(root, input_rel, output_rel);
+ add_foreign_grouping_paths(root, input_rel, output_rel,
+ (GroupPathExtraData *) extra);
}
/*
@@ -5264,13 +5268,13 @@ postgresGetForeignUpperPaths(PlannerInfo *root, UpperRelationKind stage,
*/
static void
add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
- RelOptInfo *grouped_rel)
+ RelOptInfo *grouped_rel,
+ GroupPathExtraData *extra)
{
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
ForeignPath *grouppath;
- PathTarget *grouping_target;
double rows;
int width;
Cost startup_cost;
@@ -5281,7 +5285,14 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
!root->hasHavingQual)
return;
- grouping_target = root->upper_targets[UPPERREL_GROUP_AGG];
+ Assert(extra->patype == PARTITIONWISE_AGGREGATE_NONE ||
+ extra->patype == PARTITIONWISE_AGGREGATE_FULL);
+
+ /*
+ * Get HAVING qual from extra. In case of child partition, it will have
+ * translated Vars.
+ */
+ fpinfo->havingQual = extra->havingQual;
/* save the input_rel as outerrel in fpinfo */
fpinfo->outerrel = input_rel;
@@ -5312,7 +5323,7 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
- grouping_target,
+ grouped_rel->reltarget,
rows,
startup_cost,
total_cost,
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index d37cc88..488c538 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -94,6 +94,7 @@ typedef struct PgFdwRelationInfo
/* Grouping information */
List *grouped_tlist;
+ Node *havingQual;
/* Subquery information */
bool make_outerrel_subquery; /* do we deparse outerrel as a
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 4d2e43c..cf32be4 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -1932,3 +1932,54 @@ SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE
SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE a % 25 = 0) t1 FULL JOIN (SELECT 't2_phv' phv, * FROM fprt2 WHERE b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY t1.a, t2.b;
RESET enable_partitionwise_join;
+
+
+-- ===================================================================
+-- test partitionwise aggregates
+-- ===================================================================
+
+CREATE TABLE pagg_tab (a int, b int, c text) PARTITION BY RANGE(a);
+
+CREATE TABLE pagg_tab_p1 (LIKE pagg_tab);
+CREATE TABLE pagg_tab_p2 (LIKE pagg_tab);
+CREATE TABLE pagg_tab_p3 (LIKE pagg_tab);
+
+INSERT INTO pagg_tab_p1 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 10;
+INSERT INTO pagg_tab_p2 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 20 and (i % 30) >= 10;
+INSERT INTO pagg_tab_p3 SELECT i % 30, i % 50, to_char(i/30, 'FM0000') FROM generate_series(1, 3000) i WHERE (i % 30) < 30 and (i % 30) >= 20;
+
+-- Create foreign partitions
+CREATE FOREIGN TABLE fpagg_tab_p1 PARTITION OF pagg_tab FOR VALUES FROM (0) TO (10) SERVER loopback OPTIONS (table_name 'pagg_tab_p1');
+CREATE FOREIGN TABLE fpagg_tab_p2 PARTITION OF pagg_tab FOR VALUES FROM (10) TO (20) SERVER loopback OPTIONS (table_name 'pagg_tab_p2');;
+CREATE FOREIGN TABLE fpagg_tab_p3 PARTITION OF pagg_tab FOR VALUES FROM (20) TO (30) SERVER loopback OPTIONS (table_name 'pagg_tab_p3');;
+
+ANALYZE pagg_tab;
+ANALYZE fpagg_tab_p1;
+ANALYZE fpagg_tab_p2;
+ANALYZE fpagg_tab_p3;
+
+-- When GROUP BY clause matches with PARTITION KEY.
+-- Plan with partitionwise aggregates is disabled
+SET enable_partitionwise_aggregate TO false;
+EXPLAIN (COSTS OFF)
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+
+-- Plan with partitionwise aggregates is enabled
+SET enable_partitionwise_aggregate TO true;
+EXPLAIN (COSTS OFF)
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+SELECT a, sum(b), min(b), count(*) FROM pagg_tab GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+
+-- Check with whole-row reference
+-- Should have all the columns in the target list for the given relation
+EXPLAIN (VERBOSE, COSTS OFF)
+SELECT a, count(t1) FROM pagg_tab t1 GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+SELECT a, count(t1) FROM pagg_tab t1 GROUP BY a HAVING avg(b) < 22 ORDER BY 1;
+
+-- When GROUP BY clause does not match with PARTITION KEY.
+EXPLAIN (COSTS OFF)
+SELECT b, avg(a), max(a), count(*) FROM pagg_tab GROUP BY b HAVING sum(a) < 700 ORDER BY 1;
+
+
+-- Clean-up
+RESET enable_partitionwise_aggregate;
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 0ed3a47..25915a2 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -359,7 +359,8 @@ void
GetForeignUpperPaths(PlannerInfo *root,
UpperRelationKind stage,
RelOptInfo *input_rel,
- RelOptInfo *output_rel);
+ RelOptInfo *output_rel,
+ void *extra);
</programlisting>
Create possible access paths for <firstterm>upper relation</firstterm> processing,
which is the planner's term for all post-scan/join query processing, such
@@ -379,7 +380,10 @@ GetForeignUpperPaths(PlannerInfo *root,
currently being considered. <literal>output_rel</literal> is the upper relation
that should receive paths representing computation of this step,
and <literal>input_rel</literal> is the relation representing the input to this
- step. (Note that <structname>ForeignPath</structname> paths added
+ step. <literal>extra</literal> parameter provides additional details which may
+ be needed for the paths creation, like child details in case of partitioning.
+ The details passed through this parameter is depends on the <literal>stage</literal>
+ parameter. (Note that <structname>ForeignPath</structname> paths added
to <literal>output_rel</literal> would typically not have any direct dependency
on paths of the <literal>input_rel</literal>, since their processing is expected
to be done externally. However, examining paths previously generated for
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 8b4f031..8bd15c9 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -3520,7 +3520,7 @@ create_foreignscan_plan(PlannerInfo *root, ForeignPath *best_path,
* upper rel doesn't have relids set, but it covers all the base relations
* participating in the underlying scan, so use root's all_baserels.
*/
- if (IS_UPPER_REL(rel))
+ if (rel->reloptkind == RELOPT_UPPER_REL)
scan_plan->fs_relids = root->all_baserels;
else
scan_plan->fs_relids = best_path->path.parent->relids;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 3bd63f3..30d6c28 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -2205,12 +2205,13 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
if (final_rel->fdwroutine &&
final_rel->fdwroutine->GetForeignUpperPaths)
final_rel->fdwroutine->GetForeignUpperPaths(root, UPPERREL_FINAL,
- current_rel, final_rel);
+ current_rel, final_rel,
+ NULL);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_FINAL,
- current_rel, final_rel);
+ current_rel, final_rel, NULL);
/* Note: currently, we leave it to callers to do set_cheapest() */
}
@@ -4023,12 +4024,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if (grouped_rel->fdwroutine &&
grouped_rel->fdwroutine->GetForeignUpperPaths)
grouped_rel->fdwroutine->GetForeignUpperPaths(root, UPPERREL_GROUP_AGG,
- input_rel, grouped_rel);
+ input_rel, grouped_rel,
+ extra);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_GROUP_AGG,
- input_rel, grouped_rel);
+ input_rel, grouped_rel,
+ extra);
}
/*
@@ -4460,12 +4463,13 @@ create_window_paths(PlannerInfo *root,
if (window_rel->fdwroutine &&
window_rel->fdwroutine->GetForeignUpperPaths)
window_rel->fdwroutine->GetForeignUpperPaths(root, UPPERREL_WINDOW,
- input_rel, window_rel);
+ input_rel, window_rel,
+ NULL);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_WINDOW,
- input_rel, window_rel);
+ input_rel, window_rel, NULL);
/* Now choose the best path(s) */
set_cheapest(window_rel);
@@ -4764,12 +4768,13 @@ create_distinct_paths(PlannerInfo *root,
if (distinct_rel->fdwroutine &&
distinct_rel->fdwroutine->GetForeignUpperPaths)
distinct_rel->fdwroutine->GetForeignUpperPaths(root, UPPERREL_DISTINCT,
- input_rel, distinct_rel);
+ input_rel, distinct_rel,
+ NULL);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_DISTINCT,
- input_rel, distinct_rel);
+ input_rel, distinct_rel, NULL);
/* Now choose the best path(s) */
set_cheapest(distinct_rel);
@@ -4907,12 +4912,13 @@ create_ordered_paths(PlannerInfo *root,
if (ordered_rel->fdwroutine &&
ordered_rel->fdwroutine->GetForeignUpperPaths)
ordered_rel->fdwroutine->GetForeignUpperPaths(root, UPPERREL_ORDERED,
- input_rel, ordered_rel);
+ input_rel, ordered_rel,
+ NULL);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_ORDERED,
- input_rel, ordered_rel);
+ input_rel, ordered_rel, NULL);
/*
* No need to bother with set_cheapest here; grouping_planner does not
@@ -6693,7 +6699,8 @@ create_partial_grouping_paths(PlannerInfo *root,
fdwroutine->GetForeignUpperPaths(root,
UPPERREL_PARTIAL_GROUP_AGG,
- input_rel, partially_grouped_rel);
+ input_rel, partially_grouped_rel,
+ extra);
}
return partially_grouped_rel;
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 6e510f9..5236ab3 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -1032,7 +1032,7 @@ postprocess_setop_rel(PlannerInfo *root, RelOptInfo *rel)
*/
if (create_upper_paths_hook)
(*create_upper_paths_hook) (root, UPPERREL_SETOP,
- NULL, rel);
+ NULL, rel, NULL);
/* Select cheapest path */
set_cheapest(rel);
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index e88fee3..ea83c7b 100644
--- a/src/include/foreign/fdwapi.h
+++ b/src/include/foreign/fdwapi.h
@@ -62,7 +62,8 @@ typedef void (*GetForeignJoinPaths_function) (PlannerInfo *root,
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
RelOptInfo *input_rel,
- RelOptInfo *output_rel);
+ RelOptInfo *output_rel,
+ void *extra);
typedef void (*AddForeignUpdateTargets_function) (Query *parsetree,
RangeTblEntry *target_rte,
diff --git a/src/include/optimizer/planner.h b/src/include/optimizer/planner.h
index 0d8b88d..07a3bc0 100644
--- a/src/include/optimizer/planner.h
+++ b/src/include/optimizer/planner.h
@@ -28,7 +28,8 @@ extern PGDLLIMPORT planner_hook_type planner_hook;
typedef void (*create_upper_paths_hook_type) (PlannerInfo *root,
UpperRelationKind stage,
RelOptInfo *input_rel,
- RelOptInfo *output_rel);
+ RelOptInfo *output_rel,
+ void *extra);
extern PGDLLIMPORT create_upper_paths_hook_type create_upper_paths_hook;
--
1.8.3.1
