Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
On 2015-07-17 19:57:22 +0100, Andrew Gierth wrote: > Attached is the current version of my fix (with Jeevan's regression > tests plus one of mine). Pushed, thanks for the report and fix! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
On 2015-07-17 11:37:26 +0530, Jeevan Chalke wrote: > However I wonder why we are supporting GROUPING SETS inside GROUPING SETS. > On Oracle, it is throwing an error. > We are not trying to be Oracle compatible, but just curious to know. The SQL specification seems to be pretty unambigous about supporting nested grouping set specifications. Check 7.9 : (nested inside a ) can contain . Regards, Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
Hello, At Mon, 20 Jul 2015 15:45:21 +0530, Jeevan Chalke wrote in > On Sat, Jul 18, 2015 at 12:27 AM, Andrew Gierth > wrote: > > > > "Kyotaro" == Kyotaro HORIGUCHI > > writes: > > > > Kyotaro> Hello, this looks to be a kind of thinko. The attached patch > > Kyotaro> fixes it. > > > > No, that's still wrong. Just knowing that there is a List is not enough > > to tell whether to concat it or append it. Thank you. I've missed the non-grouping-set cases. > > Jeevan's original patch tries to get around this by making the RowExpr > > case wrap another List around its result (which is then removed by the > > concat), but this is the wrong approach too because it breaks nested > > RowExprs (which isn't valid syntax in the spec, because the spec allows > > only column references in GROUP BY, not arbitrary expressions, but which > > we have no reason not to support). > > > Attached is the current version of my fix (with Jeevan's regression > > tests plus one of mine). > > > > Looks good to me. It also looks for me to work as expected and to be in good shape. The two foreach loops for T_GroupingSet and T_List became to look very simiar but they don't seem can be merged in reasonable shape. regards, -- Kyotaro Horiguchi NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
On Sat, Jul 18, 2015 at 12:27 AM, Andrew Gierth wrote: > > "Kyotaro" == Kyotaro HORIGUCHI > writes: > > Kyotaro> Hello, this looks to be a kind of thinko. The attached patch > Kyotaro> fixes it. > > No, that's still wrong. Just knowing that there is a List is not enough > to tell whether to concat it or append it. > > Jeevan's original patch tries to get around this by making the RowExpr > case wrap another List around its result (which is then removed by the > concat), but this is the wrong approach too because it breaks nested > RowExprs (which isn't valid syntax in the spec, because the spec allows > only column references in GROUP BY, not arbitrary expressions, but which > we have no reason not to support). > > Attached is the current version of my fix (with Jeevan's regression > tests plus one of mine). > Looks good to me. > > -- > Andrew (irc:RhodiumToad) > > -- Jeevan B Chalke Principal Software Engineer, Product Development EnterpriseDB Corporation The Enterprise PostgreSQL Company
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
> "Kyotaro" == Kyotaro HORIGUCHI writes:
Kyotaro> Hello, this looks to be a kind of thinko. The attached patch
Kyotaro> fixes it.
No, that's still wrong. Just knowing that there is a List is not enough
to tell whether to concat it or append it.
Jeevan's original patch tries to get around this by making the RowExpr
case wrap another List around its result (which is then removed by the
concat), but this is the wrong approach too because it breaks nested
RowExprs (which isn't valid syntax in the spec, because the spec allows
only column references in GROUP BY, not arbitrary expressions, but which
we have no reason not to support).
Attached is the current version of my fix (with Jeevan's regression
tests plus one of mine).
--
Andrew (irc:RhodiumToad)
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e90e1d6..5a48a02 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1734,7 +1734,7 @@ findTargetlistEntrySQL99(ParseState *pstate, Node *node, List **tlist,
* Inside a grouping set (ROLLUP, CUBE, or GROUPING SETS), we expect the
* content to be nested no more than 2 deep: i.e. ROLLUP((a,b),(c,d)) is
* ok, but ROLLUP((a,(b,c)),d) is flattened to ((a,b,c),d), which we then
- * normalize to ((a,b,c),(d)).
+ * (later) normalize to ((a,b,c),(d)).
*
* CUBE or ROLLUP can be nested inside GROUPING SETS (but not the reverse),
* and we leave that alone if we find it. But if we see GROUPING SETS inside
@@ -1803,9 +1803,16 @@ flatten_grouping_sets(Node *expr, bool toplevel, bool *hasGroupingSets)
foreach(l2, gset->content)
{
- Node *n2 = flatten_grouping_sets(lfirst(l2), false, NULL);
+ Node *n1 = lfirst(l2);
+ Node *n2 = flatten_grouping_sets(n1, false, NULL);
- result_set = lappend(result_set, n2);
+ if (IsA(n1, GroupingSet) &&
+ ((GroupingSet *)n1)->kind == GROUPING_SET_SETS)
+ {
+ result_set = list_concat(result_set, (List *) n2);
+ }
+ else
+ result_set = lappend(result_set, n2);
}
/*
diff --git a/src/test/regress/expected/groupingsets.out b/src/test/regress/expected/groupingsets.out
index 842c2ae..ff3ba9b 100644
--- a/src/test/regress/expected/groupingsets.out
+++ b/src/test/regress/expected/groupingsets.out
@@ -145,6 +145,127 @@ select a, b, sum(c), sum(sum(c)) over (order by a,b) as rsum
| | 12 | 36
(6 rows)
+-- nesting with grouping sets
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets((
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 12
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets(((a, b)
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 8
+ 2
+ 2
+(5 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(rollup(c), grouping sets(cube(c
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 6
+ 6
+ 6
+ 6
+(6 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(a, grouping sets(a, cube(b)))
+ order by 1 desc;
+ sum
+-
+ 12
+ 10
+ 10
+ 8
+ 4
+ 2
+ 2
+(7 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets((a, (b
+ order by 1 desc;
+ sum
+-
+ 8
+ 2
+ 2
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets((a, b)))
+ order by 1 desc;
+ sum
+-
+ 8
+ 2
+ 2
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a), a))
+ order by 1 desc;
+ sum
+-
+ 10
+ 10
+ 10
+ 2
+ 2
+ 2
+(6 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a, grouping sets(a), ((a)), a, grouping sets(a), (a)), a))
+ order by 1 desc;
+ sum
+-
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+(16 rows)
+
+select sum(c) from gstest2
+ group by grouping sets((a,(a,b)), grouping sets((a,(a,b)),a))
+ order by 1 desc;
+ sum
+-
+ 10
+ 8
+ 8
+ 2
+ 2
+ 2
+ 2
+ 2
+(8 rows)
+
-- empty input: first is 0 rows, second 1, third 3 etc.
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),a);
a | b | sum | count
diff --git a/src/test/regress/sql/groupingsets.sql b/src/test/regress/sql/groupingsets.sql
index 0bffb85..d886fae 100644
--- a/src/test/regress/sql/groupingsets.sql
+++ b/src/test/regress/sql/groupingsets.sql
@@ -73,6 +73,35 @@ select grouping(a), a, array_agg(b),
select a, b, sum(c), sum(sum(c)) over (order by a,b) as rsum
from gstest2 group by rollup (a,b) order by rsum, a, b;
+-- nesting with grouping sets
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets((
+ order by 1 desc;
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets(((a, b)
+ order by 1 desc;
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(rollup(c), groupi
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
Hello, this looks to be a kind of thinko. The attached patch
fixes it.
===
According to the comment of transformGroupingSet, it assumes that
the given GROUPING SETS node is already flatted out and
flatten_grouping_sets() does that. The details of the
transformation is described in the comment for the function.
The problmen is what does the function for nested grouping sets.
> Node *n2 = flatten_grouping_sets(lfirst(l2), false, NULL);
> result_set = lappend(result_set, n2);
This does not flattens the list as required. n2 should be
concatenated if it is a list. The attached small patch fixes it
and the problematic query returns sane (perhaps) result.
# Though I don't know the exact definition of the syntax..
=# select sum(c) from gstest2 group by grouping sets ((), grouping sets ((),
grouping sets ((;
sum
-
12
12
12
(3 rows)
=# select sum(c) from gstest2 group by grouping sets ((a), grouping sets ((b),
grouping sets ((c;
sum
-
10
2
6
6
8
4
(6 rows)
regards,
At Fri, 17 Jul 2015 11:37:26 +0530, Jeevan Chalke
wrote in
> > Jeevan> It looks like we do support nested GROUPING SETS, I mean Sets
> > Jeevan> withing Sets, not other types. However this nesting is broken.
> >
> > Good catch, but I'm not yet sure your fix is correct; I'll need to look
> > into that.
> >
>
> Sure. Thanks.
>
> However I wonder why we are supporting GROUPING SETS inside GROUPING SETS.
> On Oracle, it is throwing an error.
> We are not trying to be Oracle compatible, but just curious to know.
>
> I have tried restricting it in attached patch.
>
> But it may require few comment adjustment.
--
Kyotaro Horiguchi
NTT Open Source Software Center
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e90e1d6..708ebc9 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1804,8 +1804,10 @@ flatten_grouping_sets(Node *expr, bool toplevel, bool *hasGroupingSets)
foreach(l2, gset->content)
{
Node *n2 = flatten_grouping_sets(lfirst(l2), false, NULL);
-
- result_set = lappend(result_set, n2);
+ if (IsA(n2, List))
+ result_set = list_concat(result_set, (List *)n2);
+ else
+ result_set = lappend(result_set, n2);
}
/*
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
On Wed, Jul 15, 2015 at 10:21 PM, Andrew Gierth wrote:
> > "Jeevan" == Jeevan Chalke writes:
>
> Jeevan> Hi,
> Jeevan> It looks like we do support nested GROUPING SETS, I mean Sets
> Jeevan> withing Sets, not other types. However this nesting is broken.
>
> Good catch, but I'm not yet sure your fix is correct; I'll need to look
> into that.
>
Sure. Thanks.
However I wonder why we are supporting GROUPING SETS inside GROUPING SETS.
On Oracle, it is throwing an error.
We are not trying to be Oracle compatible, but just curious to know.
I have tried restricting it in attached patch.
But it may require few comment adjustment.
Thanks
>
> --
> Andrew (irc:RhodiumToad)
>
--
Jeevan B Chalke
Principal Software Engineer, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e0ff6f1..738715f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -371,9 +371,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
-%type group_by_list
+%type group_by_list grouping_sets_list
%type group_by_item empty_grouping_set rollup_clause cube_clause
-%type grouping_sets_clause
+%type grouping_sets_clause grouping_sets_item
%type opt_fdw_options fdw_options
%type fdw_option
@@ -10343,6 +10343,18 @@ group_by_item:
| grouping_sets_clause { $$ = $1; }
;
+grouping_sets_list:
+ grouping_sets_item{ $$ = list_make1($1); }
+ | grouping_sets_list ',' grouping_sets_item { $$ = lappend($1,$3); }
+ ;
+
+grouping_sets_item:
+ a_expr { $$ = $1; }
+ | empty_grouping_set { $$ = $1; }
+ | cube_clause { $$ = $1; }
+ | rollup_clause { $$ = $1; }
+ ;
+
empty_grouping_set:
'(' ')'
{
@@ -10371,7 +10383,7 @@ cube_clause:
;
grouping_sets_clause:
- GROUPING SETS '(' group_by_list ')'
+ GROUPING SETS '(' grouping_sets_list ')'
{
$$ = (Node *) makeGroupingSet(GROUPING_SET_SETS, $4, @1);
}
diff --git a/src/test/regress/expected/groupingsets.out b/src/test/regress/expected/groupingsets.out
index 842c2ae..e75dceb 100644
--- a/src/test/regress/expected/groupingsets.out
+++ b/src/test/regress/expected/groupingsets.out
@@ -145,6 +145,25 @@ select a, b, sum(c), sum(sum(c)) over (order by a,b) as rsum
| | 12 | 36
(6 rows)
+-- nesting with grouping sets
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets(((a, b
+ order by 1 desc;
+ERROR: syntax error at or near "sets"
+LINE 2: group by grouping sets((), grouping sets(((a, b
+ ^
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(rollup(c), grouping sets(cube(c
+ order by 1 desc;
+ERROR: syntax error at or near "sets"
+LINE 2: group by grouping sets(grouping sets(rollup(c), grouping s...
+ ^
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a), a))
+ order by 1 desc;
+ERROR: syntax error at or near "sets"
+LINE 2: group by grouping sets(grouping sets(a, grouping sets(a), ...
+ ^
-- empty input: first is 0 rows, second 1, third 3 etc.
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),a);
a | b | sum | count
diff --git a/src/test/regress/sql/groupingsets.sql b/src/test/regress/sql/groupingsets.sql
index 0bffb85..b7e4826 100644
--- a/src/test/regress/sql/groupingsets.sql
+++ b/src/test/regress/sql/groupingsets.sql
@@ -73,6 +73,17 @@ select grouping(a), a, array_agg(b),
select a, b, sum(c), sum(sum(c)) over (order by a,b) as rsum
from gstest2 group by rollup (a,b) order by rsum, a, b;
+-- nesting with grouping sets
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets(((a, b
+ order by 1 desc;
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(rollup(c), grouping sets(cube(c
+ order by 1 desc;
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a), a))
+ order by 1 desc;
+
-- empty input: first is 0 rows, second 1, third 3 etc.
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),a);
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),());
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Grouping Sets: Fix unrecognized node type bug
> "Jeevan" == Jeevan Chalke writes: Jeevan> Hi, Jeevan> It looks like we do support nested GROUPING SETS, I mean Sets Jeevan> withing Sets, not other types. However this nesting is broken. Good catch, but I'm not yet sure your fix is correct; I'll need to look into that. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Grouping Sets: Fix unrecognized node type bug
Hi,
It looks like we do support nested GROUPING SETS, I mean Sets withing
Sets, not other types. However this nesting is broken.
Here is the simple example where I would expect three rows in the
result. But unfortunately it is giving "unrecognized node type"
error. Which is something weird and need a fix.
postgres=# create table gstest2 (a integer, b integer, c integer);
postgres=# insert into gstest2 values (1,1,1), (1,1,1), (1,1,1),
(1,1,1), (1,1,1), (1,1,1), (1,1,2), (1,2,2), (2,2,2);
postgres=# select sum(c) from gstest2
group by grouping sets((), grouping sets((), grouping sets((
order by 1 desc;
ERROR: unrecognized node type: 926
I spend much time to understand the cause and was looking into
transformGroupingSet() and transformGroupClauseList() function.
I have tried fixing "unrecognized node type: 926" error there,
but later it is failing with "unrecognized node type: 656".
Later I have realized that we have actually have an issue while
flattening grouping sets. If we have nested grouping sets like
above, then we are getting GroupingSet node inside the list and
transformGroupClauseList() does not expect that and end up with
this error.
I have tried fixing this issue in flatten_grouping_sets(), after
flattening grouping sets node, we need to concat the result with
the existing list and should not append. This alone does not
solve the issue as we need a list when we have ROW expression.
Thus there, if not top level, I am creating a list now.
Attached patch with few testcases too.
Please have a look.
Thanks
--
Jeevan B Chalke
Principal Software Engineer, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index e90e1d6..31d4331 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -1779,8 +1779,19 @@ flatten_grouping_sets(Node *expr, bool toplevel, bool *hasGroupingSets)
RowExpr*r = (RowExpr *) expr;
if (r->row_format == COERCE_IMPLICIT_CAST)
- return flatten_grouping_sets((Node *) r->args,
- false, NULL);
+{
+ Node *n1 = flatten_grouping_sets((Node *) r->args,
+ false, NULL);
+
+ /*
+ * Make a list for row expression if toplevel is false,
+ * return flatten list otherwise
+ */
+ if (toplevel)
+ return (Node *) n1;
+ else
+ return (Node *) list_make1(n1);
+}
}
break;
case T_GroupingSet:
@@ -1805,7 +1816,10 @@ flatten_grouping_sets(Node *expr, bool toplevel, bool *hasGroupingSets)
{
Node *n2 = flatten_grouping_sets(lfirst(l2), false, NULL);
- result_set = lappend(result_set, n2);
+ if (IsA(n2, List))
+ result_set = list_concat(result_set, (List *) n2);
+ else
+ result_set = lappend(result_set, n2);
}
/*
diff --git a/src/test/regress/expected/groupingsets.out b/src/test/regress/expected/groupingsets.out
index adb39b3..5c47717 100644
--- a/src/test/regress/expected/groupingsets.out
+++ b/src/test/regress/expected/groupingsets.out
@@ -145,6 +145,112 @@ select a, b, sum(c), sum(sum(c)) over (order by a,b) as rsum
| | 12 | 36
(6 rows)
+-- nesting with grouping sets
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets((
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 12
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets((), grouping sets((), grouping sets(((a, b)
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 8
+ 2
+ 2
+(5 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(rollup(c), grouping sets(cube(c
+ order by 1 desc;
+ sum
+-
+ 12
+ 12
+ 6
+ 6
+ 6
+ 6
+(6 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(a, grouping sets(a, cube(b)))
+ order by 1 desc;
+ sum
+-
+ 12
+ 10
+ 10
+ 8
+ 4
+ 2
+ 2
+(7 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets((a, (b
+ order by 1 desc;
+ sum
+-
+ 8
+ 2
+ 2
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets((a, b)))
+ order by 1 desc;
+ sum
+-
+ 8
+ 2
+ 2
+(3 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a), a))
+ order by 1 desc;
+ sum
+-
+ 10
+ 10
+ 10
+ 2
+ 2
+ 2
+(6 rows)
+
+select sum(c) from gstest2
+ group by grouping sets(grouping sets(a, grouping sets(a, grouping sets(a), ((a)), a, grouping sets(a), (a)), a))
+ order by 1 desc;
+ sum
+-
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 10
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+ 2
+(16 rows)
+
-- empty input: first is 0 rows, second 1, third 3 etc.
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),a);
a | b | sum | count
diff --git a/src/test/regress/sql/groupingsets.sql b/src/test/regress/sql/groupingsets.sql
index 0883afd..e478d34 100644
--- a/src/test
