Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi, On 2019/03/26 10:15, Michael Paquier wrote: > Done as you suggested, with a minimal set enough to trigger the crash, > still the error message is rather misleading as you would expect :) Thanks for committing. >> A separate thread will definitely attract more attention, at least in due >> time. :) > > Sure. For now I have committed a lighter version of 0001, with > tweaked comments based on your suggestion, as well as a minimum set of > test cases. I have added on the way some tests for range partitions > which have been missing from the start, and improved the existing set > by removing the original "a.a" references, and switching to use > max(date) for range partitions to bump correctly on the aggregate > error. I am just updating the second patch now and I'll begin a new > thread soon. Thanks. Regards, Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Fri, Mar 22, 2019 at 02:49:42PM +0900, Amit Langote wrote:
> This comment sounds a bit misleading. The code above this "did" find
> field names, but there were too many. What this comment should mention is
> that parsing didn't return a single field name, which is the format that
> the code below this can do something useful with. I had proposed that
> upthread, but maybe that feedback got lost in the discussion about other
> related issues.
True. I was reviewing that stuff yesterday and I have not been able
to finish wrapping it.
> +CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
> + FOR VALUES FROM (sum(a)) TO ('2019-01-01');
> +ERROR: function sum(date) does not exist
> +LINE 2: FOR VALUES FROM (sum(a)) TO ('2019-01-01');
>
> Maybe, we should add to this patch only the tests relevant to the cases
> that would lead to crash without this patch.
Done as you suggested, with a minimal set enough to trigger the crash,
still the error message is rather misleading as you would expect :)
> A separate thread will definitely attract more attention, at least in due
> time. :)
Sure. For now I have committed a lighter version of 0001, with
tweaked comments based on your suggestion, as well as a minimum set of
test cases. I have added on the way some tests for range partitions
which have been missing from the start, and improved the existing set
by removing the original "a.a" references, and switching to use
max(date) for range partitions to bump correctly on the aggregate
error. I am just updating the second patch now and I'll begin a new
thread soon.
--
Michael
signature.asc
Description: PGP signature
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi,
Thanks for splitting. It makes sense, because, as you know, the bug that
causes the crash is a separate problem from unintuitive error messages
which result from the way in which we parse expressions.
On 2019/03/22 14:09, Michael Paquier wrote:
> On Wed, Mar 20, 2019 at 06:17:27PM +0900, Michael Paquier wrote:
>> The thing is that in order to keep the tests for the crash, we finish
>> with the inintuitive RTE-related errors, so it is also inconsistent to
>> not group things..
>
> As I have seen no feedback from others regarding the changes in error
> messages depending on the parsing context, so I have been looking at
> splitting the fix for the crash and changing the error messages, and
> attached is the result of the split (minus the commit messages). The
> first patch fixes the crash, and includes test cases to cover the
> crash as well as extra cases for list and range strategies with
> partition bounds. Some of the error messages are confusing, but that
> fixes the issue. This is not the most elegant thing without the
> second patch, but well that could be worse.
A comment on this one:
+ if (cname == NULL)
+ {
+ /*
+* No field names have been found, meaning that there
+* is not much to do with special value handling. Instead
+* let the expression transformation handle any errors and
+* limitations.
+*/
This comment sounds a bit misleading. The code above this "did" find
field names, but there were too many. What this comment should mention is
that parsing didn't return a single field name, which is the format that
the code below this can do something useful with. I had proposed that
upthread, but maybe that feedback got lost in the discussion about other
related issues.
I had proposed this:
+ /*
+ * There should be a single field named "minvalue" or "maxvalue".
+ */
if (list_length(cref->fields) == 1 &&
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
- Assert(cname != NULL);
- if (strcmp("minvalue", cname) == 0)
+ if (cname == NULL)
+ {
+ /*
+ * ColumnRef is not in the desired single-field-name form; for
+ * consistency, let transformExpr() report the error rather
+ * than doing it ourselves.
+ */
+ }
Maybe that could use few more tweaks, but hope I've made my point.
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM (sum(a)) TO ('2019-01-01');
+ERROR: function sum(date) does not exist
+LINE 2: FOR VALUES FROM (sum(a)) TO ('2019-01-01');
Maybe, we should add to this patch only the tests relevant to the cases
that would lead to crash without this patch.
Tests regarding error messages fine tuning can be added in the other patch.
> The second patch adds better error context for the different error
> messages, and includes tests for default expressions, which we could
> discuss in a separate thread. So I am not proposing to commit that
> without more feedback.
A separate thread will definitely attract more attention, at least in due
time. :)
Thanks,
Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Wed, Mar 20, 2019 at 06:17:27PM +0900, Michael Paquier wrote:
> The thing is that in order to keep the tests for the crash, we finish
> with the inintuitive RTE-related errors, so it is also inconsistent to
> not group things..
As I have seen no feedback from others regarding the changes in error
messages depending on the parsing context, so I have been looking at
splitting the fix for the crash and changing the error messages, and
attached is the result of the split (minus the commit messages). The
first patch fixes the crash, and includes test cases to cover the
crash as well as extra cases for list and range strategies with
partition bounds. Some of the error messages are confusing, but that
fixes the issue. This is not the most elegant thing without the
second patch, but well that could be worse.
The second patch adds better error context for the different error
messages, and includes tests for default expressions, which we could
discuss in a separate thread. So I am not proposing to commit that
without more feedback.
--
Michael
From 61e77542fe720edcb7b80689ca0191ea195071ec Mon Sep 17 00:00:00 2001
From: Michael Paquier
Date: Fri, 22 Mar 2019 13:49:41 +0900
Subject: [PATCH 1/2] Fix crash in partition bounds
---
src/backend/parser/parse_utilcmd.c | 12 +++-
src/test/regress/expected/create_table.out | 65 +-
src/test/regress/sql/create_table.sql | 26 -
3 files changed, 99 insertions(+), 4 deletions(-)
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..ee129ba18f 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -3750,8 +3750,16 @@ transformPartitionRangeBounds(ParseState *pstate, List *blist,
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
- Assert(cname != NULL);
- if (strcmp("minvalue", cname) == 0)
+ if (cname == NULL)
+ {
+/*
+ * No field names have been found, meaning that there
+ * is not much to do with special value handling. Instead
+ * let the expression transformation handle any errors and
+ * limitations.
+ */
+ }
+ else if (strcmp("minvalue", cname) == 0)
{
prd = makeNode(PartitionRangeDatum);
prd->kind = PARTITION_RANGE_DATUM_MINVALUE;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..130dccb241 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -489,19 +489,35 @@ Partitions: part_null FOR VALUES IN (NULL),
part_p2 FOR VALUES IN (2),
part_p3 FOR VALUES IN (3)
--- forbidden expressions for partition bound
+-- forbidden expressions for partition bound with list partitioned table
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (somename);
ERROR: column "somename" does not exist
LINE 1: ...expr_fail PARTITION OF list_parted FOR VALUES IN (somename);
^
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (somename.somename);
+ERROR: missing FROM-clause entry for table "somename"
+LINE 1: ...expr_fail PARTITION OF list_parted FOR VALUES IN (somename.s...
+ ^
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a);
ERROR: cannot use column references in partition bound expression
LINE 1: ..._bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a);
^
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a.a);
+ERROR: missing FROM-clause entry for table "a"
+LINE 1: ...ogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a.a);
+ ^
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (sum(a));
ERROR: aggregate functions are not allowed in partition bound
LINE 1: ...s_expr_fail PARTITION OF list_parted FOR VALUES IN (sum(a));
^
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (sum(somename));
+ERROR: column "somename" does not exist
+LINE 1: ..._fail PARTITION OF list_parted FOR VALUES IN (sum(somename))...
+ ^
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (sum(1));
+ERROR: aggregate functions are not allowed in partition bound
+LINE 1: ...s_expr_fail PARTITION OF list_parted FOR VALUES IN (sum(1));
+ ^
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN ((select 1));
ERROR: cannot use subquery in partition bound
LINE 1: ...expr_fail PARTITION OF list_parted FOR VALUES IN
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Wed, Mar 20, 2019 at 06:07:23PM +0900, Amit Langote wrote: > because we can notice the aggregate before we look into its arguments. > Maybe, we should move the error-checking switch to a point before checking > the arguments? That looks slightly more drastic change to make though. Yeah, I think that it would be more invasive because the parsing logic looks at the column references first. This way of doing things is also something which is logic in its own way as the most internal portions of an expression get checked first, so I quite like that way of doing things. And that's just consistent with the point of view of the parsing check order. > We may want to fix the crash first. It might be better to hear other > opinions before doing something about the error messages. The thing is that in order to keep the tests for the crash, we finish with the inintuitive RTE-related errors, so it is also inconsistent to not group things.. -- Michael signature.asc Description: PGP signature
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi, On 2019/03/20 11:07, Michael Paquier wrote: > On Thu, Mar 14, 2019 at 01:23:08PM +0900, Michael Paquier wrote: >> I actually think that what you propose here makes more sense than what >> HEAD does because the most inner expression gets evaluated first. >> This for example generates the same error as on HEAD: >> =# create table foo (a int default (avg(1))); >> ERROR: 42803: aggregate functions are not allowed in DEFAULT expressions > > I have been working on that, Thanks a lot. > and in the case of a non-existing column > the patch would generate the following error on HEAD: > ERROR: 42703: column "non_existent" does not exist > But with the patch we get that: > ERROR: cannot use column reference in DEFAULT expression > > Still I think that this looks right as we should not have any direct > column reference anyway, and it keeps the code more simple. So I have > added more tests to cover those grounds. I agree that we should error out immediately, once we encounter an (sub-) expression that's not supported by a given feature (default, partition bounds, etc.) Actually, doesn't that mean we should error out because of the aggregate in the following example: create table foo (a int default (avg(a)); because we can notice the aggregate before we look into its arguments. Maybe, we should move the error-checking switch to a point before checking the arguments? That looks slightly more drastic change to make though. > The docs of CREATE TABLE are actually wrong, right? It mentions that > "subqueries and cross-references to other columns in the current table > are not allowed", but cookDefault() rejects any kind of column > references anyway for default expressions, including references to the > column which uses the default expression (or we talk about generated > columns here still if I recall Peter Eisentraunt's patch correctly > generated columns don't allow references to the column using the > expression itself, which is logic by the way). Yeah, the documentation in your patch looks correct at a first glance. > +* transformExpr() should have already rejected column references, > +* subqueries, aggregates, window functions, and SRFs, based on the > +* EXPR_KIND_ for a default expression. > */ > I would have added an assertion here, perhaps an elog(). Same remark > for cookDefault(). The attached patch has an assertion. +1 > CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES > IN (sum(a)); > -ERROR: aggregate functions are not allowed in partition bound > +ERROR: cannot use column reference in partition bound expression > It would be nice to also test the case where an aggregate is > forbidden, so I have added a test with sum(1) instead of a column > reference. As I said above, maybe we should try to rearrange things so that we get the former error in both cases. > We never actually tested in the tree the case of subqueries and SRFs > used in default expressions, so added. > > The last patch you sent did not fix the original problem of the > thread. That was intentional from your side I guess to show your > point, Yeah, that's right. > still we are touching the same area of the code so I propose to > fix everything together, and to improve the test coverage for list and > range strategies. In order to achieve that, I have merged my previous > proposal into your patch, and added more tests. The new tests for the > range strategy reproduce the crash. The result is attached. We may want to fix the crash first. It might be better to hear other opinions before doing something about the error messages. Thanks, Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Thu, Mar 14, 2019 at 01:23:08PM +0900, Michael Paquier wrote:
> I actually think that what you propose here makes more sense than what
> HEAD does because the most inner expression gets evaluated first.
> This for example generates the same error as on HEAD:
> =# create table foo (a int default (avg(1)));
> ERROR: 42803: aggregate functions are not allowed in DEFAULT expressions
I have been working on that, and in the case of a non-existing column
the patch would generate the following error on HEAD:
ERROR: 42703: column "non_existent" does not exist
But with the patch we get that:
ERROR: cannot use column reference in DEFAULT expression
Still I think that this looks right as we should not have any direct
column reference anyway, and it keeps the code more simple. So I have
added more tests to cover those grounds.
The docs of CREATE TABLE are actually wrong, right? It mentions that
"subqueries and cross-references to other columns in the current table
are not allowed", but cookDefault() rejects any kind of column
references anyway for default expressions, including references to the
column which uses the default expression (or we talk about generated
columns here still if I recall Peter Eisentraunt's patch correctly
generated columns don't allow references to the column using the
expression itself, which is logic by the way).
+* transformExpr() should have already rejected column references,
+* subqueries, aggregates, window functions, and SRFs, based on the
+* EXPR_KIND_ for a default expression.
*/
I would have added an assertion here, perhaps an elog(). Same remark
for cookDefault(). The attached patch has an assertion.
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES
IN (sum(a));
-ERROR: aggregate functions are not allowed in partition bound
+ERROR: cannot use column reference in partition bound expression
It would be nice to also test the case where an aggregate is
forbidden, so I have added a test with sum(1) instead of a column
reference.
We never actually tested in the tree the case of subqueries and SRFs
used in default expressions, so added.
The last patch you sent did not fix the original problem of the
thread. That was intentional from your side I guess to show your
point, still we are touching the same area of the code so I propose to
fix everything together, and to improve the test coverage for list and
range strategies. In order to achieve that, I have merged my previous
proposal into your patch, and added more tests. The new tests for the
range strategy reproduce the crash. The result is attached.
What do you think?
--
Michael
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index e94fe2c3b6..5fe16aa6f6 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -784,7 +784,7 @@ WITH ( MODULUS numeric_literal, REM
The DEFAULT clause assigns a default data value for
the column whose column definition it appears within. The value
is any variable-free expression (subqueries and cross-references
- to other columns in the current table are not allowed). The
+ to any columns in the current table are not allowed). The
data type of the default expression must match the data type of the
column.
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index c7b5ff62f9..fc682e0b52 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2939,19 +2939,11 @@ cookDefault(ParseState *pstate,
expr = transformExpr(pstate, raw_default, EXPR_KIND_COLUMN_DEFAULT);
/*
- * Make sure default expr does not refer to any vars (we need this check
- * since the pstate includes the target table).
- */
- if (contain_var_clause(expr))
- ereport(ERROR,
-(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
- errmsg("cannot use column references in default expression")));
-
- /*
- * transformExpr() should have already rejected subqueries, aggregates,
- * window functions, and SRFs, based on the EXPR_KIND_ for a default
- * expression.
+ * transformExpr() should have already rejected column references,
+ * subqueries, aggregates, window functions, and SRFs, based on the
+ * EXPR_KIND_ for a default expression.
*/
+ Assert(!contain_var_clause(expr));
/*
* Coerce the expression to the correct type and typmod, if given. This
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index e559353529..3e648dc8ef 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -520,6 +520,79 @@ transformColumnRef(ParseState *pstate, ColumnRef *cref)
CRERR_WRONG_DB,
CRERR_TOO_MANY
} crerr = CRERR_NO_COLUMN;
+ const char *err;
+
+ /*
+ * Check to see if the column reference is in an invalid place within the
+ * query. We allow column references in most places, except in default
+ * expressions and partition bound expressions.
+ */
+
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Wed, Mar 13, 2019 at 03:17:47PM +0900, Amit Langote wrote: > but on HEAD, you get: > > create table foo (a int default (avg(foo.a))); > ERROR: aggregate functions are not allowed in DEFAULT expressions I actually think that what you propose here makes more sense than what HEAD does because the most inner expression gets evaluated first. This for example generates the same error as on HEAD: =# create table foo (a int default (avg(1))); ERROR: 42803: aggregate functions are not allowed in DEFAULT expressions -- Michael signature.asc Description: PGP signature
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On 2019/03/13 14:15, Amit Langote wrote:
> On 2019/03/11 16:21, Michael Paquier wrote:
>> On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote:
>>> We could make the error message more meaningful depending on the context,
>>> but maybe it'd better be pursue it as a separate project.
>>
>> Yeah, I noticed that stuff when working on it this afternoon. The
>> error message does not completely feel right even in your produced
>> tests. Out of curiosity I have been working on this thing myself,
>> and it is possible to have a context-related message. Please see
>> attached, that's in my opinion less confusing, and of course
>> debatable. Still this approach does not feel completely right either
>> as that means hijacking the code path which generates a generic
>> message for missing RTEs. :(
>
> @@ -3259,6 +3259,9 @@ errorMissingRTE(ParseState *pstate, RangeVar
> + *
> + * Also, in the context of parsing a partition bound, produce a more
> + * helpful error message.
>*/
> if (rte && rte->alias &&
> strcmp(rte->eref->aliasname, relation->relname) != 0 &&
>
>
> - if (rte)
> + if (pstate->p_expr_kind == EXPR_KIND_PARTITION_BOUND)
> + ereport(ERROR,
> + (errcode(ERRCODE_UNDEFINED_TABLE),
> + errmsg("invalid reference in partition bound
> expression for table
> \"%s\"",
> + relation->relname)));
> + else if (rte)
>
> Hmm, it seems odd to me that it's OK for default expressions to emit the
> "missing RTE" error, whereas partition bound expressions would emit this
> special error message?
>
> create table foo (a int default (bar.a));
> ERROR: missing FROM-clause entry for table "bar"
Looking into this a bit more, I wonder if it would be a good idea to to
have one of those error-emitting switch (pstate->p_expr_kind) blocks in
transformColumnRef(), as shown in the attached patch?
For example, the following error is emitted by one of such blocks that's
in transformSubLink():
create table foo (a int default (select * from non_existent_table));
ERROR: cannot use subquery in DEFAULT expression
However, if we decide to go with this approach, we will start getting
different error messages from what HEAD gives in certain cases. With the
patch, you will get the following error when trying to use an aggregate
function in for DEFAULT expression:
create table foo (a int default (avg(foo.a)));
ERROR: cannot use column reference in DEFAULT expression
but on HEAD, you get:
create table foo (a int default (avg(foo.a)));
ERROR: aggregate functions are not allowed in DEFAULT expressions
That's because, on HEAD, transformAggregateCall() (or something that calls
it) first calls transformColumnRef() to resolve 'foo.a', which checks that
foo.a is a valid column reference, but it doesn't concern itself with the
fact that the bigger expression it's part of is being used for DEFAULT
expression. It's only after 'foo.a' has been resolved as a valid column
reference that check_agglevels_and_constraints(), via
transformAggregateCall, emits an error that the overall expression is
invalid to use as a DEFAULT expression. With patches, error will be
emitted even before resolving 'foo.a'.
While transformAggregateCall() works like that, transformSubLink(), which
I mentioned in the first example, doesn't bother to analyze the query
first (select * from non_existent_table) to notice that the referenced
table doesn't exist. If it had bothered to analyze the query first, we
would've most likely gotten an error from errorMissingRTE(), not what we
get today. So, there are certainly some inconsistencies even today in how
these errors are emitted.
Thanks,
Amit
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index c7b5ff62f9..aa36d19117 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2939,18 +2939,9 @@ cookDefault(ParseState *pstate,
expr = transformExpr(pstate, raw_default, EXPR_KIND_COLUMN_DEFAULT);
/*
-* Make sure default expr does not refer to any vars (we need this check
-* since the pstate includes the target table).
-*/
- if (contain_var_clause(expr))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
-errmsg("cannot use column references in
default expression")));
-
- /*
-* transformExpr() should have already rejected subqueries, aggregates,
-* window functions, and SRFs, based on the EXPR_KIND_ for a default
-* expression.
+* transformExpr() should have already rejected column references,
+* subqueries, aggregates, window functions, and SRFs, based on the
+* EXPR_KIND_ for a default expression.
*/
/*
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On 2019/03/11 16:21, Michael Paquier wrote:
> On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote:
>> We could make the error message more meaningful depending on the context,
>> but maybe it'd better be pursue it as a separate project.
>
> Yeah, I noticed that stuff when working on it this afternoon. The
> error message does not completely feel right even in your produced
> tests. Out of curiosity I have been working on this thing myself,
> and it is possible to have a context-related message. Please see
> attached, that's in my opinion less confusing, and of course
> debatable. Still this approach does not feel completely right either
> as that means hijacking the code path which generates a generic
> message for missing RTEs. :(
@@ -3259,6 +3259,9 @@ errorMissingRTE(ParseState *pstate, RangeVar
+*
+* Also, in the context of parsing a partition bound, produce a more
+* helpful error message.
*/
if (rte && rte->alias &&
strcmp(rte->eref->aliasname, relation->relname) != 0 &&
- if (rte)
+ if (pstate->p_expr_kind == EXPR_KIND_PARTITION_BOUND)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+errmsg("invalid reference in partition bound
expression for table
\"%s\"",
+ relation->relname)));
+ else if (rte)
Hmm, it seems odd to me that it's OK for default expressions to emit the
"missing RTE" error, whereas partition bound expressions would emit this
special error message?
create table foo (a int default (bar.a));
ERROR: missing FROM-clause entry for table "bar"
Thanks,
Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On 2019/03/13 1:35, Tom Lane wrote:
> Robert Haas writes:
>> On Mon, Mar 11, 2019 at 2:45 AM Amit Langote
>> wrote:
>>> I noticed another issue with the code -- it's using strcmp() to compare
>>> specified string against "minvalue" and "maxvalue", which causes the
>>> following silly error:
>>>
>>> create table q2 partition of q for values from ("MINVALUE") to (maxvalue);
>>> ERROR: column "MINVALUE" does not exist
>>> LINE 1: create table q2 partition of q for values from ("MINVALUE") ...
>>>
>>> It should be using pg_strncasecmp().
>
>> Uh, why? Generally, an unquoted keyword is equivalent to a quoted
>> lowercase version of that same keyword, not anything else. Like
>> CREATE TABLE "foo" = CREATE TABLE FOO <> CREATE TABLE "FOO".
OK. Perhaps, I reacted too strongly to encountering the following
behavior with HEAD:
create table p1 partition of p for values from ("minValue") to (1);
ERROR: column "minValue" does not exist
but,
create table p1 partition of p for values from ("minvalue") to (1);
\d p1
Table "public.p1"
Column │ Type │ Collation │ Nullable │ Default
┼─┼───┼──┼─
a │ integer │ │ │
Partition of: p FOR VALUES FROM (MINVALUE) TO (1)
But as you and Tom have pointed out, maybe it's normal.
> Yeah. The behavior shown above is entirely correct, and accepting the
> statement would be flat out wrong; it would cause trouble if somebody
> created a table containing multiple case-variations of MINVALUE.
Sorry, I didn't understand this last part. Different case-variations will
all be interpreted as a minvalue (negative infinity) range bound and
flagged if the resulting range bound constraint would be invalid.
Did you mean something like the following:
create table p1 partition of ... from ("minValue") to ("MINVALUE");
which using pg_strncasecmp() comparisons gives:
create table p1 partition of p for values from ("minValue") to ("MINVALUE");
ERROR: empty range bound specified for partition "p1"
DETAIL: Specified lower bound (MINVALUE) is greater than or equal to
upper bound (MINVALUE).
which is same as the behavior with unquoted keyword syntax:
create table p1 partition of p for values from (minValue) to (MINVALUE);
ERROR: empty range bound specified for partition "p1"
DETAIL: Specified lower bound (MINVALUE) is greater than or equal to
upper bound (MINVALUE).
whereas quoted identifier syntax on HEAD gives:
create table p1 partition of p for values from ("minValue") to ("MINVALUE");
ERROR: column "minValue" does not exist
LINE 1: create table p1 partition of p for values from ("minValue") ...
However, as you guys said, HEAD is behaving sanely.
Thanks,
Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Robert Haas writes:
> On Mon, Mar 11, 2019 at 2:45 AM Amit Langote
> wrote:
>> I noticed another issue with the code -- it's using strcmp() to compare
>> specified string against "minvalue" and "maxvalue", which causes the
>> following silly error:
>>
>> create table q2 partition of q for values from ("MINVALUE") to (maxvalue);
>> ERROR: column "MINVALUE" does not exist
>> LINE 1: create table q2 partition of q for values from ("MINVALUE") ...
>>
>> It should be using pg_strncasecmp().
> Uh, why? Generally, an unquoted keyword is equivalent to a quoted
> lowercase version of that same keyword, not anything else. Like
> CREATE TABLE "foo" = CREATE TABLE FOO <> CREATE TABLE "FOO".
Yeah. The behavior shown above is entirely correct, and accepting the
statement would be flat out wrong; it would cause trouble if somebody
created a table containing multiple case-variations of MINVALUE.
regards, tom lane
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Mon, Mar 11, 2019 at 2:45 AM Amit Langote
wrote:
> I noticed another issue with the code -- it's using strcmp() to compare
> specified string against "minvalue" and "maxvalue", which causes the
> following silly error:
>
> create table q2 partition of q for values from ("MINVALUE") to (maxvalue);
> ERROR: column "MINVALUE" does not exist
> LINE 1: create table q2 partition of q for values from ("MINVALUE") ...
>
> It should be using pg_strncasecmp().
Uh, why? Generally, an unquoted keyword is equivalent to a quoted
lowercase version of that same keyword, not anything else. Like
CREATE TABLE "foo" = CREATE TABLE FOO <> CREATE TABLE "FOO".
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Mon, Mar 11, 2019 at 03:44:39PM +0900, Amit Langote wrote:
> We could make the error message more meaningful depending on the context,
> but maybe it'd better be pursue it as a separate project.
Yeah, I noticed that stuff when working on it this afternoon. The
error message does not completely feel right even in your produced
tests. Out of curiosity I have been working on this thing myself,
and it is possible to have a context-related message. Please see
attached, that's in my opinion less confusing, and of course
debatable. Still this approach does not feel completely right either
as that means hijacking the code path which generates a generic
message for missing RTEs. :(
--
Michael
diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c
index f3b6d193aa..6b248f116f 100644
--- a/src/backend/parser/parse_relation.c
+++ b/src/backend/parser/parse_relation.c
@@ -3259,6 +3259,9 @@ errorMissingRTE(ParseState *pstate, RangeVar *relation)
* If we found a match that doesn't meet those criteria, assume the
* problem is illegal use of a relation outside its scope, as in the
* MySQL-ism "SELECT ... FROM a, b LEFT JOIN c ON (a.x = c.y)".
+ *
+ * Also, in the context of parsing a partition bound, produce a more
+ * helpful error message.
*/
if (rte && rte->alias &&
strcmp(rte->eref->aliasname, relation->relname) != 0 &&
@@ -3267,7 +3270,12 @@ errorMissingRTE(ParseState *pstate, RangeVar *relation)
_up) == rte)
badAlias = rte->eref->aliasname;
- if (rte)
+ if (pstate->p_expr_kind == EXPR_KIND_PARTITION_BOUND)
+ ereport(ERROR,
+(errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("invalid reference in partition bound expression for table \"%s\"",
+ relation->relname)));
+ else if (rte)
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_TABLE),
errmsg("invalid reference to FROM-clause entry for table \"%s\"",
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..ee129ba18f 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -3750,8 +3750,16 @@ transformPartitionRangeBounds(ParseState *pstate, List *blist,
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
- Assert(cname != NULL);
- if (strcmp("minvalue", cname) == 0)
+ if (cname == NULL)
+ {
+/*
+ * No field names have been found, meaning that there
+ * is not much to do with special value handling. Instead
+ * let the expression transformation handle any errors and
+ * limitations.
+ */
+ }
+ else if (strcmp("minvalue", cname) == 0)
{
prd = makeNode(PartitionRangeDatum);
prd->kind = PARTITION_RANGE_DATUM_MINVALUE;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..0b3c9fb1ff 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -494,6 +494,10 @@ CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (somena
ERROR: column "somename" does not exist
LINE 1: ...expr_fail PARTITION OF list_parted FOR VALUES IN (somename);
^
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (somename.somename);
+ERROR: invalid reference in partition bound expression for table "somename"
+CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (somename.somename.somename);
+ERROR: invalid reference in partition bound expression for table "somename"
CREATE TABLE part_bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a);
ERROR: cannot use column references in partition bound expression
LINE 1: ..._bogus_expr_fail PARTITION OF list_parted FOR VALUES IN (a);
@@ -558,6 +562,33 @@ DROP TABLE bigintp;
CREATE TABLE range_parted (
a date
) PARTITION BY RANGE (a);
+-- forbidden expressions for partition bounds
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM (a) TO (1);
+ERROR: cannot use column references in partition bound expression
+LINE 2: FOR VALUES FROM (a) TO (1);
+ ^
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM (somename) TO (1);
+ERROR: column "somename" does not exist
+LINE 2: FOR VALUES FROM (somename) TO (1);
+ ^
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM (somename.somename) TO (1);
+ERROR: invalid reference in partition bound expression for table "somename"
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM (somename.somename.somename) TO (1);
+ERROR: invalid reference in partition bound expression for table "somename"
+CREATE TABLE part_bogus_expr_fail PARTITION OF range_parted
+ FOR VALUES FROM ((select 1)) TO (10);
+ERROR: cannot use subquery in partition bound
+LINE 2: FOR VALUES
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi,
On 2019/03/06 17:27, Michael Paquier wrote:
> On Wed, Mar 06, 2019 at 04:00:42PM +0900, Amit Langote wrote:
>> Thanks for looking at this. Your patch seems better, because it allows us
>> to keep the error message consistent with the message one would get with
>> list-partitioned syntax.
>
> Thanks for confirming. I think that it would be nice as well to add
> more test coverage for such error patterns with all the strategies.
> It would be good to fix that first, so I can take care of that.
I've added some tests to your patch. Also improved the comments a bit.
I noticed another issue with the code -- it's using strcmp() to compare
specified string against "minvalue" and "maxvalue", which causes the
following silly error:
create table q2 partition of q for values from ("MINVALUE") to (maxvalue);
ERROR: column "MINVALUE" does not exist
LINE 1: create table q2 partition of q for values from ("MINVALUE") ...
It should be using pg_strncasecmp().
> Now I don't really find the error "missing FROM-clause entry for
> table" quite convincing when this is applied to a partition bound when
> using a column defined in the relation. Adding more error classes in
> the set of CRERR_NO_RTE would be perhaps nice, still I am not sure how
> elegant it could be made when looking at expressions for partition
> bounds.
Note that this is not just a problem for partition bounds. You can see it
with default expressions too.
create table foo (a int default (aa.a));
ERROR: missing FROM-clause entry for table "aa"
LINE 1: create table foo (a int default (aa.a));
create table foo (a int default (a.a.aa.a.a.a.a.aa));
ERROR: improper qualified name (too many dotted names): a.a.aa.a.a.a.a.aa
LINE 1: create table foo (a int default (a.a.aa.a.a.a.a.aa));
We could make the error message more meaningful depending on the context,
but maybe it'd better be pursue it as a separate project.
Thanks,
Amit
diff --git a/src/backend/parser/parse_utilcmd.c
b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..67b05b8039 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -3746,18 +3746,28 @@ transformPartitionRangeBounds(ParseState *pstate, List
*blist,
ColumnRef *cref = (ColumnRef *) expr;
char *cname = NULL;
+ /*
+* There should be a single field named "minvalue" or
"maxvalue".
+*/
if (list_length(cref->fields) == 1 &&
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
- Assert(cname != NULL);
- if (strcmp("minvalue", cname) == 0)
+ if (cname == NULL)
+ {
+ /*
+* ColumnRef is not in the desired
single-field-name form; for
+* consistency, let transformExpr() report the
error rather
+* than doing it ourselves.
+*/
+ }
+ else if (pg_strncasecmp(cname, "minvalue", 6) == 0)
{
prd = makeNode(PartitionRangeDatum);
prd->kind = PARTITION_RANGE_DATUM_MINVALUE;
prd->value = NULL;
}
- else if (strcmp("maxvalue", cname) == 0)
+ else if (pg_strncasecmp(cname, "maxvalue", 6) == 0)
{
prd = makeNode(PartitionRangeDatum);
prd->kind = PARTITION_RANGE_DATUM_MAXVALUE;
diff --git a/src/test/regress/expected/create_table.out
b/src/test/regress/expected/create_table.out
index d51e547278..503dca5866 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -709,6 +709,66 @@ ERROR: modulus for hash partition must be a positive
integer
-- remainder must be greater than or equal to zero and less than modulus
CREATE TABLE fail_part PARTITION OF hash_parted2 FOR VALUES WITH (MODULUS 8,
REMAINDER 8);
ERROR: remainder for hash partition must be less than modulus
+-- more tests for generalized expression syntax for list/range partition bounds
+CREATE TABLE list_parted_bound_exprs(a int) PARTITION BY LIST (a);
+CREATE TABLE list_parted_bound_exprs1 PARTITION OF list_parted_bound_exprs FOR
VALUES IN ("UNKNOWN");
+ERROR: column "UNKNOWN" does not exist
+LINE 1: ...RTITION OF list_parted_bound_exprs FOR VALUES IN ("UNKNOWN")...
+ ^
+CREATE TABLE list_parted_bound_exprs1 PARTITION OF list_parted_bound_exprs FOR
VALUES IN (a);
+ERROR: cannot use column references in partition bound expression
+LINE 1: ...prs1
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Wed, Mar 06, 2019 at 04:00:42PM +0900, Amit Langote wrote: > Thanks for looking at this. Your patch seems better, because it allows us > to keep the error message consistent with the message one would get with > list-partitioned syntax. Thanks for confirming. I think that it would be nice as well to add more test coverage for such error patterns with all the strategies. It would be good to fix that first, so I can take care of that. Now I don't really find the error "missing FROM-clause entry for table" quite convincing when this is applied to a partition bound when using a column defined in the relation. Adding more error classes in the set of CRERR_NO_RTE would be perhaps nice, still I am not sure how elegant it could be made when looking at expressions for partition bounds. -- Michael signature.asc Description: PGP signature
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi, On 2019/03/06 15:48, Michael Paquier wrote: > On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote: >> Maybe we should error out as follows in >> transformPartitionRangeBounds(), although that means we'll get >> different error message than when using list partitioning syntax: > > Hm. I don't think that this is a good idea as you could lose some > information for the expression transformation handling, and the error > handling becomes inconsistent depending on the partition bound > strategy. It seems to me that if we cannot extract any special value > from the ColumnRef expression generated, then we ought to let > transformPartitionBoundValue() and particularly transformExprRecurse() > do the analysis work and complain if needed: > =# CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM > (unknown.unknown) TO (1); > ERROR: 42P01: missing FROM-clause entry for table "unknown" > LINE 1: ...p_part PARTITION OF range_parted FOR VALUES FROM > (unknown.un... > =# CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM > (a.a.a.a.a.a.a.a.a.a.a.a) TO (1); > ERROR: 42601: improper qualified name (too many dotted names): > a.a.a.a.a.a.a.a.a.a.a.a > LINE 1: ...p_part PARTITION OF range_parted FOR VALUES FROM > (a.a.a.a.a > > What about something like the attached instead? Minus the test > cases which should go to create_table.sql of course. Thanks for looking at this. Your patch seems better, because it allows us to keep the error message consistent with the message one would get with list-partitioned syntax. Thanks, Amit
Re: BUG #15668: Server crash in transformPartitionRangeBounds
On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote:
> Maybe we should error out as follows in
> transformPartitionRangeBounds(), although that means we'll get
> different error message than when using list partitioning syntax:
Hm. I don't think that this is a good idea as you could lose some
information for the expression transformation handling, and the error
handling becomes inconsistent depending on the partition bound
strategy. It seems to me that if we cannot extract any special value
from the ColumnRef expression generated, then we ought to let
transformPartitionBoundValue() and particularly transformExprRecurse()
do the analysis work and complain if needed:
=# CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM
(unknown.unknown) TO (1);
ERROR: 42P01: missing FROM-clause entry for table "unknown"
LINE 1: ...p_part PARTITION OF range_parted FOR VALUES FROM
(unknown.un...
=# CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM
(a.a.a.a.a.a.a.a.a.a.a.a) TO (1);
ERROR: 42601: improper qualified name (too many dotted names):
a.a.a.a.a.a.a.a.a.a.a.a
LINE 1: ...p_part PARTITION OF range_parted FOR VALUES FROM
(a.a.a.a.a
What about something like the attached instead? Minus the test
cases which should go to create_table.sql of course.
--
Michael
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..ee129ba18f 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -3750,8 +3750,16 @@ transformPartitionRangeBounds(ParseState *pstate, List *blist,
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
- Assert(cname != NULL);
- if (strcmp("minvalue", cname) == 0)
+ if (cname == NULL)
+ {
+/*
+ * No field names have been found, meaning that there
+ * is not much to do with special value handling. Instead
+ * let the expression transformation handle any errors and
+ * limitations.
+ */
+ }
+ else if (strcmp("minvalue", cname) == 0)
{
prd = makeNode(PartitionRangeDatum);
prd->kind = PARTITION_RANGE_DATUM_MINVALUE;
signature.asc
Description: PGP signature
Re: BUG #15668: Server crash in transformPartitionRangeBounds
Hi,
(cc'ing -hackers and Peter E)
On Tue, Mar 5, 2019 at 8:02 PM PG Bug reporting form
wrote:
>
> The following bug has been logged on the website:
>
> Bug reference: 15668
> Logged by: Alexander Lakhin
> Email address: exclus...@gmail.com
> PostgreSQL version: Unsupported/Unknown
> Operating system: Ubuntu 18.04
> Description:
>
> The following query:
> CREATE TABLE range_parted (a int) PARTITION BY RANGE (a);
> CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM
> (unknown.unknown) TO (1);
>
> crashes server (on the master branch) with the stack trace:
> Core was generated by `postgres: law regression [local] CREATE TABLE
> '.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 0x560ab19ea0bc in transformPartitionRangeBounds
> (pstate=pstate@entry=0x560ab3290da8, blist=,
> parent=parent@entry=0x7f7846a6bea8) at parse_utilcmd.c:3754
> 3754if (strcmp("minvalue", cname) == 0)
Thanks for the report. Seems to be a bug of the following commit in
HEAD, of which I was one of the authors:
commit 7c079d7417a8f2d4bf5144732e2f85117db9214f
Author: Peter Eisentraut
Date: Fri Jan 25 11:27:59 2019 +0100
Allow generalized expression syntax for partition bounds
That seems to be caused by some over-optimistic coding in
transformPartitionRangeBounds. Following will crash too.
CREATE TABLE rp_part PARTITION OF range_parted FOR VALUES FROM
(a.a.a.a.a.a.a.a.a.a.a.a) TO (1);
If I try the list partitioning syntax, it doesn't crash but gives the
following error:
create table lparted1 partition of lparted for values in (a.a.a.a.a.a);
ERROR: improper qualified name (too many dotted names): a.a.a.a.a.a
LINE 1: ...able lparted1 partition of lparted for values in (a.a.a.a.a
^
Maybe we should error out as follows in
transformPartitionRangeBounds(), although that means we'll get
different error message than when using list partitioning syntax:
@@ -3749,6 +3749,12 @@ transformPartitionRangeBounds(ParseState
*pstate, List *blist,
if (list_length(cref->fields) == 1 &&
IsA(linitial(cref->fields), String))
cname = strVal(linitial(cref->fields));
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+errmsg("invalid expression for range bound"),
+parser_errposition(pstate,
+ exprLocation((Node *) expr;
Assert(cname != NULL);
if (strcmp("minvalue", cname) == 0)
Thanks,
Amit
