[COMMITTERS] pgsql: Fix incorrect varlevelsup in security_barrier_replace_vars().

[Dean Rasheed]

Fix incorrect varlevelsup in security_barrier_replace_vars().

When converting an RTE with securityQuals into a security barrier
subquery RTE, ensure that the Vars in the new subquery's targetlist
all have varlevelsup = 0 so that they correctly refer to the
underlying base relation being wrapped.

The original code was creating new Vars by copying them from existing
Vars referencing the base relation found elsewhere in the query, but
failed to account for the fact that such Vars could come from sublink
subqueries, and hence have varlevelsup > 0. In practice it looks like
this could only happen with nested security barrier views, where the
outer view has a WHERE clause containing a correlated subquery, due to
the order in which the Vars are processed.

Bug: #13988
Reported-by: Adam Guthrie
Backpatch-to: 9.4, where updatable SB views were introduced

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/41fedc24626696fdf55d0c43131d91757dbe1c66

Modified Files
--
src/backend/optimizer/prep/prepsecurity.c |  1 +
src/test/regress/expected/updatable_views.out | 39 +++
src/test/regress/sql/updatable_views.sql  | 34 +++
3 files changed, 74 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix incorrect varlevelsup in security_barrier_replace_vars().

[Dean Rasheed]

Fix incorrect varlevelsup in security_barrier_replace_vars().

When converting an RTE with securityQuals into a security barrier
subquery RTE, ensure that the Vars in the new subquery's targetlist
all have varlevelsup = 0 so that they correctly refer to the
underlying base relation being wrapped.

The original code was creating new Vars by copying them from existing
Vars referencing the base relation found elsewhere in the query, but
failed to account for the fact that such Vars could come from sublink
subqueries, and hence have varlevelsup > 0. In practice it looks like
this could only happen with nested security barrier views, where the
outer view has a WHERE clause containing a correlated subquery, due to
the order in which the Vars are processed.

Bug: #13988
Reported-by: Adam Guthrie
Backpatch-to: 9.4, where updatable SB views were introduced

Branch
--
REL9_4_STABLE

Details
---
http://git.postgresql.org/pg/commitdiff/9b69d5c1daeb62804eeaeab2f804191dbe0a943d

Modified Files
--
src/backend/optimizer/prep/prepsecurity.c |  1 +
src/test/regress/expected/updatable_views.out | 39 +++
src/test/regress/sql/updatable_views.sql  | 34 +++
3 files changed, 74 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add pg_size_bytes() to parse human-readable size strings.

[Dean Rasheed]

Add pg_size_bytes() to parse human-readable size strings.

This will parse strings in the format produced by pg_size_pretty() and
return sizes in bytes. This allows queries to be written with clauses
like "pg_total_relation_size(oid) > pg_size_bytes('10 GB')".

Author: Pavel Stehule with various improvements by Vitaly Burovoy
Discussion: 
http://www.postgresql.org/message-id/cafj8prd-tgodknxdygecza4on01_urqprwf-8ldkse-6bdh...@mail.gmail.com
Reviewed-by: Vitaly Burovoy, Oleksandr Shulgin, Kyotaro Horiguchi,
Michael Paquier and Robert Haas

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/53874c5228fe16589a4d01b3e1fab3678e0fd8e3

Modified Files
--
doc/src/sgml/func.sgml   |  32 +++-
src/backend/utils/adt/dbsize.c   | 149 +++
src/include/catalog/catversion.h |   2 +-
src/include/catalog/pg_proc.h|   2 +
src/include/utils/builtins.h |   1 +
src/test/regress/expected/dbsize.out | 109 +
src/test/regress/sql/dbsize.sql  |  39 +
7 files changed, 331 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_size_bytes() to parse human-readable size strings.

[Dean Rasheed]

On 20 February 2016 at 10:12, Michael Paquier  wrote:
> Happy first commit.

Arg. Not so much.

Looks like I broke something -- looking into it now :-(

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_size_bytes() to parse human-readable size strings.

[Dean Rasheed]

On 20 February 2016 at 10:33, Michael Paquier <michael.paqu...@gmail.com> wrote:
> On Sat, Feb 20, 2016 at 7:17 PM, Dean Rasheed <dean.a.rash...@gmail.com> 
> wrote:
>> On 20 February 2016 at 10:12, Michael Paquier <michael.paqu...@gmail.com> 
>> wrote:
>>> Happy first commit.
>>
>> Arg. Not so much.
>>
>> Looks like I broke something -- looking into it now :-(
>
> The terabyte conversion is at fault:
> Expected:
> !  -1tb  |-1099511627776
> Result:
> !  -1tb  |-1
>
> +   else if (pg_strcasecmp(strptr, "gb") == 0)
> +   multiplier = 1024 * 1024 * 1024;
> +   else if (pg_strcasecmp(strptr, "tb") == 0)
> +   multiplier = 1024 * 1024 * 1024 * 1024L;
> Why adding an 'L' here?

Ah, looks like it needs to be 'LL' because it needs to be a 64-bit literal.

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix pg_size_bytes() to be more portable.

[Dean Rasheed]

Fix pg_size_bytes() to be more portable.

Commit 53874c5228fe16589a4d01b3e1fab3678e0fd8e3 broke various 32-bit
buildfarm machines because it incorrectly used an 'L' suffix for what
needed to be a 64-bit literal. Thanks to Michael Paquier for helping
to diagnose this.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/ad7cc1c554980145b226a066afe56d9c777ce7ae

Modified Files
--
src/backend/utils/adt/dbsize.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Fix pg_size_bytes() to be more portable.

[Dean Rasheed]

On 20 February 2016 at 14:54, Tom Lane <t...@sss.pgh.pa.us> wrote:
> Dean Rasheed <dean.a.rash...@gmail.com> writes:
>> Fix pg_size_bytes() to be more portable.
>> Commit 53874c5228fe16589a4d01b3e1fab3678e0fd8e3 broke various 32-bit
>> buildfarm machines because it incorrectly used an 'L' suffix for what
>> needed to be a 64-bit literal. Thanks to Michael Paquier for helping
>> to diagnose this.
>
> That's still not right: not all compilers support "long long", and the
> ones that don't won't have "LL" notation either.
>
> Project style is to use something like "(uint64) 1024" instead.
>

OK, will fix.

BTW, I found a couple of instances of 'LL' in ecpglib/prepare.c, which
is why I thought it was OK to use it.

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Further fixing to make pg_size_bytes() portable.

[Dean Rasheed]

Further fixing to make pg_size_bytes() portable.

Not all compilers support "long long" and the "LL" integer literal
suffix, so use a cast to int64 instead.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/740d71842b8e0e798c80f4f841227b6de81b5f43

Modified Files
--
src/backend/utils/adt/dbsize.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Fix pg_size_bytes() to be more portable.

[Dean Rasheed]

On 20 February 2016 at 16:32, Tom Lane  wrote:
> After further thought about the portability implications of this ---
>
> 1. We probably gave up support for long-long-less compilers when we agreed
> to start requiring a working int64 type.  On a 32-bit machine that's
> highly likely to be "long long", and 64-bit machines are mostly new enough
> that they'd have C99-compliant compilers.
>
> 2. Nonetheless, LL is not the same as int64; on modern 64-bit machines it
> probably means int128.  So the right thing to do when writing a constant
> you mean to be int64 is to use a cast or [U]INT64CONST().  (You need that
> macro if the constant value might be too wide for plain int, since pickier
> compilers may reject an unsuffixed constant wider than int.)
>

OK, thanks that's all good to know.


> Your updated code looks fine from here.  I looked into changing that code
> in ecpg, but it would be more invasive than I thought because ecpg doesn't
> use c.h.  Some rearrangement of the ecpg headers would be required, and
> in view of point #1, it's unlikely to be worth it; it might buy a bit of
> micro-efficiency but not much.
>

It looks to me as though it doesn't need long long anyway, since the
rotation it's doing can be done just as easily with ints, for example:

int hashVal = 0;
for (...)
{
hashVal = hashVal + (int) ecpgQuery[stmtIx];
hashVal = (((unsigned int) hashVal) >> 19) | (hashVal << 13);
}

but it's probably not worth changing, for risk of breaking it.

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Fix pg_size_bytes() to be more portable.

[Dean Rasheed]

On 20 February 2016 at 17:24, Dean Rasheed <dean.a.rash...@gmail.com> wrote:
> It looks to me as though it doesn't need long long anyway, since the
> rotation it's doing can be done just as easily with ints, for example:

Oh, scratch that -- I was assuming int would be 32-bit, which it might not be.

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Improve estimate of distinct values in estimate_num_groups().

[Dean Rasheed]

Improve estimate of distinct values in estimate_num_groups().

When adjusting the estimate for the number of distinct values from a
rel in a grouped query to take into account the selectivity of the
rel's restrictions, use a formula that is less likely to produce
under-estimates.

The old formula simply multiplied the number of distinct values in the
rel by the restriction selectivity, which would be correct if the
restrictions were fully correlated with the grouping expressions, but
can produce significant under-estimates in cases where they are not
well correlated.

The new formula is based on the random selection probability, and so
assumes that the restrictions are not correlated with the grouping
expressions. This is guaranteed to produce larger estimates, and of
course risks over-estimating in cases where the restrictions are
correlated, but that has less severe consequences than
under-estimating, which might lead to a HashAgg that consumes an
excessive amount of memory.

This could possibly be improved upon in the future by identifying
correlated restrictions and using a hybrid of the old and new
formulae.

Author: Tomas Vondra, with some hacking be me
Reviewed-by: Mark Dilger, Alexander Korotkov, Dean Rasheed and Tom Lane
Discussion: 
http://www.postgresql.org/message-id/flat/56cd0381.5060...@2ndquadrant.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/84f9a35e398f863c62440d3f82fc57b4fedc5d08

Modified Files
--
src/backend/utils/adt/selfuncs.c| 64 +++--
src/test/regress/expected/subselect.out | 25 ++---
2 files changed, 64 insertions(+), 25 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix corner-case loss of precision in numeric pow() calculation

[Dean Rasheed]

Fix corner-case loss of precision in numeric pow() calculation

Commit 7d9a4737c268f61fb8800957631f12d3f13be218 greatly improved the
accuracy of the numeric transcendental functions, however it failed to
consider the case where the result from pow() is close to the overflow
threshold, for example 0.12 ^ -2345.6. For such inputs, where the
result has more than 2000 digits before the decimal point, the decimal
result weight estimate was being clamped to 2000, leading to a loss of
precision in the final calculation.

Fix this by replacing the clamping code with an overflow test that
aborts the calculation early if the final result is sure to overflow,
based on the overflow limit in exp_var(). This provides the same
protection against integer overflow in the subsequent result scale
computation as the original clamping code, but it also ensures that
precision is never lost and saves compute cycles in cases that are
sure to overflow.

The new early overflow test works with the initial low-precision
result (expected to be accurate to around 8 significant digits) and
includes a small fuzz factor to ensure that it doesn't kick in for
values that would not overflow exp_var(), so the overall overflow
threshold of pow() is unchanged and consistent for all inputs with
non-integer exponents.

Author: Dean Rasheed
Reviewed-by: Tom Lane
Discussion: 
http://www.postgresql.org/message-id/CAEZATCUj3U-cQj0jjoia=qgs0sje3auroxh8swvnkvzwuqe...@mail.gmail.com
See-also: 
http://www.postgresql.org/message-id/CAEZATCV7w+8iB=07dj8q0zihxqt1semcqutek+4_rogc_zq...@mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/18a02ad2a506e4425c6dd2ea235039cd5659467d

Modified Files
--
src/backend/utils/adt/numeric.c   | 20 
src/test/regress/expected/numeric_big.out | 14 ++
src/test/regress/sql/numeric_big.sql  | 10 ++
3 files changed, 40 insertions(+), 4 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Move and rename fmtReloptionsArray().

[Dean Rasheed]

Move and rename fmtReloptionsArray().

Move fmtReloptionsArray() from pg_dump.c to string_utils.c so that it
is available to other frontend code. In particular psql's \ev and \sv
commands need it to handle view reloptions. Also rename the function
to appendReloptionsArray(), which is a more accurate description of
what it does.

Author: Dean Rasheed
Reviewed-by: Peter Eisentraut
Discussion: 
http://www.postgresql.org/message-id/CAEZATCWZjCgKRyM-agE0p8ax15j9uyQoF=qew7d2xb6cf76...@mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/93a8c6fd6c5eeb61c12402f616a327d998a731c4

Modified Files
--
src/bin/pg_dump/pg_dump.c   | 77 +++--
src/fe_utils/string_utils.c | 72 ++
src/include/fe_utils/string_utils.h |  3 ++
3 files changed, 89 insertions(+), 63 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix psql's \ev and \sv commands so that they handle view relopti

[Dean Rasheed]

Fix psql's \ev and \sv commands so that they handle view reloptions.

Commit 8eb6407aaeb6cbd972839e356b436bb698f51cff added support for
editing and showing view definitions, but neglected to account for
view options such as security_barrier and WITH CHECK OPTION which are
not returned by pg_get_viewdef() and so need special handling.

Author: Dean Rasheed
Reviewed-by: Peter Eisentraut
Discussion: 
http://www.postgresql.org/message-id/CAEZATCWZjCgKRyM-agE0p8ax15j9uyQoF=qew7d2xb6cf76...@mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/9b66aa006f81b2705337ca223daeeabf4db6453a

Modified Files
--
src/bin/psql/command.c | 77 +-
1 file changed, 70 insertions(+), 7 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix order of operations in CREATE OR REPLACE VIEW.

[Dean Rasheed]

Fix order of operations in CREATE OR REPLACE VIEW.

When CREATE OR REPLACE VIEW acts on an existing view, don't update the
view options until after the view query has been updated.

This is necessary in the case where CREATE OR REPLACE VIEW is used on
an existing view that is not updatable, and the new view is updatable
and specifies the WITH CHECK OPTION. In this case, attempting to apply
the new options to the view before updating its query fails, because
the options are applied using the ALTER TABLE infrastructure which
checks that WITH CHECK OPTION is only applied to an updatable view.

If new columns are being added to the view, that is also done using
the ALTER TABLE infrastructure, but it is important that that still be
done before updating the view query, because the rules system checks
that the query columns match those on the view relation. Added a
comment to explain that, in case someone is tempted to move that to
where the view options are now being set.

Back-patch to 9.4 where WITH CHECK OPTION was added.

Report: 
https://postgr.es/m/CAEZATCUp%3Dz%3Ds4SzZjr14bfct_bdJNwMPi-gFi3Xc5k1ntbsAgQ%40mail.gmail.com

Branch
--
REL9_4_STABLE

Details
---
http://git.postgresql.org/pg/commitdiff/cad24980ef40a648fc72727ca14557968db29295

Modified Files
--
src/backend/commands/view.c   | 78 +--
src/test/regress/expected/updatable_views.out | 13 +
src/test/regress/sql/updatable_views.sql  | 14 +
3 files changed, 76 insertions(+), 29 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix order of operations in CREATE OR REPLACE VIEW.

[Dean Rasheed]

Fix order of operations in CREATE OR REPLACE VIEW.

When CREATE OR REPLACE VIEW acts on an existing view, don't update the
view options until after the view query has been updated.

This is necessary in the case where CREATE OR REPLACE VIEW is used on
an existing view that is not updatable, and the new view is updatable
and specifies the WITH CHECK OPTION. In this case, attempting to apply
the new options to the view before updating its query fails, because
the options are applied using the ALTER TABLE infrastructure which
checks that WITH CHECK OPTION is only applied to an updatable view.

If new columns are being added to the view, that is also done using
the ALTER TABLE infrastructure, but it is important that that still be
done before updating the view query, because the rules system checks
that the query columns match those on the view relation. Added a
comment to explain that, in case someone is tempted to move that to
where the view options are now being set.

Back-patch to 9.4 where WITH CHECK OPTION was added.

Report: 
https://postgr.es/m/CAEZATCUp%3Dz%3Ds4SzZjr14bfct_bdJNwMPi-gFi3Xc5k1ntbsAgQ%40mail.gmail.com

Branch
--
REL9_5_STABLE

Details
---
http://git.postgresql.org/pg/commitdiff/78a98b7674cf7d5d82001f6d8d4ebcb8344fc0cd

Modified Files
--
src/backend/commands/view.c   | 78 +--
src/test/regress/expected/updatable_views.out | 13 +
src/test/regress/sql/updatable_views.sql  | 14 +
3 files changed, 76 insertions(+), 29 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix order of operations in CREATE OR REPLACE VIEW.

[Dean Rasheed]

Fix order of operations in CREATE OR REPLACE VIEW.

When CREATE OR REPLACE VIEW acts on an existing view, don't update the
view options until after the view query has been updated.

This is necessary in the case where CREATE OR REPLACE VIEW is used on
an existing view that is not updatable, and the new view is updatable
and specifies the WITH CHECK OPTION. In this case, attempting to apply
the new options to the view before updating its query fails, because
the options are applied using the ALTER TABLE infrastructure which
checks that WITH CHECK OPTION is only applied to an updatable view.

If new columns are being added to the view, that is also done using
the ALTER TABLE infrastructure, but it is important that that still be
done before updating the view query, because the rules system checks
that the query columns match those on the view relation. Added a
comment to explain that, in case someone is tempted to move that to
where the view options are now being set.

Back-patch to 9.4 where WITH CHECK OPTION was added.

Report: 
https://postgr.es/m/CAEZATCUp%3Dz%3Ds4SzZjr14bfct_bdJNwMPi-gFi3Xc5k1ntbsAgQ%40mail.gmail.com

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/58b1362642d47bd7a7ed1157035a38671555e860

Modified Files
--
src/backend/commands/view.c   | 78 +--
src/test/regress/expected/updatable_views.out | 13 +
src/test/regress/sql/updatable_views.sql  | 14 +
3 files changed, 76 insertions(+), 29 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix order of operations in CREATE OR REPLACE VIEW.

[Dean Rasheed]

Fix order of operations in CREATE OR REPLACE VIEW.

When CREATE OR REPLACE VIEW acts on an existing view, don't update the
view options until after the view query has been updated.

This is necessary in the case where CREATE OR REPLACE VIEW is used on
an existing view that is not updatable, and the new view is updatable
and specifies the WITH CHECK OPTION. In this case, attempting to apply
the new options to the view before updating its query fails, because
the options are applied using the ALTER TABLE infrastructure which
checks that WITH CHECK OPTION is only applied to an updatable view.

If new columns are being added to the view, that is also done using
the ALTER TABLE infrastructure, but it is important that that still be
done before updating the view query, because the rules system checks
that the query columns match those on the view relation. Added a
comment to explain that, in case someone is tempted to move that to
where the view options are now being set.

Back-patch to 9.4 where WITH CHECK OPTION was added.

Report: 
https://postgr.es/m/CAEZATCUp%3Dz%3Ds4SzZjr14bfct_bdJNwMPi-gFi3Xc5k1ntbsAgQ%40mail.gmail.com

Branch
--
REL9_6_STABLE

Details
---
http://git.postgresql.org/pg/commitdiff/a46ee6b30ae9cba3ece5b4b5912969c74969f274

Modified Files
--
src/backend/commands/view.c   | 78 +--
src/test/regress/expected/updatable_views.out | 13 +
src/test/regress/sql/updatable_views.sql  | 14 +
3 files changed, 76 insertions(+), 29 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Fix base backup rate limiting in presence of slow i/o

[Dean Rasheed]

On 19 December 2016 at 09:17, Magnus Hagander  wrote:
> Fix base backup rate limiting in presence of slow i/o
>
> Branch
> --
> REL9_4_STABLE

I'm seeing a warning on the 9.4 branch that looks to have been caused
by this commit:

basebackup.c: In function ‘throttle’:
basebackup.c:1284:8: warning: variable ‘wait_result’ set but not used
[-Wunused-but-set-variable]
  int   wait_result;

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Fix base backup rate limiting in presence of slow i/o

[Dean Rasheed]

On 21 December 2016 at 20:20, Magnus Hagander <mag...@hagander.net> wrote:
> On Wed, Dec 21, 2016 at 6:55 PM, Dean Rasheed <dean.a.rash...@gmail.com>
>> basebackup.c: In function ‘throttle’:
>> basebackup.c:1284:8: warning: variable ‘wait_result’ set but not used
>> [-Wunused-but-set-variable]
>>   int   wait_result;
>
> Interesting.
>
> ff44fba4 replaced the latch in walsender, which was not backported (of
> course).
>
> But it also added a CHECK_FOR_INTERRUPTS there.
>
> 9.4 does not have a CHECK_FOR_INTERRUPTS anywhere near there. Perhaps what's
> really needed is to put one of those in regardless?
>

Yeah, it seems reasonable that there should be a CHECK_FOR_INTERRUPTS() there.

I'm not really familiar with that code, but it looks like the signal
handler in 9.4 might not set that latch, so throttle() would have to
rely on setting ImmediateInterruptOK to make the sleep interruptable,
in which case the wait result would be irrelevant, right?

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Comment fix for partition_rbound_cmp().

[Dean Rasheed]

Comment fix for partition_rbound_cmp().

This was an oversight in d363d42.

Beena Emerson

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/4de6216877a32e869fe1cf168c1fe1ffb8c3d576

Modified Files
--
src/backend/catalog/partition.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition b

[Dean Rasheed]

Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition bounds.

Previously, UNBOUNDED meant no lower bound when used in the FROM list,
and no upper bound when used in the TO list, which was OK for
single-column range partitioning, but problematic with multiple
columns. For example, an upper bound of (10.0, UNBOUNDED) would not be
collocated with a lower bound of (10.0, UNBOUNDED), thus making it
difficult or impossible to define contiguous multi-column range
partitions in some cases.

Fix this by using MINVALUE and MAXVALUE instead of UNBOUNDED to
represent a partition column that is unbounded below or above
respectively. This syntax removes any ambiguity, and ensures that if
one partition's lower bound equals another partition's upper bound,
then the partitions are contiguous.

Also drop the constraint prohibiting finite values after an unbounded
column, and just document the fact that any values after MINVALUE or
MAXVALUE are ignored. Previously it was necessary to repeat UNBOUNDED
multiple times, which was needlessly verbose.

Note: Forces a post-PG 10 beta2 initdb.

Report by Amul Sul, original patch by Amit Langote with some
additional hacking by me.

Discussion: 
https://postgr.es/m/CAAJ_b947mowpLdxL3jo3YLKngRjrq9+Ej4ymduQTfYR+8=y...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/d363d42bb9a4399a0207bd3b371c966e22e06bd3

Modified Files
--
doc/src/sgml/ref/create_table.sgml |  61 +++--
src/backend/catalog/partition.c| 211 ++---
src/backend/nodes/copyfuncs.c  |   2 +-
src/backend/nodes/equalfuncs.c |   2 +-
src/backend/nodes/outfuncs.c   |   2 +-
src/backend/nodes/readfuncs.c  |   2 +-
src/backend/parser/gram.y  |  16 ++-
src/backend/parser/parse_utilcmd.c |  34 -
src/backend/utils/adt/ruleutils.c  |  12 +-
src/include/catalog/catversion.h   |   2 +-
src/include/nodes/parsenodes.h |  16 ++-
src/test/regress/expected/create_table.out |  41 +++---
src/test/regress/expected/inherit.out  |   6 +-
src/test/regress/expected/insert.out   | 130 +-
src/test/regress/sql/create_table.sql  |  31 ++---
src/test/regress/sql/inherit.sql   |   6 +-
src/test/regress/sql/insert.sql|  39 +-
src/tools/pgindent/typedefs.list   |   1 +
18 files changed, 377 insertions(+), 237 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition b

[Dean Rasheed]

On 21 July 2017 at 09:24, Dean Rasheed <dean.a.rash...@gmail.com> wrote:
> Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition bounds.
>

Hmm, looks like the buildfarm doesn't like this.

It looks like the order of partitions listed by \d+ isn't entirely
predictable ... looking into it now.

Regards,
Dean


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Simplify the logic checking new range partition bounds.

[Dean Rasheed]

Simplify the logic checking new range partition bounds.

The previous logic, whilst not actually wrong, was overly complex and
involved doing two binary searches, where only one was really
necessary. This simplifies that logic and improves the comments.

One visible change is that if the new partition overlaps multiple
existing partitions, the error message now always reports the overlap
with the first existing partition (the one with the lowest
bounds). The old code would sometimes report the clash with the first
partition and sometimes with the last one.

Original patch idea from Amit Langote, substantially rewritten by me.

Discussion: 
https://postgr.es/m/CAAJ_b947mowpLdxL3jo3YLKngRjrq9+Ej4ymduQTfYR+8=y...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c03911d9454a0cf5d88910ad46b433ab342c39e0

Modified Files
--
src/backend/catalog/partition.c| 92 +-
src/test/regress/expected/create_table.out |  2 +-
2 files changed, 40 insertions(+), 54 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Clarify the contract of partition_rbound_cmp().

[Dean Rasheed]

Clarify the contract of partition_rbound_cmp().

partition_rbound_cmp() is intended to compare range partition bounds
in a way such that if all the bound values are equal but one is an
upper bound and one is a lower bound, the upper bound is treated as
smaller than the lower bound. This particular ordering is required by
RelationBuildPartitionDesc() when building the PartitionBoundInfoData,
so that it can consistently keep only the upper bounds when upper and
lower bounds coincide.

Update the function comment to make that clearer.

Also, fix a (currently unreachable) corner-case bug -- if the bound
values coincide and they contain unbounded values, fall through to the
lower-vs-upper comparison code, rather than immediately returning
0. Currently it is not possible to define coincident upper and lower
bounds containing unbounded columns, but that may change in the
future, so code defensively.

Discussion: 
https://postgr.es/m/CAAJ_b947mowpLdxL3jo3YLKngRjrq9+Ej4ymduQTfYR+8=y...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f1dae097f2945ffcb59a9f236843e0e0bbf0920d

Modified Files
--
src/backend/catalog/partition.c | 44 -
1 file changed, 30 insertions(+), 14 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Prevent table partitions from being turned into views.

[Dean Rasheed]

Prevent table partitions from being turned into views.

A table partition must be a table, not a view, so don't allow a
"_RETURN" rule to be added that would convert an existing table
partition into a view.

Amit Langote

Discussion: 
https://postgr.es/m/CAEZATCVzFcAjZwC1bTFvJ09skB_sgkF4SwPKMywev-XTnimp9Q%40mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/bcbf392ec84362040faf72daad22c647c74e2b2a

Modified Files
--
src/backend/rewrite/rewriteDefine.c | 6 ++
src/test/regress/expected/rules.out | 5 +
src/test/regress/sql/rules.sql  | 5 +
3 files changed, 16 insertions(+)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Teach PL/pgSQL about partitioned tables.

[Dean Rasheed]

Teach PL/pgSQL about partitioned tables.

Table partitioning, introduced in commit f0e44751d7, added a new
relkind - RELKIND_PARTITIONED_TABLE. Update a couple of places in
PL/pgSQL to handle it. Specifically plpgsql_parse_cwordtype() and
build_row_from_class() needed updating in order to make table%ROWTYPE
and table.col%TYPE work for partitioned tables.

Dean Rasheed, reviewed by Amit Langote.

Discussion: 
https://postgr.es/m/CAEZATCUnNOKN8sLML9jUzxecALWpEXK3a3W7y0PgFR4%2Buhgc%3Dg%40mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/d3c3f2b1e25cc96d3078bf4d47a2f58fefb70560

Modified Files
--
src/pl/plpgsql/src/pl_comp.c  |  6 +++--
src/test/regress/expected/plpgsql.out | 45 +++
src/test/regress/sql/plpgsql.sql  | 39 ++
3 files changed, 88 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Teach relation_is_updatable() about partitioned tables.

[Dean Rasheed]

Teach relation_is_updatable() about partitioned tables.

Table partitioning, introduced in commit f0e44751d7, added a new
relkind - RELKIND_PARTITIONED_TABLE. Update relation_is_updatable() to
handle it. Specifically, partitioned tables and simple views built on
top of them are updatable.

This affects the SQL-callable functions pg_relation_is_updatable() and
pg_column_is_updatable(), and the views information_schema.views and
information_schema.columns.

Dean Rasheed, reviewed by Ashutosh Bapat.

Discussion: 
https://postgr.es/m/CAEZATCXnbiFkMXgF4Ez1pmM2c-tS1z33bSq7OGbw7QQhHov%2B6Q%40mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/b6263cd851ef245a5dc38119448e029ca1592da4

Modified Files
--
src/backend/rewrite/rewriteHandler.c  |  6 +++--
src/test/regress/expected/updatable_views.out | 36 +++
src/test/regress/sql/updatable_views.sql  | 10 
3 files changed, 50 insertions(+), 2 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Improve the CREATE POLICY documentation.

[Dean Rasheed]

Improve the CREATE POLICY documentation.

Provide a correct description of how multiple policies are combined,
clarify when SELECT permissions are required, mention SELECT FOR
UPDATE/SHARE, and do some other more minor tidying up.

Reviewed by Stephen Frost

Discussion: 
https://postgr.es/m/CAEZATCVrxyYbOFU8XbGHicz%2BmXPYzw%3DhfNL2XTphDt-53TomQQ%40mail.gmail.com

Back-patch to 9.5.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/af44cbd5ecd7e1db0ae4bce75c8f1bce14b1d6db

Modified Files
--
doc/src/sgml/ref/create_policy.sgml | 206 +++-
1 file changed, 134 insertions(+), 72 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Improve the CREATE POLICY documentation.

[Dean Rasheed]

Improve the CREATE POLICY documentation.

Provide a correct description of how multiple policies are combined,
clarify when SELECT permissions are required, mention SELECT FOR
UPDATE/SHARE, and do some other more minor tidying up.

Reviewed by Stephen Frost

Discussion: 
https://postgr.es/m/CAEZATCVrxyYbOFU8XbGHicz%2BmXPYzw%3DhfNL2XTphDt-53TomQQ%40mail.gmail.com

Back-patch to 9.5.

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/af4bd075911d449ff1627188a4175d71d4e317d2

Modified Files
--
doc/src/sgml/ref/create_policy.sgml | 170 +++-
1 file changed, 108 insertions(+), 62 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Improve the CREATE POLICY documentation.

[Dean Rasheed]

Improve the CREATE POLICY documentation.

Provide a correct description of how multiple policies are combined,
clarify when SELECT permissions are required, mention SELECT FOR
UPDATE/SHARE, and do some other more minor tidying up.

Reviewed by Stephen Frost

Discussion: 
https://postgr.es/m/CAEZATCVrxyYbOFU8XbGHicz%2BmXPYzw%3DhfNL2XTphDt-53TomQQ%40mail.gmail.com

Back-patch to 9.5.

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/6a21eb31b6deb0f76e8304fb9f12df272ed59f0c

Modified Files
--
doc/src/sgml/ref/create_policy.sgml | 170 +++-
1 file changed, 108 insertions(+), 62 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Improve the CREATE POLICY documentation.

[Dean Rasheed]

Improve the CREATE POLICY documentation.

Provide a correct description of how multiple policies are combined,
clarify when SELECT permissions are required, mention SELECT FOR
UPDATE/SHARE, and do some other more minor tidying up.

Reviewed by Stephen Frost

Discussion: 
https://postgr.es/m/CAEZATCVrxyYbOFU8XbGHicz%2BmXPYzw%3DhfNL2XTphDt-53TomQQ%40mail.gmail.com

Back-patch to 9.5.

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4d5d08c1cd52add02bdfadc00854135a3b6c88f6

Modified Files
--
doc/src/sgml/ref/create_policy.sgml | 206 +++-
1 file changed, 134 insertions(+), 72 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/045a1f38bd46f5b50e145470095f461cc41c

Modified Files
--
src/backend/catalog/pg_constraint.c   | 99 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint.h   |  2 +
src/test/regress/expected/privileges.out  | 18 +-
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 21 ++-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 197 insertions(+), 13 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/87b2ebd352c4afe1ded0841604b59a3afbae97d1

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3f80895723037c0d1c684dbdd50b7e03453df90f

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Always require SELECT permission for ON CONFLICT DO UPDATE.

[Dean Rasheed]

Always require SELECT permission for ON CONFLICT DO UPDATE.

The update path of an INSERT ... ON CONFLICT DO UPDATE requires SELECT
permission on the columns of the arbiter index, but it failed to check
for that in the case of an arbiter specified by constraint name.

In addition, for a table with row level security enabled, it failed to
check updated rows against the table's SELECT policies when the update
path was taken (regardless of how the arbiter index was specified).

Backpatch to 9.5 where ON CONFLICT DO UPDATE and RLS were introduced.

Security: CVE-2017-15099

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1f23d1cd21ed46dba882729bedd9c40b71995989

Modified Files
--
src/backend/catalog/pg_constraint.c   | 98 +++
src/backend/parser/parse_clause.c | 21 ++-
src/backend/rewrite/rowsecurity.c | 20 ++-
src/include/catalog/pg_constraint_fn.h|  2 +
src/test/regress/expected/privileges.out  | 16 -
src/test/regress/expected/rowsecurity.out | 15 -
src/test/regress/sql/privileges.sql   | 19 +-
src/test/regress/sql/rowsecurity.sql  | 14 -
8 files changed, 194 insertions(+), 11 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers