[COMMITTERS] pgsql: Make WAL segment size configurable at initdb time.

2017-09-19 Thread Andres Freund
Make WAL segment size configurable at initdb time. For performance reasons a larger segment size than the default 16MB can be useful. A larger segment size has two main benefits: Firstly, in setups using archiving, it makes it easier to write scripts that can keep up with higher amounts of WAL, se

[COMMITTERS] pgsql: Accept that server might not be able to send error in crash reco

2017-09-19 Thread Andres Freund
Accept that server might not be able to send error in crash recovery test. As it turns out we can't rely that the script's monitoring session is terminated with a proper error by the server, because the session might be terminated while already trying to send data. Also improve robustness and err

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 19:00:38 -0700, Andres Freund wrote: > Given this fact pattern, I'll allow the case without a received error > message in the recovery test. Objections? Hearing none. Pushed. While debugging this, I've also introduced a pump wrapper so that we now get: ok 4 - exactly one process kil

[COMMITTERS] pgsql: Allow no-op GiST support functions to be omitted.

2017-09-19 Thread Tom Lane
Allow no-op GiST support functions to be omitted. There are common use-cases in which the compress and/or decompress functions can be omitted, with the result being that we make no data transformation when storing or retrieving index values. Previously, you had to provide a no-op function anyway,

[COMMITTERS] pgsql: Remove no-op GiST support functions in the core GiST opclasses.

2017-09-19 Thread Tom Lane
Remove no-op GiST support functions in the core GiST opclasses. The preceding patch allowed us to remove useless GiST support functions. This patch actually does that for all the no-op cases in the core GiST code. This buys us whatever performance gain is to be had, and more importantly exercises

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 18:06:29 -0700, Andres Freund wrote: > On 2017-09-19 16:46:58 -0400, Tom Lane wrote: > > Have we forgotten an fflush() or something? > > After hacking a fix for my previous theory, I started adding strace into > the mix, to verify this. Takes longer to reproduce, but after filtering

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 16:46:58 -0400, Tom Lane wrote: > Andres Freund writes: > > So this is geniuinely interesting. When the machine is really loaded (as > > in 6 animals running on a vm at the same time, incuding valgrind), psql > > sometimes doesn't get the WARNING message from a shutdown. Instead it >

Re: [COMMITTERS] pgsql: Set partitioned_rels appropriately when UNION ALL is used.

2017-09-19 Thread Thomas Munro
On Fri, Sep 15, 2017 at 3:06 AM, Robert Haas wrote: > Set partitioned_rels appropriately when UNION ALL is used. > > In most cases, this omission won't matter, because the appropriate > locks will have been acquired during parse/plan or by AcquireExecutorLocks. > But it's a bug all the same. > > R

[COMMITTERS] pgsql: s/NULL byte/NUL byte/ in comment refering to C string terminator

2017-09-19 Thread Andres Freund
s/NULL byte/NUL byte/ in comment refering to C string terminator. Reported-By: Robert Haas Discussion: https://postgr.es/m/CA+Tgmoa+YBvWgFST2NVoeXjVSohEpK=vqnvcsockhtvvxfl...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/896537f078ba4d709ce754ecaff

Re: [COMMITTERS] pgsql: Avoid use of non-portable strnlen() in pgstat_clip_activity().

2017-09-19 Thread Robert Haas
On Tue, Sep 19, 2017 at 5:27 PM, Andres Freund wrote: > Avoid use of non-portable strnlen() in pgstat_clip_activity(). > > The use of strnlen rather than strlen was just paranoia. Instead of > giving up on the paranoia, just implement the safeguard > differently. And add a comment explaining why w

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 13:53:18 -0700, Andres Freund wrote: > On 2017-09-19 16:46:58 -0400, Tom Lane wrote: > > Have we forgotten an fflush() or something? > > > > Also, maybe problem is on client side. I vaguely recall a libpq bug > > wherein it would complain about socket EOF even though data remained >

[COMMITTERS] pgsql: Add basic TAP test setup for pg_upgrade

2017-09-19 Thread Peter Eisentraut
Add basic TAP test setup for pg_upgrade The plan is to convert the current pg_upgrade test to the TAP framework. This commit just puts a basic TAP test in place so that we can see how the build farm behaves, since the build farm client has some special knowledge of the pg_upgrade tests. Author:

[COMMITTERS] pgsql: Tag refs/tags/REL_10_RC1 was created

2017-09-19 Thread pgsql
Tag refs/tags/REL_10_RC1 was created. -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Avoid use of non-portable strnlen() in pgstat_clip_activity().

2017-09-19 Thread Andres Freund
Avoid use of non-portable strnlen() in pgstat_clip_activity(). The use of strnlen rather than strlen was just paranoia. Instead of giving up on the paranoia, just implement the safeguard differently. And add a comment explaining why we're careful. Author: Andres Freund Discussion: https://postgr.

Re: [COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Tom Lane
Andres Freund writes: > I'll just change things so that the pnstrdup() happens first, and then > do a plain strlen() on that. Same paranoia, less cost. WFM. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Tom Lane
Andres Freund writes: > I'll see if I can spot the bug in an hour or two. If not I'll make the > test temporarily accept both outputs while investigating? Seems reasonable. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To

Re: [COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Andres Freund
On 2017-09-19 16:53:09 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-09-19 16:25:31 -0400, Tom Lane wrote: > >> Looks like you're going to need to not depend on strnlen(). > > > Yup noticed - we have pg_strnlen as a static function in > > src/port/snprintf. I'm tempted to just make t

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 16:46:58 -0400, Tom Lane wrote: > Andres Freund writes: > > So this is geniuinely interesting. When the machine is really loaded (as > > in 6 animals running on a vm at the same time, incuding valgrind), psql > > sometimes doesn't get the WARNING message from a shutdown. Instead it >

Re: [COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Tom Lane
Andres Freund writes: > On 2017-09-19 16:25:31 -0400, Tom Lane wrote: >> Looks like you're going to need to not depend on strnlen(). > Yup noticed - we have pg_strnlen as a static function in > src/port/snprintf. I'm tempted to just make that public, rather than > open code the equivalent? No, w

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Tom Lane
Andres Freund writes: > So this is geniuinely interesting. When the machine is really loaded (as > in 6 animals running on a vm at the same time, incuding valgrind), psql > sometimes doesn't get the WARNING message from a shutdown. Instead it > gets > # psql::3: server closed the connection unexpe

Re: [COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Andres Freund
On 2017-09-19 16:25:31 -0400, Tom Lane wrote: > Andres Freund writes: > > Speedup pgstat_report_activity by moving mb-aware truncation to read side. > > Looks like you're going to need to not depend on strnlen(). Yup noticed - we have pg_strnlen as a static function in src/port/snprintf. I'm tem

Re: [COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Tom Lane
Andres Freund writes: > Speedup pgstat_report_activity by moving mb-aware truncation to read side. Looks like you're going to need to not depend on strnlen(). regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 15:24:49 -0400, Tom Lane wrote: > Andres Freund writes: > > Checkining on calliphoridae why that's not sufficient - the machine's > > busy, so the build & test will take a bit. > > FWIW, prairiedog got through the recovery tests this time --- run's > still going though. So this is

[COMMITTERS] pgsql: Speedup pgstat_report_activity by moving mb-aware truncation to

2017-09-19 Thread Andres Freund
Speedup pgstat_report_activity by moving mb-aware truncation to read side. Previously multi-byte aware truncation was done on every pgstat_report_activity() call - proving to be a bottleneck for workloads with long query strings that execute quickly. Instead move the truncation to the read side,

[COMMITTERS] pgsql: Disable multi-byte citext tests

2017-09-19 Thread Andrew Dunstan
Disable multi-byte citext tests This reverts commit 890faaf1 which attempted unsuccessfully to deal with the problem, and instead just comments out these tests like other similar tests elsewhere in the script. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d1687c69

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Tom Lane
Andres Freund writes: > Checkining on calliphoridae why that's not sufficient - the machine's > busy, so the build & test will take a bit. FWIW, prairiedog got through the recovery tests this time --- run's still going though. regards, tom lane -- Sent via pgsql-commit

Re: [COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
On 2017-09-19 17:40:20 +, Andres Freund wrote: > Make new crash restart test a bit more robust. > > Add timeouts in case psql doesn't deliver the expected output, and try > to cause the monitoring psql to be fully connected to a backend. This > isn't necessarily everything needed, but at leas

[COMMITTERS] pgsql: Cache datatype-output-function lookup info across calls of conca

2017-09-19 Thread Tom Lane
Cache datatype-output-function lookup info across calls of concat(). Testing indicates this can save a third to a half of the runtime of the function. Pavel Stehule, reviewed by Alexander Kuzmenkov Discussion: https://postgr.es/m/cafj8prat62prgjohbgtfjuc2ulmeq4sauj+yvjaezuimwnc...@mail.gmail.co

[COMMITTERS] pgsql: Set client encoding to UTF8 for the citext regression script

2017-09-19 Thread Andrew Dunstan
Set client encoding to UTF8 for the citext regression script Problem introduced with non-ascii characters in commit f2464997644c and discovered on various buildfarm animals. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/890faaf1957759c6e17fbcbfd16f7cabc4a59d07 Mo

Re: [COMMITTERS] pgsql: Add test for postmaster crash restarts.

2017-09-19 Thread Andres Freund
On 2017-09-19 09:58:26 -0700, Andres Freund wrote: > > > On September 19, 2017 9:53:28 AM PDT, Tom Lane wrote: > >Well, please fix it ASAP, if you don't want to take it out pending > >the fixes. > > Will as soon as I finished my morning coffee. Uncaffeinated, which my phone > fittingly autocor

[COMMITTERS] pgsql: Make new crash restart test a bit more robust.

2017-09-19 Thread Andres Freund
Make new crash restart test a bit more robust. Add timeouts in case psql doesn't deliver the expected output, and try to cause the monitoring psql to be fully connected to a backend. This isn't necessarily everything needed, but at least the timeouts should reduce the pain for buildfarm owners.

Re: [COMMITTERS] pgsql: Add test for postmaster crash restarts.

2017-09-19 Thread Andres Freund
On September 19, 2017 9:53:28 AM PDT, Tom Lane wrote: >Well, please fix it ASAP, if you don't want to take it out pending >the fixes. Will as soon as I finished my morning coffee. Uncaffeinated, which my phone fittingly autocorrects to unvaccinated, commits aren't a good idea. Andres -- Sent

Re: [COMMITTERS] pgsql: Add test for postmaster crash restarts.

2017-09-19 Thread Tom Lane
Andres Freund writes: > On 2017-09-19 12:13:54 -0400, Tom Lane wrote: >> IOW, the "$monitor" instance of psql did not complete making its >> connection until after the crash/restart cycle had occurred. > That'd be easy enough to fix... > Just something like > $monitor_stdin .= q[ > SELECT $$am-

Re: [COMMITTERS] pgsql: Add test for postmaster crash restarts.

2017-09-19 Thread Andres Freund
Hi, On 2017-09-19 12:13:54 -0400, Tom Lane wrote: > IOW, the "$monitor" instance of psql did not complete making its > connection until after the crash/restart cycle had occurred. That'd be easy enough to fix... Just something like $monitor_stdin .= q[ SELECT $$am-i-up$$; ]; $monitor->pump unti

[COMMITTERS] pgsql: doc: add example of % substitution for connection URIs

2017-09-19 Thread Bruce Momjian
doc: add example of % substitution for connection URIs Reported-by: Zhou Digoal Discussion: https://postgr.es/m/20170912133722.25637...@wrigleys.postgresql.org Backpatch-through: 10 Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/ba01ef267ee48e86566886376c

[COMMITTERS] pgsql: doc: add example of % substitution for connection URIs

2017-09-19 Thread Bruce Momjian
doc: add example of % substitution for connection URIs Reported-by: Zhou Digoal Discussion: https://postgr.es/m/20170912133722.25637...@wrigleys.postgresql.org Backpatch-through: 10 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d61f5bb7c444255b064a60df782907f7d

Re: [COMMITTERS] pgsql: Add test for postmaster crash restarts.

2017-09-19 Thread Tom Lane
I discovered that prairiedog has been hung up for many hours in the 013_crash_restart.pl. It looks to me like the explanation is that the test has a race condition, because what I find in the postmaster log is 2017-09-19 00:31:48.194 EDT [27839] [unknown] LOG: connection received: host=[local]

[COMMITTERS] pgsql: Add citext_pattern_ops for citext contrib module

2017-09-19 Thread Andrew Dunstan
Add citext_pattern_ops for citext contrib module This is similar to text_pattern_ops. Alexey Chernyshov, reviewed by Jacob Champion. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f2464997644c64b5dec93ab3c08305f48bfe14f1 Modified Files -- contrib/cite