[COMMITTERS] pgsql: Test BRIN autosummarization
Test BRIN autosummarization There was no coverage for this code. Reported-by: Nikolay Shaplov, Tom Lane Discussion: https://postgr.es/m/2700647.XEouBYNZic@x200m https://postgr.es/m/13849.1506114...@sss.pgh.pa.us Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3571a53345bb4d3d055d8a720e9817038927877e Modified Files -- src/test/modules/brin/Makefile | 7 -- src/test/modules/brin/t/01_workitems.pl | 39 + 2 files changed, 44 insertions(+), 2 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Test BRIN autosummarization
Test BRIN autosummarization There was no coverage for this code. Reported-by: Nikolay Shaplov, Tom Lane Discussion: https://postgr.es/m/2700647.XEouBYNZic@x200m https://postgr.es/m/13849.1506114...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/404ba54e8fd3036eee0f9241f68b17092ce734ee Modified Files -- src/test/modules/brin/Makefile | 7 -- src/test/modules/brin/t/01_workitems.pl | 39 + 2 files changed, 44 insertions(+), 2 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL9_5_STABLE Details --- https://git.postgresql.org/pg/commitdiff/acae13faabc505146817f4834a8c9e9b43788312 Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL9_6_STABLE Details --- https://git.postgresql.org/pg/commitdiff/a1f30ecc5115b0186ad02ec76f75296906813d26 Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL9_3_STABLE Details --- https://git.postgresql.org/pg/commitdiff/e0f5710c5e8b9502ac8bcd821d3418053ed38f7a Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL9_2_STABLE Details --- https://git.postgresql.org/pg/commitdiff/2eb84e54a25098618724ee633fbad78d5e417489 Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL9_4_STABLE Details --- https://git.postgresql.org/pg/commitdiff/9bf04994697dd094032d08682a25cbba7aba523f Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix saving and restoring umask
Fix saving and restoring umask In two cases, we set a different umask for some piece of code and restore it afterwards. But if the contained code errors out, the umask is not restored. So add TRY/CATCH blocks to fix that. Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3d7f11a0fabb038ce5c630b87dfadd8b625347fe Modified Files -- src/backend/commands/copy.c| 11 ++- src/backend/libpq/be-fsstubs.c | 13 +++-- 2 files changed, 21 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Fix saving and restoring umask
On 9/22/17 17:48, Tom Lane wrote: > Peter Eisentraut writes: >> Fix saving and restoring umask >> In two cases, we set a different umask for some piece of code and >> restore it afterwards. But if the contained code errors out, the umask >> is not restored. So add TRY/CATCH blocks to fix that. > > Doesn't that need to be back-patched? done -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Refactor new file permission handling
Refactor new file permission handling The file handling functions from fd.c were called with a diverse mix of notations for the file permissions when they were opening new files. Almost all files created by the server should have the same permissions set. So change the API so that e.g. OpenTransientFile() automatically uses the standard permissions set, and OpenTransientFilePerm() is a new function that takes an explicit permissions set for the few cases where it is needed. This also saves an unnecessary argument for call sites that are just opening an existing file. While we're reviewing these APIs, get rid of the FileName typedef and use the standard const char * for the file name and mode_t for the file mode. This makes these functions match other file handling functions and removes an unnecessary layer of mysteriousness. We can also get rid of a few casts that way. Author: David Steele Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0c5803b450e0cc29b3527df3f352e6f18a038cc6 Modified Files -- contrib/pg_stat_statements/pg_stat_statements.c | 5 +- src/backend/access/heap/rewriteheap.c | 8 ++- src/backend/access/transam/slru.c | 7 ++- src/backend/access/transam/timeline.c | 8 ++- src/backend/access/transam/twophase.c | 5 +- src/backend/access/transam/xlog.c | 28 -- src/backend/access/transam/xlogutils.c | 2 +- src/backend/catalog/catalog.c | 2 +- src/backend/libpq/be-fsstubs.c | 6 +-- src/backend/replication/logical/origin.c| 7 ++- src/backend/replication/logical/reorderbuffer.c | 7 ++- src/backend/replication/logical/snapbuild.c | 5 +- src/backend/replication/slot.c | 6 +-- src/backend/replication/walsender.c | 4 +- src/backend/storage/file/copydir.c | 5 +- src/backend/storage/file/fd.c | 68 ++--- src/backend/storage/ipc/dsm_impl.c | 2 +- src/backend/storage/smgr/md.c | 12 ++--- src/backend/utils/cache/relmapper.c | 7 +-- src/backend/utils/misc/guc.c| 3 +- src/include/storage/fd.h| 15 +++--- 21 files changed, 110 insertions(+), 102 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Ten-second timeout in 013_crash_restart.pl is not enough, let's
Ten-second timeout in 013_crash_restart.pl is not enough, let's try 60. Per buildfarm member topminnow. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/01c7d3ef85d4b0e1c52cc1a3542864f95f386f76 Modified Files -- src/test/recovery/t/013_crash_restart.pl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
Remove pgbench "progress" test pending solution of its timing issues. Buildfarm member skink shows that this is even more flaky than I thought. There are probably some actual pgbench bugs here as well as a timing dependency. But we can't have stuff this unstable in the buildfarm, it obscures other issues. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ad51c6fb5708342e603d12a730bbc4e663bd637e Modified Files -- src/bin/pgbench/t/001_pgbench_with_server.pl | 22 -- 1 file changed, 22 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Improve memory management in autovacuum.c.
Improve memory management in autovacuum.c. Invoke vacuum(), as well as "work item" processing, in the PortalContext that do_autovacuum() has manufactured, which will be reset before each such invocation. This ensures cleanup of any memory leaked by these operations. It also avoids the rather dangerous practice of calling vacuum() in a context that vacuum() itself will destroy while it runs. There's no known live bug there, but it's not hard to imagine introducing one if we leave it like this. Tom Lane, reviewed by Michael Paquier and Alvaro Herrera Discussion: https://postgr.es/m/13849.1506114...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/335f3d04e4c8dd495c4dd30ab1049b6fe8f25052 Modified Files -- src/backend/postmaster/autovacuum.c | 22 +- 1 file changed, 17 insertions(+), 5 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
Hello Tom, Remove pgbench "progress" test pending solution of its timing issues. It would have been nicer to simply comment them out... Buildfarm member skink shows that this is even more flaky than I thought. There are probably some actual pgbench bugs here as well as a timing dependency. According to the trace, the host was so loaded that it could not do anything for over two seconds, so that the first output is "progress: 2.6", i.e. the 0 thread did not get any significant time for the first 2.6 seconds... of a 1 second test. Hmmm, not very kind. Somehow this cannot be helped: if the system does not give any execution time to some pgbench thread, the expected output cannot be there. The checked condition could be relaxed to "progress: \d", which still assumes that the system was kind enough to give enough cpu time to the process in the two second run so that it could output something. The issue is probably the same on the second test on the generated log. My guess is that some transactions (we are talking about "-S"...) took several seconds, thus were completed and aggregate in rows over the expected limit. Could be considered bug, that is the aggregation should discard over the limit results rather than counting them in some late entry. I'll investigate that. Globally these test failures reveals pathological testing conditions: the inability to execute a simple select over several seconds. It could have been OOM or whatever else. If the test must run even when postgres doesn't, it is a little bit hard as a starting assumption for a benchmarking tool:-( -- Fabien. -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
Hi, On 2017-09-23 19:57:40 +0200, Fabien COELHO wrote: > According to the trace, the host was so loaded that it could not do anything > for over two seconds, so that the first output is "progress: 2.6", i.e. the > 0 thread did not get any significant time for the first 2.6 seconds... of a > 1 second test. Hmmm, not very kind. > > Somehow this cannot be helped: if the system does not give any execution > time to some pgbench thread, the expected output cannot be there. skink runs postgres under valgrind, so it's going to be slower. > If the test must run even when postgres doesn't, it is a little bit hard as > a starting assumption for a benchmarking tool:-( I'm unsure what the point of this is. It's not like we discussing removing pgbench, we're talking about an unreliable test. Greetings, Andres Freund -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
Fabien COELHO writes: > Globally these test failures reveals pathological testing conditions: the > inability to execute a simple select over several seconds. It could have > been OOM or whatever else. Well, I think it's mostly about valgrind making everything really slow. Since we have seen some passes from skink recently, perhaps there was also a component of more-load-on-the-machine-than-usual. But in the end this is just evidence for my point that regression tests have to be very much not timing-sensitive. We run them under all kinds of out-of-the-ordinary stress. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix bogus size calculation in strlist_to_textarray().
Fix bogus size calculation in strlist_to_textarray(). It's making an array of Datum, not an array of text *. The mistake is harmless since those are currently the same size, but it's still wrong. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/737639017c87d5a0a466e8676f1eadc61d775c78 Modified Files -- src/backend/catalog/objectaddress.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: ... and the very same bug in publicationListToArray().
... and the very same bug in publicationListToArray(). Sigh. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/24541ffd788d56009126fff52b2341ada6c84245 Modified Files -- src/backend/commands/subscriptioncmds.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
If the test must run even when postgres doesn't, it is a little bit hard as a starting assumption for a benchmarking tool:-( I'm unsure what the point of this is. It's not like we discussing removing pgbench, we're talking about an unreliable test. My sentence was probably not very clear. I just meant that devising a coverage test which does not fail when the benchmarking tool is not able to run because of load/valgrind/... is kind of hard. Anything even loosely related to time, here having a few simple SELECT transactions over 2 seconds and a few printf into a file or stdout every second, can always fail if conditions are bad enough. So this means somehow giving up on coverage, because of one host which can fail under very unusual testing conditions once in a while. I would prefer to keep the test and have a warning instead, something like "ok, although a test which is allowed to rarely fail failed", but at least the feature are tested most of the time. Sigh. -- Fabien. -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
Fabien COELHO writes: > So this means somehow giving up on coverage, because of one host which can > fail under very unusual testing conditions once in a while. You are attacking a straw man. There is no reason whatever to believe that the problem is confined to one host. We have quite a number of slow buildfarm hosts: some are using CLOBBER_CACHE_ALWAYS, some are overloaded, some are just plain slow. Also, regression test failures outside the context of our buildfarm are just as bad if not worse. I still remember the pain I had with mysql's regression tests, which routinely fell over in Red Hat's package buildfarm even though they always succeeded on a personal workstation with nothing else going on. I do not want our own packagers to have that kind of experience with Postgres. > I would prefer to keep the test and have a warning instead, something > like "ok, although a test which is allowed to rarely fail failed", but at > least the feature are tested most of the time. If the failures are ignored, how much of a test have you really got? Nobody inquires into nominally-passed buildfarm runs to see what actually happened. In the TAP-test context, you do have quite a bit of freedom to decide what is a pass and what is a fail. So maybe the path forward is to be more flexible about the pass conditions for this test. But I can't help thinking that these results have exposed some outright bugs too. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix pg_basebackup test to original intent
Fix pg_basebackup test to original intent One test case was meant to check that pg_basebackup does not succeed when a slot is specified with -S but WAL streaming is not selected, which used to require specifying -X stream. Since -X stream is the default in PostgreSQL 10, this test case no longer covers that meaning, but the pg_basebackup invocation happened to fail anyway for the unrelated reason that the specified replication slot does not exist. To fix, move the test case to later in the file where the slot does exist, and add -X none to the invocation so that it covers the originally meant behavior. extracted from a patch by Michael Banck Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/74ca8f9b9077017529fe658e445a11da296ac6ab Modified Files -- src/bin/pg_basebackup/t/010_pg_basebackup.pl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: Fix pg_basebackup test to original intent
Fix pg_basebackup test to original intent One test case was meant to check that pg_basebackup does not succeed when a slot is specified with -S but WAL streaming is not selected, which used to require specifying -X stream. Since -X stream is the default in PostgreSQL 10, this test case no longer covers that meaning, but the pg_basebackup invocation happened to fail anyway for the unrelated reason that the specified replication slot does not exist. To fix, move the test case to later in the file where the slot does exist, and add -X none to the invocation so that it covers the originally meant behavior. extracted from a patch by Michael Banck Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/7c8ce791d4b3e774ffe4c7c987a7055619663818 Modified Files -- src/bin/pg_basebackup/t/010_pg_basebackup.pl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: doc: Expand user documentation on SCRAM
doc: Expand user documentation on SCRAM Explain more about how the different password authentication methods and the password_encryption settings relate to each other, give some upgrading advice, and set a better link from the release notes. Reviewed-by: Jeff Janes Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/33e2f346b0689cf7631fcfe7db91dc56767f2659 Modified Files -- doc/src/sgml/client-auth.sgml | 127 -- doc/src/sgml/config.sgml | 2 +- doc/src/sgml/release-10.sgml | 2 +- 3 files changed, 100 insertions(+), 31 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
[COMMITTERS] pgsql: doc: Expand user documentation on SCRAM
doc: Expand user documentation on SCRAM Explain more about how the different password authentication methods and the password_encryption settings relate to each other, give some upgrading advice, and set a better link from the release notes. Reviewed-by: Jeff Janes Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9b31c72a9492880e657b68b1ed971dec3c361c95 Modified Files -- doc/src/sgml/client-auth.sgml | 127 -- doc/src/sgml/config.sgml | 2 +- doc/src/sgml/release-10.sgml | 2 +- 3 files changed, 100 insertions(+), 31 deletions(-) -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is
You are attacking a straw man. I thought it was tilting at windmills:-) In the TAP-test context, you do have quite a bit of freedom to decide what is a pass and what is a fail. So maybe the path forward is to be more flexible about the pass conditions for this test. Hmmm. I'm not sure that there is a solution under "any" testing conditions if -T & -P are to be tested and some sensible output expected out of them. I'll think about it. But I can't help thinking that these results have exposed some outright bugs too. Yep, I agree that there is probably one bug/race/unexpected condition wrt what is output and time out handling. -- Fabien. -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
