pgsql: Update configure for ab969a376
Update configure for ab969a376 Commit ab969a376 updated configure.ac but neglected to update the actual configure script. Fix that. Pointed out by Tom Lane Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ca07a6e7bd2cb1d6ea38a036be8d60e196935428 Modified Files -- configure | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-)
pgsql: Update Kerberos/GSSAPI configure/meson check
Update Kerberos/GSSAPI configure/meson check Instead of checking for the much older gss_init_sec_context, check for gss_store_cred_into which was added in MIT Kerberos 1.11 (circa 2012). Discussion: https://postgr.es/m/2313469.1681695223%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/1c52f9c042fb939ed55ff572daee71036720db57 Modified Files -- configure.ac | 4 ++-- meson.build | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
pgsql: De-Revert "Add support for Kerberos credential delegation"
De-Revert "Add support for Kerberos credential delegation" This reverts commit 3d03b24c3 (Revert Add support for Kerberos credential delegation) which was committed on the grounds of concern about portability, but on further review and discussion, it's clear that we are better off explicitly requiring MIT Kerberos as that appears to be the only GSSAPI library currently that's under proper maintenance and ongoing development. The API used for storing credentials was added to MIT Kerberos over a decade ago while for the other libraries which appear to be mainly based on Heimdal, which exists explicitly to be a re-implementation of MIT Kerberos, the API never made it to a released version (even though it was added to the Heimdal git repo over 5 years ago..). This post-feature-freeze change was approved by the RMT. Discussion: https://postgr.es/m/ZDDO6jaESKaBgej0%40tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6633cfb21691840c33816a6dacaca0b504efb895 Modified Files -- contrib/dblink/dblink.c| 127 ++ contrib/dblink/expected/dblink.out | 4 +- contrib/postgres_fdw/connection.c | 72 -- contrib/postgres_fdw/expected/postgres_fdw.out | 19 +- contrib/postgres_fdw/option.c | 6 + contrib/postgres_fdw/sql/postgres_fdw.sql | 3 +- doc/src/sgml/config.sgml | 17 ++ doc/src/sgml/dblink.sgml | 5 +- doc/src/sgml/libpq.sgml| 41 +++ doc/src/sgml/monitoring.sgml | 9 + doc/src/sgml/postgres-fdw.sgml | 7 +- src/backend/catalog/system_views.sql | 3 +- src/backend/foreign/foreign.c | 1 + src/backend/libpq/auth.c | 13 +- src/backend/libpq/be-gssapi-common.c | 53 src/backend/libpq/be-secure-gssapi.c | 26 +- src/backend/utils/activity/backend_status.c| 1 + src/backend/utils/adt/pgstatfuncs.c| 21 +- src/backend/utils/init/postinit.c | 8 +- src/backend/utils/misc/guc_tables.c| 10 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/catalog/pg_proc.dat| 6 +- src/include/libpq/auth.h | 1 + src/include/libpq/be-gssapi-common.h | 3 + src/include/libpq/libpq-be.h | 2 + src/include/utils/backend_status.h | 1 + src/interfaces/libpq/exports.txt | 1 + src/interfaces/libpq/fe-auth.c | 15 +- src/interfaces/libpq/fe-connect.c | 17 ++ src/interfaces/libpq/fe-secure-gssapi.c| 23 +- src/interfaces/libpq/libpq-fe.h| 1 + src/interfaces/libpq/libpq-int.h | 2 + src/test/kerberos/Makefile | 3 + src/test/kerberos/t/001_auth.pl| 331 ++--- src/test/perl/PostgreSQL/Test/Utils.pm | 27 ++ src/test/regress/expected/rules.out| 11 +- 36 files changed, 755 insertions(+), 136 deletions(-)
pgsql: Explicitly require MIT Kerberos for GSSAPI
Explicitly require MIT Kerberos for GSSAPI WHen building with GSSAPI support, explicitly require MIT Kerberos and check for gssapi_ext.h in configure.ac and meson.build. Also add documentation explicitly stating that we now require MIT Kerberos when building with GSSAPI support. Reveiwed by: Johnathan Katz Discussion: https://postgr.es/m/abcc73d0-acf7-6896-e0dc-f5bc12a61...@postgresql.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f7431bca8b0138bdbce7025871560d39119565a0 Modified Files -- configure| 27 +++ configure.ac | 2 ++ doc/src/sgml/client-auth.sgml| 2 +- doc/src/sgml/installation.sgml | 21 +++-- meson.build | 10 ++ src/backend/libpq/auth.c | 5 +++-- src/backend/libpq/be-secure-gssapi.c | 5 +++-- 7 files changed, 57 insertions(+), 15 deletions(-)
pgsql: Revert "Add support for Kerberos credential delegation"
Revert "Add support for Kerberos credential delegation" This reverts commit 3d4fa227bce4294ce1cc214b4a9d3b7caa3f0454. Per discussion and buildfarm, this depends on APIs that seem to not be available on at least one platform (NetBSD). Should be certainly possible to rework to be optional on that platform if necessary but bit late for that at this point. Discussion: https://postgr.es/m/3286097.1680922...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3d03b24c350ab060bb223623bdff38835bd7afd0 Modified Files -- contrib/dblink/dblink.c| 127 -- contrib/dblink/expected/dblink.out | 4 +- contrib/postgres_fdw/connection.c | 72 ++ contrib/postgres_fdw/expected/postgres_fdw.out | 19 +- contrib/postgres_fdw/option.c | 6 - contrib/postgres_fdw/sql/postgres_fdw.sql | 3 +- doc/src/sgml/config.sgml | 17 -- doc/src/sgml/dblink.sgml | 5 +- doc/src/sgml/libpq.sgml| 41 --- doc/src/sgml/monitoring.sgml | 9 - doc/src/sgml/postgres-fdw.sgml | 7 +- src/backend/catalog/system_views.sql | 3 +- src/backend/foreign/foreign.c | 1 - src/backend/libpq/auth.c | 13 +- src/backend/libpq/be-gssapi-common.c | 53 src/backend/libpq/be-secure-gssapi.c | 26 +- src/backend/utils/activity/backend_status.c| 1 - src/backend/utils/adt/pgstatfuncs.c| 21 +- src/backend/utils/init/postinit.c | 8 +- src/backend/utils/misc/guc_tables.c| 10 - src/backend/utils/misc/postgresql.conf.sample | 1 - src/include/catalog/pg_proc.dat| 6 +- src/include/libpq/auth.h | 1 - src/include/libpq/be-gssapi-common.h | 3 - src/include/libpq/libpq-be.h | 2 - src/include/utils/backend_status.h | 1 - src/interfaces/libpq/exports.txt | 1 - src/interfaces/libpq/fe-auth.c | 15 +- src/interfaces/libpq/fe-connect.c | 17 -- src/interfaces/libpq/fe-secure-gssapi.c| 23 +- src/interfaces/libpq/libpq-fe.h| 1 - src/interfaces/libpq/libpq-int.h | 2 - src/test/kerberos/Makefile | 3 - src/test/kerberos/t/001_auth.pl| 331 +++-- src/test/perl/PostgreSQL/Test/Utils.pm | 27 -- src/test/regress/expected/rules.out| 11 +- 36 files changed, 136 insertions(+), 755 deletions(-)
pgsql: Add support for Kerberos credential delegation
Add support for Kerberos credential delegation Support GSSAPI/Kerberos credentials being delegated to the server by a client. With this, a user authenticating to PostgreSQL using Kerberos (GSSAPI) credentials can choose to delegate their credentials to the PostgreSQL server (which can choose to accept them, or not), allowing the server to then use those delegated credentials to connect to another service, such as with postgres_fdw or dblink or theoretically any other service which is able to be authenticated using Kerberos. Both postgres_fdw and dblink are changed to allow non-superuser password-less connections but only when GSSAPI credentials have been delegated to the server by the client and GSSAPI is used to authenticate to the remote system. Authors: Stephen Frost, Peifeng Qiu Reviewed-By: David Christensen Discussion: https://postgr.es/m/co1pr05mb8023cc2cb575e0faad7df4f8a8...@co1pr05mb8023.namprd05.prod.outlook.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3d4fa227bce4294ce1cc214b4a9d3b7caa3f0454 Modified Files -- contrib/dblink/dblink.c| 127 ++ contrib/dblink/expected/dblink.out | 4 +- contrib/postgres_fdw/connection.c | 72 -- contrib/postgres_fdw/expected/postgres_fdw.out | 19 +- contrib/postgres_fdw/option.c | 6 + contrib/postgres_fdw/sql/postgres_fdw.sql | 3 +- doc/src/sgml/config.sgml | 17 ++ doc/src/sgml/dblink.sgml | 5 +- doc/src/sgml/libpq.sgml| 41 +++ doc/src/sgml/monitoring.sgml | 9 + doc/src/sgml/postgres-fdw.sgml | 7 +- src/backend/catalog/system_views.sql | 3 +- src/backend/foreign/foreign.c | 1 + src/backend/libpq/auth.c | 13 +- src/backend/libpq/be-gssapi-common.c | 53 src/backend/libpq/be-secure-gssapi.c | 26 +- src/backend/utils/activity/backend_status.c| 1 + src/backend/utils/adt/pgstatfuncs.c| 21 +- src/backend/utils/init/postinit.c | 8 +- src/backend/utils/misc/guc_tables.c| 10 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/catalog/pg_proc.dat| 6 +- src/include/libpq/auth.h | 1 + src/include/libpq/be-gssapi-common.h | 3 + src/include/libpq/libpq-be.h | 2 + src/include/utils/backend_status.h | 1 + src/interfaces/libpq/exports.txt | 1 + src/interfaces/libpq/fe-auth.c | 15 +- src/interfaces/libpq/fe-connect.c | 17 ++ src/interfaces/libpq/fe-secure-gssapi.c| 23 +- src/interfaces/libpq/libpq-fe.h| 1 + src/interfaces/libpq/libpq-int.h | 2 + src/test/kerberos/Makefile | 3 + src/test/kerberos/t/001_auth.pl| 331 ++--- src/test/perl/PostgreSQL/Test/Utils.pm | 27 ++ src/test/regress/expected/rules.out| 11 +- 36 files changed, 755 insertions(+), 136 deletions(-)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/bbf7336d5ada714a9270e3e1230554182f7d2b30 Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/f0bd3361e09d206c47c0dc439c6642f96f67fab9 Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/cd018a2b3f2479377ec62e9d8f738c603df24192 Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/ced712f1a1d60d5e24d207d97efbc4dddc051079 Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/52d83e927015ddfc7fdbc5bb65d555cdd0355522 Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/154bcce3ba9d9817bdb5bd628b611d56fc434c46 Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/0a46b2f89e188c9139fc095c0da0c145bec12f3d Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable DNS lookups
For Kerberos testing, disable DNS lookups Similar to 8dff2f224, this disables DNS lookups by the Kerberos library to look up the KDC and the realm while the Kerberos tests are running. In some environments, these lookups can take a long time and end up timing out and causing tests to fail. Further, since this isn't really our domain, we shouldn't be sending out these DNS requests during our tests. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ce5e23408568452fd206b79ee411c6f34d80c46b Modified Files -- src/test/kerberos/t/001_auth.pl | 7 +++ 1 file changed, 7 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/1beb70d810734812edbdf9ab6072425afa6eac40 Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/1fdd1fbe10fe6f7f404238e586cac212ef1451c8 Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/0787432f33fbbda4f628f4499468a3f7d0bc11ad Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: For Kerberos testing, disable reverse DNS lookup
For Kerberos testing, disable reverse DNS lookup In our Kerberos test suite, there isn't much need to worry about the normal canonicalization that Kerberos provides by looking up the reverse DNS for the IP address connected to, and in some cases it can actively cause problems (eg: a captive portal wifi where the normally not resolvable localhost address used ends up being resolved anyway, and not to the domain we are using for testing, causing the entire regression test to fail with errors about not being able to get a TGT for the remote realm for cross-realm trust). Therefore, disable it by adding rdns = false into the krb5.conf that's generated for the test. Reviewed-By: Heikki Linnakangas Discussion: https://postgr.es/m/Y/qd2zdkdyqa1...@tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8dff2f224f4f49ddd4f45f590e7f46b160a4951b Modified Files -- src/test/kerberos/t/001_auth.pl | 12 1 file changed, 12 insertions(+)
pgsql: Forgotten catversion bump for 39969e2a1e4d7f5a37f3ef37d53bbfe171
Forgotten catversion bump for 39969e2a1e4d7f5a37f3ef37d53bbfe171e7d77a Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e99546f56670491370d7dc63b0693c3aadaa3112 Modified Files -- src/include/catalog/catversion.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Re: pgsql: Remove exclusive backup mode
Greetings, * Stephen Frost (sfr...@snowman.net) wrote: > Remove exclusive backup mode ... and that should have included a catversion bump, sorry about that, though it had just been bumped a couple commits before too. Thanks, Stephen signature.asc Description: PGP signature
pgsql: Remove exclusive backup mode
Remove exclusive backup mode Exclusive-mode backups have been deprecated since 9.6 (when non-exclusive backups were introduced) due to the issues they can cause should the system crash while one is running and generally because non-exclusive provides a much better interface. Further, exclusive backup mode wasn't really being tested (nor was most of the related code- like being able to log in just to stop an exclusive backup and the bits of the state machine related to that) and having to possibly deal with an exclusive backup and the backup_label file existing during pg_basebackup, pg_rewind, etc, added other complexities that we are better off without. This patch removes the exclusive backup mode, the various special cases for dealing with it, and greatly simplifies the online backup code and documentation. Authors: David Steele, Nathan Bossart Reviewed-by: Chapman Flack Discussion: https://postgr.es/m/ac7339ca-3718-3c93-929f-99e725d11...@pgmasters.net https://postgr.es/m/CAHg+QDfiM+WU61tF6=npzocmzvhdzck47kneyb0zrulyzv5...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/39969e2a1e4d7f5a37f3ef37d53bbfe171e7d77a Modified Files -- doc/src/sgml/backup.sgml | 241 ++ doc/src/sgml/func.sgml | 111 + doc/src/sgml/high-availability.sgml| 6 +- doc/src/sgml/monitoring.sgml | 4 +- doc/src/sgml/ref/pg_ctl-ref.sgml | 6 +- doc/src/sgml/ref/pgupgrade.sgml| 2 +- doc/src/sgml/runtime.sgml | 8 +- src/backend/access/transam/xlog.c | 519 +++-- src/backend/access/transam/xlogfuncs.c | 253 ++ src/backend/access/transam/xlogrecovery.c | 20 +- src/backend/catalog/system_functions.sql | 18 +- src/backend/postmaster/postmaster.c| 75 +-- src/backend/replication/basebackup.c | 20 +- src/backend/utils/init/postinit.c | 18 - src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 + src/bin/pg_ctl/pg_ctl.c| 30 -- src/bin/pg_rewind/filemap.c| 6 +- src/include/access/xlog.h | 7 +- src/include/catalog/pg_control.h | 4 +- src/include/catalog/pg_proc.dat| 28 +- src/include/libpq/libpq-be.h | 3 +- src/include/miscadmin.h| 4 - src/test/perl/PostgreSQL/Test/Cluster.pm | 56 +-- .../recovery/t/010_logical_decoding_timelines.pl | 4 +- 24 files changed, 247 insertions(+), 1200 deletions(-)
pgsql: Use maintenance_io_concurrency for ANALYZE prefetch
Use maintenance_io_concurrency for ANALYZE prefetch When prefetching pages for ANALYZE, we should be using maintenance_io_concurrenty (by calling get_tablespace_maintenance_io_concurrency(), not get_tablespace_io_concurrency()). ANALYZE prefetching was introduced in c6fc50c, so back-patch to 14. Backpatch-through: 14 Reported-By: Egor Rogov Discussion: https://postgr.es/m/9beada99-34ce-8c95-fadb-451768d08c64%40postgrespro.ru Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ce42efaa2696fa74dffcbaa7d25c4ef00e93e1c0 Modified Files -- src/backend/commands/analyze.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Use maintenance_io_concurrency for ANALYZE prefetch
Use maintenance_io_concurrency for ANALYZE prefetch When prefetching pages for ANALYZE, we should be using maintenance_io_concurrenty (by calling get_tablespace_maintenance_io_concurrency(), not get_tablespace_io_concurrency()). ANALYZE prefetching was introduced in c6fc50c, so back-patch to 14. Backpatch-through: 14 Reported-By: Egor Rogov Discussion: https://postgr.es/m/9beada99-34ce-8c95-fadb-451768d08c64%40postgrespro.ru Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/9efa998a6403c5fe973ce5801d09ffa63e706eb6 Modified Files -- src/backend/commands/analyze.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: docs: Add command tags for SQL commands
docs: Add command tags for SQL commands Commit 6c3ffd6 added a couple new predefined roles but didn't properly wrap the SQL commands mentioned in the description of those roles with command tags, so add them now. Backpatch-through: 14 Reported-by: Michael Banck Discussion: https://postgr.es/m/606d8b1c.1c69fb81.3df04.1...@mx.google.com Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/8f6c110349769e2b6375cd01e632199a104dc4a1 Modified Files -- doc/src/sgml/user-manag.sgml | 25 + 1 file changed, 13 insertions(+), 12 deletions(-)
pgsql: docs: Add command tags for SQL commands
docs: Add command tags for SQL commands Commit 6c3ffd6 added a couple new predefined roles but didn't properly wrap the SQL commands mentioned in the description of those roles with command tags, so add them now. Backpatch-through: 14 Reported-by: Michael Banck Discussion: https://postgr.es/m/606d8b1c.1c69fb81.3df04.1...@mx.google.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f01727290fe0c7fdf7bb5a0c2526a15db8c2c52f Modified Files -- doc/src/sgml/user-manag.sgml | 25 + 1 file changed, 13 insertions(+), 12 deletions(-)
pgsql: Rename Default Roles to Predefined Roles
Rename Default Roles to Predefined Roles The term 'default roles' wasn't quite apt as these roles aren't able to be modified or removed after installation, so rename them to be 'Predefined Roles' instead, adding an entry into the newly added Obsolete Appendix to help users of current releases find the new documentation. Bruce Momjian and Stephen Frost Discussion: https://postgr.es/m/157742545062.1149.11052653770497832538%40wrigleys.postgresql.org and https://www.postgresql.org/message-id/20201120211304.gg16...@tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b Modified Files -- contrib/adminpack/adminpack.c | 7 +-- contrib/file_fdw/file_fdw.c | 4 ++-- contrib/pg_stat_statements/pg_stat_statements.c | 2 +- contrib/pgrowlocks/pgrowlocks.c | 2 +- doc/src/sgml/appendix-obsolete-default-roles.sgml | 22 ++ doc/src/sgml/appendix-obsolete.sgml | 1 + doc/src/sgml/file-fdw.sgml| 4 ++-- doc/src/sgml/filelist.sgml| 1 + doc/src/sgml/monitoring.sgml | 2 +- doc/src/sgml/ref/copy.sgml| 2 +- doc/src/sgml/user-manag.sgml | 18 +- src/backend/commands/copy.c | 6 +++--- src/backend/commands/user.c | 6 +++--- src/backend/replication/walreceiver.c | 2 +- src/backend/replication/walsender.c | 2 +- src/backend/storage/ipc/procarray.c | 2 +- src/backend/storage/ipc/signalfuncs.c | 2 +- src/backend/utils/adt/acl.c | 4 ++-- src/backend/utils/adt/dbsize.c| 4 ++-- src/backend/utils/adt/genfile.c | 7 +-- src/backend/utils/adt/pgstatfuncs.c | 2 +- src/backend/utils/misc/guc.c | 14 +++--- src/include/catalog/pg_authid.dat | 18 +- 23 files changed, 82 insertions(+), 52 deletions(-)
pgsql: Add a docs section for obsoleted and renamed functions and setti
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced for HTML docs. This prevents /current/ links on the web from breaking, and allows users of the web docs to follow links from old version pages to info on the changes in the new version. Prior to this change, a link to /current/ for renamed sections like the recovery.conf docs would just 404. Similarly if someone searched for recovery.conf they would find the pg11 docs, but there would be no /12/ or /current/ link, so they couldn't easily find out that it was removed in pg12 or how to adapt. Index entries are also added so that there's a breadcrumb trail for users to follow when they know the old name, but not what we changed it to. So a user who is trying to find out how to set standby_mode in PostgreSQL 12+, or where pg_resetxlog went, now has more chance of finding that information. Craig Ringer and Stephen Frost Reviewed-by: Euler Taveira Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com Backpatch-through: 10 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/448e86c3345bb41d3e530ab204b8f59301cdfb36 Modified Files -- doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml | 24 + doc/src/sgml/appendix-obsolete-pgresetxlog.sgml| 24 + doc/src/sgml/appendix-obsolete-pgxlogdump.sgml | 24 + .../sgml/appendix-obsolete-recovery-config.sgml| 58 ++ doc/src/sgml/appendix-obsolete.sgml| 41 +++ doc/src/sgml/config.sgml | 5 +- doc/src/sgml/filelist.sgml | 7 +++ doc/src/sgml/high-availability.sgml| 16 -- doc/src/sgml/postgres.sgml | 1 + doc/src/sgml/ref/pg_basebackup.sgml| 4 +- 10 files changed, 198 insertions(+), 6 deletions(-)
pgsql: Add a docs section for obsoleted and renamed functions and setti
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced for HTML docs. This prevents /current/ links on the web from breaking, and allows users of the web docs to follow links from old version pages to info on the changes in the new version. Prior to this change, a link to /current/ for renamed sections like the recovery.conf docs would just 404. Similarly if someone searched for recovery.conf they would find the pg11 docs, but there would be no /12/ or /current/ link, so they couldn't easily find out that it was removed in pg12 or how to adapt. Index entries are also added so that there's a breadcrumb trail for users to follow when they know the old name, but not what we changed it to. So a user who is trying to find out how to set standby_mode in PostgreSQL 12+, or where pg_resetxlog went, now has more chance of finding that information. Craig Ringer and Stephen Frost Reviewed-by: Euler Taveira Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com Backpatch-through: 10 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3b0c647bbfc52894d979976f1e6d60e40649bba7 Modified Files -- doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml | 24 + doc/src/sgml/appendix-obsolete-pgresetxlog.sgml| 24 + doc/src/sgml/appendix-obsolete-pgxlogdump.sgml | 24 + .../sgml/appendix-obsolete-recovery-config.sgml| 58 ++ doc/src/sgml/appendix-obsolete.sgml| 41 +++ doc/src/sgml/config.sgml | 4 +- doc/src/sgml/filelist.sgml | 7 +++ doc/src/sgml/high-availability.sgml| 16 -- doc/src/sgml/postgres.sgml | 1 + doc/src/sgml/ref/pg_basebackup.sgml| 5 +- 10 files changed, 199 insertions(+), 5 deletions(-)
pgsql: Add a docs section for obsoleted and renamed functions and setti
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced for HTML docs. This prevents /current/ links on the web from breaking, and allows users of the web docs to follow links from old version pages to info on the changes in the new version. Prior to this change, a link to /current/ for renamed sections like the recovery.conf docs would just 404. Similarly if someone searched for recovery.conf they would find the pg11 docs, but there would be no /12/ or /current/ link, so they couldn't easily find out that it was removed in pg12 or how to adapt. Index entries are also added so that there's a breadcrumb trail for users to follow when they know the old name, but not what we changed it to. So a user who is trying to find out how to set standby_mode in PostgreSQL 12+, or where pg_resetxlog went, now has more chance of finding that information. Craig Ringer and Stephen Frost Reviewed-by: Euler Taveira Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com Backpatch-through: 10 Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/625f5aae7e42dd15b1aa51e13abe3ebe523e511a Modified Files -- doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml | 24 ++ doc/src/sgml/appendix-obsolete-pgresetxlog.sgml | 24 ++ doc/src/sgml/appendix-obsolete-pgxlogdump.sgml| 24 ++ doc/src/sgml/appendix-obsolete.sgml | 40 +++ doc/src/sgml/filelist.sgml| 6 doc/src/sgml/postgres.sgml| 1 + 6 files changed, 119 insertions(+)
pgsql: Add a docs section for obsoleted and renamed functions and setti
Add a docs section for obsoleted and renamed functions and settings The new appendix groups information on renamed or removed settings, commands, etc into an out-of-the-way part of the docs. The original id elements are retained in each subsection to ensure that the same filenames are produced for HTML docs. This prevents /current/ links on the web from breaking, and allows users of the web docs to follow links from old version pages to info on the changes in the new version. Prior to this change, a link to /current/ for renamed sections like the recovery.conf docs would just 404. Similarly if someone searched for recovery.conf they would find the pg11 docs, but there would be no /12/ or /current/ link, so they couldn't easily find out that it was removed in pg12 or how to adapt. Index entries are also added so that there's a breadcrumb trail for users to follow when they know the old name, but not what we changed it to. So a user who is trying to find out how to set standby_mode in PostgreSQL 12+, or where pg_resetxlog went, now has more chance of finding that information. Craig Ringer and Stephen Frost Reviewed-by: Euler Taveira Discussion: https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com Backpatch-through: 10 Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/58960fca4ef1953220e626aa2bce2446da04bfc7 Modified Files -- doc/src/sgml/appendix-obsolete-pgreceivexlog.sgml | 24 ++ doc/src/sgml/appendix-obsolete-pgresetxlog.sgml | 24 ++ doc/src/sgml/appendix-obsolete-pgxlogdump.sgml| 24 ++ doc/src/sgml/appendix-obsolete.sgml | 40 +++ doc/src/sgml/filelist.sgml| 6 doc/src/sgml/postgres.sgml| 1 + 6 files changed, 119 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/31536bf93622664574c215bb5b757cbd1b6cc5da Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3c3221a138413bf04d86a815733c321849792bd8 Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/b8be9690735c85d0b91769f4002e494156c60276 Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b64654d6c450eb9fb04c6e3456915790510af482 Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/092d3db05d34e2cd122b066cc61c139e7b90d635 Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: doc: Define TLS as an acronym
doc: Define TLS as an acronym Commit c6763156589 added an acronym reference for "TLS" but the definition was never added. Author: Daniel Gustafsson Reviewed-by: Michael Paquier Backpatch-through: 9.6 Discussion: https://postgr.es/m/27109504-82db-41a8-8e63-c0498314f...@yesql.se Branch -- REL9_6_STABLE Details --- https://git.postgresql.org/pg/commitdiff/ef924828fd84b3ebaaae97f8dd8349112cb5d8fa Modified Files -- doc/src/sgml/acronyms.sgml | 10 ++ 1 file changed, 10 insertions(+)
pgsql: Change checkpoint_completion_target default to 0.9
Change checkpoint_completion_target default to 0.9 Common recommendations are that the checkpoint should be spread out as much as possible, provided we avoid having it take too long. This change updates the default to 0.9 (from 0.5) to match that recommendation. There was some debate about possibly removing the option entirely but it seems there may be some corner-cases where having it set much lower to try to force the checkpoint to be as fast as possible could result in fewer periods of time of reduced performance due to kernel flushing. General agreement is that the "spread more" is the preferred approach though and those who need to tune away from that value are much less common. Reviewed-By: Michael Paquier, Peter Eisentraut, Tom Lane, David Steele, Nathan Bossart Discussion: https://postgr.es/m/20201207175329.GM16415%40tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/bbcc4eb2e08fb6e4535c7f84b2c00f3ad508bb9b Modified Files -- doc/src/sgml/config.sgml | 12 --- doc/src/sgml/wal.sgml | 29 +-- src/backend/postmaster/checkpointer.c | 2 +- src/backend/utils/misc/guc.c | 2 +- src/backend/utils/misc/postgresql.conf.sample | 2 +- src/test/recovery/t/015_promotion_pages.pl| 1 - 6 files changed, 30 insertions(+), 18 deletions(-)
pgsql: Use pre-fetching for ANALYZE
Use pre-fetching for ANALYZE When we have posix_fadvise() available, we can improve the performance of an ANALYZE by quite a bit by using it to inform the kernel of the blocks that we're going to be asking for. Similar to bitmap index scans, the number of buffers pre-fetched is based off of the maintenance_io_concurrency setting (for the particular tablespace or, if not set, globally, via get_tablespace_maintenance_io_concurrency()). Reviewed-By: Heikki Linnakangas, Tomas Vondra Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c6fc50cb40285141fad401321ae21becbaea1c59 Modified Files -- src/backend/commands/analyze.c | 73 -- 1 file changed, 71 insertions(+), 2 deletions(-)
pgsql: Improve logging of auto-vacuum and auto-analyze
Improve logging of auto-vacuum and auto-analyze When logging auto-vacuum and auto-analyze activity, include the I/O timing if track_io_timing is enabled. Also, for auto-analyze, add the read rate and the dirty rate, similar to how that information has historically been logged for auto-vacuum. Stephen Frost and Jakub Wartak Reviewed-By: Heikki Linnakangas, Tomas Vondra Discussion: https://www.postgresql.org/message-id/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0%40VI1PR0701MB6960.eurprd07.prod.outlook.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/94d13d474dc61800e8a17cc1959c55815b050ecd Modified Files -- doc/src/sgml/config.sgml | 8 +-- src/backend/access/heap/vacuumlazy.c | 18 +++ src/backend/commands/analyze.c | 100 --- 3 files changed, 116 insertions(+), 10 deletions(-)
pgsql: Add GSS information to connection authorized log message
Add GSS information to connection authorized log message GSS information (if used) such as if the connection was authorized using GSS or if it was encrypted using GSS, and perhaps most importantly, what the GSS principal used for the authentication was, is extremely useful but wasn't being included in the connection authorized log message. Therefore, add to the connection authorized log message that information, in a similar manner to how we log SSL information when SSL is used for a connection. Author: Vignesh C Reviewed-by: Bharath Rupireddy Discussion: https://www.postgresql.org/message-id/CALDaNm2N1385_Ltoo%3DS7VGT-ESu_bRQa-sC1wg6ikrM2L2Z49w%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/dc11f31a1a891f8aa8890644e837556bcc5a75e7 Modified Files -- src/backend/utils/init/postinit.c | 82 ++ src/test/kerberos/t/001_auth.pl | 118 +++--- 2 files changed, 114 insertions(+), 86 deletions(-)
pgsql: Fix GSS client to non-GSS server connection
Fix GSS client to non-GSS server connection If the client is compiled with GSSAPI support and tries to start up GSS with the server, but the server is not compiled with GSSAPI support, we would mistakenly end up falling through to call ProcessStartupPacket with secure_done = true, but the client might then try to perform SSL, which the backend wouldn't understand and we'd end up failing the connection with: FATAL: unsupported frontend protocol 1234.5679: server supports 2.0 to 3.0 Fix by arranging to track ssl_done independently from gss_done, instead of trying to use the same boolean for both. Author: Andrew Gierth Discussion: https://postgr.es/m/87h82kzwqn@news-spur.riddles.org.uk Backpatch: 12-, where GSSAPI encryption was added. Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/79e594cf04754d55196d2ce54fc869ccad5fa9c3 Modified Files -- src/backend/postmaster/postmaster.c | 37 - 1 file changed, 24 insertions(+), 13 deletions(-)
pgsql: Fix GSS client to non-GSS server connection
Fix GSS client to non-GSS server connection If the client is compiled with GSSAPI support and tries to start up GSS with the server, but the server is not compiled with GSSAPI support, we would mistakenly end up falling through to call ProcessStartupPacket with secure_done = true, but the client might then try to perform SSL, which the backend wouldn't understand and we'd end up failing the connection with: FATAL: unsupported frontend protocol 1234.5679: server supports 2.0 to 3.0 Fix by arranging to track ssl_done independently from gss_done, instead of trying to use the same boolean for both. Author: Andrew Gierth Discussion: https://postgr.es/m/87h82kzwqn@news-spur.riddles.org.uk Backpatch: 12-, where GSSAPI encryption was added. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b68a560f8ebfc7eed679d09facdce5512a38c9c2 Modified Files -- src/backend/postmaster/postmaster.c | 37 - 1 file changed, 24 insertions(+), 13 deletions(-)
Re: pgsql: Avoid -Wconversion warnings when using checksum_impl.h
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Thu, Mar 05, 2020 at 07:32:59PM -0500, David Steele wrote: > > FWIW, we use static values in our unit tests (which are written in C), then > > test against packaged versions of Postgres for integration tests. > > > > When I saw the commit I pulled it in so I could remove instructions for the > > manual step to add the cast. So in this case the issue was apparent really > > quickly. Normally we only pull in new code from PostgreSQL once a year. > > > > We think our unit tests against static values may have endianess issues but > > we have not verified that one way or the other. Here's what they look like: > > > > https://github.com/pgbackrest/pgbackrest/blob/e55443c890181ea63a350275447885331c8254e4/test/src/module/postgres/interfaceTest.c#L182 > > By doing so, the tests still fail if the page size is something else > than 8k, no? Of course, but we check that from pg_controldata before we actually start doing anything with the cluster. Even a test which at least validated that the checksum code was returning the right value when page size is the (rather common...) 8K would be better than a change like this not breaking any tests.. Thanks, Stephen signature.asc Description: PGP signature
Re: pgsql: Prevent running pg_basebackup as root
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Thu, Feb 06, 2020 at 09:44:07AM -0500, Stephen Frost wrote: > > Erm- no, with -Ft + untar-as-root they get owned by "postgres", NOT the > > original user. That's what I was pointing out up-thread (since it seems > > to be confusing- and clearly not always well understood..) and it's an > > issue imv, but it's independent of this, so probably deserves its own > > thread if someone wants to do something about that. > > Hmm. I don't think that you are completely correct here either as it > depends on if the OS user "postgres" exists or not. Yes, I do know what happens if the named user doesn't exist, but in the general case, where the 'postgres' user does exist, they'll get owned by 'postgres'. > As mentioned in > https://www.gnu.org/software/tar/manual/tar.html#SEC138, if the user > name cannot be found in /etc/passwd, then tar switches to the user ID > (if one does not have any user or group named "postgres", then the > files are untar'ed with the same user and group as the one running the > cluster and that's to the UID and GID set by tarCreateHeader, as you > say). I think that it is a problem to not have more documentation on > the matter (now there is just a small mention in the base backup > restore about being sure to have the proper permissions). And it may > be interesting to add into pg_basebackup options to enforce the user > and/or group similarly to what tar does with --owner and --group? Yes, I agree with improving the documentation and with adding such options. > >> I agree with Stephen that this seems to be misguided, and my vote is > >> to revert. I would've also objected had you given more than 2 days > >> warning before committing, and it happened to be during FOSDEM. I saw > >> the original email which clearly said it'd be in the March commitfest, > >> so I figured I'd have time... > > > > Yeah, I also agree with reverting this change. Even if we can come to > > something we all agree on, I'm pretty confident it's not going to be > > exactly this patch, so let's back it out for now and discuss it further > > on the -hackers thread. > > OK, done that part as of dcddc3f. Great, thanks! Stephen signature.asc Description: PGP signature
Re: pgsql: Prevent running pg_basebackup as root
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Feb 6, 2020 at 8:04 AM Michael Paquier wrote: > > > > On Wed, Feb 05, 2020 at 12:22:59PM -0500, Stephen Frost wrote: > > > In any case, sorry for not responding on this sooner (was traveling for > > > FOSDEM and such), but I'm not really convinced this is something we want > > > and it certainly breaks at least somewhat reasonable use-cases when you > > > think about using pg_basebackup with -Ft. In that vein, this change is > > > kinda like saying "you can't run pg_dump as root".. > > > > It seems to me that this is entirely different than the case of > > pg_dump, as it is possible to restore a dump even as root, something > > that cannot happen with physical backups without an extra chmod -R. > > I don't see how that's relevant? And yes, you can restore physical > backups this way too, if the userids match. (though see Stephens > comment about the username, but that's independent of this issue) Right. > And pg_basebackup is about taking backups, not restores :) Yes- one of the downsides of pg_basebackup is that it doesn't really do much for you when it comes to restores, in fact.. Something that will have to change if it starts doing incrementals of some kind. That's mostly orthogonal to this discussion though. > > You have a point with -Ft as untaring the tarballs from a base backup > > taken with pg_basebackup -Ft used by root generates files owned by the > > original user. -Fp enforces the files to be owned by the user taking > > the backup, which makes the most sense, so for consistency with the > > other tools preventing root to run pg_basebackup makes sense to me > > with -Fp. Any thoughts from others to restrict the tool with -Fp but > > not with -Ft? The argument of consistency mattered for me first for > > both formats. Erm- no, with -Ft + untar-as-root they get owned by "postgres", NOT the original user. That's what I was pointing out up-thread (since it seems to be confusing- and clearly not always well understood..) and it's an issue imv, but it's independent of this, so probably deserves its own thread if someone wants to do something about that. Having -Fp run-as-root result in the files being owned by root isn't good and I agree that's unfortunate and it would be good to fix it, but preventing pg_basebackup from ever being run as root isn't a good solution to that issue. > I think having -Fp and -Ft consistent is a lot more important than > being consistent with other tools that aren't really that closely > related. And it's already inconsistent against probably the most > related command, being pg_dump. Yeah, I agree on consistency here being important too, and that pg_dump is a closer command to be thinking about than initdb and friends. > So *very* strong objection to makeing -Fp and -Ft behave differently > in this regard. What we aren't consistent about today is what happens when you do: - Backup as root with -Ft - Untar results as root - Backup as root with -Fp and that really seems less than ideal, but I don't think the answer is "don't allow backing up as root". > I agree with Stephen that this seems to be misguided, and my vote is > to revert. I would've also objected had you given more than 2 days > warning before committing, and it happened to be during FOSDEM. I saw > the original email which clearly said it'd be in the March commitfest, > so I figured I'd have time... Yeah, I also agree with reverting this change. Even if we can come to something we all agree on, I'm pretty confident it's not going to be exactly this patch, so let's back it out for now and discuss it further on the -hackers thread. Thanks, Stephen signature.asc Description: PGP signature
Re: pgsql: Prevent running pg_basebackup as root
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > Prevent running pg_basebackup as root > > Similarly to pg_upgrade, pg_ctl and initdb, a root user is able to use > --version and --help, but cannot execute the actual operation to avoid > the creation of files with permissions incompatible with the > postmaster. > > This is a behavior change, so not back-patching is done. While it's maybe not ideal, surely there isn't an actual issue if pg_basebackup is run as root with -Ft, is there..? There's possibly something to be said about the fact that we hard-code the username/groupname in the tar file too (interestingly, we actually do pass through the uid/gid..)- perhaps we should actually be passing the username/groupname through, but if we did do something like that then having pg_basebackup running as root would be necessary if we want to preserve the file ownership. In any case, sorry for not responding on this sooner (was traveling for FOSDEM and such), but I'm not really convinced this is something we want and it certainly breaks at least somewhat reasonable use-cases when you think about using pg_basebackup with -Ft. In that vein, this change is kinda like saying "you can't run pg_dump as root".. Thanks, Stephen signature.asc Description: PGP signature
pgsql: Improve GSSAPI Encryption startup comment in libpq
Improve GSSAPI Encryption startup comment in libpq The original comment was a bit confusing, pointed out by Alvaro Herrera. Thread: https://postgr.es/m/20191224151520.GA16435%40alvherre.pgsql Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8dd1511e39acd729020e151deb15a958300ebff5 Modified Files -- src/interfaces/libpq/fe-connect.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-)
Re: pgsql: Make crash recovery ignore recovery target settings.
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Sep 30, 2019 at 7:28 AM David Steele wrote: > > I'm surprised that we would consider going to GA with an issue like this > > outstanding. > > Yeah, this is a *bad* problem. I have to agree with this and I'm also pretty disappointed that we seem to be 'damn the torpedos, full steam ahead' with these issues. I don't like the idea of a delay either but ... Thanks, Stephen signature.asc Description: PGP signature
Re: pgsql: Detect unused steps in isolation specs and do some cleanup
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > Detect unused steps in isolation specs and do some cleanup > > This is useful for developers to find out if an isolation spec is > over-engineered or if it needs more work by warning at the end of a > test run if a step is not used, generating a failure with extra diffs. > > While on it, clean up all the specs which include steps not used in any > permutations to simplify them. Buildfarm didn't particularly like this. Thanks, Stephen signature.asc Description: PGP signature
Re: pgsql: Drop test user when done with it.
Greetings, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > On 2019-Jun-26, Stephen Frost wrote: > > > Isn’t this the one run from pg_upgrade’s tests? We don’t want to break that > > (and hopefully we haven’t but maybe something did...). Pretty sure we had > > nearly the same discussion this past fall... > > https://postgr.es/m/20180904203012.gg20...@paquier.xyz ? Yes, thanks, and specifically this: https://www.postgresql.org/message-id/20181130163728.GE3415%40tamriel.snowman.net Adding 'DROP IF EXISTS' to the top to have things cleaned up so that the run can be repeated is fine. Removing things at the end of the test script would defeat the entire purpose of those tests. Thanks! Stephen signature.asc Description: PGP signature
Re: pgsql: Drop test user when done with it.
Greetings, On Wed, Jun 26, 2019 at 10:51 Tom Lane wrote: > I wrote: > > Michael Paquier writes: > >> If we are on that, we still have src/test/modules/test_pg_dump/ which > >> is not repeatable with multiple installchecks: > >> > https://www.postgresql.org/message-id/20181130163728.ge3...@tamriel.snowman.net > > > OK, hadn't run into that personally, but let's fix that too. Anything > > that can be run with "installcheck" has to satisfy the restrictions > > of being re-runnable and careful about what global names it uses. > > Actually, now that I re-read that thread, maybe what we need to do is > mark test_pg_dump as something not to be run by "make installcheck"? > I'm not quite sure what *does* run it, but the context is evidently > that it's supposed to be run by some overarching script that's then > going to run pg_dump on the ending database state. So it's not meant > to be run against a generic pre-existing installation, and it doesn't > have to follow the rules for being safe for that --- but then we need > to be sure that it doesn't get called that way if someone does "make > installcheck" in a parent directory. On my phone atm, but this feels very deja vu... Isn’t this the one run from pg_upgrade’s tests? We don’t want to break that (and hopefully we haven’t but maybe something did...). Pretty sure we had nearly the same discussion this past fall... Thanks, Stephen
pgsql: GSSAPI: Improve documentation and tests
GSSAPI: Improve documentation and tests The GSSAPI encryption patch neglected to update the protocol documentation to describe how to set up a GSSAPI encrypted connection from a client to the server, so fix that by adding the appropriate documentation to protocol.sgml. The tests added for encryption support were overly long and couldn't be run in parallel due to race conditions; this was largely because each test was setting up its own KDC to perform the tests. Instead, merge the authentication tests and the encryption tests into the original test, where we only create one KDC to run the tests with. Also, have the tests check what the server's opinion is of the connection and if it was GSS authenticated or encrypted using the pg_stat_gssapi view. In passing, fix the libpq label for GSSENC-Mode to be consistent with the "PGGSSENCMODE" environment variable. Missing protocol documentation pointed out by Michael Paquier. Issues with the tests pointed out by Tom Lane and Peter Eisentraut. Refactored tests and added documentation by me. Reviewed by Robbie Harwood (protocol documentation) and Michael Paquier (rework of the tests). Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/eb882a1b717589777e068dc6671830226f3aa7f0 Modified Files -- doc/src/sgml/protocol.sgml| 106 src/interfaces/libpq/fe-connect.c | 2 +- src/test/kerberos/t/001_auth.pl | 122 --- src/test/kerberos/t/002_enc.pl| 197 -- 4 files changed, 216 insertions(+), 211 deletions(-)
pgsql: Handle errors during GSSAPI startup better
Handle errors during GSSAPI startup better There was some confusion over the format of the error message returned from the server during GSSAPI startup; specifically, it was expected that a length would be returned when, in reality, at this early stage in the startup sequence, no length is returned from the server as part of an error message. Correct the client-side code for dealing with error messages sent by the server during startup by simply reading what's available into our buffer, after we've discovered it's an error message, and then reporting back what was returned. In passing, also add in documentation of the environment variable PGGSSENCMODE which was missed previously, and adjust the code to look for the PGGSSENCMODE variable (the environment variable change was missed in the prior GSSMODE -> GSSENCMODE commit). Error-handling issue discovered by Peter Eisentraut, the rest were items discovered during testing of the error handling. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c46c85d4594d52fb34d36d4761bb9cfc5626f20b Modified Files -- doc/src/sgml/libpq.sgml | 10 ++ src/interfaces/libpq/fe-connect.c | 4 ++-- src/interfaces/libpq/fe-secure-gssapi.c | 34 - 3 files changed, 20 insertions(+), 28 deletions(-)
pgsql: GSSAPI encryption support
GSSAPI encryption support On both the frontend and backend, prepare for GSSAPI encryption support by moving common code for error handling into a separate file. Fix a TODO for handling multiple status messages in the process. Eliminate the OIDs, which have not been needed for some time. Add frontend and backend encryption support functions. Keep the context initiation for authentication-only separate on both the frontend and backend in order to avoid concerns about changing the requested flags to include encryption support. In postmaster, pull GSSAPI authorization checking into a shared function. Also share the initiator name between the encryption and non-encryption codepaths. For HBA, add "hostgssenc" and "hostnogssenc" entries that behave similarly to their SSL counterparts. "hostgssenc" requires either "gss", "trust", or "reject" for its authentication. Similarly, add a "gssencmode" parameter to libpq. Supported values are "disable", "require", and "prefer". Notably, negotiation will only be attempted if credentials can be acquired. Move credential acquisition into its own function to support this behavior. Add a simple pg_stat_gssapi view similar to pg_stat_ssl, for monitoring if GSSAPI authentication was used, what principal was used, and if encryption is being used on the connection. Finally, add documentation for everything new, and update existing documentation on connection security. Thanks to Michael Paquier for the Windows fixes. Author: Robbie Harwood, with changes to the read/write functions by me. Reviewed in various forms and at different times by: Michael Paquier, Andres Freund, David Steele. Discussion: https://www.postgresql.org/message-id/flat/jlg1tgq1ktm@thriss.redhat.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b0b39f72b9904bcb80f97b35837ccff1578aa4b8 Modified Files -- doc/src/sgml/client-auth.sgml | 78 +++- doc/src/sgml/libpq.sgml | 59 ++- doc/src/sgml/monitoring.sgml| 57 +++ doc/src/sgml/runtime.sgml | 77 +++- src/backend/catalog/system_views.sql| 8 + src/backend/libpq/Makefile | 4 + src/backend/libpq/auth.c| 115 +++--- src/backend/libpq/be-gssapi-common.c| 74 src/backend/libpq/be-gssapi-common.h| 26 ++ src/backend/libpq/be-secure-gssapi.c| 627 +++ src/backend/libpq/be-secure.c | 16 + src/backend/libpq/hba.c | 51 ++- src/backend/postmaster/pgstat.c | 46 +++ src/backend/postmaster/postmaster.c | 40 +- src/backend/utils/adt/pgstatfuncs.c | 20 +- src/bin/psql/command.c | 17 + src/include/catalog/pg_proc.dat | 6 +- src/include/libpq/hba.h | 4 +- src/include/libpq/libpq-be.h| 20 + src/include/libpq/libpq.h | 3 + src/include/libpq/pqcomm.h | 5 +- src/include/pgstat.h| 24 +- src/interfaces/libpq/Makefile | 4 + src/interfaces/libpq/exports.txt| 2 + src/interfaces/libpq/fe-auth.c | 84 + src/interfaces/libpq/fe-connect.c | 236 +++- src/interfaces/libpq/fe-gssapi-common.c | 130 +++ src/interfaces/libpq/fe-gssapi-common.h | 23 ++ src/interfaces/libpq/fe-secure-gssapi.c | 635 src/interfaces/libpq/fe-secure.c| 31 ++ src/interfaces/libpq/libpq-fe.h | 9 +- src/interfaces/libpq/libpq-int.h| 23 ++ src/test/kerberos/t/002_enc.pl | 197 ++ src/test/regress/expected/rules.out | 11 +- src/tools/msvc/Mkvcbuild.pm | 10 + 35 files changed, 2575 insertions(+), 197 deletions(-)
pgsql: Add support for partial TOAST decompression
Add support for partial TOAST decompression When asked for a slice of a TOAST entry, decompress enough to return the slice instead of decompressing the entire object. For use cases where the slice is at, or near, the beginning of the entry, this avoids a lot of unnecessary decompression work. This changes the signature of pglz_decompress() by adding a boolean to indicate if it's ok for the call to finish before consuming all of the source or destination buffers. Author: Paul Ramsey Reviewed-By: Rafia Sabih, Darafei Praliaskouski, Regina Obe Discussion: https://postgr.es/m/CACowWR07EDm7Y4m2kbhN_jnys%3DBBf9A6768RyQdKm_%3DNpkcaWg%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4d0e994eed83c845a05da6e9a417b4efec67efaf Modified Files -- src/backend/access/heap/tuptoaster.c| 38 +++-- src/backend/access/transam/xlogreader.c | 2 +- src/backend/utils/adt/varlena.c | 22 ++--- src/common/pg_lzcompress.c | 42 +++-- src/include/common/pg_lzcompress.h | 2 +- 5 files changed, 70 insertions(+), 36 deletions(-)
Re: pgsql: pg_regress: Don't use absolute paths for the diff
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Fri, Feb 22, 2019 at 03:26:35PM -0800, Andres Freund wrote: > > that's useless, because I can't trivially copy the result file into the > > expected file anymore. I have to figure out where in the tree it > > is. Which isn't exactly predictable, between the different tests we > > have as they locate their test results in different places. > > I haven't paid much attention to this commit, but the new behavior is > likely something which will become annoying in some not-so-far future > for me as well. > > > This is a bad idea. > > +1. Yeah, same here, if I can't easily copy the result file into the expected file, I'm going to be *most* annoyed... Thanks! Stephen signature.asc Description: PGP signature
Re: pgsql: Allow some recovery parameters to be changed with reload
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > > On 2019-Feb-08, Michael Paquier wrote: > >> The timestamp of this commit is a bit messed up: > >> commit: 13b89f96d07ad3da67b57f66c134c3609bd3e98f > >> author: Peter Eisentraut > >> date: Mon, 4 Feb 2019 09:28:17 +0100 > >> committer: Peter Eisentraut > >> date: Thu, 7 Feb 2019 08:34:48 +0100 > >> > >> Perhaps you overlooked a --reset-author switch? > > > I don't think we actually have a rule about these timestamps, and I > > don't think we really care, do we? > > Yeah, if you want to see a sequence of dates that makes sense, > you need to look at the commit-date. Whether the author-date > closely matches that depends on the particular committer's > workflow. While I agree that we don't really have a formal policy, there are certainly some who do (or, at least did) seem to care quite a bit about this and that's why I've been using '--ignore-date' for quite some time in my workflow: https://www.postgresql.org/message-id/CA%2BTgmobEgs1%3DAT0_SRvf6K9XrG7QAUyRNeuv5D9oaXrmpST9fw%40mail.gmail.com Thanks! Stephen signature.asc Description: PGP signature
pgsql: Add additional partition tests to pg_dump
Add additional partition tests to pg_dump This adds a few tests for non-inherited constraints. Author: Amit Langote Discussion: https://postgr.es/m/20181208001735.GT3415%40tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2d7eeb1b14925fd4ba6d2d7012636489570eaee8 Modified Files -- src/bin/pg_dump/t/002_pg_dump.pl | 10 +-- src/test/regress/expected/create_table.out | 45 -- src/test/regress/sql/create_table.sql | 23 --- 3 files changed, 57 insertions(+), 21 deletions(-)
pgsql: Remove dead code in toast_fetch_datum_slice
Remove dead code in toast_fetch_datum_slice In toast_fetch_datum_slice(), we Assert() that what is passed in isn't compressed, but we then later had a check to see what the length of if what was passed in is compressed. That later check is rather confusing since toast_fetch_datum_slice() is only ever called with non-compressed datums and the Assert() earlier makes it clear that one shouldn't be passing in compressed datums. Add a comment to make it clear that toast_fetch_datum_slice() is just for non-compressed datums, and remove the dead code. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/96c702c1edbde8a3f5013bd0ac6c25c85710258d Modified Files -- src/backend/access/heap/tuptoaster.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-)
pgsql: Improve planner stats documentation
Improve planner stats documentation It was pointed out that in the planner stats documentation under Extended Statistics, one of the sentences was a bit awkward. Improve that by rewording it slightly. Discussion: https://postgr.es/m/154409976780.14137.2785644488950047...@wrigleys.postgresql.org Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/aa175f61e0fcef16af57f82968a59ed5324146ca Modified Files -- doc/src/sgml/perform.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Improve planner stats documentation
Improve planner stats documentation It was pointed out that in the planner stats documentation under Extended Statistics, one of the sentences was a bit awkward. Improve that by rewording it slightly. Discussion: https://postgr.es/m/154409976780.14137.2785644488950047...@wrigleys.postgresql.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7a55ccc477b58863f6c73c243c7adb79c9717eda Modified Files -- doc/src/sgml/perform.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Improve planner stats documentation
Improve planner stats documentation It was pointed out that in the planner stats documentation under Extended Statistics, one of the sentences was a bit awkward. Improve that by rewording it slightly. Discussion: https://postgr.es/m/154409976780.14137.2785644488950047...@wrigleys.postgresql.org Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/23f9e3e36b1b58625f79a741c0fa6468c682da4e Modified Files -- doc/src/sgml/perform.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Cleanup minor pg_dump memory leaks
Cleanup minor pg_dump memory leaks In dumputils, we may have successfully parsed the acls when we discover that we can't parse the reverse ACLs and then return- check and free aclitems if that happens. In dumpTableSchema, move ftoptions and srvname under the relkind != RELKIND_VIEW branch (since they're only used there) and then check if they've been allocated and, if so, free them at the end of that block. Pointed out by Pavel Raiskup, though I didn't use those patches. Discussion: https://postgr.es/m/2183976.vkcjmhd...@nb.usersys.redhat.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/369d494a4f5c1494a46adcf5c56f5462a83ef20d Modified Files -- src/bin/pg_dump/dumputils.c | 2 ++ src/bin/pg_dump/pg_dump.c | 14 -- 2 files changed, 10 insertions(+), 6 deletions(-)
pgsql: Cleanup comments in xlog compression
Cleanup comments in xlog compression Skipping over the "hole" in full page images in the XLOG code was described as being a form of compression, but this got a bit confusing since we now have PGLZ-based compression happening, so adjust the wording to discuss "removing" the "hole" and keeping the talk about compression to where we're talking about using PGLZ-based compression of the full page images. Reviewed-By: Kyotaro Horiguchi Discussion: https://postgr.es/m/20181127234341.gm3...@tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a243c55326686b284fbe2c7b70f06d8a022b7dcc Modified Files -- src/backend/access/transam/xloginsert.c | 2 +- src/include/access/xlogrecord.h | 15 +++ 2 files changed, 8 insertions(+), 9 deletions(-)
pgsql: Fix typo
Fix typo Backends don't typically exist uncleanly, but they can certainly exit uncleanly, and it's exiting uncleanly that's being discussed here. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f502fc88b3c8b4c619a4bcbb86c3225a699c1d45 Modified Files -- src/backend/libpq/be-secure.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Remove extra word from create sub docs
Remove extra word from create sub docs Improve the documentation in the CREATE SUBSCRIPTION command a bit by removing an extraneous word and spelling out 'information'. Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/cf2245159172836db29d8f0bc7d7164fd33714b1 Modified Files -- doc/src/sgml/ref/create_subscription.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix for globals.c- c.h must come first
Fix for globals.c- c.h must come first Commit da9b580 mistakenly put a system header before postgres.h (which includes c.h). That can cause portability issues and broke (at least) builds with older Windows compilers. Discovered by Mark Dilger. Discussion: https://postgr.es/m/bf04a27a-d132-4927-a80a-bad18695e...@gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e2b83ff556deb9a0001bdf6b511f8cfc9189ac10 Modified Files -- src/backend/utils/init/globals.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: adminpack: Revoke EXECUTE on pg_logfile_rotate()
adminpack: Revoke EXECUTE on pg_logfile_rotate() In 9.6, we moved a number of functions over to using the GRANT system to control access instead of having hard-coded superuser checks. As it turns out, adminpack was creating another function in the catalog for one of those backend functions where the superuser check was removed, specifically pg_rotate_logfile(), but it didn't get the memo about having to REVOKE EXECUTE on the alternative-name function (pg_logfile_rotate()), meaning that in any installations with adminpack on 9.6 and higher, any user is able to run the pg_logfile_rotate() function, which then calls pg_rotate_logfile() and rotates the logfile. Fix by adding a new version of adminpack (1.1) which handles the REVOKE. As this function should have only been available to the superuser, this is a security issue, albeit a minor one. Security: CVE-2018-1115 Branch -- REL9_6_STABLE Details --- https://git.postgresql.org/pg/commitdiff/53b79ab4fe722b1030b7e9a1580b283fca956e30 Modified Files -- contrib/adminpack/Makefile| 2 +- contrib/adminpack/adminpack--1.0--1.1.sql | 6 contrib/adminpack/adminpack--1.1.sql | 55 +++ contrib/adminpack/adminpack.control | 2 +- 4 files changed, 63 insertions(+), 2 deletions(-)
pgsql: adminpack: Revoke EXECUTE on pg_logfile_rotate()
adminpack: Revoke EXECUTE on pg_logfile_rotate() In 9.6, we moved a number of functions over to using the GRANT system to control access instead of having hard-coded superuser checks. As it turns out, adminpack was creating another function in the catalog for one of those backend functions where the superuser check was removed, specifically pg_rotate_logfile(), but it didn't get the memo about having to REVOKE EXECUTE on the alternative-name function (pg_logfile_rotate()), meaning that in any installations with adminpack on 9.6 and higher, any user is able to run the pg_logfile_rotate() function, which then calls pg_rotate_logfile() and rotates the logfile. Fix by adding a new version of adminpack (1.1) which handles the REVOKE. As this function should have only been available to the superuser, this is a security issue, albeit a minor one. Security: CVE-2018-1115 Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/20f01fc45996238f7f1007ba704d30663955150a Modified Files -- contrib/adminpack/Makefile| 2 +- contrib/adminpack/adminpack--1.0--1.1.sql | 6 ++ contrib/adminpack/adminpack.control | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-)
pgsql: adminpack: Revoke EXECUTE on pg_logfile_rotate()
adminpack: Revoke EXECUTE on pg_logfile_rotate() In 9.6, we moved a number of functions over to using the GRANT system to control access instead of having hard-coded superuser checks. As it turns out, adminpack was creating another function in the catalog for one of those backend functions where the superuser check was removed, specifically pg_rotate_logfile(), but it didn't get the memo about having to REVOKE EXECUTE on the alternative-name function (pg_logfile_rotate()), meaning that in any installations with adminpack on 9.6 and higher, any user is able to run the pg_logfile_rotate() function, which then calls pg_rotate_logfile() and rotates the logfile. Fix by adding a new version of adminpack (1.1) which handles the REVOKE. As this function should have only been available to the superuser, this is a security issue, albeit a minor one. In HEAD, move the changes implemented for adminpack up to be adminpack 2.0 instead of 1.1. Security: CVE-2018-1115 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7b347409fa2776fbaa4ec9c57365f48a2bbdb80c Modified Files -- contrib/adminpack/Makefile| 2 +- contrib/adminpack/adminpack--1.0--1.1.sql | 47 +--- contrib/adminpack/adminpack--1.1--2.0.sql | 51 +++ contrib/adminpack/adminpack.control | 2 +- 4 files changed, 54 insertions(+), 48 deletions(-)
Re: pgsql: Fix EXEC BACKEND + Windows builds for group privs
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost <sfr...@snowman.net> writes: > > Fix EXEC BACKEND + Windows builds for group privs > > jacana seems to think there's still an issue here: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana=2018-04-09%2000%3A00%3A41 > > It's pretty much the same symptom: > > Apr 08 20:58:22 Running this script will delete the old cluster's data files: > Apr 08 20:58:22 delete_old_cluster.bat > Apr 08 20:58:33 ++ find > /home/pgrunner/bf/root/HEAD/pgsql.build/src/bin/pg_upgrade/tmp_check/data > -type f '!' -perm 640 > Apr 08 20:58:33 ++ wc -l > Apr 08 20:58:42 + '[' 3361 -ne 0 ']' > Apr 08 20:58:42 + echo 'files in PGDATA with permission != 640' > Apr 08 20:58:42 files in PGDATA with permission != 640 > Apr 08 20:58:42 + exit 1 > Apr 08 20:58:42 make: *** [check] Error 1 Yes, I was just looking at that and working up what I believe will be a fix. This is exactly the kind of test I was anticipating getting run unintentionally on Windows, sorry about that, but I don't have a Windows test environment currently. I've pushed what I believe will fix this, which is essentially "don't try to run this test on this platform." Tested it locally (under Linux) and it didn't blow up, so hopefully this doesn't make things worse, in any case. If there's more that's needed to fix jacana, I'll see about dealing with it first thing in the morning. Thanks! Stephen signature.asc Description: PGP signature
pgsql: Skip permissions test under MINGW/Windows
Skip permissions test under MINGW/Windows We don't support the same kind of permissions tests on Windows/MINGW, so these tests really shouldn't be getting run on that platform. Per buildfarm. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9d0383c015aabd7565f3df010a5ff56b46aa8756 Modified Files -- src/bin/pg_upgrade/test.sh | 25 - 1 file changed, 16 insertions(+), 9 deletions(-)
pgsql: Fix EXEC BACKEND + Windows builds for group privs
Fix EXEC BACKEND + Windows builds for group privs Under EXEC BACKEND we also need to be going through the group privileges setup since we do support that on Unixy systems, so add that to SubPostmasterMain(). Under Windows, we need to simply return true from GetDataDirectoryCreatePerm(), but that wasn't happening due to a missing #else clause. Per buildfarm. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2b74022473f0c7a7fbe11a4973f80478226275bd Modified Files -- src/backend/postmaster/postmaster.c | 8 src/common/file_perm.c | 9 +++-- 2 files changed, 15 insertions(+), 2 deletions(-)
pgsql: Add default roles for file/program access
Add default roles for file/program access This patch adds new default roles named 'pg_read_server_files', 'pg_write_server_files', 'pg_execute_server_program' which allow an administrator to GRANT to a non-superuser role the ability to access server-side files or run programs through PostgreSQL (as the user the database is running as). Having one of these roles allows a non-superuser to use server-side COPY to read, write, or with a program, and to use file_fdw (if installed by a superuser and GRANT'd USAGE on it) to read from files or run a program. The existing misc file functions are also changed to allow a user with the 'pg_read_server_files' default role to read any files on the filesystem, matching the privileges given to that role through COPY and file_fdw from above. Reviewed-By: Michael Paquier Discussion: https://postgr.es/m/20171231191939.GR2416%40tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0fdc8495bff02684142a44ab3bc5b18a8ca1863a Modified Files -- contrib/file_fdw/file_fdw.c | 51 + contrib/file_fdw/output/file_fdw.source | 2 +- doc/src/sgml/file-fdw.sgml | 8 -- doc/src/sgml/func.sgml | 27 +++-- doc/src/sgml/ref/copy.sgml | 8 -- doc/src/sgml/user-manag.sgml| 28 +- src/backend/commands/copy.c | 46 - src/backend/utils/adt/genfile.c | 16 +++ src/include/catalog/pg_authid.h | 6 9 files changed, 145 insertions(+), 47 deletions(-)
pgsql: Support new default roles with adminpack
Support new default roles with adminpack This provides a newer version of adminpack which works with the newly added default roles to support GRANT'ing to non-superusers access to read and write files, along with related functions (unlinking files, getting file length, renaming/removing files, scanning the log file directory) which are supported through adminpack. Note that new versions of the functions are required because an environment might have an updated version of the library but still have the old adminpack 1.0 catalog definitions (where EXECUTE is GRANT'd to PUBLIC for the functions). This patch also removes the long-deprecated alternative names for functions that adminpack used to include and which are now included in the backend, in adminpack v1.1. Applications using the deprecated names should be updated to use the backend functions instead. Existing installations which continue to use adminpack v1.0 should continue to function until/unless adminpack is upgraded. Reviewed-By: Michael Paquier Discussion: https://postgr.es/m/20171231191939.GR2416%40tamriel.snowman.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/11523e860f8fe29f9142fb63c44e01cd0d5e7375 Modified Files -- contrib/adminpack/Makefile| 2 +- contrib/adminpack/adminpack--1.0--1.1.sql | 51 +++ contrib/adminpack/adminpack.c | 245 +++--- contrib/adminpack/adminpack.control | 2 +- contrib/adminpack/expected/adminpack.out | 19 ++- contrib/adminpack/sql/adminpack.sql | 14 +- doc/src/sgml/adminpack.sgml | 55 +-- src/backend/utils/adt/genfile.c | 53 ++- src/backend/utils/adt/misc.c | 27 +++- src/include/catalog/pg_proc.h | 8 +- 10 files changed, 388 insertions(+), 88 deletions(-)
pgsql: Rewrite pg_dump TAP tests
Rewrite pg_dump TAP tests
This reworks how the tests to run are defined. Instead of having to
define all runs for all tests, we define those tests which should pass
(generally using one of the defined broad hashes), add in any which
should be specific for this test, and exclude any specific runs that
shouldn't pass for this test. This ends up removing some 4k+ lines
(more than half the file) but, more importantly, greatly simplifies the
way runs-to-be-tested are defined.
As discussed in the updated comments, for example, take the test which
does CREATE TABLE test_table. That CREATE TABLE should show up in all
'full' runs of pg_dump, except those cases where 'test_table' is
excluded, of course, and that's exactly how the test gets defined now
(modulo a few other related cases, like where we dump only that table,
or we dump the schema it's in, or we exclude the schema it's in):
like => {
%full_runs,
%dump_test_schema_runs,
only_dump_test_table=> 1,
section_pre_data=> 1, },
unlike => {
exclude_dump_test_schema => 1,
exclude_test_table => 1, }, },
Next, we no longer expect every run to be listed for every test. If a
run is listed in 'like' (directly or through a hash) then it's a 'like',
unless it's listed in 'unlike' in which case it's an 'unlike'. If it
isn't listed in either, then it's considered an 'unlike' automatically.
Lastly, this changes the code to no longer use like/unlike but rather to
use 'ok()' with 'diag()' which allows much more control over what gets
spit out to the screen. Gone are the days of the entire dump being sent
to the console, now you'll just get a couple of lines for each failing
test which say the test that failed and the run that it failed on.
This covers both the pg_dump TAP tests in src/bin/pg_dump and those in
src/test/modules/test_pg_dump.
Branch
--
master
Details
---
https://git.postgresql.org/pg/commitdiff/446f7f5d789fe9ecfacd998407b5bee70aaa64f7
Modified Files
--
src/bin/pg_dump/t/002_pg_dump.pl| 5037 ---
src/test/modules/test_pg_dump/t/001_base.pl | 443 +--
2 files changed, 780 insertions(+), 4700 deletions(-)
Re: pgsql: Remove pg_class.relhaspkey
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2018-03-14 19:35:40 +, Peter Eisentraut wrote: > > Remove pg_class.relhaspkey > > > > It is not used for anything internally, and it cannot be relied on for > > external uses, so it can just be removed. To correct recommended way to > > check for a primary key is in pg_index. > > > > Discussion: > > https://www.postgresql.org/message-id/flat/b1a24c6c-6913-f89c-674e-0704f0ed6...@2ndquadrant.com > > I'm not clear on the mechanics, and the animal doesn't show the critical > log. But this might have caused the issue, being the only commit between > runs: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake=2018-03-14%2019%3A37%3A20 > > Andrew, any chance you could modify the module to capture > pg_upgrade_dump_*.log? I've not quite tracked it down, but I would caution against blaming this commit- when doing some parallel regression test runs, I was seeing failures also, but they've not been consistent and I was trying to get something else done so didn't focus on them, so they may have been failures due to my environment, but might also be some kind of race condition in an earlier commit (my guess at the moment is actually the INOUT arguments for procedures commit...). I'll try doing some more runs to see if I can reproduce it, but wanted to mention it, just to encourage people to consider looking more broadly than just this commit if they run into similar issues. Thanks! Stephen signature.asc Description: PGP signature
pgsql: Fix comment for ExecProcessReturning
Fix comment for ExecProcessReturning resultRelInfo is the argument for the function, not projectReturning. Author: Etsuro Fujita Discussion: https://postgr.es/m/5aa8e11e.1040...@lab.ntt.co.jp Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/97d18ce27da47de2de60de8dfca8e364e71ff6fe Modified Files -- src/backend/executor/nodeModifyTable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Improve ALTER TABLE synopsis
Improve ALTER TABLE synopsis Add into the ALTER TABLE synopsis the definition of partition_bound_spec, column_constraint, index_parameters and exclude_element. Initial patch by Lætitia Avrot, with further improvements by Amit Langote and Thomas Munro. Discussion: https://postgr.es/m/flat/27ec4df3-d1ab-3411-f87f-647f944897e1%40lab.ntt.co.jp Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a2a22057617dc84b500f85938947c125183f1289 Modified Files -- doc/src/sgml/ref/alter_table.sgml | 31 +++ 1 file changed, 31 insertions(+)
Re: pgsql: postgres_fdw: Judge password use by run-as user, not session use
Robert, * Robert Haas (rh...@postgresql.org) wrote: > postgres_fdw: Judge password use by run-as user, not session user. There was apparently a regression test that was broken by this and is making the buildfarm rather unhappy.. Thanks! Stephen signature.asc Description: Digital signature
Migration to PGLister - After
Greetings! This list has now been migrated to new mailing list software known as 'PGLister'. This migration will impact all users of this mailing list in one way or another. If you would like to unsubscribe from this mailing list, please click on 'Show Original' or 'Show Headers' in your client and find the 'List-Unsubscribe' header which will include a link that you can click on (or copy/paste into your browser) and use to unsubscribe yourself from this list. The changes which we expect to be most significant to users can be found on the wiki here: https://wiki.postgresql.org/wiki/PGLister_Announce the current version of which is also included below. Thank you! Stephen (on behalf of the pginfra team) signature.asc Description: Digital signature
