pgsql: Improve estimation of OR clauses using extended statistics.
Improve estimation of OR clauses using extended statistics. Formerly we only applied extended statistics to an OR clause as part of the clauselist_selectivity() code path for an OR clause appearing in an implicitly-ANDed list of clauses. This meant that it could only use extended statistics if all sub-clauses of the OR clause were covered by a single extended statistics object. Instead, teach clause_selectivity() how to apply extended statistics to an OR clause by handling its ORed list of sub-clauses in a similar manner to an implicitly-ANDed list of sub-clauses, but with different combination rules. This allows one or more extended statistics objects to be used to estimate all or part of the list of sub-clauses. Any remaining sub-clauses are then treated as if they are independent. Additionally, to avoid double-application of extended statistics, this introduces "extended" versions of clause_selectivity() and clauselist_selectivity(), which include an option to ignore extended statistics. This replaces the old clauselist_selectivity_simple() function which failed to completely ignore extended statistics when called from the extended statistics code. A known limitation of the current infrastructure is that an AND clause under an OR clause is not treated as compatible with extended statistics (because we don't build RestrictInfos for such sub-AND clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will currently be treated as two independent AND clauses (each of which may be estimated using extended statistics), but extended statistics will not currently be used to account for any possible overlap between those clauses. Improving that is left as a task for the future. Original patch by Tomas Vondra, with additional improvements by me. Discussion: https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/25a9e54d2db31b8031c2d8166114b187e8347098 Modified Files -- src/backend/optimizer/path/clausesel.c | 301 +++ src/backend/statistics/dependencies.c| 4 +- src/backend/statistics/extended_stats.c | 217 +++- src/backend/statistics/mcv.c | 169 - src/include/optimizer/optimizer.h| 18 +- src/include/statistics/extended_stats_internal.h | 15 ++ src/include/statistics/statistics.h | 3 +- src/test/regress/expected/stats_ext.out | 189 +- src/test/regress/sql/stats_ext.sql | 85 ++- 9 files changed, 798 insertions(+), 203 deletions(-)
pgsql: Small code simplifications
Small code simplifications strVal() can be used in a couple of places instead of coding the same thing by hand. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/6114040711affa2b0bcf47fa2791187daf8455fb Modified Files -- src/backend/foreign/foreign.c | 2 +- src/backend/parser/gram.y | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/8b5c353ecec81102680ba6ab82ce73961dd266a1 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 56 + src/bin/pg_rewind/t/008_min_recovery_point.pl | 161 ++ 3 files changed, 205 insertions(+), 23 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL9_5_STABLE Details --- https://git.postgresql.org/pg/commitdiff/81e3c868f891bf291a3ad38d81b04096146c2f6f Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 60 ++ src/bin/pg_rewind/t/008_min_recovery_point.pl | 165 ++ 3 files changed, 211 insertions(+), 25 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/020f153dbb48519ebb9780ddbfe87b0532324d49 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 56 + src/bin/pg_rewind/t/008_min_recovery_point.pl | 156 ++ 3 files changed, 200 insertions(+), 23 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2b4f3130382fe2f8705863e4d38589d4d69cd695 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 56 + src/bin/pg_rewind/t/008_min_recovery_point.pl | 156 ++ 3 files changed, 200 insertions(+), 23 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/63e316f0bc1d73276b94283ab482513892c5ee75 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 56 + src/bin/pg_rewind/t/008_min_recovery_point.pl | 161 ++ 3 files changed, 205 insertions(+), 23 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL9_6_STABLE Details --- https://git.postgresql.org/pg/commitdiff/0740857de7804609783eb9f11f055776c1fff4e8 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 56 + src/bin/pg_rewind/t/008_min_recovery_point.pl | 165 ++ 3 files changed, 209 insertions(+), 23 deletions(-)
pgsql: Fix pg_rewind bugs when rewinding a standby server.
Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/abd0abfb749d39f46682fe84a1c6f973d2d8ddc2 Modified Files -- src/bin/pg_rewind/parsexlog.c | 11 +- src/bin/pg_rewind/pg_rewind.c | 58 ++ src/bin/pg_rewind/t/008_min_recovery_point.pl | 156 ++ 3 files changed, 201 insertions(+), 24 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/673b3c7ed934a5b3271671b4431880253f459971 Modified Files -- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/792d000e53b398f88c1e4bb22826a408b1cc0b8c Modified Files -- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 6 +++--- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/d415eec304c3398caa12b910877f5b2bd8545d5f Modified Files -- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 6 +++--- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL9_6_STABLE Details --- https://git.postgresql.org/pg/commitdiff/89cdf1b65e6b64c3bd784717ee4429e9db5eaf8d Modified Files -- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a659e789b743f1a4b977a1ffc7402ab618a9f68c Modified Files -- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 6 +++--- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/610e9f5b361e19f160fa27747e47d5495eb0f2ba Modified Files -- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 6 +++--- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-)
pgsql: docs: list single-letter options first in command-line summary
docs: list single-letter options first in command-line summary In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 Branch -- REL9_5_STABLE Details --- https://git.postgresql.org/pg/commitdiff/47c4843a03cac9c35d97ddc42868b157731d2032 Modified Files -- doc/src/sgml/ref/reindexdb.sgml | 10 +- doc/src/sgml/ref/vacuumdb.sgml | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
pgsql: doc: remove unnecessary blank before command option text
doc: remove unnecessary blank before command option text Backpatch-through: 11 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3f8971d92e767acf6e3d6e27c4cab7bfd88b71f4 Modified Files -- doc/src/sgml/ref/pg_checksums.sgml | 2 +- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
pgsql: doc: remove unnecessary blank before command option text
doc: remove unnecessary blank before command option text Backpatch-through: 11 Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/aee1f7496113bfe5e7ea6acdef49b86405507a83 Modified Files -- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
pgsql: doc: remove unnecessary blank before command option text
doc: remove unnecessary blank before command option text Backpatch-through: 11 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/7b0bd08a325fd4f21269dbc9a6dd82809342644d Modified Files -- doc/src/sgml/ref/pg_checksums.sgml | 2 +- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
pgsql: doc: remove unnecessary blank before command option text
doc: remove unnecessary blank before command option text Backpatch-through: 11 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/f3f033a28426796653cadbc576fe83ec3ab60673 Modified Files -- doc/src/sgml/ref/pg_checksums.sgml | 2 +- doc/src/sgml/ref/pg_controldata.sgml | 2 +- doc/src/sgml/ref/pg_resetwal.sgml| 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
pgsql: Change SHA2 implementation based on OpenSSL to use EVP digest ro
Change SHA2 implementation based on OpenSSL to use EVP digest routines The use of low-level hash routines is not recommended by upstream OpenSSL since 2000, and pgcrypto already switched to EVP as of 5ff4a67. This takes advantage of the refactoring done in 87ae969 that has introduced the allocation and free routines for cryptographic hashes. Since 1.1.0, OpenSSL does not publish the contents of the cryptohash contexts, forcing any consumers to rely on OpenSSL for all allocations. Hence, the resource owner callback mechanism gains a new set of routines to track and free cryptohash contexts when using OpenSSL, preventing any risks of leaks in the backend. Nothing is needed in the frontend thanks to the refactoring of 87ae969, and the resowner knowledge is isolated into cryptohash_openssl.c. Note that this also fixes a failure with SCRAM authentication when using FIPS in OpenSSL, but as there have been few complaints about this problem and as this causes an ABI breakage, no backpatch is done. Author: Michael Paquier Reviewed-by: Daniel Gustafsson, Heikki Linnakangas Discussion: https://postgr.es/m/[email protected] Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4f48a6fbe2b28d8281dbbfa2d334fa2ed8472734 Modified Files -- src/backend/replication/basebackup.c | 8 ++- src/backend/utils/resowner/resowner.c | 62 src/common/cryptohash_openssl.c | 128 +- src/include/utils/resowner_private.h | 7 ++ src/tools/pgindent/typedefs.list | 1 + 5 files changed, 157 insertions(+), 49 deletions(-)
pgsql: Rename cryptohashes.c to cryptohashfuncs.c
Rename cryptohashes.c to cryptohashfuncs.c
87ae969 has created two new files called cryptohash{_openssl}.c in
src/common/, whose names overlap with the existing backend file called
cryptohashes.c dedicated to the SQL wrappers for SHA2 and MD5. This
file is renamed to cryptohashfuncs.c to be more consistent with the
surroundings and reduce the confusion with the new cryptohash interface
of src/common/.
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/[email protected]
Branch
--
master
Details
---
https://git.postgresql.org/pg/commitdiff/bd94a9c04e04bb3b626e88981a50fcca2bd99d60
Modified Files
--
src/backend/utils/adt/Makefile | 2 +-
src/backend/utils/adt/{cryptohashes.c => cryptohashfuncs.c} | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
