# PostgreSQL Weekly News - December 6, 2020 Person of the week: https://postgresql.life/post/stephane_schildknecht/
# PostgreSQL Product News check_pgactivity 2.5, a Nagios remote agent for monitoring PostgreSQL, released. https://github.com/OPMDG/check_pgactivity/releases/latest WAL-G 0.2.19, a backup management system for PostgreSQL written in Go, released. https://github.com/wal-g/wal-g/releases # PostgreSQL Jobs for December [http://archives.postgresql.org/pgsql-jobs/2020-12/](http://archives.postgresql.org/pgsql-jobs/2020-12/) # PostgreSQL in the News Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to [email protected]. == Applied Patches == Tom Lane pushed: - Fix recently-introduced breakage in psql's \connect command. Through my misreading of what the existing code actually did, commits 85c54287a et al. broke psql's behavior for the case where "\c connstring" provides a password in the connstring. We should use that password in such a case, but as of 85c54287a we ignored it (and instead, prompted for a password). Commit 94929f1cf fixed that in HEAD, but since I thought it was cleaning up a longstanding misbehavior and not one I'd just created, I didn't back-patch it. Hence, back-patch the portions of 94929f1cf having to do with password management. In addition to fixing the introduced bug, this means that "\c -reuse-previous=on connstring" will allow re-use of an existing connection's password if the connstring doesn't change user/host/port. That didn't happen before, but it seems like a bug fix, and anyway I'm loath to have significant differences in this code across versions. Also fix an error with the same root cause about whether or not to override a connstring's setting of client_encoding. As of 85c54287a we always did so; restore the previous behavior of overriding only when stdin/stdout are a terminal and there's no environment setting of PGCLIENTENCODING. (I find that definition a bit surprising, but right now doesn't seem like the time to revisit it.) Per bug #16746 from Krzysztof Gradek. As with the previous patch, back-patch to all supported branches. Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/7e5e1bba034ee02245e8a3833aa1f6ea7253b584](https://git.postgresql.org/pg/commitdiff/7e5e1bba034ee02245e8a3833aa1f6ea7253b584) - Fix miscomputation of direct_lateral_relids for join relations. If a PlaceHolderVar is to be evaluated at a join relation, but its value is only needed there and not at higher levels, we neglected to update the joinrel's direct_lateral_relids to include the PHV's source rel. This causes problems because join_is_legal() then won't allow joining the joinrel to the PHV's source rel at all, leading to "failed to build any N-way joins" planner failures. Per report from Andreas Seltenreich. Back-patch to 9.5 where the problem originated. Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/b1738ff6ab73203cbbc02d7fb82941dbc061d301](https://git.postgresql.org/pg/commitdiff/b1738ff6ab73203cbbc02d7fb82941dbc061d301) - Prevent parallel index build in a standalone backend. This can't work if there's no postmaster, and indeed the code got an assertion failure trying. There should be a check on IsUnderPostmaster gating the use of parallelism, as the planner has for ordinary parallel queries. Commit 40d964ec9 got this right, so follow its model of checking IsUnderPostmaster at the same place where we check for max_parallel_maintenance_workers == 0. In general, new code implementing parallel utility operations should do the same. Report and patch by Yulin Pei, cosmetically adjusted by me. Back-patch to v11 where this code came in. Discussion: [https://postgr.es/m/hk0pr01mb22747d839f77142d7e76a45df4...@hk0pr01mb2274.apcprd01.prod.exchangelabs.com](https://postgr.es/m/hk0pr01mb22747d839f77142d7e76a45df4...@hk0pr01mb2274.apcprd01.prod.exchangelabs.com) [https://git.postgresql.org/pg/commitdiff/275b3411d9189b3974687766db06727d64d22979](https://git.postgresql.org/pg/commitdiff/275b3411d9189b3974687766db06727d64d22979) - Remove configure-time probe for DocBook DTD. Checking for DocBook being installed was valuable when we were on the OpenSP docs toolchain, because that was rather hard to get installed fully. Nowadays, as long as you have xmllint and xsltproc installed, you're good, because those programs will fetch the DocBook files off the net at need. Moreover, testing this at configure time means that a network access may well occur whether or not you have any interest in building the docs later. That can be slow (typically 2 or 3 seconds, though much higher delays have been reported), and it seems not very nice to be doing an off-machine access without warning, too. Hence, drop the PGAC_CHECK_DOCBOOK probe, and adjust related documentation. Without that macro, there's not much left of config/docbook.m4 at all, so I just removed it. Back-patch to v11, where we started to use xmllint in the PGAC_CHECK_DOCBOOK probe. Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/4823c4f6ac770ebd49b63b0ce64ecfe82987af57](https://git.postgresql.org/pg/commitdiff/4823c4f6ac770ebd49b63b0ce64ecfe82987af57) - Fix missing outfuncs.c support for IncrementalSortPath. For debugging purposes, Path nodes are supposed to have outfuncs support, but this was overlooked in the original incremental sort patch. While at it, clean up a couple other minor oversights, as well as bizarre choice of return type for create_incremental_sort_path(). (All the existing callers just cast it to "Path *" immediately, so they don't care, but some future caller might care.) outfuncs.c fix by Zhijie Hou, the rest by me Discussion: [https://postgr.es/m/324c4d81d8134117972a5b1f6cdf9560@G08CNEXMBPEKD05.g08.fujitsu.local](https://postgr.es/m/324c4d81d8134117972a5b1f6cdf9560@G08CNEXMBPEKD05.g08.fujitsu.local) [https://git.postgresql.org/pg/commitdiff/8286223f3d820c39f2d5f14222f7ccde53bdf502](https://git.postgresql.org/pg/commitdiff/8286223f3d820c39f2d5f14222f7ccde53bdf502) - Ensure that expandTableLikeClause() re-examines the same table. As it stood, expandTableLikeClause() re-did the same relation_openrv call that transformTableLikeClause() had done. However there are scenarios where this would not find the same table as expected. We hold lock on the LIKE source table, so it can't be renamed or dropped, but another table could appear before it in the search path. This explains the odd behavior reported in bug #16758 when cloning a table as a temp table of the same name. This case worked as expected before commit 502898192 introduced the need to open the source table twice, so we should fix it. To make really sure we get the same table, let's re-open it by OID not name. That requires adding an OID field to struct TableLikeClause, which is a little nervous-making from an ABI standpoint, but as long as it's at the end I don't think there's any serious risk. Per bug #16758 from Marc Boeren. Like the previous patch, back-patch to all supported branches. Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/f7f83a55bf6051818a0e4387d718867ecfa8561b](https://git.postgresql.org/pg/commitdiff/f7f83a55bf6051818a0e4387d718867ecfa8561b) - Fix missed step in removal of useless RESULT RTEs in the planner. Commit 4be058fe9 forgot that the append_rel_list would already be populated at the time we remove useless result RTEs, and it might contain PlaceHolderVars that need to be adjusted like the ones in the main parse tree. This could lead to "no relation entry for relid N" failures later on, when the planner tries to do something with an unadjusted PHV. Per report from Tom Ellis. Back-patch to v12 where the bug came in. Discussion: [https://postgr.es/m/20201205173056.GF30712@cloudinit-builder](https://postgr.es/m/20201205173056.GF30712@cloudinit-builder) [https://git.postgresql.org/pg/commitdiff/e98c900993e89ad9278cdfbf0ba5495381a1faac](https://git.postgresql.org/pg/commitdiff/e98c900993e89ad9278cdfbf0ba5495381a1faac) Fujii Masao pushed: - Improve log message about termination of background workers. Previously the shutdown of a background worker that uses die() as SIGTERM signal handler produced the log message "terminating connection due to administrator command". This log message was confusing because a background worker is not a connection. This commit improves that log message to "terminating background worker XXX due to administrator command" (XXX is replaced with the name of the background worker). This is the same log message as another SIGTERM signal handler bgworker_die() for a background worker reports. Author: Bharath Rupireddy Reviewed-by: Fujii Masao Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/98e2d58d66c81d9f1e49813c1acd4d7312089f4a](https://git.postgresql.org/pg/commitdiff/98e2d58d66c81d9f1e49813c1acd4d7312089f4a) - Fix typo in comment. Author: Haiying Tang <[email protected]> Discussion: [https://postgr.es/m/48a0928ac94b497d9c40acf1de394c15@G08CNEXMBPEKD05.g08.fujitsu.local](https://postgr.es/m/48a0928ac94b497d9c40acf1de394c15@G08CNEXMBPEKD05.g08.fujitsu.local) [https://git.postgresql.org/pg/commitdiff/6742e14959a3033d946ab3d67f5ce4c99367d332](https://git.postgresql.org/pg/commitdiff/6742e14959a3033d946ab3d67f5ce4c99367d332) - doc: Add additional index entries for progress reporting views. In the docs, the index entries for progress reporting views link to the "Viewing Statistics" section, but previously they did not link to the dedicated section (e.g., "ANALYZE Progress Reporting") for each view. This was inconvenient when finding the section describing the detailed information about each view, from the index. This commit adds additional index entries linking to those dedicated sections. Author: Fujii Masao Reviewed-by: Shinya Kato Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/0a4db67b5ed05c4013ea968930af36853f088404](https://git.postgresql.org/pg/commitdiff/0a4db67b5ed05c4013ea968930af36853f088404) - Allow restore_command parameter to be changed with reload. This commit changes restore_command from PGC_POSTMASTER to PGC_SIGHUP. As the side effect of this commit, restore_command can be reset to empty during archive recovery. In this setting, archive recovery tries to replay only WAL files available in pg_wal directory. This is the same behavior as when the command that always fails is specified in restore_command. Note that restore_command still must be specified (not empty) when starting archive recovery, even after applying this commit. This is necessary as the safeguard to prevent users from forgetting to specify restore_command and starting archive recovery. Thanks to Peter Eisentraut, Michael Paquier, Andres Freund, Robert Haas and Anastasia Lubennikova for discussion. Author: Sergei Kornilov Reviewed-by: Kyotaro Horiguchi, Fujii Masao Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/942305a36365433eff3c1937945758f2dbf1662b](https://git.postgresql.org/pg/commitdiff/942305a36365433eff3c1937945758f2dbf1662b) - Track total number of WAL records, FPIs and bytes generated in the cluster. Commit 6b466bf5f2 allowed pg_stat_statements to track the number of WAL records, full page images and bytes that each statement generated. Similarly this commit allows us to track the cluster-wide WAL statistics counters. New columns wal_records, wal_fpi and wal_bytes are added into the pg_stat_wal view, and reports the total number of WAL records, full page images and bytes generated in the , respectively. Author: Masahiro Ikeda Reviewed-by: Amit Kapila, Movead Li, Kyotaro Horiguchi, Fujii Masao Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/01469241b2ff6f17fc0b3f20f01559145ffab29e](https://git.postgresql.org/pg/commitdiff/01469241b2ff6f17fc0b3f20f01559145ffab29e) Heikki Linnakangas pushed: - Remove leftover comments, left behind by removal of WITH OIDS. Author: Amit Langote Discussion: [https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com](https://www.postgresql.org/message-id/CA%2BHiwqGaRoF3XrhPW-Y7P%2BG7bKo84Z_h%3DkQHvMh-80%3Dav3wmOw%40mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/2bc588798bdd0cdaa8f6cb6713ba8c8cc039fcb1](https://git.postgresql.org/pg/commitdiff/2bc588798bdd0cdaa8f6cb6713ba8c8cc039fcb1) - docs: ulink all references to RFC's. Make sure that the first mentions of RFC's are ulinked to their ietf.org entry, and subsequent ones are marked as acronyms. This makes references to RFC's consistent across the documentation. Author: Daniel Gustafsson Discussion: [https://www.postgresql.org/message-id/2C697878-4D01-4F06-8312-2FEDE931E973%40yesql.se](https://www.postgresql.org/message-id/2C697878-4D01-4F06-8312-2FEDE931E973%40yesql.se) [https://git.postgresql.org/pg/commitdiff/415dc2009683904f337a1837b6b4eb7f31c4dc55](https://git.postgresql.org/pg/commitdiff/415dc2009683904f337a1837b6b4eb7f31c4dc55) - Fix pg_rewind bugs when rewinding a standby server. If the target is a standby server, its WAL doesn't end at the last checkpoint record, but at minRecoveryPoint. We must scan all the WAL from the last common checkpoint all the way up to minRecoveryPoint for modified pages, and also consider that portion when determining whether the server needs rewinding. Backpatch to all supported versions. Author: Ian Barwick and me Discussion: [https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com](https://www.postgresql.org/message-id/CABvVfJU-LDWvoz4-Yow3Ay5LZYTuPD7eSjjE4kGyNZpXC6FrVQ%40mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/2b4f3130382fe2f8705863e4d38589d4d69cd695](https://git.postgresql.org/pg/commitdiff/2b4f3130382fe2f8705863e4d38589d4d69cd695) - Fix race conditions in newly-added test. Buildfarm has been failing sporadically on the new test. I was able to reproduce this by adding a random 0-10 s delay in the walreceiver, just before it connects to the primary. There's a race condition where node_3 is promoted before it has fully caught up with node_1, leading to diverged timelines. When node_1 is later reconfigured as standby following node_3, it fails to catch up: LOG: primary server contains no more WAL on requested timeline 1 LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/30000A0 That's the situation where you'd need to use pg_rewind, but in this case it happens already when we are just setting up the actual pg_rewind scenario we want to test, so change the test so that it waits until node_3 is connected and fully caught up before promoting it, so that you get a clean, controlled failover. Also rewrite some of the comments, for clarity. The existing comments detailed what each step in the test did, but didn't give a good overview of the situation the steps were trying to create. For reasons I don't understand, the test setup had to be written slightly differently in 9.6 and 9.5 than in later versions. The 9.5/9.6 version needed node 1 to be reinitialized from backup, whereas in later versions it could be shut down and reconfigured to be a standby. But even 9.5 should support "clean switchover", where primary makes sure that pending WAL is replicated to standby on shutdown. It would be nice to figure out what's going on there, but that's independent of pg_rewind and the scenario that this test tests. Discussion: [https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi](https://www.postgresql.org/message-id/b0a3b95b-82d2-6089-6892-40570f8c5e60%40iki.fi) [https://git.postgresql.org/pg/commitdiff/36a4ac20fcf31361bd42b63b1b3390b28827a69e](https://git.postgresql.org/pg/commitdiff/36a4ac20fcf31361bd42b63b1b3390b28827a69e) Michaël Paquier pushed: - Refactor parsing rules for option lists of EXPLAIN, VACUUM and ANALYZE. Those three commands have been using the same grammar rules to handle a a list of parenthesized options. This refactors the code so as they use the same parsing rules, shaving some code. A future commit will make use of those option parsing rules for more utility commands, like REINDEX and CLUSTER. Author: Alexey Kondratov, Justin Pryzby Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/873ea9ee692e7829614f913685db540b17998ba6](https://git.postgresql.org/pg/commitdiff/873ea9ee692e7829614f913685db540b17998ba6) - doc: Remove more notes about compatibilities with past versions. This is a follow-up of the work done in fa42c2e, that did not include all the fixes previously agreed on. The contents removed here can be confusing to the reader as they refer to rather old server versions. Author: Stephen Frost, Tom Lane, Heikki Linnakangas, Yaroslav Schekin Discussion: [https://postgr.es/m/CAB8KJ=jyhgnxllzsnjz7gbtck4txomngcmgkw3nemsnf0yl...@mail.gmail.com](https://postgr.es/m/CAB8KJ=jyhgnxllzsnjz7gbtck4txomngcmgkw3nemsnf0yl...@mail.gmail.com) Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/8a17f44c1e7a9f3d2a9da97dc3eba4184a2a453c](https://git.postgresql.org/pg/commitdiff/8a17f44c1e7a9f3d2a9da97dc3eba4184a2a453c) - Move SHA2 routines to a new generic API layer for crypto hashes. Two new routines to allocate a hash context and to free it are created, as these become necessary for the goal behind this refactoring: switch the all cryptohash implementations for OpenSSL to use EVP (for FIPS and also because upstream does not recommend the use of low-level cryptohash functions for 20 years). Note that OpenSSL hides the internals of cryptohash contexts since 1.1.0, so it is necessary to leave the allocation to OpenSSL itself, explaining the need for those two new routines. This part is going to require more work to properly track hash contexts with resource owners, but this not introduced here. Still, this refactoring makes the move possible. This reduces the number of routines for all SHA2 implementations from twelve (SHA{224,256,386,512} with init, update and final calls) to five (create, free, init, update and final calls) by incorporating the hash type directly into the hash context data. The new cryptohash routines are moved to a new file, called cryptohash.c for the fallback implementations, with SHA2 specifics becoming a part internal to src/common/. OpenSSL specifics are part of cryptohash_openssl.c. This infrastructure is usable for more hash types, like MD5 or HMAC. Any code paths using the internal SHA2 routines are adapted to report correctly errors, which are most of the changes of this commit. The zones mostly impacted are checksum manifests, libpq and SCRAM. Note that e21cbb4 was a first attempt to switch SHA2 to EVP, but it lacked the refactoring needed for libpq, as done here. This patch has been tested on Linux and Windows, with and without OpenSSL, and down to 1.0.1, the oldest version supported on HEAD. Author: Michael Paquier Reviewed-by: Daniel Gustafsson Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/87ae9691d25379785f8c0f81b06a14818cfd8c56](https://git.postgresql.org/pg/commitdiff/87ae9691d25379785f8c0f81b06a14818cfd8c56) - Fix compilation warnings in cryptohash_openssl.c. These showed up with -O2. Oversight in 87ae969. Author: Fujii Masao Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/91624c2ff8809145880383b0fa84be0ee98f55b5](https://git.postgresql.org/pg/commitdiff/91624c2ff8809145880383b0fa84be0ee98f55b5) - Refactor CLUSTER and REINDEX grammar to use DefElem for option lists. This changes CLUSTER and REINDEX so as a parenthesized grammar becomes possible for options, while unifying the grammar parsing rules for option lists with the existing ones. This is a follow-up of the work done in 873ea9e for VACUUM, ANALYZE and EXPLAIN. This benefits REINDEX for a potential backend-side filtering for collatable-sensitive indexes and TABLESPACE, while CLUSTER would benefit from the latter. Author: Alexey Kondratov, Justin Pryzby Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/b5913f6120792465f4394b93c15c2e2ac0c08376](https://git.postgresql.org/pg/commitdiff/b5913f6120792465f4394b93c15c2e2ac0c08376) - Change SHA2 implementation based on OpenSSL to use EVP digest routines. The use of low-level hash routines is not recommended by upstream OpenSSL since 2000, and pgcrypto already switched to EVP as of 5ff4a67. This takes advantage of the refactoring done in 87ae969 that has introduced the allocation and free routines for cryptographic hashes. Since 1.1.0, OpenSSL does not publish the contents of the cryptohash contexts, forcing any consumers to rely on OpenSSL for all allocations. Hence, the resource owner callback mechanism gains a new set of routines to track and free cryptohash contexts when using OpenSSL, preventing any risks of leaks in the backend. Nothing is needed in the frontend thanks to the refactoring of 87ae969, and the resowner knowledge is isolated into cryptohash_openssl.c. Note that this also fixes a failure with SCRAM authentication when using FIPS in OpenSSL, but as there have been few complaints about this problem and as this causes an ABI breakage, no backpatch is done. Author: Michael Paquier Reviewed-by: Daniel Gustafsson, Heikki Linnakangas Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/4f48a6fbe2b28d8281dbbfa2d334fa2ed8472734](https://git.postgresql.org/pg/commitdiff/4f48a6fbe2b28d8281dbbfa2d334fa2ed8472734) - Rename cryptohashes.c to cryptohashfuncs.c. 87ae969 has created two new files called cryptohash{_openssl}.c in src/common/, whose names overlap with the existing backend file called cryptohashes.c dedicated to the SQL wrappers for SHA2 and MD5. This file is renamed to cryptohashfuncs.c to be more consistent with the surroundings and reduce the confusion with the new cryptohash interface of src/common/. Author: Michael Paquier Reviewed-by: Daniel Gustafsson Discussion: [https://postgr.es/m/[email protected]](https://postgr.es/m/[email protected]) [https://git.postgresql.org/pg/commitdiff/bd94a9c04e04bb3b626e88981a50fcca2bd99d60](https://git.postgresql.org/pg/commitdiff/bd94a9c04e04bb3b626e88981a50fcca2bd99d60) Álvaro Herrera pushed: - Document concurrent indexes waiting on each other. Because regular CREATE INDEX commands are independent, and there's no logical data dependency, it's not immediately obvious that transactions held by concurrent index builds on one table will block the second phase of concurrent index creation on an unrelated table, so document this caveat. Backpatch this all the way back. In branch master, mention that only some indexes are involved. Author: James Coleman <[email protected]> Reviewed-by: David Johnston <[email protected]> Discussion: [https://postgr.es/m/CAAaqYe994=purn8cjz4ueo_s-ffrr_3ogeryhtdghab2wg_...@mail.gmail.com](https://postgr.es/m/CAAaqYe994=purn8cjz4ueo_s-ffrr_3ogeryhtdghab2wg_...@mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/58ebe967f8a1e34a824d6a0a35728027272041c6](https://git.postgresql.org/pg/commitdiff/58ebe967f8a1e34a824d6a0a35728027272041c6) - Avoid memcpy() with a NULL source pointer and count == 0. When memcpy() is called on a pointer, the compiler is entitled to assume that the pointer is not null, which can lead to optimizing nearby code in potentially undesirable ways. We still want such optimizations (gcc's -fdelete-null-pointer-checks) in cases where they're valid. Related: commit 13bba02271dc. Backpatch to pg11, where this particular instance appeared. Reported-by: Ranier Vilela <[email protected]> Reported-by: Zhihong Yu <[email protected]> Discussion: [https://postgr.es/m/caeudqapundmqkr5flrckxq7+ib44i7s+kk93pyvths85png...@mail.gmail.com](https://postgr.es/m/caeudqapundmqkr5flrckxq7+ib44i7s+kk93pyvths85png...@mail.gmail.com) Discussion: [https://postgr.es/m/calnj-vsdhwsm5f4tnnn1cdlhvxmve_s+v3nr5gwnrmcpnb2...@mail.gmail.com](https://postgr.es/m/calnj-vsdhwsm5f4tnnn1cdlhvxmve_s+v3nr5gwnrmcpnb2...@mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/677f74e5bb8360122ebf2f473d7169ed65ce4dba](https://git.postgresql.org/pg/commitdiff/677f74e5bb8360122ebf2f473d7169ed65ce4dba) Thomas Munro pushed: - Free disk space for dropped relations on commit. When committing a transaction that dropped a relation, we previously truncated only the first segment file to free up disk space (the one that won't be unlinked until the next checkpoint). Truncate higher numbered segments too, even though we unlink them on commit. This frees the disk space immediately, even if other backends have open file descriptors and might take a long time to get around to handling shared invalidation events and closing them. Also extend the same behavior to the first segment, in recovery. Back-patch to all supported releases. Bug: #16663 Reported-by: Denis Patron <[email protected]> Reviewed-by: Pavel Borisov <[email protected]> Reviewed-by: Neil Chen <[email protected]> Reviewed-by: David Zhang <[email protected]> Discussion: [https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org](https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org) [https://git.postgresql.org/pg/commitdiff/9f35f943732eaf9121a107f54ea043925e9a5d7b](https://git.postgresql.org/pg/commitdiff/9f35f943732eaf9121a107f54ea043925e9a5d7b) - Use truncate(2) where appropriate. When truncating files by name, use truncate(2). Windows hasn't got it, so keep our previous coding based on ftruncate(2) as a fallback. Discussion: [https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org](https://postgr.es/m/16663-fe97ccf9932fc800%40postgresql.org) [https://git.postgresql.org/pg/commitdiff/57faaf376e1961fa48866c6e5d6926463c6671b1](https://git.postgresql.org/pg/commitdiff/57faaf376e1961fa48866c6e5d6926463c6671b1) Bruce Momjian pushed: - pg_checksums: data_checksum_version is unsigned so use %u not %d. While the previous behavior didn't generate a warning, we might as well use an accurate *printf specification. Backpatch-through: 12 [https://git.postgresql.org/pg/commitdiff/888671a8cda5896d36d3ec523e201ab6a11e1855](https://git.postgresql.org/pg/commitdiff/888671a8cda5896d36d3ec523e201ab6a11e1855) - docs: list single-letter options first in command-line summary. In a few places, the long-version options were listed before the single-letter ones in the command summary of a few commands. This didn't match other commands, and didn't match the option ordering later in the same reference page. Backpatch-through: 9.5 [https://git.postgresql.org/pg/commitdiff/a659e789b743f1a4b977a1ffc7402ab618a9f68c](https://git.postgresql.org/pg/commitdiff/a659e789b743f1a4b977a1ffc7402ab618a9f68c) - doc: remove unnecessary blank before command option text. Backpatch-through: 11 [https://git.postgresql.org/pg/commitdiff/3f8971d92e767acf6e3d6e27c4cab7bfd88b71f4](https://git.postgresql.org/pg/commitdiff/3f8971d92e767acf6e3d6e27c4cab7bfd88b71f4) Stephen Frost pushed: - Add GSS information to connection authorized log message. GSS information (if used) such as if the connection was authorized using GSS or if it was encrypted using GSS, and perhaps most importantly, what the GSS principal used for the authentication was, is extremely useful but wasn't being included in the connection authorized log message. Therefore, add to the connection authorized log message that information, in a similar manner to how we log SSL information when SSL is used for a connection. Author: Vignesh C Reviewed-by: Bharath Rupireddy Discussion: [https://www.postgresql.org/message-id/CALDaNm2N1385_Ltoo%3DS7VGT-ESu_bRQa-sC1wg6ikrM2L2Z49w%40mail.gmail.com](https://www.postgresql.org/message-id/CALDaNm2N1385_Ltoo%3DS7VGT-ESu_bRQa-sC1wg6ikrM2L2Z49w%40mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/dc11f31a1a891f8aa8890644e837556bcc5a75e7](https://git.postgresql.org/pg/commitdiff/dc11f31a1a891f8aa8890644e837556bcc5a75e7) Dean Rasheed pushed: - Improve estimation of OR clauses using extended statistics. Formerly we only applied extended statistics to an OR clause as part of the clauselist_selectivity() code path for an OR clause appearing in an implicitly-ANDed list of clauses. This meant that it could only use extended statistics if all sub-clauses of the OR clause were covered by a single extended statistics object. Instead, teach clause_selectivity() how to apply extended statistics to an OR clause by handling its ORed list of sub-clauses in a similar manner to an implicitly-ANDed list of sub-clauses, but with different combination rules. This allows one or more extended statistics objects to be used to estimate all or part of the list of sub-clauses. Any remaining sub-clauses are then treated as if they are independent. Additionally, to avoid double-application of extended statistics, this introduces "extended" versions of clause_selectivity() and clauselist_selectivity(), which include an option to ignore extended statistics. This replaces the old clauselist_selectivity_simple() function which failed to completely ignore extended statistics when called from the extended statistics code. A known limitation of the current infrastructure is that an AND clause under an OR clause is not treated as compatible with extended statistics (because we don't build RestrictInfos for such sub-AND clauses). Thus, for example, "(a=1 AND b=1) OR (a=2 AND b=2)" will currently be treated as two independent AND clauses (each of which may be estimated using extended statistics), but extended statistics will not currently be used to account for any possible overlap between those clauses. Improving that is left as a task for the future. Original patch by Tomas Vondra, with additional improvements by me. Discussion: [https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development](https://postgr.es/m/20200113230008.g67iyk4cs3xbnjju@development) [https://git.postgresql.org/pg/commitdiff/25a9e54d2db31b8031c2d8166114b187e8347098](https://git.postgresql.org/pg/commitdiff/25a9e54d2db31b8031c2d8166114b187e8347098) Peter Eisentraut pushed: - Small code simplifications. strVal() can be used in a couple of places instead of coding the same thing by hand. [https://git.postgresql.org/pg/commitdiff/6114040711affa2b0bcf47fa2791187daf8455fb](https://git.postgresql.org/pg/commitdiff/6114040711affa2b0bcf47fa2791187daf8455fb) - Remove unnecessary grammar symbols. Instead of publication_name_list, we can use name_list. We already refer to publications everywhere else by the 'name' or 'name_list' symbols, so this only improves consistency. Reviewed-by: [https://www.postgresql.org/message-id/flat/3e3ccddb-41bd-ecd8-29fe-195e34d9886f%40enterprisedb.com](https://www.postgresql.org/message-id/flat/3e3ccddb-41bd-ecd8-29fe-195e34d9886f%40enterprisedb.com) Discussion: Tom Lane <[email protected]> [https://git.postgresql.org/pg/commitdiff/a6964bc1bb0793e20636ccb573cd2a5ad3ef7667](https://git.postgresql.org/pg/commitdiff/a6964bc1bb0793e20636ccb573cd2a5ad3ef7667) - Convert elog(LOG) calls to ereport() where appropriate. User-visible log messages should go through ereport(), so they are subject to translation. Many remaining elog(LOG) calls are really debugging calls. Reviewed-by: Alvaro Herrera <[email protected]> Reviewed-by: Michael Paquier <[email protected]> Reviewed-by: Noah Misch <[email protected]> Discussion: [https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com](https://www.postgresql.org/message-id/flat/92d6f545-5102-65d8-3c87-489f71ea0a37%40enterprisedb.com) [https://git.postgresql.org/pg/commitdiff/eb93f3a0b633ad6afb0f37391b87f460c4b0663b](https://git.postgresql.org/pg/commitdiff/eb93f3a0b633ad6afb0f37391b87f460c4b0663b) Amit Kapila pushed: - Remove incorrect assertion in reorderbuffer.c. We start recording changes in ReorderBufferTXN even before we reach SNAPBUILD_CONSISTENT state so that if the commit is encountered after reaching that we should be able to send the changes of the entire transaction. Now, while recording changes if the reorder buffer memory has exceeded logical_decoding_work_mem then we can start streaming if it is allowed and we haven't yet streamed that data. However, we must not allow streaming to start unless the snapshot has reached SNAPBUILD_CONSISTENT state. In passing, improve the comments atop ReorderBufferResetTXN to mention the case when we need to continue streaming after getting an error. Author: Amit Kapila Reviewed-by: Dilip Kumar Discussion: [https://postgr.es/m/caa4ek1kooh0byboyyy40nbcc7fe812trwta+wy3jqf7wqwz...@mail.gmail.com](https://postgr.es/m/caa4ek1kooh0byboyyy40nbcc7fe812trwta+wy3jqf7wqwz...@mail.gmail.com) [https://git.postgresql.org/pg/commitdiff/8ae4ef4fb0e0331f02c4615182600546c8e5c4d4](https://git.postgresql.org/pg/commitdiff/8ae4ef4fb0e0331f02c4615182600546c8e5c4d4) == Pending Patches == Tatsuro Yamada sent in another revision of a patch to add \dX (extended statistics) to psql. Andrey V. Lepikhov sent in another revision of a patch to remove unneeded self-joins. David G. Johnston sent in another revision of a patch to make some minor cleanups and rewording of plpgsql docs. Seino Yuki sent in another revision of a patch to add a new column, reset-time, to `pg_stat_statements_info`. Justin Pryzby sent in a patch to make \dt+ `pg_toast`.\* work in psql. Justin Pryzby sent in another revision of a patch to make EXPLAIN show JIT details in non-text format, even if zero. Peter Eisentraut sent in another revision of a patch to pause recovery for insufficient parameter settings. Michaël Paquier sent in another revision of a patch to fix a bug that manifested as `vac_update_datfrozenxid` will raise "wrong tuple length" if pg_database tuple contains toast attribute. Michaël Paquier sent in another revision of a patch to remove some references to long-obsolete behavior. Peter Geoghegan sent in another revision of a patch to teach heapam to support bottom-up index deletion, pass down "logically unchanged index" hint, and teach nbtree to use bottom-up index deletion. Bertrand Drouvot sent in two revisions of a patch to prevent functions from becoming orphaned by dropping their containing schemas. Andrey V. Lepikhov sent in a patch to fix a wild overestimation of the cost of a foreign join. Konstantin Knizhnik sent in another revision of a patch to implement custom compression for libpq. Álvaro Herrera sent in a patch to improve operations spelled like INDEX CONCURRENTLY. Peter Eisentraut sent in another revision of a patch to pageinspect to change the block number arguments to bigint, as this actually fits uint32. Aleksey Kondratov, Michaël Paquier, and Justin Pryzby traded patches to allow CLUSTER and VACUUM FULL to change tablespace on the fly. Bertrand Drouvot and Fujii Masao traded patches to log the standby recovery conflict waits. Justin Pryzby sent in two more revisions of a patch to make it possible for INSERT SELECT to use a BulkInsertState. Bharath Rupireddy sent in two more revisions of a patch to make it possible to use parallel inserts in CREATE TABLE AS. James Coleman sent in another revision of a patch to error if gather merge paths aren't sufficiently sorted. James Coleman sent in another revision of a patch to ensure that `generate_useful_gather_paths` doesn't skip unsorted subpaths, enforce parallel safety of pathkeys in `generate_useful_gather_paths`, disallow SRFs in proactive sort, remove volatile expr target search, and document `find_em_expr_usable_for_sorting_rel` in `prepare_sort_from_pathkeys`. Craig Ringer sent in a patch to implement a TAP test utility module `PG_LSN.pm` to test things that involve logical sequence numbers. Greg Nancarrow sent in another revision of a patch to add an `in_hot_standby` reportable GUC and make `transaction_read_only` GUC reportable, and enhance the connection parameter `target_session_attrs` to support new values: read-only/primary/standby/prefer-standby. Jürgen Purtz sent in another revision of a patch to put more explicit JOINs in the tutorial. Zhihong Yu sent in another revision of a patch to check nparts for defining index. Hou Zhejie sent in a patch to replace lcons and `list_delete_first` in `plan_union_children()`, with lappend and `list_delete_last`. Laurenz Albe sent in a patch to discard the query buffer if editor is quit in psql's \e. Andres Freund sent in a patch to fix a JIT problem that came up with LLVM 12. Fujii Masao sent in another revision of a patch to add basic statistics to the `pg_stat_wal view`.
