Re: TerminateOtherDBBackends code comments inconsistency.

2024-05-16 Thread Noah Misch
On Tue, Apr 30, 2024 at 09:10:52AM +0530, Amit Kapila wrote: > On Tue, Apr 30, 2024 at 2:58 AM Noah Misch wrote: > > On Mon, Apr 29, 2024 at 10:18:35AM +0530, Amit Kapila wrote: > > > 3a9b18b309 didn't change the docs of pg_terminate_backend and whatever > > > is

Re: AIX support

2024-05-15 Thread Noah Misch
On Wed, May 15, 2024 at 03:33:25PM +, Sriram RK wrote: > Hi Team, we have any updated from the XLC team, the issue specific to the > alignment is fixed > and XLC had released it as part of 16.1.0.18. The PTF is available at the > below location, > > You can also find a link here: >

Re: race condition in pg_class

2024-05-13 Thread Noah Misch
On Mon, May 13, 2024 at 03:53:08PM -0400, Robert Haas wrote: > On Sun, May 12, 2024 at 7:29 PM Noah Misch wrote: > > - [consequences limited to transient failure] Since a PROC_IN_VACUUM > > backend's > > xmin does not stop pruning, an MVCC scan in that backend can find z

Re: race condition in pg_class

2024-05-13 Thread Noah Misch
On Mon, May 13, 2024 at 04:59:59PM +0900, Michael Paquier wrote: > About inplace050-tests-inj-v1.patch. > > + /* Check if blocked_pid is in injection_wait(). */ > + proc = BackendPidGetProc(blocked_pid); > + if (proc == NULL) > + PG_RETURN_BOOL(false); /* session gone:

Re: Weird test mixup

2024-05-12 Thread Noah Misch
On Fri, May 10, 2024 at 10:04:17AM +0900, Michael Paquier wrote: > Attached is an updated patch for now, indented with a happy CI. I am > still planning to look at that a second time on Monday with a fresher > mind, in case I'm missing something now. This looks correct, and it works well in my

Hot standby queries see transient all-zeros pages

2024-05-12 Thread Noah Misch
XLogReadBufferForRedoExtended() precedes RestoreBlockImage() with RBM_ZERO_AND_LOCK. Per src/backend/storage/buffer/README: Once one has determined that a tuple is interesting (visible to the current transaction) one may drop the content lock, yet continue to access the tuple's data for as

Re: Weird test mixup

2024-05-09 Thread Noah Misch
On Thu, May 09, 2024 at 04:40:43PM +0900, Michael Paquier wrote: > So I like your suggestion. This version closes the race window > between the shmem exit detach in backend A and a concurrent backend B > running a point local to A so as B will never run the local point of > A. However, it can

Re: Weird test mixup

2024-05-08 Thread Noah Misch
On Thu, May 09, 2024 at 09:37:54AM +0900, Michael Paquier wrote: > On Tue, May 07, 2024 at 11:53:10AM -0700, Noah Misch wrote: > > The problem I'm trying to tackle in this thread is to make > > src/test/modules/gin installcheck-safe. $SUBJECT's commit 5105c90 started > > t

Re: Weird test mixup

2024-05-07 Thread Noah Misch
On Tue, May 07, 2024 at 11:53:10AM -0700, Noah Misch wrote: > On Tue, May 07, 2024 at 10:17:49AM +0900, Michael Paquier wrote: > > Overall, this switches from one detach behavior to a different one, > > Can you say more about that? The only behavior change known to me is that a &g

Re: Weird test mixup

2024-05-07 Thread Noah Misch
On Tue, May 07, 2024 at 10:17:49AM +0900, Michael Paquier wrote: > On Mon, May 06, 2024 at 02:23:24PM -0700, Noah Misch wrote: > > Here's how I've patched it locally. It does avoid changing the > > backend-side, > > which has some attraction. Shall I just push this? >

Re: 2024-05-09 release announcement draft

2024-05-07 Thread Noah Misch
On Mon, May 06, 2024 at 11:09:24PM -0400, Jonathan S. Katz wrote: > * Avoid leaking a query result from > [`psql`](https://www.postgresql.org/docs/current/app-psql.html) > after the query is cancelled. I'd delete this item about a psql-lifespan memory leak, because (a) it's so minor and (b)

Re: Weird test mixup

2024-05-06 Thread Noah Misch
On Mon, May 06, 2024 at 10:03:37AM +0900, Michael Paquier wrote: > On Thu, May 02, 2024 at 12:35:55PM -0700, Noah Misch wrote: > > I should have given a simpler example: > > > > s1: local-attach to POINT > > s2: enter InjectionPointRun(POINT), yield CPU just bef

Re: Weird test mixup

2024-05-02 Thread Noah Misch
On Thu, May 02, 2024 at 04:27:12PM +0900, Michael Paquier wrote: > On Wed, May 01, 2024 at 04:12:14PM -0700, Noah Misch wrote: > > While writing an injection point test, I encountered a variant of the race > > condition that f4083c4 fixed. It had three sessions and this sequenc

Re: Weird test mixup

2024-05-01 Thread Noah Misch
While writing an injection point test, I encountered a variant of the race condition that f4083c4 fixed. It had three sessions and this sequence of events: s1: local-attach to POINT s2: enter InjectionPointRun(POINT), yield CPU just before injection_callback() s3: detach POINT, deleting the

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-04-30 Thread Noah Misch
On Thu, Apr 25, 2024 at 04:59:54PM +0400, Pavel Borisov wrote: > 0001: Optimize speed by avoiding heap visibility checking for different > non-deduplicated index tuples as proposed by Noah Misch > > Speed measurements on my laptop using the exact method recommended by Noah > upth

Re: TerminateOtherDBBackends code comments inconsistency.

2024-04-30 Thread Noah Misch
On Tue, Apr 30, 2024 at 09:10:52AM +0530, Amit Kapila wrote: > On Tue, Apr 30, 2024 at 2:58 AM Noah Misch wrote: > > On Mon, Apr 29, 2024 at 10:18:35AM +0530, Amit Kapila wrote: > > > On Mon, Apr 22, 2024 at 9:56 PM Noah Misch wrote: > > > > >

Re: TerminateOtherDBBackends code comments inconsistency.

2024-04-29 Thread Noah Misch
On Mon, Apr 29, 2024 at 10:18:35AM +0530, Amit Kapila wrote: > On Mon, Apr 22, 2024 at 9:56 PM Noah Misch wrote: > > > > On Mon, Apr 15, 2024 at 11:17:54AM +0530, Amit Kapila wrote: > > > On Thu, Apr 11, 2024 at 6:58 PM Kirill Reshke > > > wrote: > >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-24 Thread Noah Misch
t have code that replaces an existing value > with a value of a different length. I saw a SIGSEGV there when using tidstore to write a fix for something else. Patch attached. Author: Noah Misch Commit: Noah Misch radixtree: Fix SIGSEGV at update of embeddable value to non-embedd

datfrozen/relfrozen update race condition

2024-04-22 Thread Noah Misch
mprove the chance of future copy/paste finding the safe style. I'm attaching a patch for that, too. I didn't add "volatile", because I couldn't think of how we'd care if the load moved earlier. Author: Noah Misch Commit: Noah Misch Test datfrozenxid/relfrozenxid update r

Re: TerminateOtherDBBackends code comments inconsistency.

2024-04-22 Thread Noah Misch
t I wonder if we should > add a check similar to 3a9b18b [1] for the reason given in the commit > message. I have added Noah to see if he has any suggestions on this > matter. > > [1] - > commit 3a9b18b3095366cd0c4305441d426d04572d88c1 > Author: Noah Misch > Date: Mon

Re: DROP OWNED BY fails to clean out pg_init_privs grants

2024-04-05 Thread Noah Misch
On Fri, Apr 05, 2024 at 07:10:59PM -0400, Tom Lane wrote: > I wondered why buildfarm member copperhead has started to fail > xversion-upgrade-HEAD-HEAD tests. I soon reproduced the problem here: > > pg_restore: creating ACL "regress_pg_dump_schema.TYPE "test_type"" > pg_restore: while PROCESSING

Re: AIX support

2024-04-05 Thread Noah Misch
On Fri, Apr 05, 2024 at 04:12:06PM +, Sriram RK wrote: > > > What you do need to do to reproduce the described problems is > > check out the Postgres git tree and rewind to just before > > commit 0b16bb877, where we deleted AIX support. Any attempt > > to restore AIX support would have to

Re: Query execution in Perl TAP tests needs work

2024-03-30 Thread Noah Misch
On Wed, Aug 30, 2023 at 09:48:42AM +1200, Thomas Munro wrote: > On Wed, Aug 30, 2023 at 1:49 AM Noah Misch wrote: > > On Tue, Aug 29, 2023 at 04:25:24PM +1200, Thomas Munro wrote: > > > On Tue, Aug 29, 2023 at 1:48 PM Noah Misch wrote: > > > > https://github.com/c

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-03-29 Thread Noah Misch
On Fri, Mar 29, 2024 at 02:17:08PM -0400, Peter Geoghegan wrote: > On Mon, Mar 25, 2024 at 2:24 PM Noah Misch wrote: > > I wasn't thinking about changing the pre-v17 bt_right_page_check_scankey() > > code. I got interested in this area when I saw the interaction of the new

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-29 Thread Noah Misch
On Fri, Mar 29, 2024 at 09:17:55AM +0100, Jelte Fennema-Nio wrote: > On Thu, 28 Mar 2024 at 19:03, Tom Lane wrote: > > Could we make this test bulletproof by using an injection point? > > If not, I remain of the opinion that we're better off without it. > > Possibly, and if so, I agree that

Re: AIX support

2024-03-28 Thread Noah Misch
On Thu, Mar 28, 2024 at 11:09:43AM +, Sriram RK wrote: > We are setting up the build environment and trying to build the source and > also trying to analyze the assert from the Aix point of view. The thread Alvaro and Tom cited contains an analysis. It's a compiler bug. You can get past the

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-03-25 Thread Noah Misch
On Mon, Mar 25, 2024 at 12:03:10PM -0400, Peter Geoghegan wrote: > On Sun, Mar 24, 2024 at 10:03 PM Noah Misch wrote: > Separately, I now see that the committed patch just reuses the code > that has long been used to check that things are in the correct order > across pag

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2024-03-24 Thread Noah Misch
On Mon, Oct 30, 2023 at 11:29:04AM +0400, Pavel Borisov wrote: > On Wed, 25 Oct 2023 at 00:13, Alexander Korotkov wrote: > > I think this patch is ready to go. I'm going to push it if there are > > no objections. > It's very good that this long-standing patch is finally committed. Thanks a >

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-20 Thread Noah Misch
On Mon, Mar 18, 2024 at 07:40:10PM +0100, Alvaro Herrera wrote: > I enabled the test again and also pushed the changes to dblink, > isolationtester and fe_utils (which AFAICS is used by pg_dump, I recommend adding a libpqsrv_cancel() function to libpq-be-fe-helpers.h, to use from dblink and

Re: Autogenerate some wait events code and documentation

2024-03-18 Thread Noah Misch
On Mon, Mar 18, 2024 at 09:04:44AM +, Bertrand Drouvot wrote: > --- a/src/backend/utils/activity/wait_event_names.txt > +++ b/src/backend/utils/activity/wait_event_names.txt > @@ -24,7 +24,12 @@ > # SGML tables of wait events for inclusion in the documentation. > # > # When adding a

Re: Autogenerate some wait events code and documentation

2024-03-17 Thread Noah Misch
On Mon, Mar 18, 2024 at 08:02:24AM +0900, Michael Paquier wrote: > > 1. Don't back-patch wait events to v17+. Use the closest existing event. > > 2. Let wait_event_names.txt back-patches control the enum order. For > > example, > >a line could have an annotation that controls its position

Re: Autogenerate some wait events code and documentation

2024-03-17 Thread Noah Misch
On Wed, Jul 05, 2023 at 10:57:19AM +0900, Michael Paquier wrote: > I have applied it. I like the new developer experience of adding a wait event. After release of v17, how should we approach back-patching an event, like was done in commits 8fa4a1a 1396b5c 78c0f85? Each of those commits put the

Re: 035_standby_logical_decoding unbounded hang

2024-03-07 Thread Noah Misch
On Thu, Mar 07, 2024 at 02:46:55PM +0500, Andrey M. Borodin wrote: > I’m not sure if it is connected, but so far many patches in CFbot keep > hanging in this test. For example [0]. > [0] https://cirrus-ci.com/task/5911176840740864?logs=check_world#L292 Relevant part: [22:03:10.292] stderr:

Re: Relation bulk write facility

2024-02-27 Thread Noah Misch
On Wed, Feb 28, 2024 at 12:24:01AM +0400, Heikki Linnakangas wrote: > Here's a patch to fully remove AIX support. > Subject: [PATCH 1/1] Remove AIX support > > There isn't a lot of user demand for AIX support, no one has stepped > up to the plate to properly maintain it, so it's best to remove

Re: Relation bulk write facility

2024-02-25 Thread Noah Misch
On Sun, Feb 25, 2024 at 04:34:47PM +0200, Heikki Linnakangas wrote: > I agree with Andres though, that unless someone raises their hand and > volunteers to properly maintain the AIX support, we should drop it. There's no way forward in which AIX support stops doing net harm. Even if AIX

Re: Relation bulk write facility

2024-02-24 Thread Noah Misch
On Sun, Feb 25, 2024 at 09:13:47AM +1300, Thomas Munro wrote: > On Sun, Feb 25, 2024 at 9:12 AM Thomas Munro wrote: > > On Sun, Feb 25, 2024 at 8:50 AM Noah Misch wrote: > > > On GNU/Linux x64, gcc correctly records alignment=2**12 for the associated > > > sectio

Re: Relation bulk write facility

2024-02-24 Thread Noah Misch
On Sun, Feb 25, 2024 at 07:52:16AM +1300, Thomas Munro wrote: > On Sun, Feb 25, 2024 at 6:24 AM Noah Misch wrote: > > On Fri, Feb 23, 2024 at 04:27:34PM +0200, Heikki Linnakangas wrote: > > > Committed this. Thanks everyone! > > > > https://buildfarm.postgresql.org/c

Re: Relation bulk write facility

2024-02-24 Thread Noah Misch
On Fri, Feb 23, 2024 at 04:27:34PM +0200, Heikki Linnakangas wrote: > Committed this. Thanks everyone! https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill=2024-02-24%2015%3A13%3A14 got: TRAP: failed Assert("(uintptr_t) buffer == TYPEALIGN(PG_IO_ALIGN_SIZE, buffer)"), File: "md.c",

Re: Removing unneeded self joins

2024-02-24 Thread Noah Misch
Hello, On Sat, Feb 24, 2024 at 01:02:01PM +0200, Alexander Korotkov wrote: > On Sat, Feb 24, 2024 at 7:12 AM Noah Misch wrote: > > On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > > > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > > > wrote:

Re: incremental backup mishandles XLOG_DBASE_CREATE_FILE_COPY

2024-02-24 Thread Noah Misch
On Sat, Feb 24, 2024 at 04:16:24PM +0530, Robert Haas wrote: > On Sat, Feb 24, 2024 at 10:05 AM Noah Misch wrote: > > On Fri, Feb 23, 2024 at 08:47:52PM +0530, Robert Haas wrote: > > > I thought about whether there were any other WAL records that have >

Re: Removing unneeded self joins

2024-02-23 Thread Noah Misch
On Sat, Feb 24, 2024 at 12:36:59AM +0200, Alexander Korotkov wrote: > On Thu, Feb 22, 2024 at 10:51 AM Andrei Lepikhov > wrote: > > On 21/2/2024 14:26, Richard Guo wrote: > > > I think the right fix for these issues is to introduce a new element > > > 'sublevels_up' in ReplaceVarnoContext, and

Re: incremental backup mishandles XLOG_DBASE_CREATE_FILE_COPY

2024-02-23 Thread Noah Misch
On Fri, Feb 23, 2024 at 08:47:52PM +0530, Robert Haas wrote: > If XLOG_DBASE_CREATE_FILE_COPY occurs between an incremental backup > and its reference backup, every relation whose DB OID and tablespace > OID match the corresponding values in that record should be backed up > in full. Currently

Re: 035_standby_logical_decoding unbounded hang

2024-02-19 Thread Noah Misch
On Fri, Feb 16, 2024 at 06:37:38AM +, Bertrand Drouvot wrote: > On Thu, Feb 15, 2024 at 12:48:16PM -0800, Noah Misch wrote: > > On Wed, Feb 14, 2024 at 03:31:16PM +, Bertrand Drouvot wrote: > > > What about creating a sub, say wait_for_restart_lsn_calculation() in

Re: 035_standby_logical_decoding unbounded hang

2024-02-15 Thread Noah Misch
On Wed, Feb 14, 2024 at 03:31:16PM +, Bertrand Drouvot wrote: > On Sat, Feb 10, 2024 at 05:02:27PM -0800, Noah Misch wrote: > > The 035_standby_logical_decoding.pl hang is > > a race condition arising from an event sequence like this: > > > > - Test scrip

Re: common signal handler protection

2024-02-14 Thread Noah Misch
On Mon, Dec 18, 2023 at 11:32:47AM -0600, Nathan Bossart wrote: > rebased for cfbot I took a look over each of these. +1 for all. Thank you.

035_standby_logical_decoding unbounded hang

2024-02-10 Thread Noah Misch
Coincidentally, one of my buildfarm animals hanged several weeks in a different test, 035_standby_logical_decoding.pl. A LOG_SNAPSHOT_INTERVAL_MS reduction was part of making it reproducible: On Fri, Feb 02, 2024 at 04:01:45PM -0800, Noah Misch wrote: > On Fri, Feb 02, 2024 at 02:30:03PM -0

Re: Popcount optimization using AVX512

2024-02-10 Thread Noah Misch
On Fri, Feb 09, 2024 at 08:33:23PM -0800, Andres Freund wrote: > On 2024-02-09 15:27:57 -0800, Noah Misch wrote: > > On Fri, Feb 09, 2024 at 10:24:32AM -0800, Andres Freund wrote: > > > On 2024-01-26 07:42:33 +0100, Alvaro Herrera wrote: > > > > This suggests that f

Re: Popcount optimization using AVX512

2024-02-09 Thread Noah Misch
On Fri, Feb 09, 2024 at 10:24:32AM -0800, Andres Freund wrote: > On 2024-01-26 07:42:33 +0100, Alvaro Herrera wrote: > > This suggests that finding a way to make the ifunc stuff work (with good > > performance) is critical to this work. > > Ifuncs are effectively implemented as a function call

Re: Should we remove -Wdeclaration-after-statement?

2024-02-07 Thread Noah Misch
On Mon, Jan 29, 2024 at 04:03:44PM +0100, Jelte Fennema-Nio wrote: > I feel like this is the type of change where there's not much > discussion to be had. And the only way to resolve it is to use some > voting to gauge community opinion. > > So my suggestion is for people to respond with -1,

Re: Draft release notes for minor releases are up

2024-02-04 Thread Noah Misch
On Sun, Feb 04, 2024 at 01:13:53PM -0500, Tom Lane wrote: > I now have this text for your CREATE DATABASE fixes: > > > Ensure durability of CREATE DATABASE (Noah Misch) > > > > If an operating system crash occurred during or shortly >

Re: Draft release notes for minor releases are up

2024-02-02 Thread Noah Misch
On Fri, Feb 02, 2024 at 08:18:50PM -0500, Tom Lane wrote: > Noah Misch writes: > > Shall the top of the notes advise to reindex GIN indexes? > > I thought about that, but it's a pretty low-probability failure > I think, so I didn't write that advice. Maybe I misjudged it. I c

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-02 Thread Noah Misch
On Fri, Feb 02, 2024 at 02:30:03PM -0800, Noah Misch wrote: > On Fri, Feb 02, 2024 at 05:07:14PM -0500, Tom Lane wrote: > > If you look at the buildfarm's failures page and filter down to > > just subscriptionCheck failures, what you find is that all of the > >

Re: Draft release notes for minor releases are up

2024-02-02 Thread Noah Misch
rocesses > + attempted the cleanup concurrently. > + > + Shall the top of the notes advise to reindex GIN indexes? > + > + > + > + Add more interlocks between CREATE DATABASE and > + base backup (Noah Misch) > + > + > +

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-02 Thread Noah Misch
On Fri, Feb 02, 2024 at 05:07:14PM -0500, Tom Lane wrote: > If you look at the buildfarm's failures page and filter down to > just subscriptionCheck failures, what you find is that all of the > last 6 such failures are in 031_column_list.pl: > >

Re: dblink query interruptibility

2024-01-26 Thread Noah Misch
On Thu, Jan 25, 2024 at 12:28:43PM +0900, Fujii Masao wrote: > On Thu, Jan 25, 2024 at 5:45 AM Noah Misch wrote: > > On Thu, Jan 25, 2024 at 04:23:39AM +0900, Fujii Masao wrote: > > > I found that this patch was committed at d3c5f37dd5 and changed the > > > error messa

Re: dblink query interruptibility

2024-01-24 Thread Noah Misch
On Thu, Jan 25, 2024 at 04:23:39AM +0900, Fujii Masao wrote: > I found that this patch was committed at d3c5f37dd5 and changed the > error message in postgres_fdw slightly. Here's an example: > > #1. Begin a new transaction. > #2. Execute a query accessing to a foreign table, like SELECT * FROM >

Re: Recovering from detoast-related catcache invalidations

2024-01-14 Thread Noah Misch
On Fri, Jan 12, 2024 at 03:47:13PM -0500, Tom Lane wrote: > I wrote: > > This is uncomfortably much in bed with the tuple table slot code, > > perhaps, but I don't see a way to do it more cleanly unless we want > > to add some new provisions to that API. Andres, do you have any > > thoughts about

Re: broken master regress tests

2023-12-12 Thread Noah Misch
On Tue, Oct 10, 2023 at 05:54:34PM -0700, Andres Freund wrote: > On 2023-10-10 17:08:25 -0700, Jeff Davis wrote: > > After this, it seems "make check" no longer picks up the locale from > > the system environment by default. > > Yea. I wonder if the better fix would have been to copy

Re: Is WAL_DEBUG related code still relevant today?

2023-12-07 Thread Noah Misch
On Thu, Dec 07, 2023 at 04:50:30PM +0530, Bharath Rupireddy wrote: > On Mon, Dec 4, 2023 at 12:37 AM Noah Misch wrote: > > On Sun, Dec 03, 2023 at 08:30:24PM +0530, Bharath Rupireddy wrote: > > > On Sun, Dec 3, 2023 at 4:00 AM Nathan Bossart > > > wrote: > >

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

2023-12-06 Thread Noah Misch
On Wed, Dec 06, 2023 at 03:17:12PM +0900, Michael Paquier wrote: > On Sat, Nov 18, 2023 at 04:32:36PM -0800, Noah Misch wrote: > > On Sat, Nov 18, 2023 at 03:09:58PM -0800, Andres Freund wrote: > >> Unfortunately, there is a case of such an sqlstate that's not at al

Re: Is WAL_DEBUG related code still relevant today?

2023-12-03 Thread Noah Misch
On Sun, Dec 03, 2023 at 08:30:24PM +0530, Bharath Rupireddy wrote: > On Sun, Dec 3, 2023 at 4:00 AM Nathan Bossart > wrote: > > On Sat, Dec 02, 2023 at 07:36:29PM +0530, Bharath Rupireddy wrote: > > > b) Remove both the WAL_DEBUG macro and the wal_debug GUC. I don't > > > think (2) is needed to

dblink query interruptibility

2023-11-21 Thread Noah Misch
est suite checks that. Hence, I standardized on just inner_query. I wrote this because pglogical needs these functions to cooperate with v15+ DROP DATABASE (https://github.com/2ndQuadrant/pglogical/issues/418). Thanks, nm Author: Noah Misch Commit: Noah Misch Make dblink interruptible, via n

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

2023-11-18 Thread Noah Misch
On Sat, Nov 18, 2023 at 03:09:58PM -0800, Andres Freund wrote: > We currently provide no way to learn about a postgres instance having > corruption than searching the logs for corruption events than matching by > sqlstate, for ERRCODE_DATA_CORRUPTED and ERRCODE_INDEX_CORRUPTED. > > Unfortunately,

Re: 2023-11-09 release announcement draft

2023-11-07 Thread Noah Misch
On Tue, Nov 07, 2023 at 09:02:03AM -0500, Jonathan S. Katz wrote: > On 11/6/23 9:52 PM, Noah Misch wrote: > > On Mon, Nov 06, 2023 at 05:04:25PM -0500, Jonathan S. Katz wrote: > > Delete lines starting here ... > > > > > This is the **final release of

Re: Popcount optimization using AVX512

2023-11-06 Thread Noah Misch
On Mon, Nov 06, 2023 at 09:59:26PM -0600, Nathan Bossart wrote: > On Mon, Nov 06, 2023 at 07:15:01PM -0800, Noah Misch wrote: > > On Mon, Nov 06, 2023 at 09:52:58PM -0500, Tom Lane wrote: > >> Nathan Bossart writes: > >> > Like I said, I don't have any proposal

Re: Popcount optimization using AVX512

2023-11-06 Thread Noah Misch
On Mon, Nov 06, 2023 at 09:52:58PM -0500, Tom Lane wrote: > Nathan Bossart writes: > > Like I said, I don't have any proposals yet, but assuming we do want to > > support newer intrinsics, either open-coded or via auto-vectorization, I > > suspect we'll need to gather consensus for a new

Re: 2023-11-09 release announcement draft

2023-11-06 Thread Noah Misch
On Mon, Nov 06, 2023 at 05:04:25PM -0500, Jonathan S. Katz wrote: > The PostgreSQL Global Development Group has released an update to all > supported > versions of PostgreSQL, including 16.1, 15.5, 14.10, 13.13, 12.17, and 11.22 > This release fixes over 55 bugs reported over the last several

Re: Tab completion regression test failed on illumos

2023-11-01 Thread Noah Misch
On Wed, Nov 01, 2023 at 03:19:39PM +0800, Japin Li wrote: > I try to run regression test on illumos, the 010_tab_completion will > failed because of timeout. > Any suggestions? Thanks in advance! This test failed for me, in a different way, when I briefly installed IO::Pty on a Solaris

Re: race condition in pg_class

2023-11-01 Thread Noah Misch
I prototyped two ways, one with a special t_ctid and one with LockTuple(). On Fri, Oct 27, 2023 at 04:26:12PM -0700, Noah Misch wrote: > On Fri, Oct 27, 2023 at 06:40:55PM -0400, Tom Lane wrote: > > Noah Misch writes: > > > On Fri, Oct 27, 2023 at 03:32:26PM -0400, Tom La

Re: race condition in pg_class

2023-10-27 Thread Noah Misch
On Fri, Oct 27, 2023 at 06:40:55PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Fri, Oct 27, 2023 at 03:32:26PM -0400, Tom Lane wrote: > >> I'm inclined to propose that heap_inplace_update should check to > >> make sure that it's operating on the latest version o

Re: race condition in pg_class

2023-10-27 Thread Noah Misch
On Fri, Oct 27, 2023 at 03:32:26PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Thu, Oct 26, 2023 at 09:44:04PM -0700, Noah Misch wrote: > >> We'll likely need to change how we maintain relhasindex or perhaps take a > >> lock > >> in GRANT. > &g

Re: race condition in pg_class

2023-10-27 Thread Noah Misch
On Thu, Oct 26, 2023 at 09:44:04PM -0700, Noah Misch wrote: > On Wed, Oct 25, 2023 at 01:39:41PM +0300, Smolkin Grigory wrote: > > We are running PG13.10 and recently we have encountered what appears to be > > a bug due to some race condition between ALTER TABLE ... ADD CONSTRAI

Re: race condition in pg_class

2023-10-26 Thread Noah Misch
On Wed, Oct 25, 2023 at 01:39:41PM +0300, Smolkin Grigory wrote: > We are running PG13.10 and recently we have encountered what appears to be > a bug due to some race condition between ALTER TABLE ... ADD CONSTRAINT and > some other catalog-writer, possibly ANALYZE. > The problem is that after

Re: post-recovery amcheck expectations

2023-10-24 Thread Noah Misch
On Tue, Oct 24, 2023 at 07:03:34PM -0700, Peter Geoghegan wrote: > On Mon, Oct 23, 2023 at 7:28 PM Noah Misch wrote: > > > That makes sense to me. I believe that it's not possible to have a > > > string of consecutive sibling pages that are all half-dead (regardless > >

Re: post-recovery amcheck expectations

2023-10-23 Thread Noah Misch
On Mon, Oct 23, 2023 at 04:46:23PM -0700, Peter Geoghegan wrote: > On Fri, Oct 20, 2023 at 8:55 PM Noah Misch wrote: > > > > I lean toward fixing this by > > > > having amcheck scan left; if left links reach only half-dead or deleted > > > > pages, >

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-23 Thread Noah Misch
On Mon, Oct 16, 2023 at 11:21:20PM -0700, Donghang Lin wrote: > > I've also caught btree posting lists where one TID refers to a '1d' heap > > tuple, while another TID refers to a '24h' heap tuple. amcheck complains. > Index-only scans can return the '1d' bits where the actual tuple had the >

Re: pgstatindex vs. !indisready

2023-10-22 Thread Noah Misch
On Wed, Oct 04, 2023 at 09:00:23AM +0900, Michael Paquier wrote: > On Sun, Oct 01, 2023 at 06:31:26PM -0700, Noah Misch wrote: > > The !indisvalid index may be missing tuples, yes. In what way does that > > make > > one of those four operations incorrect? > > Remindin

Re: post-recovery amcheck expectations

2023-10-20 Thread Noah Misch
On Mon, Oct 09, 2023 at 04:46:26PM -0700, Peter Geoghegan wrote: > On Wed, Oct 4, 2023 at 7:52 PM Noah Misch wrote: > Might make sense to test the fix for this issue using a similar > approach: by adding custom code that randomly throws errors at a point > that stresses the impleme

Re: Remove last traces of HPPA support

2023-10-20 Thread Noah Misch
On Fri, Oct 20, 2023 at 12:40:00PM -0700, Andres Freund wrote: > On 2023-10-19 17:23:04 -0700, Noah Misch wrote: > > On Thu, Oct 19, 2023 at 11:16:28AM -0400, Tom Lane wrote: > > > We removed support for the HP-UX OS in v16, but left in support > > > for the PA-RISC arc

Re: Remove last traces of HPPA support

2023-10-19 Thread Noah Misch
On Thu, Oct 19, 2023 at 11:16:28AM -0400, Tom Lane wrote: > We removed support for the HP-UX OS in v16, but left in support > for the PA-RISC architecture, mainly because I thought that its > spinlock mechanism is weird enough to be a good stress test > for our spinlock infrastructure. It still

Re: Add support for AT LOCAL

2023-10-18 Thread Noah Misch
On Wed, Oct 18, 2023 at 04:45:46PM -0400, Tom Lane wrote: > > On Wed, Oct 18, 2023 at 12:02 PM Tom Lane wrote: > >> Probably. Independent of that, it's fair to ask why we're still > >> testing against xlc 12.1 and not the considerably-more-recent xlclang, > >> or at least xlc 16.1. (I also

Re: Add support for AT LOCAL

2023-10-16 Thread Noah Misch
On Mon, Oct 16, 2023 at 01:54:23AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Sun, Oct 15, 2023 at 09:58:04PM -0700, Noah Misch wrote: > >> Works for me. I've started a test run with the xlc version change. > > > It failed similarly: > > > + 23:59

Re: Add support for AT LOCAL

2023-10-15 Thread Noah Misch
On Sun, Oct 15, 2023 at 09:58:04PM -0700, Noah Misch wrote: > On Sun, Oct 15, 2023 at 11:30:17PM -0400, Tom Lane wrote: > > Thomas Munro writes: > > > On Mon, Oct 16, 2023 at 4:02 PM Tom Lane wrote: > > >> I'm having a hard time not believing that this is a com

Re: Add support for AT LOCAL

2023-10-15 Thread Noah Misch
On Sun, Oct 15, 2023 at 11:30:17PM -0400, Tom Lane wrote: > Thomas Munro writes: > > On Mon, Oct 16, 2023 at 4:02 PM Tom Lane wrote: > >> I'm having a hard time not believing that this is a compiler bug. > >> Looking back at 8d2a01ae12cd and its speculation that xlc is overly > >> liberal about

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-12 Thread Noah Misch
On Wed, Oct 11, 2023 at 01:00:44PM -0700, Peter Geoghegan wrote: > On Wed, Oct 11, 2023 at 11:38 AM Noah Misch wrote: > > Interesting. So, >99% of interval-type indexes, even ones WITH > > (deduplicate_items=off), will get amcheck failures. The <1% of exceptions > > m

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-11 Thread Noah Misch
On Tue, Oct 10, 2023 at 09:35:45PM -0700, Peter Geoghegan wrote: > On Tue, Oct 10, 2023 at 8:51 PM Peter Geoghegan wrote: > > I don't see any reason to delay committing your fix. The issue that > > you've highlighted is exactly the kind of issue that I anticipated > > might happen at some point.

Re: REL_15_STABLE: pgbench tests randomly failing on CI, Windows only

2023-10-10 Thread Noah Misch
On Sun, Oct 08, 2023 at 10:00:03PM -0700, David G. Johnston wrote: > On Sun, Oct 8, 2023 at 9:10 PM Noah Misch wrote: > > I didn't think of any phrasing that clearly explained things without the > > reader consulting the code. I considered these: I'm leaning toward: "so

Re: interval_ops shall stop using btequalimage (deduplication)

2023-10-10 Thread Noah Misch
On Tue, Oct 10, 2023 at 08:12:36PM -0700, Peter Geoghegan wrote: > On Tue, Oct 10, 2023 at 6:33 PM Noah Misch wrote: > > interval_ops, however, recognizes equal-but-distinguishable values: > > > Fails with: > > > > 2498151 2023-10-10 05:06:46.177 GMT DEBUG:

interval_ops shall stop using btequalimage (deduplication)

2023-10-10 Thread Noah Misch
─ amproc│ btvarstrequalimage l_eq_r│ t array_agg │ {{bpchar_ops,character},{text_ops,text},{text_ops,name}} Thanks, nm Author: Noah Misch Commit: Noah Misch Dissociate btequalimage() from interval

Re: REL_15_STABLE: pgbench tests randomly failing on CI, Windows only

2023-10-08 Thread Noah Misch
On Mon, Oct 09, 2023 at 04:22:52PM +1300, Thomas Munro wrote: > On Mon, Oct 9, 2023 at 3:25 PM Noah Misch wrote: > > The "fd >= FD_SETSIZE" check is irrelevant on Windows. See comments in the > > attached patch; in brief, Windows assigns FDs and uses FD_SETSIZE &g

Re: REL_15_STABLE: pgbench tests randomly failing on CI, Windows only

2023-10-08 Thread Noah Misch
or 2040. Thanks, nm P.S. Later, we should change test code so the pgbench stderr can't grow an extra line without that line appearing in test logs. Author: Noah Misch Commit: Noah Misch Don't spuriously report FD_SETSIZE exhaustion on Windows. Starting on 2023-08-03, this inte

Re: Trigger violates foreign key constraint

2023-10-08 Thread Noah Misch
On Mon, Oct 02, 2023 at 09:49:53AM -0400, Tom Lane wrote: > Laurenz Albe writes: > > CREATE FUNCTION silly() RETURNS trigger LANGUAGE plpgsql AS 'BEGIN RETURN > > NULL; END;'; > > CREATE TRIGGER silly BEFORE DELETE ON child FOR EACH ROW EXECUTE FUNCTION > > silly(); > > > The trigger function

Re: Unlogged relation copy is not fsync'd

2023-10-07 Thread Noah Misch
On Wed, Sep 20, 2023 at 11:22:10PM -0700, Noah Misch wrote: > On Fri, Sep 15, 2023 at 02:47:45PM +0300, Heikki Linnakangas wrote: > > On 05/09/2023 21:20, Robert Haas wrote: > > > Thinking about this some more, I think this is still not 100% correct, even > > with th

post-recovery amcheck expectations

2023-10-04 Thread Noah Misch
haps recovery can trick them the same way. Examples: errmsg("downlink or sibling link points to deleted block in index \"%s\"", errmsg("block %u is not leftmost in index \"%s\"", errmsg("block %u is not true root in index \"%s\"",

Re: pgstatindex vs. !indisready

2023-10-01 Thread Noah Misch
On Mon, Oct 02, 2023 at 09:24:33AM +0900, Michael Paquier wrote: > On Sun, Oct 01, 2023 at 04:20:42PM -0700, Peter Geoghegan wrote: > > On Sun, Oct 1, 2023 at 2:00 PM Noah Misch wrote: > >> [brin_desummarize_range brin_summarize_new_values brin_summarize_range > >

Re: pgstatindex vs. !indisready

2023-10-01 Thread Noah Misch
On Sun, Oct 01, 2023 at 04:37:25PM -0400, Tom Lane wrote: > Noah Misch writes: > > Running pgstatindex on some !indisready indexes fails with "ERROR: XX001: > > could not read block 0 in file". This reproduces it: > > ... > > Since XX001 = ERRCODE_DATA_C

pgstatindex vs. !indisready

2023-10-01 Thread Noah Misch
-XX messages: 2023-10-01 12:24:05.992 PDT [646914:11] 001_stream_rep.pl ERROR: 58P01: could not open file "base/5/16862": No such file or directory 2023-10-01 12:24:05.996 PDT [646914:118] 001_stream_rep.pl ERROR: 22023: fork "main" does not exist for this relation Thanks, n

Re: Testing autovacuum wraparound (including failsafe)

2023-09-29 Thread Noah Misch
On Fri, Sep 29, 2023 at 12:17:04PM +0200, Daniel Gustafsson wrote: > +# Bump the query timeout to avoid false negatives on slow test systems. > +my $psql_timeout_secs = 4 * $PostgreSQL::Test::Utils::timeout_default; > > Should we bump the timeout like this for all systems? I interpreted Noah's >

Re: Fail hard if xlogreader.c fails on out-of-memory

2023-09-26 Thread Noah Misch
On Wed, Sep 27, 2023 at 11:06:37AM +1300, Thomas Munro wrote: > On Tue, Sep 26, 2023 at 8:38 PM Michael Paquier wrote: > > Thoughts and/or comments are welcome. > > I don't have an opinion yet on your other thread about making this > stuff configurable for replicas, but for the simple crash

Re: Unlogged relation copy is not fsync'd

2023-09-21 Thread Noah Misch
On Fri, Sep 15, 2023 at 02:47:45PM +0300, Heikki Linnakangas wrote: > On 05/09/2023 21:20, Robert Haas wrote: > Thinking about this some more, I think this is still not 100% correct, even > with the patch I posted earlier: > > > /* > > * When we WAL-logged rel pages, we must nonetheless

  1   2   3   4   5   6   7   8   9   >