> On Sep 9, 2026, at 8:18 AM, Greg Burd <[email protected]> wrote: > > > On Tue, Sep 8, 2026, at 8:03 PM, Michael Paquier wrote: >> On Tue, Sep 08, 2026 at 01:28:22PM -0400, Greg Burd wrote: >>> ExecForceStoreHeapTuple() does not set slot->tts_tid when the target >>> slot is a TTS_IS_BUFFERTUPLE slot. Any plan that re-stores a heap tuple >>> through it and then projects ctid therefore gets (4294967295,0) instead >>> of the row's real heap TID. >> >> Oops. > > Yeah, oops indeed. :) > >> My question would be what kind of testing you have done to spot that.. > > Honestly, not by a test. I am working on a new index AM for vector > similarity search [1] adding a 1-bit encoding with rerank. To do that > rerank it sets xs_recheckorderby = true, because its distances are > quantised and only the executor's exact re-check can order the top-k > correctly. That routes tuples through nodeIndexscan.c's reorder queue. > One of our documented recipes harvests ctid from a scan and chains it > downstream; that quietly started matching nothing, with the ctids > coming back as (4294967295,0). > > My first assumption was that I'd broken something, so I went looking for > the boundary: > > - xs_heaptid was correct and every real column was right; only the > projected ctid was wrong, which pointed at the slot, not the AM. > - ExecForceStoreHeapTuple's TTS_IS_BUFFERTUPLE branch calls > ExecClearTuple (hence ItemPointerSetInvalid on tts_tid) and never > restores it, while the sibling tts_heap_store_tuple does; and > slot_getsysattr answers SelfItemPointerAttributeNumber straight out of > tts_tid. > - Then I reproduced it with core GiST and nothing else loaded, which is > the version in the patch. > > To confirm the fix I built stock 18.4 with only that hunk applied and > ran a test: > unpatched patched > ctid self-join, expect 5 1 5 > UPDATE ... WHERE ctid, expect 5 1 5 > sentinel ctids at LIMIT 50 49/50 0/50 > > The UPDATE line is the one that bothers me most. No error, it just > affects the wrong number of rows. > > As for why the tree doesn't catch it, AFAICT nothing in core projects > ctid from an ORDER BY-op index scan. The GiST kNN tests check ordering > and results, which are fine here the row data is never wrong. That's > what v1 adds a regress case for, and I checked it fails > (ctid_matches = 1) without the hunk and passes (5) with it, so it gates > the fix rather than just recording current output. > > One detail that probably explains the longevity, IndexNextWithReorder > only queues a tuple when the AM's advertised ORDER BY value doesn't > compare equal to the recomputed one. GiST's bounding-box distance is > sometimes exact, so some rows keep their real ctid. Note the 49 of 50 > above rather than 50. My index AM can't usefully bound its distance and > advertises -inf, so every tuple goes through the queue and every one > shows the sentinel. Partial in core, total out here. > >>> ExecClearTuple() reaches tts_buffer_heap_clear(), which does >>> ItemPointerSetInvalid(&slot->tts_tid). The tuple is then copied in, but >>> tts_tid is left invalid. The sibling path, ExecStoreHeapTuple() -> >>> tts_heap_store_tuple() — *does* slot->tts_tid = tuple->t_self, so this >>> reads as a plain asymmetry rather than an intentional choice. >> >> That's strange. Once thing that I can see why scanning this file is >> the same code pattern in tts_buffer_heap_copyslot(), where a slot is >> similarly cleared in a copy-paste fashion. > > Agreed. > >> Andres? >> -- >> Michael > > best, thanks for looking, > > -greg > > [1] https://codeberg.org/gregburd/pg_turbovec > https://github.com/gburd/pg_turbovec
As can happen it turns out I'm not the first to report this or propose a patch for it. [1] So I suggest we continue on that thread. Also, the other report found that the FOR UPDATE can extend the relation and leave a block that later breaks seqscans so essentially this bug can cause on-disk damage. best. -greg [1] https://postgr.es/m/CAM6Zo8wZOLnCWRO_tuuXVX9J4N4JN6GsEnk8WJtT0%3D_0zy-1dw%40mail.gmail.com
