pgsql: Remove unused check in heap_xlog_insert()

2025-06-26 Thread Melanie Plageman
, as of now, we do not support any freezing of pages in the heap_insert() path. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/flat/CAAKRu_Zp4Pi-t51OFWm1YZ-cctDfBhHCMZ%3DEx6PKxv0o8y2GvA%40mail.gmail.com Backpatch-through: 14 Branch -- master Details

pgsql: Remove unused check in heap_xlog_insert()

2025-06-26 Thread Melanie Plageman
, as of now, we do not support any freezing of pages in the heap_insert() path. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/flat/CAAKRu_Zp4Pi-t51OFWm1YZ-cctDfBhHCMZ%3DEx6PKxv0o8y2GvA%40mail.gmail.com Backpatch-through: 14 Branch -- REL_15_STABLE

pgsql: Remove unused check in heap_xlog_insert()

2025-06-26 Thread Melanie Plageman
, as of now, we do not support any freezing of pages in the heap_insert() path. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/flat/CAAKRu_Zp4Pi-t51OFWm1YZ-cctDfBhHCMZ%3DEx6PKxv0o8y2GvA%40mail.gmail.com Backpatch-through: 14 Branch -- REL_16_STABLE

pgsql: Remove unused check in heap_xlog_insert()

2025-06-26 Thread Melanie Plageman
, as of now, we do not support any freezing of pages in the heap_insert() path. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/flat/CAAKRu_Zp4Pi-t51OFWm1YZ-cctDfBhHCMZ%3DEx6PKxv0o8y2GvA%40mail.gmail.com Backpatch-through: 14 Branch -- REL_14_STABLE

pgsql: Remove unused check in heap_xlog_insert()

2025-06-26 Thread Melanie Plageman
, as of now, we do not support any freezing of pages in the heap_insert() path. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/flat/CAAKRu_Zp4Pi-t51OFWm1YZ-cctDfBhHCMZ%3DEx6PKxv0o8y2GvA%40mail.gmail.com Backpatch-through: 14 Branch -- REL_17_STABLE

pgsql: Simplify vacuum VM update logging counters

2025-06-26 Thread Melanie Plageman
ed on this knowledge. Doing so is worthwhile because the old logic was confusing and misguided. Author: Melanie Plageman Reviewed-by: Nazir Bilal Yavuz Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/flat/CAAKRu_a9w_n2mwY%3DG4LjfWTvRTJtjbfvnYAKi4WjO8QXHHrA0g%40mail.gmail.com Branch ---

pgsql: Test that vacuum removes tuples older than OldestXmin

2025-06-25 Thread Melanie Plageman
it required a large number of dead rows to generate enough dead TIDs to force multiple index vacuuming passes. Once the source code changes were made to allow a minimum maintenance_work_mem value of 64kB, the test could be made much faster and more stable. Author: Melanie Plageman Reviewed-by: J

pgsql: Test that vacuum removes tuples older than OldestXmin

2025-06-24 Thread Melanie Plageman
it required a large number of dead rows to generate enough dead TIDs to force multiple index vacuuming passes. Once the source code changes were made to allow a minimum maintenance_work_mem value of 64kB, the test could be made much faster and more stable. Author: Melanie Plageman Reviewed-by: J

Re: pgsql: Improve runtime and output of tests for replication slots checkp

2025-06-20 Thread Melanie Plageman
On Thu, Jun 19, 2025 at 7:31 PM Alexander Korotkov wrote: > Improve runtime and output of tests for replication slots checkpointing. > > The TAP tests that verify logical and physical replication slot behavior > during checkpoints (046_checkpoint_logical_slot.pl and > 047_checkpoint_physical_slot

pgsql: Correct heap vacuum boundary state setup ordering

2025-06-02 Thread Melanie Plageman
: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzntqvVEdbbpqG5JqSZGuLWmy4PBfUO-OswfivKchr2gvw%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/31a7e175fd8f6079555f2bba95c7cc118236bb20 Modified Files

pgsql: Replace deprecated log_connections values in docs and tests

2025-05-22 Thread Melanie Plageman
on and the 'receipt' aspect in src/test/postmaster). In other cases, the new value for log_connections is a superset of the previous included aspects (e.g. 'all' in src/test/kerberos/t/001_auth.pl). Reported-by: Peter Eisentraut Author: Melanie Plageman Reviewed-by: Peter

pgsql: Add maintenance_io_concurrency flag to some read stream users

2025-04-28 Thread Melanie Plageman
Add maintenance_io_concurrency flag to some read stream users Index vacuuming and [auto]prewarm AIO concurrency should be governed by maintenance_io_concurrency. As such, pass those read stream users the READ_STREAM_MAINTENANCE flag which will calculate their read stream distance with maintenance_

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Melanie Plageman
On Mon, Apr 7, 2025 at 5:15 AM David Rowley wrote: > Are these failures from patches applied to master prior to 3ba2cdaa? Yea, my email was held in moderation for days. I guess cross-posting is flagged. I thought I saw people regularly cc pgsql-hackers when replying to pgsql-committers, but I g

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Melanie Plageman
On Fri, Apr 4, 2025 at 9:17 AM Alexander Korotkov wrote: > > Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate > > This commit implements the automatic conversion of 'x IN (VALUES ...)' into > ScalarArrayOpExpr. That simplifies the query tree, eliminating the appearance > of an unnece

pgsql: Use streaming read I/O in btree vacuuming

2025-04-05 Thread Melanie Plageman
explicit ReadBuffer() calls. Author: Andrey Borodin Reviewed-by: Melanie Plageman Reviewed-by: Junwang Zhao Reviewed-by: Kirill Reshke Discussion: https://postgr.es/m/flat/CAAKRu_bW1UOyup%3DjdFw%2BkOF9bCaAm%3D9UpiyZtbPMn8n_vnP%2Big%40mail.gmail.com#3b3a84132fc683b3ee5b40bc4c2ea2a5 Branch

pgsql: aio: Add errcontext for processing I/Os for another backend

2025-04-05 Thread Melanie Plageman
worker is processing. For io_method io_uring, this adds context only when a backend is *completing* I/O for another backend. It specifies the pid of the owning process. Author: Melanie Plageman Reviewed-by: Andres Freund Discussion: https://postgr.es/m

pgsql: Remove superfluous autoprewarm check

2025-04-04 Thread Melanie Plageman
Remove superfluous autoprewarm check autoprewarm_database_main() prewarms blocks from the same database. It is passed an array of sorted BlockInfoRecords and a start and stop index into the array. The range represented should include only blocks belonging to global objects or blocks from a single

pgsql: Refactor autoprewarm_database_main() in preparation for read str

2025-04-04 Thread Melanie Plageman
avuz Co-authored-by: Melanie Plageman Reviewed-by: Heikki Linnakangas Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/flat/CAN55FZ3n8Gd%2BhajbL%3D5UkGzu_aHGRqnn%2BxktXq2fuds%3D1AOR6Q%40mail.gmail.com Branch -- master Details --- https://git.postgres

pgsql: Use streaming read I/O in autoprewarm

2025-04-04 Thread Melanie Plageman
: Melanie Plageman Reviewed-by: Heikki Linnakangas Reviewed-by: Daniel Gustafsson Reviewed-by: Andrey M. Borodin (earlier versions) Reviewed-by: Kirill Reshke (earlier versions) Reviewed-by: Matheus Alcantara (earlier versions) Discussion: https://postgr.es/m/flat/CAN55FZ3n8Gd%2BhajbL

pgsql: Fix autoprewarm neglect of tablespaces

2025-04-04 Thread Melanie Plageman
Fix autoprewarm neglect of tablespaces While prewarming blocks from a dump file, autoprewarm_database_main() mistakenly ignored tablespace when detecting the beginning of the next relation to prewarm. Because RelFileNumbers are only unique within a tablespace, autoprewarm could miss prewarming blo

pgsql: Remove misleading read stream asserts in a few users

2025-04-03 Thread Melanie Plageman
Remove misleading read stream asserts in a few users Several read stream users asserted that the read stream was exhausted after looping on that very condition. It was pointed out in an a review of an as-of-yet uncommitted read stream user [1] that this was confusing and could lead the reader to t

pgsql: Use AIO batchmode for bitmap heap scans

2025-04-03 Thread Melanie Plageman
Use AIO batchmode for bitmap heap scans Previously bitmap heap scan was not AIO batchmode safe because of the visibility map reads potentially done for the "skip fetch" optimization (which skipped fetching tuples from the heap if the pages were all visible and none of the columns were used in the

pgsql: Use streaming read I/O in heap amcheck

2025-03-27 Thread Melanie Plageman
Use streaming read I/O in heap amcheck Instead of directly invoking ReadBuffer() for each unskippable block in the heap relation, verify_heapam() now uses the read stream API to acquire the next buffer to check for corruption. Author: Matheus Alcantara Co-authored-by: Melanie Plageman Reviewed

pgsql: Fix bitmapheapscan incorrect recheck of NULL tuples

2025-03-24 Thread Melanie Plageman
Fix bitmapheapscan incorrect recheck of NULL tuples The bitmap heap scan skip fetch optimization skips fetching the heap block when a page is set all-visible in the visibility map and no columns from the table are needed to satisfy the query. 2b73a8cd33b and c3953226a07 changed the control flow o

pgsql: Use streaming read I/O in SP-GiST vacuuming

2025-03-21 Thread Melanie Plageman
list which is later processed to ensure no dead tuples are left behind. Pages containing such tuples are still read by directly calling ReadBuffer() and do not use the read stream API. Author: Andrey M. Borodin Reviewed-by: Melanie Plageman Discussion: https://postgr.es/m/37432403-8657-403B

pgsql: Use streaming read I/O in GiST vacuuming

2025-03-21 Thread Melanie Plageman
backtrack and reprocess those pages. These pages are still read with explicit ReadBuffer() calls. Author: Andrey M. Borodin Reviewed-by: Melanie Plageman Discussion: https://postgr.es/m/EFEBED92-18D1-4C0F-A4EB-CD47072EF071%40yandex-team.ru Branch -- master Details --- https

pgsql: Assorted trivial cleanup of c5c239e26e

2025-03-21 Thread Melanie Plageman
Assorted trivial cleanup of c5c239e26e c5c239e26e made btree vacuum use the read stream API. Though it used functions declared in read_stream.h, it relied on transitively including it. Explicitly include that file. Also remove an extraneous newline and decrease the scope of one of the local variab

pgsql: Increase default maintenance_io_concurrency to 16

2025-03-18 Thread Melanie Plageman
Increase default maintenance_io_concurrency to 16 Since its introduction in fc34b0d9de27a, the default maintenance_io_concurrency has been larger than the default effective_io_concurrency. maintenance_io_concurrency primarily controlled prefetching done on behalf of the whole system, for operation

pgsql: Remove table AM callback scan_bitmap_next_block

2025-03-15 Thread Melanie Plageman
Remove table AM callback scan_bitmap_next_block After pushing the bitmap iterator into table-AM specific code (as part of making bitmap heap scan use the read stream API in 2b73a8cd33b7), scan_bitmap_next_block() no longer returns the current block number. Since scan_bitmap_next_block() isn't retu

pgsql: Separate TBM[Shared|Private]Iterator and TBMIterateResult

2025-03-15 Thread Melanie Plageman
Separate TBM[Shared|Private]Iterator and TBMIterateResult Remove the TBMIterateResult member from the TBMPrivateIterator and TBMSharedIterator and make tbm_[shared|private_]iterate() take a TBMIterateResult as a parameter. This allows tidbitmap API users to manage multiple TBMIterateResults per s

pgsql: BitmapHeapScan uses the read stream API

2025-03-15 Thread Melanie Plageman
: Melanie Plageman Reviewed-by: Tomas Vondra Tested-by: Tomas Vondra Tested-by: Andres Freund Tested-by: Thomas Munro Tested-by: Nazir Bilal Yavuz Discussion: https://postgr.es/m/flat/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com Branch -- master Details

pgsql: Increase default effective_io_concurrency to 16

2025-03-12 Thread Melanie Plageman
Increase default effective_io_concurrency to 16 The default effective_io_concurrency has been 1 since it was introduced in b7b8f0b6096d2ab6e. Referencing the associated discussion [1], it seems 1 was chosen as a conservative value that seemed unlikely to cause regressions. Experimentation on high

Re: pgsql: Modularize log_connections output

2025-03-12 Thread Melanie Plageman
Thanks for taking a look! On Wed, Mar 12, 2025 at 11:50 AM Daniel Westermann (DWE) wrote: > > > great, thanks for this. One question: > > >The current log_connections options are 'receipt', 'authentication', and > >'authorization'. The empty string disables all connection logging. 'all' > >enable

pgsql: Modularize log_connections output

2025-03-12 Thread Melanie Plageman
at previously supported substrings of on, off, true, false, yes, and no are no longer supported. Author: Melanie Plageman Reviewed-by: Bertrand Drouvot Reviewed-by: Fujii Masao Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/flat/CAAKRu_b_smAHK0ZjrnL5GRxnAVWujEXQWpLXYzGbmpcZd3nLY

pgsql: Add connection establishment duration logging

2025-03-12 Thread Melanie Plageman
. To make this portable, the timings captured in the postmaster (socket creation time, fork initiation time) are passed through the BackendStartupData. Author: Melanie Plageman Reviewed-by: Bertrand Drouvot Reviewed-by: Fujii Masao Reviewed-by: Daniel Gustafsson Reviewed-by: Jacob Champion R

pgsql: Trigger more frequent autovacuums with relallfrozen

2025-03-03 Thread Melanie Plageman
-triggered autovacuums, as anti-wraparound vacuums are not automatically canceled by conflicting lock requests. We calculate the unfrozen percentage of the table using the recently added (99f8f3fbbc8f) relallfrozen column of pg_class. Author: Melanie Plageman Reviewed-by: Nathan Bossart Reviewed-by: Greg

Re: pgsql: Add relallfrozen to pg_class

2025-03-03 Thread Melanie Plageman
On Mon, Mar 3, 2025 at 11:44 AM Álvaro Herrera wrote: > > On 2025-Mar-03, Melanie Plageman wrote: > > > relallfrozen, together with relallvisible, is useful for estimating the > > outstanding number of all-visible but not all-frozen pages in the > > relation for the pu

Re: pgsql: Add relallfrozen to pg_class

2025-03-03 Thread Melanie Plageman
On Mon, Mar 3, 2025 at 11:48 AM Melanie Plageman wrote: > > On Mon, Mar 3, 2025 at 11:44 AM Álvaro Herrera > wrote: > > > > I'm confused about this. Why was the new value added to pg_class > > instead of to the pgstat system? I don't think relallvisible is

Re: pgsql: Set amcancrosscompare to true for hash

2025-03-03 Thread Melanie Plageman
Is there a discussion somewhere about why valgrind has been failing like this on skink since Saturday? valgrind: Fatal error at startup: a function redirection valgrind: which is mandatory for this platform-tool combination valgrind: cannot be set up. Details of the redirection are: It doesn'

pgsql: Add relallfrozen to pg_class

2025-03-03 Thread Melanie Plageman
insert-focused workloads with significant volume of frozen data. Bump catalog version Author: Melanie Plageman Reviewed-by: Nathan Bossart Reviewed-by: Robert Treat Reviewed-by: Corey Huinker Reviewed-by: Greg Sabino Mullane Discussion: https://postgr.es/m/flat/CAAKRu_aj-P7YyBz_cPNwztz6ohP

pgsql: Delay extraction of TIDBitmap per page offsets

2025-02-24 Thread Melanie Plageman
Delay extraction of TIDBitmap per page offsets Pages from the bitmap created by the TIDBitmap API can be exact or lossy. The TIDBitmap API extracts the tuple offsets from exact pages into an array for the convenience of the caller. This was done in tbm_private|shared_iterate() right after advanci

pgsql: Add lossy indicator to TBMIterateResult

2025-02-24 Thread Melanie Plageman
Add lossy indicator to TBMIterateResult TBMIterateResult->ntuples is -1 when the page in the bitmap is lossy. Add an explicit lossy indicator so that we can move ntuples out of the TBMIterateResult in a future commit. Discussion: https://postgr.es/m/CA%2BhUKGLHbKP3jwJ6_%2BhnGi37Pw3BD5j2amjV3oSk7

Re: pgsql: Reduce scope of heap vacuum per_buffer_data

2025-02-18 Thread Melanie Plageman
On Tue, Feb 18, 2025 at 9:30 AM Melanie Plageman wrote: > > Reduce scope of heap vacuum per_buffer_data > > Move lazy_scan_heap()'s per_buffer_data variable into a tighter scope. > In lazy_scan_heap()'s phase I heap vacuuming, the read stream API > returns a pointer

pgsql: Reduce scope of heap vacuum per_buffer_data

2025-02-18 Thread Melanie Plageman
Reduce scope of heap vacuum per_buffer_data Move lazy_scan_heap()'s per_buffer_data variable into a tighter scope. In lazy_scan_heap()'s phase I heap vacuuming, the read stream API returns a pointer to the next block number to vacuum. As long as read_stream_next_buffer() returns a valid buffer, pe

Re: pgsql: Convert heap_vac_scan_next_block() boolean parameters to flags

2025-02-14 Thread Melanie Plageman
On Fri, Feb 14, 2025 at 3:45 PM Daniel Gustafsson wrote: > > > On 14 Feb 2025, at 18:59, Melanie Plageman > > wrote: > > > > Convert heap_vac_scan_next_block() boolean parameters to flags > > This seems to have made skink upset with a Valgrind error: > > h

Re: pgsql: Use streaming read I/O in VACUUM's third phase

2025-02-14 Thread Melanie Plageman
On Fri, Feb 14, 2025 at 1:31 PM Melanie Plageman wrote: > > On Fri, Feb 14, 2025 at 12:59 PM Melanie Plageman > wrote: > > > > Use streaming read I/O in VACUUM's third phase > > > > Make vacuum's third phase (its second pass over the heap), which reaps

Re: pgsql: Use streaming read I/O in VACUUM's third phase

2025-02-14 Thread Melanie Plageman
On Fri, Feb 14, 2025 at 12:59 PM Melanie Plageman wrote: > > Use streaming read I/O in VACUUM's third phase > > Make vacuum's third phase (its second pass over the heap), which reaps > dead items collected in the first phase and marks them as reusable, use > the read

pgsql: Use streaming read I/O in VACUUM's first phase

2025-02-14 Thread Melanie Plageman
Use streaming read I/O in VACUUM's first phase Make vacuum's first phase, which prunes and freezes tuples and records dead TIDs, use the read stream API by by converting heap_vac_scan_next_block() to a read stream callback. Reviewed-by: Masahiko Sawada Reviewed-by: Thomas Munro Discussion: htt

pgsql: Use streaming read I/O in VACUUM's third phase

2025-02-14 Thread Melanie Plageman
xt(), that looks ahead in the TidStore and returns the next block number to read for vacuum. Author: Melanie Plageman Co-authored-by: Thomas Munro Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/CA%2BhUKGKN3oy0bN_3yv8hd78a4%2BM1tJC9z7mD8%2Bf%2ByA%2BGeoFUwQ%40mail.gmail.c

pgsql: Convert heap_vac_scan_next_block() boolean parameters to flags

2025-02-14 Thread Melanie Plageman
Convert heap_vac_scan_next_block() boolean parameters to flags The read stream API only allows one piece of extra per block state to be passed back to the API user (per_buffer_data). lazy_scan_heap() needs two pieces of per-buffer data: whether or not the block was all-visible in the visibility ma

pgsql: Limit pgbench COPY FREEZE to ordinary relations

2025-02-11 Thread Melanie Plageman
Limit pgbench COPY FREEZE to ordinary relations pgbench client-side data generation uses COPY FREEZE to load data for most tables. COPY FREEZE isn't supported for partitioned tables and since pgbench only supports partitioning pgbench_accounts, pgbench used a hard-coded check to skip COPY FREEZE a

pgsql: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-02-11 Thread Melanie Plageman
Eagerly scan all-visible pages to amortize aggressive vacuum Aggressive vacuums must scan every unfrozen tuple in order to advance the relfrozenxid/relminmxid. Because data is often vacuumed before it is old enough to require freezing, relations may build up a large backlog of pages that are set a

pgsql: Move BitmapTableScan per-scan setup into a helper

2025-01-30 Thread Melanie Plageman
Move BitmapTableScan per-scan setup into a helper Add BitmapTableScanSetup(), a helper which contains all of the code that must be done on every scan of the table in a bitmap table scan. This includes scanning the index, building the bitmap, and setting up the scan descriptors. Pushing this setup

pgsql: Add and use BitmapHeapScanDescData struct

2025-01-16 Thread Melanie Plageman
Add and use BitmapHeapScanDescData struct Move the several members of HeapScanDescData which are specific to Bitmap Heap Scans into a new struct, BitmapHeapScanDescData, which inherits from HeapScanDescData. This reduces the size of the HeapScanDescData for other types of scans and will allow us

Re: pgsql: Change gist stratnum function to use CompareType

2025-01-15 Thread Melanie Plageman
On Wed, Jan 15, 2025 at 5:41 AM Peter Eisentraut wrote: > > Change gist stratnum function to use CompareType > > This changes commit 7406ab623fe in that the gist strategy number > mapping support function is changed to use the CompareType enum as > input, instead of the "well-known" RT*StrategyNum

pgsql: Add more general summary to vacuumlazy.c

2025-01-15 Thread Melanie Plageman
Add more general summary to vacuumlazy.c Add more comments at the top of vacuumlazy.c on heap relation vacuuming implementation. Previously vacuumlazy.c only had details related to dead TID storage. This commit adds a more general summary to help future developers understand the heap relation vac

pgsql: Synchronize guc_tables.c categories with vacuum docs categories

2025-01-14 Thread Melanie Plageman
Synchronize guc_tables.c categories with vacuum docs categories ca9c6a5680d consolidated most of the vacuum-related GUCs' documentation into a new subsection. af2317652d5daf8b then enforced this order in postgresql.conf.sample. This commit reorganizes the GUC groups in guc_tables.c/h to match the

pgsql: Reorder vacuum GUCs in postgresql.conf.sample to match docs

2025-01-13 Thread Melanie Plageman
Reorder vacuum GUCs in postgresql.conf.sample to match docs ca9c6a5680d consolidated most of vacuum-related GUCs' documentation into a new subsection. It neglected, however, to reorganize postgresql.conf.sample to match the new order. Do this now. Reported-by: Álvaro Herrera Discussion: https://p

pgsql: Consolidate docs for vacuum-related GUCs in new subsection

2025-01-10 Thread Melanie Plageman
Consolidate docs for vacuum-related GUCs in new subsection GUCs related to vacuum's freezing behavior were documented in a subsection of the Client Connection Defaults documentation. These GUCs don't belong there, as they affect the freezing behavior of all vacuums -- including autovacuums. There

pgsql: Fix overflow danger in SampleHeapTupleVisible(), take 2

2024-12-20 Thread Melanie Plageman
Fix overflow danger in SampleHeapTupleVisible(), take 2 28328ec87b45725 addressed one overflow danger in SampleHeapTupleVisible() but introduced another, albeit a less likely one. Modify the binary search code to remove this danger. Reported-by: Richard Guo Reviewed-by: Richard Guo, Ranier Vilela

pgsql: Remove final mention of FREEZE_PAGE from comments

2024-12-19 Thread Melanie Plageman
Remove final mention of FREEZE_PAGE from comments b7493e1ab35 removed leftover mentions of XLOG_HEAP2_FREEZE_PAGE records from comments but neglected to remove one mention of FREEZE_PAGE. Reported off-list by Alexander Lakhin Branch -- master Details --- https://git.postgresql.org/pg/co

pgsql: Fix bitmap table scan crash on iterator release

2024-12-19 Thread Melanie Plageman
Fix bitmap table scan crash on iterator release 1a0da347a7ac98db replaced Bitmap Table Scan's individual private and shared iterators with a unified iterator. It neglected, however, to check if the iterator had already been cleaned up before doing so on rescan. Add this check both on rescan and en

pgsql: Remove extra prefetch iterator setup for Bitmap Table Scan

2024-12-19 Thread Melanie Plageman
Remove extra prefetch iterator setup for Bitmap Table Scan 1a0da347a7ac98db replaced Bitmap Table Scan's separate private and shared bitmap iterators with a unified iterator. It accidentally set up the prefetch iterator twice for non-parallel bitmap table scans. Remove the extra set up call to tbm

pgsql: Remove leftover mentions of XLOG_HEAP2_FREEZE_PAGE records

2024-12-18 Thread Melanie Plageman
Remove leftover mentions of XLOG_HEAP2_FREEZE_PAGE records f83d709760d merged the separate XLOG_HEAP2_FREEZE_PAGE records into a new combined prune, freeze, and vacuum record with opcode XLOG_HEAP2_PRUNE_VACUUM_SCAN. Remove the last few references to XLOG_HEAP2_FREEZE_PAGE records which were accid

pgsql: Bitmap Table Scans use unified TBMIterator

2024-12-18 Thread Melanie Plageman
around a bit of code, so a few variables are initialized elsewhere. Author: Melanie Plageman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/c736f6aa-8b35-4e20-9621-62c7c82e2168%40vondra.me Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Add common interface for TBMIterators

2024-12-18 Thread Melanie Plageman
. This encapsulation simplifies call sites for callers supporting both parallel and serial TID Bitmap access. TBMIterator is not yet used in this commit. Author: Melanie Plageman Reviewed-by: Tomas Vondra, Heikki Linnakangas Discussion: https://postgr.es/m/063e4eb4-32d9-439e-a0b1-75565a9835a8%40iki.fi

pgsql: Fix overflow danger in SampleHeapTupleVisible()

2024-12-18 Thread Melanie Plageman
Fix overflow danger in SampleHeapTupleVisible() 68d9662be1c4b70 made HeapScanDesc->rs_ntuples unsigned but neglected to change how it was being used in SampleHeapTupleVisible(). Return early if rs_ntuples is 0 to avoid overflowing and incorrectly executing the loop code in SampleHeapTupleVisible(

Re: pgsql: seg: pure parser and reentrant scanner

2024-12-18 Thread Melanie Plageman
On Wed, Dec 18, 2024 at 2:57 AM Peter Eisentraut wrote: > > seg: pure parser and reentrant scanner sifaka is failing [1] with cubescan.c:348:15: error: redefinition of typedef 'yyscan_t' is a C11 feature [-Werror,-Wtypedef-redefinition] 348 | typedef void* yyscan_t; | ^ ./c

pgsql: Make rs_cindex and rs_ntuples unsigned

2024-12-18 Thread Melanie Plageman
test in heapam_scan_bitmap_next_tuple() that checks if rs_cindex < 0. This was never true, but now that rs_cindex is unsigned, it makes even less sense. While we are at it, initialize both rs_cindex and rs_ntuples to 0 in initscan(). Author: Melanie Plageman Reviewed-by: Dilip Kumar Discussion: https://

pgsql: Rename LVRelState->frozen_pages

2024-12-17 Thread Melanie Plageman
zen tuple. It also includes pages that are not all-frozen. Author: Melanie Plageman Reviewed-by: Andres Freund, Masahiko Sawada, Nitin Jadhav, Bilal Yavuz Discussion: https://postgr.es/m/ctdjzroezaxmiyah3gwbwm67defsrwj2b5fpfs4ku6msfpxeia%40mwjyqlhwr2wu Branch -- master Details --- ht

pgsql: Count pages set all-visible and all-frozen in VM during vacuum

2024-12-17 Thread Melanie Plageman
aggressive vacuums. The counts of newly all-visible and all-frozen pages give us insight into the rate at which this debt is being accrued and paid down. Author: Melanie Plageman Reviewed-by: Masahiko Sawada, Alastair Turner, Nitin Jadhav, Andres Freund, Bilal Yavuz, Tomas Vondra Discussion: https

pgsql: Make visibilitymap_set() return previous state of vmbits

2024-12-17 Thread Melanie Plageman
e. Author: Melanie Plageman Reviewed-by: Masahiko Sawada, Andres Freund, Nitin Jadhav, Bilal Yavuz Discussion: https://postgr.es/m/flat/CAAKRu_ZQe26xdvAqo4weHLR%3DivQ8J4xrSfDDD8uXnh-O-6P6Lg%40mail.gmail.com#6d8d2b4219394f774889509bf3bdc13d, https://p

pgsql: Move EXPLAIN counter increment to heapam_scan_bitmap_next_block

2024-10-25 Thread Melanie Plageman
required to be able to use the read stream API for bitmap heap scans. The bitmap iterator must be advanced from inside the read stream callback, so TBMIterateResults cannot be used as a flow control mechanism in BitmapHeapNext(). Author: Melanie Plageman Reviewed-by: Tomas Vondra, Heikki Linnakangas

pgsql: Make table_scan_bitmap_next_block() async-friendly

2024-10-25 Thread Melanie Plageman
and 3) the prefetch iterator will be removed once the read stream API is used. Author: Melanie Plageman Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas, Mark Dilger Discussion: https://postgr.es/m/063e4eb4-32d9-439e-a0b1-75565a9835a8%40iki.fi Branch -- master Details ---

pgsql: Add accidentally omitted test to meson build file

2024-07-29 Thread Melanie Plageman
Add accidentally omitted test to meson build file 01e2b7f0fd02a44e introduced a test that vacuum correctly removes tuples older than OldestXmin. The same commit was backpatched on 14-16, but 16 is the only version with meson and the test was mistakenly left off of the recovery test meson build fil

pgsql: Use DELETE instead of UPDATE to speed up vacuum test

2024-07-29 Thread Melanie Plageman
Use DELETE instead of UPDATE to speed up vacuum test d42f60ccf07d89c introduced a test which generated dead tuples for vacuum with an UPDATE. The test only required enough dead TIDs for two rounds of index vacuuming. This can be accomplished with a DELETE instead of an UPDATE -- which generates ab

pgsql: Use DELETE instead of UPDATE to speed up vacuum test

2024-07-29 Thread Melanie Plageman
Use DELETE instead of UPDATE to speed up vacuum test e2e992820f104def introduced a test which generated dead tuples for vacuum with an UPDATE. The test only required enough dead TIDs for two rounds of index vacuuming. This can be accomplished with a DELETE instead of an UPDATE -- which generates a

pgsql: Use DELETE instead of UPDATE to speed up vacuum test

2024-07-29 Thread Melanie Plageman
Use DELETE instead of UPDATE to speed up vacuum test 01e2b7f0fd02a44e introduced a test which generated dead tuples for vacuum with an UPDATE. The test only required enough dead TIDs for two rounds of index vacuuming. This can be accomplished with a DELETE instead of an UPDATE -- which generates a

pgsql: Revert "Test that vacuum removes tuples older than OldestXmin"

2024-07-22 Thread Melanie Plageman
Revert "Test that vacuum removes tuples older than OldestXmin" This reverts commit 80c34692e8e674e3b2f150f248ef2002ae2ac3a7. This test proved to be unstable on the buildfarm, timing out before the standby could catch up on 32-bit machines where more rows were required and failing to reliably trig

pgsql: Revert "Test that vacuum removes tuples older than OldestXmin"

2024-07-22 Thread Melanie Plageman
Revert "Test that vacuum removes tuples older than OldestXmin" This reverts commit aa607980aee08416211f003ab41aa750f5559712. This test proved to be unstable on the buildfarm, timing out before the standby could catch up on 32-bit machines where more rows were required and failing to reliably trig

pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX

2024-07-19 Thread Melanie Plageman
s/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5aadee Back-patch through 14 Author: Melanie Plageman Reviewed-by: Peter Geoghegan, Robert Haas, Andres Freund, Heikki Linnakangas, and Noah Misch Discussion: https://postgr.es/m/CAAKRu_bDD7oq9ZwB2OJqub5BovMG6UjEYsoK2LVt

pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX

2024-07-19 Thread Melanie Plageman
val until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://post

pgsql: Test that vacuum removes tuples older than OldestXmin

2024-07-19 Thread Melanie Plageman
est stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Pe

pgsql: Test that vacuum removes tuples older than OldestXmin

2024-07-19 Thread Melanie Plageman
est stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Pe

pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX

2024-07-19 Thread Melanie Plageman
s/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5aadee Back-patch through 14 Author: Melanie Plageman Reviewed-by: Peter Geoghegan, Robert Haas, Andres Freund, Heikki Linnakangas, and Noah Misch Discussion: https://postgr.es/m/CAAKRu_bDD7oq9ZwB2OJqub5BovMG6UjEYsoK2LVt

pgsql: Test that vacuum removes tuples older than OldestXmin

2024-07-19 Thread Melanie Plageman
est stability issues. Discussion of the bug: https://postgr.es/m/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com Discussion of the test: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Author: Melanie Plageman Reviewed-by: Pe

pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX

2024-07-19 Thread Melanie Plageman
val until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://post

pgsql: Ensure vacuum removes all visibly dead tuples older than OldestX

2024-07-19 Thread Melanie Plageman
val until the tuple is removable. Thus, the worst that can happen is a recovery conflict. Fixes BUG# 17257 Back-patched in versions 14-17 Author: Melanie Plageman Reviewed-by: Noah Misch, Peter Geoghegan, Robert Haas, Andres Freund, and Heikki Linnakangas Discussion: https://post

pgsql: Test that vacuum removes tuples older than OldestXmin

2024-07-19 Thread Melanie Plageman
est stability issues. Author: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/aa607980aee08416211f003ab41aa750f5559

pgsql: Test that vacuum removes tuples older than OldestXmin

2024-07-19 Thread Melanie Plageman
est stability issues. Author: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com Branch -- REL_17_STABLE Details --- https://git.postgresql.org/pg/commitd

pgsql: BitmapHeapScan: Remove incorrect assert and reset field

2024-05-16 Thread Melanie Plageman
tuple. Remove the assert and reset the field on which it previously asserted to avoid incorrectly emitting NULL-filled tuples from a previous scan on rescan. Author: Melanie Plageman Reviewed-by: Tomas Vondra, Michael Paquier, Alvaro Herrera Reported-by: Melanie Plageman Reproduced-by: Tomas

Re: pgsql: Transform OR clauses to ANY expression

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov wrote: > > Transform OR clauses to ANY expression > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) > on the preliminary stage of optimization when we are still working with the > expression tree. > > Here Cn is a n-

Re: pgsql: Use bump memory context for tuplesorts

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 8:32 AM David Rowley wrote: > > Use bump memory context for tuplesorts > > 29f6a959c added a bump allocator type for efficient compact allocations. > Here we make use of this for non-bounded tuplesorts to store tuples. > This is very space efficient when storing narrow tuple

Re: pgsql: Reduce branches in heapgetpage()'s per-tuple loop

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 3:29 AM Andres Freund wrote: > > Reduce branches in heapgetpage()'s per-tuple loop > > Until now, heapgetpage()'s loop over all tuples performed some conditional > checks for each tuple, even though condition did not change across the loop. I haven't reviewed exactly what t