On Tue, May 14, 2024 at 4:05 PM Alvaro Herrera <alvhe...@alvh.no-ip.org> wrote: > > On 2024-May-14, Tomas Vondra wrote: > > I wonder why it resets enable_indexscan at all. I see that this query > first tries a seqscan, then if you disable that it tries an index only > scan, and if you disable that you get the expected bitmap indexscan. > But an indexscan doesn't seem to be in the cards.
Ah, yes. That is true. I think I added that when, in an older version of the test, I had a query that did try an index scan before bitmap heap scan. I've removed that guc from the attached v7. > > IMHO if the test requires a specific plan, it's better to do an actual > > "explain (rows off, costs off)" to check that. > > That's already in the patch, right? Yep. > I do wonder how do we _know_ that the test is testing what it wants to > test: > QUERY PLAN > ───────────────────────────────────────────────────────── > Nested Loop Anti Join > -> Seq Scan on skip_fetch t1 > -> Materialize > -> Bitmap Heap Scan on skip_fetch t2 > Recheck Cond: (a = 1) > -> Bitmap Index Scan on skip_fetch_a_idx > Index Cond: (a = 1) > > Is it because of the shape of the index condition? Maybe it's worth > explaining in the comments for the tests. There is a comment in the test that explains what it is exercising and how. We include the explain output (the plan) to ensure it is still using a bitmap heap scan. The test exercises the skip fetch optimization in bitmap heap scan when not all of the inner tuples are emitted. Without the patch, the test fails, so it is protection against someone adding back that assert in the future. It is not protection against someone deleting the line scan->rs_empty_tuples_pending = 0 That is, it doesn't verify that the empty unused tuples count is discarded. Do you think that is necessary? - Melanie
From 031012be9d77d5f5888b306d546322e6176e527e Mon Sep 17 00:00:00 2001 From: Melanie Plageman <melanieplage...@gmail.com> Date: Tue, 14 May 2024 13:36:42 -0400 Subject: [PATCH v7] BitmapHeapScan: Remove incorrect assert and reset field 04e72ed617be pushed the skip fetch optimization (allowing bitmap heap scans to operate like index-only scans if none of the underlying data is needed) into heap AM implementations of bitmap table scan callbacks. 04e72ed617be added an assert that all tuples in blocks eligible for the optimization had been NULL-filled and emitted by the end of the scan. This assert is incorrect when not all tuples need be scanned to execute the query; for example: a join in which not all inner tuples need to be scanned before skipping to the next outer tuple. Remove the assert and reset the field on which it previously asserted to avoid incorrectly emitting NULL-filled tuples from a previous scan on rescan. Author: Melanie Plageman Reviewed-by: Tomas Vondra, Michael Paquier, Alvaro Herrera Reported-by: Melanie Plageman Reproduced-by: Tomas Vondra, Richard Guo Discussion: https://postgr.es/m/CAMbWs48orzZVXa7-vP9Nt7vQWLTE04Qy4PePaLQYsVNQgo6qRg%40mail.gmail.com --- src/backend/access/heap/heapam.c | 9 +++++--- src/test/regress/expected/join.out | 36 ++++++++++++++++++++++++++++++ src/test/regress/sql/join.sql | 24 ++++++++++++++++++++ 3 files changed, 66 insertions(+), 3 deletions(-) diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 4be0dee4de..82bb9cb33b 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1184,7 +1184,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params, scan->rs_vmbuffer = InvalidBuffer; } - Assert(scan->rs_empty_tuples_pending == 0); + /* + * Reset rs_empty_tuples_pending, a field only used by bitmap heap scan, + * to avoid incorrectly emitting NULL-filled tuples from a previous scan + * on rescan. + */ + scan->rs_empty_tuples_pending = 0; /* * The read stream is reset on rescan. This must be done before @@ -1216,8 +1221,6 @@ heap_endscan(TableScanDesc sscan) if (BufferIsValid(scan->rs_vmbuffer)) ReleaseBuffer(scan->rs_vmbuffer); - Assert(scan->rs_empty_tuples_pending == 0); - /* * Must free the read stream before freeing the BufferAccessStrategy. */ diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out index 0246d56aea..6b16c3a676 100644 --- a/src/test/regress/expected/join.out +++ b/src/test/regress/expected/join.out @@ -7924,3 +7924,39 @@ where exists (select 1 from j3 (13 rows) drop table j3; +-- Exercise the "skip fetch" Bitmap Heap Scan optimization when candidate +-- tuples are discarded. This may occur when: +-- 1. A join doesn't require all inner tuples to be scanned for each outer +-- tuple, and +-- 2. The inner side is scanned using a bitmap heap scan, and +-- 3. The bitmap heap scan is eligible for the "skip fetch" optimization. +-- This optimization is usable when no data from the underlying table is +-- needed. Use a temp table so it is only visible to this backend and +-- vacuum may reliably mark all blocks in the table all visible in the +-- visibility map. +CREATE TEMP TABLE skip_fetch (a INT, b INT) WITH (fillfactor=10); +INSERT INTO skip_fetch SELECT i % 3, i FROM generate_series(0,30) i; +CREATE INDEX ON skip_fetch(a); +VACUUM (ANALYZE) skip_fetch; +SET enable_indexonlyscan = off; +SET enable_seqscan = off; +EXPLAIN (COSTS OFF) +SELECT t1.a FROM skip_fetch t1 LEFT JOIN skip_fetch t2 ON t2.a = 1 WHERE t2.a IS NULL; + QUERY PLAN +--------------------------------------------------------- + Nested Loop Anti Join + -> Seq Scan on skip_fetch t1 + -> Materialize + -> Bitmap Heap Scan on skip_fetch t2 + Recheck Cond: (a = 1) + -> Bitmap Index Scan on skip_fetch_a_idx + Index Cond: (a = 1) +(7 rows) + +SELECT t1.a FROM skip_fetch t1 LEFT JOIN skip_fetch t2 ON t2.a = 1 WHERE t2.a IS NULL; + a +--- +(0 rows) + +RESET enable_indexonlyscan; +RESET enable_seqscan; diff --git a/src/test/regress/sql/join.sql b/src/test/regress/sql/join.sql index 923e7c5549..8bfe3b7ba6 100644 --- a/src/test/regress/sql/join.sql +++ b/src/test/regress/sql/join.sql @@ -2904,3 +2904,27 @@ where exists (select 1 from j3 and t1.unique1 < 1; drop table j3; + +-- Exercise the "skip fetch" Bitmap Heap Scan optimization when candidate +-- tuples are discarded. This may occur when: +-- 1. A join doesn't require all inner tuples to be scanned for each outer +-- tuple, and +-- 2. The inner side is scanned using a bitmap heap scan, and +-- 3. The bitmap heap scan is eligible for the "skip fetch" optimization. +-- This optimization is usable when no data from the underlying table is +-- needed. Use a temp table so it is only visible to this backend and +-- vacuum may reliably mark all blocks in the table all visible in the +-- visibility map. +CREATE TEMP TABLE skip_fetch (a INT, b INT) WITH (fillfactor=10); +INSERT INTO skip_fetch SELECT i % 3, i FROM generate_series(0,30) i; +CREATE INDEX ON skip_fetch(a); +VACUUM (ANALYZE) skip_fetch; + +SET enable_indexonlyscan = off; +SET enable_seqscan = off; +EXPLAIN (COSTS OFF) +SELECT t1.a FROM skip_fetch t1 LEFT JOIN skip_fetch t2 ON t2.a = 1 WHERE t2.a IS NULL; +SELECT t1.a FROM skip_fetch t1 LEFT JOIN skip_fetch t2 ON t2.a = 1 WHERE t2.a IS NULL; + +RESET enable_indexonlyscan; +RESET enable_seqscan; -- 2.34.1
