On Fri, Mar 1, 2024 at 3:58 PM Tomas Vondra
wrote:
[..]
> TBH I don't have a clear idea what to do. It'd be cool to have at least
> some benefits in v17, but I don't know how to do that in a way that
> would be useful in the future.
>
> For example, the v20240124 patch implements this in the
On Fri, Mar 1, 2024 at 10:18 AM Tomas Vondra
wrote:
> But I have very hard time figuring out what the MVP version should be,
> because I have very limited understanding on how much control the index
> AM ought to have :-( And it'd be a bit silly to do something in v17,
> only to have to rip it
On 2/15/24 21:30, Peter Geoghegan wrote:
> On Thu, Feb 15, 2024 at 3:13 PM Andres Freund wrote:
>>> This is why I don't think that the tuples with lower page offset
>>> numbers are in any way significant here. The significant part is
>>> whether or not you'll actually need to visit more than one
Hi,
Thanks for looking at the patch!
On 3/1/24 09:20, Jakub Wartak wrote:
> On Wed, Jan 24, 2024 at 7:13 PM Tomas Vondra
> wrote:
> [
>>
>> (1) Melanie actually presented a very different way to implement this,
>> relying on the StreamingRead API. So chances are this struct won't
>> actually
On Wed, Jan 24, 2024 at 7:13 PM Tomas Vondra
wrote:
[
>
> (1) Melanie actually presented a very different way to implement this,
> relying on the StreamingRead API. So chances are this struct won't
> actually be used.
Given lots of effort already spent on this and the fact that is thread
is
On Thu, Feb 15, 2024 at 3:13 PM Andres Freund wrote:
> > This is why I don't think that the tuples with lower page offset
> > numbers are in any way significant here. The significant part is
> > whether or not you'll actually need to visit more than one leaf page
> > in the first place (plus the
Hi,
On 2024-02-15 12:53:10 -0500, Peter Geoghegan wrote:
> On Thu, Feb 15, 2024 at 12:26 PM Tomas Vondra
> wrote:
> > I may be missing something, but it seems fairly self-evident to me an
> > entry at the beginning of an index page won't get prefetched (assuming
> > the page-at-a-time thing).
>
On Thu, Feb 15, 2024 at 12:26 PM Tomas Vondra
wrote:
> I may be missing something, but it seems fairly self-evident to me an
> entry at the beginning of an index page won't get prefetched (assuming
> the page-at-a-time thing).
Sure, if the first item on the page is also the first item that we
On 2/15/24 17:42, Peter Geoghegan wrote:
> On Thu, Feb 15, 2024 at 9:36 AM Tomas Vondra
> wrote:
>> On 2/15/24 00:06, Peter Geoghegan wrote:
>>> I suppose that it might be much more important than I imagine it is
>>> right now, but it'd be nice to have something a bit more concrete to
>>> go
On Thu, Feb 15, 2024 at 9:36 AM Tomas Vondra
wrote:
> On 2/15/24 00:06, Peter Geoghegan wrote:
> > I suppose that it might be much more important than I imagine it is
> > right now, but it'd be nice to have something a bit more concrete to
> > go on.
> >
>
> This probably depends on which corner
On 2/15/24 00:06, Peter Geoghegan wrote:
> On Wed, Feb 14, 2024 at 4:46 PM Melanie Plageman
> wrote:
>
>> ...
>
> 2. Are you sure that the leaf-page-at-a-time thing is such a huge
> hindrance to effective prefetching?
>
> I suppose that it might be much more important than I imagine it is
>
On Thu, Feb 15, 2024 at 10:33 AM Andres Freund wrote:
> The issue here is that we need to read index leaf pages (synchronously for
> now!) to get the tids to do readahead of table data. What you describe is done
> for the table data (IMO not a good idea medium term [1]), but the problem at
> hand
Hi,
On 2024-02-15 09:59:27 +0530, Robert Haas wrote:
> I would have thought that the way this prefetching would work is that
> we would bring pages into shared_buffers sooner than we currently do,
> but not actually pin them until we're ready to use them, so that it's
> possible they might be
On Wed, Feb 14, 2024 at 7:43 PM Tomas Vondra
wrote:
> I don't think it's just a bookkeeping problem. In a way, nbtree already
> does keep an array of tuples to kill (see btgettuple), but it's always
> for the current index page. So it's not that we immediately go and kill
> the prior tuple -
On Wed, Feb 14, 2024 at 7:28 PM Andres Freund wrote:
> On 2024-02-13 14:54:14 -0500, Peter Geoghegan wrote:
> > This property of index scans is fundamental to how index scans work.
> > Pinning an index page as an interlock against concurrently TID
> > recycling by VACUUM is directly described by
Hi,
On 2024-02-13 14:54:14 -0500, Peter Geoghegan wrote:
> This property of index scans is fundamental to how index scans work.
> Pinning an index page as an interlock against concurrently TID
> recycling by VACUUM is directly described by the index API docs [1],
> even (the docs actually use
Hi,
On 2024-02-14 16:45:57 -0500, Melanie Plageman wrote:
> > > The LIMIT problem is not very clear to me either. Yes, if we get close
> > > to the end of the leaf page, we may need to visit the next leaf page.
> > > But that's kinda the whole point of prefetching - reading stuff ahead,
> > > and
On Wed, Feb 14, 2024 at 4:46 PM Melanie Plageman
wrote:
> So, a pin on the index leaf page is sufficient to keep line pointers
> from being reused? If we stick to prefetching heap blocks referred to
> by index tuples in a single index leaf page, and we keep that page
> pinned, will we still have
On Wed, Feb 14, 2024 at 1:21 PM Peter Geoghegan wrote:
>
> On Wed, Feb 14, 2024 at 8:34 AM Tomas Vondra
> wrote:
> > > Another thing that argues against doing this is that we might not need
> > > to visit any more B-Tree leaf pages when there is a LIMIT n involved.
> > > We could end up scanning
On Wed, Feb 14, 2024 at 11:40 AM Melanie Plageman
wrote:
>
> On Tue, Feb 13, 2024 at 2:01 PM Tomas Vondra
> wrote:
> >
> > On 2/7/24 22:48, Melanie Plageman wrote:
> > > ...
> > > - switching scan directions
> > >
> > > If the index scan switches directions on a given invocation of
> > >
On Wed, Feb 14, 2024 at 11:40 AM Melanie Plageman
wrote:
> I wasn't quite sure how we could use
> index_compute_xid_horizon_for_tuples() for inspiration -- per Peter's
> suggestion. But, I'd like to understand.
The point I was trying to make with that example was: a highly generic
mechanism can
On Wed, Feb 14, 2024 at 8:34 AM Tomas Vondra
wrote:
> > Another thing that argues against doing this is that we might not need
> > to visit any more B-Tree leaf pages when there is a LIMIT n involved.
> > We could end up scanning a whole extra leaf page (including all of its
> > tuples) for want
On Tue, Feb 13, 2024 at 2:01 PM Tomas Vondra
wrote:
>
> On 2/7/24 22:48, Melanie Plageman wrote:
> > ...
Issues
> > ---
> > - kill prior tuple
> >
> > This optimization doesn't work with index prefetching with the current
> > design. Kill prior tuple relies on alternating between fetching a
> >
On 2/14/24 08:10, Robert Haas wrote:
> On Thu, Feb 8, 2024 at 3:18 AM Melanie Plageman
> wrote:
>> - kill prior tuple
>>
>> This optimization doesn't work with index prefetching with the current
>> design. Kill prior tuple relies on alternating between fetching a
>> single index tuple and
On 2/13/24 20:54, Peter Geoghegan wrote:
> On Tue, Feb 13, 2024 at 2:01 PM Tomas Vondra
> wrote:
>> On 2/7/24 22:48, Melanie Plageman wrote:
>> I admit I haven't thought about kill_prior_tuple until you pointed out.
>> Yeah, prefetching separates (de-synchronizes) the two scans (index and
>>
On Thu, Feb 8, 2024 at 3:18 AM Melanie Plageman
wrote:
> - kill prior tuple
>
> This optimization doesn't work with index prefetching with the current
> design. Kill prior tuple relies on alternating between fetching a
> single index tuple and visiting the heap. After visiting the heap we
> can
On Tue, Feb 13, 2024 at 2:01 PM Tomas Vondra
wrote:
> On 2/7/24 22:48, Melanie Plageman wrote:
> I admit I haven't thought about kill_prior_tuple until you pointed out.
> Yeah, prefetching separates (de-synchronizes) the two scans (index and
> heap) in a way that prevents this optimization. Or at
On 2/7/24 22:48, Melanie Plageman wrote:
> ...
>
> Attached is a patch which implements a real queue and fixes some of
> the issues with the previous version. It doesn't pass tests yet and
> has issues. Some are bugs in my implementation I need to fix. Some are
> issues we would need to solve in
On Wed, Jan 24, 2024 at 3:20 PM Melanie Plageman
wrote:
>
> On Wed, Jan 24, 2024 at 4:19 AM Tomas Vondra
> wrote:
> >
> > On 1/24/24 01:51, Melanie Plageman wrote:
> > >> But I'm not sure what to do about optimizations that are more specific
> > >> to the access path. Consider for example the
On 1/25/24 11:45, Dilip Kumar wrote:
> On Wed, Jan 24, 2024 at 11:43 PM Tomas Vondra
> wrote:
>
>> On 1/22/24 07:35, Konstantin Knizhnik wrote:
>>>
>>> On 22/01/2024 1:47 am, Tomas Vondra wrote:
h, right. Well, you're right in this case we perhaps could set just one
of those flags,
On Wed, Jan 24, 2024 at 11:43 PM Tomas Vondra
wrote:
> On 1/22/24 07:35, Konstantin Knizhnik wrote:
> >
> > On 22/01/2024 1:47 am, Tomas Vondra wrote:
> >> h, right. Well, you're right in this case we perhaps could set just one
> >> of those flags, but the "purpose" of the two places is quite
On Wed, Jan 24, 2024 at 4:19 AM Tomas Vondra
wrote:
>
> On 1/24/24 01:51, Melanie Plageman wrote:
>
> >>> There are also table AM layering violations in my sketch which would
> >>> have to be worked out (not to mention some resource leakage I didn't
> >>> bother investigating [which causes it to
On 1/22/24 07:35, Konstantin Knizhnik wrote:
>
> On 22/01/2024 1:47 am, Tomas Vondra wrote:
>> h, right. Well, you're right in this case we perhaps could set just one
>> of those flags, but the "purpose" of the two places is quite different.
>>
>> The "prefetch" flag is fully controlled by the
On 1/22/24 08:21, Konstantin Knizhnik wrote:
>
> On 22/01/2024 1:39 am, Tomas Vondra wrote:
>>> Why we can prefer covering index to compound index? I see only two good
>>> reasons:
>>> 1. Extra columns type do not have comparison function need for AM.
>>> 2. The extra columns are never used in
On 1/24/24 01:51, Melanie Plageman wrote:
> On Tue, Jan 23, 2024 at 12:43 PM Tomas Vondra
> wrote:
>>
>> On 1/19/24 22:43, Melanie Plageman wrote:
>>
>>> We fill a queue with blocks from TIDs that we fetched from the index.
>>> The queue is saved in a scan descriptor that is made available to the
On Tue, Jan 23, 2024 at 12:43 PM Tomas Vondra
wrote:
>
> On 1/19/24 22:43, Melanie Plageman wrote:
>
> > We fill a queue with blocks from TIDs that we fetched from the index.
> > The queue is saved in a scan descriptor that is made available to the
> > streaming read callback. Once the queue is
On 1/19/24 22:43, Melanie Plageman wrote:
> On Fri, Jan 12, 2024 at 11:42 AM Tomas Vondra
> wrote:
>>
>> On 1/9/24 21:31, Robert Haas wrote:
>>> On Thu, Jan 4, 2024 at 9:55 AM Tomas Vondra
>>> wrote:
Here's a somewhat reworked version of the patch. My initial goal was to
see if it
On 22/01/2024 1:39 am, Tomas Vondra wrote:
Why we can prefer covering index to compound index? I see only two good
reasons:
1. Extra columns type do not have comparison function need for AM.
2. The extra columns are never used in query predicate.
Or maybe you don't want to include the
On 22/01/2024 1:47 am, Tomas Vondra wrote:
h, right. Well, you're right in this case we perhaps could set just one
of those flags, but the "purpose" of the two places is quite different.
The "prefetch" flag is fully controlled by the prefetcher, and it's up
to it to change it (e.g. I can
2024-01 Commitfest.
Hi, This patch has a CF status of "Needs Review" [1], but it seems
like there were CFbot test failures last time it was run [2]. Please
have a look and post an updated version if necessary.
==
[1] https://commitfest.postgresql.org/46/4351/
[2]
On 1/21/24 20:56, Konstantin Knizhnik wrote:
>
> On 19/01/2024 2:35 pm, Tomas Vondra wrote:
>>
>> On 1/19/24 09:34, Konstantin Knizhnik wrote:
>>> On 18/01/2024 6:00 pm, Tomas Vondra wrote:
On 1/17/24 09:45, Konstantin Knizhnik wrote:
> I have integrated your prefetch patch in Neon
On 1/21/24 20:50, Konstantin Knizhnik wrote:
>
> On 20/01/2024 12:14 am, Tomas Vondra wrote:
>> Looks like I was not true, even if it is not index-only scan but index
>>> condition involves only index attributes, then heap is not accessed
>>> until we find tuple satisfying search condition.
On 19/01/2024 2:35 pm, Tomas Vondra wrote:
On 1/19/24 09:34, Konstantin Knizhnik wrote:
On 18/01/2024 6:00 pm, Tomas Vondra wrote:
On 1/17/24 09:45, Konstantin Knizhnik wrote:
I have integrated your prefetch patch in Neon and it actually works!
Moreover, I combined it with prefetch of leaf
On 20/01/2024 12:14 am, Tomas Vondra wrote:
Looks like I was not true, even if it is not index-only scan but index
condition involves only index attributes, then heap is not accessed
until we find tuple satisfying search condition.
Inclusive index case described above
On 1/19/24 16:19, Konstantin Knizhnik wrote:
>
> On 18/01/2024 5:57 pm, Tomas Vondra wrote:
>> On 1/16/24 21:10, Konstantin Knizhnik wrote:
>>> ...
>>>
4. I think that performing prefetch at executor level is really great
> idea and so prefetch can be used by all indexes, including
On Fri, Jan 12, 2024 at 11:42 AM Tomas Vondra
wrote:
>
> On 1/9/24 21:31, Robert Haas wrote:
> > On Thu, Jan 4, 2024 at 9:55 AM Tomas Vondra
> > wrote:
> >> Here's a somewhat reworked version of the patch. My initial goal was to
> >> see if it could adopt the StreamingRead API proposed in [1],
On 18/01/2024 5:57 pm, Tomas Vondra wrote:
On 1/16/24 21:10, Konstantin Knizhnik wrote:
...
4. I think that performing prefetch at executor level is really great
idea and so prefetch can be used by all indexes, including custom
indexes. But prefetch will be efficient only if index can
On 1/19/24 09:34, Konstantin Knizhnik wrote:
>
> On 18/01/2024 6:00 pm, Tomas Vondra wrote:
>> On 1/17/24 09:45, Konstantin Knizhnik wrote:
>>> I have integrated your prefetch patch in Neon and it actually works!
>>> Moreover, I combined it with prefetch of leaf pages for IOS and it also
>>>
On 18/01/2024 6:00 pm, Tomas Vondra wrote:
On 1/17/24 09:45, Konstantin Knizhnik wrote:
I have integrated your prefetch patch in Neon and it actually works!
Moreover, I combined it with prefetch of leaf pages for IOS and it also
seems to work.
Cool! And do you think this is the right
On 1/17/24 09:45, Konstantin Knizhnik wrote:
> I have integrated your prefetch patch in Neon and it actually works!
> Moreover, I combined it with prefetch of leaf pages for IOS and it also
> seems to work.
>
Cool! And do you think this is the right design/way to do this?
> Just small notice:
On 1/16/24 21:10, Konstantin Knizhnik wrote:
>
> ...
>
>> 4. I think that performing prefetch at executor level is really great
>>> idea and so prefetch can be used by all indexes, including custom
>>> indexes. But prefetch will be efficient only if index can provide fast
>>> access to next TID
I have integrated your prefetch patch in Neon and it actually works!
Moreover, I combined it with prefetch of leaf pages for IOS and it also
seems to work.
Just small notice: you are reporting `blks_prefetch_rounds` in explain,
but it is not incremented anywhere.
Moreover, I do not precisely
On 16/01/2024 11:58 pm, Jim Nasby wrote:
On 1/16/24 2:10 PM, Konstantin Knizhnik wrote:
Amazon RDS is just vanilla Postgres with file system mounted on EBS
(Amazon distributed file system).
EBS provides good throughput but larger latencies comparing with
local SSDs.
I am not sure if
On 16/01/2024 11:58 pm, Jim Nasby wrote:
On 1/16/24 2:10 PM, Konstantin Knizhnik wrote:
Amazon RDS is just vanilla Postgres with file system mounted on EBS
(Amazon distributed file system).
EBS provides good throughput but larger latencies comparing with
local SSDs.
I am not sure if
On 1/16/24 2:10 PM, Konstantin Knizhnik wrote:
Amazon RDS is just vanilla Postgres with file system mounted on EBS
(Amazon distributed file system).
EBS provides good throughput but larger latencies comparing with local SSDs.
I am not sure if read-ahead works for EBS.
Actually, EBS only
On 16/01/2024 6:25 pm, Tomas Vondra wrote:
On 1/16/24 09:13, Konstantin Knizhnik wrote:
Hi,
On 12/01/2024 6:42 pm, Tomas Vondra wrote:
Hi,
Here's an improved version of this patch, finishing a lot of the stuff
that I alluded to earlier - moving the code from indexam.c, renaming a
bunch of
On Tue, Jan 16, 2024 at 11:25 AM Tomas Vondra
wrote:
> > 3. It doesn't perform prefetch of leave pages for IOS, only referenced
> > heap pages which are not marked as all-visible. It seems to me that if
> > optimized has chosen IOS (and not bitmap heap scan for example), then
> > there should be
On 1/16/24 09:13, Konstantin Knizhnik wrote:
> Hi,
>
> On 12/01/2024 6:42 pm, Tomas Vondra wrote:
>> Hi,
>>
>> Here's an improved version of this patch, finishing a lot of the stuff
>> that I alluded to earlier - moving the code from indexam.c, renaming a
>> bunch of stuff, etc. I've also
Hi,
On 12/01/2024 6:42 pm, Tomas Vondra wrote:
Hi,
Here's an improved version of this patch, finishing a lot of the stuff
that I alluded to earlier - moving the code from indexam.c, renaming a
bunch of stuff, etc. I've also squashed it into a single patch, to make
it easier to review.
I am
Not a full response, but just to address a few points:
On Fri, Jan 12, 2024 at 11:42 AM Tomas Vondra
wrote:
> Thinking about this, I think it should be possible to make prefetching
> work even for plans with execute_once=false. In particular, when the
> plan changes direction it should be
Hi,
Here's an improved version of this patch, finishing a lot of the stuff
that I alluded to earlier - moving the code from indexam.c, renaming a
bunch of stuff, etc. I've also squashed it into a single patch, to make
it easier to review.
I'll briefly go through the main changes in the patch,
On Thu, Jan 4, 2024 at 9:55 AM Tomas Vondra
wrote:
> Here's a somewhat reworked version of the patch. My initial goal was to
> see if it could adopt the StreamingRead API proposed in [1], but that
> turned out to be less straight-forward than I hoped, for two reasons:
I guess we need Thomas or
Hi,
Here's a somewhat reworked version of the patch. My initial goal was to
see if it could adopt the StreamingRead API proposed in [1], but that
turned out to be less straight-forward than I hoped, for two reasons:
(1) The StreamingRead API seems to be designed for pages, but the index
code
On 12/21/23 18:14, Robert Haas wrote:
> On Thu, Dec 21, 2023 at 11:08 AM Andres Freund wrote:
>> But I'd like you to feel guilty (no, not really) and fix it (yes, really) :)
>
> Sadly, you're more likely to get the first one than you are to get the
> second one. I can't really see going back to
On Thu, Dec 21, 2023 at 11:08 AM Andres Freund wrote:
> But I'd like you to feel guilty (no, not really) and fix it (yes, really) :)
Sadly, you're more likely to get the first one than you are to get the
second one. I can't really see going back to revisit that decision as
a basis for somebody
Hi,
On 2023-12-21 11:00:34 -0500, Robert Haas wrote:
> On Thu, Dec 21, 2023 at 10:33 AM Tomas Vondra
> wrote:
> > > I continue to think that we should not have split plain and index only
> > > scans
> > > into separate files...
> >
> > I do agree with that opinion. Not just because of this
On Thu, Dec 21, 2023 at 10:33 AM Tomas Vondra
wrote:
> > I continue to think that we should not have split plain and index only scans
> > into separate files...
>
> I do agree with that opinion. Not just because of this prefetching
> thread, but also because of the discussions about index-only
Hi,
On 2023-12-21 16:20:45 +0100, Tomas Vondra wrote:
> On 12/21/23 14:43, Andres Freund wrote:
> >> AFAICS this seems similar to some of the AIO patch, I wonder what that
> >> plans to do. I need to check.
> >
> > Yes, most of this exists there. The difference that with the AIO you don't
> >
On 12/21/23 14:27, Andres Freund wrote:
> Hi,
>
> On 2023-12-09 19:08:20 +0100, Tomas Vondra wrote:
>> But there's a layering problem that I don't know how to solve - I don't
>> see how we could make indexam.c entirely oblivious to the prefetching,
>> and move it entirely to the executor.
On 12/21/23 14:43, Andres Freund wrote:
> Hi,
>
> On 2023-12-21 13:30:42 +0100, Tomas Vondra wrote:
>> You're right a lot of this is a guesswork. I don't think we can do much
>> better, because it depends on stuff that's out of our control - each OS
>> may do things differently, or perhaps
Hi,
On 2023-12-21 13:30:42 +0100, Tomas Vondra wrote:
> You're right a lot of this is a guesswork. I don't think we can do much
> better, because it depends on stuff that's out of our control - each OS
> may do things differently, or perhaps it's just configured differently.
>
> But I don't
Hi,
On 2023-12-09 19:08:20 +0100, Tomas Vondra wrote:
> But there's a layering problem that I don't know how to solve - I don't
> see how we could make indexam.c entirely oblivious to the prefetching,
> and move it entirely to the executor. Because how else would you know
> what to prefetch?
>
On 12/21/23 07:49, Dilip Kumar wrote:
> On Wed, Dec 20, 2023 at 7:11 AM Tomas Vondra
> wrote:
>>
> I was going through to understand the idea, couple of observations
>
> --
> + for (int i = 0; i < PREFETCH_LRU_SIZE; i++)
> + {
> + entry = >prefetchCache[lru * PREFETCH_LRU_SIZE + i];
> +
> +
On 12/20/23 20:09, Robert Haas wrote:
> On Tue, Dec 19, 2023 at 8:41 PM Tomas Vondra
> ...
>> I have imagined something like this:
>>
>> nodeIndexscan / index_getnext_slot()
>> -> no callback, all TIDs are prefetched
>>
>> nodeIndexonlyscan / index_getnext_tid()
>> -> callback checks VM for the
On Wed, Dec 20, 2023 at 7:11 AM Tomas Vondra
wrote:
>
I was going through to understand the idea, couple of observations
--
+ for (int i = 0; i < PREFETCH_LRU_SIZE; i++)
+ {
+ entry = >prefetchCache[lru * PREFETCH_LRU_SIZE + i];
+
+ /* Is this the oldest prefetch request in this LRU? */
+ if
On Tue, Dec 19, 2023 at 8:41 PM Tomas Vondra
wrote:
> Whatever the right abstraction is, it probably needs to do these VM
> checks only once.
Makes sense.
> Yeah, after you pointed out the "leaky" abstraction, I also started to
> think about customizing the behavior using a callback. Not sure
On 12/18/23 22:00, Robert Haas wrote:
> On Sat, Dec 9, 2023 at 1:08 PM Tomas Vondra
> wrote:
>> But there's a layering problem that I don't know how to solve - I don't
>> see how we could make indexam.c entirely oblivious to the prefetching,
>> and move it entirely to the executor. Because how
On Sat, Dec 9, 2023 at 1:08 PM Tomas Vondra
wrote:
> But there's a layering problem that I don't know how to solve - I don't
> see how we could make indexam.c entirely oblivious to the prefetching,
> and move it entirely to the executor. Because how else would you know
> what to prefetch?
Yeah,
Here's a v5 of the patch, rebased to current master and fixing a couple
compiler warnings reported by cfbot (%lu vs. UINT64_FORMAT in some debug
messages). No other changes compared to v4.
cfbot also reported a failure on windows in pg_dump [1], but it seem
pretty strange:
[11:42:48.708]
Hi,
attached is a v4 of the patch, with a fairly major shift in the approach.
Until now the patch very much relied on the AM to provide information
which blocks to prefetch next (based on the current leaf index page).
This seemed like a natural approach when I started working on the PoC,
but
Hi,
I have results from the new extended round of prefetch tests. I've
pushed everything to
https://github.com/tvondra/index-prefetch-tests-2
There are scripts I used to run this (run-*.sh), raw results and various
kinds of processed summaries (pdf, ods, ...) that I'll mention later.
As
On Thu, Jun 8, 2023 at 9:10 PM Tomas Vondra
wrote:
> We already do prefetching for bitmap index scans, where the bitmap heap
> scan prefetches future pages based on effective_io_concurrency. I'm not
> sure why exactly was prefetching implemented only for bitmap scans, but
> I suspect the
On Thu, 2023-06-08 at 17:40 +0200, Tomas Vondra wrote:
> Hi,
>
> At pgcon unconference I presented a PoC patch adding prefetching for
> indexes, along with some benchmark results demonstrating the (pretty
> significant) benefits etc. The feedback was quite positive, so let me
> share the current
On 6/10/23 22:34, Andres Freund wrote:
> Hi,
>
> On 2023-06-09 12:18:11 +0200, Tomas Vondra wrote:
>>>
2) prefetching from executor
Another question is whether the prefetching shouldn't actually happen
even higher - in the executor. That's what Andres suggested during the
Hi,
On 2023-06-09 12:18:11 +0200, Tomas Vondra wrote:
> >
> >> 2) prefetching from executor
> >>
> >> Another question is whether the prefetching shouldn't actually happen
> >> even higher - in the executor. That's what Andres suggested during the
> >> unconference, and it kinda makes sense.
On Thu, Jun 8, 2023 at 11:40 AM Tomas Vondra
wrote:
> We already do prefetching for bitmap index scans, where the bitmap heap
> scan prefetches future pages based on effective_io_concurrency. I'm not
> sure why exactly was prefetching implemented only for bitmap scans
At the point Greg Stark
On Fri, Jun 9, 2023 at 3:45 AM Tomas Vondra
wrote:
> > What the exact historical timeline is may not be that important. My
> > emphasis on ScalarArrayOpExpr is partly due to it being a particularly
> > compelling case for both parallel index scan and prefetching, in
> > general. There are many
On 6/9/23 01:38, Peter Geoghegan wrote:
> On Thu, Jun 8, 2023 at 3:17 PM Tomas Vondra
> wrote:
>> Normal index scans are an even more interesting case but I'm not
>> sure how hard it would be to get that information. It may only be
>> convenient to get the blocks from the last leaf
On 6/9/23 02:06, Andres Freund wrote:
> Hi,
>
> On 2023-06-08 17:40:12 +0200, Tomas Vondra wrote:
>> At pgcon unconference I presented a PoC patch adding prefetching for
>> indexes, along with some benchmark results demonstrating the (pretty
>> significant) benefits etc. The feedback was quite
On Thu, Jun 8, 2023 at 4:38 PM Peter Geoghegan wrote:
> This is conceptually a "mini bitmap index scan", though one that takes
> place "inside" a plain index scan, as it processes one particular leaf
> page. That's the kind of design that "plain index scan vs bitmap index
> scan as a continuum"
Hi,
On 2023-06-08 17:40:12 +0200, Tomas Vondra wrote:
> At pgcon unconference I presented a PoC patch adding prefetching for
> indexes, along with some benchmark results demonstrating the (pretty
> significant) benefits etc. The feedback was quite positive, so let me
> share the current patch
On Thu, Jun 8, 2023 at 3:17 PM Tomas Vondra
wrote:
> Normal index scans are an even more interesting case but I'm not
> sure how hard it would be to get that information. It may only be
> convenient to get the blocks from the last leaf page we looked at,
> for example.
>
> So this
On 6/8/23 20:56, Peter Geoghegan wrote:
> On Thu, Jun 8, 2023 at 8:40 AM Tomas Vondra
> wrote:
>> We already do prefetching for bitmap index scans, where the bitmap heap
>> scan prefetches future pages based on effective_io_concurrency. I'm not
>> sure why exactly was prefetching implemented only
On Thu, Jun 8, 2023 at 8:40 AM Tomas Vondra
wrote:
> We already do prefetching for bitmap index scans, where the bitmap heap
> scan prefetches future pages based on effective_io_concurrency. I'm not
> sure why exactly was prefetching implemented only for bitmap scans, but
> I suspect the
94 matches
Mail list logo
