On Wed, Jul 16, 2025 at 10:37 AM Tomas Vondra <to...@vondra.me> wrote: > What sounds weird? That the read_stream works like a stream of blocks, > or that it can't do "pause" and we use "reset" as a workaround?
The fact that prefetch distance is in any way affected by a temporary inability to return more blocks. Just starting from scratch seems particularly bad. Doesn't that mean that it's simply impossible for us to remember ramping up the distance on an earlier leaf page? There is nothing about leaf page boundaries that should be meaningful to the read stream/our heap accesses. I get that index characteristics could be the limiting factor, especially in a world where we're not yet eagerly reading leaf pages. But that in no way justifies just forgetting about prefetch distance like this. > >> In an ideal world we'd have a function that'd "pause" the stream, > >> without resetting the distance etc. But we don't have that, and the > >> reset thing was suggested to me as a workaround. > > > > Does the "complex" patch require a similar workaround? Why or why not? > > > > I think it'll need to do something like that in some cases, when we need > to limit the number of leaf pages kept in memory to something sane. That's the only reason? The memory usage for batches? That doesn't seem like a big deal. It's something to keep an eye on, but I see no reason why it'd be particularly difficult. Doesn't this argue for the "complex" patch's approach? -- Peter Geoghegan
