Re: Processing btree walks as a batch to parallelize IO

2021-05-07 Thread Peter Geoghegan
On Fri, May 7, 2021 at 3:34 PM Greg Stark wrote: > We've talked before about buffering inserts even just for disk-based > indexes. Much like how GIN buffers inserts and periodically flushes > them out. We talked about doing a local buffer in each session since > no other session even needs to see

Re: Processing btree walks as a batch to parallelize IO

2021-05-07 Thread Greg Stark
On Fri, 9 Apr 2021 at 16:58, Tomas Vondra wrote: > > > > On 4/9/21 7:33 PM, James Coleman wrote: > > A specific area where this is particularly painful is btree index reads. > > Walking the tree to leaf pages isn't naturally prefetchable, and so for > > each level you pay the random page cost.

Re: Processing btree walks as a batch to parallelize IO

2021-05-07 Thread James Coleman
On Fri, Apr 9, 2021 at 4:57 PM Tomas Vondra wrote: > > > > On 4/9/21 7:33 PM, James Coleman wrote: > > $SUBJECT is still a very loosely formed idea, so forgive lack of detail > > or things I've likely missed, but I wanted to get it out there to see if > > it sounded at all intriguing to people. >

Re: Processing btree walks as a batch to parallelize IO

2021-04-09 Thread Tomas Vondra
On 4/9/21 7:33 PM, James Coleman wrote: > $SUBJECT is still a very loosely formed idea, so forgive lack of detail > or things I've likely missed, but I wanted to get it out there to see if > it sounded at all intriguing to people.  > > Background: One of the big problems with non-local storage

Processing btree walks as a batch to parallelize IO

2021-04-09 Thread James Coleman
$SUBJECT is still a very loosely formed idea, so forgive lack of detail or things I've likely missed, but I wanted to get it out there to see if it sounded at all intriguing to people. Background: One of the big problems with non-local storage such as AWS EBS volumes or a SAN is that in a large