On Wed, Sep 4, 2019 at 06:18:31PM +1200, Thomas Munro wrote: > Hello, > > A few years back[1] I experimented with a simple readiness API that > would allow Append to start emitting tuples from whichever Foreign > Scan has data available, when working with FDW-based sharding. I used > that primarily as a way to test Andres's new WaitEventSet stuff and my > kqueue implementation of that, but I didn't pursue it seriously > because I knew we wanted a more ambitious async executor rewrite and > many people had ideas about that, with schedulers capable of jumping > all over the tree etc. > > Anyway, Stephen Frost pinged me off-list to ask about that patch, and > asked why we don't just do this naive thing until we have something > better. It's a very localised feature that works only between Append > and its immediate children. The patch makes it work for postgres_fdw, > but it should work for any FDW that can get its hands on a socket. > > Here's a quick rebase of that old POC patch, along with a demo. Since > 2016, Parallel Append landed, but I didn't have time to think about > how to integrate with that so I did a quick "sledgehammer" rebase that > disables itself if parallelism is in the picture.
Yes, sharding has been waiting on parallel FDW scans. Would this work for parallel partition scans if the partitions were FDWs? -- Bruce Momjian <br...@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
