On Thu, Dec 5, 2019 at 4:26 PM Kyotaro Horiguchi <horikyota....@gmail.com> wrote: > There's my pending (somewhat stale) patch, which allows to run local > scans while waiting for remote servers. > > https://www.postgresql.org/message-id/20180515.202945.69332784.horiguchi.kyot...@lab.ntt.co.jp > > I (or we) wanted to introduce the asynchronous node mechanism as the > basis of async-capable postgres_fdw. The reason why it is stopping is > that we are seeing and I am waiting the executor change that makes > executor push-up style, on which the async-node mechanism will be > constructed. If that won't happen shortly, I'd like to continue that > work..
After rereading some threads to remind myself what happened here... right, my little patch began life in March 2016[1] when I wanted a test case to test Andres's work on WaitEventSets, and your patch set started a couple of months later and is vastly more ambitious[2][3]. It wants to escape from the volcano give-me-one-tuple-or-give-me-EOF model. And I totally agree that there are lots of reason to want to do that (including yielding to other parts of the plan instead of waiting for I/O, locks and some parallelism primitives enabling new kinds of parallelism), and I'm hoping to help with some small pieces of that if I can. My patch set (rebased upthread) was extremely primitive, with no new planner concepts, and added only a very simple new executor node method: ExecReady(). Append used that to try to ask its children if they'd like some time to warm up. By default, ExecReady() says "I don't know what you're talking about, go away", but FDWs can provide an implementation that says "yes, please call me again when this fd is ready" or "yes, I am ready, please call ExecProc() now". It doesn't deal with anything more complicated than that, and in particular it doesn't work if there are extra planner nodes in between Append and the foreign scan. (It also doesn't mix particularly well with parallelism, as mentioned.) The reason I reposted this unambitious work is because Stephen keeps asking me why we don't consider the stupidly simple thing that would help with simple foreign partition-based queries today, instead of waiting for someone to redesign the entire executor, because that's ... really hard. [1] https://www.postgresql.org/message-id/CAEepm%3D1CuAWfxDk%3D%3DjZ7pgCDCv52fiUnDSpUvmznmVmRKU5zpA%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/CA%2BTgmobx8su_bYtAa3DgrqB%2BR7xZG6kHRj0ccMUUshKAQVftww%40mail.gmail.com [3] https://www.postgresql.org/message-id/flat/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
