On Sun, Feb 5, 2023 at 7:46 PM Nathan Bossart <nathandboss...@gmail.com> wrote: > Got it. I suspect we'll want to do something similar for archive modules > eventually, too.
+1. I felt like the archive modules work was a step forward when we did, because basic_archive does some things that you're not likely to get right if you do it on your own. And a similar approach to restore_command might be also be valuable, at least in my opinion. However, the gains that we can get out of the archive module facility in its present form do seem to be somewhat limited, for exactly the kinds of reasons being discussed here. I kind of wonder whether we ought to try to flip the model around. At present, the idea is that the archiver is doing its thing and it makes callbacks into the archive module. But what if we got rid of the archiver main loop altogether and put the main loop inside of the archive module, and have it call back to some functions that we provide? One function could be like char *pgarch_next_file_to_be_archived_if_there_is_one_ready(void) and the other could be like void pgarch_some_file_that_you_gave_me_previously_is_now_fully_archived(char *which_one). That way, we'd break the tight coupling where you have to get a unit of work and perform it in full before you can get the next unit of work. Some variant of this could work on the restore side, too, I think, although we have less certainty about how much it makes to prefetch for restore than we do about what needs to be archived. -- Robert Haas EDB: http://www.enterprisedb.com
