On Thu, Sep 7, 2017 at 2:47 PM, Alvaro Herrera <alvhe...@alvh.no-ip.org> wrote:
> After reading this discussion, I agree that pg_rewind needs to become > smarter in order to be fully useful in production environments; right > now there are many warts and corner cases that did not seem to have been > considered during the initial development (which I think is all right, > taking into account that its mere concept was complicated enough; so we > need not put any blame on the original developers, rather the contrary). > Agreed with this assessment. And as a solution to the problem of "base backups take too long to take and transfer" the solution and the corner cases make a lot of sense. > > I think we need to make the program simple to use (i.e. not have the > user write shell globs for the common log file naming patterns) while > remaining powerful (i.e. not forcibly copy any files that do not match > hardcoded globs). I would add that well-defined tasks are a key aspect of powerful software in my view and here the well defined task is to restore data states to a particular usable timeline point taken from another system. If that is handled well, that opens up new uses and makes some problems that are difficult right now much easier to solve. > Is the current dry-run mode enough to give the user > peace of mind regarding what would be done in terms of testing globs > etc? If not, maybe the debug mode needs to be improved (for instance, > have it report the file size for each file that would be copied; > otherwise you may not notice it's going to copy the 20GB log file until > it's too late ...) > The dry run facility solves one problem in one circumstance, namely a manually invoked run of the software along with the question of "will this actually re-wind?" I suppose software developers might be able to use it to backup and restore things that are to be clobbered (but is anyone likely to on the software development side?). I don't see anything in that corner that can be improved without over engineering the solution. There are two reasons I am skeptical that a dry-run mode will ever be "enough." The first is that pg_rewind is often integrated into auto-failover/back tools and the chance of running it in a dry-run mode before it is automatically triggered is more or less nil. These are also the cases where you won't notice it does something bad until much later. The second is that there are at least some corner cases we may need to define as outside the responsibility of pg_rewind. The one that comes to mind here is if I am rewinding back past the creation of a small table. I don't see an easy or safe way to address that from inside pg_rewind without a lot of complication. It might be better to have a dedicated tool for that. > > Now, in order for any of this to happen, there will need to be a > champion to define what the missing pieces are, write all those patches > and see them through the usual (cumbersome, slow) procedure. Are you, > Chris, willing to take the lead on that? > Yeah. I think the first step would be list of the corner cases and a proposal for how I think it should work. Then maybe a roadmap of patches, and then submitting them as they become complete. > -- > Álvaro Herrera https://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services > -- Best Regards, Chris Travers Database Administrator Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com Saarbrücker Straße 37a, 10405 Berlin
