Re: [PATCHES] PITR Archive Recovery plus WIP PITR
I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. We really need to get this right the first time. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Please ignore- seems some old mail of mine got sent waaay late... Christopher Kings-Lynne wrote: I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. We really need to get this right the first time. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
On Wed, 2004-07-14 at 05:45, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. I assume they could just restore from backup and try again. Sure, if they don't mind losing whatever transactions they processed before realizing how broken their database was. That's not going to be an acceptable answer for the sort of installations that need PITR in the first place. I think it's really important to get this right the first time, both for reliability's sake and because we are expecting people to write their own archiving scripts. If we change the xlog segment naming convention later on, then we will break all those scripts. I agree, but I'm going to have a rest day while people test what is already there in case there are further code changeswhich nods towards both of your concerns. BTW, one test last night broke because of the lack of timelines... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. I assume they could just restore from backup and try again. Sure, if they don't mind losing whatever transactions they processed before realizing how broken their database was. That's not going to be an acceptable answer for the sort of installations that need PITR in the first place. I think it's really important to get this right the first time, both for reliability's sake and because we are expecting people to write their own archiving scripts. If we change the xlog segment naming convention later on, then we will break all those scripts. We don't have anything hardcoded based on those file names, at last in PostgreSQL. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think it's really important to get this right the first time, both for reliability's sake and because we are expecting people to write their own archiving scripts. If we change the xlog segment naming convention later on, then we will break all those scripts. We don't have anything hardcoded based on those file names, at last in PostgreSQL. That's because we've punted the whole problem of archive-segment management off to the users. If we did implement this functionality ourselves then I'd be less worried, since we'd know that future changes would affect only our own code. But as things stand, we will have very unhappy PITR users if we change the naming convention later. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
On Wed, 2004-07-14 at 16:00, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think it's really important to get this right the first time, both for reliability's sake and because we are expecting people to write their own archiving scripts. If we change the xlog segment naming convention later on, then we will break all those scripts. We don't have anything hardcoded based on those file names, at last in PostgreSQL. Well, I think we do. There's two places where the filename format and length matters and there are numerous calls to calculate filenames from log,seg pairs. ...and much of the current patch would need reworking thoroughly to make sure all differences were changed. Which is why I was striving for a solution that retained the filename make-up, by adding a very large number to the log value (we just aren't gonna run out...I did the math in another post). That's because we've punted the whole problem of archive-segment management off to the users. If we did implement this functionality ourselves then I'd be less worried, since we'd know that future changes would affect only our own code. But as things stand, we will have very unhappy PITR users if we change the naming convention later. Yes, if we are going to change the xlog filename format, we must do it now. The change must be in effect whether or not you use archive_mode. ...Is there agreement with my previous posts on thismarked Point in Time Recovery over the last few days? Is that what we should implement? Overall, the timeline concept is only worth implementing if: - we archive xlogs - we recover them - we recover them to a point in time/txnid We agreed that the last part wasn't the priority for beta freeze. I'm willing to spend more time on the timeline idea as long as I've got some idea that we will be committing what has been developed so far. It takes effort to keep the patch viable against changes because new commits update the catalog version, which invalidates all my test databases, as well as any changes I have to track down. ...and I've been doing that for a month now - getting much better though, thanks. If we can review what we have now, I would be most pleased. Until we commit at least some of it, I'm the only developer and I would like to open this up to allow others to contribute more easily. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Following a suggestion and patch from Klaus Naumann, the recovery.conf file can now accept comments No patch supplied at present (anoncvs is down), but here is the annotated recovery.conf.sample Best Regards, Simon Riggs # --- # PostgreSQL recovery config file # --- # # Edit this file to provide the parameters that PostgreSQL # needs to perform an archive recovery of a database # # This file consists of lines of the form: # # name = value # # (The ' = ' is NOT optional.) # # Comments are introduced with '#' at beginning of a line. # # The complete list of option names and # allowed values can be found in the PostgreSQL documentation. The # commented-out settings shown in this file show optional values. # # These options cannot be specified in any other way. There are # no command line switches, to ensure that when recovery completes # we do not accidentally re-enter archive recovery. # # This file is read on postmaster startup. # #--- # REQUIRED PARAMETERS #--- # # restore program # # specifies the program that is called to copy log files # back from archive for use # command string must specify 3 parameters, using %s # parameter1 will be substituted with archive_dest # parameter2 will be substituted with the single log file # that will be restored (one file per execution) # parameter3 will be substituted with the full path of # the file to which the restore program will restore # # #restore_program = 'cp %s/%s %s' # # #--- # RECOVERY TARGET #--- # # By default, recovery will rollforward to the end of logs # If you want to stop rollforward before that point, you # MUST set a recovery target. # # You may set a recovery target either by transactionId, or # by timestamp. Recovery may either include or exclude the # records with the recovery target value. # # #recovery_target_time = 'YY-MM-DD-hh:mm:ss' #recovery_target_txnid = '1256166' # # true or false #recovery_target_inclusive = 'Y' # #--- ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
On Tue, 2004-07-13 at 23:58, Bruce Momjian wrote: Simon Riggs wrote: New release of patch, at v5_1 ... for serious testing what's in - Point in Time Recovery now worksplease check carefully - additional options in recovery.conf (including code contributed to PITR from Klaus Naumann) what's not (yet) - Timelines...though I think they are useful, they may not be critical I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. Well, I really want to finish this, so I do agree. Exhaustion is setting inI need other eyes to test and fix the bugs. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Simon Riggs wrote: New release of patch, at v5_1 ... for serious testing what's in - Point in Time Recovery now worksplease check carefully - additional options in recovery.conf (including code contributed to PITR from Klaus Naumann) what's not (yet) - Timelines...though I think they are useful, they may not be critical I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Bruce Momjian [EMAIL PROTECTED] writes: Simon Riggs wrote: what's not (yet) - Timelines...though I think they are useful, they may not be critical I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. We really need to get this right the first time. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Simon Riggs wrote: what's not (yet) - Timelines...though I think they are useful, they may not be critical I am not fond of the timeline idea, especially for 7.5. Let's get usage cases submitted first. I can imagine timelines as causing significant confusion during restore, which is the last thing we want to do. I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. We really need to get this right the first time. I assume they could just restore from backup and try again. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I think that judgment is exactly backward. *Not* having timelines is what will cause serious and possibly fatal mistakes during restore: people will hand the wrong xlog files to restore and the software will be unable to recognize the inconsistency. I assume they could just restore from backup and try again. Sure, if they don't mind losing whatever transactions they processed before realizing how broken their database was. That's not going to be an acceptable answer for the sort of installations that need PITR in the first place. I think it's really important to get this right the first time, both for reliability's sake and because we are expecting people to write their own archiving scripts. If we change the xlog segment naming convention later on, then we will break all those scripts. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] PITR Archive Recovery plus WIP PITR
On Fri, 2004-07-09 at 12:53, Klaus Naumann wrote: archive_program is provided with a string which contains the target directory. That doesn't really make sense. archive_dest is used for both archive and restore, thats why its set as a separate parameter. Thats the rationale...lets see what others think First of all it introduces the problem you mentioned in the README file (if the directory doesn't exist you loose xlogs). Your example quoted later is the answer use archive_dest = '/mnt/pgarch/' rather than archive_dest = '/mnt/pgarch' which is ambiguous... I thought about checking if this is a dir within the code. But this would make things too unfelxible. Yes, otherwise the check would be there Second, we could make the user responsible of what he's doing by not giving him any target. Remember, the user is specifying the archive_dest also, so the user is completely responsible for how archiving actually occurs. Like you could then do things like: archive_program = 'gzip -d %s | tar rf /dev/nst0 - ' archive_program = 'gzip -d %s | tar rf %s - ' would be how I would use it in the example you give Which adds the file to a tar archive on his tape. If he wants to archive it on disk, let him do it this way: archive_program = 'cp %s /mnt/pgarch/' archive_program = 'cp %s %s' would be the way to specify that... Thank you very much for feedback and your other contributions, Best regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
