Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-29 Thread Greg Stark
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: > Also, I need to be sure that pg_dumpall is enough, and I don't need to make > sure I issue a checkpoint before the pg_dumpall or anything. I think if you're using PITR you don't use pg_dump you just tar up the PGDATA directory. Of course you

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Christopher Kings-Lynne
That gives us enough to talk through and begin some testing. Anybody have any other horror stories, bring 'em on. I think that the PITR docs will have to be written in two sections. One will need to be a pure reference that orthogonally describes the options, etc. The other section will need to

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > PITR should cope with these scenarios. These are described reasonably > closely but not as exact mechanical tests, so as to ensure that if > multiple solutions exist to these recovery scenarios that all paths are > tested. > [ snip ... ] Now *my* head is

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Simon Riggs
On Wed, 2004-07-21 at 23:42, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > More verbosely (not numbered because they're not a sequence or > > progression) > > > - if no recovery.conf is present we do crash recovery to end of logs on > > pg_control timeline (i.e. current) > > Chec

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > More verbosely (not numbered because they're not a sequence or > progression) > - if no recovery.conf is present we do crash recovery to end of logs on > pg_control timeline (i.e. current) Check. > - if recovery.conf is present and we do not specify a t

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-20 Thread Simon Riggs
On Tue, 2004-07-20 at 00:58, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > where the default is and if you specify a target, the default > > target_in_timeline is . > > I think actually the default target has to be the timeline ID found in > pg_control -- otherwise you get weird

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-20 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Anybody see any holes in this design? God help the DBA who deletes a history file with needed information. Or edits it inappropriately. Why can't every log file contain a header that says which timeline it's part of and which timeline the preceding log fil

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 23:15, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Mon, 2004-07-19 at 19:33, Tom Lane wrote: > >> * When we need to do recovery, we first identify the source timeline > >> (either by reading the current timeline ID from pg_control, or the DBA > >> can tel

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I think we're heatedly agreeing again. Yeah, I think so. I'll get started on this tomorrow. > where the default is and if you specify a target, the default > target_in_timeline is . I think actually the default target has to be the timeline ID found in

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2004-07-19 at 19:33, Tom Lane wrote: >> When doing WAL archiving a history file can be copied >> off to the archive area by the existing archiver mechanism (ie, we'll >> make a .ready file for it as soon as it's written). > Need to check the archiv

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > The crucial time is when re-running recoveries repeatedly and if we > write the manual with sufficient red ink then we'll avoid this. But > heck, not having your history file is only as bad as not having added > timelines in the first place. Not great, just

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 19:33, Tom Lane wrote: > I wrote: > > I think there's really no way around the issue: somehow we've got to > > keep some meta-history outside the $PGDATA area, if we want to do this > > in a clean fashion. > > After further thought I think we can fix this stuff by creating a

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 16:58, Tom Lane wrote: > I think there's really no way around the issue: somehow we've got to > keep some meta-history outside the $PGDATA area, if we want to do this > in a clean fashion. We could perhaps expect the archive area to store > it, but I'm a bit worried about the

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
I wrote: > I think there's really no way around the issue: somehow we've got to > keep some meta-history outside the $PGDATA area, if we want to do this > in a clean fashion. After further thought I think we can fix this stuff by creating a "history file" for each timeline. This will make recover

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Some further thinking from that base... > Perhaps timelines should be nest-numbered: (using 0 as a counter also) > 0 etc is the original branch > 0.1 is the first recovery off the original branch > 0.2 is the second recovery off the original branch > 0.1.1

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-19 Thread Simon Riggs
On Mon, 2004-07-19 at 04:31, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > The way you write this makes me think you might mean you would allow: we > > can start recovering in one timelines, then rollforward takes us through > > all the timeline nexus points required to get us to th

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-18 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > The way you write this makes me think you might mean you would allow: we > can start recovering in one timelines, then rollforward takes us through > all the timeline nexus points required to get us to the target > timeline. Sure. Let's draw a diagram:

Re: [HACKERS] Why we really need timelines *now* in PITR

2004-07-18 Thread Simon Riggs
On Sat, 2004-07-17 at 21:36, Tom Lane wrote: > If we do not add timeline numbers to WAL file names, we will be forced > to destroy information during recovery. Consider the following > scenario: > > 1. You have a WAL directory containing, say, WAL segments 0010 to 0020 > (for the purposes of this

[HACKERS] Why we really need timelines *now* in PITR

2004-07-17 Thread Tom Lane
If we do not add timeline numbers to WAL file names, we will be forced to destroy information during recovery. Consider the following scenario: 1. You have a WAL directory containing, say, WAL segments 0010 to 0020 (for the purposes of this example I won't bother typing out realistic 16-digit fil