Re: [HACKERS] Forcing current WAL file to be archived

2006-07-31 Thread Simon Riggs
On Tue, 2006-08-01 at 00:40 +0300, Hannu Krosing wrote: > Ühel kenal päeval, T, 2006-07-25 kell 17:05, kirjutas Simon Riggs: > > On Tue, 2006-07-25 at 11:53 -0400, Tom Lane wrote: > > > That's fine, but feature freeze is in a week and we don't even have > > > the > > > basic function for manually d

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-31 Thread Simon Riggs
On Tue, 2006-08-01 at 00:40 +0300, Hannu Krosing wrote: > Ühel kenal päeval, T, 2006-07-25 kell 17:05, kirjutas Simon Riggs: > > On Tue, 2006-07-25 at 11:53 -0400, Tom Lane wrote: > > > That's fine, but feature freeze is in a week and we don't even have > > > the > > > basic function for manually d

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-31 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-07-25 kell 17:05, kirjutas Simon Riggs: > On Tue, 2006-07-25 at 11:53 -0400, Tom Lane wrote: > > That's fine, but feature freeze is in a week and we don't even have > > the > > basic function for manually doing a log file switch. Let's get that > > done first and then th

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Stephen Frost
* Bruce Momjian ([EMAIL PROTECTED]) wrote: > Tom Lane wrote: > > You are assuming here that the continuous archiving process is identical > > to the WAL part of the base-backup process. If what you want is an > > identifiable self-contained base backup then you copy off the WAL files > > along wit

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Albe Laurenz
Tom Lane wrote: >> The point is until that last WAL file is backed up, the whole backup is >> useless. It isn't good policy to have a backup's value be contingent on >> some future event. > > You are assuming here that the continuous archiving process is identical > to the WAL part of the base-bac

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Assuming such a case, would it be possible to have two functions? > > > pg_stop_backup() > > pg_stop_backup(boolean); --parameter says log switch or not > > Well, it seems everyone but me thinks that pg_stop_backup should > force a WAL

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Assuming such a case, would it be possible to have two functions? > pg_stop_backup() > pg_stop_backup(boolean); --parameter says log switch or not Well, it seems everyone but me thinks that pg_stop_backup should force a WAL switch, so I'll yield on that p

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, 2006-07-25 at 11:57 -0400, Bruce Momjian wrote: > > Tom Lane wrote: > > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > > I was planning to add a new GUC > > > > archive_timeout (integer) = max # secs between log file switches > > > > > > That's fine, but featu

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Csaba Nagy
> > The problems I see with this is if in this case the normal postgres > > WAL > > archiving won't conflict with this streaming ? > > You are not forced to use it if your shell scripts do conflict. > > What I envisioned, was that the current WAL archiving shell script would > just do some CRC c

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Tue, 2006-07-25 at 11:31 -0400, Tom Lane wrote: > >> My point here is that forcing the current segment to archive is a > >> function of whatever your continuous-archiving process is, and it's > >> not necessarily tied to backups. We

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:57 -0400, Bruce Momjian wrote: > Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > I was planning to add a new GUC > > > archive_timeout (integer) = max # secs between log file switches > > > > That's fine, but feature freeze is in a week and we don't eve

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-07-25 kell 17:52, kirjutas Csaba Nagy: > > OK, "offset" added to TODO item. What would the offset give us? > > The last offset could be remembered by the external program, and it > only > has to transfer from the last offset to the new one. It allows > incremental strea

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2006-07-25 at 11:31 -0400, Tom Lane wrote: >> My point here is that forcing the current segment to archive is a >> function of whatever your continuous-archiving process is, and it's >> not necessarily tied to backups. We should not prejudge when p

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:45 -0400, Tom Lane wrote: > there are scenarios in which you don't need a WAL > switch at the end of a backup. My mind's blank today, so forgive me that I cannot see what that might be. Assuming such a case, would it be possible to have two functions? pg_stop_backup() p

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-07-25 kell 11:48, kirjutas Bruce Momjian: > Hannu Krosing wrote: > > ?hel kenal p?eval, T, 2006-07-25 kell 11:27, kirjutas Bruce Momjian: > > > Hannu Krosing wrote: > > > > ?hel kenal p?eval, T, 2006-07-25 kell 10:51, kirjutas Bruce Momjian: > > > > > Where are we on thes

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:53 -0400, Tom Lane wrote: > That's fine, but feature freeze is in a week and we don't even have > the > basic function for manually doing a log file switch. Let's get that > done first and then think about automatic switches. Agreed. -- Simon Riggs Ente

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
OK, makes sense. That is much more sophisticated that I imagined. --- Csaba Nagy wrote: > > OK, "offset" added to TODO item. What would the offset give us? > > The last offset could be remembered by the external program,

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Csaba Nagy
> OK, "offset" added to TODO item. What would the offset give us? The last offset could be remembered by the external program, and it only has to transfer from the last offset to the new one. It allows incremental streaming of the WAL files... of course the external program will be a lot more com

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, 2006-07-25 at 11:31 -0400, Tom Lane wrote: > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > For example, if you do pg_stop_backup(), in what cases would you not > > > also call pg_finish_wal_segment()? I can't think of one. > > > > I can't see why you would need to

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Stephen Frost
* Simon Riggs ([EMAIL PROTECTED]) wrote: > On Tue, 2006-07-25 at 11:20 -0400, Bruce Momjian wrote: > > Yes, perhaps, though I can envision a GUC that does regularly partial > > archiving. I will add a question mark to the item. > > I was planning to add a new GUC > > archive_timeout (inte

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I was planning to add a new GUC > > archive_timeout (integer) = max # secs between log file switches > > That's fine, but feature freeze is in a week and we don't even have the > basic function for manually doing a log file switch.

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I was planning to add a new GUC > archive_timeout (integer) = max # secs between log file switches That's fine, but feature freeze is in a week and we don't even have the basic function for manually doing a log file switch. Let's get that done first

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Tue, 2006-07-25 at 11:07 -0400, Tom Lane wrote: > >> I see no need for that to be "automatic". I'd vote for a simple > >> function pg_finish_wal_segment() or something like that, which you > >> call just after pg_stop_backup() if you

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:31 -0400, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > For example, if you do pg_stop_backup(), in what cases would you not > > also call pg_finish_wal_segment()? I can't think of one. > > I can't see why you would need to, unless your intention is not

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2006-07-25 at 11:07 -0400, Tom Lane wrote: >> I see no need for that to be "automatic". I'd vote for a simple >> function pg_finish_wal_segment() or something like that, which you >> call just after pg_stop_backup() if you want this behavior. Tryi

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Hannu Krosing wrote: > ?hel kenal p?eval, T, 2006-07-25 kell 11:27, kirjutas Bruce Momjian: > > Hannu Krosing wrote: > > > ?hel kenal p?eval, T, 2006-07-25 kell 10:51, kirjutas Bruce Momjian: > > > > Where are we on these TODO items: > > > > > > > > > > > o Add reporting of the current

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-07-25 kell 11:27, kirjutas Bruce Momjian: > Hannu Krosing wrote: > > ?hel kenal p?eval, T, 2006-07-25 kell 10:51, kirjutas Bruce Momjian: > > > Where are we on these TODO items: > > > > > > > > o Add reporting of the current WAL file, perhaps as part of > > >

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:07 -0400, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Where are we on these TODO items: > > > o Allow point-in-time recovery to archive partially filled > > write-ahead logs [pitr] > > I believe we'd agreed that the necessary infr

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Simon Riggs
On Tue, 2006-07-25 at 11:20 -0400, Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > Where are we on these TODO items: > > > > > o Allow point-in-time recovery to archive partially filled > > > write-ahead logs [pitr] > > > > I believ

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > For example, if you do pg_stop_backup(), in what cases would you not > also call pg_finish_wal_segment()? I can't think of one. I can't see why you would need to, unless your intention is not to run PITR at all but only to make a filesystem backup inste

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Hannu Krosing wrote: > ?hel kenal p?eval, T, 2006-07-25 kell 10:51, kirjutas Bruce Momjian: > > Where are we on these TODO items: > > > > > o Add reporting of the current WAL file, perhaps as part of > > partial log file archiving > > It would be nice to have a function tha

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I assumed we would have a function like pg_finish_wal_segment(), and > > server stop and stop_backup would call it too, > > That idea is *exactly* what I'm objecting to. > > > the reason being, it > > would greatly simplify our docum

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-07-25 kell 10:51, kirjutas Bruce Momjian: > Where are we on these TODO items: > > o Add reporting of the current WAL file, perhaps as part of > partial log file archiving It would be nice to have a function that tells both filename and offset of c

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > I assumed we would have a function like pg_finish_wal_segment(), and > server stop and stop_backup would call it too, That idea is *exactly* what I'm objecting to. > the reason being, it > would greatly simplify our documentation on how to use PITR if t

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Where are we on these TODO items: > > > o Allow point-in-time recovery to archive partially filled > > write-ahead logs [pitr] > > I believe we'd agreed that the necessary infrastructure for this is > just a fun

Re: [HACKERS] Forcing current WAL file to be archived

2006-07-25 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Where are we on these TODO items: > o Allow point-in-time recovery to archive partially filled > write-ahead logs [pitr] I believe we'd agreed that the necessary infrastructure for this is just a function to tell the current WAL se