Re: [HACKERS] PITR logging control program

2004-05-10 Thread Simon Riggs
Further design plans for PITR...as posted previously, Bruce and I had a long discussion recently to iron out the major thinking and a good deal of the detail also. In overview, major change is introducing an ARCHIVE process running under control of the Postmaster, similar to Stats collector. Due

Re: [HACKERS] PITR logging control program

2004-05-04 Thread Simon Riggs
On Fri, 2004-04-30 at 04:02, Bruce Momjian wrote: Simon Riggs wrote: Agreed we want to allow the superuser control over writing of the archive logs. The question is how do they get access to that. Is it by running a client program continuously or calling an interface script from the

Re: [HACKERS] PITR logging control program

2004-04-30 Thread Simon Riggs
On Fri, 2004-04-30 at 04:02, Bruce Momjian wrote: Let me also add that I am not terribly worried about having the feature to restore to an arbitrary point in time for 7.5. I would much rather have a good PITR solution that works cleanly in 7.5 and add it to 7.6, than to have retore to an

Re: [HACKERS] PITR logging control program

2004-04-30 Thread Zeugswetter Andreas SB SD
Basically it is updating the logs as soon as it receives the notifications. Writing 16 MB of xlogs could take some time. In my experience with archiving logs, 16 Mb is on the contrary way too small for a single log. The overhead of starting e.g. a tape session is so high that you cannot

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Bruce Momjian
Simon Riggs wrote: Agreed we want to allow the superuser control over writing of the archive logs. The question is how do they get access to that. Is it by running a client program continuously or calling an interface script from the backend? My point was that having the backend

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Alvaro Herrera
On Thu, Apr 29, 2004 at 12:18:38AM -0400, Bruce Momjian wrote: OK, I looked over the code. Basically it appears pg_arch is a client-side program that copies files from pg_xlog to a specified directory, and marks completion in a new pg_rlog directory. The driving part of the program seems

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Bruce Momjian
Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 12:18:38AM -0400, Bruce Momjian wrote: OK, I looked over the code. Basically it appears pg_arch is a client-side program that copies files from pg_xlog to a specified directory, and marks completion in a new pg_rlog directory. The

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Alvaro Herrera
On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could take a while to be filled.

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Bruce Momjian
Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill it later. This may be needed because a segment could

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Simon Riggs
On Thu, 2004-04-29 at 15:22, Bruce Momjian wrote: Alvaro Herrera wrote: On Thu, Apr 29, 2004 at 10:07:01AM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: Is the API able to indicate a written but not-yet-filled WAL segment? So an archiver could copy the filled part, and refill

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Bruce Momjian
Simon Riggs wrote: Also, I think this archiver should be able to log to a local drive, network drive (trivial), tape drive, ftp, or use an external script to transfer the logs somewhere. (ftp would probably be an external script with 'expect'). Bruce is correct, the API waits for the

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Alvaro Herrera
On Thu, Apr 29, 2004 at 07:34:47PM +0100, Simon Riggs wrote: Bruce is correct, the API waits for the archive to be full before archiving. I had thought about the case for partial archiving: basically, if you want to archive in smaller chunks, make your log files smaller...this is now a

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Simon Riggs
On Thu, 2004-04-29 at 20:24, Bruce Momjian wrote: Simon Riggs wrote: The archiver should be able to do a whole range of things. Basically, that point was discussed and the agreed approach was to provide an API that would allow anybody and everybody to write whatever they wanted. The

Re: [HACKERS] PITR logging control program

2004-04-29 Thread Simon Riggs
On Thu, 2004-04-29 at 20:24, Bruce Momjian wrote: I am willing to work on this... There is much work still to be done to make PITR work..accepting all of the many comments made. If anybody wants this by 1 June, I think we'd better look sharp. My aim has been to knock one of the URGENT items on