Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-18 Thread J. R. Nield
On Thu, 2002-07-18 at 15:36, Richard Tucker wrote: > Sorry, I didn't delimit my comments correctly before. > I'm offering our pg_copy/roll forward implementation to PostgreSQL.org if it > finds it an acceptable contribution. Progress/Multera would hand over all > rights to any code accepted. > >

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-18 Thread Richard Tucker
; PostgreSQL Hacker Subject: Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR) Richard: I can't quite follow this; maybe you sent a draft by accident. If you want to post a patch against 7.2.1, or even better against HEAD in CVS, that would be great. Or if you'd rather p

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-18 Thread J. R. Nield
ED]] > Sent: Wednesday, July 17, 2002 8:13 PM > To: [EMAIL PROTECTED] > Cc: Bruce Momjian > Subject: RE: [HACKERS] Issues Outstanding for Point In Time Recovery > (PITR) > > > On Wed, 2002-07-17 at 19:25, Richard Tucker wrote: > > Regarding hot backup. Our implementat

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Bruce Momjian
J. R. Nield wrote: > I will do something like this, but reserve 16 bytes for it just in case > we change our minds. It needs to be different among systems on the same > machine, so there needs to be a time value and a pseudo-random part as > well. Also, 'hostname' will likely be the same on many m

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread J. R. Nield
On Wed, 2002-07-17 at 01:25, Bruce Momjian wrote: > > We only patch configure.in. If you post to hackers, they can give you > assistance and I will try to help however I can. I can so some > configure.in stuff for you myself. Thanks for the offer. The only thing I was changing it for was to te

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Richard Tucker
nt In Time Recovery and not multera marketing spin. -regards richt -Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED]] Sent: Wednesday, July 17, 2002 6:29 PM To: Rod Taylor Cc: PostgreSQL-development; [EMAIL PROTECTED]; J. R. Nield Subject: Re: [HACKERS] Issues Outstanding fo

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread D'Arcy J.M. Cain
On July 17, 2002 05:45 pm, Richard Tucker wrote: > We also have implemented a roll forward recovery mechanism. We modified a > 7.2.1 version of Postgres. > ... Excellent! I can't wait. When will it be in current? > The BIG THING we have not done is address the issue that add/drop tables > and

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Bruce Momjian
Rod Taylor wrote: > > server powered by PostgreSQL, the third most popular database, and a > > > >^ > > > Wonder why we are "the third most popular database". I think that's > > good? > > You'll notice they didn't qualify where. On this list, it's probably > #1.

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Rod Taylor
> server powered by PostgreSQL, the third most popular database, and a > >^ > Wonder why we are "the third most popular database". I think that's > good? You'll notice they didn't qualify where. On this list, it's probably #1. Within Progress software perhaps we'

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Bruce Momjian
Bruce Momjian wrote: > Richard Tucker wrote: > > > > We also have implemented a roll forward recovery mechanism. We modified a > > 7.2.1 version of Postgres. > > The mechanism is designed to provide a means of recoverying from the loss or > > corruption of media. It provides for duplicating wal_

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Bruce Momjian
Richard Tucker wrote: > > We also have implemented a roll forward recovery mechanism. We modified a > 7.2.1 version of Postgres. > The mechanism is designed to provide a means of recoverying from the loss or > corruption of media. It provides for duplicating wal_files so that if a > wal_file is

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-17 Thread Richard Tucker
We also have implemented a roll forward recovery mechanism. We modified a 7.2.1 version of Postgres. The mechanism is designed to provide a means of recoverying from the loss or corruption of media. It provides for duplicating wal_files so that if a wal_file is lost roll forward recovery can rec

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-16 Thread Bruce Momjian
J. R. Nield wrote: > On Tue, 2002-07-16 at 15:36, Bruce Momjian wrote: > > > > J.R., just checking to see how PITR recovery is going. Do you need any > > assistance or have any questions for us? > > > > Also, do you have any idea how close you are to having something > > completed? Are you awa

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-16 Thread Tom Lane
"J. R. Nield" <[EMAIL PROTECTED]> writes: > Related to that, the other place I need advice is on adding Ted Tso's > LGPL'd UUID library (stolen from e2fsprogs) to the source. Are we > allowed to use this? Uh, why exactly is UUID essential for this? (The correct answer is "it's not", IMHO.) > We

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-16 Thread J. R. Nield
On Tue, 2002-07-16 at 15:36, Bruce Momjian wrote: > > J.R., just checking to see how PITR recovery is going. Do you need any > assistance or have any questions for us? > > Also, do you have any idea how close you are to having something > completed? Are you aware we are closing development of

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-16 Thread Bruce Momjian
J.R., just checking to see how PITR recovery is going. Do you need any assistance or have any questions for us? Also, do you have any idea how close you are to having something completed? Are you aware we are closing development of 7.3 at the end of August and start beta September 1? Is there

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-09 Thread Hannu Krosing
On Tue, 2002-07-09 at 17:26, Tom Lane wrote: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > 1) record the lowest uncommitted transaction number (LUTN) , this may > > have problems with wraparound, but I guess they are solvable. Disllow > > VACUUM. Do a CHECKPOINT ('alter database begin backup') >

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-09 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > 1) record the lowest uncommitted transaction number (LUTN) , this may > have problems with wraparound, but I guess they are solvable. Disllow > VACUUM. Do a CHECKPOINT ('alter database begin backup') > 3) make a file-level (.tar) backup of data directory

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-08 Thread Barry Lind
I know that in Oracle there are 'alter database begin backup' and 'alter database end backup' commands that allow you to script your hot backups through a cron job by calling the begin backup command first, then using disk backup method of choice and then finally call the end backup command. -

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-08 Thread Bruce Momjian
Zeugswetter Andreas SB SD wrote: > > > OK, so you do a tar backup of a file. While you are doing the tar, > > certain 8k blocks are being modified in the file. There is no way to > > know what blocks are modified as you are doing the tar, and in fact you > > could read partial page writes durin

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-08 Thread Patrick Macdonald
Zeugswetter Andreas SB SD wrote: > > > As noted, one of the main problems is knowing where to begin > > in the log. This can be handled by having backup processing > > update the control file with the first lsn and log file > > required. At the time of the backup, this information is or > > can

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-08 Thread Zeugswetter Andreas SB SD
> As noted, one of the main problems is knowing where to begin > in the log. This can be handled by having backup processing > update the control file with the first lsn and log file > required. At the time of the backup, this information is or > can be made available. The control file can

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-08 Thread Zeugswetter Andreas SB SD
> OK, so you do a tar backup of a file. While you are doing the tar, > certain 8k blocks are being modified in the file. There is no way to > know what blocks are modified as you are doing the tar, and in fact you > could read partial page writes during the tar. No, I think all OS's (Unix and

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-07 Thread Bruce Momjian
Patrick Macdonald wrote: > The idea of using the last lsn on the page to detect a partial > write is used by other dbms systems. You already have that > information available so there is no overhead in computing it. > Nothing wrong with CRC though. Agreed. Just thought I would point out that

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-06 Thread Patrick Macdonald
J.R., Nice first draft and a good read. Was going to comment in-line but thought this method would be easier to follow. The comments/suggestions below assume that PIT recovery is being performed at the cluster level with a data backup image created by a tar-like utility. As noted, one of the

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-05 Thread Bruce Momjian
Bruce Momjian wrote: > You are saying, "How do we know what WAL records go with that backup > snapshot of the file?" OK, lets assume we are shutdown. You can grab > the WAL log info from pg_control using contrib/pg_controldata and that > tells you what WAL logs to roll forward when you need to P

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-05 Thread Bruce Momjian
J. R. Nield wrote: > On Fri, 2002-07-05 at 01:42, Bruce Momjian wrote: > > > > We have needed > > point-in-time recovery for a long time, > > Most thanks should go to vadim (and whoever else worked on this), since > his WAL code already does most of the work. The key thing is auditing > the bac

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-05 Thread Zeugswetter Andreas SB SD
> Let me re-write it, and I'll post it in the next version. The section > dealt with what to do when you have a valid restored controlfile from a > backup system, which is in the DB_SHUTDOWNED state, and that points to a > valid shutdown/checkpoint record in the log; only the checkpoint record >

Re: [HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-04 Thread J. R. Nield
On Thu, 2002-07-04 at 11:45, J. R. Nield wrote: One other item that should be here: > The big items so-far are: > §1 - Logging Relation file creation, truncation, and removal > This is mostly done. Can do infinte play-forward from > online logs. > §2 -

[HACKERS] Issues Outstanding for Point In Time Recovery (PITR)

2002-07-04 Thread J. R. Nield
Hello: I've got the logging system to the point where I can take a shutdown consistent copy of a system, and play forward through multiple checkpoints. It seems to handle CREATE TABLE/DROP TABLE/TRUNCATE properly, and things are moving forward well. Recovery to an arbitrary point-in-time should b