Re: Backing up PostgreSQL?

2002-06-27 Thread Fathi Ben Nasr
Probably mirroring the database and then making a backup of the mirror would allow you to never stop the master server but just the slave. I don't know if it is possible to have a slave running on a different port on the same server is possible or not. If not, you will need and second computer.

Re: Backing up PostgreSQL?

2002-06-22 Thread Martín Marqués
On Vie 21 Jun 2002 20:32, Greg A. Woods wrote: [ On Friday, June 21, 2002 at 18:00:19 (-0300), Martín Marqués wrote: ] Other UNIX? If you could rely on it, why does Informix (and other database servers, but this is the one I used) still use it's own FS to write the database files?

Re: Backing up PostgreSQL?

2002-06-22 Thread Ragnar Kjørstad
On Fri, Jun 21, 2002 at 08:43:27PM -0400, Greg A. Woods wrote: There are no writes to the filesystem at all while the snapshot is in progress. The LVM-driver will block all data-access until it is completed. If not, it wouldn't be a snapshot, would it? I realize that -- but it doesn't

Re: Backing up PostgreSQL?

2002-06-22 Thread Ragnar Kjørstad
On Fri, Jun 21, 2002 at 08:30:44PM -0400, Greg A. Woods wrote: It should only require REDO. No changes are made to the actual database-files until the transaction is commited, written to the WAL and fsynced. At this point there is no longer a need for UNDO. Hmmm possibly. I'm not

Re: Backing up PostgreSQL?

2002-06-22 Thread Ragnar Kjørstad
On Fri, Jun 21, 2002 at 09:01:17PM -0400, Michael H.Collins wrote: I thought i just read that postgresql on ext3 outran oracle on raw devices. Yes, it's funny how all database engines are faster than all other database engines, isn't it? ~Because there are tremendous performance advantages

Re: Backing up PostgreSQL?

2002-06-21 Thread Kirk Strauser
At 2002-06-21T04:36:19Z, [EMAIL PROTECTED] (Greg A. Woods) writes: Wasn't this question originally about FreeBSD anyway? It was, but this is pretty interesting too. Feel free to continue. :) -- Kirk Strauser The Strauser Group - http://www.strausergroup.com/

Re: Backing up PostgreSQL?

2002-06-21 Thread Paul Jakma
On Fri, 14 Jun 2002, Greg A. Woods wrote: That's irrelevant from PostgreSQL's point of view. There's no sure way to tell the postgresql process(es) to make the on-disk database image consistent before you create the snapshot. if it's pg 7.1+ then it has write-ahead-logging, and the on-disk

Re: Backing up PostgreSQL?

2002-06-21 Thread Paul Jakma
On Sat, 15 Jun 2002, Greg A. Woods wrote: Unless you can do a global database lock at a point where there are no open transactions and no uncommitted data in-core, the only way to guarantee a consistent database on disk is to close all the database files. Even then if you're very paranoid

Re: Backing up PostgreSQL?

2002-06-21 Thread Paul Jakma
On Thu, 20 Jun 2002, Greg A. Woods wrote: Where does it say that close/open will flush metadata? That's how the unix filesystem works. UTSL. that's an implementation issue - you cant rely on it. portable synchronous metadata is a murky issue on plain unix filesystems. maybe above works,

Re: Backing up PostgreSQL?

2002-06-21 Thread Greg A. Woods
[ On Friday, June 21, 2002 at 18:37:20 (+0100), Paul Jakma wrote: ] Subject: Re: Backing up PostgreSQL? On Thu, 20 Jun 2002, Greg A. Woods wrote: Where does it say that close/open will flush metadata? That's how the unix filesystem works. UTSL. that's an implementation issue

ext2 and mail (Was: Backing up PostgreSQL?)

2002-06-21 Thread Martín Marqués
On Jue 20 Jun 2002 17:17, Ragnar Kjørstad wrote: This issue has been discussed in depth on the reiserfs and reiserfs-dev lists; I propse you subscribe or browse the archives for more information. In particular there is a thread about filesystem-features required for mailservers, and there

Re: Backing up PostgreSQL?

2002-06-21 Thread Martín Marqués
On Vie 21 Jun 2002 17:33, Greg A. Woods wrote: [ On Friday, June 21, 2002 at 18:37:20 (+0100), Paul Jakma wrote: ] Subject: Re: Backing up PostgreSQL? On Thu, 20 Jun 2002, Greg A. Woods wrote: Where does it say that close/open will flush metadata? That's how the unix filesystem

Re: Backing up PostgreSQL?

2002-06-21 Thread Greg A. Woods
[ On Friday, June 21, 2002 at 18:15:28 (+0100), Paul Jakma wrote: ] Subject: Re: Backing up PostgreSQL? On Fri, 14 Jun 2002, Greg A. Woods wrote: That's irrelevant from PostgreSQL's point of view. There's no sure way to tell the postgresql process(es) to make the on-disk database

Re: ext2 and mail (Was: Backing up PostgreSQL?)

2002-06-21 Thread Brandon D. Valentine
On Fri, 21 Jun 2002, [iso-8859-1] Martín Marqués wrote: On Jue 20 Jun 2002 17:17, Ragnar Kjørstad wrote: In particular there is a thread about filesystem-features required for mailservers, and there is a post from Wietse where he writes: ext2fs isn't a great file system for mail. Queue

Re: Backing up PostgreSQL?

2002-06-21 Thread Ragnar Kjørstad
Remember I've only been talking about the backed up files being in a self-consistent state and not requiring roll-back or roll-forward of any transaction logs after restore. If you don't want your database to do roll-back or roll-forward backup-snapshots will work for you. Neither

Re: Backing up PostgreSQL?

2002-06-21 Thread Ragnar Kjørstad
On Fri, Jun 21, 2002 at 05:17:35PM -0400, Greg A. Woods wrote: Yes, I agree with this, assuming you're talking about an increment to the currently available release that puts you up ahead to some mythical future vapour-ware. The ability to reliably restore consistency in such a backup copy

Re: Backing up PostgreSQL?

2002-06-21 Thread Ragnar Kjørstad
On Fri, Jun 21, 2002 at 06:28:59PM +0100, Paul Jakma wrote: Now you're getting a little out of hand. A journaling filesystem is a piling of one set of warts ontop of another. Now you've got a situation where even though the filesystem might be 100% consistent even after a catastrophic

Re: Backing up PostgreSQL?

2002-06-21 Thread Greg A. Woods
[ On Friday, June 21, 2002 at 18:00:19 (-0300), Martín Marqués wrote: ] Subject: Re: Backing up PostgreSQL? On Vie 21 Jun 2002 17:33, Greg A. Woods wrote: [ On Friday, June 21, 2002 at 18:37:20 (+0100), Paul Jakma wrote: ] Subject: Re: Backing up PostgreSQL? On Thu, 20 Jun 2002

Re: Backing up PostgreSQL?

2002-06-21 Thread Greg A. Woods
[ On Saturday, June 22, 2002 at 00:07:05 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? On Fri, Jun 21, 2002 at 05:17:35PM -0400, Greg A. Woods wrote: Yes, I agree with this, assuming you're talking about an increment to the currently available release that puts you

Re: Backing up PostgreSQL?

2002-06-21 Thread Greg A. Woods
[ On Friday, June 21, 2002 at 23:59:27 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? Remember I've only been talking about the backed up files being in a self-consistent state and not requiring roll-back or roll-forward of any transaction logs after

Re: Backing up PostgreSQL?

2002-06-21 Thread Michael H.Collins
I thought i just read that postgresql on ext3 outran oracle on raw devices. ~Because there are tremendous performance advantages to using RAW I/O if ~you're prepared to go to the trouble of dealing with all the niggling ~details ~ ~(some of these advantages are supposedly less visible on

Re: Backing up PostgreSQL?

2002-06-20 Thread Greg A. Woods
[ On Wednesday, June 19, 2002 at 23:53:14 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? By this definition postgresql is consistant at all times That's simply not possible to be true. PostgreSQL uses multiple files in the filesystem namespace to contain its data

Re: Backing up PostgreSQL?

2002-06-20 Thread Ragnar Kjørstad
19, 2002 at 23:53:14 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? By this definition postgresql is consistant at all times That's simply not possible to be true. PostgreSQL uses multiple files in the filesystem namespace to contain its data -- sometimes even

Re: Backing up PostgreSQL?

2002-06-20 Thread Greg A. Woods
[ On Thursday, June 20, 2002 at 22:17:55 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? On Thu, Jun 20, 2002 at 01:11:09PM -0400, Greg A. Woods wrote: [ On Wednesday, June 19, 2002 at 23:53:14 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL

Re: Backing up PostgreSQL?

2002-06-18 Thread Greg A. Woods
[ On Sunday, June 16, 2002 at 14:42:50 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? The term consistency has no meaning by itself unless there are well defined rules about the allowed data-content. If the rule is that alle the postgresql-datafiles together holds

Re: Backing up PostgreSQL?

2002-06-16 Thread Ragnar Kjørstad
On Sat, Jun 15, 2002 at 04:32:23PM -0400, Greg A. Woods wrote: That's irrelevant from PostgreSQL's point of view. There's no sure way to tell the postgresql process(es) to make the on-disk database image consistent before you create the snapshot. The race condition is between the

Re: Backing up PostgreSQL?

2002-06-15 Thread Ragnar Kjørstad
On Fri, Jun 14, 2002 at 09:15:15PM -0400, Greg A. Woods wrote: [ On Saturday, June 15, 2002 at 00:45:12 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? snapshots (LVM snapshots) are not supposedly nearly instantaneous, but instantaneous. All write-access

Re: Backing up PostgreSQL?

2002-06-15 Thread Greg A. Woods
[ On Saturday, June 15, 2002 at 14:42:35 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? On Fri, Jun 14, 2002 at 09:15:15PM -0400, Greg A. Woods wrote: [ On Saturday, June 15, 2002 at 00:45:12 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL

Re: Backing up PostgreSQL?

2002-06-14 Thread Ragnar Kjørstad
On Thu, Jun 13, 2002 at 04:57:16PM -0500, Kirk Strauser wrote: At 2002-06-13T21:26:22Z, Ragnar Kjørstad [EMAIL PROTECTED] writes: The easiest way to snapshot the filesystem is to use a logical volume manager (LVM or EVMS on linux) and then do: 1. take database offline 2. take

Re: Backing up PostgreSQL?

2002-06-14 Thread Ragnar Kjørstad
On Thu, Jun 13, 2002 at 05:33:31PM -0500, Brandon D. Valentine wrote: The easiest way to snapshot the filesystem is to use a logical volume manager (LVM or EVMS on linux) and then do: 1. take database offline 2. take snapshot 3. take database online 4. backup from snapshot 5. remove

Re: Backing up PostgreSQL?

2002-06-14 Thread Kirk Strauser
At 2002-06-14T05:40:24Z, Brandon D. Valentine [EMAIL PROTECTED] writes: Space permitting? Most natural language plain text files compress very well. A file full of database queries is going to have lots of easy to predict repetitive query statements and lots of natural language within

Re: Backing up PostgreSQL?

2002-06-14 Thread Kirk Strauser
At 2002-06-14T07:41:53Z, Ragnar Kjørstad [EMAIL PROTECTED] writes: Hmm, is the pg_dump consistent? From man 1 pg_dump: pg_dump makes consistent backups even if the database is being used concurrently. pg_dump does not block other users accessing the database

Re: Backing up PostgreSQL?

2002-06-14 Thread Niall O Broin
On Thu, Jun 13, 2002 at 10:29:33PM -0500, Kirk Strauser wrote: I would re-do the backup steps as 1) Make a snapshot 2) Use dump to back up that completely static filesystem image 3) Remove the snapshot This is NOT guaranteed to work - it may, if you're lucky. By doing this you're

Re: Backing up PostgreSQL?

2002-06-14 Thread Greg A. Woods
[ On , June 14, 2002 at 09:23:33 (-0500), Kirk Strauser wrote: ] Subject: Re: Backing up PostgreSQL? At 2002-06-14T07:41:53Z, Ragnar Kjørstad [EMAIL PROTECTED] writes: Hmm, is the pg_dump consistent? From man 1 pg_dump: pg_dump makes consistent backups even

Re: Backing up PostgreSQL?

2002-06-14 Thread Greg A. Woods
[ On , June 13, 2002 at 22:29:33 (-0500), Kirk Strauser wrote: ] Subject: Re: Backing up PostgreSQL? At 2002-06-13T23:07:54Z, Niall O Broin [EMAIL PROTECTED] writes: Nothing to do with BSD, Linux, or even, Gawd help us, Windows. If you don't take the database offline then the files

RE: Backing up PostgreSQL?

2002-06-14 Thread Greg A. Woods
[ On Friday, June 14, 2002 at 12:20:21 (-0400), Bort, Paul wrote: ] Subject: RE: Backing up PostgreSQL? I don't know much about PostgresQL, but on MS SQL server and Oracle (IIRC), any update that would leave the database inconsistent should be inside a transaction. Any snapshot

RE: Backing up PostgreSQL?

2002-06-14 Thread Bort, Paul
PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, June 14, 2002 2:53 PM To: Bort, Paul Cc: [EMAIL PROTECTED] Subject: RE: Backing up PostgreSQL? [ On Friday, June 14, 2002 at 12:20:21 (-0400), Bort, Paul wrote: ] Subject: RE: Backing up PostgreSQL? I don't know much about PostgresQL

RE: Backing up PostgreSQL?

2002-06-14 Thread Greg A. Woods
[ On Friday, June 14, 2002 at 15:07:40 (-0400), Bort, Paul wrote: ] Subject: RE: Backing up PostgreSQL? Sorry, when the previous poster mentioned 'snapshot', I was thinking of SQL Server's 'dump', which is transactionally consistent, because it's done by the SQL engine. I thought Oracle had

Re: Backing up PostgreSQL?

2002-06-14 Thread Ragnar Kjørstad
On Fri, Jun 14, 2002 at 04:32:20PM +0100, Niall O Broin wrote: 1) Make a snapshot 2) Use dump to back up that completely static filesystem image 3) Remove the snapshot This is NOT guaranteed to work - it may, if you're lucky. By doing this you're guaranteeing that the database files, no

Re: Backing up PostgreSQL?

2002-06-14 Thread Greg A. Woods
[ On Saturday, June 15, 2002 at 00:45:12 (+0200), Ragnar Kjørstad wrote: ] Subject: Re: Backing up PostgreSQL? snapshots (LVM snapshots) are not supposedly nearly instantaneous, but instantaneous. All write-access to the device is _locked_ while the snapshot is in progress (the process takes

Backing up PostgreSQL?

2002-06-13 Thread Kirk Strauser
I just finished reading `Unix Backup and Recovery' and realized that there are quite a few holes in my disaster recovery plan. In particular, I'm looking for a way to backup PostgreSQL without taking the database offline. I'm assuming that whatever method I use will involve writing a wrapper

Re: Backing up PostgreSQL?

2002-06-13 Thread Brandon D. Valentine
On 13 Jun 2002, Kirk Strauser wrote: I just finished reading `Unix Backup and Recovery' and realized that there are quite a few holes in my disaster recovery plan. In particular, I'm looking for a way to backup PostgreSQL without taking the database offline. I'm assuming that whatever method I

Re: Backing up PostgreSQL?

2002-06-13 Thread Doug Silver
The latest Postgres has a script someone wrote called '502.pgsql' which vacuums and dumps all the databases. Should be in the files area of the source. It was intended for FreeBSD, but I don't think it really matters. HTH On Thu, 13 Jun 2002, Brandon D. Valentine wrote: On 13 Jun 2002, Kirk

Re: Backing up PostgreSQL?

2002-06-13 Thread Kirk Strauser
At 2002-06-13T20:40:20Z, Doug Silver [EMAIL PROTECTED] writes: The latest Postgres has a script someone wrote called '502.pgsql' which vacuums and dumps all the databases. Should be in the files area of the source. It was intended for FreeBSD, but I don't think it really matters. As luck

Re: Backing up PostgreSQL?

2002-06-13 Thread Ragnar Kjørstad
On Thu, Jun 13, 2002 at 02:13:39PM -0500, Kirk Strauser wrote: I just finished reading `Unix Backup and Recovery' and realized that there are quite a few holes in my disaster recovery plan. In particular, I'm looking for a way to backup PostgreSQL without taking the database offline. I'm

Re: Backing up PostgreSQL?

2002-06-13 Thread Kirk Strauser
At 2002-06-13T21:26:22Z, Ragnar Kjørstad [EMAIL PROTECTED] writes: The easiest way to snapshot the filesystem is to use a logical volume manager (LVM or EVMS on linux) and then do: 1. take database offline 2. take snapshot 3. take database online 4. backup from snapshot 5. remove

Re: Backing up PostgreSQL?

2002-06-13 Thread Brandon D. Valentine
On Thu, 13 Jun 2002, [iso-8859-1] Ragnar Kjørstad wrote: Alternatively to the pg_dump approach others have suggested you can snapshot your filesystem and then use regular backup. The easiest way to snapshot the filesystem is to use a logical volume manager (LVM or EVMS on linux) and then do: 1.

Re: Backing up PostgreSQL?

2002-06-13 Thread Kirk Strauser
is what prompted me to ask about backing up PostgreSQL in the first place. If your environment is such that taking the database offline is unacceptable then use must use the database's own utilities to dump the database into some form which may be backed up. Space permitting, I'd think

Re: Backing up PostgreSQL?

2002-06-13 Thread Brandon D. Valentine
On 13 Jun 2002, Kirk Strauser wrote: Space permitting, I'd think that making an (ASCII) copy of the database contents into a file is a pretty smart idea, even if you're backing up the DB files in another manner. Space permitting? Most natural language plain text files compress very well. A