Re: [GENERAL] pitr archive_command cp fsync

2015-03-16 Thread Peter Eisentraut
On 3/14/15 3:27 PM, Миша Тюрин wrote: should we add disclaimer in pitr documentation about cp and fsync? cp does not fsync. and dd for example can do fsync. only on some platforms -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

[GENERAL] pitr archive_command cp fsync

2015-03-14 Thread Миша Тюрин
hi all! should we add disclaimer in pitr documentation about cp and fsync? cp does not fsync. and dd for example can do fsync. -- misha

[GENERAL] PITR manual doesn't reference pg_receivexlog?

2012-11-19 Thread Joe Van Dyk
http://www.postgresql.org/docs/current/static/continuous-archiving.htmldoesn't mention pg_receivexlog. But http://www.postgresql.org/docs/current/static/app-pgreceivexlog.htmlsays pg_receivexlog can be used for PITR backups. Should the PITR page reference pg_receivexlog?

[GENERAL] PITR / progress

2012-10-27 Thread Andreas Brandl
Hi, I'm currently doing a point in time recovery with a recovery_target_time set. As it takes quite a while, I was wondering if there is a way to see the progress in terms of 'realtime' somehow? Is there any way of getting the timestamp of the last replayed transaction or the like? I know

[GENERAL] PITR backup - estimating size

2012-03-23 Thread Mike Blackwell
I'd like to switch to PITR backups, but have limited disk space. Is there a way to get a ballpark estimate by monitoring a running system, without actually creating the WAL files and risking filling a filesystem? Mike

Re: [GENERAL] PITR backup - estimating size

2012-03-23 Thread Fabrízio de Royes Mello
2012/3/23 Mike Blackwell mike.blackw...@rrd.com I'd like to switch to PITR backups, but have limited disk space. Is there a way to get a ballpark estimate by monitoring a running system, without actually creating the WAL files and risking filling a filesystem? Usually each wal segment size

[GENERAL] Pitr

2010-12-27 Thread Geoffrey Myers
Set up wal shipping on postgresql 8.3.9 and rhel 5.5. When I start the postmaster on the primary, there is no reference to archiving in the log and files do not get shipped. Had this working earlier, but the ssh keys were accidentally over written breaking the shipping. Any clues? -- Later,

Re: [GENERAL] Pitr

2010-12-27 Thread Geoffrey Myers
Patience is my friend. No transactions so no archiving. Waiting long enough produced results. Sorry for the noise. -- Later, Geoffrey Sent from my iPhone On Dec 27, 2010, at 3:18 PM, Geoffrey Myers li...@serioustechnology.com wrote: Set up wal shipping on postgresql 8.3.9 and rhel 5.5. When

Re: [GENERAL] PITR on different machine/architecture?

2010-11-06 Thread Craig Ringer
On 11/06/2010 02:48 AM, Vick Khera wrote: On Fri, Nov 5, 2010 at 1:54 PM, Andreas Brandlm...@andreas-brandl.de wrote: we are implementing archiving/PITR for a postgresql instance operating on OpenBSD/64-bit. Is it possible to restore the backup on a completely different machine (i.e. other

[GENERAL] PITR on different machine/architecture?

2010-11-05 Thread Andreas Brandl
Hi, we are implementing archiving/PITR for a postgresql instance operating on OpenBSD/64-bit. Is it possible to restore the backup on a completely different machine (i.e. other OS/32-bit)? What about restoring on (slightly) different versions of postgresql? Thanks! Best regards, Andreas

Re: [GENERAL] PITR on different machine/architecture?

2010-11-05 Thread Vick Khera
On Fri, Nov 5, 2010 at 1:54 PM, Andreas Brandl m...@andreas-brandl.de wrote: we are implementing archiving/PITR for a postgresql instance operating on OpenBSD/64-bit. Is it possible to restore the backup on a completely different machine (i.e. other OS/32-bit)? What about restoring on

Re: [GENERAL] pitr question

2010-10-14 Thread Craig Ringer
On 14/10/10 00:18, Joshua D. Drake wrote: On Wed, 2010-10-13 at 11:40 -0400, Geoffrey Myers wrote: On 10/13/2010 11:30 AM, zhong ming wu wrote: On Wed, Oct 13, 2010 at 11:17 AM, Geoffrey Myers li...@serioustechnology.com mailto:li...@serioustechnology.com wrote: Excuse the ignorance, but I

[GENERAL] pitr question

2010-10-13 Thread Geoffrey Myers
Excuse the ignorance, but I see the following in the docs: 'In any case the hardware architecture must be the same — shipping from, say, a 32-bit to a 64-bit system will not work.' Is this specific to the hardware? That is to say, can I use pitr wal shipping from 64 bit hardware to 64 bit

Re: [GENERAL] pitr question

2010-10-13 Thread Vick Khera
On Wed, Oct 13, 2010 at 11:17 AM, Geoffrey Myers li...@serioustechnology.com wrote: 'In any case the hardware architecture must be the same — shipping from, say, a 32-bit to a 64-bit system will not work.' Is this specific to the hardware?  That is to say, can I use pitr wal shipping from 64

Re: [GENERAL] pitr question

2010-10-13 Thread Geoffrey Myers
On 10/13/2010 11:30 AM, zhong ming wu wrote: On Wed, Oct 13, 2010 at 11:17 AM, Geoffrey Myers li...@serioustechnology.com mailto:li...@serioustechnology.com wrote: Excuse the ignorance, but I see the following in the docs: 'In any case the hardware architecture must be the same — shipping

Re: [GENERAL] pitr question

2010-10-13 Thread Tom Lane
Vick Khera vi...@khera.org writes: On Wed, Oct 13, 2010 at 11:17 AM, Geoffrey Myers li...@serioustechnology.com wrote: 'In any case the hardware architecture must be the same — shipping from, say, a 32-bit to a 64-bit system will not work.' Is this specific to the hardware?  That is to say,

Re: [GENERAL] pitr question

2010-10-13 Thread Joshua D. Drake
On Wed, 2010-10-13 at 11:40 -0400, Geoffrey Myers wrote: On 10/13/2010 11:30 AM, zhong ming wu wrote: On Wed, Oct 13, 2010 at 11:17 AM, Geoffrey Myers li...@serioustechnology.com mailto:li...@serioustechnology.com wrote: Excuse the ignorance, but I see the following in the docs:

[GENERAL] pitr errors

2009-09-29 Thread Louis Fridkis
I am testing PITR, following the instructions in: 23.3.3. Recovering with an On-line Backup. In step 9 I inspect the database and find that it is working perfectly. All the data from the original is present and I am able to create a new table and insert rows. The problem is that there are errors

Re: [GENERAL] pitr errors

2009-09-29 Thread Greg Smith
On Tue, 29 Sep 2009, Louis Fridkis wrote: When I check the WAL file directories I find that all is in order. The file 0004.history does not exist, but it is not supposed to exist. The file 000300EF00A6 is not in the backup archive (archive_log) because it had not been moved

Re: [GENERAL] PITR - warm standby switchover question

2009-04-15 Thread Erik Jones
Well, if you don't delete the recovery.conf and you *do* delete pg_standby's stop file (or it gets deleted, for example if you set it to go under /tmp and the server is restarted for whatever reason) the server will attempt to go back into recovery mode using your configured recovery

Re: [GENERAL] PITR - warm standby switchover question

2009-04-15 Thread Fujii Masao
Hi, On Wed, Apr 15, 2009 at 9:23 AM, Dan Hayes dhayes...@gmail.com wrote: Excellent!  Thanks.  One other quick question...  What would happen if I didn't delete the recovery.conf file?  Is that step just to prevent accidentally restarting the server with it there? recovery.conf is

Re: [GENERAL] PITR - warm standby switchover question

2009-04-15 Thread Chander Ganesan
Fujii Masao wrote: Hi, On Wed, Apr 15, 2009 at 9:23 AM, Dan Hayes dhayes...@gmail.com wrote: Excellent! Thanks. One other quick question... What would happen if I didn't delete the recovery.conf file? Is that step just to prevent accidentally restarting the server with it there?

Re: [GENERAL] PITR - warm standby switchover question

2009-04-15 Thread Erik Jones
On Apr 15, 2009, at 12:42 AM, Fujii Masao wrote: Hi, On Wed, Apr 15, 2009 at 9:23 AM, Dan Hayes dhayes...@gmail.com wrote: Excellent! Thanks. One other quick question... What would happen if I didn't delete the recovery.conf file? Is that step just to prevent accidentally restarting

[GENERAL] PITR - warm standby switchover question

2009-04-14 Thread Dan Hayes
I've followed the implementation instructions at 24.4.2: http://www.postgresql.org/docs/current/static/warm-standby.html And I've used the archive/restore commands from the example in F23.2: http://www.postgresql.org/docs/current/static/pgstandby.html This all works great. The primary backs up

Re: [GENERAL] PITR - warm standby switchover question

2009-04-14 Thread Erik Jones
On Apr 14, 2009, at 3:47 PM, Dan Hayes wrote: I've followed the implementation instructions at 24.4.2: http://www.postgresql.org/docs/current/static/warm-standby.html And I've used the archive/restore commands from the example in F23.2:

Re: [GENERAL] PITR - warm standby switchover question

2009-04-14 Thread Dan Hayes
Excellent! Thanks. One other quick question... What would happen if I didn't delete the recovery.conf file? Is that step just to prevent accidentally restarting the server with it there? On Tue, Apr 14, 2009 at 6:26 PM, Erik Jones ejo...@engineyard.com wrote: On Apr 14, 2009, at 3:47 PM,

[GENERAL] PITR archive_status/%p.done files

2008-12-16 Thread salman
A couple of questions about these files: 1) When are these generated? Is this once an archive is generated, or once the full archive_command has run? 2) Is there any harm in removing these files by appending an rm command at the end of archive_command? -salman -- Sent via pgsql-general

[GENERAL] PITR and base + full backups

2008-09-16 Thread Joey K.
Hello, Just to be sure of our backups we plan to do a base + full backups (yes, we are overly paranoid) (1) pg_start_backup(`date`) (2) perform hot rsync first (while the database is running) $ rsync -avr pgdata /backup/`date`/ (3) stop pg (4) perform cold rsync $ rsync -avr --delete pgdata

Re: [GENERAL] PITR and base + full backups

2008-09-16 Thread Alan Hodgson
On Tuesday 16 September 2008, Joey K. [EMAIL PROTECTED] wrote: Hello, Just to be sure of our backups we plan to do a base + full backups (yes, we are overly paranoid) (1) (`date`) (2) perform hot rsync first (while the database is running) $ rsync -avr pgdata /backup/`date`/ (3) stop

Re: [GENERAL] PITR and base + full backups

2008-09-16 Thread Greg Smith
On Tue, 16 Sep 2008, Joey K. wrote: (1) pg_start_backup(`date`) (2) perform hot rsync first (while the database is running) $ rsync -avr pgdata /backup/`date`/ (3) stop pg You need to call pg_stop_backup() here and wait until the last WAL file it references has been archived before this

Re: [GENERAL] PITR and base + full backups

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 21:16 +0530, Joey K. wrote: This didn't work and not sure if this is supposed to work ;-) Or should I stick to just plain PITR? Yes, just drop steps (3) and (5). If you don't trust it for some reason, then don't use it at all - mixing modes like that won't work. But we

Re: [ADMIN] [GENERAL] PITR - base backup question

2008-08-28 Thread Julio Leyva
Date: Wed, 27 Aug 2008 06:58:33 -0700 From: [EMAIL PROTECTED] Subject: Re: [ADMIN] [GENERAL] PITR - base backup question To: pgsql-general@postgresql.org; [EMAIL PROTECTED]; [EMAIL PROTECTED] --- On Tue, 8/26/08, Richard Broersma wrote

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Merlin Moncure
On Tue, Aug 26, 2008 at 9:04 PM, Richard Broersma [EMAIL PROTECTED] wrote: On Tue, Aug 26, 2008 at 5:38 PM, Merlin Moncure [EMAIL PROTECTED] wrote: If you ever want to mess around with log shipping I strongly suggest you go through the motions of setting up a warm standby vi the pg_standby

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Michael Nolan
I have what I have sometimes called a 'tepid spare' backup. Once a week I copy the physical files over to another system (actually to two of them) and every few hours I make sure the archived WAL log files are in sync (using rsync.) Anyway, here's the cookbook guide I wrote for updating one of

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Merlin Moncure
On Wed, Aug 27, 2008 at 9:18 AM, Michael Nolan [EMAIL PROTECTED] wrote: I have what I have sometimes called a 'tepid spare' backup. Once a week I copy the physical files over to another system (actually to two of them) and every few hours I make sure the archived WAL log files are in sync

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Michael Nolan
On Wed, Aug 27, 2008 at 8:32 AM, Merlin Moncure [EMAIL PROTECTED] wrote: 3. Shut down the Postgresql server running on the backup server, if any pg_ctl stop (Use 'ps ax' to make sure the server is stopped.) probably pg_ctl -m fast stop or -m immediate...since we are

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Lennin Caro
--- On Tue, 8/26/08, Richard Broersma [EMAIL PROTECTED] wrote: From: Richard Broersma [EMAIL PROTECTED] Subject: [GENERAL] PITR - base backup question To: pgsql-general@postgresql.org pgsql-general@postgresql.org, [EMAIL PROTECTED] Date: Tuesday, August 26, 2008, 10:53 PM From

Re: [GENERAL] PITR - base backup question

2008-08-27 Thread Merlin Moncure
On Wed, Aug 27, 2008 at 9:52 AM, Michael Nolan [EMAIL PROTECTED] wrote: This is a nice touch. With a little bash-fu you could do a find | xargs rm and list/kill the files in one pass. In the standby setups I've done I usually script the whole process, a prep on the main and a startup on the

Re: [GENERAL] PITR - base backup question

2008-08-26 Thread Merlin Moncure
On Tue, Aug 26, 2008 at 6:53 PM, Richard Broersma [EMAIL PROTECTED] wrote: From the following link: http://www.postgresql.org/docs/8.3/interactive/continuous-archiving.html#BACKUP-BASE-BACKUP Step 3 says to perform the back up. Does this mean a File System Backup of the Data directory?

Re: [GENERAL] PITR - base backup question

2008-08-26 Thread Richard Broersma
On Tue, Aug 26, 2008 at 5:38 PM, Merlin Moncure [EMAIL PROTECTED] wrote: If you ever want to mess around with log shipping I strongly suggest you go through the motions of setting up a warm standby vi the pg_standby utility and practice popping the standby out of recovery. Thanks for the

[GENERAL] PITR base backup -- stop server or not?

2008-06-19 Thread Rob Adams
The docs for Making a Base Backup (tar) say that it can be done live without stopping the server: http://www.postgresql.org/docs/8.3/interactive/continuous-archiving.html#BACKUP-BASE-BACKUP (step #3) However, the docs for straight File System Level Backup (tar) say the server must be shut

Re: [GENERAL] PITR base backup -- stop server or not?

2008-06-19 Thread Scott Marlowe
On Thu, Jun 19, 2008 at 12:14 AM, Rob Adams [EMAIL PROTECTED] wrote: The docs for Making a Base Backup (tar) say that it can be done live without stopping the server: http://www.postgresql.org/docs/8.3/interactive/continuous-archiving.html#BACKUP-BASE-BACKUP (step #3) However, the docs for

Re: [GENERAL] PITR problem

2008-05-01 Thread wstrzalka
On 29 Kwi, 17:16, [EMAIL PROTECTED] (Erik Jones) wrote: On Apr 29, 2008, at 3:20 AM, wstrzalka wrote: What is the full pg_standby command string (restore_command=) in your recovery.conf. It sound's like you have pg_standby set to delete archived WALs and possibly have that a

Re: [GENERAL] PITR problem

2008-04-29 Thread wstrzalka
What is the full pg_standby command string (restore_command=) in your recovery.conf. It sound's like you have pg_standby set to delete archived WALs and possibly have that a little too aggressive. Do you have the -k flag set in your pg_standby call in your restore_command? My restore

Re: [GENERAL] PITR problem

2008-04-29 Thread Erik Jones
On Apr 29, 2008, at 3:20 AM, wstrzalka wrote: What is the full pg_standby command string (restore_command=) in your recovery.conf. It sound's like you have pg_standby set to delete archived WALs and possibly have that a little too aggressive. Do you have the -k flag set in your

[GENERAL] PITR problem

2008-04-28 Thread wstrzalka
I have some problem with setting up PITR recovery on the database. I have archive_command set properly and logs are shipping OK. Archive timeout is also set (5 min). When performing pg_start_backup the WAL is lets say on position 0001000100D9, then I start copy database to the second

Re: [GENERAL] PITR problem

2008-04-28 Thread Erik Jones
On Apr 26, 2008, at 5:11 PM, wstrzalka wrote: I have some problem with setting up PITR recovery on the database. I have archive_command set properly and logs are shipping OK. Archive timeout is also set (5 min). When performing pg_start_backup the WAL is lets say on position

[GENERAL] PITR - filter by database?

2008-01-02 Thread Martin Langhoff
Is it possible to segregate the PITR data by database at any stage? We are - taking regular (daily) snapshots straight from the disk - storing WALs - restoring the snapshot - replaying the WALs My guess is that at snapshot time, I could use oid2name to focus on the database I'm interested

Re: [GENERAL] PITR - filter by database?

2008-01-02 Thread Usama Dar
On Jan 3, 2008 2:27 AM, Martin Langhoff [EMAIL PROTECTED] wrote: Is it possible to segregate the PITR data by database at any stage? We are i don't think so. My guess is that at snapshot time, I could use oid2name to focus on the database I'm interested in plus core Pg data structures, can

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Dhaval Shah
I am on 8.2 production and it will be difficult to upgrade to 8.3. Is it possible to backport the %r fix from 8.3 to 8.2? Regards Dhaval On Nov 13, 2007 11:26 PM, Simon Riggs [EMAIL PROTECTED] wrote: On Tue, 2007-11-13 at 00:07 -0500, Greg Smith wrote: On Mon, 12 Nov 2007, Mason Hale wrote:

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Bruce Momjian
Dhaval Shah wrote: I am on 8.2 production and it will be difficult to upgrade to 8.3. Is it possible to backport the %r fix from 8.3 to 8.2? You need to troll through the CVS archives to find that patch and try to apply it to 8.2. This feature will not be backpatched because we don't backpatch

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Dhaval Shah
No problem. One more question, is there a way to find out, without going through a test install, and from release notes etc. for 8.3 if the database needs migration from 8.2 to 8.3 or not. Regards Dhaval On Nov 14, 2007 10:44 AM, Bruce Momjian [EMAIL PROTECTED] wrote: Dhaval Shah wrote: I am

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Bruce Momjian
Dhaval Shah wrote: No problem. One more question, is there a way to find out, without going through a test install, and from release notes etc. for 8.3 if the database needs migration from 8.2 to 8.3 or not. What is migration? Application changes? The release notes pretty much tell you

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Scott Marlowe
On Nov 14, 2007 5:19 PM, Dhaval Shah [EMAIL PROTECTED] wrote: No problem. One more question, is there a way to find out, without going through a test install, and from release notes etc. for 8.3 if the database needs migration from 8.2 to 8.3 or not. Well, you HAVE to do a dump from one to

Re: [GENERAL] PITR and warm standby setup questions

2007-11-14 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 14 Nov 2007 18:35:00 -0500 (EST) Bruce Momjian [EMAIL PROTECTED] wrote: Dhaval Shah wrote: No problem. One more question, is there a way to find out, without going through a test install, and from release notes etc. for 8.3 if the

Re: [GENERAL] PITR and warm standby setup questions

2007-11-13 Thread Decibel!
On Nov 12, 2007, at 11:07 PM, Greg Smith wrote: On Mon, 12 Nov 2007, Mason Hale wrote: After the wal segment file is copied by the restore_command script, is it safe to delete it from my archive? While I believe you can toss them immediately, you should considering keeping those around

Re: [GENERAL] PITR and warm standby setup questions

2007-11-13 Thread Simon Riggs
On Tue, 2007-11-13 at 00:07 -0500, Greg Smith wrote: On Mon, 12 Nov 2007, Mason Hale wrote: After the wal segment file is copied by the restore_command script, is it safe to delete it from my archive? While I believe you can toss them immediately, This is almost never possible. The

[GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Mason Hale
I am setting up a warm standby configuration as described here: http://www.postgresql.org/docs/8.2/static/warm-standby.html Using PostgreSql 8.2.5 My production server is archiving 16MB wal segment files at a rate of 1 every 5 to 10 seconds My standby server is processing the wal segment files

Re: [GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Merlin Moncure
On Nov 12, 2007 6:59 PM, Mason Hale [EMAIL PROTECTED] wrote: I am setting up a warm standby configuration as described here: http://www.postgresql.org/docs/8.2/static/warm-standby.html Using PostgreSql 8.2.5 My production server is archiving 16MB wal segment files at a rate of 1 every 5 to

Re: [GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Mason Hale
your i/o must be really random to be seeing numbers that lousy (10 seconds to replay a file is 1.6 megabytes/sec), or there is some other unexplained problem with your server. is your raid controller properly caching wites? have you benchmarked the volume with bonnie++ or similar tool (pay

Re: [GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Greg Smith
On Mon, 12 Nov 2007, Mason Hale wrote: After the wal segment file is copied by the restore_command script, is it safe to delete it from my archive? While I believe you can toss them immediately, you should considering keeping those around for a bit regardless as an additional layer of

Re: [GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Merlin Moncure
On Nov 12, 2007 11:03 PM, Mason Hale [EMAIL PROTECTED] wrote: your i/o must be really random to be seeing numbers that lousy (10 seconds to replay a file is 1.6 megabytes/sec), or there is some other unexplained problem with your server. is your raid controller properly caching wites?

Re: [GENERAL] PITR and warm standby setup questions

2007-11-12 Thread Robert Treat
On Tuesday 13 November 2007 00:07, Greg Smith wrote: On Mon, 12 Nov 2007, Mason Hale wrote: After the wal segment file is copied by the restore_command script, is it safe to delete it from my archive? While I believe you can toss them immediately, you should considering keeping those

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-04 Thread Simon Riggs
On Tue, 2007-10-02 at 17:11 -0600, Brian Wipf wrote: Both servers have identical Intel processors and both are running 64- bit PostgreSQL 8.2.4. The original server is running 64-bit openSUSE 10.2 (Linux 2.6.18.2-34-default #1 SMP Mon Jul 16 01:16:32 GMT 2007 x86_64 x86_64 x86_64

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-04 Thread Brian Wipf
On 4-Oct-07, at 8:14 AM, Simon Riggs wrote: The First Commandment is Make Thy Servers Identical, which applies to OS, OS version, disk layouts/config as well as basic hardware. If they're not then you're going to get some strange results. Other than the corrupt indexes on varchar columns,

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-04 Thread Simon Riggs
On Thu, 2007-10-04 at 09:21 -0600, Brian Wipf wrote: On 4-Oct-07, at 8:14 AM, Simon Riggs wrote: The First Commandment is Make Thy Servers Identical, which applies to OS, OS version, disk layouts/config as well as basic hardware. If they're not then you're going to get some strange results.

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-04 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Thu, 2007-10-04 at 09:21 -0600, Brian Wipf wrote: The Apple Xserve is easy to maintain and rock solid reliable. If it had better performance to its Fibre Channel RAID array, it would be a better main server too. The Linux box is a better performer,

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Richard Huxton
Brian Wipf wrote: We are running a production server off of a new database that was synchronized using PITR recovery. We found that many of the btree indexes were out of sync with the underlying data after bringing the new server out of recovery mode, but the data itself appeared to be okay.

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Tom Lane
Richard Huxton [EMAIL PROTECTED] writes: Brian Wipf wrote: Both servers have identical Intel processors and both are running 64-bit PostgreSQL 8.2.4. The original server is running 64-bit openSUSE 10.2 (Linux 2.6.18.2-34-default #1 SMP Mon Jul 16 01:16:32 GMT 2007 x86_64 x86_64 x86_64

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Brian Wipf
On 3-Oct-07, at 8:07 AM, Tom Lane wrote: PG 8.2 does store data in the pg_control file with which it can check for the most common disk-format-incompatibility problems (to wit, endiannness, maxalign, and --enable-integer-datetimes). If Brian has stumbled on another such foot-gun, it'd be good

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Tom Lane
Brian Wipf [EMAIL PROTECTED] writes: PG tried to enforce the same LC_COLLATE and LC_CTYPE. On OS X, the value of en_US.utf8 didn't exist, so I created a soft link to en_US.UTF-8 in the /usr/share/locale/ directory. When I sort the values of product_id_from_source on both systems using

Re: [GENERAL] PITR and Compressed WALS

2007-10-03 Thread Tom Lane
Brian Wipf [EMAIL PROTECTED] writes: Last night, I brought the database out of its perpetual recovery mode. Here are the lines from the log when this was done: [2007-10-01 23:43:03 MDT] LOG: restored log file 000104660060 from archive [2007-10-01 23:45:50 MDT] LOG: could not

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Richard Huxton
Tom Lane wrote: Brian Wipf [EMAIL PROTECTED] writes: PG tried to enforce the same LC_COLLATE and LC_CTYPE. On OS X, the value of en_US.utf8 didn't exist, so I created a soft link to en_US.UTF-8 in the /usr/share/locale/ directory. When I sort the values of product_id_from_source on both

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Brian Wipf
On 3-Oct-07, at 12:46 PM, Richard Huxton wrote: Tom Lane wrote: Brian Wipf [EMAIL PROTECTED] writes: PG tried to enforce the same LC_COLLATE and LC_CTYPE. On OS X, the value of en_US.utf8 didn't exist, so I created a soft link to en_US.UTF-8 in the /usr/share/locale/ directory. When I

Re: [GENERAL] PITR and Compressed WALS

2007-10-03 Thread Brian Wipf
On 3-Oct-07, at 12:38 PM, Tom Lane wrote: What this sounds like to me is a problem in your recovery procedures. What exactly did you do to bring the database out of recovery mode? The script looked for a trigger file and once found, aborts. Unfortunately, it would abort without doing the

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Richard Huxton
Brian Wipf wrote: On 3-Oct-07, at 12:46 PM, Richard Huxton wrote: Could you run Linux in a virtual-machine in OS X? That's an idea. Performance-wise though, I think we'd be better off wiping OS X and installing Linux. As an added bonus, we'll be able to get way better performance out of our

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Richard Huxton wrote: Could you run Linux in a virtual-machine in OS X? I think it would be easier (and more performant) to define a new locale on OS/X (or on Linux) to match the behavior of the other system. (Perhaps define a new locale on both, with

Re: [GENERAL] PITR Recovery and out-of-sync indexes

2007-10-03 Thread Alvaro Herrera
Richard Huxton wrote: Tom Lane wrote: Brian Wipf [EMAIL PROTECTED] writes: PG tried to enforce the same LC_COLLATE and LC_CTYPE. On OS X, the value of en_US.utf8 didn't exist, so I created a soft link to en_US.UTF-8 in the /usr/share/locale/ directory. When I sort the values of

[GENERAL] PITR and Compressed WALS

2007-10-02 Thread Brian Wipf
We have two PostgreSQL 8.2.4 servers. On one database, WALs are archived with a simple script that gzips and transfers them to an NFS file server. The other database is in perpetual recovery mode, ungizipping and processing the WALs as they appear and become complete on the file server.

[GENERAL] PITR Recovery and out-of-sync indexes

2007-10-02 Thread Brian Wipf
We are running a production server off of a new database that was synchronized using PITR recovery. We found that many of the btree indexes were out of sync with the underlying data after bringing the new server out of recovery mode, but the data itself appeared to be okay. Both servers

Re: [GENERAL] PITR for postgresql-7.3

2007-08-13 Thread Mary Ellen Fitzpatrick
I am trying to run pg_dump on the database with the corrupt table, and try to restore the database. I also tried to vacuumdb the database and get the same error. I get the following error. pg_dump database pg_dump: query to obtain list of data types failed: PANIC: read of clog

Re: [GENERAL] PITR for postgresql-7.3

2007-08-13 Thread Tom Lane
Mary Ellen Fitzpatrick [EMAIL PROTECTED] writes: I am trying to run pg_dump on the database with the corrupt table, and try to restore the database. I also tried to vacuumdb the database and get the same error. I get the following error. pg_dump database pg_dump: query to

Re: [GENERAL] PITR for postgresql-7.3

2007-08-10 Thread Merlin Moncure
On 8/10/07, Mary Ellen Fitzpatrick [EMAIL PROTECTED] wrote: Hi, We are running postgresql-7.3.3 and we had a hardware controller and disk failure on the system. And of course the database does not appear to be backup anywhere. I was reading about PITR and was wondering if that is

Re: [GENERAL] PITR for postgresql-7.3

2007-08-10 Thread Merlin Moncure
On 8/10/07, Mary Ellen Fitzpatrick [EMAIL PROTECTED] wrote: Merlin, I am willing to spend the time, as it is an important table. I am a newbie at this and it has fallen into my lap. From what the user tells me, it is only the one table. Not sure if fsync was running, how can I tell? check

[GENERAL] PITR for postgresql-7.3

2007-08-10 Thread Mary Ellen Fitzpatrick
Hi, We are running postgresql-7.3.3 and we had a hardware controller and disk failure on the system. And of course the database does not appear to be backup anywhere. I was reading about PITR and was wondering if that is applicable to my version. We do have pg_xlog files and I am

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-06 Thread Marco Colombo
Greg Smith wrote: On Tue, 5 Jun 2007, Marco Colombo wrote: AFAIK, files in pg_xlog are first renamed (and only if and after the archive_command returned true) and later overwritten to. Never deleted. No, they get deleted sometimes, too. Not often, but it can happen under heavy load if more

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-06 Thread Marco Colombo
Simon Riggs wrote: On Tue, 2007-06-05 at 18:39 +0200, Marco Colombo wrote: I'm asking: what _exactly_ can go wrong? If a checkpoint occurs while taking the backup then the contents of the files will be overwritten ^ Data files or WAL segments? My archive command prevents WAL segments

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-05 Thread Marco Colombo
Simon Riggs wrote: Marco Colombo wrote: my method ...is dangerous Ok, but why? Once again, I'm asking: what _exactly_ can go wrong? so we don't get loads of new DBAs picking up this idea but missing the exact point of danger. I'm one of them. I'm _am_ missing the exact point of danger.

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-05 Thread Simon Riggs
On Tue, 2007-06-05 at 18:39 +0200, Marco Colombo wrote: I'm asking: what _exactly_ can go wrong? If a checkpoint occurs while taking the backup then the contents of the files will be overwritten and you will be unable to rollforward from before the backup until after the backup. This will give

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-05 Thread Greg Smith
On Tue, 5 Jun 2007, Marco Colombo wrote: AFAIK, files in pg_xlog are first renamed (and only if and after the archive_command returned true) and later overwritten to. Never deleted. No, they get deleted sometimes, too. Not often, but it can happen under heavy load if more segments get

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-04 Thread Marco Colombo
Greg Smith wrote: The way you're grabbing files directly from the xlog directory only works because your commit workload is so trivial that you can get away with it, and because you haven't then tried to apply future archive logs. Well, it's only because I don't need future logs, just like I

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-04 Thread Simon Riggs
On Mon, 2007-06-04 at 12:55 +0200, Marco Colombo wrote: Greg Smith wrote: The way you're grabbing files directly from the xlog directory only works because your commit workload is so trivial that you can get away with it, and because you haven't then tried to apply future archive logs.

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-02 Thread Greg Smith
On Fri, 1 Jun 2007, Marco Colombo wrote: If you need *both* a full backup *and* PITR, just add a real cp to the archive_command above. The important part is to return failure during the backup process, I think. You seem to have worked out a way for your application to do a base backup in a

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-01 Thread Simon Riggs
On Wed, 2007-05-30 at 22:41 -0400, Greg Smith wrote: -Find something harmless I can execute in a loop that will generate WAL activity, run that until the segment gets archived. Haven't really thought of something good to use for that purpose yet. create table xlog_switch as select

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-06-01 Thread Marco Colombo
Greg Smith wrote: On Thu, 31 May 2007, Marco Colombo wrote: archive_command = 'test ! -f /var/lib/pgsql/backup_lock /dev/null' Under normal condition (no backup running) this will trick PG into thinking that segments get archived. If I'm not mistaken, PG should behave exactly as if no

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-05-31 Thread Marco Colombo
Greg Smith wrote: [...] -Find something harmless I can execute in a loop that will generate WAL activity, run that until the segment gets archived. Haven't really thought of something good to use for that purpose yet. Some time ago I started a thread about taking on-the-fly backups at file

Re: [GENERAL] PITR Base Backup on an idle 8.1 server

2007-05-31 Thread Greg Smith
On Thu, 31 May 2007, Marco Colombo wrote: archive_command = 'test ! -f /var/lib/pgsql/backup_lock /dev/null' Under normal condition (no backup running) this will trick PG into thinking that segments get archived. If I'm not mistaken, PG should behave exactly as if no archive_command is

[GENERAL] PITR Base Backup on an idle 8.1 server

2007-05-30 Thread Greg Smith
I'm trying to figure out the best way to cope with creating a PITR base backup on a 8.1 server that is essentially idle during that time (and for hours afterwards). Because there's no activity when the backup is going on, I get the same segment file for FIRST WAL and LAST WAL. Unfortunately,

Re: [GENERAL] PITR and tar

2007-05-13 Thread Jim C. Nasby
Moving to -docs... Does anyone know what the history of the docs saying that GNU tar had issues with files changing underneath it? According to this report it's actually BSD tar that has the issue. On Wed, May 09, 2007 at 10:19:05AM -0700, Jeff Davis wrote: On Wed, 2007-05-09 at 11:40 -0500,

Re: [GENERAL] PITR and tar

2007-05-09 Thread Dhaval Shah
Looks like a problem specific to FreeBSD. I use Centos/postgres 8.2.3 and I do not see that problem at all. Dhaval On 5/8/07, Jeff Davis [EMAIL PROTECTED] wrote: On Tue, 2007-05-08 at 13:24 -0400, Merlin Moncure wrote: On 5/8/07, Jeff Davis [EMAIL PROTECTED] wrote: On Tue, 2007-05-08 at

  1   2   >