Re: [HACKERS] postgresql.conf archive_command example

2011-09-10 Thread Andres Freund
On Friday, September 09, 2011 08:59:43 PM Florian Pflug wrote: On Sep8, 2011, at 15:09 , Aidan Van Dyk wrote: Personally, I think both of these show examples of why PG should be looking hard at either providing a simple robust local directory based archive_command, or very seriously

Re: [HACKERS] postgresql.conf archive_command example

2011-09-09 Thread Florian Pflug
On Sep8, 2011, at 15:09 , Aidan Van Dyk wrote: Personally, I think both of these show examples of why PG should be looking hard at either providing a simple robust local directory based archive_command, or very seriously pointing users at properly written tools like omniptr, or ptrtools,

Re: [HACKERS] postgresql.conf archive_command example

2011-09-09 Thread Martijn van Oosterhout
On Fri, Sep 09, 2011 at 08:59:43PM +0200, Florian Pflug wrote: Archiving WAL should be done by copying to a temp file and moving it into place. Before returning success, one should probably also do the fsync incantations the linux kernel guys argued are necessary to prevent the file from

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Fujii Masao
On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat r...@xzilla.net wrote: On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: I agree that basically archive_command should not overwrite an existing file. But if the size of existing file is less than 16MB, it should do that.

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Simon Riggs
On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat r...@xzilla.net wrote: On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: I agree that basically archive_command should not overwrite an existing file.

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 3:26 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote: That's an option. But I don't think that finding an existing file is so serious problem. The recommendation should be that the archived files are

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Aidan Van Dyk
On Thu, Sep 8, 2011 at 2:05 AM, Fujii Masao masao.fu...@gmail.com wrote: That's an option. But I don't think that finding an existing file is so serious problem. The most common cases which cause a partially-filled archived file are; 1. The server crashes while WAL file is being archived,

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Kevin Grittner
Aidan Van Dyk ai...@highrise.ca wrote: If you're copying a file into the archive, and making it appear non-atomically in your archive, your doing something wrong. Period. No excuses. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] postgresql.conf archive_command example

2011-09-07 Thread Robert Treat
On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Sep 3, 2011 at 5:10 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: (2)  It should copy, not move, with protection against overwriting an existing file. I agree that basically archive_command should not

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Kevin Grittner
Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: (1) We're talking about a new /bin executable to do this which could be referenced in an archive_command string or run from a script called by archive_command, right? That, or an internal

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Kevin Grittner
Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I think the solution to that problem is to provide a default archive command that just does the very simple thing, namely moving the WAL file to some place given as parameter. Some *local* mount point. I think we've been mostly in agreement,

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Fujii Masao
On Sat, Sep 3, 2011 at 5:10 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: (2)  It should copy, not move, with protection against overwriting an existing file. I agree that basically archive_command should not overwrite an existing file. But if the size of existing file is less than

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. I think it would. My usual advice is to avoid having to write one if possible, because it's more complex than it looks. What

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. I think it would. My usual advice is to avoid having to write one if possible,

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Kevin Grittner kevin.gritt...@wicourts.gov writes: In a green field I might argue for having an archvie_directory GUC instead of archive_command. As it stands, it might be a really good I would think we then would need both. archive_command with parameters offers both. idea to provide a

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Robert Haas
On Fri, Sep 2, 2011 at 10:34 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful.

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Andrew Dunstan
On 09/02/2011 01:00 PM, Robert Haas wrote: On Fri, Sep 2, 2011 at 10:34 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Dimitri Fontainedimi...@2ndquadrant.fr wrote: Robert Haasrobertmh...@gmail.com writes: I'm also wondering if providing some shell script examples of a

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: maybe instead of shipping a more complicated script we should be trying to eliminate (or at least reduce) the need for a more complicated script. That was the intent of my pg_archiveto suggestion. I'll amend it (based on other comments) to allow for

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Andrew Dunstan and...@dunslane.net wrote: For example, you might want to copy the archives to more than one place for safety reasons. We've never felt that the way to do that was to put the logic for it in the archive script -- we archive to a local directory and set up rsync tasks on cron

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Magnus Hagander
On Fri, Sep 2, 2011 at 19:13, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Andrew Dunstan and...@dunslane.net wrote: For example, you might want to copy the archives to more than one place for safety reasons. We've never felt that the way to do that was to put the logic for it in the

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Another thought I have here is to wonder whether we should change something on the server side so that we don't NEED such a complicated archive_command. I mean, copying a file to a directory somewhere is not fundamentally a complex operation. Nor is

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Robert Haas
On Fri, Sep 2, 2011 at 3:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Another thought I have here is to wonder whether we should change something on the server side so that we don't NEED such a complicated archive_command.  I mean, copying a file to a

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I think the notion that we should get rid of archive_command in favor of something more hard-wired is sheer lunacy. It's a good thing nobody proposed that. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I wasn't really proposing to get rid of it, but I do wonder if there are some configuration parameters we could add somewhere that would make common cases easier without making really complex things impossible. I think the solution to that problem is

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Kevin Grittner kevin.gritt...@wicourts.gov writes: (1) We're talking about a new /bin executable to do this which could be referenced in an archive_command string or run from a script called by archive_command, right? That, or an internal implementation. That would be a function in the

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: Well, we could make initdb patch it up, but that might seem excessive. I sometime wonder if archive_mode shouldn't default to on with the archive_command set to either '/bin/true' or 'rem' for windows. That allows to install proper archiving without

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Robert Haas
2011/9/1 Dimitri Fontaine dimi...@2ndquadrant.fr: Peter Eisentraut pete...@gmx.net writes: Well, we could make initdb patch it up, but that might seem excessive. I sometime wonder if archive_mode shouldn't default to on with the archive_command set to either '/bin/true' or 'rem' for windows.

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Josh Berkus
Seems like it would be better to fix archive_mode so that it can be changed without a restart. +1 I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Robert Haas
On Thu, Sep 1, 2011 at 3:05 PM, Josh Berkus j...@agliodbs.com wrote: +1 I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. I think it would. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Cédric Villemain
2011/8/30 Peter Eisentraut pete...@gmx.net: I think it would be useful to add the following explanation and sample to the postgresql.conf sample file: diff --git i/src/backend/utils/misc/postgresql.conf.sample w/src/backend/utils/misc/postgresql.conf.sample ---

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Peter Eisentraut
On ons, 2011-08-31 at 11:18 +0200, Cédric Villemain wrote: Just a question: can we build a different postgresql.conf for windows or do we add a windows command example here as well ? Well, we could make initdb patch it up, but that might seem excessive. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Cédric Villemain
2011/8/31 Peter Eisentraut pete...@gmx.net: On ons, 2011-08-31 at 11:18 +0200, Cédric Villemain wrote: Just a question: can we build a different postgresql.conf for windows or do we add a windows command example here as well ? Well, we could make initdb patch it up, but that might seem

Re: [HACKERS] postgresql.conf archive_command example

2011-08-30 Thread Brendan Jurd
On 31 August 2011 04:39, Peter Eisentraut pete...@gmx.net wrote: I think it would be useful to add the following explanation and sample to the postgresql.conf sample file: Good idea Peter, +1. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes