Re: [HACKERS] postgresql.conf archive_command example

2011-09-10 Thread Andres Freund
On Friday, September 09, 2011 08:59:43 PM Florian Pflug wrote: > On Sep8, 2011, at 15:09 , Aidan Van Dyk wrote: > > Personally, I think both of these show examples of why PG should be > > looking hard at either providing a simple robust local directory based > > archive_command, or very seriously p

Re: [HACKERS] postgresql.conf archive_command example

2011-09-09 Thread Martijn van Oosterhout
On Fri, Sep 09, 2011 at 08:59:43PM +0200, Florian Pflug wrote: > Archiving WAL should be done by copying to a temp file and moving it > into place. Before returning success, one should probably also do the > fsync incantations the linux kernel guys argued are necessary to prevent > the file from ap

Re: [HACKERS] postgresql.conf archive_command example

2011-09-09 Thread Florian Pflug
On Sep8, 2011, at 15:09 , Aidan Van Dyk wrote: > Personally, I think both of these show examples of why PG should be > looking hard at either providing a simple robust local directory based > archive_command, or very seriously pointing users at properly written > tools like omniptr, or ptrtools, wa

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Kevin Grittner
Aidan Van Dyk wrote: > If you're copying a file into the archive, and making it appear > non-atomically in your archive, your doing something wrong. > > Period. > > No excuses. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Aidan Van Dyk
On Thu, Sep 8, 2011 at 2:05 AM, Fujii Masao wrote: > That's an option. But I don't think that finding an existing file is so > serious > problem. The most common cases which cause a partially-filled archived > file are; > > 1. The server crashes while WAL file is being archived, and then the ser

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 3:26 PM, Simon Riggs wrote: > On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao wrote: >> That's an option. But I don't think that finding an existing file is so >> serious >> problem. > > The recommendation should be that the archived files are never > overwritten because that

Re: [HACKERS] postgresql.conf archive_command example

2011-09-07 Thread Simon Riggs
On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao wrote: > On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat wrote: >> On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao wrote: >>> I agree that basically archive_command should not overwrite an existing >>> file. >>> But if the size of existing file is less tha

Re: [HACKERS] postgresql.conf archive_command example

2011-09-07 Thread Fujii Masao
On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat wrote: > On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao wrote: >> I agree that basically archive_command should not overwrite an existing file. >> But if the size of existing file is less than 16MB, it should do that. >> Otherwise, >> that WAL file woul

Re: [HACKERS] postgresql.conf archive_command example

2011-09-07 Thread Robert Treat
On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao wrote: > On Sat, Sep 3, 2011 at 5:10 AM, Kevin Grittner > wrote: >> (2)  It should copy, not move, with protection against overwriting >> an existing file. > > I agree that basically archive_command should not overwrite an existing file. > But if the s

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Fujii Masao
On Sat, Sep 3, 2011 at 5:10 AM, Kevin Grittner wrote: > (2)  It should copy, not move, with protection against overwriting > an existing file. I agree that basically archive_command should not overwrite an existing file. But if the size of existing file is less than 16MB, it should do that. Other

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Kevin Grittner
Dimitri Fontaine wrote: > I think the solution to that problem is to provide a default > archive command that just does the very simple thing, namely > moving the WAL file to some place given as parameter. Some > *local* mount point. I think we've been mostly in agreement, but I have a few qu

Re: [HACKERS] postgresql.conf archive_command example

2011-09-06 Thread Kevin Grittner
Dimitri Fontaine wrote: > "Kevin Grittner" writes: >> (1) We're talking about a new /bin executable to do this which >> could be referenced in an archive_command string or run from a >> script called by archive_command, right? > > That, or an internal implementation. That would be a function i

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
"Kevin Grittner" writes: > (1) We're talking about a new /bin executable to do this which > could be referenced in an archive_command string or run from a > script called by archive_command, right? That, or an internal implementation. That would be a function in the backend that would be called

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Robert Haas writes: > I wasn't really proposing to get rid of it, but I do wonder if there > are some configuration parameters we could add somewhere that would > make common cases easier without making really complex things > impossible. I think the solution to that problem is to provide a defau

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Tom Lane wrote: > I think the notion that we should get rid of archive_command in > favor of something more hard-wired is sheer lunacy. It's a good thing nobody proposed that. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Robert Haas
On Fri, Sep 2, 2011 at 3:29 PM, Tom Lane wrote: > Robert Haas writes: >> Another thought I have here is to wonder whether we should change >> something on the server side so that we don't NEED such a complicated >> archive_command.  I mean, copying a file to a directory somewhere is >> not fundam

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Tom Lane
Robert Haas writes: > Another thought I have here is to wonder whether we should change > something on the server side so that we don't NEED such a complicated > archive_command. I mean, copying a file to a directory somewhere is > not fundamentally a complex operation. Nor is using ssh to copy

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Magnus Hagander
On Fri, Sep 2, 2011 at 19:13, Kevin Grittner wrote: > Andrew Dunstan wrote: > >> For example, you might want to copy the archives to more than one >> place for safety reasons. > > We've never felt that the way to do that was to put the logic for it > in the archive script -- we archive to a local

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Andrew Dunstan wrote: > For example, you might want to copy the archives to more than one > place for safety reasons. We've never felt that the way to do that was to put the logic for it in the archive script -- we archive to a local directory and set up rsync tasks on cron to distribute it.

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Robert Haas wrote: > maybe instead of shipping a more complicated script we should be > trying to eliminate (or at least reduce) the need for a more > complicated script. That was the intent of my pg_archiveto suggestion. I'll amend it (based on other comments) to allow for a URL as an altern

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Andrew Dunstan
On 09/02/2011 01:00 PM, Robert Haas wrote: On Fri, Sep 2, 2011 at 10:34 AM, Kevin Grittner wrote: Dimitri Fontaine wrote: Robert Haas writes: I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. I think it would. My

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Robert Haas
On Fri, Sep 2, 2011 at 10:34 AM, Kevin Grittner wrote: > Dimitri Fontaine wrote: >> Robert Haas writes: I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. >>> >>> I think it would. >> >> My usual advice is to avoid

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
"Kevin Grittner" writes: > In a green field I might argue for having an archvie_directory GUC > instead of archive_command. As it stands, it might be a really good I would think we then would need both. archive_command with parameters offers both. > idea to provide a pg_archiveto executable wh

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Kevin Grittner
Dimitri Fontaine wrote: > Robert Haas writes: >>> I'm also wondering if providing some shell script examples of a >>> fault-tolerant script to handle archiving would be useful. >> >> I think it would. > > My usual advice is to avoid having to write one if possible, > because it's more complex th

Re: [HACKERS] postgresql.conf archive_command example

2011-09-02 Thread Dimitri Fontaine
Robert Haas writes: >> I'm also wondering if providing some shell script examples of a >> fault-tolerant script to handle archiving would be useful. > > I think it would. My usual advice is to avoid having to write one if possible, because it's more complex than it looks. What about recommending

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Robert Haas
On Thu, Sep 1, 2011 at 3:05 PM, Josh Berkus wrote: > +1 > > I'm also wondering if providing some shell script examples of a > fault-tolerant script to handle archiving would be useful. I think it would. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Josh Berkus
> Seems like it would be better to fix archive_mode so that it can be > changed without a restart. +1 I'm also wondering if providing some shell script examples of a fault-tolerant script to handle archiving would be useful. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Robert Haas
2011/9/1 Dimitri Fontaine : > Peter Eisentraut writes: >> Well, we could make initdb patch it up, but that might seem excessive. > > I sometime wonder if archive_mode shouldn't default to "on" with the > archive_command set to either '/bin/true' or 'rem' for windows. > > That allows to install pro

Re: [HACKERS] postgresql.conf archive_command example

2011-09-01 Thread Dimitri Fontaine
Peter Eisentraut writes: > Well, we could make initdb patch it up, but that might seem excessive. I sometime wonder if archive_mode shouldn't default to "on" with the archive_command set to either '/bin/true' or 'rem' for windows. That allows to install proper archiving without restart, but the

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Cédric Villemain
2011/8/31 Peter Eisentraut : > On ons, 2011-08-31 at 11:18 +0200, Cédric Villemain wrote: >> Just a question: can we build a different postgresql.conf for windows >> or do we add a windows command example here as well ? > > Well, we could make initdb patch it up, but that might seem excessive. sur

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Peter Eisentraut
On ons, 2011-08-31 at 11:18 +0200, Cédric Villemain wrote: > Just a question: can we build a different postgresql.conf for windows > or do we add a windows command example here as well ? Well, we could make initdb patch it up, but that might seem excessive. -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] postgresql.conf archive_command example

2011-08-31 Thread Cédric Villemain
2011/8/30 Peter Eisentraut : > I think it would be useful to add the following explanation and sample > to the postgresql.conf sample file: > > diff --git i/src/backend/utils/misc/postgresql.conf.sample > w/src/backend/utils/misc/postgresql.conf.sample > --- i/src/backend/utils/misc/postgresql.con

Re: [HACKERS] postgresql.conf archive_command example

2011-08-30 Thread Brendan Jurd
On 31 August 2011 04:39, Peter Eisentraut wrote: > I think it would be useful to add the following explanation and sample > to the postgresql.conf sample file: > Good idea Peter, +1. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

[HACKERS] postgresql.conf archive_command example

2011-08-30 Thread Peter Eisentraut
I think it would be useful to add the following explanation and sample to the postgresql.conf sample file: diff --git i/src/backend/utils/misc/postgresql.conf.sample w/src/backend/utils/misc/postgresql.conf.sample --- i/src/backend/utils/misc/postgresql.conf.sample +++ w/src/backend/utils/misc/po