Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-29 Thread Jeff Janes
On Saturday, March 29, 2014, Tom Lane wrote: > Jeff Janes > writes: > > But, it is hard to tell what the real solution is, because the doc > doesn't > > explain why it should refuse (and fail) to overwrite an existing file. > The > > only reason I can think of to make that recommendation is beca

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-29 Thread Tom Lane
Jeff Janes writes: > But, it is hard to tell what the real solution is, because the doc doesn't > explain why it should refuse (and fail) to overwrite an existing file. The > only reason I can think of to make that recommendation is because it is > easy to accidentally configure two clusters to a

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-29 Thread Jeff Janes
On Fri, Mar 21, 2014 at 2:22 PM, MauMau wrote: > From: "Jeff Janes" > > Do people really just copy the files from one directory of local storage >> to >> another directory of local storage? I don't see the point of that. >> > > It makes sense to archive WAL to a directory of local storage for

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-22 Thread Martijn van Oosterhout
On Sat, Mar 22, 2014 at 06:22:37AM +0900, MauMau wrote: > From: "Jeff Janes" > >Do people really just copy the files from one directory of local > >storage to > >another directory of local storage? I don't see the point of that. > > It makes sense to archive WAL to a directory of local storage f

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread Bruce Momjian
On Fri, Mar 21, 2014 at 01:16:08PM -0700, Jeff Janes wrote: > On Sun, Mar 16, 2014 at 3:23 AM, MauMau wrote: > > Hello, > > The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > Windows) as an example for archive_command. However, cp/copy does not > sync > the

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread MauMau
From: "Jeff Janes" Do people really just copy the files from one directory of local storage to another directory of local storage? I don't see the point of that. It makes sense to archive WAL to a directory of local storage for media recovery. Here, the local storage is a different disk dr

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread Jeff Janes
On Sun, Mar 16, 2014 at 3:23 AM, MauMau wrote: > Hello, > > The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > Windows) as an example for archive_command. However, cp/copy does not sync > the copied data to disk. As a result, the completed WAL segments would be > lost in th

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread MauMau
From: "Michael Paquier" On Fri, Mar 21, 2014 at 8:54 PM, MauMau wrote: * Create pg_copy in C so that it can be used on Windows as well as on UNIX/Linux. It just copies one file. Its source code is located in src/bin/pg_copy/. Please recommend a better name if you have one in mind. I'd rat

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread MauMau
From: "Michael Paquier" On Fri, Mar 21, 2014 at 8:54 PM, MauMau wrote: * Create pg_copy in C so that it can be used on Windows as well as on UNIX/Linux. It just copies one file. Its source code is located in src/bin/pg_copy/. Please recommend a better name if you have one in mind. I'd rat

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread Michael Paquier
On Fri, Mar 21, 2014 at 8:54 PM, MauMau wrote: > * Create pg_copy in C so that it can be used on Windows as well as on > UNIX/Linux. It just copies one file. Its source code is located in > src/bin/pg_copy/. Please recommend a better name if you have one in mind. I'd rather see that as a part o

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-21 Thread MauMau
From: "Mitsumasa KONDO" 2014-03-17 21:12 GMT+09:00 Fujii Masao : On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas wrote: > On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: >> * Improve the example in the documentation. >> But what command can we use to reliably sync just one file? >> >> * Provide

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Mitsumasa KONDO
2014-03-17 21:12 GMT+09:00 Fujii Masao : > On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas > wrote: > > On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: > >> The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > >> Windows) as an example for archive_command. However, cp/copy does

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-17 Thread Fujii Masao
On Mon, Mar 17, 2014 at 10:20 AM, Robert Haas wrote: > On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: >> The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on >> Windows) as an example for archive_command. However, cp/copy does not sync >> the copied data to disk. As a result, t

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread Amit Kapila
On Sun, Mar 16, 2014 at 7:53 PM, MauMau wrote: > From: "Amit Kapila" > >> How about using pg_receivexlog for archiving purpose? > > > pg_receivexlog is good in that it does fsync(). But it seems difficult to > use correctly, and I'm not sure if I can catch all WAL segments without any > loss. p

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread Robert Haas
On Sun, Mar 16, 2014 at 6:23 AM, MauMau wrote: > The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > Windows) as an example for archive_command. However, cp/copy does not sync > the copied data to disk. As a result, the completed WAL segments would be > lost in the following

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread Andreas Karlsson
On 03/16/2014 03:23 PM, MauMau wrote: From: "Amit Kapila" How about using pg_receivexlog for archiving purpose? pg_receivexlog is good in that it does fsync(). But it seems difficult to use correctly, and I'm not sure if I can catch all WAL segments without any loss. pg_receivexlog must be

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread Greg Stark
On Sun, Mar 16, 2014 at 10:23 AM, MauMau wrote: > The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > Windows) as an example for archive_command. However, cp/copy does not sync > the copied data to disk I'm actually a lot less concerned about fsyncing the backup than I am abo

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread MauMau
From: "Amit Kapila" How about using pg_receivexlog for archiving purpose? pg_receivexlog is good in that it does fsync(). But it seems difficult to use correctly, and I'm not sure if I can catch all WAL segments without any loss. pg_receivexlog must be started with postmaster and monitored

Re: [HACKERS] [RFC] What should we do for reliable WAL archiving?

2014-03-16 Thread Amit Kapila
On Sun, Mar 16, 2014 at 3:53 PM, MauMau wrote: > Hello, > > The PostgreSQL documentation describes cp (on UNIX/Linux) or copy (on > Windows) as an example for archive_command. However, cp/copy does not sync > the copied data to disk. As a result, the completed WAL segments would be > lost in the