Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Mon, Jun 4, 2012 at 6:53 PM, Magnus Hagander wrote: > No, it's more a "there's no reason to do that". I don't think it > should necessarily be an actual problem. Ok, good to know. > In your case the missing piece of information is why was there a > timeline switch? pg_basebackup shouldn't cau

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Magnus Hagander
On Mon, Jun 4, 2012 at 5:48 PM, Ants Aasma wrote: > On Mon, Jun 4, 2012 at 6:20 PM, Fujii Masao wrote: >> On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma wrote: >>> On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander >>> wrote: > it doesn't say that is not possible to use this for a standby

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Mon, Jun 4, 2012 at 6:20 PM, Fujii Masao wrote: > On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma wrote: >> On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander >> wrote: it doesn't say that is not possible to use this for a standby server... probably that's why i get the error i put a re

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Fujii Masao
On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma wrote: > On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander wrote: >>> it doesn't say that is not possible to use this for a standby >>> server... probably that's why i get the error i put a recovery.conf >>> after pg_basebackup finished... maybe we can

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander wrote: >> it doesn't say that is not possible to use this for a standby >> server... probably that's why i get the error i put a recovery.conf >> after pg_basebackup finished... maybe we can say that  more loudly? > > The idea is, if you use it wit

Re: pg_receivexlog and sync rep Re: [HACKERS] Updated version of pg_receivexlog

2012-02-09 Thread Magnus Hagander
On Wed, Feb 8, 2012 at 19:39, Fujii Masao wrote: > On Tue, Oct 25, 2011 at 7:37 PM, Magnus Hagander wrote: >> On Mon, Oct 24, 2011 at 14:40, Magnus Hagander wrote: >>> On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas >>> wrote: How does this interact with synchronous replication? If a bas

pg_receivexlog and sync rep Re: [HACKERS] Updated version of pg_receivexlog

2012-02-08 Thread Fujii Masao
On Tue, Oct 25, 2011 at 7:37 PM, Magnus Hagander wrote: > On Mon, Oct 24, 2011 at 14:40, Magnus Hagander wrote: >> On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas >> wrote: >>> How does this interact with synchronous replication? If a base backup that >>> streams WAL is in progress, and you ha

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 11:57 PM, Magnus Hagander wrote: > On Thu, Oct 27, 2011 at 16:54, Tom Lane wrote: >> Magnus Hagander writes: >>> On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas >>> wrote: On 27.10.2011 14:09, Fujii Masao wrote: > Yes. But that sounds unuserfriendly. Padding t

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Dimitri Fontaine
Magnus Hagander writes: > On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas > wrote: >> Perhaps we should add automatic padding in the server, though. It wouldn't >> take much code in the server, and would make life easier for people writing >> their scripts. Maybe even have the backend check for

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 16:54, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas >> wrote: >>> On 27.10.2011 14:09, Fujii Masao wrote: Yes. But that sounds unuserfriendly. Padding the WAL file manually is easy-to-do for a user? > >> I'd defi

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Tom Lane
Magnus Hagander writes: > On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas > wrote: >> On 27.10.2011 14:09, Fujii Masao wrote: >>> Yes. But that sounds unuserfriendly. Padding the WAL file manually >>> is easy-to-do for a user? > I'd definitely want to avoid anything that requires pg_receivexlo

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Robert Haas
On Thu, Oct 27, 2011 at 7:19 AM, Heikki Linnakangas wrote: > On 27.10.2011 14:09, Fujii Masao wrote: >> >> On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagander >>  wrote: >>> >>> I'm rewriting the handling of partial files per the other thread >>> started by Heikki. The idea is that there will be an a

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas wrote: > On 27.10.2011 14:09, Fujii Masao wrote: >> >> On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagander >>  wrote: >>> >>> I'm rewriting the handling of partial files per the other thread >>> started by Heikki. The idea is that there will be an act

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Heikki Linnakangas
On 27.10.2011 14:09, Fujii Masao wrote: On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagander wrote: I'm rewriting the handling of partial files per the other thread started by Heikki. The idea is that there will be an actual .partial file in there when pg_receivexlog has ended, and you have to deal

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagander wrote: > On Thu, Oct 27, 2011 at 12:29, Fujii Masao wrote: >> On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao wrote: >>> On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander >>> wrote: Not sure I follow. When we arrive at PQgetCopyData() there sho

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 12:29, Fujii Masao wrote: > On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao wrote: >> On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander wrote: >>> Not sure I follow. When we arrive at PQgetCopyData() there should be >>> nothing buffered, and if the end of stream happens there

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao wrote: > On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander wrote: >> Not sure I follow. When we arrive at PQgetCopyData() there should be >> nothing buffered, and if the end of stream happens there it returns >> -1, and we exit, no? So where is the data

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander wrote: > Not sure I follow. When we arrive at PQgetCopyData() there should be > nothing buffered, and if the end of stream happens there it returns > -1, and we exit, no? So where is the data that's lost? > > I do realize we don't actually fsync() a

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 10:12, Fujii Masao wrote: > On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander wrote: >> On Thu, Oct 27, 2011 at 09:46, Fujii Masao wrote: >>> On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander >>> wrote: On Thu, Oct 27, 2011 at 09:29, Fujii Masao wrote: > On Thu,

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander wrote: > On Thu, Oct 27, 2011 at 09:46, Fujii Masao wrote: >> On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander wrote: >>> On Thu, Oct 27, 2011 at 09:29, Fujii Masao wrote: On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander wrote: >

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 09:46, Fujii Masao wrote: > On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander wrote: >> On Thu, Oct 27, 2011 at 09:29, Fujii Masao wrote: >>> On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander >>> wrote: I've applied this version with a few more minor changes that Hei

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander wrote: > On Thu, Oct 27, 2011 at 09:29, Fujii Masao wrote: >> On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander wrote: >>> I've applied this version with a few more minor changes that Heikki found. >> >> Cool! >> >> When I tried pg_receivexlog and

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 09:29, Fujii Masao wrote: > On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander wrote: >> I've applied this version with a few more minor changes that Heikki found. > > Cool! > > When I tried pg_receivexlog and checked the contents of streamed WAL file by > xlogdump, I found

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander wrote: > I've applied this version with a few more minor changes that Heikki found. Cool! When I tried pg_receivexlog and checked the contents of streamed WAL file by xlogdump, I found that recent WAL records that walsender has already sent don't

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-26 Thread Magnus Hagander
On Tue, Oct 25, 2011 at 12:37, Magnus Hagander wrote: > On Mon, Oct 24, 2011 at 14:40, Magnus Hagander wrote: >> On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas >> wrote: +               /* +                * Looks like an xlog file. Parse it's position. >>> >>> s/it's/its/ >>>

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Heikki Linnakangas
+ /* +* Looks like an xlog file. Parse it's position. s/it's/its/ +*/ + if (sscanf(dirent->d_name, "%08X%08X%08X", &tli, &log, &seg) != 3) + { + fprintf(stderr, _("%s: could not parse xlog filename

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Jaime Casanova
On Mon, Oct 24, 2011 at 7:40 AM, Magnus Hagander wrote: > >> synchronous_standby_names='*' is prone to such confusion in general, but it >> seems that it's particularly surprising if a running pg_basebackup lets a >> commit in synchronous replication to proceed. Maybe we just need a warning >> in

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Magnus Hagander
On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas wrote: >> +               /* >> +                * Looks like an xlog file. Parse it's position. > > s/it's/its/ > >> +                */ >> +               if (sscanf(dirent->d_name, "%08X%08X%08X", &tli, &log, >> &seg) != 3) >> +               {

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Magnus Hagander
On Mon, Oct 24, 2011 at 16:12, Jaime Casanova wrote: > On Mon, Oct 24, 2011 at 7:40 AM, Magnus Hagander wrote: >> >>> synchronous_standby_names='*' is prone to such confusion in general, but it >>> seems that it's particularly surprising if a running pg_basebackup lets a >>> commit in synchronous

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-29 Thread Magnus Hagander
On Thu, Sep 29, 2011 at 01:55, Jaime Casanova wrote: > On Wed, Sep 28, 2011 at 12:50 PM, Magnus Hagander wrote: >> >>> pg_receivexlog worked good in my tests. >>> >>> pg_basebackup with --xlog=stream gives me an already recycled wal >>> segment message (note that the file was in pg_xlog in the st

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Jaime Casanova
On Wed, Sep 28, 2011 at 12:50 PM, Magnus Hagander wrote: > >> pg_receivexlog worked good in my tests. >> >> pg_basebackup with --xlog=stream gives me an already recycled wal >> segment message (note that the file was in pg_xlog in the standby): >> FATAL:  could not receive data from WAL stream: FA

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Magnus Hagander
On Wed, Sep 28, 2011 at 09:30, Jaime Casanova wrote: > On Wed, Sep 28, 2011 at 1:38 AM, Jaime Casanova wrote: >> On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander wrote: >>> Here's an updated version of pg_receivexlog, that should now actually >>> work (it previously failed miserably when a repli

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Magnus Hagander
On Wed, Sep 28, 2011 at 08:38, Jaime Casanova wrote: > On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander wrote: >> Here's an updated version of pg_receivexlog, that should now actually >> work (it previously failed miserably when a replication record crossed >> a WAL file boundary - something whic

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Jaime Casanova
On Wed, Sep 28, 2011 at 1:38 AM, Jaime Casanova wrote: > On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander wrote: >> Here's an updated version of pg_receivexlog, that should now actually >> work (it previously failed miserably when a replication record crossed >> a WAL file boundary - something wh

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-27 Thread Jaime Casanova
On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander wrote: > Here's an updated version of pg_receivexlog, that should now actually > work (it previously failed miserably when a replication record crossed > a WAL file boundary - something which I at the time could not properly > reproduce, but when I