Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander mag...@hagander.net wrote: it doesn't say that is not possible to use this for a standby server... probably that's why i get the error i put a recovery.conf after pg_basebackup finished... maybe we can say that  more loudly? The idea is, if

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Fujii Masao
On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander mag...@hagander.net wrote: it doesn't say that is not possible to use this for a standby server... probably that's why i get the error i put a recovery.conf after pg_basebackup

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Mon, Jun 4, 2012 at 6:20 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander mag...@hagander.net wrote: it doesn't say that is not possible to use this for a standby server...

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Magnus Hagander
On Mon, Jun 4, 2012 at 5:48 PM, Ants Aasma a...@cybertec.at wrote: On Mon, Jun 4, 2012 at 6:20 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Jun 4, 2012 at 11:25 PM, Ants Aasma a...@cybertec.at wrote: On Thu, Sep 29, 2011 at 11:30 PM, Magnus Hagander mag...@hagander.net wrote: it

Re: [HACKERS] Updated version of pg_receivexlog

2012-06-04 Thread Ants Aasma
On Mon, Jun 4, 2012 at 6:53 PM, Magnus Hagander mag...@hagander.net wrote: No, it's more a there's no reason to do that. I don't think it should necessarily be an actual problem. Ok, good to know. In your case the missing piece of information is why was there a timeline switch? pg_basebackup

Re: pg_receivexlog and sync rep Re: [HACKERS] Updated version of pg_receivexlog

2012-02-09 Thread Magnus Hagander
On Wed, Feb 8, 2012 at 19:39, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Oct 25, 2011 at 7:37 PM, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 14:40, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas

pg_receivexlog and sync rep Re: [HACKERS] Updated version of pg_receivexlog

2012-02-08 Thread Fujii Masao
On Tue, Oct 25, 2011 at 7:37 PM, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 14:40, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: How does this interact with synchronous

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-28 Thread Fujii Masao
On Thu, Oct 27, 2011 at 11:57 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 16:54, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 27.10.2011

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander mag...@hagander.net wrote: I've applied this version with a few more minor changes that Heikki found. Cool! When I tried pg_receivexlog and checked the contents of streamed WAL file by xlogdump, I found that recent WAL records that walsender has

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 09:29, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander mag...@hagander.net wrote: I've applied this version with a few more minor changes that Heikki found. Cool! When I tried pg_receivexlog and checked the contents of

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 09:29, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander mag...@hagander.net wrote: I've applied this version with a few more minor changes that Heikki

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 09:46, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 09:29, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 3:29 AM, Magnus Hagander mag...@hagander.net

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 09:46, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 09:29, Fujii Masao masao.fu...@gmail.com wrote:

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 10:12, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 4:49 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 09:46, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 4:40 PM, Magnus Hagander mag...@hagander.net

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander mag...@hagander.net wrote: Not sure I follow. When we arrive at PQgetCopyData() there should be nothing buffered, and if the end of stream happens there it returns -1, and we exit, no? So where is the data that's lost? I do realize we don't

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander mag...@hagander.net wrote: Not sure I follow. When we arrive at PQgetCopyData() there should be nothing buffered, and if the end of stream happens there it returns -1, and

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 12:29, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander mag...@hagander.net wrote: Not sure I follow. When we arrive at PQgetCopyData() there should be

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Fujii Masao
On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, Oct 27, 2011 at 12:29, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 6:25 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 27, 2011 at 5:18 PM, Magnus Hagander mag...@hagander.net

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Heikki Linnakangas
On 27.10.2011 14:09, Fujii Masao wrote: On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagandermag...@hagander.net wrote: I'm rewriting the handling of partial files per the other thread started by Heikki. The idea is that there will be an actual .partial file in there when pg_receivexlog has ended,

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 27.10.2011 14:09, Fujii Masao wrote: On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagandermag...@hagander.net  wrote: I'm rewriting the handling of partial files per the other thread started by

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Robert Haas
On Thu, Oct 27, 2011 at 7:19 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 27.10.2011 14:09, Fujii Masao wrote: On Thu, Oct 27, 2011 at 7:48 PM, Magnus Hagandermag...@hagander.net  wrote: I'm rewriting the handling of partial files per the other thread started by

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 27.10.2011 14:09, Fujii Masao wrote: Yes. But that sounds unuserfriendly. Padding the WAL file manually is easy-to-do for a user? I'd definitely want

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Magnus Hagander
On Thu, Oct 27, 2011 at 16:54, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 27.10.2011 14:09, Fujii Masao wrote: Yes. But that sounds unuserfriendly. Padding the

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-27 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: On Thu, Oct 27, 2011 at 13:19, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Perhaps we should add automatic padding in the server, though. It wouldn't take much code in the server, and would make life easier for people writing their

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-26 Thread Magnus Hagander
On Tue, Oct 25, 2011 at 12:37, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 14:40, Magnus Hagander mag...@hagander.net wrote: On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: +               /* +                * Looks like an

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Magnus Hagander
On Mon, Oct 24, 2011 at 16:12, Jaime Casanova ja...@2ndquadrant.com wrote: On Mon, Oct 24, 2011 at 7:40 AM, Magnus Hagander mag...@hagander.net wrote: synchronous_standby_names='*' is prone to such confusion in general, but it seems that it's particularly surprising if a running pg_basebackup

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Magnus Hagander
On Mon, Oct 24, 2011 at 13:46, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: +               /* +                * Looks like an xlog file. Parse it's position. s/it's/its/ +                */ +               if (sscanf(dirent-d_name, %08X%08X%08X, tli, log, seg) != 3) +  

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Jaime Casanova
On Mon, Oct 24, 2011 at 7:40 AM, Magnus Hagander mag...@hagander.net wrote: synchronous_standby_names='*' is prone to such confusion in general, but it seems that it's particularly surprising if a running pg_basebackup lets a commit in synchronous replication to proceed. Maybe we just need a

Re: [HACKERS] Updated version of pg_receivexlog

2011-10-24 Thread Heikki Linnakangas
+ /* +* Looks like an xlog file. Parse it's position. s/it's/its/ +*/ + if (sscanf(dirent-d_name, %08X%08X%08X, tli, log, seg) != 3) + { + fprintf(stderr, _(%s: could not parse xlog filename

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-29 Thread Magnus Hagander
On Thu, Sep 29, 2011 at 01:55, Jaime Casanova ja...@2ndquadrant.com wrote: On Wed, Sep 28, 2011 at 12:50 PM, Magnus Hagander mag...@hagander.net wrote: pg_receivexlog worked good in my tests. pg_basebackup with --xlog=stream gives me an already recycled wal segment message (note that the

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Jaime Casanova
On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander mag...@hagander.net wrote: Here's an updated version of pg_receivexlog, that should now actually work (it previously failed miserably when a replication record crossed a WAL file boundary - something which I at the time could not properly

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Jaime Casanova
On Wed, Sep 28, 2011 at 1:38 AM, Jaime Casanova ja...@2ndquadrant.com wrote: On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander mag...@hagander.net wrote: Here's an updated version of pg_receivexlog, that should now actually work (it previously failed miserably when a replication record crossed

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Magnus Hagander
On Wed, Sep 28, 2011 at 08:38, Jaime Casanova ja...@2ndquadrant.com wrote: On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander mag...@hagander.net wrote: Here's an updated version of pg_receivexlog, that should now actually work (it previously failed miserably when a replication record crossed a

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Magnus Hagander
On Wed, Sep 28, 2011 at 09:30, Jaime Casanova ja...@2ndquadrant.com wrote: On Wed, Sep 28, 2011 at 1:38 AM, Jaime Casanova ja...@2ndquadrant.com wrote: On Tue, Aug 16, 2011 at 9:32 AM, Magnus Hagander mag...@hagander.net wrote: Here's an updated version of pg_receivexlog, that should now

Re: [HACKERS] Updated version of pg_receivexlog

2011-09-28 Thread Jaime Casanova
On Wed, Sep 28, 2011 at 12:50 PM, Magnus Hagander mag...@hagander.net wrote: pg_receivexlog worked good in my tests. pg_basebackup with --xlog=stream gives me an already recycled wal segment message (note that the file was in pg_xlog in the standby): FATAL:  could not receive data from WAL