Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-12 Thread Fujii Masao
On Tue, Aug 12, 2014 at 4:34 PM, Michael Paquier wrote: > On Fri, Aug 8, 2014 at 5:10 PM, Fujii Masao wrote: >> Thanks for the review! Applied the patch. > I noticed that the tab padding for the new option -F in the getops > switch is incorrect. Attached patch corrects that. pgindent would have >

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-12 Thread Michael Paquier
On Fri, Aug 8, 2014 at 5:10 PM, Fujii Masao wrote: > Thanks for the review! Applied the patch. I noticed that the tab padding for the new option -F in the getops switch is incorrect. Attached patch corrects that. pgindent would have caught that anyway, but it doesn't hurt to be correct now. Thanks

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-08 Thread Fujii Masao
On Thu, Aug 7, 2014 at 5:24 PM, wrote: >> Okay, applied the patch. >> >> I heavily modified your patch based on the master that the refactoring >> patch has been applied. Attached is that modified version. Could you >> review that? > > Thank you for the patch. > I did a review of the patch. > > N

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-07 Thread furuyao
> Okay, applied the patch. > > I heavily modified your patch based on the master that the refactoring > patch has been applied. Attached is that modified version. Could you > review that? Thank you for the patch. I did a review of the patch. No problem in the patch. Behavior after the true re

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-06 Thread Fujii Masao
On Wed, Aug 6, 2014 at 5:10 PM, wrote: >> > - break; /* ignore >> the rest of this XLogData packet */ >> > >> > + return true;/* ignore the rest of >> this XLogData packet */ >> > >> > For break statement at close of

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-06 Thread furuyao
> > - break; /* ignore > the rest of this XLogData packet */ > > > > + return true;/* ignore the rest of > this XLogData packet */ > > > > For break statement at close of wal file, it is a return to true. > > It may be

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-05 Thread Fujii Masao
On Wed, Aug 6, 2014 at 2:34 PM, wrote: >> >> I have improved the patch by making following changes: >> >> >> >> 1. Since stream_stop() was redundant, stream_stop() at the time of >> WAL file closing was deleted. >> >> >> >> 2. Change the Flash judging timing for the readability of source code. >

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-05 Thread furuyao
> >> I have improved the patch by making following changes: > >> > >> 1. Since stream_stop() was redundant, stream_stop() at the time of > WAL file closing was deleted. > >> > >> 2. Change the Flash judging timing for the readability of source code. > >>I have changed the Flash judging timing

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-05 Thread Fujii Masao
On Tue, Aug 5, 2014 at 9:04 PM, Fujii Masao wrote: > On Tue, Jul 29, 2014 at 7:07 PM, wrote: >> I have improved the patch by making following changes: >> >> 1. Since stream_stop() was redundant, stream_stop() at the time of WAL file >> closing was deleted. >> >> 2. Change the Flash judging tim

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-05 Thread Fujii Masao
On Tue, Jul 29, 2014 at 7:07 PM, wrote: > I have improved the patch by making following changes: > > 1. Since stream_stop() was redundant, stream_stop() at the time of WAL file > closing was deleted. > > 2. Change the Flash judging timing for the readability of source code. >I have changed

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-29 Thread furuyao
I have improved the patch by making following changes: 1. Since stream_stop() was redundant, stream_stop() at the time of WAL file closing was deleted. 2. Change the Flash judging timing for the readability of source code. I have changed the Flash judging timing , from the continuous messag

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-25 Thread furuyao
This patch was made by the following process. 1. post patch for additional pg_receivexlog synchronous mode. 2. In response to comment for the flush frequency, I revise the patch to do flush every consecutive message in reference to walreceiver. 3. The synchronization mode was necessary to reply

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-10 Thread furuyao
This patch is modified the comment. Each comment is coping as follows. > Could you update the status of this patch from "Waiting on Author" to > "Needs Review" when you post the revised version of the patch? Thank you for pointing out. From now on, I will update status when I post the patch. > +

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-07 Thread Fujii Masao
On Fri, Jul 4, 2014 at 7:45 PM, wrote: >> > Thanks for reviewing the patch! >> > >> > I think that this refactoring patch is useful for improving source >> > code readability and making the future patches simpler, whether we >> > adopt your patch or not. So, barring any objections, I'm thinking t

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-04 Thread furuyao
> > Thanks for reviewing the patch! > > > > I think that this refactoring patch is useful for improving source > > code readability and making the future patches simpler, whether we > > adopt your patch or not. So, barring any objections, I'm thinking to > > commit this refactoring patch. > > Comm

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-03 Thread Fujii Masao
On Tue, Jul 1, 2014 at 10:11 PM, Fujii Masao wrote: > On Mon, Jun 30, 2014 at 7:09 PM, wrote: >>> Thanks for the review! >>> >>> +if (secs <= 0) >>> +secs = 1;/* Always sleep at least 1 sec */ >>> + >>> +sleeptime = secs * 1000 + usecs / 1000; >>> >>>

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-07-01 Thread Fujii Masao
On Mon, Jun 30, 2014 at 7:09 PM, wrote: >> Thanks for the review! >> >> +if (secs <= 0) >> +secs = 1;/* Always sleep at least 1 sec */ >> + >> +sleeptime = secs * 1000 + usecs / 1000; >> >> The above is the code which caused that problem. 'usecs' should

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-30 Thread furuyao
> > > > Synchronous(synchronous_commit = on) mode offers the ability to > > > confirm WAL have been streamed in the same way as synchronous > > > replication. > > > > If an output is used as a different disk from the directory where > > > > the > > > transaction log should be stored. > > > > Preven

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-30 Thread furuyao
> Thanks for the review! > > +if (secs <= 0) > +secs = 1;/* Always sleep at least 1 sec */ > + > +sleeptime = secs * 1000 + usecs / 1000; > > The above is the code which caused that problem. 'usecs' should have been > reset to zero when 'secs' are round

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-27 Thread Andres Freund
On 2014-06-05 19:13:34 +0900, furu...@pm.nttdata.co.jp wrote: > > -Original Message- > > Hi, > > > > On 2014-06-05 17:09:44 +0900, furu...@pm.nttdata.co.jp wrote: > > > Synchronous(synchronous_commit = on) mode offers the ability to > > confirm WAL have been streamed in the same way as syn

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-27 Thread Fujii Masao
On Thu, Jun 26, 2014 at 7:01 PM, wrote: >> The patch looks somewhat complicated and bugs can be easily introduced >> because it tries to not only add new feature but also reorganize the main >> loop in HandleCopyStream at the same time. To keep the patch simple, I'm >> thinking to firstly apply t

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-26 Thread furuyao
> The patch looks somewhat complicated and bugs can be easily introduced > because it tries to not only add new feature but also reorganize the main > loop in HandleCopyStream at the same time. To keep the patch simple, I'm > thinking to firstly apply the attached patch which just refactors the > m

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-25 Thread Fujii Masao
On Wed, Jun 25, 2014 at 3:50 AM, Fujii Masao wrote: > On Tue, Jun 24, 2014 at 3:18 PM, wrote: >>> I found that this patch breaks --status-interval option of >>> pg_receivexlog when -m option which the patch introduced is supplied. >>> When -m is set, pg_receivexlog tries to send the feedback mes

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-24 Thread Fujii Masao
On Tue, Jun 24, 2014 at 3:18 PM, wrote: >> I found that this patch breaks --status-interval option of >> pg_receivexlog when -m option which the patch introduced is supplied. >> When -m is set, pg_receivexlog tries to send the feedback message as soon >> as it flushes WAL file even if status inte

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-23 Thread furuyao
> I found that this patch breaks --status-interval option of > pg_receivexlog when -m option which the patch introduced is supplied. > When -m is set, pg_receivexlog tries to send the feedback message as soon > as it flushes WAL file even if status interval timeout has not been passed > yet. If you

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-19 Thread Fujii Masao
On Mon, Jun 16, 2014 at 7:03 PM, wrote: >> You introduced the state machine using the flag "flush_flg" into >> pg_receivexlog. >> That's complicated and would reduce the readability of the source code. >> I think that the logic should be simpler like walreceiver's one. >> >> Maybe I found one pro

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-16 Thread furuyao
> You introduced the state machine using the flag "flush_flg" into > pg_receivexlog. > That's complicated and would reduce the readability of the source code. > I think that the logic should be simpler like walreceiver's one. > > Maybe I found one problematic path as follows: > > 1. WAL is writte

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-11 Thread Fujii Masao
On Tue, Jun 10, 2014 at 5:01 PM, wrote: >> No. IIUC walreceiver does flush *less* frequently than what you >> implemented on pg_receivexlog. Your version of pg_receivexlog tries to >> do flush every time when it receives one WAL chunk. OTOH, walreceiver >> does flush only when there is no extra W

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-10 Thread furuyao
> No. IIUC walreceiver does flush *less* frequently than what you > implemented on pg_receivexlog. Your version of pg_receivexlog tries to > do flush every time when it receives one WAL chunk. OTOH, walreceiver > does flush only when there is no extra WAL chunk in receive buffer. IOW, > after writi

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-06 Thread Fujii Masao
On Fri, Jun 6, 2014 at 8:05 PM, wrote: >> -Original Message- >> > > Flush is not performed every time write, it is performed >> > > collectively like walrecever. >> > >> > I only glanced at this, but afaics you're only flushing at the end >> > every WAL segment. That will result in absolu

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-06 Thread furuyao
> -Original Message- > > > Flush is not performed every time write, it is performed > > > collectively like walrecever. > > > > I only glanced at this, but afaics you're only flushing at the end > > every WAL segment. That will result in absolutely horrible performance, > right? > > Walrece

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-05 Thread furuyao
> -Original Message- > Hi, > > On 2014-06-05 17:09:44 +0900, furu...@pm.nttdata.co.jp wrote: > > Synchronous(synchronous_commit = on) mode offers the ability to > confirm WAL have been streamed in the same way as synchronous > replication. > > If an output is used as a different disk from

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-05 Thread Andres Freund
Hi, On 2014-06-05 17:09:44 +0900, furu...@pm.nttdata.co.jp wrote: > Synchronous(synchronous_commit = on) mode offers the ability to confirm WAL > have been streamed in the same way as synchronous replication. > If an output is used as a different disk from the directory where the > transaction l

[HACKERS] pg_receivexlog add synchronous mode

2014-06-05 Thread furuyao
Hi, This patch implements a pg_receivexlog add synchronous mode. Now, synchronous(synchronous_commit = remote_write) is supported. But synchronous(synchronous_commit = remote_write), if the server crashes then WAL file may not to be flushed to disk , causing data loss. Synchronous(synchronous_