Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Tom Lane
Jeff Janes writes: > I think the argument is that drawing the next value from a sequence can > generate xlog that needs to be flushed, but doesn't assign an xid. > I would think the sequence should flush that record before it hands out the > value, not before the commit, but... IIRC the argument

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Andres Freund
On 2014-01-16 11:01:29 -0800, Jeff Janes wrote: > I think the argument is that drawing the next value from a sequence can > generate xlog that needs to be flushed, but doesn't assign an xid. Then that should assign an xid. Which would yield correct behaviour with async commit where it's currently

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Jeff Janes
On Thu, Jan 16, 2014 at 9:37 AM, Andres Freund wrote: > On 2014-01-16 09:25:51 -0800, Jeff Janes wrote: > > On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund >wrote: > > > > > On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: > > > > But if the transaction would not have otherwise generated WAL (i.e.

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Andres Freund
On 2014-01-16 09:25:51 -0800, Jeff Janes wrote: > On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund wrote: > > > On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: > > > But if the transaction would not have otherwise generated WAL (i.e. a > > > select that did not have to do any HOT pruning, or an upda

Re: [HACKERS] Patch for fail-back without fresh backup

2014-01-16 Thread Jeff Janes
On Thu, Nov 21, 2013 at 2:43 PM, Andres Freund wrote: > On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: > > But if the transaction would not have otherwise generated WAL (i.e. a > > select that did not have to do any HOT pruning, or an update with zero > rows > > matching the where condition), doe

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-22 Thread Bruce Momjian
On Thu, Nov 21, 2013 at 11:43:34PM +0100, Andres Freund wrote: > On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: > > But if the transaction would not have otherwise generated WAL (i.e. a > > select that did not have to do any HOT pruning, or an update with zero rows > > matching the where condition

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-21 Thread Andres Freund
On 2013-11-21 14:40:36 -0800, Jeff Janes wrote: > But if the transaction would not have otherwise generated WAL (i.e. a > select that did not have to do any HOT pruning, or an update with zero rows > matching the where condition), doesn't it now have to flush and wait when > it would otherwise not?

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-21 Thread Jeff Janes
On Thu, Nov 21, 2013 at 12:31 PM, Alvaro Herrera wrote: > Bruce Momjian escribió: > > On Thu, Oct 24, 2013 at 11:14:14PM +0300, Heikki Linnakangas wrote: > > > On 24.10.2013 23:07, Josh Berkus wrote: > > > > >What kind of overhead are we talking about here? > > > > > > One extra WAL record wheneve

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-21 Thread Alvaro Herrera
Bruce Momjian escribió: > On Thu, Oct 24, 2013 at 11:14:14PM +0300, Heikki Linnakangas wrote: > > On 24.10.2013 23:07, Josh Berkus wrote: > > >What kind of overhead are we talking about here? > > > > One extra WAL record whenever a hint bit is set on a page, for the > > first time after a checkpo

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-12 Thread Bruce Momjian
On Thu, Oct 24, 2013 at 11:14:14PM +0300, Heikki Linnakangas wrote: > On 24.10.2013 23:07, Josh Berkus wrote: > >On 10/24/2013 11:12 AM, Heikki Linnakangas wrote: > >>On 24.10.2013 20:39, Josh Berkus wrote: > >>>On 10/24/2013 04:15 AM, Pavan Deolasee wrote: > If we do what you are suggesting, i

Re: [HACKERS] Patch for fail-back without fresh backup

2013-11-02 Thread Sawada Masahiko
On Fri, Oct 25, 2013 at 8:08 PM, Andres Freund wrote: > On 2013-10-24 13:51:52 -0700, Josh Berkus wrote: > > It entirely depends on your workload. If it happens to be something > like: > INSERT INTO table (lots_of_data); > CHECKPOINT; > SELECT * FROM TABLE; > > i.e. there's a checkpoint between lo

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-25 Thread Andres Freund
On 2013-10-24 22:57:29 +0200, Magnus Hagander wrote: > In fact I've been considering suggesting we might want to retire the > difference between archive and hot_standby as wal_level, because the > difference is usually so small. And the advantage of hot_standby is in > almost every case worth it. E

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-25 Thread Andres Freund
On 2013-10-24 13:51:52 -0700, Josh Berkus wrote: > On 10/24/2013 01:14 PM, Heikki Linnakangas wrote: > > One extra WAL record whenever a hint bit is set on a page, for the first > > time after a checkpoint. In other words, a WAL record needs to be > > written in the same circumstances as with page

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Michael Paquier
On Fri, Oct 25, 2013 at 5:57 AM, Magnus Hagander wrote: > In fact I've been considering suggesting we might want to retire the > difference between archive and hot_standby as wal_level, because the > difference is usually so small. And the advantage of hot_standby is in > almost every case worth i

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Sawada Masahiko
On Fri, Oct 25, 2013 at 5:57 AM, Magnus Hagander wrote: > On Thu, Oct 24, 2013 at 10:51 PM, Josh Berkus wrote: >> On 10/24/2013 01:14 PM, Heikki Linnakangas wrote: >> >> I think it would be worth estimating what this actually looks like in >> terms of log write quantity. My inclication is to say

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Magnus Hagander
On Thu, Oct 24, 2013 at 10:51 PM, Josh Berkus wrote: > On 10/24/2013 01:14 PM, Heikki Linnakangas wrote: >> One extra WAL record whenever a hint bit is set on a page, for the first >> time after a checkpoint. In other words, a WAL record needs to be >> written in the same circumstances as with pag

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
On 10/24/2013 01:14 PM, Heikki Linnakangas wrote: > One extra WAL record whenever a hint bit is set on a page, for the first > time after a checkpoint. In other words, a WAL record needs to be > written in the same circumstances as with page checksums, but the WAL > records are much smaller as they

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Heikki Linnakangas
On 24.10.2013 23:07, Josh Berkus wrote: On 10/24/2013 11:12 AM, Heikki Linnakangas wrote: On 24.10.2013 20:39, Josh Berkus wrote: On 10/24/2013 04:15 AM, Pavan Deolasee wrote: If we do what you are suggesting, it seems like a single line patch to me. In XLogSaveBufferForHint(), we probably nee

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
On 10/24/2013 11:12 AM, Heikki Linnakangas wrote: > On 24.10.2013 20:39, Josh Berkus wrote: >> On 10/24/2013 04:15 AM, Pavan Deolasee wrote: >>> If we do what you are suggesting, it seems like a single line patch >>> to me. >>> In XLogSaveBufferForHint(), we probably need to look at this >>> additi

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Heikki Linnakangas
On 24.10.2013 20:39, Josh Berkus wrote: On 10/24/2013 04:15 AM, Pavan Deolasee wrote: If we do what you are suggesting, it seems like a single line patch to me. In XLogSaveBufferForHint(), we probably need to look at this additional GUC to decide whether or not to backup the block. Wait, what?

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Josh Berkus
On 10/24/2013 04:15 AM, Pavan Deolasee wrote: > If we do what you are suggesting, it seems like a single line patch to me. > In XLogSaveBufferForHint(), we probably need to look at this additional GUC > to decide whether or not to backup the block. Wait, what? Why are we having an additional GUC?

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Alvaro Herrera
Pavan Deolasee escribió: > Yeah, I had brought up similar idea up thread. Right now wal_level is > nicely ordered. But with this additional logic, I am not sure if we would > need multiple new levels and also break that ordering (I don't know if its > important). For example, one may want to set u

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Pavan Deolasee
On Thu, Oct 24, 2013 at 5:45 PM, Heikki Linnakangas wrote: > >> > Will just have to figure out what we want the user interface to be like; > should it be a separate guc, or somehow cram it into wal_level? > > Yeah, I had brought up similar idea up thread. Right now wal_level is nicely ordered. B

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Heikki Linnakangas
On 24.10.2013 14:15, Pavan Deolasee wrote: On Thu, Oct 24, 2013 at 4:22 PM, Heikki Linnakangas wrote: To fix that, pg_rewind could always start the rewinding process from the last checkpoint before the point that the histories diverge, instead of the exact point of divergence. Is that someth

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Pavan Deolasee
On Thu, Oct 24, 2013 at 4:45 PM, Pavan Deolasee wrote: > . Or would the recovery logic apply first WAL without looking at the page > lsn ? (Sorry, may be I should read the code instead of asking you) > > Never mind. I realized it has to. That's the whole purpose of backing it up in the first place

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Pavan Deolasee
On Thu, Oct 24, 2013 at 4:22 PM, Heikki Linnakangas wrote: > . >> > > To fix that, pg_rewind could always start the rewinding process from the > last checkpoint before the point that the histories diverge, instead of the > exact point of divergence. Is that something required even if someone pl

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Heikki Linnakangas
On 24.10.2013 13:02, Pavan Deolasee wrote: Another difference AFAICS is that checksum feature needs the block to be backed up only after the first time a hint bit is updated after checkpoint. But for something like pg_rewind to work, we will need to WAL log every hint bit update on a page. So we

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-24 Thread Pavan Deolasee
On Mon, Oct 21, 2013 at 7:10 PM, Sawada Masahiko wrote: > > > I agree with you. > If writing FPW is not large performance degradation, it is just idea > that we can use to write FPW in same timing as checksum enabled. > i.g., if we support new wal_level, the system writes FPW when a simple > SELEC

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-21 Thread Sawada Masahiko
On Thu, Oct 10, 2013 at 1:41 PM, Pavan Deolasee wrote: > > > Not that I can find any flaw in the OP's patch, but given the major > objections and my own nervousness about documenting this new "failback safe" > standby mode, I am also inclining to improve pg_rewind or whatever it takes > to get it

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-09 Thread Pavan Deolasee
On Tue, Oct 8, 2013 at 9:22 PM, Heikki Linnakangas wrote: > >> > Yeah, I definitely think we should work on the pg_rewind approach instead > of this patch. It's a lot more flexible. The performance hit of WAL-logging > hint bit updates is the price you have to pay, but a lot of people were OK > wi

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-09 Thread Robert Haas
On Wed, Oct 9, 2013 at 4:54 AM, Samrat Revagade wrote: > On Tue, Oct 8, 2013 at 3:16 PM, Andres Freund > wrote: >> On 2013-10-08 15:07:02 +0530, Pavan Deolasee wrote: >> > On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko >> > wrote: >> > >> > > On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao >> > > w

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-09 Thread Samrat Revagade
On Tue, Oct 8, 2013 at 3:16 PM, Andres Freund wrote: > On 2013-10-08 15:07:02 +0530, Pavan Deolasee wrote: > > On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko >wrote: > > > > > On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao > wrote: > > > > > > > I attached the v12 patch which have modified based o

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Sawada Masahiko
On Tue, Oct 8, 2013 at 6:46 PM, Andres Freund wrote: > On 2013-10-08 15:07:02 +0530, Pavan Deolasee wrote: >> On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko wrote: >> >> > On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao wrote: >> > > >> > I attached the v12 patch which have modified based on above s

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Sawada Masahiko
On Tue, Oct 8, 2013 at 6:37 PM, Pavan Deolasee wrote: > > On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko > wrote: >> >> On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao wrote: >> > >> I attached the v12 patch which have modified based on above suggestions. > > > There are still some parts of this de

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Heikki Linnakangas
On 08.10.2013 13:00, Pavan Deolasee wrote: On Tue, Oct 8, 2013 at 3:16 PM, Andres Freundwrote: It is my impression that there still are several people having pretty fundamental doubts about this approach in general. From what I remember neither Heikki, Simon, Tom nor me were really convinced ab

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Pavan Deolasee
On Tue, Oct 8, 2013 at 3:16 PM, Andres Freund wrote: > > > It is my impression that there still are several people having pretty > fundamental doubts about this approach in general. From what I remember > neither Heikki, Simon, Tom nor me were really convinced about this > approach. > > IIRC you a

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Andres Freund
On 2013-10-08 15:07:02 +0530, Pavan Deolasee wrote: > On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko wrote: > > > On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao wrote: > > > > > I attached the v12 patch which have modified based on above suggestions. > > > > There are still some parts of this desi

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Pavan Deolasee
On Tue, Oct 8, 2013 at 2:33 PM, Sawada Masahiko wrote: > On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao wrote: > > > I attached the v12 patch which have modified based on above suggestions. > There are still some parts of this design/patch which I am concerned about. 1. The design clubs synchronou

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-08 Thread Sawada Masahiko
On Fri, Oct 4, 2013 at 4:32 PM, Fujii Masao wrote: > > You added several checks into SyncRepWaitForLSN() so that it can handle both > synchronous_transfer=data_flush and =commit. This change made the source code > of the function very complicated, I'm afraid. To simplify the source code, > what ab

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-04 Thread Fujii Masao
On Fri, Oct 4, 2013 at 1:46 PM, Sawada Masahiko wrote: > On Fri, Sep 27, 2013 at 6:44 PM, Sawada Masahiko > wrote: >> On Fri, Sep 27, 2013 at 5:18 PM, Pavan Deolasee >> wrote: >>> On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko >>> wrote: > Thank you for comment. I th

Re: [HACKERS] Patch for fail-back without fresh backup

2013-10-03 Thread Sawada Masahiko
On Fri, Sep 27, 2013 at 6:44 PM, Sawada Masahiko wrote: > On Fri, Sep 27, 2013 at 5:18 PM, Pavan Deolasee > wrote: >> On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko >> wrote: >>> >>> >>> > >>> >>> Thank you for comment. I think it is good simple idea. >>> In your opinion, if synchronous_transf

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Sawada Masahiko
On Fri, Sep 27, 2013 at 5:18 PM, Pavan Deolasee wrote: > On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko > wrote: >> >> >> > >> >> Thank you for comment. I think it is good simple idea. >> In your opinion, if synchronous_transfer is set 'all' and >> synchronous_commit is set 'on', >> the master

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Pavan Deolasee
On Fri, Sep 27, 2013 at 1:28 PM, Sawada Masahiko wrote: > > > > > Thank you for comment. I think it is good simple idea. > In your opinion, if synchronous_transfer is set 'all' and > synchronous_commit is set 'on', > the master wait for data flush eve if user sets synchronous_commit to > 'local' o

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-27 Thread Sawada Masahiko
On Thu, Sep 26, 2013 at 8:54 PM, Pavan Deolasee wrote: > > On Thu, Sep 19, 2013 at 4:02 PM, Fujii Masao wrote: >> >> >> >> Hmm... when synchronous_transfer is set to data_flush, >> IMO the intuitive behaviors are >> >> (1) synchronous_commit = on >> A data flush should wait for the corresponding

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-26 Thread Pavan Deolasee
On Thu, Sep 19, 2013 at 4:02 PM, Fujii Masao wrote: > > > Hmm... when synchronous_transfer is set to data_flush, > IMO the intuitive behaviors are > > (1) synchronous_commit = on > A data flush should wait for the corresponding WAL to be > flushed in the standby > > (2) synchronous_commit = remot

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-22 Thread Sawada Masahiko
On Fri, Sep 20, 2013 at 10:33 PM, Samrat Revagade wrote: > > > > On Fri, Sep 20, 2013 at 3:40 PM, Sameer Thakur > wrote: >> >> >> >> >>> >>> >Attached patch combines documentation patch and source-code patch. >> >> >> I have had a stab at reviewing the documentation. Have a look. >> > > Than

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-20 Thread Samrat Revagade
On Fri, Sep 20, 2013 at 3:40 PM, Sameer Thakur wrote: > > > > >>> >> >Attached patch combines documentation patch and source-code patch. >> > > I have had a stab at reviewing the documentation. Have a look. > > Thanks. Attached patch implements suggestions in documentation. But comments from Fujii

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-20 Thread Sameer Thakur
>> > >Attached patch combines documentation patch and source-code patch. > I have had a stab at reviewing the documentation. Have a look. --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -1749,6 +1749,50 @@ include 'filename' + + synchronous_transfer

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-19 Thread Sawada Masahiko
On Thu, Sep 19, 2013 at 7:32 PM, Fujii Masao wrote: > On Thu, Sep 19, 2013 at 7:07 PM, Sawada Masahiko > wrote: >> On Thu, Sep 19, 2013 at 12:25 PM, Fujii Masao wrote: >>> On Thu, Sep 19, 2013 at 11:48 AM, Sawada Masahiko >>> wrote: I attached the patch which I have modified. >>> >>> Tha

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-19 Thread Fujii Masao
On Thu, Sep 19, 2013 at 7:07 PM, Sawada Masahiko wrote: > On Thu, Sep 19, 2013 at 12:25 PM, Fujii Masao wrote: >> On Thu, Sep 19, 2013 at 11:48 AM, Sawada Masahiko >> wrote: >>> I attached the patch which I have modified. >> >> Thanks for updating the patch! >> >> Here are the review comments:

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-19 Thread Sawada Masahiko
On Thu, Sep 19, 2013 at 12:25 PM, Fujii Masao wrote: > On Thu, Sep 19, 2013 at 11:48 AM, Sawada Masahiko > wrote: >> I attached the patch which I have modified. > > Thanks for updating the patch! > > Here are the review comments: > Thank you for reviewing! > I got the compiler warning: > >

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-18 Thread Fujii Masao
On Thu, Sep 19, 2013 at 11:48 AM, Sawada Masahiko wrote: > I attached the patch which I have modified. Thanks for updating the patch! Here are the review comments: I got the compiler warning: syncrep.c:112: warning: unused variable 'i' How does synchronous_transfer work with synchronous_c

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-18 Thread Sawada Masahiko
On Wed, Sep 18, 2013 at 11:45 AM, Fujii Masao wrote: > On Wed, Sep 18, 2013 at 10:35 AM, Sawada Masahiko > wrote: >> On Tue, Sep 17, 2013 at 9:52 PM, Fujii Masao wrote: >>> I set up synchronous replication with synchronous_transfer = all, and then >>> I ran >>> pgbench -i and executed CHECKPOI

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-17 Thread Sawada Masahiko
On Wed, Sep 18, 2013 at 1:05 PM, Sawada Masahiko wrote: > On Wed, Sep 18, 2013 at 11:45 AM, Fujii Masao wrote: >> On Wed, Sep 18, 2013 at 10:35 AM, Sawada Masahiko >> wrote: >>> On Tue, Sep 17, 2013 at 9:52 PM, Fujii Masao wrote: I set up synchronous replication with synchronous_transfer

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-17 Thread Sawada Masahiko
On Wed, Sep 18, 2013 at 11:45 AM, Fujii Masao wrote: > On Wed, Sep 18, 2013 at 10:35 AM, Sawada Masahiko > wrote: >> On Tue, Sep 17, 2013 at 9:52 PM, Fujii Masao wrote: >>> I set up synchronous replication with synchronous_transfer = all, and then >>> I ran >>> pgbench -i and executed CHECKPOI

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-17 Thread Fujii Masao
On Wed, Sep 18, 2013 at 10:35 AM, Sawada Masahiko wrote: > On Tue, Sep 17, 2013 at 9:52 PM, Fujii Masao wrote: >> I set up synchronous replication with synchronous_transfer = all, and then I >> ran >> pgbench -i and executed CHECKPOINT in the master. After that, when I executed >> CHECKPOINT in

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-17 Thread Sawada Masahiko
On Tue, Sep 17, 2013 at 9:52 PM, Fujii Masao wrote: > On Tue, Sep 17, 2013 at 3:45 PM, Samrat Revagade > wrote: >> >>> > syncrep.c: In function ‘SyncRepReleaseWaiters’: >>> > syncrep.c:421:6: warning: variable ‘numdataflush’ set but not used >>> > [-Wunused-but-set-variable] >>> > >>> >>> Sorry I

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-17 Thread Fujii Masao
On Tue, Sep 17, 2013 at 3:45 PM, Samrat Revagade wrote: > >> > syncrep.c: In function ‘SyncRepReleaseWaiters’: >> > syncrep.c:421:6: warning: variable ‘numdataflush’ set but not used >> > [-Wunused-but-set-variable] >> > >> >> Sorry I forgot fix it. >> >> I have attached the patch which I modified

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-16 Thread Samrat Revagade
> > syncrep.c: In function ‘SyncRepReleaseWaiters’: > > syncrep.c:421:6: warning: variable ‘numdataflush’ set but not used > > [-Wunused-but-set-variable] > > > > Sorry I forgot fix it. > > I have attached the patch which I modified. > > Attached patch combines documentation patch and source-code p

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-12 Thread Sawada Masahiko
On Fri, Sep 13, 2013 at 1:11 AM, Peter Eisentraut wrote: > On 9/12/13 3:00 AM, Samrat Revagade wrote: >> >> We are improving the patch for Commit Fest 2 now. >> We will fix above compiler warnings as soon as possible and submit >> the patch >> >> >> Attached *synchronous_transfer_v5.pa

Re: [HACKERS] Patch for fail-back without fresh backup

2013-09-12 Thread Peter Eisentraut
On 9/12/13 3:00 AM, Samrat Revagade wrote: > > We are improving the patch for Commit Fest 2 now. > We will fix above compiler warnings as soon as possible and submit > the patch > > > Attached *synchronous_transfer_v5.patch* implements review comments from > commit fest-1 and reduces

Re: [HACKERS] Patch for fail-back without fresh backup

2013-08-25 Thread Sawada Masahiko
On Sat, Aug 24, 2013 at 11:38 PM, Peter Eisentraut wrote: > On Thu, 2013-07-11 at 23:42 +0900, Sawada Masahiko wrote: >> please find the attached patch. > > Please fix these compiler warnings: > > xlog.c:3117:2: warning: implicit declaration of function ‘SyncRepWaitForLSN’ > [-Wimplicit-function-

Re: [HACKERS] Patch for fail-back without fresh backup

2013-08-24 Thread Peter Eisentraut
On Thu, 2013-07-11 at 23:42 +0900, Sawada Masahiko wrote: > please find the attached patch. Please fix these compiler warnings: xlog.c:3117:2: warning: implicit declaration of function ‘SyncRepWaitForLSN’ [-Wimplicit-function-declaration] syncrep.c:414:6: warning: variable ‘numdataflush’ set but

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-15 Thread Samrat Revagade
> ToDo > 1. currently this patch supports synchronous transfer. so we can't set > different synchronous transfer mode to each server. > we need to improve the patch for support following cases. >- SYNC standby and make separate ASYNC failback safe standby >- ASYNC standby and make separ

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-11 Thread Sawada Masahiko
On Tue, Jul 9, 2013 at 11:45 PM, Sawada Masahiko wrote: > On Sun, Jul 7, 2013 at 4:27 PM, Sawada Masahiko wrote: > I found a bug which occurred when we do vacuum, and have fixed it. > yesterday (8th July) "Improve scalability of WAL insertions" patch is > committed to HEAD. so v2 patch does not a

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-09 Thread Sawada Masahiko
On Sun, Jul 7, 2013 at 4:27 PM, Sawada Masahiko wrote: > On Sun, Jul 7, 2013 at 4:19 PM, Sawada Masahiko wrote: >> On Mon, Jun 17, 2013 at 8:48 PM, Simon Riggs wrote: >>> On 17 June 2013 09:03, Pavan Deolasee wrote: >>> I agree. We should probably find a better name for this. Any suggestio

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-07 Thread Sawada Masahiko
On Sun, Jul 7, 2013 at 4:19 PM, Sawada Masahiko wrote: > On Mon, Jun 17, 2013 at 8:48 PM, Simon Riggs wrote: >> On 17 June 2013 09:03, Pavan Deolasee wrote: >> >>> I agree. We should probably find a better name for this. Any suggestions ? >> >> err, I already made one... >> But that's not t

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-07 Thread Sawada Masahiko
On Mon, Jun 17, 2013 at 8:48 PM, Simon Riggs wrote: > On 17 June 2013 09:03, Pavan Deolasee wrote: > >> I agree. We should probably find a better name for this. Any suggestions ? > > err, I already made one... > >>> But that's not the whole story. I can see some utility in a patch that >>> makes

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-02 Thread Amit Kapila
On Tuesday, July 02, 2013 11:16 AM Amit Kapila wrote: > On Friday, June 28, 2013 10:41 AM Sawada Masahiko wrote: > > On Wed, Jun 26, 2013 at 1:40 PM, Amit Kapila > > wrote: > > > On Tuesday, June 25, 2013 10:23 AM Amit Langote wrote: > > >> Hi, > > >> > > >> > > > >> >> So our proposal on this pro

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-02 Thread Sawada Masahiko
On Tue, Jul 2, 2013 at 2:45 PM, Amit Kapila wrote: > On Friday, June 28, 2013 10:41 AM Sawada Masahiko wrote: >> On Wed, Jun 26, 2013 at 1:40 PM, Amit Kapila >> wrote: >> > On Tuesday, June 25, 2013 10:23 AM Amit Langote wrote: >> >> Hi, >> >> >> >> > >> >> >> So our proposal on this problem is t

Re: [HACKERS] Patch for fail-back without fresh backup

2013-07-01 Thread Amit Kapila
On Friday, June 28, 2013 10:41 AM Sawada Masahiko wrote: > On Wed, Jun 26, 2013 at 1:40 PM, Amit Kapila > wrote: > > On Tuesday, June 25, 2013 10:23 AM Amit Langote wrote: > >> Hi, > >> > >> > > >> >> So our proposal on this problem is that we must ensure that > master > >> should > >> > not make

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-27 Thread Sawada Masahiko
On Wed, Jun 26, 2013 at 1:40 PM, Amit Kapila wrote: > On Tuesday, June 25, 2013 10:23 AM Amit Langote wrote: >> Hi, >> >> > >> >> So our proposal on this problem is that we must ensure that master >> should >> > not make any file system level changes without confirming that the >> >> corresponding

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-27 Thread Robert Haas
On Mon, Jun 17, 2013 at 7:48 AM, Simon Riggs wrote: >> I am told, one of the very popular setups for DR is to have one >> local sync standby and one async (may be cascaded by the local sync). Since >> this new feature is more useful for DR because taking a fresh backup on a >> slower link is even

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-27 Thread Sawada Masahiko
On Mon, Jun 24, 2013 at 10:47 PM, Sawada Masahiko wrote: > 1. synchronous standby and make same as failback safe standby > 2. asynchronous standby and make same as failback safe standby > > in above case, adding new parameter might be meaningless. but I think > that we should handle case not only

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-25 Thread Amit Kapila
On Tuesday, June 25, 2013 10:23 AM Amit Langote wrote: > Hi, > > > > >> So our proposal on this problem is that we must ensure that master > should > > not make any file system level changes without confirming that the > >> corresponding WAL record is replicated to the standby. > > > > How will

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-25 Thread Sawada Masahiko
On Tue, Jun 25, 2013 at 12:19 PM, Pavan Deolasee wrote: > > > > On Mon, Jun 24, 2013 at 7:17 PM, Sawada Masahiko > wrote: >> >> >> >> >> [Server] >> standby_name = 'slave1' >> synchronous_transfer = commit >> wal_sender_timeout = 30 >> [Server] >> standby_name = 'slave2' >> s

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-24 Thread Amit Langote
Hi, > >> So our proposal on this problem is that we must ensure that master should > not make any file system level changes without confirming that the >> corresponding WAL record is replicated to the standby. > > How will you take care of extra WAL on old master during recovery. If it > plays t

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-24 Thread Pavan Deolasee
On Mon, Jun 24, 2013 at 7:17 PM, Sawada Masahiko wrote: > > > > [Server] > standby_name = 'slave1' > synchronous_transfer = commit > wal_sender_timeout = 30 > [Server] > standby_name = 'slave2' > synchronous_transfer = all > wal_sender_timeout = 50 > --- > Wha

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-24 Thread Amit Langote
Hi, > parameter improvement idea is which we extend ini file for to set > parameter each standby. For example : > > > [Server] > standby_name = 'slave1' > synchronous_transfer = commit > wal_sender_timeout = 30 > [Server] > standby_name = 'slave2' > synchronous_transfer = all

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-24 Thread Sawada Masahiko
On Mon, Jun 17, 2013 at 8:48 PM, Simon Riggs wrote: > On 17 June 2013 09:03, Pavan Deolasee wrote: > >> I agree. We should probably find a better name for this. Any suggestions ? > > err, I already made one... > >>> But that's not the whole story. I can see some utility in a patch that >>> makes

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-19 Thread Amit Kapila
On Wednesday, June 19, 2013 10:45 PM Sawada Masahiko wrote: On Tuesday, June 18, 2013, Amit Kapila wrote: On Tuesday, June 18, 2013 12:18 AM Sawada Masahiko wrote: > On Sun, Jun 16, 2013 at 2:00 PM, Amit kapila > wrote: > > On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: > > On Sat, Jun

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-19 Thread Sawada Masahiko
On Tuesday, June 18, 2013, Amit Kapila wrote: > On Tuesday, June 18, 2013 12:18 AM Sawada Masahiko wrote: > > On Sun, Jun 16, 2013 at 2:00 PM, Amit kapila > > > > > > wrote: > > > On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: > > > On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila > > >

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-18 Thread Sawada Masahiko
On Tuesday, June 18, 2013, Amit Kapila wrote: > On Tuesday, June 18, 2013 12:18 AM Sawada Masahiko wrote: > > On Sun, Jun 16, 2013 at 2:00 PM, Amit kapila > > > > > > wrote: > > > On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: > > > On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila > > >

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-17 Thread Amit Kapila
On Tuesday, June 18, 2013 12:18 AM Sawada Masahiko wrote: > On Sun, Jun 16, 2013 at 2:00 PM, Amit kapila > wrote: > > On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: > > On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila > wrote: > >> > >> On Saturday, June 15, 2013 1:19 PM Sawada Masahiko w

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-17 Thread Sawada Masahiko
On Sun, Jun 16, 2013 at 2:00 PM, Amit kapila wrote: > On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: > On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila wrote: >> >> On Saturday, June 15, 2013 1:19 PM Sawada Masahiko wrote: >> On Fri, Jun 14, 2013 at 10:15 PM, Amit Kapila wrote: >>> On Fr

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-17 Thread Simon Riggs
On 17 June 2013 09:03, Pavan Deolasee wrote: > I agree. We should probably find a better name for this. Any suggestions ? err, I already made one... >> But that's not the whole story. I can see some utility in a patch that >> makes all WAL transfer synchronous, rather than just commits. Some >>

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-17 Thread Pavan Deolasee
On Sun, Jun 16, 2013 at 5:10 PM, Simon Riggs wrote: > > > My perspective is that if the master crashed, assuming that you know > everything about that and suddenly jumping back on seem like a recipe > for disaster. Attempting that is currently blocked by the technical > obstacles you've identifie

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-16 Thread Samrat Revagade
On Sun, Jun 16, 2013 at 11:08 PM, Simon Riggs wrote: > On 16 June 2013 17:25, Samrat Revagade wrote: > > > > > > On Sun, Jun 16, 2013 at 5:10 PM, Simon Riggs > wrote: > >> > >> > >> > >> So I strongly object to calling this patch anything to do with > >> "failback safe". You simply don't have e

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-16 Thread Simon Riggs
On 16 June 2013 17:25, Samrat Revagade wrote: > > > On Sun, Jun 16, 2013 at 5:10 PM, Simon Riggs wrote: >> >> >> >> So I strongly object to calling this patch anything to do with >> "failback safe". You simply don't have enough data to make such a bold >> claim. (Which is why we call it synchrono

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-16 Thread Samrat Revagade
On Sun, Jun 16, 2013 at 5:10 PM, Simon Riggs wrote: > > > So I strongly object to calling this patch anything to do with > "failback safe". You simply don't have enough data to make such a bold > claim. (Which is why we call it synchronous replication and not "zero > data loss", for example). > >

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-16 Thread Simon Riggs
On 14 June 2013 10:11, Samrat Revagade wrote: > We have already started a discussion on pgsql-hackers for the problem of > taking fresh backup during the failback operation here is the link for that: > > http://www.postgresql.org/message-id/caf8q-gxg3pqtf71nvece-6ozraew5pwhk7yqtbjgwrfu513...@mail

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-16 Thread Simon Riggs
On 14 June 2013 17:21, Jeff Davis wrote: > On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: >> Jeff Davis has a patch pending >> (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std >> flag down to MarkBufferDirtyHint() for exactly that reason. I thought we >> were on track

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Amit kapila
On Saturday, June 15, 2013 8:29 PM Sawada Masahiko wrote: On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila wrote: > > On Saturday, June 15, 2013 1:19 PM Sawada Masahiko wrote: > On Fri, Jun 14, 2013 at 10:15 PM, Amit Kapila wrote: >> On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: >>> Hello,

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Andres Freund
On 2013-06-15 11:36:54 -0700, Jeff Davis wrote: > On Fri, 2013-06-14 at 18:27 +0200, Andres Freund wrote: > > I'd like to see a comment around the memcpys in XLogSaveBufferForHint() > > that mentions that they are safe in a non std buffer due to > > XLogCheckBuffer setting an appropriate hole/offse

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Jeff Davis
On Fri, 2013-06-14 at 18:27 +0200, Andres Freund wrote: > I'd like to see a comment around the memcpys in XLogSaveBufferForHint() > that mentions that they are safe in a non std buffer due to > XLogCheckBuffer setting an appropriate hole/offset. Or make an explicit > change of the copy algorithm th

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Sawada Masahiko
On Sat, Jun 15, 2013 at 10:34 PM, Amit kapila wrote: > > On Saturday, June 15, 2013 1:19 PM Sawada Masahiko wrote: > On Fri, Jun 14, 2013 at 10:15 PM, Amit Kapila wrote: >> On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: >>> Hello, >> We have already started a discussion on pgsql-hac

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Amit kapila
On Saturday, June 15, 2013 1:19 PM Sawada Masahiko wrote: On Fri, Jun 14, 2013 at 10:15 PM, Amit Kapila wrote: > On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: >> Hello, > >>> We have already started a discussion on pgsql-hackers for the problem of >>> taking fresh backup during the fail

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-15 Thread Sawada Masahiko
On Fri, Jun 14, 2013 at 10:15 PM, Amit Kapila wrote: > On Friday, June 14, 2013 2:42 PM Samrat Revagade wrote: >> Hello, > >> We have already started a discussion on pgsql-hackers for the problem of > taking fresh backup during the failback operation here is the link for that: > >> > http://www.po

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Andres Freund
On 2013-06-14 09:21:12 -0700, Jeff Davis wrote: > On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: > > Jeff Davis has a patch pending > > (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std > > flag down to MarkBufferDirtyHint() for exactly that reason. I thought we > > wer

Re: [HACKERS] Patch for fail-back without fresh backup

2013-06-14 Thread Jeff Davis
On Fri, 2013-06-14 at 16:10 +0200, Andres Freund wrote: > Jeff Davis has a patch pending > (1365493015.7580.3240.camel@sussancws0025) that passes the buffer_std > flag down to MarkBufferDirtyHint() for exactly that reason. I thought we > were on track committing that, but rereading the thread it do

  1   2   >