Re: [HACKERS] WAL recycle retading based on active sync rep.

[Kyotaro HORIGUCHI]

Hello,

At Fri, 18 Nov 2016 10:16:22 -0800, Andres Freund  wrote in 
<20161118181622.hklschaizwaxo...@alap3.anarazel.de>
> Hi,
> 
> On 2016-11-18 14:12:42 +0900, Kyotaro HORIGUCHI wrote:
> > We had too-early WAL recycling during a test we had on a sync
> > replication set. This is not a bug and a bit extreme case but is
> > contrary to expectation on synchronous replication.
> 
> I don't think you can expect anything else.

My sentense was inaccurate. "is contrary to *naive* expectation
on synchronous replication." But I agree to you.

> > This is because sync replication doesn't wait non-commit WALs to
> > be replicated. This situation is artificially caused with the
> > first patch attached and the following steps.
> 
> You could get that situation even if we waited for syncrep. The
> SyncRepWaitForLSN happens after delayChkpt is unset.
> 
> Additionally a syncrep connection could break for a a short while, and
> you'd loose all guarantees anyway.

I know. Replication slots are for such cases.

> > - Is this situation required to be saved? This is caused by a
> >   large transaction, spans over two max_wal_size segments, or
> >   replication stall lasts for a chackepoint period.
> 
> I very strongly think not.
> 
> 
> > - Is the measure acceptable?  For the worst case, a master
> >   crashes from WAL space exhaustion. (But such large transaction
> >   won't/shouldn't exist?)
> 
> No, imo not.

Thanks for clarifying that.

regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL recycle retading based on active sync rep.

[Kyotaro HORIGUCHI]

Thanks for the comment.

At Fri, 18 Nov 2016 17:06:55 +0800, Craig Ringer  
wrote in 
> > We had too-early WAL recycling during a test we had on a sync
> > replication set. This is not a bug and a bit extreme case but is
> > contrary to expectation on synchronous replication.
> 
> Isn't this prevented by using a physical replication slot?
> 
> You hint that you looked at slots but they didn't meet your needs in some
> way. I'm not sure I understood the last part.

Yes, repslot does the similar. The point was whether "Do we
expect that removal of necessary WAL doesn't occur on an active
sync replication?", with a strong doubt.

At Fri, 18 Nov 2016 10:16:22 -0800, Andres Freund  wrote in 
<20161118181622.hklschaizwaxo...@alap3.anarazel.de>
> On 2016-11-18 14:12:42 +0900, Kyotaro HORIGUCHI wrote:
> > We had too-early WAL recycling during a test we had on a sync
> > replication set. This is not a bug and a bit extreme case but is
> > contrary to expectation on synchronous replication.
> 
> I don't think you can expect anything else.

I think this is the answer for it.

regards,

-- 
堀口恭太郎

日本電信電話株式会社 NTTオープンソースソフトウェアセンタ
Phone: 03-5860-5115 / Fax: 03-5463-5490




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL recycle retading based on active sync rep.

[Andres Freund]

Hi,

On 2016-11-18 14:12:42 +0900, Kyotaro HORIGUCHI wrote:
> We had too-early WAL recycling during a test we had on a sync
> replication set. This is not a bug and a bit extreme case but is
> contrary to expectation on synchronous replication.

I don't think you can expect anything else.


> This is because sync replication doesn't wait non-commit WALs to
> be replicated. This situation is artificially caused with the
> first patch attached and the following steps.

You could get that situation even if we waited for syncrep. The
SyncRepWaitForLSN happens after delayChkpt is unset.

Additionally a syncrep connection could break for a a short while, and
you'd loose all guarantees anyway.


> - Is this situation required to be saved? This is caused by a
>   large transaction, spans over two max_wal_size segments, or
>   replication stall lasts for a chackepoint period.

I very strongly think not.


> - Is the measure acceptable?  For the worst case, a master
>   crashes from WAL space exhaustion. (But such large transaction
>   won't/shouldn't exist?)

No, imo not.


Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL recycle retading based on active sync rep.

[Craig Ringer]

On 18 Nov. 2016 13:14, "Kyotaro HORIGUCHI" 
wrote:
>
> Hello.
>
> We had too-early WAL recycling during a test we had on a sync
> replication set. This is not a bug and a bit extreme case but is
> contrary to expectation on synchronous replication.

Isn't this prevented by using a physical replication slot?

You hint that you looked at slots but they didn't meet your needs in some
way. I'm not sure I understood the last part.