Re: [HACKERS] about fsync in CLOG buffer write

On Sun, Oct 4, 2015 at 3:25 PM, Andres Freund wrote: > I don't think it's worth investing time and complexity to bypass SLRU in > certain cases. We should rather rewrite the thing completely. +1. That code is considerably past its sell-by date. -- Robert Haas EnterpriseDB:

Re: [HACKERS] about fsync in CLOG buffer write

On Sat, Sep 12, 2015 at 5:21 PM, Andres Freund wrote: > On September 12, 2015 5:18:28 PM PDT, Jeff Janes > wrote: > >On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund > >wrote: > > > >> On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > >>

Re: [HACKERS] about fsync in CLOG buffer write

On 2015-10-04 12:14:05 -0700, Jeff Janes wrote: > My (naive) expectation is that no additional locking is needed. > > Once we decide to consult the clog, we already know the transaction is no > longer in progress, so it can't be in-flight to change that clog entry we > care about because it was

Re: [HACKERS] about fsync in CLOG buffer write

On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund wrote: > On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > > We found there is a fsync call when CLOG buffer > > is written out in SlruPhysicalWritePage(). It is often called when a > backend > > needs to check transaction status

Re: [HACKERS] about fsync in CLOG buffer write

On September 12, 2015 5:18:28 PM PDT, Jeff Janes wrote: >On Wed, Sep 2, 2015 at 5:32 AM, Andres Freund >wrote: > >> On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> > We found there is a fsync call when CLOG buffer >> > is written out in

Re: [HACKERS] Re: [HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

Robert Haas writes: > ... How often such a workload actually has to replace a *dirty* clog > buffer obviously depends on how often you checkpoint, but if you're > getting ~28k TPS you can completely fill 32 clog buffers (1 million > transactions) in less than 40 seconds,

[HACKERS] Re: [HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

On Tue, Sep 8, 2015 at 2:28 PM, Andres Freund wrote: > On 2015-09-08 15:58:26 +0800, 周正中(德歌) wrote: >> postgres@digoal-> cat 7.sql >> select txid_current(); >> >> postgres@digoal-> pgbench -M prepared -n -r -P 1 -f ./7.sql -c 1 -j 1 -T >> 10 >> About 32K tps. >> progress:

Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

On 2015-09-09 10:46:47 -0400, Robert Haas wrote: > Well, if you're filling ~1 clog page per second, you're doing ~1 fsync > per second too. Or if you are not, then you are thrashing the > progressively smaller and smaller number of clean slots ever-harder > until no clean pages remain and you're

[HACKERS] Re: [HACKERS] Re: [HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

On Wed, Sep 9, 2015 at 10:35 AM, Tom Lane wrote: > Robert Haas writes: >> ... How often such a workload actually has to replace a *dirty* clog >> buffer obviously depends on how often you checkpoint, but if you're >> getting ~28k TPS you can completely

[HACKERS] 答复：[HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

hou@alibaba-inc.com>，pgsql-hackers <pgsql-hackers@postgresql.org>，范孝剑(康贤) <funnyxj@alibaba-inc.com>，曾文旌(义从) <wenjing@alibaba-inc.com>，窦贤明(执白) <xianming@alibaba-inc.com>，萧少聪(铁庵) <shaocong@alibaba-inc.com>，陈新坚(惧留孙) <xinjian.c...@alibaba-inc.com>

[HACKERS] Re: [HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

On Sun, Sep 6, 2015 at 7:34 AM, 周正中(德歌) wrote: > > >On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > >> We found there is a fsync call when CLOG buffer > > >> is written out in SlruPhysicalWritePage(). It is often called when a > >> backend > >> needs to check

[HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

/transam/slru.c:699").return,  par:'\001' fsync interval about 1 sec, every 32KB xacts, per CLOG PAGE. postgres=# select 1441503928836-1441503927731;  ?column?  --  1105 (1 row) postgres=# select 1441503929819-1441503928836;  ?column?  --   983 (1 row) ---

Re: [HACKERS] 答复：[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

Hi, On 2015-09-08 15:58:26 +0800, 周正中(德歌) wrote: > postgres@digoal-> cat 7.sql > select txid_current(); > > postgres@digoal-> pgbench -M prepared -n -r -P 1 -f ./7.sql -c 1 -j 1 -T  > 10 > About 32K tps. > progress: 240.0 s, 31164.4 tps, lat 0.031 ms stddev 0.183 > progress: 241.0 s, 33243.3 

Re: [HACKERS] about fsync in CLOG buffer write

Andres, Thanks for the reply! I will try the ForwardFsyncRequest-like approach. 在 15-9-2 下午8:32， "Andres Freund" 写入: >On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> We found there is a fsync call when CLOG buffer >> is written out in SlruPhysicalWritePage(). It is often

[HACKERS] 答复：[HACKERS] about fsync in CLOG buffer write

>On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: >> We found there is a fsync call when CLOG buffer >> is written out in SlruPhysicalWritePage(). It is often called when a backend >> needs to check transaction status with SimpleLruReadPage(). >That's when there's not enough buffers available some 

[HACKERS] about fsync in CLOG buffer write

Hi Hackers, We host many PG instances on each of our machines, and we want to minimize the calls to fsync within each instance, in order to minimize possible impact on other instances. We found there is a fsync call when CLOG buffer is written out in SlruPhysicalWritePage(). It is often called

Re: [HACKERS] about fsync in CLOG buffer write

On 2015-09-10 19:39:59 +0800, 张广舟(明虚) wrote: > We found there is a fsync call when CLOG buffer > is written out in SlruPhysicalWritePage(). It is often called when a backend > needs to check transaction status with SimpleLruReadPage(). That's when there's not enough buffers available some other,