Re: shared-memory based stats collector - v66

2022-04-05 Thread Andres Freund
Hi, On 2022-04-02 01:16:48 -0700, Andres Freund wrote: > I just noticed that the code doesn't appear to actually work like that right > now. Whenever the timeout is reached, pgstat_report_stat() is called with > force = true. > > And even if the backend is busy running queries, once there's conte

Re: shared-memory based stats collector - v66

2022-04-02 Thread Andres Freund
Hi, On 2022-03-25 17:24:18 +0900, Kyotaro Horiguchi wrote: > > * AFIXME: Should all the stats drop code be moved into pgstat_drop.c? > > Or pgstat_xact.c? I wasn't initially happy with that suggestion, but after running with it, it looks pretty good. I also moved a fair bit of code into pgstat_

Re: shared-memory based stats collector - v66

2022-04-02 Thread Andres Freund
Hi, On 2022-03-25 17:24:18 +0900, Kyotaro Horiguchi wrote: > > AFIXME: Isn't PGSTAT_MIN_INTERVAL way too long? What is the justification > > for increasing it? > > It is 1000ms in the comment just above but actually 1ms. The > number came from a discussion that if we have 1000 clients and eac

Re: shared-memory based stats collector - v66

2022-03-29 Thread Andres Freund
Hi, On 2022-03-25 17:24:18 +0900, Kyotaro Horiguchi wrote: > I'd like to dump out my humble thoughts about other AFIXMEs.. Thanks! Please have another look at the code in https://github.com/anarazel/postgres/tree/shmstat I just pushed a revised version with a lot of [a]fixmes removed. Most imp

Re: shared-memory based stats collector - v66

2022-03-25 Thread Kyotaro Horiguchi
At Fri, 25 Mar 2022 14:22:56 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 24 Mar 2022 13:21:33 -0400, Melanie Plageman > wrote in > > On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > > > > > The biggest todos are: > > > - Address all the remaining AFIXMEs and XXXs > > > > Attac

Re: shared-memory based stats collector - v66

2022-03-24 Thread Kyotaro Horiguchi
At Thu, 24 Mar 2022 13:21:33 -0400, Melanie Plageman wrote in > On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > > > The biggest todos are: > > - Address all the remaining AFIXMEs and XXXs > > Attached is a patch that addresses three of the existing AFIXMEs. Thanks! + .

Re: shared-memory based stats collector - v66

2022-03-24 Thread Melanie Plageman
On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > The biggest todos are: > - Address all the remaining AFIXMEs and XXXs Attached is a patch that addresses three of the existing AFIXMEs. From 2a975cdb5d10ec365ca2ced39b9f99a9385b6268 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Wed

Re: shared-memory based stats collector - v66

2022-03-22 Thread Melanie Plageman
On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > I've attached a substantially improved version of the shared memory stats > patch. ... > - lot of the pg_stat_ views like bgwriter, pg_stat_database have zero > coverage today Attached are some tests including tests that reset of stats wor

Re: shared-memory based stats collector - v66

2022-03-21 Thread Melanie Plageman
On Sun, Mar 20, 2022 at 4:56 PM Melanie Plageman wrote: > > Addressed all of these points in > v2-0001-add-replica-cleanup-tests.patch > > also added a new test file in > v2-0002-Add-TAP-test-for-discarding-stats-after-crash.patch > testing correct behavior after a crash and when stats file is inv

Re: shared-memory based stats collector - v66

2022-03-20 Thread Melanie Plageman
On Sun, Mar 20, 2022 at 12:58 PM Andres Freund wrote: > > Hi, > > On 2022-03-20 12:32:39 -0400, Melanie Plageman wrote: > > Attached is a TAP test to check that stats are cleaned up on a physical > > replica after the objects they concern are dropped on the primary. > > Thanks! > > > > I'm not sur

Re: shared-memory based stats collector - v66

2022-03-20 Thread Andres Freund
Hi, On 2022-03-20 12:32:39 -0400, Melanie Plageman wrote: > Attached is a TAP test to check that stats are cleaned up on a physical > replica after the objects they concern are dropped on the primary. Thanks! > I'm not sure that the extra force next flush on standby is needed after > drop on pr

Re: shared-memory based stats collector - v66

2022-03-20 Thread Melanie Plageman
On Thu, Mar 17, 2022 at 3:36 AM Andres Freund wrote: > > Starting to feel more optimistic about this! There's loads more to do, but now > the TODOs just seem to require elbow grease, rather than deep thinking. > > The biggest todos are: > - Address all the remaining AFIXMEs and XXXs > > - add long