Re: Adding wait events statistics

2025-08-01 Thread Bertrand Drouvot
Hi, On Wed, Jul 23, 2025 at 11:38:07AM -0400, Robert Haas wrote: > I think it's probably a mistake to even be thinking about this in > terms of wait events. It seems very reasonable to want more data about > what PostgreSQL backends are doing, but I don't really see any reason > why that should ha

Re: Adding wait events statistics

2025-07-29 Thread Robert Haas
On Mon, Jul 28, 2025 at 8:35 AM Bertrand Drouvot wrote: > Focusing on LWLocks, I think that for example the OidGenLock one is one > it would be interesting to have the duration for. That way one could start to > investigate if there is a very long run of used OID values with no gaps in > TOAST tab

Re: Adding wait events statistics

2025-07-28 Thread Bertrand Drouvot
Hi, On Thu, Jul 24, 2025 at 10:38:49AM -0400, Robert Haas wrote: > On Thu, Jul 24, 2025 at 7:52 AM Bertrand Drouvot > wrote: > > Well, the idea was more: as we speak about "wait" events then it would make > > sense to add their duration. And then, to have meaningful data to interpret > > the > >

Re: Adding wait events statistics

2025-07-24 Thread Robert Haas
On Thu, Jul 24, 2025 at 7:52 AM Bertrand Drouvot wrote: > Well, the idea was more: as we speak about "wait" events then it would make > sense to add their duration. And then, to have meaningful data to interpret > the > durations then it would make sense to add the counters. So that one could >

Re: Adding wait events statistics

2025-07-24 Thread Bertrand Drouvot
Hi, On Wed, Jul 23, 2025 at 11:38:07AM -0400, Robert Haas wrote: > On Tue, Jul 22, 2025 at 8:24 AM Bertrand Drouvot > wrote: > > So based on the cycles metric I think it looks pretty safe to implement for > > the > > whole majority of classes. > > I'm not convinced that this is either cheap eno

Re: Adding wait events statistics

2025-07-24 Thread Bertrand Drouvot
Hi, On Tue, Jul 22, 2025 at 10:07:30AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-22 12:24:46 +, Bertrand Drouvot wrote: > > Anyway, let's forget about eBPF, I ran another experiment by counting the > > cycles > > with: > > > > static inline uint64_t rdtsc(void) { > > uint32_t lo,

Re: Adding wait events statistics

2025-07-23 Thread Robert Haas
On Tue, Jul 22, 2025 at 8:24 AM Bertrand Drouvot wrote: > So based on the cycles metric I think it looks pretty safe to implement for > the > whole majority of classes. I'm not convinced that this is either cheap enough to implement, and I don't understand the value proposition, either. I see th

Re: Adding wait events statistics

2025-07-22 Thread Andres Freund
Hi, On 2025-07-22 12:24:46 +, Bertrand Drouvot wrote: > Anyway, let's forget about eBPF, I ran another experiment by counting the > cycles > with: > > static inline uint64_t rdtsc(void) { > uint32_t lo, hi; > __asm__ __volatile__("rdtsc" : "=a" (lo), "=d" (hi)); > return ((uint64_

Re: Adding wait events statistics

2025-07-22 Thread Bertrand Drouvot
Hi, On Fri, Jul 18, 2025 at 11:43:47AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-18 06:04:38 +, Bertrand Drouvot wrote: > > Here what I’ve done: > > > > 1. Added 2 probes: one in pgstat_report_wait_start() and one in > > pgstat_report_wait_end() > > to measure the wait events duration

Re: Adding wait events statistics

2025-07-18 Thread Andres Freund
Hi, On 2025-07-18 06:04:38 +, Bertrand Drouvot wrote: > Here what I’ve done: > > 1. Added 2 probes: one in pgstat_report_wait_start() and one in > pgstat_report_wait_end() > to measure the wait events duration. Then, I ran make check-world and computed > the duration percentiles (in microseco

Re: Adding wait events statistics

2025-07-17 Thread Bertrand Drouvot
Hi, On Tue, Jul 15, 2025 at 04:13:49PM +0200, Bertrand Drouvot wrote: > Hi, > > On Fri, Jul 11, 2025 at 11:59:39AM -0400, Andres Freund wrote: > > Hi, > > > > On 2025-07-10 08:21:56 +, Bertrand Drouvot wrote: > > > - Let's define a list of "cheap" and a list of "expensive" wait events > > >

Re: Adding wait events statistics

2025-07-15 Thread Bertrand Drouvot
Hi, On Fri, Jul 11, 2025 at 11:59:39AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-10 08:21:56 +, Bertrand Drouvot wrote: > > - Let's define a list of "cheap" and a list of "expensive" wait events > > - Change the code path to increment counters only for the "expensive" ones > > - Optimi

Re: Adding wait events statistics

2025-07-11 Thread Andres Freund
Hi, On 2025-07-10 08:21:56 +, Bertrand Drouvot wrote: > - Let's define a list of "cheap" and a list of "expensive" wait events > - Change the code path to increment counters only for the "expensive" ones > - Optimize the counter incrementation as much as possible > > Does that sound reasonabl

Re: Adding wait events statistics

2025-07-10 Thread Bertrand Drouvot
Hi, On Wed, Jul 09, 2025 at 11:19:00AM -0400, Andres Freund wrote: > Hi, > > On 2025-07-09 07:26:05 +, Bertrand Drouvot wrote: > > On Tue, Jul 08, 2025 at 10:19:07PM -0400, Andres Freund wrote: > > > On 2025-06-30 13:36:12 +, Bertrand Drouvot wrote: > > TBH, the more I think about this,

Re: Adding wait events statistics

2025-07-09 Thread Andres Freund
Hi, On 2025-07-09 07:26:05 +, Bertrand Drouvot wrote: > On Tue, Jul 08, 2025 at 10:19:07PM -0400, Andres Freund wrote: > > On 2025-06-30 13:36:12 +, Bertrand Drouvot wrote: > > > * we don't want to allocate memory in some places where the counters are > > > incremented (see 4feba03d8b9).

Re: Adding wait events statistics

2025-07-09 Thread Bertrand Drouvot
Hi, On Wed, Jul 09, 2025 at 09:56:10AM +0900, Michael Paquier wrote: > On Tue, Jul 08, 2025 at 03:25:26PM +, Bertrand Drouvot wrote: > > So, something like: > > > > ClassInfo LWLockClassInfo = { > > .numberOfEvents = NB_LWLOCK_WAIT_EVENTS, > > .pendingCounts = PendingWaitEventStats.lw

Re: Adding wait events statistics

2025-07-09 Thread Bertrand Drouvot
Hi, On Tue, Jul 08, 2025 at 10:19:07PM -0400, Andres Freund wrote: > On 2025-06-30 13:36:12 +, Bertrand Drouvot wrote: > > * we don't want to allocate memory in some places where the counters are > > incremented (see 4feba03d8b9). We could still use the same implementation > > as for > > b

Re: Adding wait events statistics

2025-07-08 Thread Andres Freund
Hi, On 2025-06-30 13:36:12 +, Bertrand Drouvot wrote: > Wait events are useful to know what backends are waiting for when there is/was > a performance issue: for this we can sample pg_stat_activity at regular > intervals > and record historical data. That’s how it is commonly used. > > It cou

Re: Adding wait events statistics

2025-07-08 Thread Michael Paquier
On Tue, Jul 08, 2025 at 03:25:26PM +, Bertrand Drouvot wrote: > So, something like: > > ClassInfo LWLockClassInfo = { > .numberOfEvents = NB_LWLOCK_WAIT_EVENTS, > .pendingCounts = PendingWaitEventStats.lwlock_counts > }; > > and then the "global" one: > > WaitClassInfo *AllWaitClasse

Re: Adding wait events statistics

2025-07-08 Thread Bertrand Drouvot
Hi, On Tue, Jul 08, 2025 at 04:46:28AM +, Bertrand Drouvot wrote: > On Tue, Jul 08, 2025 at 09:36:53AM +0900, Michael Paquier wrote: > > Yes, you may need some level of meta-data generated for the wait > > classes generated when the perl script generating this data is run. > > It would be nice

Re: Adding wait events statistics

2025-07-07 Thread Bertrand Drouvot
Hi, On Tue, Jul 08, 2025 at 09:36:53AM +0900, Michael Paquier wrote: > On Mon, Jul 07, 2025 at 03:07:12PM +, Bertrand Drouvot wrote: > > > I think the question is: if the extension owner does not increment it, do > > we want > > to do it in core on their behalf? I'm not sure as it's still up

Re: Adding wait events statistics

2025-07-07 Thread Michael Paquier
On Mon, Jul 07, 2025 at 03:07:12PM +, Bertrand Drouvot wrote: > On Mon, Jul 07, 2025 at 02:56:29PM +0900, Michael Paquier wrote: >> On Mon, Jun 30, 2025 at 01:36:12PM +, Bertrand Drouvot wrote: > > Yeah, I think that this is needed for the custom wait events. But do we need > to handle the

Re: Adding wait events statistics

2025-07-07 Thread Bertrand Drouvot
Hi, On Mon, Jul 07, 2025 at 02:56:29PM +0900, Michael Paquier wrote: > On Mon, Jun 30, 2025 at 01:36:12PM +, Bertrand Drouvot wrote: > > * Do we need to serialize the stats based on their names (as for > > PGSTAT_KIND_REPLSLOT)? This question is the same as "is the ordering > > preserved >

Re: Adding wait events statistics

2025-07-06 Thread Michael Paquier
On Mon, Jun 30, 2025 at 01:36:12PM +, Bertrand Drouvot wrote: > It could also be useful to observe the engine/backends behavior over time and > help answer questions like: > > * Is the engine’s wait pattern the same over time? > * Is application’s "A" wait pattern the same over time? > * I obs