Re: Track IO times in pg_stat_io

2023-04-07 Thread Andres Freund
Hi, On 2023-04-07 12:17:38 -0400, Melanie Plageman wrote: > Attached v9 addresses review feedback as well as resolving merge > conflicts with recent relation extension patchset. I've edited it a bit more: - removed pgstat_tracks_io_time() and replaced it by returning the new IO_COL_INVALID =

Re: Track IO times in pg_stat_io

2023-04-07 Thread Melanie Plageman
6 1 file changed, 16 insertions(+) diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 3846d3eaca..3639296bc1 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -176,6 +176,8 @@ GetLocalVictim

Re: Track IO times in pg_stat_io

2023-04-04 Thread Andres Freund
sage.local_blks_written++; > + > /* Pop the error context stack */ > error_context_stack = errcallback.previous; > } > -- > 2.37.2 > > From 2bdad725133395ded199ecc726096e052d6e654b Mon

Re: Track IO times in pg_stat_io

2023-03-31 Thread Melanie Plageman
e += (n)) #define pgstat_count_conn_txn_idle_time(n) \ -- 2.37.2 From 2bdad725133395ded199ecc726096e052d6e654b Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Fri, 31 Mar 2023 15:32:36 -0400 Subject: [PATCH v8 3/4] Track IO times in pg_stat_io Add IO timing for reads, writes, extends, and fsyncs to pg_

Re: Track IO times in pg_stat_io

2023-03-21 Thread Melanie Plageman
checksum_failure(void); extern void pgstat_report_connect(Oid dboid); -#define pgstat_count_buffer_read_time(n) \ - (pgStatBlockReadTime += (n)) -#define pgstat_count_buffer_write_time(n) \ - (pgStatBlockWriteTime += (n)) #define pgstat_count_conn_active_time(n) \ (pgStatActiveTime += (n)) #def

Re: Track IO times in pg_stat_io

2023-03-20 Thread Andres Freund
───┼───┼─┼┼─┼─┤ │ 1318539 │ 0 │ 1318539 │ 5013 │ 1523339 │7873 │ └─┴───┴─┴┴─┴─┘ (1 row) Greetings, Andres >From 5d4aa3f6c651006f1ec960f59e24ebc8b5a8ca25 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 6 Mar 2

Re: Track IO times in pg_stat_io

2023-03-16 Thread Melanie Plageman
store in microsec and then in pg_stat_io, I multiply to get milliseconds for display. I considered refactoring pgstat_io_end() to use INSTR_TIME_ACCUM_DIFF() like [1], but, in the end I actually think I would end up with more operations because of the various different counters needing to be updat

Re: Track IO times in pg_stat_io

2023-03-14 Thread Andres Freund
Hi, On 2023-03-09 11:50:38 -0500, Melanie Plageman wrote: > On Tue, Mar 7, 2023 at 1:39 PM Andres Freund wrote: > > On 2023-03-06 11:30:13 -0500, Melanie Plageman wrote: > > > > As pgstat_bktype_io_stats_valid() is called only in Assert(), I think > > > > that would be a good idea > > > > to

Re: Track IO times in pg_stat_io

2023-03-09 Thread Melanie Plageman
TYPES][IOOP_NUM_TYPES]; > > + instr_time > > times[IOOBJECT_NUM_TYPES][IOCONTEXT_NUM_TYPES][IOOP_NUM_TYPES]; > > } PgStat_BktypeIO; > > Ah, you're going to hate me. We can't store instr_time on disk. There's > another patch that gets substantial peformance gains by

Re: Track IO times in pg_stat_io

2023-03-09 Thread Imseih (AWS), Sami
> >>> Now I've a second thought: what do you think about resetting the related > >>> number > >>> of operations and *_time fields when enabling/disabling track_io_timing? > >>> (And mention it in the doc). > >>> > >>> That way it'd prevent bad interpretation (at least as far the time per > >>>

Re: Track IO times in pg_stat_io

2023-03-09 Thread Drouvot, Bertrand
Hi, On 3/9/23 1:34 AM, Andres Freund wrote: Hi, On 2023-03-08 12:55:34 +0100, Drouvot, Bertrand wrote: On 3/7/23 7:47 PM, Andres Freund wrote: On 2023-03-07 13:43:28 -0500, Melanie Plageman wrote: No, I don't think we can do that. It can be enabled on a per-session basis. Oh right. So it's

Re: Track IO times in pg_stat_io

2023-03-08 Thread Andres Freund
Hi, On 2023-03-08 12:55:34 +0100, Drouvot, Bertrand wrote: > On 3/7/23 7:47 PM, Andres Freund wrote: > > On 2023-03-07 13:43:28 -0500, Melanie Plageman wrote: > > > > Now I've a second thought: what do you think about resetting the > > > > related number > > > > of operations and *_time fields

Re: Track IO times in pg_stat_io

2023-03-08 Thread Drouvot, Bertrand
Hi, On 3/7/23 7:47 PM, Andres Freund wrote: On 2023-03-07 13:43:28 -0500, Melanie Plageman wrote: Now I've a second thought: what do you think about resetting the related number of operations and *_time fields when enabling/disabling track_io_timing? (And mention it in the doc). That way

Re: Track IO times in pg_stat_io

2023-03-07 Thread Andres Freund
On 2023-03-07 13:43:28 -0500, Melanie Plageman wrote: > > Now I've a second thought: what do you think about resetting the related > > number > > of operations and *_time fields when enabling/disabling track_io_timing? > > (And mention it in the doc). > > > > That way it'd prevent bad

Re: Track IO times in pg_stat_io

2023-03-07 Thread Melanie Plageman
Thanks for taking another look! On Tue, Mar 7, 2023 at 10:52 AM Drouvot, Bertrand wrote: > On 3/6/23 5:30 PM, Melanie Plageman wrote: > > Thanks for the review! > > > > On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand > > wrote: > >> On 2/26/23 5:03 PM, Melanie Plageman wrote: > >>> The

Re: Track IO times in pg_stat_io

2023-03-07 Thread Andres Freund
Hi, On 2023-03-06 11:30:13 -0500, Melanie Plageman wrote: > > As pgstat_bktype_io_stats_valid() is called only in Assert(), I think that > > would be a good idea > > to also check that if counts are not Zero then times are not Zero. > > Yes, I think adding some validation around the

Re: Track IO times in pg_stat_io

2023-03-07 Thread Drouvot, Bertrand
Hi, On 3/6/23 5:30 PM, Melanie Plageman wrote: Thanks for the review! On Tue, Feb 28, 2023 at 4:49 AM Drouvot, Bertrand wrote: On 2/26/23 5:03 PM, Melanie Plageman wrote: The timings will only be non-zero when track_io_timing is on That could lead to incorrect interpretation if one wants

Re: Track IO times in pg_stat_io

2023-03-06 Thread Melanie Plageman
imes are not zero, IO counts are not zero. I've done this in the attached v3. - Melanie From 52d997001108a52c833b339f9b8dcb3d34ed3270 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 6 Mar 2023 10:41:51 -0500 Subject: [PATCH v3] Track IO times in pg_stat_io Add IO timing for reads, wr

Re: Track IO times in pg_stat_io

2023-02-28 Thread Drouvot, Bertrand
Hi, On 2/26/23 5:03 PM, Melanie Plageman wrote: Hi, As suggested in [1], the attached patch adds IO times to pg_stat_io; Thanks for the patch! I started to have a look at it and figured out that a tiny rebase was needed (due to 728560db7d and b9f0e54bc9), so please find the rebase (aka V2)

Track IO times in pg_stat_io

2023-02-26 Thread Melanie Plageman
Plageman Date: Sat, 25 Feb 2023 18:09:10 -0500 Subject: [PATCH v1] Track IO times in pg_stat_io Add IO timing for reads, writes, extends, and fsyncs to pg_stat_io. --- doc/src/sgml/monitoring.sgml | 48 +++ src/backend/catalog/system_views.sql | 4 ++ src/backend/storage