Re: relfilenode statistics

2025-03-13 Thread Kirill Reshke
On Fri, 3 Jan 2025 at 21:18, Bertrand Drouvot wrote: > As mentioned by Andres in [1], relying on the relation OID would not work to > "recover" the stats because we don't have access to the relation oid during > crash > recovery. So, I'm going to resume working on the "initial" idea (i.e having

Re: relfilenode statistics

2025-01-03 Thread Bertrand Drouvot
Hi, On Tue, Dec 03, 2024 at 10:31:15AM +, Bertrand Drouvot wrote: > Hi, > > On Fri, Nov 29, 2024 at 08:52:13PM +0500, Kirill Reshke wrote: > > On Fri, 29 Nov 2024 at 20:20, Bertrand Drouvot > > wrote: > > > On Fri, Nov 29, 2024 at 11:23:12AM +0500, Kirill Reshke wrote: > > > > If we don’t ha

Re: relfilenode statistics

2024-12-03 Thread Bertrand Drouvot
Hi, On Fri, Nov 29, 2024 at 08:52:13PM +0500, Kirill Reshke wrote: > On Fri, 29 Nov 2024 at 20:20, Bertrand Drouvot > wrote: > > On Fri, Nov 29, 2024 at 11:23:12AM +0500, Kirill Reshke wrote: > > > If we don’t have the relation OID when writing buffers out, can we > > > just store oid to bufferta

Re: relfilenode statistics

2024-11-29 Thread Kirill Reshke
On Fri, 29 Nov 2024 at 20:20, Bertrand Drouvot wrote: > On Fri, Nov 29, 2024 at 11:23:12AM +0500, Kirill Reshke wrote: > > If we don’t have the relation OID when writing buffers out, can we > > just store oid to buffertag mapping somewhere and use it? > > Do you mean add the relation OID into the

Re: relfilenode statistics

2024-11-29 Thread Bertrand Drouvot
Hi, On Fri, Nov 29, 2024 at 11:23:12AM +0500, Kirill Reshke wrote: > On Tue, 5 Nov 2024 at 11:06, Bertrand Drouvot > wrote: > > > > > > Does it sound ok to you to move with the above principal? (I'm +1 on it). > > > > Hi! I looked through this thread. Thanks for looking at it! > Looks like we

Re: relfilenode statistics

2024-11-28 Thread Kirill Reshke
On Tue, 5 Nov 2024 at 11:06, Bertrand Drouvot wrote: > > > Does it sound ok to you to move with the above principal? (I'm +1 on it). > Hi! I looked through this thread. Looks like we are still awaiting a patch which stores more counters (n_dead_tup, ... etc) into relfilenode stats. So, I assume t

Re: relfilenode statistics

2024-11-05 Thread Robert Haas
On Tue, Nov 5, 2024 at 1:06 AM Bertrand Drouvot wrote: > Does it sound ok to you to move with the above principal? (I'm +1 on it). Yes, provided we can get a clean implementation of it. -- Robert Haas EDB: http://www.enterprisedb.com

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
Hi, On Mon, Nov 04, 2024 at 02:51:10PM -0500, Robert Haas wrote: > On Mon, Nov 4, 2024 at 4:27 AM Bertrand Drouvot > wrote: > > Then I think we should go with the "sometimes I don't know the relation OID > > so I want to use the relfilenumber instead, without changing the user > > experience" >

Re: relfilenode statistics

2024-11-04 Thread Robert Haas
On Mon, Nov 4, 2024 at 4:27 AM Bertrand Drouvot wrote: > Then I think we should go with the "sometimes I don't know the relation OID > so I want to use the relfilenumber instead, without changing the user > experience" > way. So does the latest version of the patch implement that principal unifo

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
On Mon, Nov 04, 2024 at 09:27:38AM +, Bertrand Drouvot wrote: > On Tue, Sep 10, 2024 at 05:30:32AM +, Bertrand Drouvot wrote: > > Hi, > > > > On Thu, Sep 05, 2024 at 04:48:36AM +, Bertrand Drouvot wrote: > > > Please find attached a mandatory rebase. > > > > > > In passing, checking i

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
Hi, On Tue, Sep 10, 2024 at 05:30:32AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Sep 05, 2024 at 04:48:36AM +, Bertrand Drouvot wrote: > > Please find attached a mandatory rebase. > > > > In passing, checking if based on the previous discussion (and given that we > > don't have the re

Re: relfilenode statistics

2024-09-09 Thread Bertrand Drouvot
Hi, On Thu, Sep 05, 2024 at 04:48:36AM +, Bertrand Drouvot wrote: > Please find attached a mandatory rebase. > > In passing, checking if based on the previous discussion (and given that we > don't have the relation OID when writing buffers out) you see another approach > that the one this pat

Re: relfilenode statistics

2024-09-04 Thread Bertrand Drouvot
Hi, On Mon, Aug 05, 2024 at 05:28:22AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jul 11, 2024 at 06:10:23AM +, Bertrand Drouvot wrote: > > Hi, > > > > On Thu, Jul 11, 2024 at 01:58:19PM +0900, Michael Paquier wrote: > > > On Wed, Jul 10, 2024 at 01:38:06PM +, Bertrand Drouvot wrot

Re: relfilenode statistics

2024-08-04 Thread Bertrand Drouvot
Hi, On Thu, Jul 11, 2024 at 06:10:23AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jul 11, 2024 at 01:58:19PM +0900, Michael Paquier wrote: > > On Wed, Jul 10, 2024 at 01:38:06PM +, Bertrand Drouvot wrote: > > > So, I think it makes sense to link the hashkey to all the RelFileLocator > >

Re: relfilenode statistics

2024-07-10 Thread Bertrand Drouvot
Hi, On Thu, Jul 11, 2024 at 01:58:19PM +0900, Michael Paquier wrote: > On Wed, Jul 10, 2024 at 01:38:06PM +, Bertrand Drouvot wrote: > > So, I think it makes sense to link the hashkey to all the RelFileLocator > > fields, means: > > > > dboid (linked to RelFileLocator's dbOid) > > objoid (lin

Re: relfilenode statistics

2024-07-10 Thread Michael Paquier
On Wed, Jul 10, 2024 at 01:38:06PM +, Bertrand Drouvot wrote: > On Wed, Jul 10, 2024 at 03:02:34PM +0900, Michael Paquier wrote: >> and I am troubled by the approach taken (mentioned down by you), but >> that's invasive compared to how pgstats wants to be transparent with >> its stats kinds. >>

Re: relfilenode statistics

2024-07-10 Thread Bertrand Drouvot
Hi, On Wed, Jul 10, 2024 at 03:02:34PM +0900, Michael Paquier wrote: > On Sat, May 25, 2024 at 07:52:02AM +, Bertrand Drouvot wrote: > > But I think that it is in a state that can be used to discuss the approach > > it > > is implementing (so that we can agree or not on it) before moving > >

Re: relfilenode statistics

2024-07-09 Thread Michael Paquier
On Sat, May 25, 2024 at 07:52:02AM +, Bertrand Drouvot wrote: > But I think that it is in a state that can be used to discuss the approach it > is implementing (so that we can agree or not on it) before moving > forward. I have read through the patch to get an idea of how things are done, and

Re: relfilenode statistics

2024-06-12 Thread Bertrand Drouvot
Hi, On Tue, Jun 11, 2024 at 03:35:23PM +0900, Kyotaro Horiguchi wrote: > At Mon, 10 Jun 2024 08:09:56 +, Bertrand Drouvot > wrote in > > > > My idea was to move all that is in pg_statio_all_tables to relfilenode stats > > and 1) add new stats to pg_statio_all_tables (like heap_blks_written

Re: relfilenode statistics

2024-06-10 Thread Kyotaro Horiguchi
At Mon, 10 Jun 2024 08:09:56 +, Bertrand Drouvot wrote in > Hi, > > On Fri, Jun 07, 2024 at 09:24:41AM -0400, Robert Haas wrote: > > On Thu, Jun 6, 2024 at 11:17 PM Andres Freund wrote: > > > If we just want to keep prior stats upon arelation rewrite, we can just > > > copy > > > the stat

Re: relfilenode statistics

2024-06-10 Thread Bertrand Drouvot
Hi, On Fri, Jun 07, 2024 at 09:24:41AM -0400, Robert Haas wrote: > On Thu, Jun 6, 2024 at 11:17 PM Andres Freund wrote: > > If we just want to keep prior stats upon arelation rewrite, we can just copy > > the stats from the old relfilenode. Or we can decide that those stats don't > > really make

Re: relfilenode statistics

2024-06-07 Thread Robert Haas
On Thu, Jun 6, 2024 at 11:17 PM Andres Freund wrote: > If we just want to keep prior stats upon arelation rewrite, we can just copy > the stats from the old relfilenode. Or we can decide that those stats don't > really make sense anymore, and start from scratch. I think we need to think carefull

Re: relfilenode statistics

2024-06-07 Thread Bertrand Drouvot
Hi, On Thu, Jun 06, 2024 at 08:17:36PM -0700, Andres Freund wrote: > Hi, > > On 2024-06-06 12:27:49 -0400, Robert Haas wrote: > > On Wed, Jun 5, 2024 at 1:52 AM Bertrand Drouvot > > wrote: > > > I think we should keep the stats in the relation during relfilenode > > > changes. > > > As a POC, v

Re: relfilenode statistics

2024-06-07 Thread Bertrand Drouvot
Hi, On Thu, Jun 06, 2024 at 08:38:06PM -0700, Andres Freund wrote: > Hi, > > On 2024-06-03 11:11:46 +, Bertrand Drouvot wrote: > > The main argument is that we currently don’t have writes counters for > > relations. > > The reason is that we don’t have the relation OID when writing buffers o

Re: relfilenode statistics

2024-06-06 Thread Andres Freund
Hi, On 2024-06-03 11:11:46 +, Bertrand Drouvot wrote: > The main argument is that we currently don’t have writes counters for > relations. > The reason is that we don’t have the relation OID when writing buffers out. > Tracking writes per relfilenode would allow us to track/consolidate writes

Re: relfilenode statistics

2024-06-06 Thread Andres Freund
Hi, On 2024-06-06 12:27:49 -0400, Robert Haas wrote: > On Wed, Jun 5, 2024 at 1:52 AM Bertrand Drouvot > wrote: > > I think we should keep the stats in the relation during relfilenode changes. > > As a POC, v1 implemented a way to do so during TRUNCATE (see the changes in > > table_relation_set_n

Re: relfilenode statistics

2024-06-06 Thread Robert Haas
On Wed, Jun 5, 2024 at 1:52 AM Bertrand Drouvot wrote: > I think we should keep the stats in the relation during relfilenode changes. > As a POC, v1 implemented a way to do so during TRUNCATE (see the changes in > table_relation_set_new_filelocator() and in pg_statio_all_tables): as you can > see

Re: relfilenode statistics

2024-06-06 Thread Bertrand Drouvot
Hi, On Mon, Jun 03, 2024 at 11:11:46AM +, Bertrand Drouvot wrote: > Yeah, I’ll update the commit message in V2 with better explanations once I get > feedback on V1 (should we decide to move on with the relfilenode stats idea). > Please find attached v2, mandatory rebase due to cd312adc56. In

Re: relfilenode statistics

2024-06-04 Thread Bertrand Drouvot
On Tue, Jun 04, 2024 at 09:26:27AM -0400, Robert Haas wrote: > On Mon, Jun 3, 2024 at 7:11 AM Bertrand Drouvot > wrote: > > We’d move the current buffer read and buffer hit counters from the relation > > stats > > to the relfilenode stats (while still being able to retrieve them from the > > pg_s

Re: relfilenode statistics

2024-06-04 Thread Robert Haas
On Mon, Jun 3, 2024 at 7:11 AM Bertrand Drouvot wrote: > The main argument is that we currently don’t have writes counters for > relations. > The reason is that we don’t have the relation OID when writing buffers out. OK. > Second argument is that this is also beneficial for the "Split index an

Re: relfilenode statistics

2024-06-03 Thread Bertrand Drouvot
Hi Robert, On Mon, May 27, 2024 at 09:10:13AM -0400, Robert Haas wrote: > Hi Bertrand, > > It would be helpful to me if the reasons why we're splitting out > relfilenodestats could be more clearly spelled out. I see Andres's > comment in the thread to which you linked, but it's pretty vague about

Re: relfilenode statistics

2024-05-27 Thread Robert Haas
Hi Bertrand, It would be helpful to me if the reasons why we're splitting out relfilenodestats could be more clearly spelled out. I see Andres's comment in the thread to which you linked, but it's pretty vague about why we should do this ("it's not nice") and whether we should do this ("I wonder i