Re: Fix parallel vacuum buffer usage reporting

2024-05-13 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 19:09, Masahiko Sawada wrote: > > On Fri, May 10, 2024 at 7:26 PM Nazir Bilal Yavuz wrote: > > > > Hi, > > > > Thank you for working on this! > > > > On Wed, 1 May 2024 at 06:37, Masahiko Sawada wrote: > > > > > > Thank you for further testing! I've pushed the patch.

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Masahiko Sawada
On Fri, May 10, 2024 at 7:26 PM Nazir Bilal Yavuz wrote: > > Hi, > > Thank you for working on this! > > On Wed, 1 May 2024 at 06:37, Masahiko Sawada wrote: > > > > Thank you for further testing! I've pushed the patch. > > I realized a behaviour change while looking at 'Use pgBufferUsage for >

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:55, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Fri, 10 May 2024 at 14:49, Alena Rybakina > > wrote: > > > > > > Hi! I could try to check it with the test, but I want to ask you about > > >

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 16:21, Nazir Bilal Yavuz wrote: > > Hi, > > On Fri, 10 May 2024 at 14:49, Alena Rybakina wrote: > > > > Hi! I could try to check it with the test, but I want to ask you about > > details, because I'm not sure that I completely understand the test case. > > > > You

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 May 2024 at 14:49, Alena Rybakina wrote: > > Hi! I could try to check it with the test, but I want to ask you about > details, because I'm not sure that I completely understand the test case. > > You mean that we need to have two backends and on one of them we deleted > the tuples

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Alena Rybakina
Hi! I could try to check it with the test, but I want to ask you about details, because I'm not sure that I completely understand the test case. You mean that we need to have two backends and on one of them we deleted the tuples before vacuum called the other, do you? On 10.05.2024 13:25,

Re: Fix parallel vacuum buffer usage reporting

2024-05-10 Thread Nazir Bilal Yavuz
Hi, Thank you for working on this! On Wed, 1 May 2024 at 06:37, Masahiko Sawada wrote: > > Thank you for further testing! I've pushed the patch. I realized a behaviour change while looking at 'Use pgBufferUsage for block reporting in analyze' thread [1]. Since that change applies here as well,

Re: Fix parallel vacuum buffer usage reporting

2024-05-08 Thread Masahiko Sawada
On Fri, May 3, 2024 at 3:41 PM Anthonin Bonnefoy wrote: > > On Wed, May 1, 2024 at 5:37 AM Masahiko Sawada wrote: >> >> Thank you for further testing! I've pushed the patch. > > Thanks! > > Here is the rebased version for the follow-up patch removing VacuumPage > variables. Though I'm not sure

Re: Fix parallel vacuum buffer usage reporting

2024-05-03 Thread Anthonin Bonnefoy
On Wed, May 1, 2024 at 5:37 AM Masahiko Sawada wrote: > Thank you for further testing! I've pushed the patch. Thanks! Here is the rebased version for the follow-up patch removing VacuumPage variables. Though I'm not sure if I should create a dedicated mail thread since the bug was fixed and

Re: Fix parallel vacuum buffer usage reporting

2024-04-30 Thread Alena Rybakina
Hi! On 30.04.2024 05:18, Masahiko Sawada wrote: On Fri, Apr 26, 2024 at 9:12 PM Alena Rybakina wrote: Hi! The same script was run, but using vacuum verbose analyze, and I saw the difference again in the fifth step: with your patch: buffer usage: 32312 hits, 607 misses, 1566 dirtied master:

Re: Fix parallel vacuum buffer usage reporting

2024-04-30 Thread Masahiko Sawada
On Tue, Apr 30, 2024 at 3:34 PM Anthonin Bonnefoy wrote: > > I've done some additional tests to validate the reported numbers. Using > pg_statio, it's possible to get the minimum number of block hits (Full script > attached). > > -- Save block hits before vacuum > SELECT

Re: Fix parallel vacuum buffer usage reporting

2024-04-30 Thread Anthonin Bonnefoy
I've done some additional tests to validate the reported numbers. Using pg_statio, it's possible to get the minimum number of block hits (Full script attached). -- Save block hits before vacuum SELECT pg_stat_force_next_flush(); SELECT heap_blks_hit, idx_blks_hit FROM pg_statio_all_tables where

Re: Fix parallel vacuum buffer usage reporting

2024-04-29 Thread Masahiko Sawada
On Fri, Apr 26, 2024 at 9:12 PM Alena Rybakina wrote: > > Hi! >> >> The same script was run, but using vacuum verbose analyze, and I saw the >> difference again in the fifth step: >> with your patch: buffer usage: 32312 hits, 607 misses, 1566 dirtied >> master: buffer usage: 32346 hits, 573

Re: Fix parallel vacuum buffer usage reporting

2024-04-26 Thread Alena Rybakina
Hi! The same script was run, but using vacuum verbose analyze, and I saw the difference again in the fifth step: with your patch: buffer usage: 32312 hits, 607 misses, 1566 dirtied master: buffer usage: 32346 hits, 573 misses, 1360 dirtied Isn't there a chance for the

Re: Fix parallel vacuum buffer usage reporting

2024-04-25 Thread Anthonin Bonnefoy
On Wed, Apr 24, 2024 at 4:01 PM Alena Rybakina wrote: > I tested the main postgres branch with and without your fix using a script > that was written by me. It consists of five scenarios and I made a > comparison in the logs between the original version of the master branch > and the master

Re: Fix parallel vacuum buffer usage reporting

2024-04-24 Thread Alena Rybakina
On 22.04.2024 11:07, Anthonin Bonnefoy wrote: On Sat, Apr 20, 2024 at 2:00 PM Alena Rybakina wrote: Hi, thank you for your work with this subject. While I was reviewing your code, I noticed that your patch conflicts with another patch [0] that been committed. [0]

Re: Fix parallel vacuum buffer usage reporting

2024-04-24 Thread Anthonin Bonnefoy
Thanks for the review! > I think that if the anayze command doesn't have the same issue, we > don't need to change it. Good point, I've wrongly assumed that analyze was also impacted but there's no parallel analyze so the block count is correct. > (a) make lazy vacuum use BufferUsage instead

Re: Fix parallel vacuum buffer usage reporting

2024-04-24 Thread Masahiko Sawada
On Mon, Apr 22, 2024 at 5:07 PM Anthonin Bonnefoy wrote: > > On Sat, Apr 20, 2024 at 2:00 PM Alena Rybakina > wrote: >> >> Hi, thank you for your work with this subject. >> >> While I was reviewing your code, I noticed that your patch conflicts with >> another patch [0] that been committed. >>

Re: Fix parallel vacuum buffer usage reporting

2024-04-22 Thread Anthonin Bonnefoy
On Sat, Apr 20, 2024 at 2:00 PM Alena Rybakina wrote: > Hi, thank you for your work with this subject. > > While I was reviewing your code, I noticed that your patch conflicts with > another patch [0] that been committed. > > [0] >

Re: Fix parallel vacuum buffer usage reporting

2024-04-20 Thread Alena Rybakina
Hi, thank you for your work with this subject. While I was reviewing your code, I noticed that your patch conflicts with another patch [0] that been committed. [0] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com On

Re: Fix parallel vacuum buffer usage reporting

2024-03-28 Thread Anthonin Bonnefoy
Hi, Thanks for the review. On Thu, Mar 28, 2024 at 4:07 AM Masahiko Sawada wrote: > As for the proposed patch, the following part should handle the temp > tables too: > True, I've missed the local blocks. I've updated the patch: - read_rate and write_rate now include local block usage - I've

Re: Fix parallel vacuum buffer usage reporting

2024-03-27 Thread Masahiko Sawada
Hi, Thank you for the report. On Fri, Feb 9, 2024 at 6:10 PM Anthonin Bonnefoy wrote: > > Hi, > > With a db setup with pgbench, we add an additional index: > CREATE INDEX ON pgbench_accounts(abalance) > > And trigger several updates and vacuum to reach a stable amount of > dirtied pages: >

Fix parallel vacuum buffer usage reporting

2024-02-09 Thread Anthonin Bonnefoy
Hi, With a db setup with pgbench, we add an additional index: CREATE INDEX ON pgbench_accounts(abalance) And trigger several updates and vacuum to reach a stable amount of dirtied pages: UPDATE pgbench_accounts set abalance = abalance + 1 WHERE aid=1; CHECKPOINT; VACUUM (VERBOSE, INDEX_CLEANUP