Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-12 Thread Bossart, Nathan
On 1/11/22, 11:46 PM, "Masahiko Sawada" wrote: > Regarding the new pg_stat_progress_vacuum_index view, why do we need > to have a separate view? Users will have to check two views. If this > view is expected to be used together with and joined to > pg_stat_progress_vacuum, why don't we provide one

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-11 Thread Masahiko Sawada
(We had better avoid top-posting[1]) On Tue, Jan 11, 2022 at 10:01 AM Imseih (AWS), Sami wrote: > > I agree, Renaming "index_vacuum_count" can be taken up in a separate > discussion. > > I have attached the 3rd revision of the patch which also includes the > documentation changes. Also attache

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-11 Thread Bossart, Nathan
On 1/11/22, 12:33 PM, "Imseih (AWS), Sami" wrote: > What about something like "The number of indexes that are eligible for > vacuuming". > This covers the cases where either an individual index is skipped or the > entire "index vacuuming" phase is skipped. Hm. I don't know if "eligible" is th

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-11 Thread Imseih (AWS), Sami
On 1/11/22, 1:01 PM, "Bossart, Nathan" wrote: On 1/10/22, 5:01 PM, "Imseih (AWS), Sami" wrote: > I have attached the 3rd revision of the patch which also includes the documentation changes. Also attached is a rendered html of the docs for review. > > "max_index_vacuum_cycle_time

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-11 Thread Bossart, Nathan
On 1/10/22, 5:01 PM, "Imseih (AWS), Sami" wrote: > I have attached the 3rd revision of the patch which also includes the > documentation changes. Also attached is a rendered html of the docs for > review. > > "max_index_vacuum_cycle_time" has been removed. > "index_rows_vacuumed" renamed to "ind

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-10 Thread Imseih (AWS), Sami
I agree, Renaming "index_vacuum_count" can be taken up in a separate discussion. I have attached the 3rd revision of the patch which also includes the documentation changes. Also attached is a rendered html of the docs for review. "max_index_vacuum_cycle_time" has been removed. "index_rows_vacuu

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-10 Thread Bossart, Nathan
On 1/6/22, 6:14 PM, "Imseih (AWS), Sami" wrote: > I am hesitant to make column name changes for obvious reasons, as it breaks > existing tooling. However, I think there is a really good case to change > "index_vacuum_count" as the name is confusing. > "index_vacuum_cycles_completed" is the name

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Imseih (AWS), Sami
Thanks for the review. I am hesitant to make column name changes for obvious reasons, as it breaks existing tooling. However, I think there is a really good case to change "index_vacuum_count" as the name is confusing. "index_vacuum_cycles_completed" is the name I suggest if we agree to rename.

Re: Add index scan progress to pg_stat_progress_vacuum

2022-01-06 Thread Bossart, Nathan
On 12/29/21, 8:44 AM, "Imseih (AWS), Sami" wrote: > In "pg_stat_progress_vacuum", introduce 2 columns: > > * total_index_vacuum : This is the # of indexes that will be vacuumed. Keep > in mind that if failsafe mode kicks in mid-flight to the vacuum, Postgres may > choose to forgo index scans. Th

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-29 Thread Justin Pryzby
http://cfbot.cputube.org/sami-imseih.html You should run "make check" and update rules.out. You should also use make check-world - usually something like: make check-world -j4 >check-world.out 2>&1 ; echo ret $? > indrelid: The relid of the index currently being vacuumed I think it should be cal

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-27 Thread Imseih (AWS), Sami
I do agree that tracking progress by # of blocks scanned is not deterministic for all index types. Based on this feedback, I went back to the drawing board on this. Something like below may make more sense. In pg_stat_progress_vacuum, introduce 2 new columns: 1. total_index_vacuum - total #

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-27 Thread Justin Pryzby
: "Imseih (AWS), Sami" Date: Mon, 20 Dec 2021 17:55:03 +0000 Subject: [PATCH] Add index scan progress to pg_stat_progress_vacuum Here is a V2 attempt of the patch to include a new view called pg_stat_progress_vacuum_worker. Also, scans for index cleanups will also have an entry in th

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-23 Thread Andrey Lepikhov
On 21/12/2021 00:05, Peter Geoghegan wrote: * Some index AMs don't work like nbtree and GiST in that they cannot do their scan sequentially -- they have to do something like a logical/keyspace order scan instead, which is *totally* different to heapam (not just a bit different). There is no telli

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-23 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 3:37 AM Peter Geoghegan wrote: > > On Wed, Dec 15, 2021 at 2:10 PM Bossart, Nathan wrote: > > nitpick: Shouldn't index_blks_scanned be index_blks_vacuumed? IMO it > > is more analogous to heap_blks_vacuumed. > > +1. > > > This will tell us which indexes are currently bein

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Justin Pryzby
This view also doesn't show vacuum progress across a partitioned table. For comparison: pg_stat_progress_create_index (added in v12) has: partitions_total partitions_done pg_stat_progress_analyze (added in v13) has: child_tables_total child_tables_done pg_stat_progress_cluster should have somet

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
On Wed, Dec 1, 2021 at 2:59 PM Imseih (AWS), Sami wrote: > The current implementation of pg_stat_progress_vacuum does not provide > progress on which index is being vacuumed making it difficult for a user to > determine if the "vacuuming indexes" phase is making progress. I notice that your pat

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-20 Thread Peter Geoghegan
On Wed, Dec 15, 2021 at 2:10 PM Bossart, Nathan wrote: > nitpick: Shouldn't index_blks_scanned be index_blks_vacuumed? IMO it > is more analogous to heap_blks_vacuumed. +1. > This will tell us which indexes are currently being vacuumed and the > current progress of those operations, but it does

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-16 Thread Greg Stark
I had a similar question. And I'm still not clear from the response what exactly index_blks_total is and whether it addresses it. I think I agree that a user is likely to want to see the progress in a way they can understand which means for a single index at a time. I think what you're describing

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-16 Thread Imseih (AWS), Sami
On 12/15/21, 4:10 PM, "Bossart, Nathan" wrote: On 12/1/21, 3:02 PM, "Imseih (AWS), Sami" wrote: > The current implementation of pg_stat_progress_vacuum does not > provide progress on which index is being vacuumed making it > difficult for a user to determine if the "vacuuming

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-15 Thread Bossart, Nathan
On 12/1/21, 3:02 PM, "Imseih (AWS), Sami" wrote: > The current implementation of pg_stat_progress_vacuum does not > provide progress on which index is being vacuumed making it > difficult for a user to determine if the "vacuuming indexes" phase > is making progress. By exposing which index is bein

<    1   2