Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2022-01-14 Thread Peter Geoghegan
On Thu, Jan 6, 2022 at 10:21 AM Peter Geoghegan wrote: > The patch bitrot again, so attached is a rebased version, v3. I pushed a version of this patch earlier. This final version didn't go quite as far as v3 did: it retained a few VACUUM VERBOSE only INFO messages (it didn't demote them to

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2022-01-06 Thread Peter Geoghegan
On Tue, Dec 21, 2021 at 11:57 PM Masahiko Sawada wrote: > I've looked at the patch and here are comments: Thanks! The patch bitrot again, so attached is a rebased version, v3. > I think we should set message_level. Otherwise, index AM will set an > invalid log level, although any index AM in

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-22 Thread Peter Geoghegan
On Tue, Dec 21, 2021 at 9:46 PM Greg Stark wrote: > Or rather I think a better way to look at it is that the progress > output for the operator should be separated from the metrics logged. > As an operator what I want to see is some progress indicator > ""starting table scan", "overflow at x% of

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-21 Thread Masahiko Sawada
On Tue, Dec 21, 2021 at 2:39 AM Peter Geoghegan wrote: > > On Mon, Nov 29, 2021 at 6:51 PM Peter Geoghegan wrote: > > Attached is a WIP patch doing this. > > This has bitrot, so I attach v2, mostly just to keep the CFTester > status green. The only real change is one minor simplification to how

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-21 Thread Greg Stark
I haven't read the patch yet. But some thoughts based on the posted output: 1) At first I was quite skeptical about losing the progress reporting. I've often found it quite useful. But looking at the examples I'm convinced. Or rather I think a better way to look at it is that the progress output

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-20 Thread Peter Geoghegan
On Mon, Nov 29, 2021 at 6:51 PM Peter Geoghegan wrote: > Attached is a WIP patch doing this. This has bitrot, so I attach v2, mostly just to keep the CFTester status green. The only real change is one minor simplification to how we set everything up, inside heap_vacuum_rel(). -- Peter

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Peter Geoghegan
On Sat, Dec 11, 2021 at 2:52 PM Andres Freund wrote: > I feel one, or both, must be missing something here. My point was that you > said upthread that the patch doesn't change DEBUG2/non-verbose logging for > most messages. But the fact that those messages are only emitted inside and if >

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Andres Freund
Hi, On 2021-12-11 13:13:56 -0800, Peter Geoghegan wrote: > On Sat, Dec 11, 2021 at 12:24 PM Andres Freund wrote: > > But the ereport is inside an if (verbose), no? > > Yes -- in order to report aggressiveness in VACUUM VERBOSE. But the > autovacuum case still reports verbose-ness, in the same

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Peter Geoghegan
On Sat, Dec 11, 2021 at 1:13 PM Peter Geoghegan wrote: > Yes -- in order to report aggressiveness in VACUUM VERBOSE. But the > autovacuum case still reports verbose-ness, in the same way as it > always has -- in that same LOG entry. We don't want to repeat > ourselves in the VERBOSE case, which

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Peter Geoghegan
On Sat, Dec 11, 2021 at 12:24 PM Andres Freund wrote: > But the ereport is inside an if (verbose), no? Yes -- in order to report aggressiveness in VACUUM VERBOSE. But the autovacuum case still reports verbose-ness, in the same way as it always has -- in that same LOG entry. We don't want to

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Andres Freund
On 2021-12-11 09:52:29 -0800, Peter Geoghegan wrote: > On Fri, Dec 10, 2021 at 8:30 PM Andres Freund wrote: > > I think some actually ended up being omitted compared to the previous > > state. E.g. "aggressively vacuuming ...", but I think others as well. > > The "aggressive-ness" is reported by

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-11 Thread Peter Geoghegan
On Fri, Dec 10, 2021 at 8:30 PM Andres Freund wrote: > I think some actually ended up being omitted compared to the previous > state. E.g. "aggressively vacuuming ...", but I think others as well. The "aggressive-ness" is reported by a distinct ereport() with the patch, so you'll still see that

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-12-10 Thread Andres Freund
Hi, On 2021-11-29 18:51:37 -0800, Peter Geoghegan wrote: > One thing that's still unclear is what the new elevel should be for > the ereport messages that used to be either LOG (for VACUUM VERBOSE) > or DEBUG2 (for everything else) -- what should I change them to now? > For now I've done taken

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-29 Thread Peter Geoghegan
On Mon, Nov 29, 2021 at 8:19 PM Justin Pryzby wrote: > I think the 2nd chunk here could say "if (instrument)" like the first: I agree that that would be clearer. > Autovacuum's format doesn't show the number of scanned pages ; it shows how > many pages were skipped due to frozen bit, but not

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-29 Thread Justin Pryzby
I think the 2nd chunk here could say "if (instrument)" like the first: > @@ -482,8 +480,10 @@ heap_vacuum_rel(Relation rel, VacuumParams *params, > TransactionId FreezeLimit; > MultiXactId MultiXactCutoff; > > - /* measure elapsed time iff autovacuum logging requires it */

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-29 Thread Peter Geoghegan
On Fri, Nov 26, 2021 at 12:37 PM Peter Geoghegan wrote: > My preferred approach to this is simple: redefine VACUUM VERBOSE to > not show incremental output, which seems rather unhelpful anyway. This > does mean that VACUUM VERBOSE won't show certain information that > might occasionally be useful

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-26 Thread Peter Geoghegan
On Fri, Nov 26, 2021 at 1:57 PM Justin Pryzby wrote: > > * VACUUM VERBOSE doesn't provide much of the most useful > > instrumentation that we have available in log_autovacuum_min_duration, > > and yet produces output that is ludicrously, unmanageably verbose -- > > lots of pg_rusage_show()

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-26 Thread Peter Geoghegan
On Fri, Nov 26, 2021 at 12:37 PM Peter Geoghegan wrote: > Unifying everything cannot be approached mechanically, so doing this > requires real buy-in. It's a bit tricky because VACUUM VERBOSE is > supposed to show real time information about what just finished, as a > kind of rudimentary progress

Re: Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-26 Thread Justin Pryzby
On Fri, Nov 26, 2021 at 12:37:32PM -0800, Peter Geoghegan wrote: > My preferred approach to this is simple: redefine VACUUM VERBOSE to > not show incremental output, which seems rather unhelpful anyway. > I don't think that we need to keep the getrusage() stuff at all, though. +1 > * VACUUM

Unifying VACUUM VERBOSE and log_autovacuum_min_duration output

2021-11-26 Thread Peter Geoghegan
I think that it's worth unifying VACUUM VERBOSE and log_autovacuum_min_duration output, to remove the redundancy, and to provide more useful VACUUM VERBOSE output. Both variants already output approximately the same things. But, each variant reports on certain details that the other variant lacks