Re: Higher level questions around shared memory stats

2022-04-04 Thread Andres Freund
Hi, On 2022-03-30 14:08:41 -0700, Andres Freund wrote: > On 2022-03-30 14:42:23 -0400, Robert Haas wrote: > > On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > > > I can think of these different times: > > > > > > - Last time stats were removed due to starting up in crash recovery > > > -

Re: Higher level questions around shared memory stats

2022-04-03 Thread Alvaro Herrera
On 2022-Apr-02, Andres Freund wrote: > 6) Should any part of the "reuse_stats" logic in table_recheck_autovac() be > kept? > > With the shared memory stats patch, autovacuum can cheaply access individual > stats, so the whole scheme for avoiding stats accesses is moot. Agreed, I don't think

Re: Higher level questions around shared memory stats

2022-04-02 Thread Andres Freund
Hi, Alvaro, added you because you were the original author for a lot of that code. Fujii, you touched it last... 6) Should any part of the "reuse_stats" logic in table_recheck_autovac() be kept? With the shared memory stats patch, autovacuum can cheaply access individual stats, so the whole

Re: Higher level questions around shared memory stats

2022-04-01 Thread Andres Freund
Hi, On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > Separate from the minutia in [1] I'd like to discuss a few questions of more > general interest. I'll post another question or two later. 5) What does track_counts = off mean? I just was trying to go through the shared memory stats patch

Re: Higher level questions around shared memory stats

2022-03-31 Thread Kyotaro Horiguchi
At Thu, 31 Mar 2022 22:12:25 -0700, Andres Freund wrote in > Hi, > > On 2022-03-31 20:06:07 -0700, David G. Johnston wrote: > > Do we really think no one has taken our advice in the documentation and > > moved their stats_temp_directory to a RAM-based file system? > > I'm pretty sure some

Re: Higher level questions around shared memory stats

2022-03-31 Thread Andres Freund
Hi, On 2022-03-31 20:06:07 -0700, David G. Johnston wrote: > Do we really think no one has taken our advice in the documentation and > moved their stats_temp_directory to a RAM-based file system? I'm pretty sure some have, I've seen it in the field in the past. > The question is whether

Re: Higher level questions around shared memory stats

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 7:33 PM Kyotaro Horiguchi wrote: > At Thu, 31 Mar 2022 14:04:16 -0700, Andres Freund > wrote in > > Hi, > > > > On 2022-03-31 16:16:31 +0900, Kyotaro Horiguchi wrote: > > > After moving to shared stats, we might want to expose the GUC variable > > > itself. Then

Re: Higher level questions around shared memory stats

2022-03-31 Thread Kyotaro Horiguchi
At Thu, 31 Mar 2022 14:04:16 -0700, Andres Freund wrote in > Hi, > > On 2022-03-31 16:16:31 +0900, Kyotaro Horiguchi wrote: > > After moving to shared stats, we might want to expose the GUC variable > > itself. Then hide/remove the macro PG_STAT_TMP_DIR. This breaks the > > extensions but it

Re: Higher level questions around shared memory stats

2022-03-31 Thread Andres Freund
Hi, On 2022-03-31 16:16:31 +0900, Kyotaro Horiguchi wrote: > After moving to shared stats, we might want to expose the GUC variable > itself. Then hide/remove the macro PG_STAT_TMP_DIR. This breaks the > extensions but it is better than keeping using PG_STAT_TMP_DIR for > uncertain reasons. The

Re: Higher level questions around shared memory stats

2022-03-31 Thread Kyotaro Horiguchi
At Wed, 30 Mar 2022 17:09:44 -0700, Andres Freund wrote in > Hi, > > On 2022-03-30 16:35:50 -0700, Andres Freund wrote: > > On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > > > Separate from the minutia in [1] I'd like to discuss a few questions of > > > more > > > general interest. I'll

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 16:35:50 -0700, Andres Freund wrote: > On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > > Separate from the minutia in [1] I'd like to discuss a few questions of more > > general interest. I'll post another question or two later. > > 4) What to do with the

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-29 12:17:27 -0700, Andres Freund wrote: > Separate from the minutia in [1] I'd like to discuss a few questions of more > general interest. I'll post another question or two later. 4) What to do with the stats_temp_directory GUC / PG_STAT_TMP_DIR define / pg_stats_temp

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 14:42:23 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > > I think it's reasonably rare because in cases there'd be corruption, we'd > > typically not even have written them out / throw them away explicitly - we > > only read stats when

Re: Higher level questions around shared memory stats

2022-03-30 Thread Andres Freund
Hi, On 2022-03-30 21:44:20 +0200, Peter Eisentraut wrote: > On 29.03.22 23:01, Andres Freund wrote: > > > I think what's actually most important here is the error reporting. We > > > need > > > to make it clear, at least via log messages, that something bad has > > > happened. > > The message

Re: Higher level questions around shared memory stats

2022-03-30 Thread Peter Eisentraut
On 29.03.22 23:01, Andres Freund wrote: I think what's actually most important here is the error reporting. We need to make it clear, at least via log messages, that something bad has happened. The message currently (on HEAD, but similarly on the path) is:

Re: Higher level questions around shared memory stats

2022-03-30 Thread Robert Haas
On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > I think it's reasonably rare because in cases there'd be corruption, we'd > typically not even have written them out / throw them away explicitly - we > only read stats when starting without crash recovery. > > So the "expected" case of

Re: Higher level questions around shared memory stats

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 16:24:02 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > > 1) How to react to corrupted statsfiles? > > > > IMO it'd make more sense to just throw away all stats > > in that case. > > That seems reasonable to me. I think there's some downside,

Re: Higher level questions around shared memory stats

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > 1) How to react to corrupted statsfiles? > > IMO it'd make more sense to just throw away all stats > in that case. That seems reasonable to me. I think there's some downside, in that stats are important, and having some of them might be

Higher level questions around shared memory stats

2022-03-29 Thread Andres Freund
Hi, Separate from the minutia in [1] I'd like to discuss a few questions of more general interest. I'll post another question or two later. 1) How to react to corrupted statsfiles? In HEAD we stop reading stats at the point we detect the stats file to be corrupted. The contents of the