Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Cristian Gafton
On Tue, 29 Jan 2008, Cristian Gafton wrote: I have a ~150GB sized server, containing two databases that are active in mostly read mode. I have noticed lately that the global/pgstat.stat file is somewhere around 1MB freshly after a restart, but at some point it baloons to 74MB in size for no

[HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Cristian Gafton
Hello all, I have a ~150GB sized server, containing two databases that are active in mostly read mode. I have noticed lately that the global/pgstat.stat file is somewhere around 1MB freshly after a restart, but at some point it baloons to 74MB in size for no apparent reason, after a few

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Cristian Gafton
On Tue, 29 Jan 2008, Tom Lane wrote: (Pokes around in the code...) I think the problem here is that the only active mechanism for flushing dead stats-table entries is pgstat_vacuum_tabstat(), which is invoked by a VACUUM command or an autovacuum. Once-a-day VACUUM isn't gonna cut it for you

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Cristian Gafton
On Tue, 29 Jan 2008, Tom Lane wrote: Cristian Gafton [EMAIL PROTECTED] writes: Autovacuum is disabled, since the database is mostly read only. There is a vacuumdb -a -z running nightly on the box. However, the application that queries it does a lot of work with temporary tables - would those

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Tom Lane
Cristian Gafton [EMAIL PROTECTED] writes: Autovacuum is disabled, since the database is mostly read only. There is a vacuumdb -a -z running nightly on the box. However, the application that queries it does a lot of work with temporary tables - would those bloat the stats at all?

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Tom Lane
Cristian Gafton [EMAIL PROTECTED] writes: On Tue, 29 Jan 2008, Cristian Gafton wrote: I have a ~150GB sized server, containing two databases that are active in mostly read mode. I have noticed lately that the global/pgstat.stat file is somewhere around 1MB freshly after a restart, but at

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Cristian Gafton
On Tue, 29 Jan 2008, Tom Lane wrote: On Tue, 29 Jan 2008, Cristian Gafton wrote: I have a ~150GB sized server, containing two databases that are active in mostly read mode. I have noticed lately that the global/pgstat.stat file is somewhere around 1MB freshly after a restart, but at some point

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Tom Lane
Cristian Gafton [EMAIL PROTECTED] writes: We are churning through a bunch of short-lived temp tables. I think that's probably the root of the problem ... Since I reported the problem, the pgstat file is now sitting at 85M, yet the pg_stat* tables barely have any entries in them:

Re: [HACKERS] Large pgstat.stat file causes I/O storm

2008-01-29 Thread Tom Lane
Cristian Gafton [EMAIL PROTECTED] writes: I just ran a vacuumdb -a on the box - the pgstat file is still 90MB in size. If vacuum is supposed to clean up the cruft from pgstat, then I don't know if we're looking at the right cruft - I kind of expected the pgstat file to go down in size and