Re: [PERFORM] too many clog files

2008-09-10 Thread Kevin Grittner
Matt Smiley [EMAIL PROTECTED] wrote: Alvaro Herrera wrote: Move the old clog files back where they were, and run VACUUM FREEZE in all your databases. That should clean up all the old pg_clog files, if you're really that desperate. Has anyone actually seen a CLOG file get removed under

Re: [PERFORM] too many clog files

2008-09-10 Thread Scott Marlowe
On Wed, Sep 10, 2008 at 8:58 AM, Kevin Grittner [EMAIL PROTECTED] wrote: Matt Smiley [EMAIL PROTECTED] wrote: Alvaro Herrera wrote: Move the old clog files back where they were, and run VACUUM FREEZE in all your databases. That should clean up all the old pg_clog files, if you're really

Re: [PERFORM] too many clog files

2008-09-10 Thread Tom Lane
Scott Marlowe [EMAIL PROTECTED] writes: On Wed, Sep 10, 2008 at 8:58 AM, Kevin Grittner [EMAIL PROTECTED] wrote: Some of my high-volume databases don't quite go back to , but this does seem to be a problem. I have confirmed that VACUUM FREEZE on all but template0 (which doesn't allow

Re: [PERFORM] too many clog files

2008-09-10 Thread Kevin Grittner
Tom Lane [EMAIL PROTECTED] wrote: The expected behavior (in 8.2 and newer) is to maintain about autovacuum_freeze_max_age transactions' worth of clog; which is to say about 50MB at the default settings. The active database I checked, where it didn't go all the way back to , had 50 MB

Re: [PERFORM] too many clog files

2008-09-10 Thread Scott Carey
And potentially to tune down the number kept by modifying the appropriate freeze parameter for 8.1 (I'm not sure of the details), so that it keeps perhaps 20MB or so rather than 50MB. On Wed, Sep 10, 2008 at 10:47 AM, Kevin Grittner [EMAIL PROTECTED] wrote: Tom Lane [EMAIL PROTECTED] wrote:

Re: [PERFORM] too many clog files

2008-09-10 Thread Alvaro Herrera
Kevin Grittner escribió: It sounds like the advice to the OP that running VACUUM FREEZE on all databases to clean up the files was off base? His responses are not explicit enough to know. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company -

Re: [PERFORM] too many clog files

2008-09-09 Thread Matt Smiley
Hi Duan, As others have said, you should probably attempt to run pg_dump to export your database. If that doesn't work, consider restoring from backup. If the dump does work, you can create a clean PGDATA directory (using initdb like when you setup your original installation), and create a

Re: [PERFORM] too many clog files

2008-09-09 Thread Matt Smiley
Alvaro Herrera wrote: Move the old clog files back where they were, and run VACUUM FREEZE in all your databases. That should clean up all the old pg_clog files, if you're really that desperate. Has anyone actually seen a CLOG file get removed under 8.2 or 8.3? How about 8.1? I'm probably

Re: [PERFORM] too many clog files

2008-09-07 Thread Duan Ligong
and executing the select clause. and they are not long-running transactions, aren't they? because the procpid changes when I logout and login again. Thanks Duan --- On Thu, 9/4/08, Duan Ligong [EMAIL PROTECTED] wrote: From: Duan Ligong [EMAIL PROTECTED] Subject: Re: [PERFORM] too many clog files

Re: [PERFORM] too many clog files

2008-09-07 Thread Duan Ligong
Alvaro wrote: Move the old clog files back where they were, and run VACUUM FREEZE in all your databases. That should clean up all the old pg_clog files, if you're really that desperate. This is not something that I'd recommend doing on a periodic basis ... Thank you for your suggestions. I

Re: [PERFORM] too many clog files

2008-09-05 Thread Matthew Wakeling
On Fri, 5 Sep 2008, Duan Ligong wrote: Well, we could not wait so long and just moved the old clog files. Congratulations. You have probably just destroyed your database. Matthew -- Lord grant me patience, and I want it NOW! -- Sent via pgsql-performance mailing list

Re: [PERFORM] too many clog files

2008-09-05 Thread Alvaro Herrera
Duan Ligong wrote: Greg wrote: On Tue, 2 Sep 2008, Duan Ligong wrote: - Does Vacuum delete the old clog files? Yes, if those transactions are all done. One possibility here is that you've got some really long-running transaction floating around that is keeping normal clog

Re: [PERFORM] too many clog files

2008-09-05 Thread Scott Marlowe
On Thu, Sep 4, 2008 at 8:58 PM, Duan Ligong [EMAIL PROTECTED] wrote: Thanks for your reply. Greg wrote: On Tue, 2 Sep 2008, Duan Ligong wrote: - Does Vacuum delete the old clog files? Yes, if those transactions are all done. One possibility here is that you've got some really

Re: [PERFORM] too many clog files

2008-09-04 Thread Duan Ligong
Thanks for your reply. Greg wrote: On Tue, 2 Sep 2008, Duan Ligong wrote: - Does Vacuum delete the old clog files? Yes, if those transactions are all done. One possibility here is that you've got some really long-running transaction floating around that is keeping normal clog cleanup

Re: [PERFORM] too many clog files

2008-09-02 Thread Guillaume Lelarge
Greg Smith a écrit : [...] - When, or in what case is a new clog file produced? Every 32K transactions. Are you sure about this? y clog files get up to 262144 bytes. Which means 100 transactions' status: 262144 bytes are 2Mb (mega bits), so if a status is 2 bits, it holds 1M

Re: [PERFORM] too many clog files

2008-09-02 Thread Greg Smith
On Tue, 2 Sep 2008, Guillaume Lelarge wrote: AFAICT, 32K transactions' status are available on a single (8KB) page. You're right, I had that right on the refered to page but mangled it when writing the e-mail. 262144 bytes are 2Mb (mega bits), so if a status is 2 bits, [a clog file] holds

[PERFORM] too many clog files

2008-09-01 Thread Duan Ligong
Hi, there. I have encountered an issue that there are too many clog file under the .../pg_clog/ directory. Some of them were even produced one month ago. My questions: - Does Vacuum delete the old clog files? - Can we controll the maximum number of the clog files? - When, or in what case is a

Re: [PERFORM] too many clog files

2008-09-01 Thread Duan Ligong
Alvaro Herrera wrote: Duan Ligong wrote: I have encountered an issue that there are too many clog file under the .../pg_clog/ directory. Some of them were even produced one month ago. If you're going to repost a question, it is only polite that you link to the answers already

Re: [PERFORM] too many clog files

2008-09-01 Thread Greg Smith
On Tue, 2 Sep 2008, Duan Ligong wrote: - Does Vacuum delete the old clog files? Yes, if those transactions are all done. One possibility here is that you've got some really long-running transaction floating around that is keeping normal clog cleanup from happening. Take a look at the

Re: [PERFORM] too many clog files

2008-09-01 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: On Tue, 2 Sep 2008, Duan Ligong wrote: - Can we controll the maximum number of the clog files? The reference Alvaro suggested at http://www.postgresql.org/docs/8.3/interactive/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND goes over that. If you reduce