Re: [PATCHES] Cleaning up unreferenced table files

2005-05-07 Thread Heikki Linnakangas
Maybe we should take a different approach to the problem: 1. Create new file with an extension to mark that it's not yet committed (eg. 1234.notcommitted) 2. ... 3. Take CheckpointStartLock 4. Write commit record to WAL, with list of created files. 5. rename created file (1234.notcommitted -

Re: [PATCHES] Cleaning up unreferenced table files

2005-05-07 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Maybe we should take a different approach to the problem: 1. Create new file with an extension to mark that it's not yet committed (eg. 1234.notcommitted) This is pushing the problem into the wrong place, viz the lowest-level file access

Re: [PATCHES] Cleaning up unreferenced table files

2005-05-06 Thread Heikki Linnakangas
On Thu, 5 May 2005, Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Applied. Now that I've had a chance to look at it, this patch is thoroughly broken. Problems observed in a quick review: 1. It doesn't work at all for non-default tablespaces: it will claim that every file in such a

Re: [PATCHES] Cleaning up unreferenced table files

2005-05-05 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Applied. Now that I've had a chance to look at it, this patch is thoroughly broken. Problems observed in a quick review: 1. It doesn't work at all for non-default tablespaces: it will claim that every file in such a tablespace is stale. The fact

Re: [PATCHES] Cleaning up unreferenced table files

2005-05-04 Thread Simon Riggs
Heikki, Good patch. ...The patch makes no mention of temporary files, which are left there after a crash. Temp files are only removed on a normal startup, whereas the patch invokes removal on both normal startup and crash recovery. That doesn't make much sense... Also, temp file deletion

Re: [PATCHES] Cleaning up unreferenced table files

2005-05-04 Thread Bruce Momjian
FYI, his patch is in CVS and now only _reports_ unreferenced files, and it happens on recovery and normal startup. --- Simon Riggs wrote: Heikki, Good patch. ...The patch makes no mention of temporary files, which

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-28 Thread Bruce Momjian
Uh, you forgot to add cleanup.h. --- Heikki Linnakangas wrote: On Tue, 26 Apr 2005, Alvaro Herrera wrote: You forgot the attachment? Damn. It happens time after time... - Heikki Content-Description: [

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-28 Thread Heikki Linnakangas
How embarrasing... I hope it's all there now. On Thu, 28 Apr 2005, Bruce Momjian wrote: Uh, you forgot to add cleanup.h. --- Heikki Linnakangas wrote: On Tue, 26 Apr 2005, Alvaro Herrera wrote: You forgot the attachment? Damn.

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Heikki Linnakangas
On Tue, 26 Apr 2005, Alvaro Herrera wrote: You forgot the attachment? Damn. It happens time after time... - HeikkiIndex: doc/src/sgml/maintenance.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/maintenance.sgml,v retrieving

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Heikki Linnakangas
On Tue, 26 Apr 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I feel that crashes that leaves behind stale files are rare. Indeed, and getting more so all the time ... How so? Have changes been made in those parts of the code? which makes me question the value of doing anything

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: On Tue, 26 Apr 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I feel that crashes that leaves behind stale files are rare. Indeed, and getting more so all the time ... How so? Have changes been made in those parts of the

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Heikki Linnakangas
On Wed, 27 Apr 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Tue, 26 Apr 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I feel that crashes that leaves behind stale files are rare. Indeed, and getting more so all the time ... How so? Have changes been

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Bruce Momjian
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Tue, 26 Apr 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I feel that crashes that leaves behind stale files are rare. Indeed, and getting more so all the time ... How so? Have changes been made in

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: I think we do need the patch, at least to find out if there is an issue we don't know about. My point is that we won't find out anything, because we will have no idea if people are noticing the log entries at all, much less telling us about 'em. Of

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-27 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I think we do need the patch, at least to find out if there is an issue we don't know about. My point is that we won't find out anything, because we will have no idea if people are noticing the log entries at all, much less

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-26 Thread Heikki Linnakangas
On Mon, 25 Apr 2005, Bruce Momjian wrote: Tom Lane wrote: ... I think though that we ought to first consider the question of whether we *want* this functionality. On reflection I'm fairly nervous about the idea of actually deleting anything during startup --- seems like a good recipe for turning

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-26 Thread Heikki Linnakangas
On Tue, 26 Apr 2005, Heikki Linnakangas wrote: On Mon, 25 Apr 2005, Bruce Momjian wrote: ... I guess I am happy with just reporting during startup like the patch does now. Ok. I'll fix the design issues Tom addressed earlier, add documentation, and resubmit. Here you go. The new functionality is

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-26 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: I feel that crashes that leaves behind stale files are rare. Indeed, and getting more so all the time ... which makes me question the value of doing anything about this at all. regards, tom lane

Re: [PATCHES] Cleaning up unreferenced table files

2005-04-24 Thread Bruce Momjian
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: On Sat, 5 Mar 2005, Tom Lane wrote: xlog.c is a fairly random place to put that functionality. Didn't it strike any warning bells for you when you had to add so many new #includes? I'm not entirely sure where this should go,

Re: [PATCHES] Cleaning up unreferenced table files

2005-03-06 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: On Sat, 5 Mar 2005, Tom Lane wrote: xlog.c is a fairly random place to put that functionality. Didn't it strike any warning bells for you when you had to add so many new #includes? I'm not entirely sure where this should go, but not there. Yeah

Re: [PATCHES] Cleaning up unreferenced table files

2005-03-05 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Here's a patch for the TODO item Remove unreferenced table files created by transactions that were in-progress when the server terminated abruptly. xlog.c is a fairly random place to put that functionality. Didn't it strike any warning bells for

Re: [PATCHES] Cleaning up unreferenced table files

2005-03-05 Thread Heikki Linnakangas
On Sat, 5 Mar 2005, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Here's a patch for the TODO item Remove unreferenced table files created by transactions that were in-progress when the server terminated abruptly. xlog.c is a fairly random place to put that functionality. Didn't