Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-18 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Patch attached that does the three changes we've talked about:' - make ForgetDatabaseFsyncRequests forget unlink requests as well - make rmtree() not fail on ENOENT - force checkpoint on in dropdb on all platforms This looks fine as

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-17 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: ISTM that we must fix the bgwriter so that ForgetDatabaseFsyncRequests causes PendingUnlinkEntrys for the doomed DB to be thrown away too. Because of the asynchronous nature of ForgetDatabaseFsyncRequests, this

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-17 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Patch attached that does the three changes we've talked about:' - make ForgetDatabaseFsyncRequests forget unlink requests as well - make rmtree() not fail on ENOENT - force checkpoint on in dropdb on all platforms This looks fine as far as it goes,

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Heikki Linnakangas
Tom Lane wrote: ISTM that we must fix the bgwriter so that ForgetDatabaseFsyncRequests causes PendingUnlinkEntrys for the doomed DB to be thrown away too. This should prevent the unlink-live-data scenario, I think. Even then, concurrent deletion attempts are probably possible (since

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Florian suggested a scheme where the xid and epoch is embedded in the filename, but that's unnecessarily complex. We could just make relfilenode a 64-bit integer. 2^64 should be enough for everyone. Doesn't fix the problem unless DB and TS OIDs

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Florian suggested a scheme where the xid and epoch is embedded in the filename, but that's unnecessarily complex. We could just make relfilenode a 64-bit integer. 2^64 should be enough for everyone. Doesn't fix the problem unless

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Heikki Linnakangas
Tom Lane wrote: Actually ... what if the same DB OID and relfilenode get re-made before the checkpoint? Then we'd be unlinking live data. This is improbable but hardly less so than the scenario the PendingUnlinkEntry code was put in to prevent. ISTM that we must fix the bgwriter so that

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: ISTM that we must fix the bgwriter so that ForgetDatabaseFsyncRequests causes PendingUnlinkEntrys for the doomed DB to be thrown away too. Because of the asynchronous nature of ForgetDatabaseFsyncRequests, this still isn't enough,

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-16 Thread Andrew Dunstan
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: ISTM that we must fix the bgwriter so that ForgetDatabaseFsyncRequests causes PendingUnlinkEntrys for the doomed DB to be thrown away too. Because of the asynchronous nature of

[HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-15 Thread Tom Lane
Over the last couple days I twice saw complaints like this during DROP DATABASE: WARNING: could not remove file or directory base/80750/80825: No such file or directory WARNING: could not remove database directory base/80750 I poked at it for awhile and was eventually able to extract a

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-15 Thread Alvaro Herrera
Tom Lane wrote: I think that it should be coded to ignore ENOENT the same as the bgwriter does, and that it should press on and keep trying to delete things even if it gets a failure. Perhaps, if it gets ENOENT, record this fact -- and after rmtree returns, retry the whole thing. -- Alvaro

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-15 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I think that it should be coded to ignore ENOENT the same as the bgwriter does, and that it should press on and keep trying to delete things even if it gets a failure. Perhaps, if it gets ENOENT, record this fact -- and after rmtree

Re: [HACKERS] DROP DATABASE vs patch to not remove files right away

2008-04-15 Thread Andrew Dunstan
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I think that it should be coded to ignore ENOENT the same as the bgwriter does, and that it should press on and keep trying to delete things even if it gets a failure. Perhaps, if it gets ENOENT,