August Zajonc <[EMAIL PROTECTED]> writes: > I do have a question about jamming though. Will the system work if the > file ended up stuck in this folder? Let's say the move destination has a > duplicate file that conflicts, or permissions change under you, or disks > fill.
Yeah, the move-the-file approach seems to introduce its own set of failure modes, which hardly seems like something we want. I had an idea this morning that might be useful: back off the strength of what we try to guarantee. Specifically, does it matter if we leak a file on crash, as long as it isn't occupying a lot of disk space? (I suppose if you had enough crashes to accumulate many thousands of leaked files, the directory entries would start to be a performance drag, but if your DB crashes that much you have other problems.) This leads to the idea that we don't really need to protect the open(O_CREAT) per se. Rather, we can emit a WAL entry *after* successful creation of a file, while it's still empty. This eliminates all the issues about logging an action that might fail. The WAL entry would need to include the relfilenode and the creating XID. Crash recovery would track these until it saw the commit or abort or prepare record for the XID, and if it didn't find any, would remove the file. With this approach I think we'd not even need to force-fsync the WAL entry; instead treat it like an async COMMIT record (pass its LSN to the walwriter). Even in the absence of any subsequent WAL activity, it would reach disk via the walwriter before the new file could be filled to a size that would bother anyone. (If the new file is being filled via WAL-logged insertions, then you can probably make even stronger statements than that, but we do have operations like COPY and CREATE INDEX that can fill a file with unlogged insertions.) So the performance impact would be about nil. I still don't think that this area is in urgent need of a fix, but I wanted to get this idea into the archives. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate