Re: [PATCHES] SRA Win32 sync() code
Bruce Momjian [EMAIL PROTECTED] writes: Not sure how we are going to do this in Win32, but somehow we will have to record all open files between checkpoints in an area that the checkpoint process can read during a checkpoint. One reason I like the idea of adopting a sync-when-you-write policy is that it eliminates the need for anything as messy as that. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] SRA Win32 sync() code
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: One reason I like the idea of adopting a sync-when-you-write policy is that it eliminates the need for anything as messy as that. Yes, but can we do it without causing a performance degredation, and I would hate to change something to make things easier on Win32 while penalizing all platforms. Having to keep a list of modified files in shared memory isn't a penalty? Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view if we could stop depending on sync(). The fact that Windows hasn't got sync() is merely another reason we should stop using it. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] SRA Win32 sync() code
Tom Lane wrote: Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view if we could stop depending on sync(). Which function guarantees that renames of WAL files arrived on the disk? AFAIK sync() is the only function that guarantees that. What about the sync app from sysinternals? It seems Mark Russinovich figured out how to implement sync on Win32: http://www.sysinternals.com/ntw2k/source/misc.shtml#Sync It requires administrative priveledges, but it shouldn't be that difficult to write a tiny service that runs in the LocalSystem account, listens to a pipe and syncs all disks when asked. -- Manfred ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] SRA Win32 sync() code
Manfred Spraul [EMAIL PROTECTED] writes: Which function guarantees that renames of WAL files arrived on the disk? The OS itself is supposed to guarantee that; that's what a journaling file system is for. In any case, I don't think we care. Renaming would apply only to WAL segments that are not currently needed where they are, and would only be needed under their new names at some future time. If the rename gets lost shortly after it's done, it can be redone. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] SRA Win32 sync() code
Manfred Spraul wrote: Tom Lane wrote: Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view if we could stop depending on sync(). Which function guarantees that renames of WAL files arrived on the disk? AFAIK sync() is the only function that guarantees that. What about the sync app from sysinternals? It seems Mark Russinovich figured out how to implement sync on Win32: http://www.sysinternals.com/ntw2k/source/misc.shtml#Sync It requires administrative priveledges, but it shouldn't be that difficult to write a tiny service that runs in the LocalSystem account, listens to a pipe and syncs all disks when asked. I think we'd have to do it from scratch, because of these license terms: --- There is no charge to use any of the software published on this Web site at home or at work, so long as each user downloads and installs the product directly from www.sysinternals.com. A commercial license is required to redistribute any of these utilities directly (whether by computer media, a file server, an email attachment, etc.) or to embed them in- or link them to- another program. -- Also, do we want to force a broad brush sync() of just fsync our own files? cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] SRA Win32 sync() code
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: One reason I like the idea of adopting a sync-when-you-write policy is that it eliminates the need for anything as messy as that. Yes, but can we do it without causing a performance degredation, and I would hate to change something to make things easier on Win32 while penalizing all platforms. Having to keep a list of modified files in shared memory isn't a penalty? Seriously though, if we can move the bulk of the writing work into background processes then I don't believe that there will be any significant penalty for regular backends. And I believe that it would be a huge advantage from a correctness point of view if we could stop depending on sync(). The fact that Windows hasn't got sync() is merely another reason we should stop using it. If the background writer starts using fsync(), we can have normal backends that do a write() set a shared memory boolean. We can then test that boolean and do sync() only if other backends had to do their own writes. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] SRA Win32 sync() code
Jan Wieck [EMAIL PROTECTED] writes: Well, the bgwriter has basically the same chance the checkpointer has ... mdblindwrt() in the end, because he doesn't have the relcache handy. We could easily get rid of mdblindwrt --- there is no very good reason that we use the relcache for I/O. There could and should be a lower-level notion of open relation that bgwriter and checkpoint could use. See recent discussion with Neil, for example. Vadim had always wanted to do that, IIRC. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
