On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Jun 6, 2014 at 5:31 PM, Gurjeet Singh <gurj...@singh.im> wrote: >> On Thu, Jun 5, 2014 at 11:32 PM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: > >> > Buffer saver process itself can crash while saving or restoring >> > buffers. >> >> True. That may lead to partial list of buffers being saved. And the >> code in Reader process tries hard to read only valid data, and punts >> at the first sight of data that doesn't make sense or on ERROR raised >> from Postgres API call. > > Inspite of Reader Process trying hard, I think we should ensure by > some other means that file saved by buffer saver is valid (may be > first write in tmp file and then rename it or something else).
I see no harm in current approach, since even if the file is partially written on shutdown, or if it is corrupted due to hardware corruption, the worst that can happen is the BlockReaders will try to restore, and possibly succeed, a wrong block to shared-buffers. I am okay with your approach of first writing to a temp file, if others see an advantage of doing this and insist on it. >> > IIUC on shutdown request, postmaster will send signal to BG Saver >> > and BG Saver will save the buffers and then postmaster will send >> > signal to checkpointer to shutdown. So before writing Checkpoint >> > record, BG Saver can crash (it might have saved half the buffers) >> >> Case handled as described above. >> >> > or may BG saver saves buffers, but checkpointer crashes (due to >> > power outage or any such thing). >> >> Checkpointer process' crash seems to be irrelevant to Postgres >> Hibernator's workings. > > Yeap, but if it crashes before writing checkpoint record, it will lead to > recovery which is what we are considering. Good point. In case of such recovery, the recovery process will read in the blocks that were recently modified, and were possibly still in shared-buffers when Checkpointer crashed. So after recovery finishes, the BlockReaders will be invoked (because save-files were successfully written before the crash), and they would request the same blocks to be restored. Most likely, those blocks would already be in shared-buffers, hence no cause of concern regarding BlockReaders evicting buffers populated by recovery. >> I think you are trying to argue the wording in my claim "save-files >> are created only on clean shutdowons; not on a crash or immediate >> shutdown", by implying that a crash may occur at any time during and >> after the BufferSaver processing. I agree the wording can be improved. > > Not only wording, but in your above mail Case 2 and 1b would need to > load buffer's and perform recovery as well, so we need to decide which > one to give preference. In the cases you mention, 1b and 2, ideally there will be no save-files because the server either (1b) was still running, or (2) crashed. If there were any save-files present during the previous startup (the one that happened before (1b) hot-backup or (2) crash) of the server, they would have been removed by the BlockReaders soon after the startup. > So If you agree that we should have consideration for recovery data > along with saved files data, then I think we have below options to > consider: I don't think any of the options you mention need any consideration because recovery and buffer-restore process don't seem to be at conflict with each other; not enough to be a concern, IMHO. Thanks and best regards, -- Gurjeet Singh http://gurjeet.singh.im/ EDB www.EnterpriseDB.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
