(CC'ing pgsql-hackers, this started as an IM discussion yesterday but really belongs in the archives)

On 25.10.2011 23:52, Magnus Hagander wrote:
There's a tiny chance to get incomplete xlog files with pg_receivexlog if you 
1. pg_receivexlog finishes write()ing a file but system crashes before fsync() 
2. When pg_receivexlog restarts after crash, the last WAL file was not fully 
flushed to disk, with
holes in the middle, but it has the right length. pg_receivexlog will continue 
streaming from the next file.
not sure if we care about such a narrow window, but maybe we do

So how would we go about fixing that?  Always unlink the last file in
the directory and try from there would seem dangerous too - what if
it's not available on the master anymore, then we might have given up
on data...

Start streaming from the beginning of the last segment, but don't unlink it first. Just overwrite it as you receive the data.

Or, always create new xlog file as "0000000100000001000000D3.partial", and only when it's fully written, fsync it, and then rename it to "0000000100000001000000D3". Then you know that if a file doesn't have the .partial suffix, it's complete. The fact that the last partial file always has the .partial suffix needs some extra pushups in the restore_command, though.

  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to