Florian G. Pflug wrote:
August Zajonc wrote:
I'm confused about this.

As long as we assert the rule that the file name can't change on the move, then after commit the file can be in only one of two places. The name of the file is known (ie, pg_class). The directories are known. What needs to be carried forwarded past a checkpoint? We don't even look at WAL, so checkpoints are irrelevant it seems If there is a crash just after commit and before the move, no harm. You just move on startup. If the move fails, no harm, you can emit warning and open in /pending (or simply error, even easier).
If you're going to open the file from /pending, whats the point of moving
it in the first place?
Allow time for someone to sort out the disk situation without impacting things. Preserves the concept that after COMMIT a file exists on disk that is accessible, which is what people I think expect.
The idea would have to be that you move on commit (Or on COMMIT-record
replay, in case of a crash), and then, after recovering the whole wal,
you could remove leftover files in /pending.
I think so. What I was thinking was

Limit to moves from spclocation/file.new to spclocation/file. So given a pg_class filename the datafile can only have two possible names. relfilenode or relfilenode.new

you commit, then move.

If crash occurs before commit, you leak a .new table.

If move fails after commit you emit warning. The commit is still valid, because fopen can fall back to .new. The data is still there.

On crash recovery move .new files that show up in pg_class to their proper name if the disk has been fixed and move can succeed. For .new files that don't exist in pg_class after log replay, delete them.

Fallback open.

fopen relfilenode,
if ENOENT fopen relfilenode.new
   if ENOENT error
   elseif emit warning

So, what are you going to do if the move fails? You cannot roll back, and
you cannot update the CLOG (because than others would see your new table,
but no datafile). The only option is to PANIC. This will lead to a server
restart, WAL recovery, and probably another PANIC once the COMMIT-record
is replayed (Since the move probably still won't be possible).
That was the idea of the fallback generally, avoid this issue. You never rollback. If datafile is not where expected, it can only be one other place.
It might be even worse - I'm not sure that a rename is an atomic operation
on most filesystems. If it's not, then you might end up with two files if
power fails *just* as you rename, or, worse with no file at all. Even a slight
possibility of the second case seems unacceptable - I means loosing
a committed transaction.
Yes, atomic renames are an assumption.

I agree that we should eventually find a way to guarantee either no file
leakage, or at least an upper bound on the amount of wasted space. But
doing so at the cost of PANICing if the move fails seems like a bad

greetings, Florian Pflug

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Reply via email to