Re: [HACKERS] typo: XIDs are actually compared using modulo-2^32 arithmetic

2013-12-14 Thread Greg Stark
I don't have a source tree handy but iirc we treaty 2^31 values as being in
the past and 2^31 values as being in the future.

I've been trying to think how to protect better against the recent vacuum
freeze bug. If someone ruins vacuum freeze now and has any wrapped values
they'll destroy their possibly recoverable data.

It seems to me we shouldn't really need 2^31 values in the future. If
vacuum or hot pruning comes across an xid far in the future, say a million
xids further into the future than the most recent transaction, then it
should signal an error rather than just treat it as being in the future.

-- 
greg
On 12 Dec 2013 09:41, Tom Lane t...@sss.pgh.pa.us wrote:

 Gianni Ciolli gianni.cio...@2ndquadrant.it writes:
  It seems there is a typo here:
 
 http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND
  where we say that we compare XIDs using arithmetic modulo 2^31, which
  should instead be 2^32 (as it is with uint32, e.g. xid_age).

 [ thinks about that for awhile... ]  Yeah, I think you're right.
 Patch pushed, thanks!

 regards, tom lane


 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] typo: XIDs are actually compared using modulo-2^32 arithmetic

2013-12-14 Thread Andres Freund
On 2013-12-14 20:19:11 +, Greg Stark wrote:
 I don't have a source tree handy but iirc we treaty 2^31 values as being in
 the past and 2^31 values as being in the future.
 
 I've been trying to think how to protect better against the recent vacuum
 freeze bug. If someone ruins vacuum freeze now and has any wrapped values
 they'll destroy their possibly recoverable data.

Fortunately that's exceedingly unlikely to happen. There's basically two
consequences the bug can have:
a) we don't freeze tuples on pages that are already marked all-visible
   because we're doing a partial scan and thus don't scan them.
b) (9.2+) we don't freeze tuples on a page not marked all visible,
   because a buffer is pinned and we skip those when !scan_all.

a) can lead to the tuple vanishing again because they are reported as
being in progress, after 2^31 xids passed. But by virtue of being on an
all-visible page, they are fully hinted. Which means, that after the
wraparound they will be reported as delete-in-progress or
insert-in-progress. Luckily neither will get vacuumed away. They will
just be invisible.

What can happen with b) is that the clog gets truncated to somewhere
between the real relfrozenxid and the computed relfrozenxid. In that
case we'll get errors when later doing a
HeapTupleSatisfiesVacuum/HTSMVCC. But it's quite likely that the tuple
will get vacuumed at some point before 2^31 xids have passed since its
not marked all visible and thus will be scanned with each future vacuum.

So, for the data to be removed permanently you'd have to hit b) with
partial vacuums (scan_all vacuums do wait!) several times in a row. That
seems unlikely.

 It seems to me we shouldn't really need 2^31 values in the future. If
 vacuum or hot pruning comes across an xid far in the future, say a million
 xids further into the future than the most recent transaction, then it
 should signal an error rather than just treat it as being in the future.

Yea, I have wondered about that as well.

Greetings,

Andres Freund

-- 
 Andres Freund http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] typo: XIDs are actually compared using modulo-2^32 arithmetic

2013-12-12 Thread Tom Lane
Gianni Ciolli gianni.cio...@2ndquadrant.it writes:
 It seems there is a typo here:
   
 http://www.postgresql.org/docs/devel/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND
 where we say that we compare XIDs using arithmetic modulo 2^31, which
 should instead be 2^32 (as it is with uint32, e.g. xid_age).

[ thinks about that for awhile... ]  Yeah, I think you're right.
Patch pushed, thanks!

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers