On Fri, Mar 10, 2017 at 2:00 AM, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 10 March 2017 at 02:55, Robert Haas <robertmh...@gmail.com> wrote: >> Well, that's why I tried to advocate that their should be two separate >> XID limits in shared memory: leave what's there now the way it is, and >> then add a new limit for "don't try to look up XIDs before this point: >> splat". I still think that'd be less risky. > > I'm coming back to this "cold" after an extended break, so I hope I > get the details right.
Yeah, sorry I've been away from this for a while. > TL;DR: doing it that way duplicates a bunch of stuff and is ugly > without offering significant benefit over just fixing the original. > > I started out with the approach you suggested, but it turns out to be > less helpful than you'd think. Simply advancing a new lower limit > field before beginning truncation is no help; there's then a race > where the lower-limit field can be advanced after we test it but > before we actually do the SLRU read from clog. To guard against this > it's necessary for SLRU truncation to take an exclusive lock during > which it advances the lower bound. That way a concurrent reader can > take the lock in shared mode before reading the lower bound and hold > it until it finishes the clog read, knowing that vacuum cannot then > truncate the data out from under it because it'll block trying to > advance the lower limit. That's a good point which I hadn't fully considered. On the other hand, there really are two separate notions of the "oldest" XID. There's the oldest XID that we can safely look up, and then there's the oldest XID that we can't reuse. These two are the same when no truncation is in progress, but when a truncation is in progress then they're different: the oldest XID that's safe to look up is the first one after whatever we're truncating away, but the oldest XID that we can't reuse is the newest one preceding the stuff that we're busy truncating. IOW, when truncation is happening, there's a portion of the XID space whose clog files are being removed - and the XIDs that are in that range aren't safe to look up any more, but are also not available for reuse to prevent wraparound. Right now, all of the relevant fields in VariableCacheData are based on the ready-for-reuse concept, and I don't think that switching some but not all of them to be based on the safe-to-look-up concept necessarily qualifies as an improvement. It's different, but I'm not sure it's better. What if we approached this problem from the other end? Suppose we use a heavyweight lock on, say, transaction ID 1 to represent the right to truncate CLOG. We grab this lock in exclusive mode before beginning to truncate, and in shared mode while looking up XIDs. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
