On Fri, Jun 17, 2011 at 1:22 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Yeah, this seems like a possibly workable direction to explore. I like > this better than what Simon is proposing, because it would fix the > generic issue for all types of catalog SnapshotNow scans.
It would also avoid adding more lock manager traffic which - as recently discussed on the relevant threads - turns out to be a significant performance bottleneck for us right now on some workloads. >> IIUC, the problem with this approach is not correctness but >> performance. Taking snapshots is (currently) expensive. > > Yeah. After mulling it for awhile, what about this idea: we could > redefine SnapshotNow as a snapshot type that includes a list of > transactions-in-progress, somewhat like an MVCC snapshot, but we don't > fill that list from the PGPROC array. Instead, while running a scan > with SnapshotNow, anytime we determine that a particular XID is > still-in-progress, we add that XID to the snapshot's list. > Subsequently, the SnapshotNow code assumes that XID to be > still-in-progress without consulting its actual state. We reset the XID > list to empty when starting a new SnapshotNow scan. (We might be able > to do so less often than that, like only when we do > AcceptInvalidationMessages, but it's not clear to me that there's any > real benefit in hanging onto the state longer.) I think that something like that might possibly work, but what if the XID array overflows? A while back I proposed the idea of a "lazy" snapshot, by which I had in mind something similar to what you are suggesting but different in detail. Initially, when asked to acquire a snapshot, the snapshot manager acknowledges having taken one but does not actually do any work. As long as it sees only XIDs that either precede the oldest XID still running anywhere in the cluster, or have aborted, it can provide answers that are 100% correct without any further data. If it ever sees a newer, non-aborted XID then it goes and really gets an MVCC snapshot at that point, which it can uses from that point onward. I think that it might be possible to make such a system work even for MVCC snapshots generally, but even if not, it might be sufficient for this purpose. Unlike your approach, it would avoid both the "see no rows" and the "see multiple rows" cases, which might be thought an advantage. > I am not, however, particularly pleased with the idea of trying to make > this work in 9.1. I still think that we should back off the attempt > to reduce lock strength in 9.1, and take it up for 9.2. We need to be > stabilizing 9.1 for release, not introducing new untested mechanisms in > it. I like this feature a lot, but it's hard to imagine that any of the fixes anyone has so far suggested can be implemented without collateral damage. Nor is there any certainty that this is the last bug. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
