On Sat, Nov 26, 2011 at 10:43 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Nov 26, 2011 at 10:52 AM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: >> What we can do is when a transaction comes to compute its snapshot, it >> checks if some other transaction is already computing a snapshot for >> itself. If so, it just sleeps on the lock. When the other process >> finishes computing the snapshot, it saves the snapshot is a shared >> area and wakes up all processes waiting for the snapshot. All those >> processes then just copy the snapshot from the shared area and they >> are done. This will not only reduce the total CPU consumption by >> avoiding repetitive work, but would also reduce the total time for >> which ProcArrayLock is held in SHARED mode by avoiding pipeline of >> GetSnapshotData calls. I am currently trying the shared work queue >> mechanism to implement this, but I am sure we can do it this in some >> other way too. > > I don't quite understand how this is going to work.
I will try, keeping it simple. > Transaction A > ends, invaliding the shared snapshot. Now transactions B, C, and D > come along wishing to take snapshots. B begins taking a snapshot, so > C and D wait (blech!) Yeah, C and D waits. But thats better than concurrently doing the same computation. > for that to be complete. Then, they start > copying the snapshot. And they are holding the ProcArrayLock in shared mode. > Meanwhile, transaction E ends, invalidating the > shared snapshot, E can't end until B, C and D are done with copying the snapshot. > and then transaction F comes along, wanting to take a > snapshot. If F constructs a new shared snapshot, then doesn't that > overwrite the same memory area C and D were busy copying? It seems > like F needs some way to know that C and D are done with the old > shared snapshot before it starts writing a new one. Right. And that can easily be achieved by holding shared lock on ProcArray. The snapshot is invalidated by holding exclusive lock and set/copied while holding shared lock. I am assuming setting a boolean (valid/invalid) can safely be done with a shared lock. But I may be wrong. > Furthermore, it's > hard to understand how this could be a net improvement in the general > case, because now both B and F are copying everything twice (once to > the shared area and one to backend-local memory) instead of just once > (to backend-local memory) and C and D are sleeping (yikes!). Yes, B and F pay a price of double copy. But I think it can be a net saving because C and D (and many more hopefully) don't need to recompute the snapshot again by going over a potentially large ProcArray. So as I demonstrated in the illustration, the total time for which ProcArray lock is held will be significantly less with large number of clients. > That > could maybe be a gain at very high concurrency when spinlock > contention is eating us alive, but it doesn't seem like a good idea in > general. Maybe I'm missing something; did you mean to attach a patch? > I have a patch that I am attaching. It contains some unrelated changes (don't know why; may be I took a diff with some other branch), but you will get the idea. This needs improvement though because it just checks if a valid shared snapshot is available and copies that. If not, it goes ahead and computes one for itself. We need something more intelligent to know that a snapshot computation is in progress and we should wait instead of building our own. This patch had shown 15-20% improvement on the HP box that we are using for the benchmarks. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com
shared-snapshot-v2.patch
Description: Binary data
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
