Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-23 Thread Simon Riggs
On 15 May 2013 16:35, Heikki Linnakangas hlinnakan...@vmware.com wrote: Shared memory space is limited, but we only need the watermarks for any in-progress truncations. Let's keep them in shared memory, in a small fixed-size array. That limits the number of concurrent truncations that can be

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-22 Thread Robert Haas
On Mon, May 20, 2013 at 10:19 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: It shouldn't be difficult to restrict the set of backends that have to be signaled to those that have the relation open. You could have a special kind of catchup signal that means catch yourself up, but don't

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Robert Haas
On Fri, May 17, 2013 at 3:38 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: If we could use the catchup interrupts to speed that up though, that would be much better. I think vacuum could simply send a catchup interrupt, and wait until everyone has caught up. That would significantly

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Heikki Linnakangas
On 20.05.2013 16:59, Robert Haas wrote: On Fri, May 17, 2013 at 3:38 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: If we could use the catchup interrupts to speed that up though, that would be much better. I think vacuum could simply send a catchup interrupt, and wait until everyone

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Heikki Linnakangas
On 17.05.2013 12:35, Andres Freund wrote: On 2013-05-17 10:45:26 +0300, Heikki Linnakangas wrote: On 16.05.2013 04:15, Andres Freund wrote: Couldn't we just take the extension lock and then walk backwards from the rechecked end of relation ConditionalLockBufferForCleanup() the buffers? For

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Heikki Linnakangas
On 16.05.2013 00:18, Robert Haas wrote: On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Shared memory space is limited, but we only need the watermarks for any in-progress truncations. Let's keep them in shared memory, in a small fixed-size array. That

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Heikki Linnakangas
On 16.05.2013 04:15, Andres Freund wrote: On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short duration, it can be enough to

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Andres Freund
On 2013-05-17 10:45:26 +0300, Heikki Linnakangas wrote: On 16.05.2013 04:15, Andres Freund wrote: On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Robert Haas
On Wed, May 15, 2013 at 8:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I've been thinking for a while that we need some other system for managing other kinds of invalidations. For example, suppose we want to cache relation sizes in blocks. So we

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: 2. If you don't find an entry for your target rel in the cache, aren't you still going to have to do an lseek? Don't think of it as a cache. The caching happens inside each backend's relcache; the shared memory structure is just a tool to force

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Robert Haas
On Thu, May 16, 2013 at 1:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 2. If you don't find an entry for your target rel in the cache, aren't you still going to have to do an lseek? Don't think of it as a cache. The caching happens inside each

[HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Heikki Linnakangas
Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short duration, it can be enough to cause a hiccup in query processing while it's held. Also, if there is a continuous stream of

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Robert Haas
On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Shared memory space is limited, but we only need the watermarks for any in-progress truncations. Let's keep them in shared memory, in a small fixed-size array. That limits the number of concurrent truncations

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: To not slow down common backend operations, the values (or lack thereof) are cached in relcache. To sync the relcache when the values change, there will be a new

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Robert Haas
On Wed, May 15, 2013 at 7:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Another problem is that sinval resets are bad for performance, and anything we do that pushes more messages through sinval will increase the frequency of resets. I've been thinking that we should increase the size of the

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I've been thinking for a while that we need some other system for managing other kinds of invalidations. For example, suppose we want to cache relation sizes in blocks. So we allocate 4kB of shared memory, interpreted as an array of 512 8-byte

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Andres Freund
On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short duration, it can be enough to cause a hiccup in query processing while