Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 05/17/2013 11:38 AM, Robert Haas wrote: maybe with a bit of modest pre-extension. When it comes to pre-extension, is it realistic to get a count of backends waiting on the lock and extend the relation by (say) 2x the number of waiting

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 7:36 AM, Stephen Frost sfr...@snowman.net wrote: On the other hand, I do feel like people are worried about over-extending a relation and wasting disk space- but with the way that vacuum can clean up pages at the end, that would only be a temporary situation anyway.

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 8:38 AM, Jaime Casanova ja...@2ndquadrant.com wrote: We can also think in GUC/reloption for next_extend_blocks so formula is needed, or of course the automated calculation that has been proposed s/so formula is needed/so *no* formula is needed btw, we can also use a

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in GUCs for this. We just need to realize when the only

Re: [HACKERS] Extent Locks

2013-05-28 Thread Merlin Moncure
On Tue, May 28, 2013 at 9:07 AM, Stephen Frost sfr...@snowman.net wrote: * Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited

Re: [HACKERS] Extent Locks

2013-05-28 Thread Andres Freund
On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: * Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: I'm really not, at all, excited about adding in GUCs for this. But I thought you were in favor of doing complex stuff like mapping segments filled somewhere else into place :P That wouldn't

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: But I agree. This needs to work without much manual intervention. I think we just need to make autovacuum truncate only if it finds more free space than whatever amount we might have added at that relation size plus

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: On Tue, May 28, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: But I agree. This needs to work without much manual intervention. I think we just need to make autovacuum truncate only if it finds more free space than whatever

Re: [HACKERS] Extent Locks

2013-05-27 Thread Craig Ringer
On 05/17/2013 11:38 AM, Robert Haas wrote: maybe with a bit of modest pre-extension. When it comes to pre-extension, is it realistic to get a count of backends waiting on the lock and extend the relation by (say) 2x the number of waiting backends? Getting a list of lock waiters is always a racey

Re: [HACKERS] Extent Locks

2013-05-27 Thread Craig Ringer
On 05/18/2013 03:15 AM, Josh Berkus wrote: The drawback to this is whatever size we choose is liable to be wrong for some users. Users who currently have a lot of 16K tables would see their databases grow alarmingly. This only becomes a problem for tables that're tiny, right? If your table is

Re: [HACKERS] Extent Locks

2013-05-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: I think it's pretty unrealistic to suppose that this can be made to work. The most obvious problem is that a sequential scan is coded to assume that every block between 0 and the last block in the relation is worth reading, You don't change that.

Re: [HACKERS] Extent Locks

2013-05-17 Thread Robert Haas
On Thu, May 16, 2013 at 9:36 PM, Stephen Frost sfr...@snowman.net wrote: What we do have is a relation extension lock, but the size of the segment on disk has nothing to do with that: there's only one for the whole relation, and you hold it when adding a block to the relation. Yes, which is

Re: [HACKERS] Extent Locks

2013-05-17 Thread Robert Haas
On Thu, May 16, 2013 at 11:55 PM, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: I think it's pretty unrealistic to suppose that this can be made to work. The most obvious problem is that a sequential scan is coded to assume that every block between 0 and

Re: [HACKERS] Extent Locks

2013-05-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Thu, May 16, 2013 at 11:55 PM, Stephen Frost sfr...@snowman.net wrote: You don't change that. However, when a seq scan asks the storage layer for blocks that it knows don't actually exist, it can simply skip over them or return empty records

Re: [HACKERS] Extent Locks

2013-05-17 Thread Josh Berkus
Robert, But I still feel like that thought experiment indicates that there must be a solution here just by rejiggering the locking, and maybe with a bit of modest pre-extension. The mediocre results of my last couple tries must indicate that I wasn't entirely successful in getting the

Re: [HACKERS] Extent Locks

2013-05-16 Thread Robert Haas
On Wed, May 15, 2013 at 8:54 PM, Stephen Frost sfr...@snowman.net wrote: Starting a new thread to avoid hijacking Heikki's original, but.. * Heikki Linnakangas (hlinnakan...@vmware.com) wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires

Re: [HACKERS] Extent Locks

2013-05-16 Thread Stephen Frost
Robert, For not understanding me, we seem to be in violent agreement. ;) * Robert Haas (robertmh...@gmail.com) wrote: I think you might be confused, or else I'm confused, because I don't believe we have any such thing as an extent lock. The relation extension lock is what I was referring

[HACKERS] Extent Locks

2013-05-15 Thread Stephen Frost
All, Starting a new thread to avoid hijacking Heikki's original, but.. * Heikki Linnakangas (hlinnakan...@vmware.com) wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short