Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-04-01 Thread Jan Wieck
On 3/28/2011 12:35 PM, Jan Wieck wrote: On 3/27/2011 10:43 PM, Tom Lane wrote: In particular, I thought the direction Jan was headed was to release and reacquire the lock between truncating off limited-size chunks of the file. If we do that, we probably *don't* want or need to allow

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-04-01 Thread Jim Nasby
On Mar 27, 2011, at 9:43 PM, Tom Lane wrote: 1) move the truncating to a new transaction just like we currently do toast tables in a separate transaction from the main vacuum. +1 if we are going to continue the behavior of allowing other transactions to kick autovac off the exclusive lock.

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-28 Thread Jan Wieck
On 3/27/2011 9:51 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 9:41 PM, Jan Wieckjanwi...@yahoo.com wrote: On 3/27/2011 6:21 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieckjanwi...@yahoo.comwrote: Since we are talking about stable releases, I think just

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-28 Thread Jan Wieck
On 3/27/2011 10:43 PM, Tom Lane wrote: In particular, I thought the direction Jan was headed was to release and reacquire the lock between truncating off limited-size chunks of the file. If we do that, we probably *don't* want or need to allow autovac to be booted off the lock more quickly.

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Simon Riggs
On Sat, Mar 26, 2011 at 8:05 PM, Jan Wieck janwi...@yahoo.com wrote: On 3/26/2011 12:12 PM, Simon Riggs wrote: On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieckjanwi...@yahoo.com  wrote:  My current idea for a fix is to modify lazy_truncate_heap(). It does acquire  and release the exclusive lock,

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Mar 26, 2011, at 4:16 PM, Jan Wieck janwi...@yahoo.com wrote: That was what I meant. Go in steps of 16-64MB backwards and scan from there to the current end in forward direction to find a nondeletable block. In between these steps, release and reacquire the exclusive lock so that client

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Simon Riggs
On Sun, Mar 27, 2011 at 2:13 PM, Robert Haas robertmh...@gmail.com wrote: On Mar 26, 2011, at 4:16 PM, Jan Wieck janwi...@yahoo.com wrote: That was what I meant. Go in steps of 16-64MB backwards and scan from there to the current end in forward direction to find a nondeletable block. In

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Jan Wieck
On 3/27/2011 1:24 PM, Simon Riggs wrote: So we should truncate in 16MB chunks also. On a second though, fiddling with the scan direction is probably too much of a change for back releases anyway. That 8.3/8.4 can get into a situation, where autovacuum causes a 12 minute freeze of a

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Greg Stark
On Sun, Mar 27, 2011 at 8:25 PM, Jan Wieck janwi...@yahoo.com wrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive lock is enough. We can then try to further improve things for future releases. I like all of: 1) move the truncating to a new

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck janwi...@yahoo.com wrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive lock is enough. We can then try to further improve things for future releases. That seems unsafe - things can change under you

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Jan Wieck
On 3/27/2011 6:21 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieckjanwi...@yahoo.com wrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive lock is enough. We can then try to further improve things for future releases. That

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Robert Haas
On Sun, Mar 27, 2011 at 9:41 PM, Jan Wieck janwi...@yahoo.com wrote: On 3/27/2011 6:21 PM, Robert Haas wrote: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieckjanwi...@yahoo.com  wrote:  Since we are talking about stable releases, I think just releasing and  reacquiring the exclusive lock is

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Mar 27, 2011 at 3:25 PM, Jan Wieck janwi...@yahoo.com wrote: Since we are talking about stable releases, I think just releasing and reacquiring the exclusive lock is enough. We can then try to further improve things for future releases. That

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-27 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: I like all of: 1) move the truncating to a new transaction just like we currently do toast tables in a separate transaction from the main vacuum. +1 if we are going to continue the behavior of allowing other transactions to kick autovac off the exclusive

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck janwi...@yahoo.com wrote: My current idea for a fix is to modify lazy_truncate_heap(). It does acquire and release the exclusive lock, so it should be possible to do this in smaller chunks, releasing and reacquiring the lock so that client

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Itagaki Takahiro
On Sun, Mar 27, 2011 at 01:12, Simon Riggs si...@2ndquadrant.com wrote: At the same time I would change count_nondeletable_pages() so that it uses a forward scan direction (if that leads to a speedup). +1. Do we need that? Linux readahead works in both directions doesn't it? Guess it

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 1:44 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Sun, Mar 27, 2011 at 01:12, Simon Riggs si...@2ndquadrant.com wrote: At the same time I would change count_nondeletable_pages() so that it uses a forward scan direction (if that leads to a speedup). +1.

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 12:12 PM, Simon Riggs wrote: On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieckjanwi...@yahoo.com wrote: My current idea for a fix is to modify lazy_truncate_heap(). It does acquire and release the exclusive lock, so it should be possible to do this in smaller chunks, releasing and

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 3:17 PM, Robert Haas wrote: On Mar 26, 2011, at 1:44 PM, Itagaki Takahiroitagaki.takah...@gmail.com wrote: On Sun, Mar 27, 2011 at 01:12, Simon Riggssi...@2ndquadrant.com wrote: At the same time I would change count_nondeletable_pages() so that it uses a forward scan