Re: BUG #14941: Vacuum crashes

2018-04-06 Thread Bossart, Nathan
On 4/5/18, 9:48 PM, "Michael Paquier" wrote: > On Fri, Apr 06, 2018 at 06:41:14AM +0900, Michael Paquier wrote: >> That would be wiser. We are two days away from the end of the CF and >> this patch gets quite invasive with a set of new concepts, so my >> recommendation would be to do so. There h

Re: BUG #14941: Vacuum crashes

2018-04-05 Thread Michael Paquier
On Fri, Apr 06, 2018 at 06:41:14AM +0900, Michael Paquier wrote: > That would be wiser. We are two days away from the end of the CF and > this patch gets quite invasive with a set of new concepts, so my > recommendation would be to do so. There has been already much done with > the introduction o

Re: BUG #14941: Vacuum crashes

2018-04-05 Thread Michael Paquier
On Thu, Apr 05, 2018 at 08:29:38PM +, Bossart, Nathan wrote: > The new tests are now passing as expected, but I am still doing some > manual testing. Since there is quite a bit of added complexity in > this new patch set, I will certainly not be surprised if this gets > moved to the next commi

Re: BUG #14941: Vacuum crashes

2018-04-02 Thread Michael Paquier
On Fri, Mar 30, 2018 at 05:19:46PM -0700, Andres Freund wrote: > On 2018-03-30 18:39:01 +, Bossart, Nathan wrote: >> I think the most straightforward approach for fixing this is to add >> skip-locked functionality in find_all_inheritors(), >> find_inheritance_children(), and vac_open_indexes(),

Re: BUG #14941: Vacuum crashes

2018-03-30 Thread Andres Freund
Hi, On 2018-03-30 18:39:01 +, Bossart, Nathan wrote: > I've noticed one more problem with ACCESS EXCLUSIVE. If an ACCESS > EXCLUSIVE lock is held on a child relation of a partitioned table, > an ANALYZE on the partitioned table will be blocked at > acquire_inherited_sample_rows(). > > static

Re: BUG #14941: Vacuum crashes

2018-03-30 Thread Bossart, Nathan
On 3/7/18, 9:34 AM, "Bossart, Nathan" wrote: > On 3/6/18, 11:04 PM, "Michael Paquier" wrote: >> + if (!(options & VACOPT_SKIP_LOCKED)) >> + relid = RangeVarGetRelid(vrel->relation, AccessShareLock, >> false); >> + else >> + { >> + relid = RangeVarGetRelid(vre

Re: BUG #14941: Vacuum crashes

2018-03-07 Thread Bossart, Nathan
On 3/6/18, 11:04 PM, "Michael Paquier" wrote: > + if (!(options & VACOPT_SKIP_LOCKED)) > + relid = RangeVarGetRelid(vrel->relation, AccessShareLock, > false); > + else > + { > + relid = RangeVarGetRelid(vrel->relation, NoLock, false); > Yeah, I agree with Andr

Re: BUG #14941: Vacuum crashes

2018-03-06 Thread Michael Paquier
On Mon, Mar 05, 2018 at 09:55:13PM +, Bossart, Nathan wrote: > On 3/3/18, 6:13 PM, "Andres Freund" wrote: >> I was working on committing 0002 and 0003, when I noticed that the >> second patch doesn't actually fully works. NOWAIT does what it says on >> the tin iff the table is locked with a l

Re: BUG #14941: Vacuum crashes

2018-03-06 Thread Bossart, Nathan
On 3/5/18, 7:58 PM, "Andres Freund" wrote: > I've pushed the first one now. There seems to be no reason to wait with > it, even it takes a bit till we can get the second part right. Thanks! Nathan

Re: BUG #14941: Vacuum crashes

2018-03-05 Thread Andres Freund
On 2018-03-03 16:12:52 -0800, Andres Freund wrote: > On 2018-01-11 08:14:42 +0900, Michael Paquier wrote: > > On Wed, Jan 10, 2018 at 05:26:43PM +, Bossart, Nathan wrote: > > > Perhaps any logging changes for VACOPT_NOWAIT could be handled in a > > > separate thread. For now, I've updated 0003

Re: BUG #14941: Vacuum crashes

2018-03-03 Thread Andres Freund
On 2018-01-11 08:14:42 +0900, Michael Paquier wrote: > On Wed, Jan 10, 2018 at 05:26:43PM +, Bossart, Nathan wrote: > > Perhaps any logging changes for VACOPT_NOWAIT could be handled in a > > separate thread. For now, I've updated 0003 to remove the logging > > changes. > > Thanks. I am marki

Re: BUG #14941: Vacuum crashes

2018-01-10 Thread Masahiko Sawada
On Thu, Jan 11, 2018 at 8:14 AM, Michael Paquier wrote: > On Wed, Jan 10, 2018 at 05:26:43PM +, Bossart, Nathan wrote: >> Right. I don't have a terribly strong opinion either way. I think the >> counter-argument is that logging skipped relations might provide >> valuable feedback to users.

Re: BUG #14941: Vacuum crashes

2018-01-10 Thread Michael Paquier
On Wed, Jan 10, 2018 at 05:26:43PM +, Bossart, Nathan wrote: > Right. I don't have a terribly strong opinion either way. I think the > counter-argument is that logging skipped relations might provide > valuable feedback to users. If I ran a database-wide VACUUM and a > relation was skipped d

Re: BUG #14941: Vacuum crashes

2018-01-10 Thread Bossart, Nathan
On 1/9/18, 8:55 PM, "Michael Paquier" wrote: > On Tue, Jan 09, 2018 at 01:46:55PM -0800, Andres Freund wrote: >> On 2018-01-09 10:23:12 -0500, Robert Haas wrote: >> > On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier >> > wrote: >> > > On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrot

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Michael Paquier
On Tue, Jan 09, 2018 at 01:46:55PM -0800, Andres Freund wrote: > On 2018-01-09 10:23:12 -0500, Robert Haas wrote: > > On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier > > wrote: > > > On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrote: > > >> I agree, this makes more sense. I've made

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Michael Paquier
On Tue, Jan 09, 2018 at 09:40:50PM +, Bossart, Nathan wrote: > On 1/8/18, 10:28 PM, "Michael Paquier" wrote: >> I think that you are doing it wrong here. In get_all_vacuum_rels() you >> should build a RangeVar to be reused in the context of this error >> message, and hence you'll save an extra

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Andres Freund
On 2018-01-09 10:23:12 -0500, Robert Haas wrote: > On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier > wrote: > > On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrote: > >> I agree, this makes more sense. I've made this change in v3 of 0003. > > > > Based on the opinions gathered on thi

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Bossart, Nathan
On 1/9/18, 9:24 AM, "Robert Haas" wrote: > On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier > wrote: >> On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrote: >>> I agree, this makes more sense. I've made this change in v3 of 0003. >> >> Based on the opinions gathered on this thread, 0

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Bossart, Nathan
On 1/8/18, 10:28 PM, "Michael Paquier" wrote: > Based on the opinions gathered on this thread, 0001 and 0002 seem to be > in the shape wanted, and those look good for shipping. Now for 0003 we > are not there yet. Thanks for taking a look. > - if (relation == NULL) > + if (relation !

Re: BUG #14941: Vacuum crashes

2018-01-09 Thread Robert Haas
On Mon, Jan 8, 2018 at 11:27 PM, Michael Paquier wrote: > On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrote: >> I agree, this makes more sense. I've made this change in v3 of 0003. > > Based on the opinions gathered on this thread, 0001 and 0002 seem to be > in the shape wanted, and

Re: BUG #14941: Vacuum crashes

2018-01-08 Thread Michael Paquier
On Thu, Dec 28, 2017 at 10:46:18PM +, Bossart, Nathan wrote: > I agree, this makes more sense. I've made this change in v3 of 0003. Based on the opinions gathered on this thread, 0001 and 0002 seem to be in the shape wanted, and those look good for shipping. Now for 0003 we are not there yet.

Re: BUG #14941: Vacuum crashes

2017-12-21 Thread Masahiko Sawada
On Tue, Dec 19, 2017 at 7:18 AM, Bossart, Nathan wrote: > On 12/18/17, 3:30 PM, "Masahiko Sawada" wrote: >> According to the following old comment, there might be reason why we >> didn't pass the information to vacuum_rel(). But your patch fetches >> the relation >> name even if the "relation" is

Re: BUG #14941: Vacuum crashes

2017-12-18 Thread Bossart, Nathan
On 12/18/17, 3:30 PM, "Masahiko Sawada" wrote: > According to the following old comment, there might be reason why we > didn't pass the information to vacuum_rel(). But your patch fetches > the relation > name even if the "relation" is not provided. I wonder if it can be > problem in some cases.

Re: BUG #14941: Vacuum crashes

2017-12-18 Thread Masahiko Sawada
On Tue, Dec 19, 2017 at 2:35 AM, Bossart, Nathan wrote: > On 12/11/17, 9:41 PM, "Masahiko Sawada" wrote: >> For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't >> get the above WARNING messages, but I think we should get them. The >> reported issue also did VACUUM FULL and REINDEX

Re: BUG #14941: Vacuum crashes

2017-12-14 Thread Bossart, Nathan
On 12/11/17, 9:41 PM, "Masahiko Sawada" wrote: > I also reviewed the patches. Thanks for taking a look. > For 0003 patch, if we do VACUUM (NOWAIT) to whole database we don't > get the above WARNING messages, but I think we should get them. The > reported issue also did VACUUM FULL and REINDEX to

Re: BUG #14941: Vacuum crashes

2017-12-14 Thread Bossart, Nathan
On 12/10/17, 11:13 PM, "Michael Paquier" wrote: > Let's see what other folks think first about the ANALYZE grammar in > VACUUM, as well as having ANALYZE use a parenthesized grammer. FWIW, I > like the contents of this patch set, and the thing is non-intrusive. I > think that NOWAIT gains more val

Re: BUG #14941: Vacuum crashes

2017-12-11 Thread Masahiko Sawada
On Mon, Dec 11, 2017 at 2:13 PM, Michael Paquier wrote: > On Wed, Dec 6, 2017 at 1:52 AM, Bossart, Nathan wrote: >> 0001_fix_unparenthesized_vacuum_grammar_v1.patch >> >> One VacuumStmt grammar rule allows users to specify the VACOPT_ANALYZE >> option by including an AnalyzeStmt at the end of the

Re: BUG #14941: Vacuum crashes

2017-12-10 Thread Michael Paquier
On Wed, Dec 6, 2017 at 1:52 AM, Bossart, Nathan wrote: > 0001_fix_unparenthesized_vacuum_grammar_v1.patch > > One VacuumStmt grammar rule allows users to specify the VACOPT_ANALYZE > option by including an AnalyzeStmt at the end of the command. This > appears to have been added as part of the int

Re: BUG #14941: Vacuum crashes

2017-12-10 Thread Michael Paquier
On Sat, Dec 2, 2017 at 6:16 AM, Bossart, Nathan wrote: > On 12/1/17, 3:04 PM, "Robert Haas" wrote: >> Seems entirely reasonable to me, provided that we only update the >> extensible-options syntax: >> >> VACUUM [ ( option [, ...] ) ] [ table_and_columns [, ...] ] >> >> I don't want to add any mor

Re: BUG #14941: Vacuum crashes

2017-12-04 Thread Robert Haas
On Mon, Dec 4, 2017 at 8:59 AM, Lyes Ameddah wrote: > sorry guys, yes I'm talking about a FULL VACUUM and not about Auto-Vacuum. > Thank you very match for your feedback. OK, but that's not the confusion. What you said is that it CRASHES, but the behavior you described is that it BLOCKS waiting

Re: BUG #14941: Vacuum crashes

2017-12-04 Thread Lyes Ameddah
sorry guys, yes I'm talking about a FULL VACUUM and not about Auto-Vacuum. Thank you very match for your feedback. That's Waht I do : vacuum FULL VERBOSE ANALYZE; reindex database postgres; I would be happy if there is a patch to fix that ! 2017-12-01 22:16 GMT+01:00 Bossart, Nathan : > On 12/1

Re: BUG #14941: Vacuum crashes

2017-12-01 Thread Bossart, Nathan
On 12/1/17, 3:04 PM, "Robert Haas" wrote: > On Fri, Dec 1, 2017 at 1:53 PM, Bossart, Nathan wrote: >> There is already an internal flag called VACOPT_NOWAIT that gives >> autovacuum this ability in certain cases (introduced in the aforementioned >> commit), and I recently brought up this potentia

Re: BUG #14941: Vacuum crashes

2017-12-01 Thread Robert Haas
On Fri, Dec 1, 2017 at 1:53 PM, Bossart, Nathan wrote: > There is already an internal flag called VACOPT_NOWAIT that gives > autovacuum this ability in certain cases (introduced in the aforementioned > commit), and I recently brought up this potential use-case as > justification for a patch [0].

Re: BUG #14941: Vacuum crashes

2017-12-01 Thread Bossart, Nathan
On 12/1/17, 11:16 AM, "Tomas Vondra" wrote: >> The behavior I would like to see is that the void ignores this table and >> moves to another instead of being blocked. >> > > I believe autovacuum should not block waiting for a heavy-weight lock on > a table since this commit that went into 9.1: >