Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Robert Haas
On Fri, Nov 9, 2012 at 10:08 AM, Alvaro Herrera wrote: > Tom Lane escribió: >> Robert Haas writes: > >> > IIRC the queue has 4K entries, and IIRC a single DDL >> > operation might provoke a couple of sinvals, but I'm thinking that >> > somebody would probably have to be creating >1024 temp tables

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Alvaro Herrera
Tom Lane escribió: > Robert Haas writes: > > IIRC the queue has 4K entries, and IIRC a single DDL > > operation might provoke a couple of sinvals, but I'm thinking that > > somebody would probably have to be creating >1024 temp tables a minute > > to overrun the queue, which is very possible but

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Tom Lane
Robert Haas writes: > So, do we need a sinval overrun or just a sinval message to provoke > starvation? The former would be bad but the latter would be really, > really bad. The former. Actually, a signal will be sent as soon as a backend is determined to be unreasonably far behind, which I thi

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Kevin Grittner
Simon Riggs wrote: > Robert Haas wrote: >> One of us is confused, because IIUC Tom just fixed this this >> morning, and I'm trying to figure out how many users will be >> affected by it, and how seriously. Like, do we need an immediate >> minor release? > > You asked what provokes starvation, an

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Simon Riggs
On 9 November 2012 14:16, Robert Haas wrote: > On Fri, Nov 9, 2012 at 9:02 AM, Simon Riggs wrote: >>> So, do we need a sinval overrun or just a sinval message to provoke >>> starvation? The former would be bad but the latter would be really, >>> really bad. IIRC the queue has 4K entries, and II

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Robert Haas
On Fri, Nov 9, 2012 at 9:02 AM, Simon Riggs wrote: >> So, do we need a sinval overrun or just a sinval message to provoke >> starvation? The former would be bad but the latter would be really, >> really bad. IIRC the queue has 4K entries, and IIRC a single DDL >> operation might provoke a couple

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Simon Riggs
On 9 November 2012 13:42, Robert Haas wrote: > On Thu, Nov 8, 2012 at 6:23 PM, Tom Lane wrote: >> Simon Riggs writes: >>> On 8 November 2012 20:36, Jeff Janes wrote: It does not seem outrageous to me that there would be real-world conditions in which invalidations would be sent more t

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Robert Haas
On Thu, Nov 8, 2012 at 6:23 PM, Tom Lane wrote: > Simon Riggs writes: >> On 8 November 2012 20:36, Jeff Janes wrote: >>> It does not seem outrageous to me that there would be real-world >>> conditions in which invalidations would be sent more than once a >>> minute over prolonged periods, so thi

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-09 Thread Simon Riggs
On 8 November 2012 23:58, Jeff Janes wrote: > On Thu, Nov 8, 2012 at 2:50 PM, Hannu Krosing wrote: >> On 11/08/2012 11:40 PM, Simon Riggs wrote: >>> >>> On 8 November 2012 20:36, Jeff Janes wrote: >>> It does not seem outrageous to me that there would be real-world conditions in which

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Tom Lane
I wrote: >> Hmm. That seems like a problem, all right, but I don't believe the >> claim that it applies only to the poll() code path. On many platforms >> the select() path would have the same issue. > BTW, doesn't win32_latch.c have the identical problem? I don't see it > updating the timeout

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Jeff Janes
On Thu, Nov 8, 2012 at 2:50 PM, Hannu Krosing wrote: > On 11/08/2012 11:40 PM, Simon Riggs wrote: >> >> On 8 November 2012 20:36, Jeff Janes wrote: >> >>> It does not seem outrageous to me that there would be real-world >>> conditions in which invalidations would be sent more than once a >>> minu

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Tom Lane
Simon Riggs writes: > On 8 November 2012 20:36, Jeff Janes wrote: >> It does not seem outrageous to me that there would be real-world >> conditions in which invalidations would be sent more than once a >> minute over prolonged periods, so this total starvation seems like a >> bug. > Yes, its a b

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Hannu Krosing
On 11/08/2012 11:40 PM, Simon Riggs wrote: On 8 November 2012 20:36, Jeff Janes wrote: It does not seem outrageous to me that there would be real-world conditions in which invalidations would be sent more than once a minute over prolonged periods, so this total starvation seems like a bug. Ye

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Andres Freund
On 2012-11-08 22:40:43 +, Simon Riggs wrote: > On 8 November 2012 20:36, Jeff Janes wrote: > > > It does not seem outrageous to me that there would be real-world > > conditions in which invalidations would be sent more than once a > > minute over prolonged periods, so this total starvation see

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Simon Riggs
On 8 November 2012 20:36, Jeff Janes wrote: > It does not seem outrageous to me that there would be real-world > conditions in which invalidations would be sent more than once a > minute over prolonged periods, so this total starvation seems like a > bug. Yes, its a bug, but do you really believ

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Tom Lane
I wrote: > Jeff Janes writes: >> When the poll() gets EINTR by SIGUSR1 for the invalidation, it >> restarts at the full timeout (60 seconds in this case) with no >> accounting for the time already waited. > Hmm. That seems like a problem, all right, but I don't believe the > claim that it applie

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Tom Lane
Jeff Janes writes: > On Thu, Nov 8, 2012 at 12:36 PM, Jeff Janes wrote: >> Should gettimeofday be called before and after the poll() and then the >> difference deducted from timeout? > Something like this? Meh. Not like that, because (1) we shouldn't add overhead when no timeout is requested.

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Tom Lane
Jeff Janes writes: > Basically the WaitLatch in autovacuum.c line 602 (in head) never returns. > This was introduced by a180776f7a1c4554f214b, "Teach unix_latch.c to > use poll() where available" > When the poll() gets EINTR by SIGUSR1 for the invalidation, it > restarts at the full timeout (60

Re: [HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Jeff Janes
On Thu, Nov 8, 2012 at 12:36 PM, Jeff Janes wrote: > > Should gettimeofday be called before and after the poll() and then the > difference deducted from timeout? > Something like this? poll_timeout.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

[HACKERS] AutoVacuum starvation from sinval messages

2012-11-08 Thread Jeff Janes
While doing a rather contrived test of something else, using: perl -le 'print "create table foo (x serial); drop table foo;" foreach (1..1e6)'|psql I noticed starvation of autovacuum launcher process and bloating of catalogs. Basically the WaitLatch in autovacuum.c line 602 (in head) never retu