Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: I may have found the problem; all the hung processes show 'async_notify waiting' in ps, and the ANALYZE eventually dies with 'tuple concurrently updated'. The routine 'ProcessIncomingNotify' in async.c does indeed try to lock pg_listener (even if

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Alvaro Herrera
On Mon, May 03, 2004 at 02:14:18PM +1000, Gavin Sherry wrote: It is implemented using shared memory. I got stuck when I considered the situation where we rung out of shared memory. Some emails in the archive suggested we just fire all listeners but I didn't like that. Can this be kept in

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: At 11:04 PM 3/05/2004, Tom Lane wrote: How confident are you in those processes? I don't know of any other mechanism for 'tuple concurrently updated' failures in ANALYZE than concurrent analyze runs ... Fairly. In this particular instance the error

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Magnus Hagander
I'm not sure how would one send to the other backends. Maybe write another file on disk, one for each remote backend? Surely this can be done somehow. I've heard that on linux-2.6 they are implementing POSIX message queues (not sure what those are anyway); maybe we can do that on

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Gavin Sherry
On Mon, 3 May 2004, Alvaro Herrera wrote: On Mon, May 03, 2004 at 02:14:18PM +1000, Gavin Sherry wrote: It is implemented using shared memory. I got stuck when I considered the situation where we rung out of shared memory. Some emails in the archive suggested we just fire all listeners

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Tom Lane
Zeugswetter Andreas SB SD [EMAIL PROTECTED] writes: * Is it really a good idea for database-wide ANALYZE to run as a single transaction? Holding all those locks is a recipe for deadlocks, even if they're as inoffensive as AccessShareLocks normally are. Wasn't one idea behind that change also

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Tom Lane
I wrote: 2. As the ANALYZE proceeds, it issues sinval messages due to the updates it's making in pg_statistic. This is normal. Small correction: actually, backends only send sinval messages at commit, so the ANALYZE will just be accumulating pending messages in its private memory. Your

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-03 Thread Zeugswetter Andreas SB SD
* Is it really a good idea for database-wide ANALYZE to run as a single transaction? Holding all those locks is a recipe for deadlocks, even if they're as inoffensive as AccessShareLocks normally are. Wasn't one idea behind that change also to not make the planner create a plan from mixed

[HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Philip Warner
We are seeing occasional long lockouts from out DB. When I do a 'select * from pg_locks', I find that everybody is waiting for pg_listener, and that the lock on pg_listener is currently held by a long-running ANALYZE VERBOSE. I saw the following in the change logs (not sure if it's relevant):

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: Does this mean that ANALYZE will take an exclusive lock on pg_listener until the ANALYZE finishes? Or is there some other cause? ANALYZE does not take an exclusive lock on anything. However, the async.c functions want AccessExclusiveLock on pg_listener,

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Bruce Momjian
Tom Lane wrote: Philip Warner [EMAIL PROTECTED] writes: Does this mean that ANALYZE will take an exclusive lock on pg_listener until the ANALYZE finishes? Or is there some other cause? ANALYZE does not take an exclusive lock on anything. However, the async.c functions want

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: Would ACCESS SHARE be OK? Certainly not, since the point of the locks in async.c is that only one backend should execute those routines at a time. ExclusiveLock might work okay ... but I still haven't thought hard about it ...

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: If possible, this seems like a great option. We currently have a large database with several hundred users who get locked out for as much as half an hour while ANALYZE runs. If it takes half an hour to ANALYZE pg_listener, I think that ANALYZE is not

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Gavin Sherry
On Sun, 2 May 2004, Bruce Momjian wrote: Tom Lane wrote: Philip Warner [EMAIL PROTECTED] writes: Does this mean that ANALYZE will take an exclusive lock on pg_listener until the ANALYZE finishes? Or is there some other cause? ANALYZE does not take an exclusive lock on anything.

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: ... for some reason pg_listeners is being locked in ACCESS SHARE the entire time. Just vacuuming pg_listener produces: vacuum verbose pg_listener; INFO: vacuuming pg_catalog.pg_listener INFO: pg_listener: found 0 removable, 0 nonremovable row

Re: [HACKERS] ANALYZE locks pg_listener in EXCLUSIVE for long time?

2004-05-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: At 02:21 PM 3/05/2004, Tom Lane wrote: [blinks...] There's something pretty strange about that. Are you using LISTEN/NOTIFY at all? Nope. In that case there's no reason for anything to be taking any particular locks on pg_listener; and it's simply