Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Attached patch seems to solve the problem on my machine at least. Uses pgwin32_waitforsinglesocket() instead of pgwin32_select(). Changes pgwin32_waitforsinglesocket() to accept the timeout as a parameter (this is why it touches files

Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Magnus Hagander
Jeremy Haile wrote: Using standard build (none of the things you mentioned) on 8.2.1 currently. I really appreciate it! Ok. I've built a binary for you at http://www.hagander.net/download/pgsql_8.2.1_win32select.zip Note that while I named the file 8.2.1 it really isn't - it's current head

Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Jeremy Haile
Will do - thanks Magnus! I'll test it for a while and post the results here. Jeremy Haile On Sat, 27 Jan 2007 17:21:23 +0100, Magnus Hagander [EMAIL PROTECTED] said: Jeremy Haile wrote: Using standard build (none of the things you mentioned) on 8.2.1 currently. I really appreciate

Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Jeremy Haile
So far, the patched binary seems to have fixed the statistics issue in my environment. pgstat.stat is now updating constantly, pg_stat_user_tables is updating correctly, and autovacuum is running! Thanks for your quick response to this issue! Let me know if there is anything else I can do to

Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Michael Meskes
On Sat, Jan 27, 2007 at 03:17:14PM +0100, Magnus Hagander wrote: Seems HEAD now passes thie check stage on at least snake and bandicoot. They're now exposing another problem with ECPG though... This should be fixed now. I just missed to change MinGW expected files when I changed the tests.

Re: [GENERAL] Stats collector frozen?

2007-01-27 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: On Fri, Jan 26, 2007 at 09:55:39AM -0500, Tom Lane wrote: Keep in mind also that we have seen the stats-test failure on non-Windows machines, so we still need to explain that ... Yeah. But it *could* be two different stats issues lurking. Perhaps the

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Thu, Jan 25, 2007 at 04:29:58PM -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Jeremy Haile wrote: If anyone else is experiencing similar problems, please post your situation. All the Windows buildfarm machines are, apparently. Can't anyone with a debugger

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Thu, Jan 25, 2007 at 04:29:58PM -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Jeremy Haile wrote: If anyone else is experiencing similar problems, please post your situation. All the Windows buildfarm machines are, apparently. Can't anyone with a debugger

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Thu, Jan 25, 2007 at 04:29:58PM -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Jeremy Haile wrote: If anyone else is experiencing similar problems, please post your situation. All the Windows buildfarm machines are, apparently. Can't anyone with a debugger

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Teodor Sigaev
Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass. May be, problem is related to fixed bug in pgwin32_waitforsinglesocket() ? WaitForMultipleObjectsEx might sleep indefinitely while

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Fri, Jan 26, 2007 at 01:11:00PM +0300, Teodor Sigaev wrote: Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass. May be, problem is related to fixed bug in

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Fri, Jan 26, 2007 at 12:26:58PM +0100, Magnus Hagander wrote: On Fri, Jan 26, 2007 at 01:11:00PM +0300, Teodor Sigaev wrote: Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass.

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Jeremy Haile
Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass. May be, problem is related to fixed bug in pgwin32_waitforsinglesocket() ? WaitForMultipleObjectsEx might sleep

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Alvaro Herrera
Jeremy Haile wrote: Magnus - thanks for your investigation and work! Any chance I could get a patched exe for win32 and test on my servers? In the meantime, could you please confirm that turning row_level_stats off makes the regression test pass? And whatever tests you were doing? (Note that

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass. I guess the bug is shown because with row level stats we simply have more data to

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
On Fri, Jan 26, 2007 at 09:55:39AM -0500, Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Apparantly there is a bug lurking somewhere in pgwin32_select(). Because if I put a #undef select right before the select in pgstat.c, the regression tests pass. I guess the bug is shown

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Jeremy Haile
We have had lots of reports of issues with the stats collector on Windows. Some were definitly fixed by the patch by OT, but I don't think all. Here were a couple of other reports I found: http://archives.postgresql.org/pgsql-hackers/2006-09/msg00415.php

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Attached patch seems to solve the problem on my machine at least. Uses pgwin32_waitforsinglesocket() instead of pgwin32_select(). Changes pgwin32_waitforsinglesocket() to accept the timeout as a parameter (this is why it touches files outside of the

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Jeremy Haile
Applied to HEAD and 8.2 --- assuming the Windows buildfarm machines go green, we should probably consider back-porting this to 8.1 and 8.0. Not trying to be a nuisance, but I'd really like to try this out in my environment and see if my problems disappear. Is there anyone out there who could

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Magnus Hagander
Jeremy Haile wrote: Applied to HEAD and 8.2 --- assuming the Windows buildfarm machines go green, we should probably consider back-porting this to 8.1 and 8.0. Not trying to be a nuisance, but I'd really like to try this out in my environment and see if my problems disappear. Is there

Re: [GENERAL] Stats collector frozen?

2007-01-26 Thread Jeremy Haile
Using standard build (none of the things you mentioned) on 8.2.1 currently. I really appreciate it! On Fri, 26 Jan 2007 21:24:09 +0100, Magnus Hagander [EMAIL PROTECTED] said: Jeremy Haile wrote: Applied to HEAD and 8.2 --- assuming the Windows buildfarm machines go green, we should

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Jeremy Haile
Tom, Did this information shed any light on what the problem might be? Any solution or workaround? Thanks! Jeremy Haile On Wed, 24 Jan 2007 14:19:05 -0500, Jeremy Haile [EMAIL PROTECTED] said: pgstat.stat was last updated 1/22 12:25pm - there is no pgstat.tmp. Coincidentally (I think

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Tom Lane
Jeremy Haile [EMAIL PROTECTED] writes: Did this information shed any light on what the problem might be? It seems to buttress Magnus' theory that the intermittent (or not so intermittent) stats-test buildfarm failures we've been seeing have to do with the stats collector actually freezing up,

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Jeremy Haile
Unfortunately I don't have any debugging tools installed that would work against postgres - although I'd be glad to do something if you could tell me the steps involved. I can reproduce the issue quite easily on two different Windows machines (one is XP, the other is 2003). Please let me know if

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Tom Lane
Jeremy Haile [EMAIL PROTECTED] writes: Unfortunately I don't have any debugging tools installed that would work against postgres - although I'd be glad to do something if you could tell me the steps involved. I can reproduce the issue quite easily on two different Windows machines (one is XP,

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Magnus Hagander
Tom Lane wrote: Jeremy Haile [EMAIL PROTECTED] writes: Unfortunately I don't have any debugging tools installed that would work against postgres - although I'd be glad to do something if you could tell me the steps involved. I can reproduce the issue quite easily on two different Windows

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Jeremy Haile [EMAIL PROTECTED] writes: Do you know of any workaround other than restarting the whole server? Can the collector be restarted individually? You can use pg_ctl to send the int signal. If it's completely hung, that may not work. In that

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Jeremy Haile
Then just pick it up in Task Manager or Process Explorer or whatever and kill it off. Just make sure you pick the right process. I mentioned earlier that killing off the collector didn't work - however I was wrong. I just wasn't giving it enough time. If I kill the postgres.exe -forkcol

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Jeremy Haile
I'll try to put together a test case for hackers, although I'm not sure what exactly causes it. Basically, when I fire up PostgreSQL - after about a minute the stats collector runs once (pgstat.stat is updated, autovacuum fires up, etc.) - and then the collector seems to hang. If I watch it's

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Alvaro Herrera
Jeremy Haile wrote: I'll try to put together a test case for hackers, although I'm not sure what exactly causes it. Basically, when I fire up PostgreSQL - after about a minute the stats collector runs once (pgstat.stat is updated, autovacuum fires up, etc.) - and then the collector seems

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Jeremy Haile wrote: If anyone else is experiencing similar problems, please post your situation. All the Windows buildfarm machines are, apparently. Can't anyone with a debugger duplicate this and get a stack trace for us? If the stats collector is

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Jeremy Haile
AFAIR (Magnus can surely confirm) there were some other tables that weren't showing stats as all zeros -- but there's no way to know whether those numbers were put there before the collector had frozen (if that's really what's happening). Yeah - I have numbers that updated before the stats

Re: [GENERAL] Stats collector frozen?

2007-01-25 Thread Magnus Hagander
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Jeremy Haile wrote: If anyone else is experiencing similar problems, please post your situation. All the Windows buildfarm machines are, apparently. Can't anyone with a debugger duplicate this and get a stack trace for us? If the

Re: [GENERAL] Stats collector frozen?

2007-01-24 Thread Tom Lane
Jeremy Haile [EMAIL PROTECTED] writes: The stats collector appears to still be running, since I can see a postgres.exe process with -forkcol. However, I never notice it using I/O or CPU usage. Also, querying the pg_stat_user_tables view shows no change in table stats even for tables that

Re: [GENERAL] Stats collector frozen?

2007-01-24 Thread Jeremy Haile
pgstat.stat was last updated 1/22 12:25pm - there is no pgstat.tmp. Coincidentally (I think not) - the last auto-analyze was performed at 2007-01-22 12:24:11.424-05. The logs for 1/22 are empty - so no errors or anything like that to give clues... Thanks! Jeremy Haile On Wed, 24 Jan 2007

Re: [GENERAL] Stats collector frozen?

2007-01-24 Thread Jeremy Haile
Searching the archives, I found a couple of 2006 posts that seem somewhat related to my problem (although I don't see any solutions listed...): http://archives.postgresql.org/pgsql-hackers/2006-09/msg00415.php http://archives.postgresql.org/pgsql-hackers/2006-04/msg00127.php Tom, since you were