Re: RESOLVED: Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Dirk Lutzebäck
Josh, I cannot reproduce the excessive semop() on a Dual XEON DP on a non-bigmem kernel, HT on. Interesting to know if the problem is related to XEON MP (as Tom wrote) or bigmem. Josh Berkus wrote: Dirk, I'm not sure if this semop() problem is still an issue but the database behaves a bit

[PERFORM] very slow simple query - outer join makes it quicker

2004-04-19 Thread Gerard Isdell
Hi Everyone I am new to this group and relatively new to Postgres, having used MSSQL 7 up until now. One table in my database in returning even the simplest of queries extremely slowly. The table is user table, and even the select userid from users takes over 20 seconds to run. There are abou

Re: [PERFORM] Horribly slow hash join

2004-04-19 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Greg Stark <[EMAIL PROTECTED]> writes: > > If the hash tables were made a power of two then it would be possible to mix > > the bits of the 32 bit value and just mask off the unneeded bits. I've found > > one page via google that mentions mixing bits in a has

Re: [PERFORM] very slow simple query - outer join makes it quicker

2004-04-19 Thread Rod Taylor
> There are other tables, such as the messages table, that have 10s of > thousands of rows and they return records much more quickly. > There must be something seriously wrong for simple queries like this to take > so long. Have you run VACUUM recently? If not, run VACUUM FULL against the u

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Dave Cramer
Here's an interesting link that suggests that hyperthreading would be much worse. http://groups.google.com/groups?q=hyperthreading+dual+xeon+idle&start=10&hl=en&lr=&ie=UTF-8&c2coff=1&selm=aukkonen-FE5275.21093624062003%40shawnews.gv.shawcable.net&rnum=16 another which has some hints as to how it

Re: [PERFORM] Horribly slow hash join

2004-04-19 Thread Dave Cramer
Here's an interesting link that suggests that hyperthreading would be much worse. http://groups.google.com/groups?q=hyperthreading+dual+xeon+idle&start=10&hl=en&lr=&ie=UTF-8&c2coff=1&selm=aukkonen-FE5275.21093624062003%40shawnews.gv.shawcable.net&rnum=16 FWIW, I have anecdotal evidence that sugge

Re: [PERFORM] Horribly slow hash join

2004-04-19 Thread Greg Stark
Dave Cramer <[EMAIL PROTECTED]> writes: > Here's an interesting link that suggests that hyperthreading would be > much worse. Uh, this is the wrong thread. -- greg ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL

Re: [PERFORM] very slow simple query - outer join makes it quicker

2004-04-19 Thread Rod Taylor
On Mon, 2004-04-19 at 08:26, Gerard Isdell wrote: > Thank, that has worked. > > I've been running VACUUM regularly and thought that would have done it. > > Obviously the FULL makes a big difference It shouldn't. That FULL makes a significant difference says that you're not running regular VACU

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Anjan Dave
What about quad-XEON setups? Could that be worse? (have dual, and quad setups both) Shall we re-consider XEON-MP CPU machines with high cache (4MB+)? Very generally, what number would be considered high, especially, if it coincides with expected heavy load? Not sure a specific chipset was men

Re: [PERFORM] query slows down with more accurate stats

2004-04-19 Thread Tom Lane
Manfred Koizar <[EMAIL PROTECTED]> writes: > Random sampling is more like "every possible sample is equally likely to > be collected", and two-stage sampling doesn't satisfy this condition. Okay, I finally see the point here: in the limit as the number of pages B goes to infinity, you'd expect the

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread J. Andrew Rogers
I decided to check the context-switching behavior here for baseline since we have a rather diverse set of postgres server hardware, though nothing using Xeon MP that is also running a postgres instance, and everything looks normal under load. Some platforms are better than others, but nothing is

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > The other thing I'd like your comment on, Tom, is that Dirk appears to have > reported that when he installed a non-bigmem kernel, the issue went away. > Dirk, is this correct? I'd be really surprised if that had anything to do with it. AFAIR Dirk's t

Re: [PERFORM] possible improvement between G4 and G5

2004-04-19 Thread Vivek Khera
> "JB" == Josh Berkus <[EMAIL PROTECTED]> writes: JB> Aaron, >> I do consulting, so they're all over the place and tend to be complex. Very >> few fit in RAM, but still are very buffered. These are almost all backed >> with very high end I/O subsystems, with dozens of spindles with battery >>

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Bruce Momjian
Josh Berkus wrote: > Tom, > > > So in the short term I think we have to tell people that Xeon MP is not > > the most desirable SMP platform to run Postgres on. (Josh thinks that > > the specific motherboard chipset being used in these machines might > > share some of the blame too. I don't have

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Josh Berkus
Tom, > So in the short term I think we have to tell people that Xeon MP is not > the most desirable SMP platform to run Postgres on. (Josh thinks that > the specific motherboard chipset being used in these machines might > share some of the blame too. I don't have any evidence for or against > t

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread scott.marlowe
On Mon, 19 Apr 2004, Bruce Momjian wrote: > Josh Berkus wrote: > > Tom, > > > > > So in the short term I think we have to tell people that Xeon MP is not > > > the most desirable SMP platform to run Postgres on. (Josh thinks that > > > the specific motherboard chipset being used in these machine

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Joe Conway
scott.marlowe wrote: On Mon, 19 Apr 2004, Bruce Momjian wrote: I have BSD on a SuperMicro dual Xeon, so if folks want another hardware/OS combination to test, I can give out logins to my machine. I can probably do some nighttime testing on a dual 2800MHz non-MP Xeon machine as well. It's a Dell 2

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Josh Berkus
Joe, > I've got a quad 2.8Ghz MP Xeon (IBM x445) that I could test on. Does > anyone have a test set that can reliably reproduce the problem? Unfortunately we can't seem to come up with one.So far we have 2 machines that exhibit the issue, and their databases are highly confidential (State

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: >> I've got a quad 2.8Ghz MP Xeon (IBM x445) that I could test on. Does >> anyone have a test set that can reliably reproduce the problem? > Unfortunately we can't seem to come up with one. > It does seem to require a database which is in the many GB (> 10

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Tom Lane
Here is a test case. To set up, run the "test_setup.sql" script once; then launch two copies of the "test_run.sql" script. (For those of you with more than two CPUs, see whether you need one per CPU to make trouble, or whether two test_runs are enough.) Check that you get a nestloops-with-index-

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Tom Lane
I wrote: > Here is a test case. Hmmm ... I've been able to reproduce the CS storm on a dual Athlon, which seems to pretty much let the Xeon per se off the hook. Anybody got a multiple Opteron to try? Totally non-Intel CPUs? It would be interesting to see results with non-Linux kernels, too.

Re: [PERFORM] sunquery and estimated rows

2004-04-19 Thread Litao Wu
Well, the example shown is simplified version. Now, let's see a little 'real' example (still simplified version): Table test is same as before: \d test Table "public.test" Column | Type | Modifiers -+--+--- id | inte

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Joe Conway
Tom Lane wrote: Here is a test case. To set up, run the "test_setup.sql" script once; then launch two copies of the "test_run.sql" script. (For those of you with more than two CPUs, see whether you need one per CPU to make trouble, or whether two test_runs are enough.) Check that you get a nestl

[PERFORM] Why will vacuum not end?

2004-04-19 Thread Shea,Dan [CIS]
This vacuum is running a marathon. Why will it not end and show me free space map INFO? We have deleted a lot of data and I would like to be confident that these deletions will be used as free space, rather than creating more table files. PWFPM_DEV=# select now();vacuum verbose forecasteleme

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread Robert Creager
When grilled further on (Mon, 19 Apr 2004 20:53:09 -0400), Tom Lane <[EMAIL PROTECTED]> confessed: > I wrote: > > Here is a test case. > > Hmmm ... I've been able to reproduce the CS storm on a dual Athlon, > which seems to pretty much let the Xeon per se off the hook. Anybody > got a multiple O

Re: [PERFORM] Why will vacuum not end?

2004-04-19 Thread Christopher Kings-Lynne
This vacuum is running a marathon. Why will it not end and show me free space map INFO? We have deleted a lot of data and I would like to be confident that these deletions will be used as free space, rather than creating more table files. Does another postgres query running have a lock on that t

[PERFORM] order by index, and inheritance

2004-04-19 Thread Michiel Meeuwissen
I have a query which performs not so well: SELECT * FROM mm_mediasources ORDER BY number DESC LIMIT 20; costs nearly a minute. The table contains over 300 000 records. The table has two extensions, which are (a the moment) nearly empty, but have something to do with this, because: SELECT * FRO

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread jelle
Same problem with dual 1Ghz P3's running Postgres 7.4.2, linux 2.4.x, and 2GB ram, under load, with long transactions (i.e. 1 "cannot serialize" rollback per minute). 200K was the worst observed with vmstat. Finally moved DB to a single xeon box. ---(end of broadcast)-