Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread Alan Stange
Here's a few numbers from the Opteron 250. If I get some time I'll post a more comprehensive comparison including some other systems. The system is a Sun v20z. Dual Opteron 250, 2.4Ghz, Linux 2.6, 8 GB memory. I did a compile and install of pg 8.0 beta 3. I created a data base on a tmpfs f

Re: [PERFORM] The never ending quest for clarity on shared_buffers

2004-10-06 Thread Neil Conway
On Thu, 2004-10-07 at 08:26, Paul Ramsey wrote: > The shared_buffers are shared (go figure) :). It is all one pool shared > by all connections. Yeah, I thought this was pretty clear. Doug, can you elaborate on where you saw the misleading docs? > The sort_mem and vacuum_mem are *per*connection*

Re: [PERFORM] Caching of Queries

2004-10-06 Thread Tatsuo Ishii
> > I don't know what you are exactly referring to in above URL > > when you are talking about "potential pitfalls of pooling". > > Please explain more. > > Sorry, I wasn't implying that pgpool doesn't deal with the issues, just that > some people aren't necessarily aware of them up front. For

Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread Gaetano Mendola
Alan Stange wrote: A few quick random observations on the Xeon v. Opteron comparison: [SNIP] I don't care to go into the whole debate of Xeon v. Opteron here. We also have a lot of dual Xeon systems. In every comparison I've done with our codes, the dual Opteron clearly outperforms the dual Xeo

Re: [PERFORM] The never ending quest for clarity on shared_buffers

2004-10-06 Thread Paul Ramsey
Doug Y wrote: For idle persistent connections, do each of them allocate the memory specified by this setting (shared_buffers * 8k), or is it one pool used by all the connection (which seems the logical conclusion based on the name SHARED_buffers)? Personally I'm more inclined to think the latt

[PERFORM] Data warehousing requirements

2004-10-06 Thread Gabriele Bartolini
Hi guys, I just discussed about my problem on IRC. I am building a Web usage mining system based on Linux, PostgreSQL and C++ made up of an OLTP database which feeds several and multi-purpose data warehouses about users' behaviour on HTTP servers. I modelled every warehouse using the star

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > But regardless of how uncommon it is, it could be considered important in > another sense: when you need it there really isn't any alternative. It's an > algorithmic improvement with no bound on the performance difference. [ shrug... ] There are an infinit

[PERFORM] The never ending quest for clarity on shared_buffers

2004-10-06 Thread Doug Y
Hello, We recently upgraded os from rh 7.2 (2.4 kernel) to Suse 9.1 (2.6 kernel), and psql from 7.3.4 to 7.4.2 One of the quirks I've noticed is how the queries don't always have the same explain plans on the new psql... but that's a different email I think. My main question is I'm trying

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Greg Stark <[EMAIL PROTECTED]> writes: > > why isn't a "skip index scan" plan available? Well, nobody's written the code > > yet. > > I don't really think it would be a useful plan anyway. Well it would clearly be useful in this test case, where has a sm

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > why isn't a "skip index scan" plan available? Well, nobody's written the code > yet. I don't really think it would be a useful plan anyway. What *would* be useful is to support HashAggregate as an implementation alternative for DISTINCT --- currently I bel

Re: [PERFORM] Comparing user attributes with bitwise operators

2004-10-06 Thread Greg Stark
Patrick Clery <[EMAIL PROTECTED]> writes: > PART OF THE QUERY PLAN: > Index Scan using people_attributes_pkey on people_attributes pa (cost=0.00..5.32 > rows=1 width=20) > Index Cond: (pa.person_id = "outer".person_id) > Filter: (((ARRAY[age, gender, orientation, children, drin

Re: [PERFORM] Comparing user attributes with bitwise operators

2004-10-06 Thread Patrick Clery
Err... I REINDEX'ed it and it is now using the index. :) I'd still appreciate if anyone could tell me why this needs to be reindexed. Is the index not updated when the records are inserted? > On Wednesday 06 October 2004 12:55, I wrote: > > Another problem I should note is that when I first inser

Re: [PERFORM] Comparing user attributes with bitwise operators

2004-10-06 Thread Patrick Clery
Another problem I should note is that when I first insert all the data into the people_attributes table ("the int[] table"), the GiST index is not used: THE INDEX: "people_attributes_search" gist ((ARRAY[age, gender, orientation, children, drinking, education, ethnicity, eyecolor, haircolor, ha

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Pierre-Frédéric Caillaud
There are even three questions here : - given that 'SELECT DISTINCT field FROM table' is exactly the same as 'SELECT field FROM table GROUP BY field", postgres could transform the first into the second and avoid itself a (potentially killer) sort. On my example the table was not too la

Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread SZUCS Gábor
Hmmm... I may be mistaken (I think last time I read about optimization params was in 7.3 docs), but doesn't RPC < 1 mean that random read is faster than sequential read? In your case, do you really think reading randomly is 4x faster than reading sequentially? Doesn't seem to make sense, even with

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Greg Stark
Pierre-Frédéric Caillaud <[EMAIL PROTECTED]> writes: > I don't know WHY (oh why) postgres does not use this kind of strategy > when distinct'ing an indexed field... Anybody got an idea ? Well there are two questions here. Why given the current plans available does postgres choose a sequent

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Ole Langbehn
Am Mittwoch, 6. Oktober 2004 12:19 schrieb Pierre-Frédéric Caillaud: > You could try : > > explain analyze select "land" from "customer_dim" group by "land"; > It will be a lot faster but I can't make it use the index on my machine... this already speeds up my queries to about 1/4th of the time,

Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread Bill Montgomery
Josh Berkus wrote: I'd be thrilled to test it too, if for no other reason that to determine whether what I'm experiencing really is the "CS problem". Hmmm ... Gavin's patch is built against 8.0, and any version of the patch would require linux 2.6, probably 2.6.7 minimum. Can you test on th

Re: [PERFORM] stats on cursor and query execution troubleshooting

2004-10-06 Thread Tom Lane
=?ISO-8859-1?Q?=22Alban_M=E9dici_=28NetCentrex=29=22?= <[EMAIL PROTECTED]> writes: > I'm looking for the statistic of memory, CPU, filesystem access while=20 > executing some regular SQL query, and I want to compare them to > same kind of results while executing a cursor function. I think your

Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread Alan Stange
Greg Stark wrote: Alan Stange <[EMAIL PROTECTED]> writes: A few quick random observations on the Xeon v. Opteron comparison: - running a dual Xeon with hyperthreading turned on really isn't the same as having a quad cpu system. I haven't seen postgresql specific benchmarks, but the general case

Re: [PERFORM] sequential scan on select distinct

2004-10-06 Thread Pierre-Frédéric Caillaud
You could try : explain analyze select "land" from "customer_dim" group by "land"; It will be a lot faster but I can't make it use the index on my machine... Example : create table dummy as (select id, id%255 as number from a large table with 1M rows); so we have a t

[PERFORM] stats on cursor and query execution troubleshooting

2004-10-06 Thread "Alban Médici (NetCentrex)"
Hi, I'm looking for the statistic of memory,  CPU,  filesystem access while executing some regular SQL query,  and I want to compare them to same kind of results while executing a cursor function. The stat collector give me good results (sequencial scans ,  acceded tuple .)  for regular

[PERFORM] sequential scan on select distinct

2004-10-06 Thread Ole Langbehn
Hi, I'm using Postgres 7.4.5. Tables are analyzed & vacuumed. I am wondering why postgresql never uses an index on queries of the type 'select distinct ...' while e.g. mysql uses the index on the same query. See the following explains: postgresql: explain analyze select distinct "land" from "c

Re: [PERFORM] Excessive context switching on SMP Xeons

2004-10-06 Thread Greg Stark
Alan Stange <[EMAIL PROTECTED]> writes: > A few quick random observations on the Xeon v. Opteron comparison: > > - running a dual Xeon with hyperthreading turned on really isn't the same as > having a quad cpu system. I haven't seen postgresql specific benchmarks, but > the general case has been