Re: [PERFORM] High load,

2011-02-07 Thread Greg Smith
Michael Kohl wrote: HDD: 2x 120 GB OCZ Vertex 2 SSD; RAID 1 As a general warning here, as far as I know the regular Vertex 2 SSD doesn't cache writes properly for database use. It's possible to have a crash that leaves the database corrupted, if the drive has writes queued up in its

Re: [PERFORM] getting the most of out multi-core systems for repeated complex SELECT statements

2011-02-07 Thread Vitalii Tymchyshyn
Hi, all My small thoughts about parallelizing single query. AFAIK in the cases where it is needed, there is usually one single operation that takes a lot of CPU, e.g. hashing or sorting. And this are usually tasks that has well known algorithms to parallelize. The main problem, as for me, is

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Marti Raudsepp
On Mon, Feb 7, 2011 at 05:03, Craig Ringer cr...@postnewspapers.com.au wrote: What would possibly help would be if Pg could fall back to lower shared_buffers automatically, screaming about it in the logs but still launching. OTOH, many people don't check the logs, so they'd think their new

Re: [PERFORM] Different execution plans for semantically equivalent queries

2011-02-07 Thread Marti Raudsepp
On Mon, Feb 7, 2011 at 00:03, Mikkel Lauritsen ren...@tala.dk wrote: SELECT * FROM table t1 WHERE 0 = (SELECT COUNT(*) FROM table t2 WHERE     t2.type = t1.type AND t2.timestamp t1.timestamp) I suspect that *any* database is going to have trouble optimizing that. Just out of curiosity I've

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread felix
+1 this is exactly what I was looking for at the time: a -t (configtest) option to pg_ctl and I think it should fall back to lower shared buffers and log it. SHOW ALL; would show the used value On Mon, Feb 7, 2011 at 11:30 AM, Marti Raudsepp ma...@juffo.org wrote: On Mon, Feb 7, 2011 at

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Scott Marlowe
On Mon, Feb 7, 2011 at 8:05 AM, felix crucialfe...@gmail.com wrote: +1 this is exactly what I was looking for at the time:  a -t (configtest) option to pg_ctl and I think it should fall back to lower shared buffers and log it. SHOW ALL; would show the used value however, much like apache,

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Greg Smith
Craig Ringer wrote: What would possibly help would be if Pg could fall back to lower shared_buffers automatically, screaming about it in the logs but still launching. This is exactly what initdb does when it produces an initial setting for shared_buffers that goes into the postgresql.conf

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Josh Berkus
Anyone seen anything like this before? it is the expected behavior, IIRC OK. It just seems kind of pathological for stats file writing to be 10X the volume of data writing. I see why it's happening, but I think it's something we should fix. -- -- Josh

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Joshua D. Drake
On Mon, 2011-02-07 at 14:58 -0800, Josh Berkus wrote: Anyone seen anything like this before? it is the expected behavior, IIRC OK. It just seems kind of pathological for stats file writing to be 10X the volume of data writing. I see why it's happening, but I think it's something we

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Cédric Villemain
2011/2/8 Joshua D. Drake j...@commandprompt.com: On Mon, 2011-02-07 at 14:58 -0800, Josh Berkus wrote: Anyone seen anything like this before? it is the expected behavior, IIRC OK.  It just seems kind of pathological for stats file writing to be 10X the volume of data writing.  I see

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Craig Ringer
On 02/07/2011 06:30 PM, Marti Raudsepp wrote: On Mon, Feb 7, 2011 at 05:03, Craig Ringercr...@postnewspapers.com.au wrote: What would possibly help would be if Pg could fall back to lower shared_buffers automatically, screaming about it in the logs but still launching. OTOH, many people don't

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Craig Ringer
On 02/08/2011 03:05 AM, Greg Smith wrote: Accordingly I would expect any serious attempt to add some auto-reduction behavior to be beset with argument, and I'd never consider writing such a thing as a result. Too many non-controversial things I could work on instead. Yep. I expressed my own

[PERFORM] Indexes with condition using immutable functions applied to column not used

2011-02-07 Thread Sylvain Rabot
Hi, I am trying to understand how indexes works to get the most of them. First I would like to know if there is more advantage than overhead to split an index in several ones using conditions e.g. doing : CREATE INDEX directory_id_user_0_btree_idx ON mike.directory USING btree (id_user) WHERE

Re: [PERFORM] Write-heavy pg_stats_collector on mostly idle server

2011-02-07 Thread Josh Berkus
Ooops. It looks like you are right, see ./src/backend/postmaster/pgstat.c 3c2313f4 (Tom Lane 2008-11-03 01:17:08 + 2926) if (last_statwrite last_statrequest) 70d75697 (Magnus Hagander2008-08-05 12:09:30 + 2927) pgstat_write_statsfile(false); This is a

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread felix
On Mon, Feb 7, 2011 at 6:05 AM, Shaun Thomas stho...@peak6.com wrote: That’s one of the things I talked about. To be safe, PG will start to shut down but disallow new connections, and **that’s all**. Old connections are grandfathered in until they disconnect, and when they all go away, it

Re: [PERFORM] Really really slow select count(*)

2011-02-07 Thread Scott Marlowe
On Mon, Feb 7, 2011 at 8:17 PM, felix crucialfe...@gmail.com wrote: On Mon, Feb 7, 2011 at 6:05 AM, Shaun Thomas stho...@peak6.com wrote: That’s one of the things I talked about. To be safe, PG will start to shut down but disallow new connections, and *that’s all*. Old connections are

Re: [PERFORM] Indexes with condition using immutable functions applied to column not used

2011-02-07 Thread Jesper Krogh
On 2011-02-08 01:14, Sylvain Rabot wrote: CREATE INDEX directory_id_user_mod_cons_hash_0_btree_idx ON mike.directory USING btree (id_user) WHERE __mod_cons_hash(id_user, 4) = 0; CREATE INDEX directory_id_user_mod_cons_hash_1_btree_idx ON mike.directory USING btree (id_user) WHERE

Re: [PERFORM] Talking about optimizer, my long dream

2011-02-07 Thread Achilleas Mantzios
Greg, 1st off, thanx for your great book, and i really hope i find the time to read it thoroughly. (since i am still stuck somewhere in the middle of Administration Cookbook lol!) Well, people, speaking from the point of the occasional poster and frequent lurker i can see that smth is going a