Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Arjen van der Meijden
On 11-4-2011 22:04 da...@lang.hm wrote: in your case, try your new servers without hyperthreading. you will end up with a 4x4 core system, which should handily outperform the 2x4 core system you are replacing. the limit isn't 8 cores, it's that the hyperthreaded cores don't work well with the

Re: [PERFORM] Slow query postgres 8.3

2011-04-12 Thread Claudio Freire
I actually implemented a statistical system for measuring these kinds of correlations. It's complex, but it might be adaptable to pgsql. Furthermore, one of the latest projects of mine was to replace the purely statistical approach with SVCs. Too bad I won't be able to devote any time to that

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread Claudio Freire
On Tue, Apr 12, 2011 at 7:20 AM, Dieter Rehbein dieter.rehb...@skiline.cc wrote: Hi everybody, I have a performance-problem with a query using a LIMIT. There are other threads rergading performance issues with LIMIT, but I didn't find useful hints for our problem and it might be

[PERFORM] DBT-5 Postgres 9.0.3

2011-04-12 Thread Sethu Prasad
Hi, Anyone lucky to have dbt5 run for PostgreSQL 9.0.3?! I am trying on Novell SuSE Linux Enterprise Server 11 SP1 x86_64 with a virtual machine and bit hard with no success run yet. If you can help me with any docs will be more of a support. Regards, Sethu Prasad

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Glyn Astill
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote: The issue I'm seeing is that 8 real cores outperform 16 real cores, which outperform 32 real cores under high concurrency. With every benchmark I've done of PostgreSQL, the knee in the performance graph comes right around

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Glyn Astill
--- On Tue, 12/4/11, Scott Marlowe scott.marl...@gmail.com wrote: From: Scott Marlowe scott.marl...@gmail.com Subject: Re: [PERFORM] Linux: more cores = less concurrency. To: Glyn Astill glynast...@yahoo.co.uk Cc: pgsql-performance@postgresql.org Date: Tuesday, 12 April, 2011, 6:55 On Mon,

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread Dieter Rehbein
what I did, was an ANALYZE, which did not change anything. I just executed a VACUUM ANALYZE and now everything performs well. hm, strange. thanks Dieter Am 12.04.2011 um 09:42 schrieb Claudio Freire: On Tue, Apr 12, 2011 at 7:20 AM, Dieter Rehbein dieter.rehb...@skiline.cc wrote: Hi

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread Claudio Freire
On Tue, Apr 12, 2011 at 10:59 AM, Dieter Rehbein dieter.rehb...@skiline.cc wrote: I just executed a VACUUM ANALYZE and now everything performs well. hm, strange. That probably means you need more statistics - try increasing the newsfeed's statistics target count. ALTER TABLE newsfeed_item

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread tv
On Tue, Apr 12, 2011 at 10:59 AM, Dieter Rehbein dieter.rehb...@skiline.cc wrote: I just executed a VACUUM ANALYZE and now everything performs well. hm, strange. That probably means you need more statistics - try increasing the newsfeed's statistics target count. ALTER TABLE newsfeed_item

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread Dieter Rehbein
thank's a lot guys, I will try that out. regards Dieter Am 12.04.2011 um 11:07 schrieb Claudio Freire: On Tue, Apr 12, 2011 at 10:59 AM, Dieter Rehbein dieter.rehb...@skiline.cc wrote: I just executed a VACUUM ANALYZE and now everything performs well. hm, strange. That probably means

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Glyn Astill
--- On Mon, 11/4/11, Kevin Grittner kevin.gritt...@wicourts.gov wrote: From: Kevin Grittner kevin.gritt...@wicourts.gov Subject: Re: [PERFORM] Linux: more cores = less concurrency. To: da...@lang.hm, Steve Clark scl...@netwolves.com, Kevin Grittner kevin.gritt...@wicourts.gov, Glyn Astill

Re: [PERFORM] Poor performance when joining against inherited tables

2011-04-12 Thread Shaun Thomas
On 04/11/2011 03:11 PM, Lucas Madar wrote: EXPLAIN ANALYZE SELECT * FROM objects INNER JOIN item f USING ( id ); This scans everything over everything, and obviously takes forever (there are millions of rows in the objects table, and tens of thousands in each itemXX table). What is your

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Merlin Moncure
On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill glynast...@yahoo.co.uk wrote: --- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote: The issue I'm seeing is that 8 real cores outperform 16 real cores, which outperform 32 real cores under high concurrency. With every benchmark I've

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Merlin Moncure
On Tue, Apr 12, 2011 at 8:23 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill glynast...@yahoo.co.uk wrote: --- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote: The issue I'm seeing is that 8 real cores outperform 16 real cores, which

Re: [PERFORM] Two servers - One Replicated - Same query

2011-04-12 Thread Kevin Grittner
Ozer, Pam po...@automotive.com wrote: I have two servers one has replication the other does not. The same query on both servers. One takes 225seconds on the replicated server the first time it runs and only 125ms on the other server the first time it runs. The second time you execute the

Re: [PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-12 Thread Tom Lane
Claudio Freire klaussfre...@gmail.com writes: Did you try increasing the statistic targets? AFAIK, it looks a lot like the planner is missing stats, since it estimates the index query on idx_nfi_newsfeed will fetch 10k rows - instead of 25. BTW, this is the right suggestion, but for the

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Kevin Grittner
Glyn Astill glynast...@yahoo.co.uk wrote: Tried tweeking LOG2_NUM_LOCK_PARTITIONS between 5 and 7. My results took a dive when I changed to 32 partitions, and improved as I increaced to 128, but appeared to be happiest at the default of 16. Good to know. Also, if you can profile

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Glyn Astill
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote: Can we see some iobound and cpubound pgbench runs on both servers? Of course, I'll post when I've gotten to that. Ok, there's no writing going on -- so the i/o tets aren't necessary. Context switches are also not too

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Glyn Astill
--- On Tue, 12/4/11, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Wow, zero idle and zero wait, and single digit for system.  Did you ever run those RAM speed tests?  (I don't remember seeing results for that -- or failed to recognize them.)  At this point, my best guess at this point

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Merlin Moncure
On Tue, Apr 12, 2011 at 11:01 AM, Glyn Astill glynast...@yahoo.co.uk wrote: --- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote: Can we see some iobound and cpubound pgbench runs on both servers? Of course, I'll post when I've gotten to that. Ok, there's no writing going

[PERFORM] Performance

2011-04-12 Thread Ogden
I have been wrestling with the configuration of the dedicated Postges 9.0.3 server at work and granted, there's more activity on the production server, but the same queries take twice as long on the beefier server than my mac at home. I have pasted what I have changed in postgresql.conf - I am

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Kevin Grittner
Glyn Astill glynast...@yahoo.co.uk wrote: Results from Greg Smiths stream_scaling test are here: http://www.privatepaste.com/4338aa1196 Well, that pretty much clinches it. Your RAM access tops out at 16 processors. It appears that your processors are spending most of their time waiting

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Claudio Freire
On Tue, Apr 12, 2011 at 6:40 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Well, that pretty much clinches it.  Your RAM access tops out at 16 processors.  It appears that your processors are spending most of their time waiting for and contending for the RAM bus. It tops, but it

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread F. BROUARD / SQLpro
Hi, I think that a NUMA architecture machine can solve the problem A + Le 11/04/2011 15:04, Glyn Astill a écrit : Hi Guys, I'm just doing some tests on a new server running one of our heavy select functions (the select part of a plpgsql function to allocate seats) concurrently. We do

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Greg Smith
Kevin Grittner wrote: Glyn Astill glynast...@yahoo.co.uk wrote: Results from Greg Smiths stream_scaling test are here: http://www.privatepaste.com/4338aa1196 Well, that pretty much clinches it. Your RAM access tops out at 16 processors. It appears that your processors are

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Greg Smith
Scott Marlowe wrote: Have you tried running the memory stream benchmark Greg Smith had posted here a while back? It'll let you know if you're memory is bottlenecking. Right now my 48 core machines are the king of that benchmark with something like 70+Gig a second. The big Opterons are

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Merlin Moncure
On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith g...@2ndquadrant.com wrote: Kevin Grittner wrote: Glyn Astill glynast...@yahoo.co.uk wrote: Results from Greg Smiths stream_scaling test are here: http://www.privatepaste.com/4338aa1196  Well, that pretty much clinches it.  Your RAM access

Re: [PERFORM] Performance

2011-04-12 Thread Andreas Kretschmer
Ogden li...@darkstatic.com wrote: I have been wrestling with the configuration of the dedicated Postges 9.0.3 server at work and granted, there's more activity on the production server, but the same queries take twice as long on the beefier server than my mac at home. I have pasted what I

Re: [PERFORM] Performance

2011-04-12 Thread Ogden
On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden li...@darkstatic.com wrote: I have been wrestling with the configuration of the dedicated Postges 9.0.3 server at work and granted, there's more activity on the production server, but the same queries take twice as long on the

Re: [PERFORM] Performance

2011-04-12 Thread Tomas Vondra
Dne 12.4.2011 19:23, Ogden napsal(a): On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden li...@darkstatic.com wrote: I have been wrestling with the configuration of the dedicated Postges 9.0.3 server at work and granted, there's more activity on the production server, but

Re: [PERFORM] Performance

2011-04-12 Thread Ogden
On Apr 12, 2011, at 1:16 PM, Tomas Vondra wrote: Dne 12.4.2011 19:23, Ogden napsal(a): On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden li...@darkstatic.com wrote: I have been wrestling with the configuration of the dedicated Postges 9.0.3 server at work and granted,

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-12 Thread Strange, John W
When purchasing the intel 7500 series, please make sure to check the hemisphere mode of your memory configuration. There is a HUGE difference in the memory configuration around 50% speed if you don't populate all the memory slots on the controllers properly.

Re: [PERFORM] Performance

2011-04-12 Thread Tomas Vondra
Dne 12.4.2011 20:28, Ogden napsal(a): On Apr 12, 2011, at 1:16 PM, Tomas Vondra wrote: Dne 12.4.2011 19:23, Ogden napsal(a): On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden li...@darkstatic.com wrote: I have been wrestling with the configuration of the dedicated Postges

Re: [PERFORM] Performance

2011-04-12 Thread Ogden
On Apr 12, 2011, at 4:09 PM, Tomas Vondra wrote: Dne 12.4.2011 20:28, Ogden napsal(a): On Apr 12, 2011, at 1:16 PM, Tomas Vondra wrote: Dne 12.4.2011 19:23, Ogden napsal(a): On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden li...@darkstatic.com wrote: I have been

Re: [PERFORM] Performance

2011-04-12 Thread Tomas Vondra
Dne 12.4.2011 23:19, Ogden napsal(a): On Apr 12, 2011, at 4:09 PM, Tomas Vondra wrote: Dne 12.4.2011 20:28, Ogden napsal(a): On Apr 12, 2011, at 1:16 PM, Tomas Vondra wrote: Dne 12.4.2011 19:23, Ogden napsal(a): On Apr 12, 2011, at 12:18 PM, Andreas Kretschmer wrote: Ogden

[PERFORM] poor execution plan because column dependence

2011-04-12 Thread Václav Ovsík
Hi, I have done migration of the Request Tracker 3.8.9 (http://requesttracker.wikia.com/wiki/HomePage) from Mysql to PostgreSQL in testing environment. The RT schema used can be viewed at https://github.com/bestpractical/rt/blob/3.8-trunk/etc/schema.Pg. I have added full text search on table

Re: [PERFORM] poor execution plan because column dependence

2011-04-12 Thread Bob Lunney
Zito, Using psql log in as the database owner and run analyze verbose. Happiness will ensue. Also, when requesting help with a query its important to state the database version (select version();) and what, if any, configuration changes you have made in postgresql.conf. Listing ony the

Re: [PERFORM] poor execution plan because column dependence

2011-04-12 Thread Tom Lane
=?iso-8859-1?Q?V=E1clav_Ovs=EDk?= vaclav.ov...@i.cz writes: I think the execution plan is poor. Better would be to filter table attachments at first and then join the rest. The reason is a bad estimate on number of rows returned from table tickets (85 estimated - 25410 in the reality). ...