[PERFORM] Poor performance when joining against inherited tables

2011-04-11 Thread Lucas Madar
I have a database that contains many tables, each with some common characteristics. For legacy reasons, they have to be implemented in a way so that they are *all* searchable by an older identifier to find the newer identifier. To do this, we've used table inheritance. Each entry has an id, as

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 7:04 AM, Glyn Astill wrote: > Hi Guys, > > I'm just doing some tests on a new server running one of our heavy select > functions (the select part of a plpgsql function to allocate seats) > concurrently.  We do use connection pooling and split out some selects to > slony

[PERFORM] performance problem with LIMIT (order BY in DESC order). Wrong index used?

2011-04-11 Thread Dieter Rehbein
Hi everybody, I have a performance-problem with a query using a LIMIT. There are other threads rergading performance issues with LIMIT, but I didn't find useful hints for our problem and it might be interesting for other postgres-users. There are only 2 simple tables: CREATE TABLE newsfeed (

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Jesper Krogh
On 2011-04-11 22:39, James Cloos wrote: "GA" == Glyn Astill writes: GA> I was hoping someone had seen this sort of behaviour before, GA> and could offer some sort of explanation or advice. Jesper's reply is probably most on point as to the reason. I know that recent Opterons use some of the

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 6:50 PM, mark wrote: > > Interesting, thanks for sharing. > > I guess I have never gotten to the point where I felt I needed more than 2 > drives for my xlogs. Maybe I have been dismissing that as a possibility > something. (my biggest array is only 24 SFF drives tho) > > I

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Merlin Moncure
On Mon, Apr 11, 2011 at 5:06 PM, Kevin Grittner wrote: > Glyn Astill wrote: > >> The issue I'm seeing is that 8 real cores outperform 16 real >> cores, which outperform 32 real cores under high concurrency. > > With every benchmark I've done of PostgreSQL, the "knee" in the > performance graph co

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread mark
> -Original Message- > From: Scott Marlowe [mailto:scott.marl...@gmail.com] > Sent: Monday, April 11, 2011 6:18 PM > To: mark > Cc: Glyn Astill; Kevin Grittner; Joshua D. Drake; pgsql- > performa...@postgresql.org > Subject: Re: [PERFORM] Linux: more cores = less concurrency. > > On Mon,

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 6:18 PM, Scott Marlowe wrote: > On Mon, Apr 11, 2011 at 6:05 PM, mark wrote: >> Just wondering, which LSI card ? >> Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some >> other config? > > We were using teh LSI but I'll be switching back to Areca wh

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 6:05 PM, mark wrote: > Just wondering, which LSI card ? > Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some > other config? We were using teh LSI but I'll be switching back to Areca when we go back to HW RAID. The LSI only performed well if w

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread mark
> -Original Message- > From: pgsql-performance-ow...@postgresql.org [mailto:pgsql-performance- > ow...@postgresql.org] On Behalf Of Scott Marlowe > Sent: Monday, April 11, 2011 1:29 PM > To: Glyn Astill > Cc: Kevin Grittner; Joshua D. Drake; pgsql-performance@postgresql.org > Subject: Re:

[PERFORM] Two servers - One Replicated - Same query

2011-04-11 Thread Ozer, Pam
I have two servers one has replication the other does not. The same query on both servers. One takes 225seconds on the replicated server the first time it runs and only 125ms on the other server the first time it runs. The second time you execute the query it drops to the 125ms. They are using th

Re: [PERFORM] how explain works to Mr Nathan Boley

2011-04-11 Thread Radhya sahal
Thanks Mr Nathan Boley , i want these equations to solve thsese equtions of parameters and total time  in order to get each paramter formula i need these formula  in my experiments is very important to know the rate for each parameter in total cost for plan.  Best Radhya..

Re: [PERFORM] how explain works

2011-04-11 Thread Nathan Boley
> how explian works as math equations to estimate cost with  constatn query > parameters > such as cpu_tuple cost ,random page cost ...etc >  i want maths  expression  in order to know how these parameters will effect > in cost ??? The expressions are complicated, and they are certainly not linear

[PERFORM] how explain works

2011-04-11 Thread Radhya sahal
Dear ,all plz could any one help me !!! how explian works as math equations to estimate cost with  constatn query parameters such as cpu_tuple cost ,random page cost ...etc  i want maths  expression  in order to know how these parameters will effect in cost ??? please any one can help me ??     R

Re: [PERFORM] Postgres 9 slave lagging

2011-04-11 Thread Merlin Moncure
On Mon, Apr 11, 2011 at 9:10 AM, Saurabh Agrawal wrote: > Hi All, > > I have setup postgres 9 master slave streaming replication but > experiencing slave lagging sometimes by 50 min to 60 min. I am not > getting exact reason for slave lag delay. Below are the details: > > 1. Master table contains

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread David Rees
On Mon, Apr 11, 2011 at 6:04 AM, Glyn Astill wrote: > The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz, our current servers > are 2 x 4 core Xeon E5320 CPUs at 2Ghz. > > What I'm seeing is when the number of clients is greater than the number of > cores, the new servers perform better on f

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Kevin Grittner
"Kevin Grittner" wrote: > I don't know why you were hitting the knee sooner than I've seen > in my benchmarks If you're compiling your own executable, you might try boosting LOG2_NUM_LOCK_PARTITIONS (defined in lwlocks.h) to 5 or 6. The current value of 4 means that there are 16 partitions to

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread James Cloos
> "GA" == Glyn Astill writes: GA> I was hoping someone had seen this sort of behaviour before, GA> and could offer some sort of explanation or advice. Jesper's reply is probably most on point as to the reason. I know that recent Opterons use some of their cache to better manage cache-cohere

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Glyn Astill
--- On Mon, 11/4/11, Scott Marlowe wrote: > From: Scott Marlowe > Subject: Re: [PERFORM] Linux: more cores = less concurrency. > To: "Glyn Astill" > Cc: "Kevin Grittner" , "Joshua D. Drake" > , pgsql-performance@postgresql.org > Date: Monday, 11 April, 2011, 21:52 > On Mon, Apr 11, 2011 at 1

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Kevin Grittner
Glyn Astill wrote: > The issue I'm seeing is that 8 real cores outperform 16 real > cores, which outperform 32 real cores under high concurrency. With every benchmark I've done of PostgreSQL, the "knee" in the performance graph comes right around ((2 * cores) + effective_spindle_count). With

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 1:42 PM, Glyn Astill wrote: > A wild guess is something like multiple cores contending for cpu cache, cpu > affinity, or some kind of contention in the kernel, alas a little out of my > depth. > > It's pretty sickening to think I can't get anything else out of more than

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread david
On Mon, 11 Apr 2011, Steve Clark wrote: On 04/11/2011 02:32 PM, Scott Marlowe wrote: On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake wrote: On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner" wrote: Glyn Astill wrote: The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz Which has h

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Glyn Astill
--- On Mon, 11/4/11, da...@lang.hm wrote: > From: da...@lang.hm > Subject: Re: [PERFORM] Linux: more cores = less concurrency. > To: "Steve Clark" > Cc: "Scott Marlowe" , "Joshua D. Drake" > , "Kevin Grittner" , > pgsql-performance@postgresql.org, "Glyn Astill" > Date: Monday, 11 April, 2011

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Jesper Krogh
On 2011-04-11 21:42, Glyn Astill wrote: I'll have to try with the synthetic benchmarks next then, but somethings definately going off here. I'm seeing no disk activity at all as they're selects and all pages are in ram. Well, if you dont have enough computations to be bottlenecked on the cpu

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Steve Clark
On 04/11/2011 02:32 PM, Scott Marlowe wrote: On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake wrote: On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner" wrote: Glyn Astill wrote: The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz Which has hyperthreading. our current servers are

[PERFORM] Postgres 9 slave lagging

2011-04-11 Thread Saurabh Agrawal
Hi All, I have setup postgres 9 master slave streaming replication but experiencing slave lagging sometimes by 50 min to 60 min. I am not getting exact reason for slave lag delay. Below are the details: 1. Master table contains partition tables with frequent updates. 2. Slave is used for report g

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Glyn Astill
--- On Mon, 11/4/11, Scott Marlowe wrote: > Just FYI, in synthetic pgbench type benchmarks, a 48 core > AMD Magny > Cours with LSI HW RAID and 34 15k6 Hard drives scales > almost linearly > up to 48 or so threads, getting into the 7000+ tps > range.  With SW > RAID it gets into the 5500 tps range

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 12:23 PM, Glyn Astill wrote: > > > --- On Mon, 11/4/11, Joshua D. Drake wrote: > >> From: Joshua D. Drake >> Subject: Re: [PERFORM] Linux: more cores = less concurrency. >> To: "Kevin Grittner" >> Cc: pgsql-performance@postgresql.org, "Glyn Astill" >> Date: Monday, 11 A

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 12:39 PM, Tom Lane wrote: > Scott Marlowe writes: >> On Mon, Apr 11, 2011 at 1:41 AM, Greg Smith wrote: >>> Scott Marlowe wrote: FYI, in 8.3.13 I get this for all but one index: ERROR:  deadlock detected > >>> Is that trying to build them by hand?  The upthread

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Tom Lane
Scott Marlowe writes: > On Mon, Apr 11, 2011 at 1:41 AM, Greg Smith wrote: >> Scott Marlowe wrote: >>> FYI, in 8.3.13 I get this for all but one index: >>> ERROR:  deadlock detected >> Is that trying to build them by hand?  The upthread request here is actually >> already on the TODO list at htt

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake wrote: > On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner" > wrote: >> Glyn Astill wrote: >> >>> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz >> >> Which has hyperthreading. >> >>> our current servers are 2 x 4 core Xeon E5320 CPUs

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Glyn Astill
--- On Mon, 11/4/11, Joshua D. Drake wrote: > From: Joshua D. Drake > Subject: Re: [PERFORM] Linux: more cores = less concurrency. > To: "Kevin Grittner" > Cc: pgsql-performance@postgresql.org, "Glyn Astill" > Date: Monday, 11 April, 2011, 19:12 > On Mon, 11 Apr 2011 13:09:15 -0500, > "Kevin

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Shaun Thomas
On 04/09/2011 11:28 AM, Chris Ruprecht wrote: I'm wondering if there is a way to build these indexes in parallel while reading the table only once for all indexes and building them all at the same time. Is there an index build tool that I missed somehow, that can do this? I threw together a ve

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Joshua D. Drake
On Mon, 11 Apr 2011 13:09:15 -0500, "Kevin Grittner" wrote: > Glyn Astill wrote: > >> The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz > > Which has hyperthreading. > >> our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz. > > Which doesn't have hyperthreading. > > PostgreS

Re: [PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Kevin Grittner
Glyn Astill wrote: > The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz Which has hyperthreading. > our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz. Which doesn't have hyperthreading. PostgreSQL often performs worse with hyperthreading than without. Have you turned HT off

[PERFORM] Linux: more cores = less concurrency.

2011-04-11 Thread Glyn Astill
Hi Guys, I'm just doing some tests on a new server running one of our heavy select functions (the select part of a plpgsql function to allocate seats) concurrently.  We do use connection pooling and split out some selects to slony slaves, but the tests here are primeraly to test what an individ

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Scott Marlowe
On Mon, Apr 11, 2011 at 1:41 AM, Greg Smith wrote: > Scott Marlowe wrote: >> >> FYI, in 8.3.13 I get this for all but one index: >> >> ERROR:  deadlock detected >> DETAIL:  Process 24488 waits for ShareLock on virtual transaction >> 64/825033; blocked by process 27505. >> Process 27505 waits for S

Re: [PERFORM] Slow query postgres 8.3

2011-04-11 Thread Kevin Grittner
"Anne Rosset" wrote: >-> Index Scan using role_oper_obj_oper > on role_operation (cost=0.00..93.20 rows=45 width=9) (actual > time=0.236..71.291 rows=6108 loops=1) > Index Cond: > (((object_type_id)::text = 'SfMain.Project'::text) AN

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Greg Smith
Scott Marlowe wrote: FYI, in 8.3.13 I get this for all but one index: ERROR: deadlock detected DETAIL: Process 24488 waits for ShareLock on virtual transaction 64/825033; blocked by process 27505. Process 27505 waits for ShareUpdateExclusiveLock on relation 297369165 of database 278059474; blo

Re: [PERFORM] Multiple index builds on same table - in one sweep?

2011-04-11 Thread Scott Marlowe
On Sun, Apr 10, 2011 at 11:35 PM, Scott Marlowe wrote: > On Sun, Apr 10, 2011 at 8:29 PM, Greg Smith wrote: >> On 04/09/2011 01:23 PM, Chris Ruprecht wrote: >>> >>> Maybe, in a future release, somebody will develop something that can >>> create indexes as inactive and have a build tool build and

Re: [PERFORM] Slow query postgres 8.3

2011-04-11 Thread Anne Rosset
Hi Thomas, Here is the plan after explain. QUERY PLAN