Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Martin French
> > Hi> > > > > > > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > > statements hang indefinitely until *something* (we don't know what)> > > releases some kind of resource or no longer becomes a massive bottle> > > neck. These are the symptoms.> > > > Is this in pgAdmin? Or psq

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Craig Ringer
On 07/23/2012 10:46 PM, Jim Vanns wrote: Nothing obvious - and we log a fair amount. No tmp table creations, no locks held. To add to this EXPLAIN reports it took only 0.23ms to run (for example) whereas the wall clock time is more like 20-30 minutes (or up to n hours as I said where everything

Re: [PERFORM] Efficiency of EXISTS?

2012-07-23 Thread Kenneth Tilton
On Mon, Jul 23, 2012 at 2:52 PM, Merlin Moncure wrote: > On Mon, Jul 23, 2012 at 4:12 PM, Kenneth Tilton wrote: > > My mental model of the EXISTS clause must be off. This snippet appears at > > the end of a series of WITH clauses I suspect are irrelevant: > > > >> with etc etc ... , cids as > >>

Re: [PERFORM] Efficiency of EXISTS?

2012-07-23 Thread Merlin Moncure
On Mon, Jul 23, 2012 at 4:12 PM, Kenneth Tilton wrote: > My mental model of the EXISTS clause must be off. This snippet appears at > the end of a series of WITH clauses I suspect are irrelevant: > >> with etc etc ... , cids as >> (select distinct c.id from ddr2 c >> join claim_entries ce on ce.c

[PERFORM] Efficiency of EXISTS?

2012-07-23 Thread Kenneth Tilton
My mental model of the EXISTS clause must be off. This snippet appears at the end of a series of WITH clauses I suspect are irrelevant: with etc etc ... , cids as > (select distinct c.id from ddr2 c > join claim_entries ce on ce.claim_id = c.id > where (c.assigned_ddr = 879 > or exists (select 1

Re: [PERFORM] High CPU Usage

2012-07-23 Thread Robert Haas
On Thu, Jun 14, 2012 at 11:15 AM, Siddharth Shah wrote: > I have around 1000 schema in database, Each schema having similar data > structure with different data > Each schema has few tables which never updates (Read only table) and other > tables rewrites almost everyday so I prefer to TRUNCATE th

Re: [PERFORM] postgres clustering interactions with pg_dump

2012-07-23 Thread Robert Haas
On Mon, Jun 11, 2012 at 9:55 AM, Fitch, Britt wrote: > Hi, I have a table that I am clustering on an index. > > I am then dumping that table via pg_dump –Fc and loading it into another > database via pg_restore. > > It is unclear to me though if the clustering I did in the original database > is p

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
On Mon, 2012-07-23 at 11:09 -0400, Tom Lane wrote: > Jim Vanns writes: > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some kind of resource or no longer becomes a massive bottle > > neck. These

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Tom Lane
Jim Vanns writes: > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > statements hang indefinitely until *something* (we don't know what) > releases some kind of resource or no longer becomes a massive bottle > neck. These are the symptoms. Does anything show up as blocked in the pg

Re: [PERFORM] Shards + hash = forever running queries

2012-07-23 Thread Tom Lane
Daniele Varrazzo writes: > Since "big" was sharded, the query plan results in something like: > Hash Join (cost=101.23..30038997974.72 rows=10 width=753) >Hash Cond: (b.id = i.big_id) >-> Append (cost=0.00..20038552251.23 rows=118859245 width=11) > -> Index Scan usin

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
On Mon, 2012-07-23 at 09:53 -0400, Andrew Dunstan wrote: > On 07/23/2012 04:41 AM, Jim Vanns wrote: > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some kind of resource or no longer becomes a mass

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
Thank you all for your replies, I shall try and qualify and confirm... On Mon, 2012-07-23 at 14:46 +0100, Martin French wrote: > Hi > > > > > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > > statements hang indefinitely until *something* (we don't know what) > > releases some ki

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Andrew Dunstan
On 07/23/2012 04:41 AM, Jim Vanns wrote: We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) statements hang indefinitely until *something* (we don't know what) releases some kind of resource or no longer becomes a massive bottle neck. These are the symptoms. I have seen this sort of

Re: [PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Martin French
Hi> > We're seeing SELECT statements and even EXPLAIN (no ANAYLZE) > statements hang indefinitely until *something* (we don't know what)> releases some kind of resource or no longer becomes a massive bottle> neck. These are the symptoms.Is this in pgAdmin? Or psql on the console?> However, the syst

Re: [PERFORM] Shards + hash = forever running queries

2012-07-23 Thread Daniele Varrazzo
On Mon, Jul 23, 2012 at 11:03 AM, Daniele Varrazzo wrote: > 1. Can we fix the queries to work around this problem? As a stop-gap measure I've defined a get_big(id) function and using it to pull in the details we're interested into from the "big" table: create function get_big (id int) retur

[PERFORM] Shards + hash = forever running queries

2012-07-23 Thread Daniele Varrazzo
Hello, We are using Postgres 9.1.4. We are struggling with a class of queries that got impossible to run after sharding a large table. Everything like: select small.something, big.anything from small join big on small.big_id = big.id; and variation such as "select * from big where id in

[PERFORM] Odd blocking (or massively latent) issue - even with EXPLAIN

2012-07-23 Thread Jim Vanns
(First, apologies if this post now appears twice - it appears our mail server rewrites my address!) Hello all. I'm a pgsql performance virgin so hope I cross all the 't's and dot the lower-case 'j's when posting this query... On our production database server we're experiencing behaviour that se