Re: [PERFORM] Performance regressions in PG 9.3 vs PG 9.0

2014-04-08 Thread Tom Lane
uher dslij writes: > Thanks for your reply Tom. I've found that the culprit is the function > parentContainers(), which recurses up a folder structure and looks like > this: Hmm ... I had noticed the execution of that in a subplan, but it appeared that the subplan was being done the same number

Re: [PERFORM] query against large table not using sensible index to find very small amount of data

2014-04-08 Thread 'Andrew W. Gibbs'
Your understanding of the utility of multi-part indices does not jive with my own. While I agree that a partial index might be in order here, that ought just be a performance optimization that lowers the footprint of the index from an index size and index maintenance standpoint, not something that

Re: [PERFORM] Performance regressions in PG 9.3 vs PG 9.0

2014-04-08 Thread uher dslij
Sorry for the premature send on that last email. Here is the full one: Thanks for your reply Tom. I've found that the culprit is the function parentContainers(), which recurses up a folder structure and looks like this: create function parentContainers(numeric) returns setof numeric as ' se

Re: [PERFORM] Performance regressions in PG 9.3 vs PG 9.0

2014-04-08 Thread uher dslij
Thanks for your reply Tom. I've found that the culprit is the function parentContainers(), which recurses in a folder structure and looks like this: create function parentContainers(numeric) returns setof numeric as ' select parentContainers( (select container_id from container where id = $1

Re: [PERFORM] query against large table not using sensible index to find very small amount of data

2014-04-08 Thread Jeff Janes
On Tue, Apr 8, 2014 at 6:39 AM, Shaun Thomas wrote: > > > Other possibly relevant pieces of information... The entity type > > column has a cardinality in the neighborhood of a couple dozen. > > Meanwhile, for some of the entity types there is a large and ongoing > > number of events, and for oth

Re: [PERFORM] Performance regressions in PG 9.3 vs PG 9.0

2014-04-08 Thread Tom Lane
uher dslij writes: > The EXPLAINs all pretty much look like my original post. The planner in > 9.2 and above is simply not using bitmap heap scans or bitmap index scans? > What could be the reason for this? I don't see any reason to think this is a planner regression. The rowcount estimates ar

[PERFORM] Optimizing Time Series Access

2014-04-08 Thread Robert Burgholzer
I am looking for advice on dealing with large tables of environmental model data and looking for alternatives to my current optimization approaches. Basically, I have about 1 Billion records stored in a table which I access in groups of roughly 23 Million at a time. Which means that I have somew

Re: [PERFORM] Nested loop issue

2014-04-08 Thread Dhananjay Singh
REPLACE -- where sos_stg_aggr.tid_stg in (select distinct lehr_stg_ab_tid from lehr_stg_ab2fb) WITH -- where sos_stg_aggr.tid_stg EXISTS (select distinct lehr_stg_ab_tid from lehr_stg_ab2fb) Similarly others also like -- lehr_stg_ab.tid not in (select lehr_stg_ab_tid from lehr_stg_ab2fb) with

Re: [PERFORM] Performance regressions in PG 9.3 vs PG 9.0

2014-04-08 Thread uher dslij
As a follow up to this issue on Graeme's suggestion in a private email, I checked the statistics in both databases, and they were the same (these values as seen from pg_stat_user_tables to not seem to propagate to slave databases however). I even ran a manual analyze on the master database in the

Re: [SQL] Re: [PERFORM] performance drop when function argument is evaluated in WHERE clause

2014-04-08 Thread Tom Lane
Gerardo Herzig writes: > Tom, thanks (as allways) for your answer. This is a 9.1.12. I have to say, im > not very happy about if-elif-else'ing at all. > The "conditional filter" es a pretty common pattern in our functions, i would > have to add (and maintain) a substantial amount of extra code.

Re: [PERFORM] performance drop when function argument is evaluated in WHERE clause

2014-04-08 Thread Gerardo Herzig
Tom, thanks (as allways) for your answer. This is a 9.1.12. I have to say, im not very happy about if-elif-else'ing at all. The "conditional filter" es a pretty common pattern in our functions, i would have to add (and maintain) a substantial amount of extra code. And i dont really understand w

Re: [PERFORM] PGSQL, checkpoints, and file system syncs

2014-04-08 Thread Shaun Thomas
> Is there something I can set in the PGSQL parameters or in the file system > parameters to force a steady flow of writes to disk rather than waiting for > a sync system call? Mounting with "commit=1" did not make a difference. The PostgreSQL devs actually had a long talk with the Linux kernel de

Re: [PERFORM] query against large table not using sensible index to find very small amount of data

2014-04-08 Thread Tom Lane
"Andrew W. Gibbs" writes: > A very common query against this table is of the form... > SELECT * FROM events WHERE entity_type_id = XXX ORDER BY published_at DESC > LIMIT 25; > ... to get the most recent 25 events from the table for a given type > of entity, and generally the query planner does

Re: [PERFORM] performance drop when function argument is evaluated in WHERE clause

2014-04-08 Thread Tom Lane
Gerardo Herzig writes: > Hi all. I have a function that uses a "simple" select between 3 tables. There > is a function argument to help choose how a WHERE clause applies. This is the > code section: > select * from > [...] > where case $3 > when 'I' then [filter 1] > when 'E' then [fil

Re: [PERFORM] query against large table not using sensible index to find very small amount of data

2014-04-08 Thread Shaun Thomas
> Other possibly relevant pieces of information... The entity type > column has a cardinality in the neighborhood of a couple dozen. > Meanwhile, for some of the entity types there is a large and ongoing > number of events, and for other entity types there is a smaller and > more sporadic number

[PERFORM] query against large table not using sensible index to find very small amount of data

2014-04-08 Thread Andrew W. Gibbs
I have a fairly large table (~100M rows), let's call it "events", and among other things it has a couple of columns on it, columns that we'll call entity_type_id (an integer) and and published_at (a timestamp). It has, among others, indices on (published_at) and (entity_type_id, published_at). A

[PERFORM] performance drop when function argument is evaluated in WHERE clause

2014-04-08 Thread Gerardo Herzig
Hi all. I have a function that uses a "simple" select between 3 tables. There is a function argument to help choose how a WHERE clause applies. This is the code section: select * from [...] where case $3 when 'I' then [filter 1] when 'E' then [filter 2] when 'P' then [filter 3] else

[PERFORM] Nested loop issue

2014-04-08 Thread Manoj Gadi
Hi All, I have been looking for a solution to a problem where my query is executing for a long time because it is running into a nested loop problem. I have done explain analyze and it shows the query taking a very long time due to nested loops. On the DB side, there are indices in place for a