Re: [PERFORM] Performance for relative large DB

2005-08-26 Thread Chris Travers
tobbe wrote: Hi Chris. Thanks for the answer. Sorry that i was a bit unclear. 1) We update around 20.000 posts per night. 2) What i meant was that we suspect that the DBMS called PervasiveSQL that we are using today is much to small. That's why we're looking for alternatives. Today we base o

Re: [PERFORM] Need indexes on empty tables for good performance ?

2005-08-26 Thread Bruno Wolff III
On Tue, Aug 23, 2005 at 13:41:32 +1000, "Lenard, Rohan (Rohan)" <[EMAIL PROTECTED]> wrote: > I've read that indexes aren't used for COUNT(*) and I've noticed (7.3.x) > with EXPLAIN that indexes never seem to be used on empty tables - is > there any reason to have indexes on empty tables, or will

Re: [PERFORM] Performance for relative large DB

2005-08-26 Thread tobbe
Hi Chris. Thanks for the answer. Sorry that i was a bit unclear. 1) We update around 20.000 posts per night. 2) What i meant was that we suspect that the DBMS called PervasiveSQL that we are using today is much to small. That's why we're looking for alternatives. Today we base our solution much

[PERFORM] Need indexes on empty tables for good performance ?

2005-08-26 Thread Lenard, Rohan (Rohan)
I've read that indexes aren't used for COUNT(*) and I've noticed (7.3.x) with EXPLAIN that indexes never seem to be used on empty tables - is there any reason to have indexes on empty tables, or will postgresql never use them.   This is not as silly as it sounds - with table inheritance you

[PERFORM] Performance for relative large DB

2005-08-26 Thread tobbe
Hi. The company that I'm working for are surveying the djungle of DBMS since we are due to implement the next generation of our system. The companys buissnes is utilizing the DBMS to store data that are accessed trough the web at daytime (only SELECTs, sometimes with joins, etc). The data is a co

Re: [PERFORM] Caching by Postgres

2005-08-26 Thread Thomas Ganss
The first, to always remember - is that the move from 64-bits to 32-bits doesn't come for free. In a real 64-bit system with a 64-bit operating system, and 64-bit applications, pointers are now double their 32-bit size. This means more bytes to copy around memory, and in an extreme case, has the

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > As far as the "desc" point goes, the problem is that mergejoins aren't > capable of dealing with backward sort order, so a merge plan isn't > considered for that case (or at least, it would have to have a sort > after it, which pretty much defeats the point

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Mark Kirkwood
Interestingly enough, 7.4.8 and 8.1devel-2005-08-23 all behave the same as 8.0.3 for me (tables freshly ANALYZEd): joinlimit=# SELECT version(); version ---

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread Michael Fuhr
On Fri, Aug 26, 2005 at 05:10:49PM -0700, asif ali wrote: > "GroupAggregate (cost=195623.66..206672.52 rows=20132 > width=16) (actual time=8205.283..10139.369 rows=55291 > loops=1)" > " -> Sort (cost=195623.66..198360.71 rows=1094820 > width=16) (actual time=8205.114..9029.501 rows=863883 > loo

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread asif ali
Thanks Michael For your reply. Here is performance on the database on which i did VACUUM ANALYZE explain analyze select keyword_id ,sum(daily_impressions) as daily_impressions ,sum(daily_actions)as daily_actions from conversion_table c wherec.conversion_date BETWEEN '20

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Tom Lane
Mark Kirkwood <[EMAIL PROTECTED]> writes: > That doesn't explain why the nested loop is being kicked tho', No, but I think the fuzzy-cost bug does. There are two different issues here. regards, tom lane ---(end of broadcast)---

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread Tom Lane
=?ISO-8859-1?Q?=DCmit_=D6ztosun?= <[EMAIL PROTECTED]> writes: > We are using PostgreSQL for our business application. Recently, during > testing of our application with large volumes of data, we faced a weird > problem. Our query performance dropped *dramatically* after "VACUUM FULL > ANALYZE" comm

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Mark Kirkwood
Tom Lane wrote: Mark Kirkwood <[EMAIL PROTECTED]> writes: What is interesting is why this plan is being rejected... Which PG version are you using exactly? That mistake looks like an artifact of the 8.0 "fuzzy plan cost" patch, which we fixed recently: http://archives.postgresql.org/pgsql-c

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread Michael Fuhr
On Fri, Aug 26, 2005 at 03:52:24PM -0700, asif ali wrote: > I have the same issue. After doing "VACCUME ANALYZE" > performance of the query dropped. Your EXPLAIN output doesn't show the actual query times -- could you post the EXPLAIN ANALYZE output? That'll also show how accurate the planner's

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread Philip Hallstrom
Hi, I have the same issue. After doing "VACCUME ANALYZE" performance of the query dropped. Here is the query explain select * from conversion_table c where c.conversion_date BETWEEN '2005-06-07' and '2005-08-17' Before "VACCUME ANALYZE" "Index Scan using conversion_table_pk on keyword_conversi

Re: [PERFORM] Inefficient queryplan for query with intersectable

2005-08-26 Thread Tom Lane
Arjen van der Meijden <[EMAIL PROTECTED]> writes: > As said, it chooses sequential scans or "the wrong index plans" over a > perfectly good plan that is just not selected when the parameters are > "too well tuned" or sequential scanning of the table is allowed. I think some part of the problem c

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread asif ali
Hi, I have the same issue. After doing "VACCUME ANALYZE" performance of the query dropped. Here is the query explain select * from conversion_table c where c.conversion_date BETWEEN '2005-06-07' and '2005-08-17' Before "VACCUME ANALYZE" "Index Scan using conversion_table_pk on key

Re: [PERFORM] Sending a select to multiple servers.

2005-08-26 Thread Ligesh
On Fri, Aug 26, 2005 at 11:04:59AM -0500, Frank Wiles wrote: > On Fri, 26 Aug 2005 20:54:09 +0530 > Ligesh <[EMAIL PROTECTED]> wrote: > Mostly because every situation is different, you may have > 10 servers and need 10 rows of results, others may need something > entirely different. > N

Re: [PERFORM] Sending a select to multiple servers.

2005-08-26 Thread Ligesh
On Fri, Aug 26, 2005 at 11:04:59AM -0500, Frank Wiles wrote: > On Fri, 26 Aug 2005 20:54:09 +0530 > This is typically handled by the application layer, not a standard > client. Mostly because every situation is different, you may have > 10 servers and need 10 rows of results, others may need

[PERFORM] Weird performance drop after VACUUM

2005-08-26 Thread Ümit Öztosun
Hello, We are using PostgreSQL for our business application. Recently, during testing of our application with large volumes of data, we faced a weird problem. Our query performance dropped *dramatically* after "VACUUM FULL ANALYZE" command. We have encountered a similar problem listed on mailing l

Re: [PERFORM] Inefficient queryplan for query with intersectable

2005-08-26 Thread Arjen van der Meijden
On 26-8-2005 15:05, Richard Huxton wrote: Arjen van der Meijden wrote: I left all the configuration-stuff to the defaults since changing values didn't seem to impact much. Apart from the buffers and effective cache, increasing those made the performance worse. I've not looked at the rest

Re: [PERFORM] Performance indexing of a simple query

2005-08-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Uh, the plain english and the SQL don't match. That query will find > every job that was NOT running at the time you said. No, I think it was right. But anyway it was just an example. > On Wed, Aug 24, 2005 at 07:42:00PM -0400, Tom Lane wrote: >> AFAI

Re: [PERFORM] How does the planner execute unions?

2005-08-26 Thread Bruno Wolff III
On Fri, Aug 26, 2005 at 16:14:18 -0400, Chris Hoover <[EMAIL PROTECTED]> wrote: > Hopefully a quick question. > > In 7.3.4, how does the planner execute a query with union alls in it? > > Does it execute the unions serially, or does it launch a "thread" for > each union (or maybe something else

[PERFORM] How does the planner execute unions?

2005-08-26 Thread Chris Hoover
Hopefully a quick question. In 7.3.4, how does the planner execute a query with union alls in it? Does it execute the unions serially, or does it launch a "thread" for each union (or maybe something else entirely). Thanks, Chris Here is an explain from the view I'm thinking about, how does pos

Re: [PERFORM] difference in plan between 8.0 and 8.1?

2005-08-26 Thread Alan Stange
Tom Lane wrote: Alan Stange <[EMAIL PROTECTED]> writes: Unique (cost=2717137.08..2771407.21 rows=10854026 width=8) -> Sort (cost=2717137.08..2744272.14 rows=10854026 width=8) Sort Key: timeseriesid -> Bitmap Heap Scan on tbltimeseries (cost=48714.09..1331000.42 row

[PERFORM] OSX & Performance

2005-08-26 Thread Jeff Trout
Well folks, I've been trying to track down why this Athlon 2800 (2.1ghz) has been handing my 2.5ghz G5 its cake. I have a query that (makes no io - the dataset can live in ram easily) takes about 700ms on the athlon and about 10 seconds on the G5. Tracking ti down a bit timestamp_cmp_inter

Re: [PERFORM] Performance indexing of a simple query

2005-08-26 Thread Jim C. Nasby
On Wed, Aug 24, 2005 at 07:42:00PM -0400, Tom Lane wrote: > Mark Fox <[EMAIL PROTECTED]> writes: > > The sort of queries I want to execute (among others) are like: > > SELECT * FROM jobs > > WHERE completion_time > SOMEDATE AND start_time < SOMEDATE; > > In plain english: All the jobs that were ru

Re: [PERFORM] Sending a select to multiple servers.

2005-08-26 Thread Merlin Moncure
> Does such a solution exist now. To me this appears to be in entirety of > what should constitute a database cluster. Only the search needs to be > done on all the servers simultaneously at the low level. Once you get the > results, the writing can be determined by the upper level logic (which ca

Re: [PERFORM] Sending a select to multiple servers.

2005-08-26 Thread Frank Wiles
On Fri, 26 Aug 2005 20:54:09 +0530 Ligesh <[EMAIL PROTECTED]> wrote: > > I would like to know if the following kind of database client exists: > I need a 'select' query to be sent to say 10 db servers > simultaneously in parallel (using threading), the results should be > re-sorted and return

Re: [PERFORM] difference in plan between 8.0 and 8.1?

2005-08-26 Thread Merlin Moncure
> Hello all, > > I was hoping someone could explain the plan for a statement. > > We have a table with a column of longs being used as an index. The > query plan in 8.0 was like this: > > # explain select distinct timeseriesid from tbltimeseries where > timeseriesid > 0 order by timeseriesid;

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Tom Lane
Mark Kirkwood <[EMAIL PROTECTED]> writes: > What is interesting is why this plan is being rejected... Which PG version are you using exactly? That mistake looks like an artifact of the 8.0 "fuzzy plan cost" patch, which we fixed recently: http://archives.postgresql.org/pgsql-committers/2005-07/ms

[PERFORM] Sending a select to multiple servers.

2005-08-26 Thread Ligesh
I would like to know if the following kind of database client exists: I need a 'select' query to be sent to say 10 db servers simultaneously in parallel (using threading), the results should be re-sorted and returned. For example I have a query: 'select * from table where parent_clname = 'pare

Re: [PERFORM] difference in plan between 8.0 and 8.1?

2005-08-26 Thread Tom Lane
Alan Stange <[EMAIL PROTECTED]> writes: > Unique (cost=2717137.08..2771407.21 rows=10854026 width=8) >-> Sort (cost=2717137.08..2744272.14 rows=10854026 width=8) > Sort Key: timeseriesid > -> Bitmap Heap Scan on tbltimeseries > (cost=48714.09..1331000.42 rows=10854026 w

Re: [PERFORM] difference in plan between 8.0 and 8.1?

2005-08-26 Thread Michael Fuhr
On Fri, Aug 26, 2005 at 10:45:07AM -0400, Alan Stange wrote: > -> Bitmap Heap Scan on tbltimeseries (cost=48714.09..1331000.42 > rows=10854026 width=8) > Recheck Cond: (timeseriesid > 0) > -> Bitmap Index Scan on idx_timeseris (cost=0.00..48714.09 > rows=1085

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Merlin Moncure
Mark Kirkwood > > The 'desc' seems to be the guy triggering the sort, e.g: > > Oh; really an accident that I didn't notice myself, I was actually going > to > remove all instances of "desc" in my simplification, but seems like I > forgot. If desc is the problem you can push the query into a subqu

[PERFORM] difference in plan between 8.0 and 8.1?

2005-08-26 Thread Alan Stange
Hello all, I was hoping someone could explain the plan for a statement. We have a table with a column of longs being used as an index. The query plan in 8.0 was like this: # explain select distinct timeseriesid from tbltimeseries where timeseriesid > 0 order by timeseriesid; SET

Re: [PERFORM] Limit + group + join

2005-08-26 Thread Stephan Szabo
On Fri, 26 Aug 2005, Mark Kirkwood wrote: > However being a bit brutal: > > set enable_mergejoin=false; > set enable_hashjoin=false; > > explain select c.id from c join b on c_id=c.id group by c.id order by > c.id desc limit 5; > QUERY PLAN > > ---

Re: [PERFORM] Inefficient queryplan for query with intersectable

2005-08-26 Thread Richard Huxton
Arjen van der Meijden wrote: I left all the configuration-stuff to the defaults since changing values didn't seem to impact much. Apart from the buffers and effective cache, increasing those made the performance worse. I've not looked at the rest of your problem in detail, but using the def

[PERFORM] Inefficient queryplan for query with intersectable subselects/joins

2005-08-26 Thread Arjen van der Meijden
Hi list, I'm writing an application that will aggregate records with a few million records into averages/sums/minimums etc grouped per day. Clients can add filters and do lots of customization on what they want to see. And I've to translate that to one or more queries. Basically, I append ea

Re: [PERFORM] postmaster memory keep going up????

2005-08-26 Thread Richard Huxton
Chun Yit(Chronos) wrote: >>I have a pl/pgsql function that using temp table to perform searching logic, >>my question is,it is postmaster have memory leaking problem? First step - upgrade to the latest 7.4.x release. Second step - read the "release notes" section of the manuals for 7.4.x and 8.0