Re: [PERFORM] Newbie question: ultra fast count(*)

2005-11-28 Thread Merlin Moncure
> I have been reading all this technical talk about costs and such that > I don't (_yet_) understand. > > Now I'm scared... what's the fastest way to do an equivalent of > count(*) on a table to know how many items it has? Make sure to analyze the database frequently and check pg_class for reltup

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Luke Lonergan
The MPP test I ran was with the release version 2.0 of MPP which is based on Postgres 8.0, the upcoming 2.1 release is based on 8.1, and 8.1 is far faster at seq scan + agg. 12,937MB were counted in 4.5 seconds, or 2890MB/s from I/O cache. That's 722MB/s per host, and 360MB/s per Postgres instanc

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Merlin Moncure
> > It certainly makes quite a difference as I measure it: > > doing select(1) from a 181000 page table (completely uncached) on my PIII: > > 8.0 : 32 s > 8.1 : 25 s > > Note that the 'fastcount()' function takes 21 s in both cases - so all > the improvement seems to be from the count overhead

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Mark Kirkwood
Merlin Moncure wrote: It certainly makes quite a difference as I measure it: doing select(1) from a 181000 page table (completely uncached) on my PIII: 8.0 : 32 s 8.1 : 25 s Note that the 'fastcount()' function takes 21 s in both cases - so all the improvement seems to be from the count ove

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Luke Lonergan
Mark, On 11/28/05 1:45 PM, "Mark Kirkwood" <[EMAIL PROTECTED]> wrote: >>> 8.0 : 32 s >>> 8.1 : 25 s A 22% reduction. select count(1) on 12,900MB = 1617125 pages fully cached: MPP based on 8.0 : 6.06s MPP based on 8.1 : 4.45s A 26% reduction. I'll take it! I am looking to back-port Tom's pre

[PERFORM] Please help with this explain analyse...

2005-11-28 Thread David Gagnon
Hi all, I don't understand why this request take so long. Maybe I read the analyse correctly but It seem that the first line(Nested Loop Left Join ...) take all the time. But I don't understand where the performance problem is ??? All the time is passed in the first line ... Thanks for

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Brendan Duddridge
Forgive my ignorance, but what is MPP? Is that part of Bizgres? Is it possible to upgrade from Postgres 8.1 to Bizgres? Thanks, Brendan Duddridge | CTO | 403-277-5591 x24 | [EMAIL PROTECTED] ClickSpace Interactive Inc. Suit

Re: [PERFORM] Please help with this explain analyse...

2005-11-28 Thread Bricklen Anderson
David Gagnon wrote: > " -> Index Scan using cr_pk on cr (cost=0.00..6.02 rows=1 > width=828) (actual time=0.073..0.077 rows=1 loops=13587)" > "Index Cond: (((cr.crypnum)::text = 'M'::text) AND > (cr.crnum = "outer".cscrnum))" > "Filter: ((crda

Re: [PERFORM] Please help with this explain analyse...

2005-11-28 Thread Tom Lane
Bricklen Anderson <[EMAIL PROTECTED]> writes: > Your loops are what is causing the time spent. > eg. "actual time=0.127..17.379 rows=1154 loops=8335)" == > 8335*(17.379-0.127)/1000=>143 secs (if my math is correct). As for where the problem is, I think it's the horrid misestimate of the number of

Re: [PERFORM] Please help with this explain analyse...

2005-11-28 Thread David Gagnon
I restored my db but haven't run the analyse... That was the problem. Thanks /David "Merge Left Join (cost=2273.54..2290.19 rows=228 width=816) (actual time=2098.257..2444.472 rows=8335 loops=1)" " Merge Cond: (("outer".rrnum)::text = "inner"."?column8?")" " -> Merge Join (cost=2131.25..2

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread Brendan Duddridge
Hi David, Thanks for your reply. So how is that different than something like Slony2 or pgcluster with multi-master replication? Is it similar technology? We're currently looking for a good clustering solution that will work on our Apple Xserves and Xserve RAIDs. Thanks,

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread David Boreham
Brendan Duddridge wrote: Thanks for your reply. So how is that different than something like Slony2 or pgcluster with multi-master replication? Is it similar technology? We're currently looking for a good clustering solution that will work on our Apple Xserves and Xserve RAIDs. I think yo

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread David Lang
On Mon, 28 Nov 2005, Brendan Duddridge wrote: Hi David, Thanks for your reply. So how is that different than something like Slony2 or pgcluster with multi-master replication? Is it similar technology? We're currently looking for a good clustering solution that will work on our Apple Xserves

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-28 Thread David Lang
On Mon, 28 Nov 2005, Brendan Duddridge wrote: Forgive my ignorance, but what is MPP? Is that part of Bizgres? Is it possible to upgrade from Postgres 8.1 to Bizgres? MPP is the Greenplum propriatary extention to postgres that spreads the data over multiple machines, (raid, but with entire mac

[PERFORM] index auto changes after copying data ?

2005-11-28 Thread [EMAIL PROTECTED]
I know in mysql, index will auto change after copying data Of course, index will change after inserting a line in postgresql, but what about copying data? ---(end of broadcast)--- TIP 6: explain analyze is your friend