Re: [PERFORM] Performance of pg_dump on PGSQL 8.0

2006-06-15 Thread Merlin Moncure
Just couple of suggestions: I think on the current server you're pretty much hosed since you are look like you are cpu bottlenecked. You probably should take a good look at PITR and see if that meets your requirements. Also you definately want to go to 8.1...it's faster, and every bit helps. G

Re: [PERFORM] Optimizer internals

2006-06-15 Thread Mischa Sandberg
Mark Lewis wrote: On Thu, 2006-06-15 at 14:05 -0400, John Vincent wrote: Now I've been told by our DBA that we should have been able to wholy satisfy that query via the indexes. DB2 can satisfy the query using only indexes because DB2 doesn't do MVCC. You can get pretty much the same effect

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread Alex Turner
Given the fact that most SATA drives have only an 8MB cache, and your RAID controller should have at least 64MB, I would argue that the system with the RAID controller should always be faster.  If it's not, you're getting short-changed somewhere, which is typical on linux, because the drivers just

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread Mark Lewis
On Thu, 2006-06-15 at 18:24 -0400, Tom Lane wrote: > I agree with Brian's suspicion that the SATA drive isn't properly > fsync'ing to disk, resulting in bogusly high throughput. However, > ISTM a well-configured SAN ought to be able to match even the bogus > throughput, because it should be able t

Re: [PERFORM] Optimizer internals

2006-06-15 Thread John Vincent
decibel=# create index test on i ( sum(i) );ERROR:  cannot use aggregate function in index _expression_ decibel=#BTW, there have been a number of proposals to negate the effect of nothaving visibility info in indexes. Unfortunately, none of them have cometo fruition yet, mostly because it's a very

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread Tom Lane
Brian Hurt <[EMAIL PROTECTED]> writes: > Tim Allen wrote: >> To simplify greatly - single local SATA disk beats EMC SAN by factor >> of four. > I'm actually in a not dissimiliar position here- I was seeing the > performance of Postgres going to an EMC Raid over iSCSI running at about > 1/2 the

Re: [PERFORM] Optimizer internals

2006-06-15 Thread Jim C. Nasby
On Thu, Jun 15, 2006 at 03:43:09PM -0400, John Vincent wrote: > >Any suggestions? FYI the original question wasn't meant as a poke at > >comparing PG to MySQL to DB2. I'm not making an yvalue judgements either > >way. I'm just trying to understand how we can use it the best way possible. > > > >If

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread John Vincent
On 6/15/06, Tim Allen <[EMAIL PROTECTED]> wrote: Is that expected performance, anyone? It doesn't sound right to me. Doesanyone have any clues about what might be going on? Buggy kerneldrivers? Buggy kernel, come to think of it? Does a SAN just not provide adequate performance for a large database?

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread Brian Hurt
Tim Allen wrote: We have a customer who are having performance problems. They have a large (36G+) postgres 8.1.3 database installed on an 8-way opteron with 8G RAM, attached to an EMC SAN via fibre-channel (I don't have details of the EMC SAN model, or the type of fibre-channel card at the mo

Re: [PERFORM] SAN performance mystery

2006-06-15 Thread Scott Marlowe
On Thu, 2006-06-15 at 16:50, Tim Allen wrote: > We have a customer who are having performance problems. They have a > large (36G+) postgres 8.1.3 database installed on an 8-way opteron with > 8G RAM, attached to an EMC SAN via fibre-channel (I don't have details > of the EMC SAN model, or the ty

[PERFORM] SAN performance mystery

2006-06-15 Thread Tim Allen
We have a customer who are having performance problems. They have a large (36G+) postgres 8.1.3 database installed on an 8-way opteron with 8G RAM, attached to an EMC SAN via fibre-channel (I don't have details of the EMC SAN model, or the type of fibre-channel card at the moment). They're runn

Re: [PERFORM] Optimizer internals

2006-06-15 Thread John Vincent
Any suggestions? FYI the original question wasn't meant as a poke at comparing PG to MySQL to DB2. I'm not making an yvalue judgements either way. I'm just trying to understand how we can use it the best way possible. If anyone from the bizgres team is watching, have they done any work in this area

Re: [PERFORM] Optimizer internals

2006-06-15 Thread Scott Marlowe
On Thu, 2006-06-15 at 14:21, John Vincent wrote: > On 6/15/06, Mark Lewis <[EMAIL PROTECTED]> wrote: > Unfortunately SUM is in the same boat as COUNT; in order for > it to > return a meaningful result it must inspect visibility > information for > all of the

Re: [PERFORM] Optimizer internals

2006-06-15 Thread John Vincent
On 6/15/06, Mark Lewis <[EMAIL PROTECTED]> wrote: Unfortunately SUM is in the same boat as COUNT; in order for it toreturn a meaningful result it must inspect visibility information forall of the rows.-- MarkWe'll this is interesting news to say the least. We went with PostgreSQL for our warehouse

Re: [PERFORM] Optimizer internals

2006-06-15 Thread Mark Lewis
On Thu, 2006-06-15 at 14:46 -0400, John Vincent wrote: > One question that we came up with is how does this affect other > aggregate functions like MAX,MIN,SUM and whatnot? Being that this is > our data warehouse, we use these all the time. As I've said > previously, I didn't know a human could ge

Re: [PERFORM] Optimizer internals

2006-06-15 Thread John Vincent
On 6/15/06, Mark Lewis <[EMAIL PROTECTED]> wrote: DB2 can satisfy the query using only indexes because DB2 doesn't doMVCC.Although MVCC is generally a win in terms of making the database easierto use and applications less brittle, it also means that the database must inspect the visibility informat

Re: [PERFORM] Optimizer internals

2006-06-15 Thread Mark Lewis
On Thu, 2006-06-15 at 14:05 -0400, John Vincent wrote: > Now I've been told by our DBA that we should have been able to wholy > satisfy that query via the indexes. DB2 can satisfy the query using only indexes because DB2 doesn't do MVCC. Although MVCC is generally a win in terms of making the dat

Re: [PERFORM] Precomputed constants?

2006-06-15 Thread Zoltan Boszormenyi
Jim C. Nasby írta: On Thu, Jun 15, 2006 at 06:31:02AM +0200, Zoltan Boszormenyi wrote: Jim C. Nasby ?rta: On Wed, Jun 14, 2006 at 01:30:10PM +0200, B?sz?rm?nyi Zolt?n wrote: Replacing random() with a true constant gives me index scan even if it's hidden inside other function c

[PERFORM] Optimizer internals

2006-06-15 Thread John Vincent
I'm not a programmer so understanding the optimizer code is WAY beyond my limits.My question, that I haven't seen answered elsewhere, is WHAT things can affect the choice of an index scan over a sequence scan. I understand that sometimes a sequence scan is faster and that you still have to get the

Re: [PERFORM]Is it possible to start two instances of postgresql?

2006-06-15 Thread Bill Moran
In response to Dan Harris <[EMAIL PROTECTED]>: > > > [EMAIL PROTECTED] wrote: > > > >> both of the two database are live but use for two different web app. > >> my company don't want to spend more to buy a new server, so then I think of > >> to implement both under the same server and one instance

Re: [PERFORM] Which processor runs better for Postgresql?

2006-06-15 Thread Vivek Khera
On Jun 15, 2006, at 1:10 PM, Steve Poe wrote: Vivek, Thanks for your feedback. Which Dell server did you purchase? I have many many dell rackmounts: 1550, 1650, 1750, 1850, and SC1425 and throw in a couple of 2450. I *really* like the 1850 with built-in SCSI RAID. It is fast enough t

Re: [PERFORM]Is it possible to start two instances of postgresql?

2006-06-15 Thread Dan Harris
[EMAIL PROTECTED] wrote: both of the two database are live but use for two different web app. my company don't want to spend more to buy a new server, so then I think of to implement both under the same server and one instance.. Just as an anecdote, I am running 30 databases on a single i

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread Bruce Momjian
Added to TODO: > o Fix memory leak from exceptions > >http://archives.postgresql.org/pgsql-performance/2006-06/msg0$ --- Tom Lane wrote: > "jody brownell" <[EMAIL PROTECTED]> writes: > > B

Re: [PERFORM] Which processor runs better for Postgresql?

2006-06-15 Thread Steve Poe
Vivek, Thanks for your feedback. Which Dell server did you purchase? The client has a PowerEdge 2600 and they STILL want Dell. Again, if it were my pocketbook, Dell would not be there. The client has a 30GB DB. This is large for me, but probably not with you. Also, I am advising the client t

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread Mark Lewis
On Thu, 2006-06-15 at 11:34 -0400, Tom Lane wrote: > "jody brownell" <[EMAIL PROTECTED]> writes: > > When postgresql starts to go into this bloating state, I can only make it > > happen from my java app. > > That's interesting. The JDBC driver uses protocol features that aren't > used by psql, s

Re: [PERFORM] Which processor runs better for Postgresql?

2006-06-15 Thread Vivek Khera
On Jun 13, 2006, at 2:02 PM, Steve Poe wrote: Can anyone share what their experience has been with Intel's dual core CPUs and/or Dell's new servers? I'm one of the few Dell fans around here... but I must say that I don't buy them for my big DB servers specifically since they don't curren

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread Tom Lane
"jody brownell" <[EMAIL PROTECTED]> writes: > BTW - the fix you mentioned is that targeted for 8.2? Is there a > timeline for 8.2? There is no fix as yet, but it's on the radar screen to fix for 8.2. We expect 8.2 will go beta towards the end of summer (I forget whether Aug 1 or Sep 1 is th

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread jody brownell
Tom - that make sense... and fits the timeline of when the instability may have been introduced. I use soft references in java to track these relationships. When the GC needs memory it will collect objects referenced by soft references so I need to have this exception caught where my caches may

Re: [PERFORM] Precomputed constants?

2006-06-15 Thread Jim C. Nasby
On Thu, Jun 15, 2006 at 06:31:02AM +0200, Zoltan Boszormenyi wrote: > Jim C. Nasby ?rta: > >On Wed, Jun 14, 2006 at 01:30:10PM +0200, B?sz?rm?nyi Zolt?n wrote: > > > >>Replacing random() with a true constant gives me index scan > >>even if it's hidden inside other function calls. E.g.: > >> >

Re: [PERFORM] Postgres fsync off (not needed) with NetApp

2006-06-15 Thread Jim C. Nasby
On Thu, Jun 15, 2006 at 01:14:26AM -0400, Jonah H. Harris wrote: > On 14 Jun 2006 23:33:53 -0400, Greg Stark <[EMAIL PROTECTED]> wrote: > >In fact the benefit of the NVRAM is precisely that it makes sure you > >*don't* > >have any reason to turn fsync off. It should make the fsync essentially > >

Re: [PERFORM] Confirmation of bad query plan generated by 7.4

2006-06-15 Thread Jim C. Nasby
On Wed, Jun 14, 2006 at 10:36:55PM -0400, Tom Lane wrote: > Jim Nasby <[EMAIL PROTECTED]> writes: > > On Jun 13, 2006, at 8:50 PM, Tom Lane wrote: > >> Hmm ... worksforme. Could you provide a complete test case? > > > decibel=# create table date_test(d date not null, i int not null); > > [etc] >

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread Tom Lane
"jody brownell" <[EMAIL PROTECTED]> writes: > BEGIN > INSERT into attacker_target_link (attacker_id, target_id) values > (p_attacker, v_target); > v_returns_size := v_returns_size + 1; > v_returns[v_returns_size] := v_target; > EXCEPTION WHEN un

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread Tom Lane
"jody brownell" <[EMAIL PROTECTED]> writes: > When postgresql starts to go into this bloating state, I can only make it > happen from my java app. That's interesting. The JDBC driver uses protocol features that aren't used by psql, so it's possible that the leak is triggered by one of those feat

Re: [PERFORM] Is it possible to start two instances of postgresql?

2006-06-15 Thread Chris Browne
[EMAIL PROTECTED] writes: > Is it possible to start two instances of postgresql with different port and > directory which run simultaneously? Certainly. We have one HACMP cluster which hosts 14 PostgreSQL instances across two physical boxes. (If one went down, they'd all migrate to the survivor.

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread jody brownell
Some more information... When postgresql starts to go into this bloating state, I can only make it happen from my java app. If I simultaneously perform insert of 10million rows into another table, it behaves as expected, but the postgresql process handling the java connection slows down and blo

Re: [PERFORM] How to analyze function performance

2006-06-15 Thread Tom Lane
"Mindaugas" <[EMAIL PROTECTED]> writes: > Is it possible to somehow analyze function performance? E.g. > we are using function cleanup() which takes obviously too much time > to execute but I have problems trying to figure what is slowing things > down. > When I explain analyze function lines

Re: [PERFORM] How to analyze function performance

2006-06-15 Thread Tomas Vondra
It depends what is the purpose of the function. If it's mainly a container for a heap of SQL queries along with some simple IF, ELSE etc. then I use two simple ways to analyze the performance (or lack of performance): 1) I use a lot of debug messages 2) I print out all SQL and the execute EXPLAIN

[PERFORM] How to analyze function performance

2006-06-15 Thread Mindaugas
Hello, Is it possible to somehow analyze function performance? E.g. we are using function cleanup() which takes obviously too much time to execute but I have problems trying to figure what is slowing things down. When I explain analyze function lines step by step it show quite acceptable p

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread jody brownell
The last version of postgres we had in production was 8.1.1 actually, not 8.1.3. So far, on my stability box and older production stability boxes I dont see the same behavior. I will install 8.1.1 on these boxes and see what I see. On Thursday 15 June 2006 09:01, jody brownell wrote: > Sorry

Re: [PERFORM] Postgres consuming way too much memory???

2006-06-15 Thread jody brownell
Sorry about that, I was in a slight panic :) I am using postgresql 8.1.4. I will install 8.1.3 and see if the same behavior exists.. we may have started seeing this in 8.1.3, but I dont think before. I will check some stability machines for similar bloating. The query (calling a store proc)

Re: [PERFORM]Is it possible to start two instances of postgresql?

2006-06-15 Thread Nis Jorgensen
[EMAIL PROTECTED] wrote: > both of the two database are live but use for two different web app. > my company don't want to spend more to buy a new server, so then I think of > to implement both under the same server and one instance.. > but then my superior don't want to do that way. > they want

Re: [PERFORM]Is it possible to start two instances of postgresql?

2006-06-15 Thread kah_hang_ang
both of the two database are live but use for two different web app. my company don't want to spend more to buy a new server, so then I think of to implement both under the same server and one instance.. but then my superior don't want to do that way. they want to implement two databases in o

Re: [PERFORM]Is it possible to start two instances of postgresql?

2006-06-15 Thread A. Kretschmer
am 15.06.2006, um 14:34:51 +0800 mailte [EMAIL PROTECTED] folgendes: > > > > > so what is the best way to implement two databases in one machine? > implement with two postgresql instances with separate directory or > implement under one instance? What do you want to do? Do you need 2 separate