Re: [PERFORM] Array indexes, GIN?

2007-03-01 Thread Adam L Beberg
Oleg Bartunov wrote on 3/1/2007 10:45 PM: On Thu, 1 Mar 2007, Josh Berkus wrote: Adam, On the surface, looks like a job for GIN, but GIN seems undocumented, specifically mentions it doesn't support the deletes we'll have many of since it's designed for word searching apparently, the performan

Re: [PERFORM] Array indexes, GIN?

2007-03-01 Thread Oleg Bartunov
On Thu, 1 Mar 2007, Josh Berkus wrote: Adam, On the surface, looks like a job for GIN, but GIN seems undocumented, specifically mentions it doesn't support the deletes we'll have many of since it's designed for word searching apparently, the performance implications are undocumented. I searche

Re: [PERFORM] Improving query performance

2007-03-01 Thread Tom Lane
David Leangen <[EMAIL PROTECTED]> writes: > And this is the actual query: > SELECT COUNT(t0.ID) FROM public.MY_TABLE t0 > WHERE ((POSITION('adenosine cyclic 35-monophosphate' IN LOWER(t0.TITLE)) > - 1) >=0 OR > (POSITION('adenosine cyclic 55-monophosphate' IN LOWER(t0.TEXT)) - 1) >= > 0 OR > (P

Re: [PERFORM] Array indexes, GIN?

2007-03-01 Thread Josh Berkus
Adam, > On the surface, looks like a job for GIN, but GIN seems undocumented, > specifically mentions it doesn't support the deletes we'll have many of > since it's designed for word searching apparently, the performance > implications are undocumented. I searched, I read, and even IRC'd, and > it

[PERFORM] Array indexes, GIN?

2007-03-01 Thread Adam L Beberg
I need to cross reference 2 tables. There are O(10M) A's, each has an ordered set of 10 of the O(100K) B's associated with it. The dominant query will be finding the A's and their count associated with a given list of ~1k B's i.e. if 2 of the listed B's are in A's set of 10, it's (A,2), and we

[PERFORM] Improving query performance

2007-03-01 Thread David Leangen
Hello! I'm new to performance tuning on postgres. I've read the docs on the posgtres site, as well as: http://www.revsys.com/writings/postgresql-performance.html http://www.powerpostgresql.com/PerfList However, my query is really slow, and I'm not sure what the main cause could be, as there

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Alex Deucher wrote: > On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: >> \ >> >> Is the SAN being shared between the database servers and other >> >> servers? Maybe >> >> it was just random timing that gave you the poor write performanc

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
Alex Deucher wrote: > On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: >> \ >> >> Is the SAN being shared between the database servers and other >> >> servers? Maybe >> >> it was just random timing that gave you the poor write performance on >> >> the old >> >> server which might be also yiel

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: > On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: >> On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> >> >> Postgresql might be choosing a bad plan because your >> >> effective_cache_size >> >> >> is >> >> >>

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> Postgresql might be choosing a bad plan because your >> effective_cache_size >> >> is >> >> way off (it's the default now right?). Also, what was the block >>

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: \ >> Is the SAN being shared between the database servers and other >> servers? Maybe >> it was just random timing that gave you the poor write performance on >> the old >> server which might be also yielding occassional poor performance on >

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
\ >> Is the SAN being shared between the database servers and other >> servers? Maybe >> it was just random timing that gave you the poor write performance on >> the old >> server which might be also yielding occassional poor performance on >> the new >> one. >> > > The direct attached scsi discs

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> Postgresql might be choosing a bad plan because your >> effective_cache_size >> >> is >> >> way off (it's the default now right?). Also, what was the block >> read/write >> > >> > yes it's set to the

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > On 3/2/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> "Merlin Moncure" <[EMAIL PROTECTED]> writes: >>> I think this explains the trigger that was blowing up my FC4 box. >> >> I dug in the archives a bit and couldn't find the report you're >> referring to?

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Merlin Moncure
On 3/2/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Merlin Moncure" <[EMAIL PROTECTED]> writes: > On 3/1/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> A further report led us to realize that 8.2.x in fact has a nasty bug >> here: the stats collector is supposed to dump its stats to a file at >> most eve

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Ron
At 07:36 PM 3/1/2007, Jeff Frost wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Postgresql might be choosing a bad plan because your effective_cache_size >> is >> way off (it's the default now right?). Also, what was the block read/write > > yes it's set to the default. > >> speed of the

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: >> Postgresql might be choosing a bad plan because your effective_cache_size >> is >> way off (it's the default now right?). Also, what was the block read/write > > yes it's set to the default. > >> speed of the SAN from your bonnie tests? Probably wa

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: here are some examples. Analyze is still running on the new db, I'll post results when that is done. Mostly what our apps do is prepared row selects from different tables: select c1,c2,c3,c4,c5 from t1 where c1='XXX'; old server: db=# EXPLAIN ANALYZE se

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large database (~16 GB). The >> original postgres 7.4

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > On 3/1/07, Tom Lane <[EMAIL PROTECTED]> wrote: >> A further report led us to realize that 8.2.x in fact has a nasty bug >> here: the stats collector is supposed to dump its stats to a file at >> most every 500 milliseconds, but the code was actually wa

Re: [PERFORM] Identical Queries

2007-03-01 Thread Craig A. James
Stephan Szabo wrote: I tried posting to the bugs, and they said this is a better question for here. I have to queries. One runs in about 2 seconds. The other takes upwards of 2 minutes. I have a temp table that is created with 2 columns. This table is joined with the larger database of call detai

Re: [PERFORM] Identical Queries

2007-03-01 Thread Stephan Szabo
On Thu, 1 Mar 2007, Rob Schall wrote: > There are 4 entries (wanted to make the playing field level for this > test). There are 2 with true for istf and 2 with false. Then analyzing might help, because I think it's estimating many more rows for both cases, and with 2 rows estimated to be returned

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: > On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: >> On Thu, 1 Mar 2007, Alex Deucher wrote: >> >> >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> >> effective_cache_size? work_m

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> effective_cache_size? work_mem? >> > > I'm running the autovacuum process on the 8.1 serv

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Merlin Moncure
On 3/1/07, Tom Lane <[EMAIL PROTECTED]> wrote: Benjamin Minshall <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> It's sounding like what you had was just transient bloat, in which case >> it might be useful to inquire whether anything out-of-the-ordinary had >> been done to the database right be

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Alex Deucher wrote: >> Vacuum? Analayze? default_statistics_target? How many shared_buffers? >> effective_cache_size? work_mem? >> > > I'm running the autovacuum process on the 8.1 server. vacuuming on > the old server was don

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: Vacuum? Analayze? default_statistics_target? How many shared_buffers? effective_cache_size? work_mem? I'm running the autovacuum process on the 8.1 server. vacuuming on the old server was done manually. default_statistics_target and effective_cache_si

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Alex Deucher wrote: On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large databas

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Jeff Frost <[EMAIL PROTECTED]> wrote: On Thu, 1 Mar 2007, Joshua D. Drake wrote: > Alex Deucher wrote: >> Hello, >> >> I have noticed a strange performance regression and I'm at a loss as >> to what's happening. We have a fairly large database (~16 GB). The >> original postgres 7.4

Re: [PERFORM] Identical Queries

2007-03-01 Thread Chad Wagner
On 3/1/07, Rob Schall <[EMAIL PROTECTED]> wrote: There are 4 entries (wanted to make the playing field level for this test). There are 2 with true for istf and 2 with false. Then the difference here has to do with using orignum vs destnum as the join criteria. There must be more intersection

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
On 3/1/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Alex Deucher wrote: > Hello, > > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). The > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Jeff Frost
On Thu, 1 Mar 2007, Joshua D. Drake wrote: Alex Deucher wrote: Hello, I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). The original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB of ram running So

Re: [PERFORM] Identical Queries

2007-03-01 Thread Rob Schall
There are 4 entries (wanted to make the playing field level for this test). There are 2 with true for istf and 2 with false. Rob Stephan Szabo wrote: > On Thu, 1 Mar 2007, Rob Schall wrote: > > >> Question for anyone... >> >> I tried posting to the bugs, and they said this is a better questio

Re: [PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Joshua D. Drake
Alex Deucher wrote: > Hello, > > I have noticed a strange performance regression and I'm at a loss as > to what's happening. We have a fairly large database (~16 GB). The > original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB > of ram running Solaris on local scsi discs. The new

[PERFORM] strange performance regression between 7.4 and 8.1

2007-03-01 Thread Alex Deucher
Hello, I have noticed a strange performance regression and I'm at a loss as to what's happening. We have a fairly large database (~16 GB). The original postgres 7.4 was running on a sun v880 with 4 CPUs and 8 GB of ram running Solaris on local scsi discs. The new server is a sun Opteron box wi

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Benjamin Minshall
Tom Lane wrote: A further report led us to realize that 8.2.x in fact has a nasty bug here: the stats collector is supposed to dump its stats to a file at most every 500 milliseconds, but the code was actually waiting only 500 microseconds :-(. The larger the stats file, the more obvious this pr

Re: [PERFORM] stats collector process high CPU utilization

2007-03-01 Thread Tom Lane
Benjamin Minshall <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> It's sounding like what you had was just transient bloat, in which case >> it might be useful to inquire whether anything out-of-the-ordinary had >> been done to the database right before the excessive-CPU-usage problem >> started.

Re: [PERFORM] Identical Queries

2007-03-01 Thread Stephan Szabo
On Thu, 1 Mar 2007, Rob Schall wrote: > Question for anyone... > > I tried posting to the bugs, and they said this is a better question for here. > I have to queries. One runs in about 2 seconds. The other takes upwards > of 2 minutes. I have a temp table that is created with 2 columns. This > tab

Re: [PERFORM] increasing database connections

2007-03-01 Thread Joshua D. Drake
Magnus Hagander wrote: > On Thu, Mar 01, 2007 at 12:49:14AM -0500, Jonah H. Harris wrote: >> On 3/1/07, Shiva Sarna <[EMAIL PROTECTED]> wrote: >>> I am sorry if it is a repeat question but I want to know if database >>> performance will decrease if I increase the max-connections to 2000. At >>> pre

[PERFORM] Identical Queries

2007-03-01 Thread Rob Schall
Question for anyone... I tried posting to the bugs, and they said this is a better question for here. I have to queries. One runs in about 2 seconds. The other takes upwards of 2 minutes. I have a temp table that is created with 2 columns. This table is joined with the larger database of call deta

[PERFORM] Performance Query

2007-03-01 Thread Abu Mushayeed
1. The function: SELECT a.birth_date FROM ( SELECT indiv_fkey, birth_dt as birth_date, intern_last_update::date as last_update, 'fed' as source FROM cdm.cdm_fedcustomer WHERE birth_dt IS NOT NULL AND indiv_fkey = $1 UNION SELECT indiv_fkey, birthdate as birth_date, last_up

Re: [PERFORM] increasing database connections

2007-03-01 Thread Ron
At 01:18 AM 3/1/2007, Joshua D. Drake wrote: Jonah H. Harris wrote: > On 3/1/07, Shiva Sarna <[EMAIL PROTECTED]> wrote: >> I am sorry if it is a repeat question but I want to know if database >> performance will decrease if I increase the max-connections to 2000. At >> present it is 100. > > Most

Re: [PERFORM] increasing database connections

2007-03-01 Thread Florian Weimer
* Mark Kirkwood: > Yeah - I thought that somewhere closer to 1 connections is where > you get hit with socket management related performance issues. Huh? These sockets aren't handled in a single process, are they? Nowadays, this number of sockets does not pose any problem for most systems, e

Re: [PERFORM] increasing database connections

2007-03-01 Thread Magnus Hagander
On Thu, Mar 01, 2007 at 12:49:14AM -0500, Jonah H. Harris wrote: > On 3/1/07, Shiva Sarna <[EMAIL PROTECTED]> wrote: > >I am sorry if it is a repeat question but I want to know if database > >performance will decrease if I increase the max-connections to 2000. At > >present it is 100. > > Most cer