Re: [PERFORM] parameterized LIKE does not use index

2005-06-23 Thread Kurt De Grave
> > Of course, I could modify the application and send different SQL > > depending on which case we're in or just constructing a query with a > > literal each time, but is there a way to add a hint to the SQL that > > would cause the query to be re-planned if it's a case that could use the > > ind

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Radu-Adrian Popescu
There have been problems with Xeon processors. Can you elaborate on that please ? Thanks, -- Radu-Adrian Popescu CSA, DBA, Developer Aldrapay MD Aldratech Ltd. +40213212243 smime.p7s Description: S/MIME Cryptographic Signature

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Keith Worthington
Radu-Adrian Popescu wrote: There have been problems with Xeon processors. Can you elaborate on that please ? Thanks, Not really as I do not understand the issue. Here is one post from the archives. http://archives.postgresql.org/pgsql-performance/2005-05/msg00441.php If you search the ar

Re: [PERFORM] Querying 19million records very slowly

2005-06-23 Thread Kjell Tore Fossbakk
-I also changed now() to current_time, which increased performance quite good. I need to make further tests, before I'll post any results. -I tried to change now()- interval 'x hours' to like 2005-06-22 16:00:00+02. This also increased the performance.   changing to time > '2005-06-22 16:00:00+02'

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Dave Cramer
My understanding is that it isn't particularly XEON processors that is the problem Any dual processor will exhibit the problem, XEON's with hyperthreading exacerbate the problem though and the good news is that it has been fixed in 8.1 Dave On 23-Jun-05, at 8:16 AM, Keith Worthington wrote

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Frank Wiles
On Wed, 22 Jun 2005 22:31:29 -0400 Keith Worthington <[EMAIL PROTECTED]> wrote: > Frank, > > A couple of things I wish I had been told when I started asking how to > > configure a new machine. > > Use RAID 10 (striping across mirrored disks) > or RAID 0+1 (mirror a striped array) for your dat

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Radu-Adrian Popescu
Dave Cramer wrote: My understanding is that it isn't particularly XEON processors that is the problem Any dual processor will exhibit the problem, XEON's with hyperthreading exacerbate the problem though and the good news is that it has been fixed in 8.1 Where's that ? The only informat

Re: [PERFORM] parameterized LIKE does not use index

2005-06-23 Thread Bruno Wolff III
On Thu, Jun 23, 2005 at 10:33:18 +0200, Kurt De Grave <[EMAIL PROTECTED]> wrote: > > Now it's tempting to dream of some mechanism that could make the > database consider > replanning the query automatically once it knows the parameter, or > choose from > a set of plans depending on the parameter

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Michael Stone
On Wed, Jun 22, 2005 at 10:31:29PM -0400, Keith Worthington wrote: Use RAID 10 (striping across mirrored disks) or RAID 0+1 (mirror a striped array) for your data. yikes! never tell an unsuspecting person to use mirred stripes--that configuration has lower reliability and performance than stri

[PERFORM] select distinct on varchar - wild performance differences!

2005-06-23 Thread Elliott Bennett
Hey, all. I've bounced this around in #postgres for an hour or so, and it was suggested that I post it here as well. Hopefully someone can help me out. I have three machines. All have 512MB of ram. Machine A is a 2.0ghz celeron, running debian, pg verison 7.4.6. Machine B is a 1.8ghz celeron, r

Re: [PERFORM] Querying 19million records very slowly

2005-06-23 Thread Tobias Brox
> I'm running FreeBSD 4.11, and im editing the file in > /usr/local/etc/postgresql.conf, but it doesnt help. On my system the 'live' config file resides in /var/lib/postgresql/data/postgresql.conf - maybe you have them in /usr/local/var/lib ... -- Tobias Brox, +86-13521622905 Nordicbet, IT dep

Re: [PERFORM] select distinct on varchar - wild performance differences!

2005-06-23 Thread Tom Lane
Elliott Bennett <[EMAIL PROTECTED]> writes: > Anyone have any ideas as to what may be up with machine B? Different locale setting? strcoll() can be horribly slow in some locales ... regards, tom lane ---(end of broadcast)--

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Dave Cramer
AFAIK, the problem was the buffer manager Dave On 23-Jun-05, at 9:46 AM, Radu-Adrian Popescu wrote: Dave Cramer wrote: My understanding is that it isn't particularly XEON processors that is the problem Any dual processor will exhibit the problem, XEON's with hyperthreading exacerbate the

Re: [PERFORM] select distinct on varchar - wild performance differences!

2005-06-23 Thread Elliott Bennett
hah! That did it. Setting to 'C' makes it just as fast as the other machines. I think it defaulted to en_US... Thanks! -Elliott On Thu, Jun 23, 2005 at 11:34:55AM -0400, Tom Lane wrote: > Elliott Bennett <[EMAIL PROTECTED]> writes: > > Anyone have any ideas as to what may be up with machine B

Re: [PERFORM] Performance Tuning Article

2005-06-23 Thread Tom Lane
Dave Cramer <[EMAIL PROTECTED]> writes: > AFAIK, the problem was the buffer manager The buffer manager was the place that seemed to be hit hardest by Xeon's problems with spinlock contention. I think we've partially fixed that issue in 8.1, but as we continue to improve the system's performance,

Re: [PERFORM] parameterized LIKE does not use index

2005-06-23 Thread Josh Berkus
Bruno, > I remember some discussion about delaying planning until the first > actual query so that planning could use actual parameters to do > the planning. If you really want to have it check the parameters > every time, I think you will need to replan every time. I don't > know if there is a wa

[PERFORM] ETL optimization

2005-06-23 Thread Bricklen Anderson
Situation: I'm trying to optimize an ETL process with many upserts (~100k aggregated rows) (no duplicates allowed). The source (table t2) table holds around 14 million rows, and I'm grabbing them 100,000 rows at a time from t2, resulting in about 100,000 distinct rows in the destination table (t1).

Re: [PERFORM] ETL optimization

2005-06-23 Thread Meetesh Karia
I don't know what this will change wrt how often you need to run VACUUM (I'm a SQL Server guy), but instead of an update and insert, try a delete and insert.  You'll only have to find the duplicate rows once and your insert doesn't need a where clause. MeeteshOn 6/23/05, Bricklen Anderson <[EMAIL

Re: [PERFORM] ETL optimization

2005-06-23 Thread Bricklen Anderson
Meetesh Karia wrote: > I don't know what this will change wrt how often you need to run VACUUM > (I'm a SQL Server guy), but instead of an update and insert, try a > delete and insert. You'll only have to find the duplicate rows once and > your insert doesn't need a where clause. > > Meetesh > V

Re: [PERFORM] ETL optimization

2005-06-23 Thread Jacques Caron
Hi, At 21:38 23/06/2005, Bricklen Anderson wrote: Situation: I'm trying to optimize an ETL process with many upserts (~100k aggregated rows) (no duplicates allowed). The source (table t2) table holds around 14 million rows, and I'm grabbing them 100,000 rows at a time from t2, resulting in abo

Re: [PERFORM] ETL optimization

2005-06-23 Thread Bricklen Anderson
Jacques Caron wrote: > > I have a similar situation, and the solution I use (though I haven't > really tested many different situations): > - have a trigger ON INSERT which does: > UPDATE set whatever_value=NEW.whatever_value,... WHERE > whatever_key=NEW.whatever.key AND... > IF FOUND THEN > RETU

Re: [PERFORM] ETL optimization

2005-06-23 Thread Dennis Bjorklund
On Thu, 23 Jun 2005, Bricklen Anderson wrote: > iii. UNIQUE constraint on table "t1". This didn't seem to perform too > badly with fewer rows (preliminary tests), but as you'd expect, on error > the whole transaction would roll back. Is it possible to skip a row if > it causes an error, as opposed

[PERFORM] Postgres 8 vs Postgres 7.4/cygwin

2005-06-23 Thread Scott Goldstein
I'm currently trying to make a decision on whether to use the Cygwin port of Postgres 7.4 or Postgres 8.0 for a windows installation.  Can someone provide some comparison info from a performance point of view?  I was thinking that the Cygwin port has the overhead of the translation layer, but 8.0 i

Re: [PERFORM] Postgres 8 vs Postgres 7.4/cygwin

2005-06-23 Thread Christopher Kings-Lynne
PostgreSQL 8 for windows faster AND more reliable :) Chris Scott Goldstein wrote: I'm currently trying to make a decision on whether to use the Cygwin port of Postgres 7.4 or Postgres 8.0 for a windows installation. Can someone provide some comparison info from a performance point of view?