[PERFORM] Slow update

2005-09-12 Thread Hilary Forbes
Hello everyone I must be doing something very wrong here so help please! I have two tables tableA has 300,000 recs tableB has 20,000 recs I need to set the value of a field in table A to a value in table B depending on the existence of the record in table B. So what I have done is UPDATE

Re: [PERFORM] LEFT JOIN optimization

2005-09-12 Thread Manfred Koizar
On Mon, 12 Sep 2005 00:47:57 +0300, Ksenia Marasanova [EMAIL PROTECTED] wrote: - Seq Scan on user_ (cost=0.00..7430.63 rows=12763 width=245) (actual time=360.431..1120.012 rows=12763 loops=1) If 12000 rows of the given size are stored in more than 7000 pages, then there is a lot of free

Re: [PERFORM] Slow update

2005-09-12 Thread Richard Huxton
Hilary Forbes wrote: I need to set the value of a field in table A to a value in table B depending on the existence of the record in table B. So what I have done is UPDATE tableA set a.val1=b.somefield FROM tableA a, tableB b WHERE a.key1=b.key1; Check the EXPLAIN carefully, are you sure

Re: [PERFORM] Prepared statement not using index

2005-09-12 Thread Dave Cramer
The difference between the 7.4 driver and the 8.0.3 driver is the 8.0.3 driver is using server side prepared statements and binding the parameter to the type in setXXX(n,val). The 7.4 driver just replaces the ? with the value and doesn't use server side prepared statements. Dave On

Re: [PERFORM] Prepared statement not using index

2005-09-12 Thread Guido Neitzer
On 12.09.2005, at 14:38 Uhr, Dave Cramer wrote: The difference between the 7.4 driver and the 8.0.3 driver is the 8.0.3 driver is using server side prepared statements and binding the parameter to the type in setXXX(n,val). Would be a good idea when this were configurable. I found my

Re: [PERFORM] Prepared statement not using index

2005-09-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The difference between the 7.4 driver and the 8.0.3 driver is the 8.0.3 driver is using server side prepared statements and binding the parameter to the type in setXXX(n,val). The 7.4 driver just replaces the ? with the value and doesn't use

Re: [PERFORM] Slow update

2005-09-12 Thread Bruno Wolff III
On Mon, Sep 12, 2005 at 10:14:25 +0100, Hilary Forbes [EMAIL PROTECTED] wrote: Hello everyone I must be doing something very wrong here so help please! I have two tables tableA has 300,000 recs tableB has 20,000 recs I need to set the value of a field in table A to a value in table B

Re: [PERFORM] Slow update

2005-09-12 Thread Tom Lane
Hilary Forbes [EMAIL PROTECTED] writes: I need to set the value of a field in table A to a value in table B depending on the existence of the record in table B. So what I have done is UPDATE tableA set a.val1=b.somefield FROM tableA a, tableB b WHERE a.key1=b.key1; You've written an

Re: [PERFORM] Prepared statement not using index

2005-09-12 Thread Dave Cramer
On 12-Sep-05, at 9:22 AM, Guido Neitzer wrote: On 12.09.2005, at 14:38 Uhr, Dave Cramer wrote: The difference between the 7.4 driver and the 8.0.3 driver is the 8.0.3 driver is using server side prepared statements and binding the parameter to the type in setXXX(n,val). Would be a

Re: [PERFORM] Prepared statement not using index

2005-09-12 Thread Dave Cramer
It's added, just use the old protocol . Here are the connection parameters http://jdbc.postgresql.org/documentation/head/connect.html#connection- parameters Dave On 12-Sep-05, at 9:26 AM, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The difference between

[PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Brandon Black
I'm in the process of developing an application which uses PostgreSQL for data storage. Our database traffic is very atypical, and as a result it has been rather challenging to figure out how to best tune PostgreSQL on what development hardware we have, as well as to figure out exactly what we

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread PFC
I know I haven't provided a whole lot of application-level detail here, You did ! What about : - using COPY instead of INSERT ? (should be easy to do from the aggregators) - using Bizgres ? (which was designed for your

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Qingqing Zhou
"Brandon Black" [EMAIL PROTECTED] wrote ... Increasing shared_buffers seems to always help, even out to half of the dev box's ram (2G). Though officially PG does not prefer huge shared_buffers size, I did see several times thatperformancewas boosted in case IO is the

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Alex Turner
Split your system into multiple partitions of RAID 10s.For max performance, ten drive RAID 10 for pg_xlog (This will max out a PCI-X bus) on Bus A, multiple 4/6Drive RAID 10s for tablespaces on Bus B. For max performance I would recommend using one RAID 10 for raw data tables, one for aggregate

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Brandon Black
On 9/12/05, PFC [EMAIL PROTECTED] wrote: I know I haven't provided a whole lot of application-level detail here,You did !What about :- using COPY instead of INSERT ?(should be easy to do from the aggregators) Possibly, although it would kill the current design of returning the database

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Alan Stange
Brandon Black wrote: On 9/12/05, *PFC* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: - benchmarking something else than ext3 (xfs ? reiser3 ?) We've had bad experiences under extreme and/or strange workloads with XFS here in general, although this

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Greg Stark
Brandon Black [EMAIL PROTECTED] writes: The vast, overwhelming majority of our database traffic is pretty much a non-stop stream of INSERTs filling up tables. That part Postgres should handle pretty well. It should be pretty much limited by your I/O bandwidth so big raid 1+0 arrays are

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Christopher Petrilli
On 9/12/05, Brandon Black [EMAIL PROTECTED] wrote: I'm in the process of developing an application which uses PostgreSQL for data storage. Our database traffic is very atypical, and as a result it has been rather challenging to figure out how to best tune PostgreSQL on what development

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Brandon Black
On 12 Sep 2005 23:07:49 -0400, Greg Stark [EMAIL PROTECTED] wrote: The WAL parameters like commit_delay and commit_siblings are a bit of amystery. Nobody has done any extensive testing of them. It would be quite helpful if you find anything conclusive and post it. It would also besurprising if

Re: [PERFORM] Performance considerations for very heavy INSERT traffic

2005-09-12 Thread Brandon Black
On 9/12/05, Christopher Petrilli [EMAIL PROTECTED] wrote: 2) Tune ext3.The default configuration wrecks high-write situations. Look into data="" for mounting, turning off atime (I hopeyou've done this already) updates, and also modifying the scheduler to the elevator model.This is poorly

Re: [PERFORM] Postgresql Hardware - Recommendations

2005-09-12 Thread Christian.Kastner
Andrew, Matthew, thanks to you both four your advice. I'm sorry I couldn't provide more details to the situation, I will post again as soon I get them. Time to share your insights with the colleagues :) Best Regards, Chris -Ursprüngliche Nachricht- Von: Paul Ramsey [mailto:[EMAIL

Re: [PERFORM] Performance considerations for very heavy INSERT

2005-09-12 Thread Ron Peacetree
From: Brandon Black [EMAIL PROTECTED] Sent: Sep 12, 2005 5:04 PM To: pgsql-performance@postgresql.org Subject: [PERFORM] Performance considerations for very heavy INSERT traffic I'm in the process of developing an application which uses PostgreSQL for data storage. Our database traffic is very