Re: [PERFORM] improvement suggestions for performance design

2007-07-19 Thread Kalle Hallivuori
Hi all! 2007/7/18, Thomas Finneid [EMAIL PROTECTED]: Hi I have tested your COPY patch (actually I tested postgresql-jdbc-8.2-505-copy-20070716.jdbc3.jar) and it is really fast, actually just as fast as serverside COPY (boths tests was performed on local machine). Happy to hear there's

Re: [GENERAL] [PERFORM] Parrallel query execution for UNION ALL Queries

2007-07-19 Thread Dimitri Fontaine
Hi, Le mercredi 18 juillet 2007, Jonah H. Harris a écrit : On 7/18/07, Benjamin Arai [EMAIL PROTECTED] wrote: But I want to parrallelize searches if possible to reduce the perofrmance loss of having multiple tables. PostgreSQL does not support parallel query. Parallel query on top of

Re: [GENERAL] [PERFORM] Parrallel query execution for UNION ALL Queries

2007-07-19 Thread Luke Lonergan
Dimitri, Seems to me that : - GreenPlum provides some commercial parallel query engine on top of PostgreSQL, I certainly think so and so do our customers in production with 100s of terabytes :-) - plproxy could be a solution to the given problem.

[PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Josh Berkus
Folks, I've run into this a number of times with various PostgreSQL users, so we tested it at Sun. What seems to be happening is that at some specific number of connections average throughput drops 30% and response time quadruples or worse. The amount seems to vary per machine; I've seen it

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Joshua D. Drake
Josh Berkus wrote: Folks, 650 105.71 0.02 700 106.95 0.02 750 107.69 0.02 800 106.78 0.02 850 108.59 0.02 900 106.03 0.02 950 106.13 0.02 100064.58 0.15 105052.32 0.23 110049.79 0.25 Tinkering with shared_buffers has had no effect on this

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Alvaro Herrera
Josh Berkus wrote: Folks, I've run into this a number of times with various PostgreSQL users, so we tested it at Sun. What seems to be happening is that at some specific number of connections average throughput drops 30% and response time quadruples or worse. The amount seems to vary

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Josh Berkus
Alvaro, Have you messed with max_connections and/or max_locks_per_transaction while testing this? The lock table is sized to max_locks_per_xact times max_connections, and shared memory hash tables get slower when they are full. Of course, the saturation point would depend on the avg number

[PERFORM] Is it possible to know where is the deadlock

2007-07-19 Thread Arnau
Hi all, I have a serious problem with a server. This server holds severals DB, the problem is thet the CPU's spend most of the time waiting: Cpu0: 4.0% us, 2.3% sy, 0.0% ni, 61.5% id, 32.1% wa, 0.0% hi, 0.0% si Cpu1: 2.3% us, 0.3% sy, 0.0% ni, 84.1% id, 13.3% wa, 0.0% hi, 0.0% si Cpu2: 1.3%

Re: [PERFORM] Is it possible to know where is the deadlock

2007-07-19 Thread Bill Moran
In response to Arnau [EMAIL PROTECTED]: Hi all, I have a serious problem with a server. This server holds severals DB, the problem is thet the CPU's spend most of the time waiting: Cpu0: 4.0% us, 2.3% sy, 0.0% ni, 61.5% id, 32.1% wa, 0.0% hi, 0.0% si Cpu1: 2.3% us, 0.3% sy, 0.0% ni,

Re: [PERFORM] Is it possible to know where is the deadlock

2007-07-19 Thread Claus Guttesen
The iostat -c says about 8% of time waiting for IO. I'm afraid this is due to locks between concurrent queries, is there anyway to have more info about? I do believe that if you told what OS you're running, what pg-version you're running, what type of sql-statements you perform the list

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Alvaro, Have you messed with max_connections and/or max_locks_per_transaction while testing this? The lock table is sized to max_locks_per_xact times max_connections, and shared memory hash tables get slower when they are full. Of course, the saturation

[PERFORM] Trying to tune postgres, how is this config?

2007-07-19 Thread Pat Maddox
I'd like any advice you have on my postgres.conf. The machine in question is a 2.4 Ghz Xeon with 2 gigs of ram running freebsd 6.2 and postgres 8.24. There are 16 concurrent users. This machine is used only for the database. Usage is split out pretty evenly between reads and writes. Thanks,

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Alvaro Herrera
Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: Alvaro, Have you messed with max_connections and/or max_locks_per_transaction while testing this? The lock table is sized to max_locks_per_xact times max_connections, and shared memory hash tables get slower when they are full. Of

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: AFAIK you'd get hard failures, not slowdowns, if you ran out of lock space entirely; Well, if there still is shared memory available, the lock hash can continue to grow, but it would slow down according to this comment in

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Joshua D. Drake
Alvaro Herrera wrote: Tom Lane wrote: Josh Berkus [EMAIL PROTECTED] writes: So maybe this does not make much sense in normal operation, thus not applicable to what Josh Berkus is reporting. However I was talking to Josh Drake yesterday and he told me that pg_dump was spending some

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Greg Smith
On Thu, 19 Jul 2007, Josh Berkus wrote: What seems to be happening is that at some specific number of connections average throughput drops 30% and response time quadruples or worse. Could you characterize what each connection is doing and how you're generating the load? I don't know how

[PERFORM] Improving select peformance

2007-07-19 Thread Carlos H. Reimer
Hi, One of our end users was complaining about a report that was taking too much time to execute and I´ve discovered that the following SQL statement was the responsible for it. I would appreciate any suggestions to improve performance of it. Thank you very much in advance!

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Josh Berkus
Tom, all: Also, have you tried watching vmstat or local equivalent to confirm that the machine's not starting to swap? We're not swapping. Josh, what sort of workload is being tested here --- read-mostly, write-mostly, a mixture? It's a TPCC-like workload, so heavy single-row updates, and

[PERFORM] Problems with posting

2007-07-19 Thread Carlos H. Reimer
Hi, I'm trying to post the following message to the performance group but the message does not appears in the list. Can someone help to solve this issue? Thanks in advance!

Re: [PERFORM] Improving select peformance

2007-07-19 Thread Tom Lane
Carlos H. Reimer [EMAIL PROTECTED] writes: One of our end users was complaining about a report that was taking too much time to execute and I´ve discovered that the following SQL statement was the responsible for it. Here's part of the problem: Join Filter:

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Josh, what sort of workload is being tested here --- read-mostly, write-mostly, a mixture? It's a TPCC-like workload, so heavy single-row updates, and the updates/inserts are what's being measured. For that matter, when I've seen this before it was

Re: [PERFORM] User concurrency thresholding: where do I look?

2007-07-19 Thread Josh Berkus
Tom, Well, if the load is a lot of short writing transactions then you'd expect the throughput to depend on how fast stuff can be pushed down to WAL. What have you got wal_buffers set to? Are you using a commit delay? What's the I/O system anyway (any BB write cache on the WAL disk?) and

[PERFORM] 8.2 - 8.3 performance numbers

2007-07-19 Thread Jim Nasby
Sorry for the cross-post, but this is performance and advocacy related... Has anyone benchmarked HEAD against 8.2? I'd like some numbers to use in my OSCon lightning talk. Numbers for both with and without HOT would be even better (I know we've got HOT-specific benchmarks, but I want