Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-19 Thread Alan Stange
Another data point. We had some down time on our system today to complete some maintenance work. It took the opportunity to rebuild the 700GB file system using XFS instead of Reiser. One iostat output for 30 seconds is avg-cpu: %user %nice%sys %iowait %idle 1.580.00

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-19 Thread Alan Stange
Luke Lonergan wrote: Alan, On 11/18/05 11:39 AM, "Alan Stange" <[EMAIL PROTECTED]> wrote: Yes and no. The one cpu is clearly idle. The second cpu is 40% busy and 60% idle (aka iowait in the above numbers). The "aka iowait" is the problem here - iowait is not idle (otherwise it wo

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-19 Thread Mark Kirkwood
Luke Lonergan wrote: Mark, On 11/18/05 3:46 PM, "Mark Kirkwood" <[EMAIL PROTECTED]> wrote: If you alter this to involve more complex joins (e.g 4. way star) and (maybe add a small number of concurrent executors too) - is it still the case? 4-way star, same result, that's part of my point.

Re: [PERFORM] Perl DBD and an alarming problem

2005-11-19 Thread Craig A. James
When I set statement_timeout in the config file, it just didn't do anything - it never timed out (PG 8.0.3). ... but did you reload the server after you [changed statement_timeout]? Mystery solved. I have two servers; I was reconfiguring one and restarting the other. Duh. Thanks, Craig

Re: [PERFORM] Storage/Performance and splitting a table

2005-11-19 Thread Michael Stone
On Sat, Nov 19, 2005 at 09:54:23AM -0800, Craig A. James wrote: First, I occasionally rebuild the keywords, after which the VACUUM FULL ANALYZE takes a LONG time - like 24 hours. You know you just need vacuum, not vacuum full, right? Mike Stone ---(end of broadcast)-

[PERFORM] Storage/Performance and splitting a table

2005-11-19 Thread Craig A. James
In a recent thread, several people pointed out that UPDATE = DELETE+INSERT. This got me to wondering. I have a table that, roughly, looks like this: create table doc ( id integer primary key, document text, keywords tsvector ); where "keywords" has a GIST index. Th

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-19 Thread Luke Lonergan
Alan, On 11/18/05 11:39 AM, "Alan Stange" <[EMAIL PROTECTED]> wrote: > Yes and no. The one cpu is clearly idle. The second cpu is 40% busy > and 60% idle (aka iowait in the above numbers). The "aka iowait" is the problem here - iowait is not idle (otherwise it would be in the "idle" column).

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-19 Thread Luke Lonergan
Mark, On 11/18/05 6:27 PM, "Mark Kirkwood" <[EMAIL PROTECTED]> wrote: > That too, meaning the business of 1 executor random reading a given > relation file whilst another is sequentially scanning (some other) part > of it I think it should actually improve things - each I/O will read 16MB in

Re: [PERFORM] What is the max number of database I can create in

2005-11-19 Thread John McCawley
However, what is the max number of database I can create before performance goes down? I know I'm not directly answering your question, but you might want to consider why you're splitting things up into different logical databases. If security is a big concern, you can create different da

Re: [PERFORM] VERY slow after many updates

2005-11-19 Thread Alex Wang
Great infomation. I didn't know that update is equal to delete+insert in Postgres. I would be more careful on designing the database access method in this case. Thanks, Alex - Original Message - From: "Csaba Nagy" <[EMAIL PROTECTED]> To: "Alex Wang" <[EMAIL PROTECTED]> Cc: "postgres

Re: [PERFORM] VERY slow after many updates

2005-11-19 Thread Guido Neitzer
On 19.11.2005, at 13:05 Uhr, Alex Wang wrote: Yes, it's a "queue" table. But I did not perform many insert/delete before it becomes slow. After insert 10 records, I just do get/ update continuously. When PostgreSQL updates a row, it creates a new row with the updated values. So you should

Re: [PERFORM] VERY slow after many updates

2005-11-19 Thread Csaba Nagy
Just for clarification, update is actually equal to delete+insert in Postgres. So if you update rows, it's the same as you would delete the row and insert a new version. So the table is bloating also in this situation. I think there is an added problem when you update, namely to get to a row, postg

Re: [PERFORM] VERY slow after many updates

2005-11-19 Thread Alex Wang
Hi Csaba, Thanks for your reply. Yes, it's a "queue" table. But I did not perform many insert/delete before it becomes slow. After insert 10 records, I just do get/update continuously. After 24 hour, the whole database become very slow (not only the download_queue table but other tables, too)

Re: [PERFORM] VERY slow after many updates

2005-11-19 Thread Csaba Nagy
Alex, I suppose the table is a kind of 'queue' table, where you insert/get/delete continuously, and the life of the records is short. Considering that in postgres a delete will still leave you the record in the table's file and in the indexes, just mark it as dead, your table's actual size can gro

[PERFORM] VERY slow after many updates

2005-11-19 Thread Alex Wang
I am using PostgreSQL in an embedded system which has only 32 or 64 MB RAM (run on PPC 266 MHz or ARM 266MHz CPU). I have a table to keep downlaod tasks. There is a daemon keep looking up the table and fork a new process to download data from internet. Daemon: . Check the table every 5 seco