Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
Kelly Burkhart wrote: On 1/4/06, Steve Eckmann <[EMAIL PROTECTED]> wrote: Thanks, Steinar. I don't think we would really run with fsync off, but I need to document the performance tradeoffs. You're right that my explanation was confusing; probably because I'm confused about how to use COPY

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
Tom Lane wrote: Steve Eckmann <[EMAIL PROTECTED]> writes: <>Thanks for the suggestion, Tom. Yes, I think I could do that. But I thought what I was doing now was effectively the same, because the PostgreSQL 8.0.0 Documentation says (section 27.3.1): "It is allowed to include multi

[PERFORM] Help in avoiding a query 'Warm-Up' period/shared buffer cache

2006-01-04 Thread Mark Liberman
Hello, We have a web-application running against a postgres 8.1 database, and basically, every time I run a report after no other reports have been run for several hours, the report will take significantly longer (e.g. 30 seconds), then if I re-run the report again, or run the report when the w

Re: [PERFORM] improving write performance for logging

2006-01-04 Thread Ian Westmacott
On Wed, 2006-01-04 at 09:29 -0500, Ron wrote: > 2B is a lot of inserts. If you had to guess, > what do you think is the maximum number of inserts you could do in a day? It seems we are pushing it there. Our intentions are to scale much further, but our plans are to distribute at this point. >

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Kelly Burkhart
On 1/4/06, Steve Eckmann <[EMAIL PROTECTED]> wrote: Thanks, Steinar. I don't think we would really run with fsync off, but I need to document the performance tradeoffs. You're right that my explanation was confusing; probably because I'm confused about how to use COPY! I could batch multiple INSERT

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Tom Lane
Steve Eckmann <[EMAIL PROTECTED]> writes: > Thanks for the suggestion, Tom. Yes, I think I could do that. But I > thought what I was doing now was effectively the same, because the > PostgreSQL 8.0.0 Documentation says (section 27.3.1): "It is allowed to > include multiple SQL commands (separate

Re: [PERFORM] improving write performance for logging

2006-01-04 Thread Ron
2B is a lot of inserts. If you had to guess, what do you think is the maximum number of inserts you could do in a day? How large is each record being inserted? How much can you put in a COPY and how many COPYs can you put into a transactions? What values are you using for bgwriter* and chec

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
Ian Westmacott wrote: We have a similar application thats doing upwards of 2B inserts per day. We have spent a lot of time optimizing this, and found the following to be most beneficial: 1) use COPY (BINARY if possible) 2) don't use triggers or foreign keys 3) put WAL and tables on differen

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
dlang wrote: On Tue, 3 Jan 2006, Tom Lane wrote: Steve Eckmann <[EMAIL PROTECTED]> writes: We also found that we could improve MySQL performance significantly using MySQL's "INSERT" command extension allowing multiple value-list tuples in a single command; the rat

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
Steinar H. Gunderson wrote: On Tue, Jan 03, 2006 at 04:44:28PM -0700, Steve Eckmann wrote: Are there general guidelines for tuning the PostgreSQL server for this kind of application? The suggestions I've found include disabling fsync (done), Are you sure you really want

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Steve Eckmann
Tom Lane wrote: Steve Eckmann <[EMAIL PROTECTED]> writes: We also found that we could improve MySQL performance significantly using MySQL's "INSERT" command extension allowing multiple value-list tuples in a single command; the rate for MyISAM tables improved to about 2600 objects/

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Ian Westmacott
We have a similar application thats doing upwards of 2B inserts per day. We have spent a lot of time optimizing this, and found the following to be most beneficial: 1) use COPY (BINARY if possible) 2) don't use triggers or foreign keys 3) put WAL and tables on different spindles (channels if p