Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-16 Thread David Rees
On Thu, Jan 15, 2009 at 2:36 PM, Bill Preston wrote: > We are in Southern California. > What I need someone for when the SHTF again, and if I can't handle it, I > have some resource to get on the job right away. And it would help if they > were a company that does this kind of thing so that I can

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-15 Thread Bill Preston
Thanks Scott. We are in Southern California. What I need someone for when the SHTF again, and if I can't handle it, I have some resource to get on the job right away. And it would help if they were a company that does this kind of thing so that I can get some buy in from those above. Rusty

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-15 Thread Scott Marlowe
On Thu, Jan 15, 2009 at 2:55 PM, Bill Preston wrote: > Nothing special about that table. One index. > > It really seems that the system would grind to a stand-still when a lot of > non-transaction inserts were run combined with the creation of some large > temp tables. > > Since we added transacti

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-15 Thread Bill Preston
Nothing special about that table. One index. It really seems that the system would grind to a stand-still when a lot of non-transaction inserts were run combined with the creation of some large temp tables. Since we added transactions and started using truncate, things have cleared up nicely

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-14 Thread Heikki Linnakangas
Bill Preston wrote: Fsync is off in 8.3. You should consider turning synchronous_commit off instead. That's almost as good as fsync=off performance-wise, but doesn't leave your database corrupt in case of power loss or OS crash. -- Heikki Linnakangas EnterpriseDB http://www.enterprise

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-13 Thread Alan Hodgson
On Monday 12 January 2009, Bill Preston wrote: > As to the second example with the delete. There are no foreign keys. > For the index. If the table has fields a,b,c and d. > We have a btree index (a,b,c,d) > and we are saying DELETE FROM table_messed_up WHERE a=x. > Is there anything special abo

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-12 Thread Bill Preston
Wow thanks for all the help Tom and Alan. Sadly I was un-aware of the well-known behavior. Consider it more will known now. Fsync is off in 8.3. I am not too worried about what was before in 8.1 since we are not going back. For the first example (bad behavior when I am not using transactions)

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-12 Thread Alan Hodgson
On Monday 12 January 2009, Bill Preston wrote: > I had a data load that I was doing with 8.1. It involved about 250k sql > statements that were inserts into a table with just one index. The index > has two fields. > With the upgrade to 8.3 that process started taking all night and 1/2 a > day. I

Re: [PERFORM] Slow insert performace, 8.3 Wal related?

2009-01-12 Thread Tom Lane
Bill Preston writes: > I've noticed some performance problems that I am guessing are WAL > related based on my browsing around and wondered if someone had some > suggestions for tuning the WAL settings. It could also help if someone > just laughed at me and told me it wasn't WAL. Consider it d

Re: [PERFORM] slow insert into very large table

2005-12-01 Thread Tom Lane
Wolfgang Gehner <[EMAIL PROTECTED]> writes: > This is with PostgreSQL 8.0 final for WindowsXP on a Pentium 1.86 GHz, > 1GB Memory. HD is fast IDE. Try something more recent, like 8.0.3 or 8.0.4. IIRC we had some performance issues in 8.0.0 with tables that grew from zero to large size during a s

Re: [PERFORM] slow insert into very large table

2005-12-01 Thread Andreas Pflug
Wolfgang Gehner wrote: Hi there, I need a simple but large table with several million records. I do batch inserts with JDBC. After the first million or so records, the inserts degrade to become VERY slow (like 8 minutes vs initially 20 secondes). The table has no indices except PK while I do

Re: [PERFORM] Slow insert

2004-12-11 Thread Steinar H. Gunderson
On Wed, Dec 08, 2004 at 10:42:19AM +0800, Christopher Kings-Lynne wrote: > Why would an INSERT ever be really slow? This is what I see a lot of in > our site logs: > > Dec 5 15:57:48 marshall postgres[19599]: [3-1] LOG: duration: > 13265.492 ms statement: INSERT INTO users_sessions (sid, cob

Re: [PERFORM] Slow INSERT

2004-06-29 Thread Tom Lane
Michal Taborsky <[EMAIL PROTECTED]> writes: > I ran some tests to support this hypothesis. Every 500th insert is a tad > slower, but it is insignificant (normally the INSERT lasts 1.5ms, every > 500th is 9ms). During my tests (10 runs of 1000 INSERTS) I had > experienced only one "slow" insert (

Re: [PERFORM] Slow INSERT

2004-06-29 Thread Tom Lane
Michal Taborsky <[EMAIL PROTECTED]> writes: > I've read the discussion in "Trying to minimize the impact of > checkpoints" thread and I get it, that there is nothing I can do about > it. Well, we'll have to live with that, at least until 7.5. You could experiment with the checkpoint interval (ch

Re: [PERFORM] Slow INSERT

2004-06-29 Thread Michal Taborsky
Tom Lane wrote: Actually, the simpler theory is that the slowdown is caused by background checkpoint operations. Now a checkpoint would slow *everything* down not only this one insert, so maybe that's not the right answer either, but it's my next idea. You could check this to some extent by manua

Re: [PERFORM] Slow INSERT

2004-06-29 Thread Michal Taborsky
Tom Lane wrote: It's hard to see how inserting to such a simple table would be slow. Indeed. Is the number of inserts between slowdowns perfectly repeatable? My first thought is that the fast case is associated with inserting onto a page that is the same one last inserted to, and the slow case is

Re: [PERFORM] Slow INSERT

2004-06-29 Thread Tom Lane
=?ISO-8859-2?Q?Michal_T=E1borsk=FD?= <[EMAIL PROTECTED]> writes: > I am experiencing rather slow INSERTs on loaded server. > ... There are no indices, triggers or constraints attached to it. It's hard to see how inserting to such a simple table would be slow. > Sometimes, it takes as long as 130