Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Anjan Dave
mance@postgresql.org Subject: Re: [PERFORM] PostgreSQL strugling during high load Anjan, > As far as disk I/O is concerned for flushing the buffers out, I am not > ruling out the combination of Dell PERC4 RAID card, and the RH AS 3.0 > Update3 being a problem. You know that Update4 is out, yes

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Steve Poe
Josh Berkus wrote: >Anjan, > > > >>As far as disk I/O is concerned for flushing the buffers out, I am not >>ruling out the combination of Dell PERC4 RAID card, and the RH AS 3.0 >>Update3 being a problem. >> >> > >You know that Update4 is out, yes? >Update3 is currenly throttling your I/O

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Josh Berkus
Anjan, > As far as disk I/O is concerned for flushing the buffers out, I am not > ruling out the combination of Dell PERC4 RAID card, and the RH AS 3.0 > Update3 being a problem. You know that Update4 is out, yes? Update3 is currenly throttling your I/O by about 50%. -- --Josh Josh Berkus Ag

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Vivek Khera
On May 19, 2005, at 2:12 PM, Anjan Dave wrote: As far as disk I/O is concerned for flushing the buffers out, I am not ruling out the combination of Dell PERC4 RAID card That'd be my first guess as to I/O speed issues. I have some dell hardware that by all means should be totally blowing out m

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Donald Courtney
Update3 being a problem. Thanks, Anjan -Original Message- From: Donald Courtney [mailto:[EMAIL PROTECTED] Sent: Thu 5/19/2005 12:54 PM To: Tom Lane Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] PostgreSQL strugling during high load Tom Thanks for the post

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Anjan Dave
t: Re: [PERFORM] PostgreSQL strugling during high load Tom Thanks for the post - I think I am getting this problem for a synthetic workload at high connection loads. The whole system seems to stop. Can you give some examples on what to tr

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-19 Thread Donald Courtney
Tom Thanks for the post - I think I am getting this problem for a synthetic workload at high connection loads. The whole system seems to stop. Can you give some examples on what to try out in the .conf file? I tried bgwriter_all_percent = 30, 10, and 3 Which I understand to mean 30%, 10% and 3%

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-16 Thread Mindaugas Riauba
> > Hm. Yes. Number of locks varies quite alot (10-600). Now what to > > investigate > > further? We do not use explicit locks in our functions. We use quite simple > > update/delete where key=something; > > Some sample (select * from pg_locks order by pid) is below. > > The sample doesn't sho

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-15 Thread Thomas F. O'Connell
Actually, that solution didn't work so well. Even very small delays in the loop caused the entire loop to perform too slowly to be useful in the production environment. I ended up producing a small patch out of it :P, but we ended up using pgpool to reduce connections from another part of t

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-15 Thread Matthew T. O'Connor
Mindaugas Riauba wrote: The "vacuum cost" parameters can be adjusted to make vacuums fired by pg_autovacuum less of a burden. I haven't got any specific numbers to suggest, but perhaps someone else does. It looks like that not only vacuum causes our problems. vacuum_cost seems to lower vacuu

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Mischa Sandberg
Quoting Tom Lane <[EMAIL PROTECTED]>: > "Mindaugas Riauba" <[EMAIL PROTECTED]> writes: > > ... So contents of database changes very fast. Problem is that > when > > pg_autovacuum does vacuum those changes slows down too much. > > The "vacuum cost" parameters can be adjusted to make vacuums fired

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Tom Lane
"Mindaugas Riauba" <[EMAIL PROTECTED]> writes: > Hm. Yes. Number of locks varies quite alot (10-600). Now what to > investigate > further? We do not use explicit locks in our functions. We use quite simple > update/delete where key=something; > Some sample (select * from pg_locks order by pid)

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Steinar H. Gunderson
On Fri, May 13, 2005 at 05:45:45PM +0300, Mindaugas Riauba wrote: > But there's no checkpoint warnings in serverlog. And btw we are running > with fsync=off (yes I know the consequences). Just a note here; since you have battery-backed hardware cache, you probably won't notice that much of a slo

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Mindaugas Riauba
> >>The "vacuum cost" parameters can be adjusted to make vacuums fired > >>by pg_autovacuum less of a burden. I haven't got any specific numbers > >>to suggest, but perhaps someone else does. > > > > It looks like that not only vacuum causes our problems. vacuum_cost > > seems to lower vacuum i

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Cosimo Streppone
Mindaugas Riauba wrote: The "vacuum cost" parameters can be adjusted to make vacuums fired by pg_autovacuum less of a burden. I haven't got any specific numbers to suggest, but perhaps someone else does. It looks like that not only vacuum causes our problems. vacuum_cost seems to lower vacuum im

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Mindaugas Riauba
> > It looks like that not only vacuum causes our problems. vacuum_cost > > seems to lower vacuum impact but we are still noticing slow queries "storm". > > We are logging queries that takes >2000ms to process. > > And there is quiet periods and then suddenly 30+ slow queries appears in > > lo

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Tom Lane
"Mindaugas Riauba" <[EMAIL PROTECTED]> writes: > It looks like that not only vacuum causes our problems. vacuum_cost > seems to lower vacuum impact but we are still noticing slow queries "storm". > We are logging queries that takes >2000ms to process. > And there is quiet periods and then sudde

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Mindaugas Riauba
> > ... So contents of database changes very fast. Problem is that when > > pg_autovacuum does vacuum those changes slows down too much. > > The "vacuum cost" parameters can be adjusted to make vacuums fired > by pg_autovacuum less of a burden. I haven't got any specific numbers > to suggest, but

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Tom Lane
"Mindaugas Riauba" <[EMAIL PROTECTED]> writes: > ... So contents of database changes very fast. Problem is that when > pg_autovacuum does vacuum those changes slows down too much. The "vacuum cost" parameters can be adjusted to make vacuums fired by pg_autovacuum less of a burden. I haven't got a

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Steinar H. Gunderson
On Fri, May 13, 2005 at 03:52:38PM +0300, Mindaugas Riauba wrote: > Tables are not big (2-15 rows each) but have very high > turnover rate - 100+ updates/inserts/deletes/selects per second. > So contents of database changes very fast. Problem is that when > pg_autovacuum does vacuum those

[PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Mindaugas Riauba
Hello, We have problems with one postgresql database with high data change rate. Actually we are already under pressure to change postgresql to Oracle. I cannot post schema and queries to list but can do this privately. Tables are not big (2-15 rows each) but have very high turn