Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Kevin
> I just came to think about /proc/sys/swappiness ... > > When this one is set to a high number (say, 100 - which is maximum), the > kernel will aggressively swap out all memory that is not beeing > accessed, to allow more memory for caches. For a postgres server, OS > caches are good, because po

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Ron - Thu at 03:10:35PM -0400] > Jim is correct that traditional 7.x folklore regarding shared buffer > size is nowhere near as valid for 8.x. Jim tends to know what he is > talking about when speaking about pg operational issues. I would not doubt it, but it's always better to hear it from mo

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Merlin Moncure
On 10/19/06, Ron <[EMAIL PROTECTED]> wrote: Nonetheless, "YMMV". The only sure way to know what is best for your SW running on your HW under your load conditions is to test, test, test. anybody have/know of some data on shared buffer settings on 8.1+? merlin ---(end o

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Ron
At 12:35 PM 10/19/2006, Tobias Brox wrote: [Jim C. Nasby - Thu at 10:28:31AM -0500] > I think it'd be much better to experiment with using much larger > shared_buffers settings. The conventional wisdom there is from 7.x days > when you really didn't want a large buffer, but that doesn't really >

Re: [PERFORM] DB Performance decreases due to often written/accessed table

2006-10-19 Thread Merlin Moncure
On 10/19/06, Jens Schipkowski <[EMAIL PROTECTED]> wrote: On Thu, 19 Oct 2006 19:32:22 +0200, Merlin Moncure > > 1. your database design is the real culprit here. If you want things > to run really quickly, solve the problem there by normalizing your > schema. denomalization is the root cause of

Re: [PERFORM] DB Performance decreases due to often written/accessed table

2006-10-19 Thread Jens Schipkowski
On Thu, 19 Oct 2006 19:32:22 +0200, Merlin Moncure <[EMAIL PROTECTED]> wrote: On 10/19/06, Jens Schipkowski <[EMAIL PROTECTED]> wrote: // select finds out which one has not an twin // a twin is defined as record with the same attr* values // decreases speed over time until timeout by postgres

Re: [PERFORM] DB Performance decreases due to often written/accessed

2006-10-19 Thread Richard Huxton
Jim C. Nasby wrote: On Thu, Oct 19, 2006 at 06:19:16PM +0100, Richard Huxton wrote: OK - these plans look about the same, but the time is greatly different. Both have rows=140247 as the estimated number of rows in tbl_reg. Either you have many more rows in the second case (in which case you're

Re: [PERFORM] DB Performance decreases due to often written/accessed table

2006-10-19 Thread Merlin Moncure
On 10/19/06, Jens Schipkowski <[EMAIL PROTECTED]> wrote: // select finds out which one has not an twin // a twin is defined as record with the same attr* values // decreases speed over time until timeout by postgresql SELECT * FROM tbl_reg reg WHERE register <> loc1 AND idreg NOT IN

Re: [PERFORM] DB Performance decreases due to often written/accessed

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:19:16PM +0100, Richard Huxton wrote: > OK - these plans look about the same, but the time is greatly different. > Both have rows=140247 as the estimated number of rows in tbl_reg. Either > you have many more rows in the second case (in which case you're not > running

Re: [PERFORM] DB Performance decreases due to often written/accessed

2006-10-19 Thread Richard Huxton
Jens Schipkowski wrote: And running vacuum every 30 seconds does what? Not yet fully tested. It seems to lower the slow down. But minimizing the gain of slow down doesn't solve the problem. The Problem is the decrease of execution speed of DELETE and SELECT statements by a table row count betw

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 12:00:39PM -0500] > Well, if you're buying unreliable hardware, there's not much you can > do... you're setting yourself up for problems. I'm luckily not responsible for the hardware, but my general experience tells that you never know anything about hardware reliability unt

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 12:00:39PM -0500] > What's reasonable for work_mem depends on your workload. If you've got > some reporting queries that you know aren't run very concurrently they > might benefit from large values of work_mem. For stats.distributed.net, > I set work_mem to something like 2MB

Re: [PERFORM] DB Performance decreases due to often written/accessed table

2006-10-19 Thread Jens Schipkowski
On Thu, 19 Oct 2006 16:55:34 +0200, Richard Huxton wrote: Jens Schipkowski wrote: Hi, we've got performance problems due to repeating SELECT, UPDATE, DELETE, INSERT statements. This statements have to be executed every 10 seconds, but they run into a timeout. To obviate problems regardi

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:53:49PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 11:45:32AM -0500] > > > > The issue with pg_xlog is you don't need bandwidth... you need super-low > > > > latency. The best way to accomplish that is to get a battery-backed RAID > > > > controller that you can e

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 11:45:32AM -0500] > > > The issue with pg_xlog is you don't need bandwidth... you need super-low > > > latency. The best way to accomplish that is to get a battery-backed RAID > > > controller that you can enable write caching on. > > > > Sounds a bit risky to me :-) > > We

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:39:22PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 11:31:26AM -0500] > > Yeah, test setups are a good thing to have... > > We would need to replicate the production traffic as well to do reliable > tests. Well, we'll get to that one day ... Marginally reliable

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 11:31:26AM -0500] > Yeah, test setups are a good thing to have... We would need to replicate the production traffic as well to do reliable tests. Well, we'll get to that one day ... > The issue with pg_xlog is you don't need bandwidth... you need super-low > latency. The b

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 10:28:31AM -0500] > I think it'd be much better to experiment with using much larger > shared_buffers settings. The conventional wisdom there is from 7.x days > when you really didn't want a large buffer, but that doesn't really > apply with the new buffer management we got i

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:00:54PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 10:28:31AM -0500] > > I think it'd be much better to experiment with using much larger > > shared_buffers settings. The conventional wisdom there is from 7.x days > > when you really didn't want a large buffer, bu

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
[Jim C. Nasby - Thu at 10:28:31AM -0500] > I think it'd be much better to experiment with using much larger > shared_buffers settings. The conventional wisdom there is from 7.x days > when you really didn't want a large buffer, but that doesn't really > apply with the new buffer management we got i

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 03:54:28PM +0200, Tobias Brox wrote: > I just came to think about /proc/sys/swappiness ... > > When this one is set to a high number (say, 100 - which is maximum), the > kernel will aggressively swap out all memory that is not beeing > accessed, to allow more memory for cac

Re: [PERFORM] DB Performance decreases due to often written/accessed

2006-10-19 Thread Richard Huxton
Jens Schipkowski wrote: Hi, we've got performance problems due to repeating SELECT, UPDATE, DELETE, INSERT statements. This statements have to be executed every 10 seconds, but they run into a timeout. To obviate problems regarding to our Java Software and JDBC drivers, we put the repeating s

Re: [PERFORM] VACUUM FULL ANALYZE on 8.1.4 is slower then on 8.0

2006-10-19 Thread Tom Lane
Michael Fuhr <[EMAIL PROTECTED]> writes: > On Thu, Oct 19, 2006 at 03:30:35PM +0200, Andrzej Zawadzki wrote: >> After upgrade PostgreSQL from 8.0 to 8.1.4 a VACUUM FULL ANALYZE >> process is much slower, from logs: > Is it possible that 8.1 was built with --enable-cassert and 8.0 > wasn't? What d

Re: [PERFORM] VACUUM FULL ANALYZE on 8.1.4 is slower then on 8.0

2006-10-19 Thread Michael Fuhr
On Thu, Oct 19, 2006 at 03:30:35PM +0200, Andrzej Zawadzki wrote: > After upgrade PostgreSQL from 8.0 to 8.1.4 a VACUUM FULL ANALYZE > process is much slower, from logs: Are you sure you need VACUUM FULL? If you're vacuuming often enough and your free space map settings are adequate then plain VA

[PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Tobias Brox
I just came to think about /proc/sys/swappiness ... When this one is set to a high number (say, 100 - which is maximum), the kernel will aggressively swap out all memory that is not beeing accessed, to allow more memory for caches. For a postgres server, OS caches are good, because postgres relie

Re: [PERFORM] ACCESS EXCLUSIVE lock

2006-10-19 Thread Atesz
Scott Marlowe wrote: What if, a minute or two after the drop contraint, you issue a rollback? After the DROP CONSTRAINT I insert 4 million rekords into the TABLE b. After the inserts I remake the dropped constraints, and commit the transaction (P1). This solution is faster then the conventio

[PERFORM] VACUUM FULL ANALYZE on 8.1.4 is slower then on 8.0

2006-10-19 Thread Andrzej Zawadzki
Hello! After upgrade PostgreSQL from 8.0 to 8.1.4 a VACUUM FULL ANALYZE process is much slower, from logs: 8.0 [13666][postgres][2006-10-06 01:13:38 CEST][1340121452] LOG: statement: VACUUM FULL ANALYZE; [13666][postgres][2006-10-06 01:39:15 CEST][0] LOG: duration: 1536862.425 ms 8.1 [4535][

[PERFORM] DB Performance decreases due to often written/accessed table

2006-10-19 Thread Jens Schipkowski
Hi, we've got performance problems due to repeating SELECT, UPDATE, DELETE, INSERT statements. This statements have to be executed every 10 seconds, but they run into a timeout. To obviate problems regarding to our Java Software and JDBC drivers, we put the repeating sequence of statements

[PERFORM]

2006-10-19 Thread Rohit_Behl
Hi Thanks everybody. I have confirmed that this does not affect inserts. But the query performance has improved a lot. Regards Rohit -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Heikki Linnakangas Sent: 18 October 2006 13:52 To: Rohit_B

Re: [PERFORM] Postgresql 8.1.4 - performance issues for select on

2006-10-19 Thread Ioana Danes
Hello, It looks like some of you missed my first email but my problem is not to find a replacement for this select: select max(transid) from someunionview Thare are plenty of solutions for doing this... My point is to split a tale in two and to make this transparent for the developers as a first

Re: [PERFORM] Postgresql 8.1.4 - performance issues for select on

2006-10-19 Thread Ioana Danes
Hi, I tried and this does does not work either. Thank you, Ioana --- "Jim C. Nasby" <[EMAIL PROTECTED]> wrote: > On Wed, Oct 18, 2006 at 03:32:15PM -0700, Jeff Davis > wrote: > > On Wed, 2006-10-18 at 17:10 -0500, Jim C. Nasby > wrote: > > > Sorry, don't have the earlier part of this > thread, bu

Re: [PERFORM] Postgresql 8.1.4 - performance issues for select on view using max

2006-10-19 Thread Ioana Danes
Hello, Actually what I expected from the planner for this query (select max(transid) from view) was something like this : select max(transid) from (select max(transid) from archive.transaction union all select max(transid) from public.transaction) and to apply the max function to each query of