Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sat, Sep 27, 2008 at 4:33 PM, John Huttley <[EMAIL PROTECTED]> wrote: > > > > this is part of the trade-offs of MVCC. > > > was... was a part of the trade-offs. > > You are thinking of HOT? > I don't think it applies in the case of full table updates?? Sure, you just need a table with plenty o

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Scott Marlowe wrote: On Fri, Sep 26, 2008 at 5:03 PM, John Huttley <[EMAIL PROTECTED]> wrote: Hi Andrew, There are two problems. The first is the that if there is a table with a index and an update is performed on a non indexed field, the index is still re indexed. I assume you mean

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Tom Lane
John Huttley <[EMAIL PROTECTED]> writes: > Scott Marlowe wrote: >> was... was a part of the trade-offs. > You are thinking of HOT? > I don't think it applies in the case of full table updates?? Sure, as long as there's enough free space on each page. If you wanted to make a table that was optim

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Carey
I have had great success using FILLFACTOR on certain tables where big updates like this occur and improving performance. It is still not as fast as I would like, but there are significant gains. A big disk array won't help you as much as it should -- yes it will be faster, but it will still be ch

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Ahh! I've not dealt with that before. I'll look it up. Thanks Tom. Tom Lane wrote: John Huttley <[EMAIL PROTECTED]> writes: You are thinking of HOT? I don't think it applies in the case of full table updates?? Sure, as long as there's enough free space on each page. If you wanted t

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Thanks to everyone that responded. I've done some benchmarking checkpoint _segments=16 is fine, going to 64 made no improvement. Using "update file set size=99" as a statement, but changing 99 on each run.. With 32M shared memory, time in sec and leaving the system idle long enough between ru

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Greg Smith
On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the checkpoints can't have very much work to do, so their impact on performance is s

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Greg Smith wrote: On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the checkpoints can't have very much work to do, so their impac

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sun, Sep 28, 2008 at 8:01 PM, John Huttley <[EMAIL PROTECTED]> wrote: > Ahh bugger, I've just trashed my test setup. > I've settled on 64Mb shared memory since I've only got 1Gb or RAM and the > system impact of 256M is severe. > Also it uses FB-DIMMS which cost arm+leg+first born http://www.c

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
Ah yess... actually I can get the Kingston stuff locally. However at the moment I'm happily married and want to keep it that way! Everything is in pairs too. Actually its a lot cheaper than when it first came out, but still a lot more than your corner shop DDR-2 stuff. --John Scott Marlowe

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Dan Langille
On Sep 28, 2008, at 10:01 PM, John Huttley wrote: Greg Smith wrote: On Mon, 29 Sep 2008, John Huttley wrote: checkpoint _segments=16 is fine, going to 64 made no improvement. You might find that it does *after* increasing shared_buffers. If the buffer cache is really small, the checkp

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread John Huttley
I've canned the db and got rid my of data. I'm in the midst of doing some other benchmarking for a possible change to the bacula database. Loading up 1M records into a table of 60M records complete with indexes. It's still going... --john Dan Langille wrote: On Sep 28, 2008, at 10:01 PM, J

Re: [PERFORM] Slow updates, poor IO

2008-09-28 Thread Scott Marlowe
On Sun, Sep 28, 2008 at 9:08 PM, John Huttley <[EMAIL PROTECTED]> wrote: > Ah yess... actually I can get the Kingston stuff locally. > However at the moment I'm happily married and want to keep it that way! > > Everything is in pairs too. Actually its a lot cheaper than when it first > came out, bu