Re: [PERFORM] optimization ideas for frequent, large(ish) updates in frequently accessed DB?

2004-06-01 Thread Manfred Koizar
On Fri, 13 Feb 2004 16:21:29 +0100, I wrote: >Populate this table with > INSERT INTO idmap > SELECT id, id, true > FROM t; This should be INSERT INTO idmap SELECT DISTINCT id, id, true FROM t; Servus Manfred ---(end of broadc

Re: [PERFORM] optimization ideas for frequent, large(ish) updates

2004-02-15 Thread Tom Lane
"Marinos J. Yannikos" <[EMAIL PROTECTED]> writes: > Jeff Trout wrote: >> Remember that it is going to allocate 800MB per sort. > I didn't know that it always allocates the full amount of memory > specificed in the configuration It doesn't ... but it could use *up to* that much before starting to

Re: [PERFORM] optimization ideas for frequent, large(ish) updates

2004-02-15 Thread Marinos J. Yannikos
Jeff Trout wrote: Remember that it is going to allocate 800MB per sort. It is not "you can allocate up to 800MB, so if you need 1 meg, use one meg". Some queries may end up having a few sort steps. I didn't know that it always allocates the full amount of memory specificed in the configuratio

Re: [PERFORM] optimization ideas for frequent, large(ish) updates

2004-02-15 Thread Jeff Trout
On Feb 14, 2004, at 9:02 PM, Marinos J. Yannikos wrote: Josh Berkus wrote: 800MB for sort mem? Are you sure you typed that correctly? You must be counting on not having a lot of concurrent queries. It sure will speed up index updating, though! 800MB is correct, yes... There are usually onl

Re: [PERFORM] optimization ideas for frequent, large(ish) updates

2004-02-14 Thread Christopher Kings-Lynne
800MB is correct, yes... There are usually only 10-30 postgres processes active (imagine 5-10 people working on the web front-end while cron jobs access the db occasionally). Very few queries can use such large amounts of memory for sorting, but they do exist. But remember that means that if yo

Re: [PERFORM] optimization ideas for frequent, large(ish) updates

2004-02-14 Thread Marinos J. Yannikos
Josh Berkus wrote: 800MB for sort mem? Are you sure you typed that correctly? You must be counting on not having a lot of concurrent queries. It sure will speed up index updating, though! 800MB is correct, yes... There are usually only 10-30 postgres processes active (imagine 5-10 people

Re: [PERFORM] optimization ideas for frequent, large(ish) updates in frequently accessed DB?

2004-02-13 Thread Manfred Koizar
Marinos, while you are busy answering my first set of questions :-), here is an idea that might help even out resource consumption. On Fri, 13 Feb 2004 01:58:34 +0100, "Marinos J. Yannikos" <[EMAIL PROTECTED]> wrote: >begin transaction; > delete from t where id=5; > insert into t (id,...) valu

Re: [PERFORM] optimization ideas for frequent, large(ish) updates in frequently accessed DB?

2004-02-13 Thread Manfred Koizar
On Fri, 13 Feb 2004 01:58:34 +0100, "Marinos J. Yannikos" <[EMAIL PROTECTED]> wrote: >I'm looking for ideas that might improve the interactive performance of >the system, without slowing down the updates too much. IOW, you could accept slower updates. Did you actually try and throttle down the i

Re: [PERFORM] optimization ideas for frequent, large(ish) updates in frequently accessed DB?

2004-02-12 Thread Josh Berkus
Marinos, > shared_buffers=10 > (I tried many values, this seems to work well for us - 12GB RAM) > wal_buffers=500 > sort_mem=80 > checkpoint_segments=16 > effective_cache_size=100 > etc. 800MB for sort mem? Are you sure you typed that correctly? You must be counting on not having