Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-14 Thread Jim C. Nasby
On Sat, Mar 11, 2006 at 10:21:43PM +0200, Hannu Krosing wrote: > > table partitioning, I keep wondering whether it would be possible > > to move rows near the end of the table to the beginning in one, non- > > locking > > phase (vacuum to populate FSM with free space near beginning of table, > > t

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-11 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > At some point I had to compress a very busily updated table. I used the > following approach: > [ move a few rows at a time ] We could possibly do something similar with VACUUM FULL as well: once maintenance_work_mem is filled, start discarding per-page

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-11 Thread Hannu Krosing
Ühel kenal päeval, R, 2006-03-10 kell 12:23, kirjutas Steve Atkins: > I get bitten by this quite often (customer machines, one giant table, > purge out a lot of old data). > > CLUSTER is great for that, given the headroom, though I've often > resorted to a dump and restore because I've not had th

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-11 Thread Stefan Kaltenbrunner
Tom Lane wrote: > Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: > 3. vacuuming this table - it turned out that VACUUM FULL is completly unusable on a table(which i actually expected before) of this size not only to the locking involved but rather due to a gigantic memory require

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-11 Thread Luke Lonergan
Stefan, On 3/11/06 12:21 AM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: >> So - you're getting 20MB/s on loading from a potential of 200MB/s? > > no - I can write 110MB/s on thw WAL LUN and 110MB/s on the other LUN > concurrently. The numbers you published earlier show you are getting a

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-11 Thread Stefan Kaltenbrunner
Luke Lonergan wrote: > Stefan, > > On 3/10/06 12:23 PM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > > >>wrong(or rather extremely optimistic) the array itself only has two >>(redundant) FC-loops(@2GB )to the attached expansion chassis. The array >>has 2 active/active controllers (with a

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-10 Thread Luke Lonergan
Stefan, On 3/10/06 12:23 PM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > wrong(or rather extremely optimistic) the array itself only has two > (redundant) FC-loops(@2GB )to the attached expansion chassis. The array > has 2 active/active controllers (with a failover penalty) with two host

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-10 Thread Stefan Kaltenbrunner
Luke Lonergan wrote: > Stefan, > > On 3/10/06 11:48 AM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > > >>2 HBAs in the server, 2x2 possible paths to each LUN. >>6 disks for the WAL and 12 disks for the data > > > So - you have 18 disks worth of potential bandwidth, not factoring loss du

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-10 Thread Steve Atkins
On Mar 10, 2006, at 11:54 AM, Tom Lane wrote: Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: 3. vacuuming this table - it turned out that VACUUM FULL is completly unusable on a table(which i actually expected before) of this size not only to the locking involved but rather due to a gigan

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-10 Thread Luke Lonergan
Stefan, On 3/10/06 11:48 AM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > 2 HBAs in the server, 2x2 possible paths to each LUN. > 6 disks for the WAL and 12 disks for the data So - you have 18 disks worth of potential bandwidth, not factoring loss due to RAID. That's roughly 18 * 60 = 1,

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-10 Thread Tom Lane
Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes: >>> 3. vacuuming this table - it turned out that VACUUM FULL is completly >>> unusable on a table(which i actually expected before) of this size not >>> only to the locking involved but rather due to a gigantic memory >>> requirement and unbelievable

Re: [HACKERS] random observations while testing with a 1,8B row table

2006-03-10 Thread Stefan Kaltenbrunner
Luke Lonergan wrote: > Stefan, > > On 3/10/06 9:40 AM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > > >>I will summarize some of the just in case somebody is interested: > > > I am! heh - not surprised :-) > > >>-> table used has 5 integer columns non-indexed during the loads >>-> h

Re: [HACKERS] random observations while testing with a 1,8B row

2006-03-10 Thread Luke Lonergan
Stefan, On 3/10/06 9:40 AM, "Stefan Kaltenbrunner" <[EMAIL PROTECTED]> wrote: > I will summarize some of the just in case somebody is interested: I am! > -> table used has 5 integer columns non-indexed during the loads > -> hardware is a Dual Opteron 280 with 4 [EMAIL PROTECTED],4GHz and 16GB R