Re: [HACKERS] string_to_array eats too much memory?

Tom Lane writes: Tatsuo Ishii <[EMAIL PROTECTED]> writes: string_to_array() consumes too much memory. For example, to make ~70k array elements, string_to_array seems to eat several Gig bytes of memory. I'd argue that the problem comes from enlarging the work arrays only 64 elements at a time i

Re: [HACKERS] Introducing an advanced Frequent Update Optimization

On Mon, Nov 06, 2006 at 09:50:53PM +, Simon Riggs wrote: > - There are specific issues with the optimizer's ability to understand > dead row numbers, which can in some cases lead to SeqScan plans that are > inappropriate when tables grow because of updates. This is a red-herring > that can lea

Re: [HACKERS] 8.2 Beta 3 Now Available for Download / Testing ...

Heikki Linnakangas wrote: Andrew Dunstan wrote: Marc G. Fournier wrote: As a result of there being two *known* outstanding bugs, we have just bundled up a Beta3, to allow for testing of the recent patch concerning WAL replay ... What are the bugs? AFAIK: 1. Tuple freezing and hint bit

Re: [HACKERS] Introducing an advanced Frequent Update Optimization

On Thu, 2006-11-09 at 04:09 -0500, Andrew Sullivan wrote: > On Mon, Nov 06, 2006 at 09:50:53PM +, Simon Riggs wrote: > > - There are specific issues with the optimizer's ability to understand > > dead row numbers, which can in some cases lead to SeqScan plans that are > > inappropriate when ta

[HACKERS] Frequent Update Project: Design Overview of HOT Updates

Design Overview of HOT Updates -- The objective is to increase the speed of the UPDATE case, while minimizing the overall negative effects of the UPDATE. We refer to the general requirement as *Frequent Update Optimization*, though this design proposal is for Heap Overf

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

Nice idea, just one question: On Thu, Nov 09, 2006 at 05:13:16PM +, Simon Riggs wrote: > Behavioural Characteristics > --- > > With HOT, it is easily possible that the chain of prior versions spans > many blocks. The chain always starts with the block of the root tuple

Re: [HACKERS] Frequent Update Project: Design Overview of HOTUpdates

On Thu, 2006-11-09 at 18:49 +0100, Martijn van Oosterhout wrote: > Nice idea, just one question: > It seems to me that bitmap index scans will get these same > characteristics also, right? The bitmap scan will have to follow the > chain of any possibly matching tuple in any of the blocks that are

Re: [HACKERS] Introducing an advanced Frequent Update Optimization

On Tue, 2006-11-07 at 15:00 +1300, Mark Kirkwood wrote: > Simon Riggs wrote: > > EnterpriseDB has been running a research project to improve the > > performance of heavily updated tables. We have a number of approaches > > prototyped and we'd like to discuss the best of these now on -hackers > > fo

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

Simon, > If we perform an update that meets the HOT criteria then we put the > new version into the overflow relation; we describe this as a HOT > UPDATE. If we perform an update that does not meet the criteria, then we > carry on with the existing/old MVCC behaviour; we describe this as a > non-H

Re: [HACKERS] Introducing an advanced Frequent Update Optimization

Quoth [EMAIL PROTECTED] ("Simon Riggs"): > On Tue, 2006-11-07 at 15:00 +1300, Mark Kirkwood wrote: >> Simon Riggs wrote: >> > EnterpriseDB has been running a research project to improve the >> > performance of heavily updated tables. We have a number of approaches >> > prototyped and we'd like to d

Re: [HACKERS] plperl/plperlu interaction

Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Anyway, it is probably not expected by many users that loading a module in plperlu makes it available to plperl - I was slightly surprised myself to see it work and I am probably more aware than most of perl and plperl subtleties. I

Re: [HACKERS] Frequent Update Project: Design Overview of HOTUpdates

On Thu, 2006-11-09 at 13:21 -0800, Josh Berkus wrote: > Simon, > > > If we perform an update that meets the HOT criteria then we put the > > new version into the overflow relation; we describe this as a HOT > > UPDATE. If we perform an update that does not meet the criteria, then we > > carry on w

Re: [HACKERS] Introducing an advanced Frequent Update Optimization

"Simon Riggs" <[EMAIL PROTECTED]> writes: > ...and of course it would be good if LISTEN/NOTIFY were able to use this > concept also, to help Slony along also. LISTEN/NOTIFY are overdue to be rewritten to not use a table at all, so I'm not particularly worried about whether this idea is applicable

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

"Simon Riggs" <[EMAIL PROTECTED]> writes: > As more UPDATEs take place these tuple chains would grow, making > locating the latest tuple take progressively longer. This is the part that bothers me --- particularly the random-access nature of the search. I wonder whether you couldn't do something

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

Tom, > Actually, you omitted to mention the locking aspects of moving tuples > around --- exactly how are you going to make that work without breaking > concurrent scans? I believe that's the "unsolved technical issue" in the prototype, unless Pavan has solved it in the last two weeks. Pavan?

[HACKERS] Various breakages in new contrib/isn module

It occurred to me to run the regression tests type_sanity and opr_sanity over the contrib/isn code. (The easy way to do this is to copy the isn install script into the regress/sql directory and then add it to serial_schedule just before those two tests.) It turned up a couple of moderately seriou

Re: [HACKERS] plperl/plperlu interaction

Mark Dilger wrote: > Tom Lane wrote: >> Andrew Dunstan <[EMAIL PROTECTED]> writes: >>> Anyway, it is probably not expected by many users that loading a module >>> in plperlu makes it available to plperl - I was slightly surprised myself to see it work and I am probably more aware than most of perl

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Josh Berkus wrote: Tom,> Actually, you omitted to mention the locking aspects of moving tuples> around --- exactly how are you going to make that work without breaking> concurrent scans?I believe that's the "unsolved technical issue" in the prototype, unless Pavan h

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On 11/10/06, Josh Berkus wrote: >> I believe that's the "unsolved technical issue" in the prototype, unless >> Pavan has solved it in the last two weeks. Pavan? >> > When an overflow tuple is copied back to the main heap, the overflow tuple > is >

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes:> On 11/10/06, Josh Berkus < josh@agliodbs.com> wrote: >> I believe that's the "unsolved technical issue" in the prototype, unless>> Pavan has solved it in the last two weeks.   Pavan?>>> When an overflow

Re: [HACKERS] Frequent Update Project: Design Overview of HOT

Ühel kenal päeval, N, 2006-11-09 kell 18:28, kirjutas Tom Lane: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > As more UPDATEs take place these tuple chains would grow, making > > locating the latest tuple take progressively longer. > > This is the part that bothers me --- particularly the random

[HACKERS] beta3 CFLAGS issue on openbsd

I was trying to compile 8.2beta3 on openbsd, and ran into an interesting issue. My account on the particular openbsd box has some restrictive ulimit settings, so I don't have a lot of memory to work with. I was getting an out of memory issue linking postgres, while I did not before. I figured out

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On 11/10/06, Tom Lane <[EMAIL PROTECTED]> wrote: > (2) Isn't this full of race conditions? > I agree, there could be race conditions. But IMO we can handle those. Doubtless you can prevent races by introducing a bunch of additional locking. The qu

Re: [HACKERS] Frequent Update Project: Design Overview of HOT

Ühel kenal päeval, R, 2006-11-10 kell 09:06, kirjutas Hannu Krosing: > Ühel kenal päeval, N, 2006-11-09 kell 18:28, kirjutas Tom Lane: > > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > As more UPDATEs take place these tuple chains would grow, making > > > locating the latest tuple take progressiv

Re: [HACKERS] beta3 CFLAGS issue on openbsd

Jeremy Drake <[EMAIL PROTECTED]> writes: > I figured out that the -g flag was being surreptitiously added to my > CFLAGS. It was like pulling teeth trying to get the -g flag out. I believe that this is a default behavior of autoconf scripts. I remember having done some ugly hacks years ago to pre