On Oct 3, 2006, at 4:06 PM, Merlin Moncure wrote:
On 10/3/06, Gregory Stark <[EMAIL PROTECTED]> wrote:
I can't shake the feeling that merely tweaking the way our varlenas work with a shortvarlena or with compressed varlena headers is missing the real source of our headaches. It seems very strange to me to be trying to step through a tuple with length bits at the head of every field. It's a lot of work spent dealing with a terribly inconvenient format when we can pick the format to be
whatever we like.

one advantage of the current system is that columns with nulls do not
require any storage.  so you can alter table add column for free on a
really big table.  ISTM that your approch would require moving all the
static fields in if you added a static field regardless, right?

I'm thinking that for Greg's ideas to be workable, we'll need to divorce the on-disk format from what was specified in CREATE TABLE, specifically so we can do things like put all the fixed-width stuff in front of the variable-width stuff (of course we could also further optimize placement beyond that).

IIRC, the show-stopper for doing that is how to deal with DDL changes. While we could try and get cute about that, there is a brute- force method that would work without a doubt: store some kind of catalog version number in each tuple (or maybe just in each page, since you could theoretically convert an entire page to a different format without too big a penalty while you've already got it in memory.

There are some caveats to this... there will be some limit on how many DDL changes you can make until you run out of version numbers. Worst-case, we could provide a command that would run through the entire table, ensuring that everything is up to the current version. Of course, we'd want some way to throttle that, but I don't think that'd be terribly difficult. One nice thing is that you shouldn't need to mess with any visibility info when you run this, so it should be able to just do everything in-place.

BTW, it seems like what we're really looking at between this and discussion of visibility changes, etc. is essentially re-designing the entire storage layout (for better or for worse).
Jim Nasby                                    [EMAIL PROTECTED]
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Reply via email to