Bruce Momjian wrote: > > Uh, I thought the approach was to create type-specific in/out functions, > and add casting so every time there were referenced, they would expand > to a varlena structure in memory.
Oh, one more thing. You are going to need to teach the code that walks through a tuple attributes about the short header types. I think you should set pg_type.typlen = -3 (vs -1 for varlena) and put your macro code there too. (As an example, see the macro att_addlength().) I know it is kind of odd to have a data type that is only used on disk, and not in memory, but I see this as a baby varlena type, used only to store and get varlena values using less disk space. --------------------------------------------------------------------------- > > Gregory Stark wrote: > > > > I've been looking at this again and had a few conversations about it. This > > may > > be easier than I had originally thought but there's one major issue that's > > bugging me. Do you see any way to avoid having every user function > > everywhere > > use a new macro api instead of VARDATA/VARATT_DATA and VARSIZE/VARATT_SIZEP? > > > > The two approaches I see are either > > > > a) To have two sets of macros, one of which, VARATT_DATA and VARATT_SIZEP > > are > > for constructing new tuples and behaves exactly as it does now. So you > > always > > construct a four-byte header datum. Then in heap_form*tuple we check if you > > can use a shorter header and convert. VARDATA/VARSIZE would be for looking > > at > > existing datums and would interpret the header bits. > > > > This seems very fragile since one stray call site using VARATT_DATA to find > > the data in an existing datum would cause random bugs that only occur rarely > > in certain circumstances. It would even work as long as the size is filled > > in > > with VARATT_SIZEP first which it usually is, but fail if someone changes the > > order of the statements. > > > > or > > > > b) throw away VARATT_DATA and VARATT_SIZEP and make all user function > > everywhere change over to a new macro api. That seems like a pretty big > > burden. It's safer but means every contrib module would have to be updated > > and > > so on. > > > > I'm hoping I'm missing something and there's a way to do this without > > breaking > > the api for every user function. > > > > > > -- Start of included mail From: Tom Lane <[EMAIL PROTECTED]> > > > To: Gregory Stark <[EMAIL PROTECTED]> > > cc: Gregory Stark <[EMAIL PROTECTED]>, Bruce Momjian <[EMAIL PROTECTED]>, > > Peter Eisentraut <[EMAIL PROTECTED]>, > > pgsql-hackers@postgresql.org, > > Martijn van Oosterhout <kleptog@svana.org> > > Subject: Re: [HACKERS] Fixed length data types issue > > Date: Mon, 11 Sep 2006 13:15:43 -0400 > > Lines: 64 > > Xref: stark.xeocode.com work.enterprisedb:683 > > > Gregory Stark <[EMAIL PROTECTED]> writes: > > > In any case it seems a bit backwards to me. Wouldn't it be better to > > > preserve bits in the case of short length words where they're precious > > > rather than long ones? If we make 0xxxxxxx the 1-byte case it means ... > > > > Well, I don't find that real persuasive: you're saying that it's > > important to have a 1-byte not 2-byte header for datums between 64 and > > 127 bytes long. Which is by definition less than a 2% savings for those > > values. I think its's more important to pick bitpatterns that reduce > > the number of cases heap_deform_tuple has to think about while decoding > > the length of a field --- every "if" in that inner loop is expensive. > > > > I realized this morning that if we are going to preserve the rule that > > 4-byte-header and compressed-header cases can be distinguished from the > > data alone, there is no reason to be very worried about whether the > > 2-byte cases can represent the maximal length of an in-line datum. > > If you want to do 16K inline (and your page is big enough for that) > > you can just fall back to the 4-byte-header case. So there's no real > > disadvantage if the 2-byte headers can only go up to 4K or so. This > > gives us some more flexibility in the bitpattern choices. > > > > Another thought that occurred to me is that if we preserve the > > convention that a length word's value includes itself, then for a > > 1-byte header the bit pattern 10000000 is meaningless --- the count > > has to be at least 1. So one trick we could play is to take over > > this value as the signal for "toast pointer follows", with the > > assumption that the tuple-decoder code knows a-priori how big a > > toast pointer is. I am not real enamored of this, because it certainly > > adds one case to the inner heap_deform_tuple loop and it'll give us > > problems if we ever want more than one kind of toast pointer. But > > it's a possibility. > > > > Anyway, a couple of encodings that I'm thinking about now involve > > limiting uncompressed data to 1G (same as now), so that we can play > > with the first 2 bits instead of just 1: > > > > 00xxxxxx 4-byte length word, aligned, uncompressed data (up to 1G) > > 01xxxxxx 4-byte length word, aligned, compressed data (up to 1G) > > 100xxxxx 1-byte length word, unaligned, TOAST pointer > > 1010xxxx 2-byte length word, unaligned, uncompressed data (up to 4K) > > 1011xxxx 2-byte length word, unaligned, compressed data (up to 4K) > > 11xxxxxx 1-byte length word, unaligned, uncompressed data (up to 63b) > > > > or > > > > 00xxxxxx 4-byte length word, aligned, uncompressed data (up to 1G) > > 010xxxxx 2-byte length word, unaligned, uncompressed data (up to 8K) > > 011xxxxx 2-byte length word, unaligned, compressed data (up to 8K) > > 10000000 1-byte length word, unaligned, TOAST pointer > > 1xxxxxxx 1-byte length word, unaligned, uncompressed data (up to 127b) > > (xxxxxxx not all zero) > > > > This second choice allows longer datums in both the 1-byte and 2-byte > > header formats, but it hardwires the length of a TOAST pointer and > > requires four cases to be distinguished in the inner loop; the first > > choice only requires three cases, because TOAST pointer and 1-byte > > header can be handled by the same rule "length is low 6 bits of byte". > > The second choice also loses the ability to store in-line compressed > > data above 8K, but that's probably an insignificant loss. > > > > There's more than one way to do it ... > > > > regards, tom lane > > > -- End of included mail. > > > > > > > -- > > Gregory Stark > > EnterpriseDB http://www.enterprisedb.com > > -- > Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us > EnterpriseDB http://www.enterprisedb.com > > + If your life is a hard drive, Christ can be your backup. + > > ---------------------------(end of broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org
