Uh, I thought the approach was to create type-specific in/out functions,
and add casting so every time there were referenced, they would expand
to a varlena structure in memory.


Gregory Stark wrote:
> I've been looking at this again and had a few conversations about it. This may
> be easier than I had originally thought but there's one major issue that's
> bugging me. Do you see any way to avoid having every user function everywhere
> use a new macro api instead of VARDATA/VARATT_DATA and VARSIZE/VARATT_SIZEP?
> The two approaches I see are either 
> a) To have two sets of macros, one of which, VARATT_DATA and VARATT_SIZEP are
> for constructing new tuples and behaves exactly as it does now. So you always
> construct a four-byte header datum. Then in heap_form*tuple we check if you
> can use a shorter header and convert. VARDATA/VARSIZE would be for looking at
> existing datums and would interpret the header bits.
> This seems very fragile since one stray call site using VARATT_DATA to find
> the data in an existing datum would cause random bugs that only occur rarely
> in certain circumstances. It would even work as long as the size is filled in
> with VARATT_SIZEP first which it usually is, but fail if someone changes the
> order of the statements.
> or 
> b) throw away VARATT_DATA and VARATT_SIZEP and make all user function
> everywhere change over to a new macro api. That seems like a pretty big
> burden. It's safer but means every contrib module would have to be updated and
> so on.
> I'm hoping I'm missing something and there's a way to do this without breaking
> the api for every user function.

-- Start of included mail From: Tom Lane <[EMAIL PROTECTED]>

> To: Gregory Stark <[EMAIL PROTECTED]>
> cc: Gregory Stark <[EMAIL PROTECTED]>, Bruce Momjian <[EMAIL PROTECTED]>, 
>               Peter Eisentraut <[EMAIL PROTECTED]>, 
> pgsql-hackers@postgresql.org, 
>               Martijn van Oosterhout <kleptog@svana.org>
> Subject: Re: [HACKERS] Fixed length data types issue 
> Date: Mon, 11 Sep 2006 13:15:43 -0400
> Lines: 64
> Xref: stark.xeocode.com work.enterprisedb:683

> Gregory Stark <[EMAIL PROTECTED]> writes:
> > In any case it seems a bit backwards to me. Wouldn't it be better to
> > preserve bits in the case of short length words where they're precious
> > rather than long ones? If we make 0xxxxxxx the 1-byte case it means ...
> Well, I don't find that real persuasive: you're saying that it's
> important to have a 1-byte not 2-byte header for datums between 64 and
> 127 bytes long.  Which is by definition less than a 2% savings for those
> values.  I think its's more important to pick bitpatterns that reduce
> the number of cases heap_deform_tuple has to think about while decoding
> the length of a field --- every "if" in that inner loop is expensive.
> I realized this morning that if we are going to preserve the rule that
> 4-byte-header and compressed-header cases can be distinguished from the
> data alone, there is no reason to be very worried about whether the
> 2-byte cases can represent the maximal length of an in-line datum.
> If you want to do 16K inline (and your page is big enough for that)
> you can just fall back to the 4-byte-header case.  So there's no real
> disadvantage if the 2-byte headers can only go up to 4K or so.  This
> gives us some more flexibility in the bitpattern choices.
> Another thought that occurred to me is that if we preserve the
> convention that a length word's value includes itself, then for a
> 1-byte header the bit pattern 10000000 is meaningless --- the count
> has to be at least 1.  So one trick we could play is to take over
> this value as the signal for "toast pointer follows", with the
> assumption that the tuple-decoder code knows a-priori how big a
> toast pointer is.  I am not real enamored of this, because it certainly
> adds one case to the inner heap_deform_tuple loop and it'll give us
> problems if we ever want more than one kind of toast pointer.  But
> it's a possibility.
> Anyway, a couple of encodings that I'm thinking about now involve
> limiting uncompressed data to 1G (same as now), so that we can play
> with the first 2 bits instead of just 1:
> 00xxxxxx      4-byte length word, aligned, uncompressed data (up to 1G)
> 01xxxxxx      4-byte length word, aligned, compressed data (up to 1G)
> 100xxxxx      1-byte length word, unaligned, TOAST pointer
> 1010xxxx      2-byte length word, unaligned, uncompressed data (up to 4K)
> 1011xxxx      2-byte length word, unaligned, compressed data (up to 4K)
> 11xxxxxx      1-byte length word, unaligned, uncompressed data (up to 63b)
> or
> 00xxxxxx      4-byte length word, aligned, uncompressed data (up to 1G)
> 010xxxxx      2-byte length word, unaligned, uncompressed data (up to 8K)
> 011xxxxx      2-byte length word, unaligned, compressed data (up to 8K)
> 10000000      1-byte length word, unaligned, TOAST pointer
> 1xxxxxxx      1-byte length word, unaligned, uncompressed data (up to 127b)
>               (xxxxxxx not all zero)
> This second choice allows longer datums in both the 1-byte and 2-byte
> header formats, but it hardwires the length of a TOAST pointer and
> requires four cases to be distinguished in the inner loop; the first
> choice only requires three cases, because TOAST pointer and 1-byte
> header can be handled by the same rule "length is low 6 bits of byte".
> The second choice also loses the ability to store in-line compressed
> data above 8K, but that's probably an insignificant loss.
> There's more than one way to do it ...
>                       regards, tom lane
-- End of included mail.

> -- 
>   Gregory Stark
>   EnterpriseDB          http://www.enterprisedb.com

  Bruce Momjian  <[EMAIL PROTECTED]>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Reply via email to