Hi, On 2020-09-17 14:20:50 +1200, Thomas Munro wrote: > I wonder if there is a way we could make "Minimal Tuples but with the > length travelling separately (and perhaps chopped off?)" into a > first-class concept... It's also a shame to be schlepping a bunch of > padding bytes around.
There really is no justification for having MinimalTuples, as we have them today at least, anymore. We used to rely on being able to construct pointers to MinimalTuples that are mostly compatible with HeapTuple. But I think there's none of those left since PG 12. I think it'd make a bit more sense to do some steps towards having a more suitable "minimal" tuple representation, rather than doing this local, pretty ugly, hacks. A good way would be to just starting to remove the padding, unnecessary fields etc from MinimalTuple. I also think that it'd be good to look at a few of the other places that are heavily bottlenecked by MinimalTuple overhead before designing new API around this. IIRC it's e.g. very easy to see hash joins spending a lot of time doing MinimalTuple copies & conversions. > > tuple = (MinimalTuple) data; > - Assert(tuple->t_len == nbytes); > + tuple->t_len = nbytes; > > Hmm, so you have to scribble on shared memory on the receiving side. Ick, I would really like to avoid this. > > Thomas, I guess you had a different approach in mind when you said > > about "returning either success or > > hey-that-buffer's-too-small-I-need-N-bytes". But what looks clear to > > Yeah I tried some things like that, but I wasn't satisfied with any of > them; basically the extra work involved in negotiating the size was a > bit too high. On the other hand, because my interface was "please > write a MinimalTuple here!", it had the option to *form* a > MinimalTuple directly in place, whereas your approach can only avoid > creating and destroying a temporary tuple when the source is a heap > tuple. There's a lot of cases where the source is a virtual slot (since we'll often project stuff below Gather). So it'd be quite advantageous to avoid building an unnecessary HeapTuple in those cases. I wonder if it would be sensible to build minimal tuples using tts_values/isnull in some cases. This might even be advantageous in case of heap / minimal tuples, because IIRC right now the code will materialize the slot unnecessarily. Not sure how common that is. Greetings, Andres Freund
