[HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
While looking at the recently-noticed problem that HashAggregate nodes store more columns of the input than they need to, I couldn't help noticing how much of the hashtable space goes into HeapTuple header overhead. A couple months ago we were able to get a useful improvement in sorting by not

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Martijn van Oosterhout
On Mon, Jun 26, 2006 at 10:36:00AM -0400, Tom Lane wrote: While looking at the recently-noticed problem that HashAggregate nodes store more columns of the input than they need to, I couldn't help noticing how much of the hashtable space goes into HeapTuple header overhead. A couple months ago

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: On Mon, Jun 26, 2006 at 10:36:00AM -0400, Tom Lane wrote: Unlike the case with sort temp files, it's important to be able to access the stored data without moving/copying it. So, not wishing to duplicate all the tuple access machinery we have

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Bort, Paul
Tom Lane said: To make use of a TruncatedTuple, we'd set up a temporary HeapTupleData struct with its t_data field pointing 16 bytes before the start of the TruncatedTuple. As long as the code using it never tries to access any of the missing fields (t_xmin through t_ctid), this would

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
Bort, Paul [EMAIL PROTECTED] writes: Tom Lane said: As long as the code using it never tries to access any of the missing fields (t_xmin through t_ctid), this would work exactly like a normal HeapTuple. This sounds like a security risk. No more than any other wild-pointer problem. At the

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Simon Riggs
On Mon, 2006-06-26 at 16:48 +0200, Martijn van Oosterhout wrote: On Mon, Jun 26, 2006 at 10:36:00AM -0400, Tom Lane wrote: While looking at the recently-noticed problem that HashAggregate nodes store more columns of the input than they need to, I couldn't help noticing how much of the

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2006-06-26 at 16:48 +0200, Martijn van Oosterhout wrote: Anyway, I think it's a good idea. Most places in the backend after the SeqScan/IndexScan node really don't care about most of the header fields and being able to drop them would be nice. I

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Simon Riggs
On Mon, 2006-06-26 at 14:36 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2006-06-26 at 16:48 +0200, Martijn van Oosterhout wrote: Anyway, I think it's a good idea. Most places in the backend after the SeqScan/IndexScan node really don't care about most of the header

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2006-06-26 at 14:36 -0400, Tom Lane wrote: I thought for awhile about MemoryTuple (as contrasted to HeapTuple) but that seems too generic. Any other thoughts? I like MemoryTuple but since we only use it when we go to disk... ExecutorTuple,

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
I wrote: There isn't any benefit except where we collect lots of tuples, which is to say tuplesort/tuplestore/tuplehashtable. In other places in the executor, there's basically only one transient tuple in existence per plan node; jumping through hoops to save 16 bytes per plan node is just

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Simon Riggs
On Mon, 2006-06-26 at 16:43 -0400, Tom Lane wrote: I wrote: There isn't any benefit except where we collect lots of tuples, which is to say tuplesort/tuplestore/tuplehashtable. In other places in the executor, there's basically only one transient tuple in existence per plan node; jumping

Re: [HACKERS] Truncated tuples for tuple hash tables

2006-06-26 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2006-06-26 at 16:43 -0400, Tom Lane wrote: analyze.c (tuples collected in-memory for stats analysis) Do we save enough there for us to care? Possibly not --- it's certainly low-priority, but I listed it for completeness.