Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Jim C. Nasby
On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Stupid question... how are varlena's handled in Datum? The Datum is just a pointer into the original tuple in that case. That would still result in a speedup since you don't have to figure out

Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: The Datum is just a pointer into the original tuple in that case. That would still result in a speedup since you don't have to figure out where the field begins though, right? I'm curious as to

Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Jim C. Nasby
On Thu, Feb 23, 2006 at 01:15:32PM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: The Datum is just a pointer into the original tuple in that case. That would still result in a speedup since you don't have to figure

Re: [PATCHES] WIP: further sorting speedup

2006-02-22 Thread Jim C. Nasby
On Sun, Feb 19, 2006 at 09:40:46PM -0500, Tom Lane wrote: tape. This increases the space needed by 8 or 12 bytes (depending on sizeof(Datum)) per in-memory tuple, but doesn't cost anything as far as Stupid question... how are varlena's handled in Datum? -- Jim C. Nasby, Sr. Engineering

Re: [PATCHES] WIP: further sorting speedup

2006-02-22 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: Stupid question... how are varlena's handled in Datum? The Datum is just a pointer into the original tuple in that case. regards, tom lane ---(end of broadcast)--- TIP 3: Have you

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Simon Riggs
On Sun, 2006-02-19 at 21:40 -0500, Tom Lane wrote: After applying Simon's recent sort patch, I was doing some profiling and noticed that sorting spends an unreasonably large fraction of its time extracting datums from tuples (heap_getattr or index_getattr). The attached patch does something

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Michael Glaesemann
On Feb 21, 2006, at 3:45 , Simon Riggs wrote: On Sun, 2006-02-19 at 21:40 -0500, Tom Lane wrote: After applying Simon's recent sort patch, I was doing some profiling and noticed that sorting spends an unreasonably large fraction of its time extracting datums from tuples (heap_getattr or

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Tom Lane
Michael Glaesemann [EMAIL PROTECTED] writes: Most of this is way above my head, but I'm trying to follow along: when you say first key and full key, are these related to relation keys (e.g., primary key) or attributes that are used in sorting (regardless of whether they're a key or not)?

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Michael Glaesemann
On Feb 21, 2006, at 14:24 , Tom Lane wrote: Michael Glaesemann [EMAIL PROTECTED] writes: Most of this is way above my head, but I'm trying to follow along: Right, it's whatever is the sort key for this particular sort. Thanks, Tom. I think I may actually be starting to understand this a

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Luke Lonergan
Title: Re: [PATCHES] WIP: further sorting speedup Cool! Well test this sometime soon and get back to you. Were kind of jammed this week, hopefully well get some time. So you know, weve done some more work on the external sort to remove the tape abstraction from the code, which makes

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Tom Lane
Luke Lonergan [EMAIL PROTECTED] writes: So you know, we=B9ve done some more work on the external sort to remove the =B3tape=B2 abstraction from the code, which makes a significant improvement. Improvement where? That code's down in the noise so far as I can tell. I see results like this (with

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Luke Lonergan
Title: Re: [PATCHES] WIP: further sorting speedup The improvement was pre-Simons patch, and it came from implementing a single pass merge instead of a variable pass based on the number of tapes, as it is in Knuths tape algorithm. Also, the additional tricks in logtape.c were higher