Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Jim C. Nasby
On Thu, Feb 23, 2006 at 01:15:32PM -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: > >> The Datum is just a pointer into the original tuple in that case. > > > That would still result in a speedup since you don't hav

Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: >> The Datum is just a pointer into the original tuple in that case. > That would still result in a speedup since you don't have to figure out > where the field begins though, right? I'm curious

Re: [PATCHES] WIP: further sorting speedup

2006-02-23 Thread Jim C. Nasby
On Wed, Feb 22, 2006 at 10:18:48PM -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > Stupid question... how are varlena's handled in Datum? > > The Datum is just a pointer into the original tuple in that case. That would still result in a speedup since you don't have to figu

Re: [PATCHES] WIP: further sorting speedup

2006-02-22 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Stupid question... how are varlena's handled in Datum? The Datum is just a pointer into the original tuple in that case. regards, tom lane ---(end of broadcast)--- TIP 3: Have you

Re: [PATCHES] WIP: further sorting speedup

2006-02-22 Thread Jim C. Nasby
On Sun, Feb 19, 2006 at 09:40:46PM -0500, Tom Lane wrote: > "tape". This increases the space needed by 8 or 12 bytes (depending on > sizeof(Datum)) per in-memory tuple, but doesn't cost anything as far as Stupid question... how are varlena's handled in Datum? -- Jim C. Nasby, Sr. Engineering Con

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Michael Glaesemann
On Feb 21, 2006, at 14:24 , Tom Lane wrote: Michael Glaesemann <[EMAIL PROTECTED]> writes: Most of this is way above my head, but I'm trying to follow along: Right, it's whatever is the sort key for this particular sort. Thanks, Tom. I think I may actually be starting to understand this

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Tom Lane
Michael Glaesemann <[EMAIL PROTECTED]> writes: > Most of this is way above my head, but I'm trying to follow along: > when you say first key and full key, are these related to relation > keys (e.g., primary key) or attributes that are used in sorting > (regardless of whether they're a key or

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Michael Glaesemann
On Feb 21, 2006, at 3:45 , Simon Riggs wrote: On Sun, 2006-02-19 at 21:40 -0500, Tom Lane wrote: After applying Simon's recent sort patch, I was doing some profiling and noticed that sorting spends an unreasonably large fraction of its time extracting datums from tuples (heap_getattr or in

Re: [PATCHES] WIP: further sorting speedup

2006-02-20 Thread Simon Riggs
On Sun, 2006-02-19 at 21:40 -0500, Tom Lane wrote: > After applying Simon's recent sort patch, I was doing some profiling and > noticed that sorting spends an unreasonably large fraction of its time > extracting datums from tuples (heap_getattr or index_getattr). The > attached patch does somethin

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Luke Lonergan
Title: Re: [PATCHES] WIP: further sorting speedup The improvement was pre-Simon’s patch, and it came from implementing a single pass merge instead of a variable pass based on the number of tapes, as it is in Knuth’s tape algorithm.  Also, the additional tricks in logtape.c were higher in the

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > So you know, we=B9ve done some more work on the external sort to remove the > =B3tape=B2 abstraction from the code, which makes a significant improvement. Improvement where? That code's down in the noise so far as I can tell. I see results like this (

Re: [PATCHES] WIP: further sorting speedup

2006-02-19 Thread Luke Lonergan
Title: Re: [PATCHES] WIP: further sorting speedup Cool! We’ll test this sometime soon and get back to you.  We’re kind of jammed this week, hopefully we’ll get some time. So you know, we’ve done some more work on the external sort to remove the “tape” abstraction from the code, which makes a