Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-29 Thread Luke Lonergan
Jeff, On 9/29/05 10:44 AM, Jeffrey W. Baker [EMAIL PROTECTED] wrote: On Thu, 2005-09-29 at 10:06 -0700, Luke Lonergan wrote: Looking through tuplesort.c, I have a couple of initial ideas. Are we allowed to fork here? That would open up the possibility of using the CPU and the I/O in

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-28 Thread Kevin Grittner
I can't help wondering how a couple thousand context switches per second would affect the attempt to load disk info into the L1 and L2 caches. That's pretty much the low end of what I see when the server is under any significant load. ---(end of

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-28 Thread Ron Peacetree
From: Josh Berkus josh@agliodbs.com ent: Sep 27, 2005 12:15 PM To: Ron Peacetree [EMAIL PROTECTED] Subject: Re: [HACKERS] [PERFORM] A Better External Sort? I've somehow missed part of this thread, which is a shame since this is an area of primary concern for me. Your suggested algorithm seems

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-28 Thread Jeffrey W. Baker
On Wed, 2005-09-28 at 12:03 -0400, Ron Peacetree wrote: From: Jeffrey W. Baker [EMAIL PROTECTED] Sent: Sep 27, 2005 1:26 PM To: Ron Peacetree [EMAIL PROTECTED] Subject: Re: [HACKERS] [PERFORM] A Better External Sort? On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote: That Btree can

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Jeffrey W. Baker
On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote: That Btree can be used to generate a physical reordering of the data in one pass, but that's the weakest use for it. The more powerful uses involve allowing the Btree to persist and using it for more efficient re-searches or combining

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Ron Peacetree
From: Dann Corbit [EMAIL PROTECTED] Sent: Sep 26, 2005 5:13 PM To: Ron Peacetree [EMAIL PROTECTED], pgsql-hackers@postgresql.org, pgsql-performance@postgresql.org Subject: RE: [HACKERS] [PERFORM] A Better External Sort? I think that the btrees are going to be O(n*log(n)) in construction

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Ron Peacetree
SECOND ATTEMPT AT POST. Web mailer appears to have eaten first one. I apologize in advance if anyone gets two versions of this post. =r From: Tom Lane [EMAIL PROTECTED] Sent: Sep 26, 2005 9:42 PM Subject: Re: [HACKERS] [PERFORM] A Better External Sort? So far, you've blithely assumed that you

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Jonah H. Harris
to haveeaten first one.I apologize in advance if anyone gets twoversions of this post.=rFrom: Tom Lane [EMAIL PROTECTED] Sent: Sep 26, 2005 9:42 PMSubject: Re: [HACKERS] [PERFORM] A Better External Sort?So far, you've blithely assumed that you know the size of a cache line,the sizes of L1 and L2 cache

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Josh Berkus
Ron, I've somehow missed part of this thread, which is a shame since this is an area of primary concern for me. Your suggested algorithm seems to be designed to relieve I/O load by making more use of the CPU. (if I followed it correctly). However, that's not PostgreSQL's problem;

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-26 Thread Dann Corbit
-Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Ron Peacetree Sent: Monday, September 26, 2005 10:47 AM To: pgsql-hackers@postgresql.org; pgsql-performance@postgresql.org Subject: [HACKERS] [PERFORM] A Better External Sort? From:

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-26 Thread Jonah H. Harris
Ron, Having rested my brain for the last few days, your theory made for interesting reading... Rather than argue the technical specs, I'd love to see an implementation :) -JonahOn 9/26/05, Dann Corbit [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED]

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-26 Thread Tom Lane
Ron Peacetree [EMAIL PROTECTED] writes: Let's start by assuming that an element is = in size to a cache line and a node fits into L1 DCache. [ much else snipped ] So far, you've blithely assumed that you know the size of a cache line, the sizes of L1 and L2 cache, and that you are working

<    1   2