Re: [HACKERS] Replacement Selection

2007-12-01 Thread mac_man2005
in puttuple_common(), the transition from an internal to external sort is performed at the bottom of the TSS_INITIAL case in the main switch statement. The transition? Do we internal sort somewhere else and then external sort here in tuplesort.c? The function dumptuples() heapifies the

Re: [HACKERS] Replacement Selection

2007-12-01 Thread Gregory Stark
[EMAIL PROTECTED] writes: The function dumptuples() heapifies the in-core tuples (divides the in-core tuples into initial runs and then advances the state to TSS_BUILDRUNS). Cannot see where dumptuples() advances the state to TSS_BUILDRUNS. I expected something like state-status =

Re: [HACKERS] Replacement Selection

2007-11-27 Thread mac_man2005
Hi to all. It seems a previous mail of mine with following body hasn't been sent. Sorry for possibly getting it twice. Actually I have now modified that body, so it's worth to read it once again. Thanks for your attention. Regards. PREVIOUS MAIL-- Well,

Re: [HACKERS] Replacement Selection

2007-11-27 Thread Simon Riggs
On Tue, 2007-11-27 at 09:25 +0100, [EMAIL PROTECTED] wrote: Others optimizations, for example, can be done with the virtual concatenation technique: storing a cache of couples (first_element,last_element) for each created run. This could be useful in case we can find 2 couples

Re: [HACKERS] Replacement Selection

2007-11-27 Thread mac_man2005
] Replacement Selection On Tue, 2007-11-27 at 09:25 +0100, [EMAIL PROTECTED] wrote: Others optimizations, for example, can be done with the virtual concatenation technique: storing a cache of couples (first_element,last_element) for each created run. This could be useful in case we can find 2 couples

Re: [HACKERS] Replacement Selection

2007-11-27 Thread Simon Riggs
On Tue, 2007-11-27 at 17:49 +0100, [EMAIL PROTECTED] wrote: Any comment about Two Ways Replacement Selection (two heaps instead of just one) ? It might allow dynamic heap size management more easily than with a single heap. If you really think it will be better, try it. You'll learn loads,

[HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
Hi to all. I'm new. I'd like to integrate my code into PostgreSQL. It's the implementation of some refinements of Replacement Selection algorithm used for External Sorting. I have got some issue and preferibly I'd like to be supported by some developers that have something to do with it.

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Heikki Linnakangas
[EMAIL PROTECTED] wrote: I'm new. I'd like to integrate my code into PostgreSQL. It's the implementation of some refinements of Replacement Selection algorithm used for External Sorting. I have got some issue and preferibly I'd like to be supported by some developers that have something to do

Re: [HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
for your attention. -- From: Heikki Linnakangas [EMAIL PROTECTED] Sent: Monday, November 26, 2007 1:35 PM To: [EMAIL PROTECTED] Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Replacement Selection [EMAIL PROTECTED] wrote: I'm new. I'd like

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Alvaro Herrera
[EMAIL PROTECTED] wrote: Thanks for your support. I downloaded the source code of the last stable version of PostgreSQL. Where can I find the part related to the External Sorting algorithm (supposed to be Replacement Selection)? I mean, which is the file to be studied and/or modified

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Heikki Linnakangas
[EMAIL PROTECTED] wrote: I downloaded the source code of the last stable version of PostgreSQL. Where can I find the part related to the External Sorting algorithm (supposed to be Replacement Selection)? I mean, which is the file to be studied and/or modified and/or substituted? In

Re: [HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
, November 26, 2007 2:34 PM To: [EMAIL PROTECTED] Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Replacement Selection [EMAIL PROTECTED] wrote: I downloaded the source code of the last stable version of PostgreSQL. Where can I find the part related to the External Sorting algorithm (supposed

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Heikki Linnakangas
[EMAIL PROTECTED] wrote: Unfortunately I'm lost into the code... any good soul helping me to understand what should be the precise part to be modified? You haven't given any details on what you're trying to do. What are you trying to do? -- Heikki Linnakangas EnterpriseDB

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Alvaro Herrera
[EMAIL PROTECTED] wrote: Ok guys! Thanks for your help. Unfortunately I'm lost into the code... any good soul helping me to understand what should be the precise part to be modified? I think you should print the file and read it several times until you understand what's going on. Then you

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: Unfortunately I'm lost into the code... any good soul helping me to understand what should be the precise part to be modified? I think you should print the file and read it several times until you understand what's going on.

Fw: [HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
PROTECTED] Sent: Monday, November 26, 2007 6:00 PM To: Alvaro Herrera [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Replacement Selection Alvaro Herrera [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: Unfortunately I'm lost into the code... any good soul

Re: [HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
-hackers@postgresql.org Subject: Re: [HACKERS] Replacement Selection [EMAIL PROTECTED] wrote: Unfortunately I'm lost into the code... any good soul helping me to understand what should be the precise part to be modified? You haven't given any details on what you're trying to do. What are you

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Tom Lane
[EMAIL PROTECTED] writes: 3) Start run generation. As for this phase, I see PostgreSQL code (as Knuth algorithm) marks elements belonging to runs in otder to know which run they belong to and to know when the current heap has finished building the current run. I don't memorize this kind of

Re: [HACKERS] Replacement Selection

2007-11-26 Thread mac_man2005
@postgresql.org Subject: Re: [HACKERS] Replacement Selection [EMAIL PROTECTED] writes: 3) Start run generation. As for this phase, I see PostgreSQL code (as Knuth algorithm) marks elements belonging to runs in otder to know which run they belong to and to know when the current heap has finished

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Timothy J. Kordas
[EMAIL PROTECTED] wrote: I also implemented Replacement Selection (RS) so if I'm able to integrate my RS I hope I would be able to integrate the others too. The existing code implements RS. Tom asked you to describe what improvements you hope to make; I'm confident that he already

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Tom Lane
[EMAIL PROTECTED] writes: Anyway, even in my RS implementation a longer run is created. The first M initialization elements will surely form part of the current run. M is the memory size so at least a run sized M will be created. After initialization, the elements are not suddenly output,

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: AFAICS that produces runs that are *exactly* the same length as Knuth's method --- you're just using a different technique for detecting when the run is over, to wit record is not in heap vs record is in heap but with a higher run number. I guess you would

Re: [HACKERS] Replacement Selection

2007-11-26 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I guess you would save some comparisons while the heap is shrinking, but it's not at all clear that you'd save more than what it will cost you to re-heapify all the dead records once the run is over. This sounded