Re: [HACKERS] modifying the tbale function

2007-03-20 Thread Andrew Dunstan
Florian G. Pflug wrote: Are we really sure that this isn't a solution in search of a problem? I think this really depends on how you define problem. Some people might think that select * from myfunc(...) limit 1 should stop and return a result after myfunc(...) has returned one row. Others will

Re: [HACKERS] modifying the tbale function

2007-03-20 Thread Hannu Krosing
Ühel kenal päeval, T, 2007-03-20 kell 09:35, kirjutas Andrew Dunstan: Florian G. Pflug wrote: Are we really sure that this isn't a solution in search of a problem? I think this really depends on how you define problem. Some people might think that select * from myfunc(...) limit 1 should

Re: [HACKERS] modifying the tbale function

2007-03-20 Thread Florian G. Pflug
Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: Here is a paper about portable userspace threads that I just googled. http://www.gnu.org/software/pth/rse-pmt.ps I'm not impressed --- the list of unsolved problems is a bit too long. One that seems a showstopper for our purposes is

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Heikki Linnakangas
Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: Returning control to the backend for every row returned would likely be excessive, but you could return once every k rows and get most of the benefits of both approaches (k might be on the order of 1000). However, this still leaves us

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Andrew Dunstan
Heikki Linnakangas wrote: Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: Returning control to the backend for every row returned would likely be excessive, but you could return once every k rows and get most of the benefits of both approaches (k might be on the order of 1000).

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Richard Huxton
Andrew Dunstan wrote: Actually, I think we could teach the PLs to do it - just not transparently, so we'd need to mark which functions used the new protocol. Such functions would get a state object as an implied first argument, so in plperl it might work like this (for a

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Andrew Dunstan
Richard Huxton wrote: Andrew Dunstan wrote: Actually, I think we could teach the PLs to do it - just not transparently, so we'd need to mark which functions used the new protocol. Such functions would get a state object as an implied first argument, so in plperl it might work like this (for

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Richard Huxton wrote: Is this not what we do with aggregate functions at present? Yes, more or less. That's what made me think of it. OTOH, before we rush out and do it someone needs to show that it's a net win. Yeah, because this isn't doing

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Gregory Stark
Andrew Dunstan [EMAIL PROTECTED] writes: Yes, more or less. That's what made me think of it. OTOH, before we rush out and do it someone needs to show that it's a net win. I agree with Tom that making tuplestore faster would probably be a much better investment of time. I don't think the

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Gregory Stark wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Yes, more or less. That's what made me think of it. OTOH, before we rush out and do it someone needs to show that it's a net win. I agree with Tom that making tuplestore faster would probably be a much better investment of time.

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Andrew Dunstan
Florian G. Pflug wrote: Just a thought - I believe that there are portable user-space thread implementations that contain little or no machine-specific code. What if postgres used one of those to switch from the PL into the executor and back after, say, 1000 rows were returned by the SFR? What

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Islam Hegazy
, March 19, 2007 12:18 PM Subject: Re: [HACKERS] modifying the tbale function Florian G. Pflug wrote: Just a thought - I believe that there are portable user-space thread implementations that contain little or no machine-specific code. What if postgres used one of those to switch from the PL

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Joe Conway
Andrew Dunstan wrote: Are we really sure that this isn't a solution in search of a problem? The need for value-per-call is real (examples mentioned down-thread) and was anticipated from day one of the SRF implementation (in fact the first patch I wrote was value-per-call, not materialize).

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Andrew Dunstan
Joe Conway wrote: Andrew Dunstan wrote: Are we really sure that this isn't a solution in search of a problem? The need for value-per-call is real (examples mentioned down-thread) and was anticipated from day one of the SRF implementation (in fact the first patch I wrote was value-per-call,

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Joe Conway
Andrew Dunstan wrote: Joe Conway wrote: Andrew Dunstan wrote: Are we really sure that this isn't a solution in search of a problem? The need for value-per-call is real (examples mentioned down-thread) and was anticipated from day one of the SRF implementation (in fact the first patch I wrote

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Gregory Stark
Florian G. Pflug [EMAIL PROTECTED] writes: Since context switching would occur only at two well-defined places (Some return_next_row function that PLs call when a SFR returns a row, and in the executor if no more previously returned rows from that SFR are available), this wouldn't introduce

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Andrew Dunstan wrote: Florian G. Pflug wrote: Just a thought - I believe that there are portable user-space thread implementations that contain little or no machine-specific code. What if postgres used one of those to switch from the PL into the executor and back after, say, 1000 rows were

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Florian G. Pflug
Gregory Stark wrote: Florian G. Pflug [EMAIL PROTECTED] writes: Since context switching would occur only at two well-defined places (Some return_next_row function that PLs call when a SFR returns a row, and in the executor if no more previously returned rows from that SFR are available), this

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Here is a paper about portable userspace threads that I just googled. http://www.gnu.org/software/pth/rse-pmt.ps I'm not impressed --- the list of unsolved problems is a bit too long. One that seems a showstopper for our purposes is lack of ability to

Re: [HACKERS] modifying the tbale function

2007-03-19 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: There might be trouble if a second function has to be executed with the same PL as an already running (but currently stopped) function. This would only work for PL that is thread-safe in some way. Seems a bit iffy. It strikes me that at least for

[HACKERS] modifying the tbale function

2007-03-18 Thread Islam Hegazy
Hi there I am trying to modify the execution of the table function to work in iterator fashion instead of materializing the output. I have been digging the Postgresql code source for about a month now and I can figure out where the execution of the table function works. I will be very grateful

Re: [HACKERS] modifying the tbale function

2007-03-18 Thread Martijn van Oosterhout
On Sun, Mar 18, 2007 at 01:54:55PM -0600, Islam Hegazy wrote: I am trying to modify the execution of the table function to work in iterator fashion instead of materializing the output. I have been digging the Postgresql code source for about a month now and I can figure out where the execution

Re: [HACKERS] modifying the tbale function

2007-03-18 Thread Andrew Dunstan
Martijn van Oosterhout wrote: What you want is that when you call a perl tablefunction that as soon as the perl function returns a row to return that to the caller. That means the perl interpreter has to be able to save all its state, return to the caller and when next called resume where it

Re: [HACKERS] modifying the tbale function

2007-03-18 Thread Neil Conway
Andrew Dunstan wrote: I'm not convinced it would be a huge gain anyway. Switching madly in and out of the perl interpreter at least is a known performance problem, IIRC Returning control to the backend for every row returned would likely be excessive, but you could return once every k rows

Re: [HACKERS] modifying the tbale function

2007-03-18 Thread Islam Hegazy
-hackers@postgresql.org Sent: Sunday, March 18, 2007 4:57 PM Subject: Re: [HACKERS] modifying the tbale function Andrew Dunstan wrote: I'm not convinced it would be a huge gain anyway. Switching madly in and out of the perl interpreter at least is a known performance problem, IIRC Returning

Re: [HACKERS] modifying the tbale function

2007-03-18 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: Returning control to the backend for every row returned would likely be excessive, but you could return once every k rows and get most of the benefits of both approaches (k might be on the order of 1000). However, this still leaves us with no idea how to