Re: [HACKERS] plpython is broken for recursive use

2015-10-17 Thread Andrew Dunstan
On 10/16/2015 10:03 PM, Tom Lane wrote: I wrote: This seems like a very Rube-Goldbergian way of setting up a local namespace for the user-defined code. I think perhaps what we should do is: 1. Compile the user-supplied code directly into a code object, without wrapping it in a "def". (Hence,

Re: [HACKERS] plpython is broken for recursive use

2015-10-16 Thread Tom Lane
I wrote: > This seems like a very Rube-Goldbergian way of setting up a local > namespace for the user-defined code. I think perhaps what we should do > is: > 1. Compile the user-supplied code directly into a code object, without > wrapping it in a "def". (Hence, PLy_procedure_munge_source goes a

Re: [HACKERS] plpython is broken for recursive use

2015-10-16 Thread Tom Lane
I wrote: > Anyway, the real problem here is the decision to pass procedure arguments > by assigning them to keys in the global dict. That is just brain-dead, > both because it means that recursive calls can't possibly work and because > it creates the bizarre scoping behavior mentioned in > http:/

Re: [HACKERS] plpython is broken for recursive use

2015-10-15 Thread Tom Lane
Andrew Dunstan writes: > On 10/15/2015 05:16 PM, Josh Berkus wrote: >> On 10/15/2015 01:10 PM, Tom Lane wrote: >>> I think this means that we should get rid of proc->globals and instead >>> manufacture a new globals dict locally in each call to PLy_exec_function >>> or PLy_exec_trigger. >> Don't

Re: [HACKERS] plpython is broken for recursive use

2015-10-15 Thread Joshua D. Drake
On 10/15/2015 02:16 PM, Josh Berkus wrote: On 10/15/2015 01:10 PM, Tom Lane wrote: I think this means that we should get rid of proc->globals and instead manufacture a new globals dict locally in each call to PLy_exec_function or PLy_exec_trigger. For SETOF functions it would be necessary to ke

Re: [HACKERS] plpython is broken for recursive use

2015-10-15 Thread Andrew Dunstan
On 10/15/2015 05:16 PM, Josh Berkus wrote: On 10/15/2015 01:10 PM, Tom Lane wrote: I think this means that we should get rid of proc->globals and instead manufacture a new globals dict locally in each call to PLy_exec_function or PLy_exec_trigger. For SETOF functions it would be necessary to

Re: [HACKERS] plpython is broken for recursive use

2015-10-15 Thread Josh Berkus
On 10/15/2015 01:10 PM, Tom Lane wrote: > I think this means that we should get rid of proc->globals and instead > manufacture a new globals dict locally in each call to PLy_exec_function > or PLy_exec_trigger. For SETOF functions it would be necessary to keep > the globals dict reference somewher

[HACKERS] plpython is broken for recursive use

2015-10-15 Thread Tom Lane
I looked into bug #13683, http://www.postgresql.org/message-id/20151015135804.3019.31...@wrigleys.postgresql.org It looks to me like plpython basically doesn't work at all for re-entrant calls to plpython functions, because all executions of a given function share the same "globals" dict, which do

Re: PL/Lua (was: [HACKERS] plpython implementation)

2013-07-01 Thread Andreas Karlsson
On 07/02/2013 01:54 AM, Luis Carvalho wrote: Peter Eisentraut wrote: On Mon, 2013-07-01 at 18:15 -0400, Luis Carvalho wrote: The project is maintained -- I don't know how to say when something is well-maintained, but small frequency of code updates is not one of my criteria; The bug tracker c

Re: PL/Lua (was: [HACKERS] plpython implementation)

2013-07-01 Thread Luis Carvalho
Peter Eisentraut wrote: > On Mon, 2013-07-01 at 18:15 -0400, Luis Carvalho wrote: > > The project is maintained -- I don't know how to say when something is > > well-maintained, but small frequency of code updates is not one of my > > criteria; > > The bug tracker contains bugs about build proble

Re: PL/Lua (was: [HACKERS] plpython implementation)

2013-07-01 Thread Peter Eisentraut
On Mon, 2013-07-01 at 18:15 -0400, Luis Carvalho wrote: > The project is maintained -- I don't know how to say when something is > well-maintained, but small frequency of code updates is not one of my > criteria; The bug tracker contains bugs about build problems with PG 8.4, 9.2, and 9.3, which

PL/Lua (was: [HACKERS] plpython implementation)

2013-07-01 Thread Luis Carvalho
Hi all, Claudio Freire wrote: > On Mon, Jul 1, 2013 at 2:29 AM, james wrote: > > On 01/07/2013 02:43, Claudio Freire wrote: > >> > >> In essence, you'd have to use another implementation. CPython guys > >> have left it very clear they don't intend to "fix" that, as they don't > >> consider it a b

Re: [HACKERS] plpython implementation

2013-07-01 Thread Peter Eisentraut
On 7/1/13 1:29 AM, james wrote: > Given how useful it is to have a scripting language that can be used > outside > of the database as well as inside it, would it be reasonable to consider > 'promoting' pllua? You can start promoting pllua by making it work with current PostgreSQL versions. It has

Re: [HACKERS] plpython implementation

2013-07-01 Thread Andres Freund
On 2013-06-30 22:43:52 -0300, Claudio Freire wrote: > Not only that, the CPython interpreter is rather fuzzy about the > division between interpreters. You can initialize multiple > interpreters, but they share a lot of state, so you can never fully > separate them. You'd have some state from the u

Re: [HACKERS] plpython implementation

2013-06-30 Thread Hannu Krosing
On 07/01/2013 07:53 AM, Claudio Freire wrote: > On Mon, Jul 1, 2013 at 2:29 AM, james wrote: >> On 01/07/2013 02:43, Claudio Freire wrote: >>> In essence, you'd have to use another implementation. CPython guys >>> have left it very clear they don't intend to "fix" that, as they don't >>> consider

Re: [HACKERS] plpython implementation

2013-06-30 Thread Claudio Freire
On Mon, Jul 1, 2013 at 2:29 AM, james wrote: > On 01/07/2013 02:43, Claudio Freire wrote: >> >> In essence, you'd have to use another implementation. CPython guys >> have left it very clear they don't intend to "fix" that, as they don't >> consider it a bug. It's just how it is. > > Given how usef

Re: [HACKERS] plpython implementation

2013-06-30 Thread james
On 01/07/2013 02:43, Claudio Freire wrote: In essence, you'd have to use another implementation. CPython guys have left it very clear they don't intend to "fix" that, as they don't consider it a bug. It's just how it is. Given how useful it is to have a scripting language that can be used outsid

Re: [HACKERS] plpython implementation

2013-06-30 Thread Claudio Freire
On Sun, Jun 30, 2013 at 9:45 AM, Andres Freund wrote: > On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: >> On 30 June 2013 14:31, Martijn van Oosterhout wrote: >> >> > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: >> > > > python does not any any sort of reliable sandbox, so there i

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:45, Andres Freund wrote: > On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: > > On 30 June 2013 14:31, Martijn van Oosterhout wrote: > > > > > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > > > python does not any any sort of reliable sandbox, so there is no

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andres Freund
On 2013-06-30 14:42:24 +0200, Szymon Guz wrote: > On 30 June 2013 14:31, Martijn van Oosterhout wrote: > > > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > > python does not any any sort of reliable sandbox, so there is no > > plpython, > > > > only plpythonu - hence only one i

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:31, Martijn van Oosterhout wrote: > On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > > python does not any any sort of reliable sandbox, so there is no > plpython, > > > only plpythonu - hence only one interpreter per backend is needed. > > > > > Is there any tra

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 08:18 AM, Szymon Guz wrote: python does not any any sort of reliable sandbox, so there is no plpython, only plpythonu - hence only one interpreter per backend is needed. Is there any track of the discussion that there is no way to make the sandbox? I managed to cr

Re: [HACKERS] plpython implementation

2013-06-30 Thread Martijn van Oosterhout
On Sun, Jun 30, 2013 at 02:18:07PM +0200, Szymon Guz wrote: > > python does not any any sort of reliable sandbox, so there is no plpython, > > only plpythonu - hence only one interpreter per backend is needed. > > > Is there any track of the discussion that there is no way to make the > sandbox? I

Re: [HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
On 30 June 2013 14:13, Andrew Dunstan wrote: > > On 06/30/2013 07:49 AM, Szymon Guz wrote: > >> I'm reading through plperl and plpython implementations and I don't >> understand the way they work. >> >> Comments for plperl say that there are two interpreters (trusted and >> untrusted) for each us

Re: [HACKERS] plpython implementation

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 07:49 AM, Szymon Guz wrote: I'm reading through plperl and plpython implementations and I don't understand the way they work. Comments for plperl say that there are two interpreters (trusted and untrusted) for each user session, and they are stored in a hash. Plpython version

Re: [HACKERS] plpython implementation

2013-06-30 Thread Martijn van Oosterhout
On Sun, Jun 30, 2013 at 01:49:53PM +0200, Szymon Guz wrote: > I'm reading through plperl and plpython implementations and I don't > understand the way they work. > > Comments for plperl say that there are two interpreters (trusted and > untrusted) for each user session, and they are stored in a ha

[HACKERS] plpython implementation

2013-06-30 Thread Szymon Guz
I'm reading through plperl and plpython implementations and I don't understand the way they work. Comments for plperl say that there are two interpreters (trusted and untrusted) for each user session, and they are stored in a hash. Plpython version looks quite different, there is no such global h

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-05 Thread Jan Urbański
On 05/07/12 21:37, Heikki Linnakangas wrote: Committed. This bug was present in versions >= 9.0, so backpatched. Thanks! I used ereport() rather than elog() in the error message. Correct me if that was wrong, but the point was to avoid PLy_elog(), because that might cause recursion, and erepo

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-05 Thread Heikki Linnakangas
On 04.07.2012 15:11, Jan Urbański wrote: On 04/07/12 13:58, Asif Naeem wrote: I have test the patch on Win64. postgres server is working fine now for WIN1252. Thanks. create function enctest() returns text as $$ return b'tr\xc3\xb3spido'.decode('**utf-8') $$ language plpython3u; select enctes

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-04 Thread Jan Urbański
On 04/07/12 13:58, Asif Naeem wrote: Patch attached. Asif, could you try a few things on a CP1252 database? First verify if your original test case now works and then try this: I have test the patch on Win64. postgres server is working fine now for WIN1252. Thanks. create function enctes

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-04 Thread Asif Naeem
> Patch attached. Asif, could you try a few things on a CP1252 database? First verify if your original test case now works and then try this: > > I have test the patch on Win64. postgres server is working fine now for WIN1252. Thanks. > create function enctest() returns text as $$ > return b't

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 03/07/12 17:45, Jan Urbański wrote: On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx into CPx

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-07-03 Thread Jan Urbański
On 29/06/12 00:36, Jan Urbański wrote: On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. So: I'd add code to translate WINxxx into CPxxx when choosing the Python to use, chan

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-29 Thread Asif Naeem
Thank you. Please do let me know once fix check-in. I will test it and share feedback with you. Thanks. Best Regards, Asif Naeem On Fri, Jun 29, 2012 at 3:36 AM, Jan Urbański wrote: > On 27/06/12 13:57, Jan Urbański wrote: > >> On 27/06/12 11:51, Asif Naeem wrote: >> >>> Hi, >>> >>> On Windows

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-28 Thread Jan Urbański
On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a> b

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-27 Thread Jan Urbański
On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a> b: return a return b $$ LA

[HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-27 Thread Asif Naeem
Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; > CREATE OR REPLACE FUNCTION pymax (a integer, b integer) > RETURNS integer > AS $$ > if a > b: > return a > return b > $$ LANGUAGE plpython3u; > SELECT py

Re: [HACKERS] plpython crash (PG 92)

2012-05-02 Thread Peter Eisentraut
On lör, 2012-04-28 at 00:32 -0400, Tom Lane wrote: > I'm inclined to think that the best fix is for > PLy_spi_execute_fetch_result to copy the tupledesc into > TopMemoryContext, not the current context. This is a tad scary from a > memory leakage standpoint, but I suppose that if python fails to r

Re: [HACKERS] plpython crash (PG 92)

2012-04-27 Thread Tom Lane
Peter Eisentraut writes: > On tor, 2012-04-26 at 17:32 +0500, Asif Naeem wrote: >> PFA test case. It used simple select statement to retrieve data via >> plpython. It crashes latest pg 9.2 with the following stack trace i.e. >> Apparently it is being crashed because of invalid related pointer val

Re: [HACKERS] plpython crash (PG 92)

2012-04-27 Thread Tom Lane
Peter Eisentraut writes: > I must have been confused about the tuple descriptor APIs. ob->tupdesc > is created using CreateTupleDescCopy(), which copies the refcount of the > original tuple descriptor, Um, surely not. That would be nonsensical, and anyway a look at the code shows it isn't doing

Re: [HACKERS] plpython crash (PG 92)

2012-04-27 Thread Peter Eisentraut
On tor, 2012-04-26 at 17:32 +0500, Asif Naeem wrote: > PFA test case. It used simple select statement to retrieve data via > plpython. It crashes latest pg 9.2 with the following stack trace i.e. > Apparently it is being crashed because of invalid related pointer value of > pfree() *header->contex

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-26 Thread Peter Eisentraut
On mån, 2012-04-23 at 02:25 +0200, Jan Urbański wrote: > It turned out not to be as straightforward as I though :( Yeah, been there ... > > The I/O code in PL/Python is a bit of a mess and that's something that > I'd like to address somewhere in the 9.3 development cycle. Right now > making the

Re: [HACKERS] plpython crash (PG 92)

2012-04-26 Thread Asif Naeem
FYI, I have observed this crash on Linux64. Thanks. Best Regards, Muhammad Asif Naeem On Thu, Apr 26, 2012 at 5:32 PM, Asif Naeem wrote: > Hi, > > PFA test case. It used simple select statement to retrieve data via > plpython. It crashes latest pg 9.2 with the following stack trace i.e. > > #0

[HACKERS] plpython crash (PG 92)

2012-04-26 Thread Asif Naeem
Hi, PFA test case. It used simple select statement to retrieve data via plpython. It crashes latest pg 9.2 with the following stack trace i.e. #0 0x0073021f in pfree () > #1 0x7fa74b632f7a in PLy_result_dealloc () from > /home/masif/work/postgresql/postgresql/inst/lib/plpython2.so >

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-22 Thread Jan Urbański
On 10/04/12 21:47, Jan Urbański wrote: On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive dictionaries in NEW? H

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 21:27, Tom Lane wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Yes, that would be ideal, even though not backwards-compatible. Back-patching is out of the question, but do we want to change trigger functions to receive dictionaries in NEW? Hm, I was not thinking of this as being

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: > On 10/04/12 20:47, Tom Lane wrote: >> On reflection, can't we fix this as follows: if the value coming in from >> Python is a string, just feed it to record_in, the same as we used to. >> When I traced through the logic before, it seemed like it was faili

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 20:47, Tom Lane wrote: I wrote: =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: Now that I understand what's been going on, I'll try to think of a non-invasive way of fixing that... ISTM that conversion of a composite value to Python ought to produce a dict, now that the other directio

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Tom Lane
I wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> Now that I understand what's been going on, I'll try to think of a >> non-invasive way of fixing that... > ISTM that conversion of a composite value to Python ought to produce a > dict, now that the other direction expects a dict. I can see

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> On 10/04/12 04:20, Tom Lane wrote: >>> Don't know if anybody noticed bug #6559 >>> http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php > So, I know what's going on, I still don't know what's the best way to > handle it. > The function that

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-10 Thread Jan Urbański
On 10/04/12 07:35, Jan Urbański wrote: On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. So, I know what's going on, I still

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-09 Thread Jan Urbański
On 10/04/12 04:20, Tom Lane wrote: Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. I find this code pretty unreadable, though, and know nothing to speak of abou

[HACKERS] plpython triggers are broken for composite-type columns

2012-04-09 Thread Tom Lane
Don't know if anybody noticed bug #6559 http://archives.postgresql.org/pgsql-bugs/2012-03/msg00180.php I've confirmed that the given test case works in 9.0 but fails in 9.1 and HEAD. It's not terribly sensitive to the details of the SQL: any non-null value for the composite column fails, for inst

Re: [HACKERS] plpython SPI cursors

2011-12-10 Thread Peter Eisentraut
On mån, 2011-12-05 at 13:12 -0500, Bruce Momjian wrote: > Jan Urbański wrote: > > On 05/12/11 18:58, Peter Eisentraut wrote: > > > On ons, 2011-11-23 at 19:58 +0100, Jan Urba?ski wrote: > > >> On 20/11/11 19:14, Steve Singer wrote: > > >> Responding now to all questions and attaching a revised patc

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 19:12, Bruce Momjian wrote: > Jan Urbański wrote: >> On 05/12/11 18:58, Peter Eisentraut wrote: >>> On ons, 2011-11-23 at 19:58 +0100, Jan Urba?ski wrote: On 20/11/11 19:14, Steve Singer wrote: Responding now to all questions and attaching a revised patch based on your c

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Bruce Momjian
Jan Urba??ski wrote: > On 05/12/11 18:58, Peter Eisentraut wrote: > > On ons, 2011-11-23 at 19:58 +0100, Jan Urba?ski wrote: > >> On 20/11/11 19:14, Steve Singer wrote: > >> Responding now to all questions and attaching a revised patch based on > >> your comments. > > > > Committed > > > > Pleas

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Jan Urbański
On 05/12/11 18:58, Peter Eisentraut wrote: > On ons, 2011-11-23 at 19:58 +0100, Jan Urbański wrote: >> On 20/11/11 19:14, Steve Singer wrote: >> Responding now to all questions and attaching a revised patch based on >> your comments. > > Committed > > Please refresh the other patch. Great, than

Re: [HACKERS] plpython SPI cursors

2011-12-05 Thread Peter Eisentraut
On ons, 2011-11-23 at 19:58 +0100, Jan Urbański wrote: > On 20/11/11 19:14, Steve Singer wrote: > > On 11-10-15 07:28 PM, Jan Urbański wrote: > >> Hi, > >> > >> attached is a patch implementing the usage of SPI cursors in PL/Python. > >> Currently when trying to process a large table in PL/Python y

Re: [HACKERS] plpython SPI cursors

2011-11-28 Thread Peter Eisentraut
On lör, 2011-11-26 at 11:21 -0500, Steve Singer wrote: > I've looked over the revised version of the patch and it now seems > fine. > > Ready for committer. I can take it from here. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: htt

Re: [HACKERS] plpython SPI cursors

2011-11-26 Thread Steve Singer
On 11-11-23 01:58 PM, Jan Urbański wrote: On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (

Re: [HACKERS] plpython SPI cursors

2011-11-23 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). J I foun

Re: [HACKERS] plpython SPI cursors

2011-11-20 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. I found a few bugs (see my testing section below) that will need fixing + a few questions about the code Hi Steve, thanks a lot for

Re: [HACKERS] plpython SPI cursors

2011-11-20 Thread Steve Singer
On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). J I found a few bugs (see my testing section be

Re: [HACKERS] plpython extension control files installation

2011-11-08 Thread Robert Haas
On Sat, Nov 5, 2011 at 10:42 AM, Peter Eisentraut wrote: > We only build the language plpython2u or plpython3u, not both, in any > build, but we always install the extension control files for all > variants.  Is there a reason for this, or just an oversight? Sounds like an oversight. -- Robert

[HACKERS] plpython extension control files installation

2011-11-05 Thread Peter Eisentraut
We only build the language plpython2u or plpython3u, not both, in any build, but we always install the extension control files for all variants. Is there a reason for this, or just an oversight? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subsc

[HACKERS] plpython SPI cursors

2011-10-15 Thread Jan Urbański
Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). This patch allows reading the result set in smaller chunks, using a SPI cursor behind the scen

Re: [HACKERS] plpython crash

2011-08-18 Thread Jan Urbański
On 17/08/11 23:10, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> Here are two patches that fix two separate bugs that you found >> simultaneously. Because they're actually separate issues, it turned out >> fixing them was a bit more tricky than I expected (fixing one was >> unmaski

Re: [HACKERS] plpython crash

2011-08-17 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: > Here are two patches that fix two separate bugs that you found > simultaneously. Because they're actually separate issues, it turned out > fixing them was a bit more tricky than I expected (fixing one was > unmasking the other one etc). Applied with one

Re: [HACKERS] plpython crash

2011-08-17 Thread Tom Lane
=?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >> On 16/08/11 19:07, Jean-Baptiste Quenot wrote: >>> [plpython is buggy] > Here are two patches that fix two separate bugs that you found > simultaneously. Because they're actually separate issues, it turned out > fixing them was a bit more tricky than I

Re: [HACKERS] plpython crash

2011-08-17 Thread Jan Urbański
On 17/08/11 11:40, Jan Urbański wrote: > On 16/08/11 19:12, Jan Urbański wrote: >> On 16/08/11 19:07, Jean-Baptiste Quenot wrote: >>> >>> [plpython is buggy] > I'll have a patch ready soon. Here are two patches that fix two separate bugs that you found simultaneously. Because they're actually sep

Re: [HACKERS] plpython crash

2011-08-17 Thread Jan Urbański
On 16/08/11 19:12, Jan Urbański wrote: > On 16/08/11 19:07, Jean-Baptiste Quenot wrote: >> Dear Jan, >> >> Sorry I typed the wrong git commands. With latest plpython from >> branch master I got the same gdb backtrace as reported before. I >> managed to wrap up a testcase that fails 100% of times

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 19:07, Jean-Baptiste Quenot wrote: > Dear Jan, > > Sorry I typed the wrong git commands. With latest plpython from > branch master I got the same gdb backtrace as reported before. I > managed to wrap up a testcase that fails 100% of times on my setup: > https://gist.github.com/114951

Re: [HACKERS] plpython crash

2011-08-16 Thread Jean-Baptiste Quenot
Dear Jan, Sorry I typed the wrong git commands. With latest plpython from branch master I got the same gdb backtrace as reported before. I managed to wrap up a testcase that fails 100% of times on my setup: https://gist.github.com/1149512 Hope it crashes on your side too :-) This is the result

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 17:06, Jan Urbański wrote: > On 16/08/11 16:52, Jean-Baptiste Quenot wrote: >> After backporting plpython.c from HEAD, this is the error message I get: >> >> ERROR: key "pg.dropped.6" not found in mapping >> HINT: To return null in a column, add the value None to the m

Re: [HACKERS] plpython crash

2011-08-16 Thread Jan Urbański
On 16/08/11 16:52, Jean-Baptiste Quenot wrote: > After backporting plpython.c from HEAD, this is the error message I get: > > ERROR: key "pg.dropped.6" not found in mapping > HINT: To return null in a column, add the value None to the mapping > with the key named after the column

Re: [HACKERS] plpython crash

2011-08-16 Thread Jean-Baptiste Quenot
After backporting plpython.c from HEAD, this is the error message I get: ERROR: key "pg.dropped.6" not found in mapping HINT: To return null in a column, add the value None to the mapping with the key named after the column. CONTEXT: while creating return value PL/Python functio

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 12/08/11 13:55, Jean-Baptiste Quenot wrote: > Here is the same with -O0: > > https://gist.github.com/1140005 > > sys.version reports this: > > INFO: 2.6.6 (r266:84292, Sep 15 2010, 16:41:53) > [GCC 4.4.5] I'm still at a loss. Did you reproduce it with git HEAD? I see that the query being ex

Re: [HACKERS] plpython crash

2011-08-12 Thread Jean-Baptiste Quenot
Here is the same with -O0: https://gist.github.com/1140005 sys.version reports this: INFO: 2.6.6 (r266:84292, Sep 15 2010, 16:41:53) [GCC 4.4.5] -- Jean-Baptiste Quenot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.post

Re: [HACKERS] plpython crash

2011-08-12 Thread Jan Urbański
On 11/08/11 18:01, Jean-Baptiste Quenot wrote: > Hi there, > > plpython crashes on me on various 64-bit Ubuntu hosts, see the gdb > backtrace at: https://gist.github.com/1140005 > > Do you believe there was recent bugfixes regarding PLyMapping_ToTuple() ? > > This is PG 9.0.4 with HEAD of plpyth

[HACKERS] plpython crash

2011-08-11 Thread Jean-Baptiste Quenot
Hi there, plpython crashes on me on various 64-bit Ubuntu hosts, see the gdb backtrace at: https://gist.github.com/1140005 Do you believe there was recent bugfixes regarding PLyMapping_ToTuple() ? This is PG 9.0.4 with HEAD of plpython taken in march 2011 and backported. Please tell me if you n

Re: [HACKERS] plpython thinks it's hooked into "make distprep", but not so much

2011-07-02 Thread Tom Lane
I wrote: > I noticed while wrapping 9.1beta2 that plpython tries to build a file > spiexceptions.h to be included in the tarballs, but no such file is > actually appearing therein. The reason is that src/pl/Makefile doesn't > recurse into the plpython subdirectory unless with_python is set. > Whic

[HACKERS] plpython thinks it's hooked into "make distprep", but not so much

2011-06-09 Thread Tom Lane
I noticed while wrapping 9.1beta2 that plpython tries to build a file spiexceptions.h to be included in the tarballs, but no such file is actually appearing therein. The reason is that src/pl/Makefile doesn't recurse into the plpython subdirectory unless with_python is set. Which it isn't, because

Re: [HACKERS] plpython versus gcc 4.5.x

2011-01-22 Thread Jan Urbański
On 22/01/11 19:38, Tom Lane wrote: > Buildfarm members bobcat (Fedora 14 x86) and frogmouth (mingw) have > been showing an assertion failure in the plpython regression tests > since this patch went in: > http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=740e54c > Given these facts,

Re: [HACKERS] plpython versus gcc 4.5.x

2011-01-22 Thread Andrew Dunstan
On 01/22/2011 01:38 PM, Tom Lane wrote: Buildfarm members bobcat (Fedora 14 x86) and frogmouth (mingw) have been showing an assertion failure in the plpython regression tests since this patch went in: http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=740e54c I have been able to

[HACKERS] plpython versus gcc 4.5.x

2011-01-22 Thread Tom Lane
Buildfarm members bobcat (Fedora 14 x86) and frogmouth (mingw) have been showing an assertion failure in the plpython regression tests since this patch went in: http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=740e54c I have been able to reproduce the problem locally on Fedora 14 x

Re: [HACKERS] PlPython bug in 9.0/8.4.4

2010-06-08 Thread Tom Lane
Teodor Sigaev writes: > The way to reproduce: Well, of course plpython's error handling is fundamentally brain dead. The immediate problem here seems to be that the PLy_error_in_progress struct is stored in a memory context that's been cleared by the time that control gets to the place in PLy_pr

[HACKERS] PlPython bug in 9.0/8.4.4

2010-06-08 Thread Teodor Sigaev
The way to reproduce: CREATE OR REPLACE FUNCTION foobar(a integer[]) RETURNS SETOF int8 AS $$ def getplan(name, query, args): if SD.has_key(name): plpy.warning("Using cached plan %s" % name) return SD[name] plpy.warning("Prepare plan %s" % name);

Re: [HACKERS] plpython sets

2006-07-18 Thread Hannu Krosing
Ühel kenal päeval, E, 2006-07-17 kell 23:22, kirjutas Matteo Bertini: > Tino Wildenhain ha scritto: > > http://python.projects.postgresql.org/ > > This works very well for me - although it needs some more > > finish (docs and so on) maybe if more people using it > > it can get better. > > > > SRF -

Re: [HACKERS] plpython sets

2006-07-18 Thread Matteo Bertini
Tino Wildenhain ha scritto: > http://python.projects.postgresql.org/ > This works very well for me - although it needs some more > finish (docs and so on) maybe if more people using it > it can get better. > > SRF - even lazy ones (e.g. generators) work nicely there. > > > Regards > Tino Wildenhain

Re: [HACKERS] plpython sets

2006-07-17 Thread Hannu Krosing
Ühel kenal päeval, E, 2006-07-17 kell 22:54, kirjutas Tino Wildenhain: > Matteo Bertini wrote: > > Hello all, > > I'm working with pl/python and I'd like to use the set returning > > function feature. > > > > I'm not working in a debug python, so the iterator bug is not a problem me. > > > > Can

Re: [HACKERS] plpython sets

2006-07-17 Thread Tino Wildenhain
Matteo Bertini wrote: > Hello all, > I'm working with pl/python and I'd like to use the set returning > function feature. > > I'm not working in a debug python, so the iterator bug is not a problem me. > > Can someone point me to some plpython.c "setof" enabled sources? > > Hint to build them in

[HACKERS] plpython sets

2006-07-17 Thread Matteo Bertini
Hello all, I'm working with pl/python and I'd like to use the set returning function feature. I'm not working in a debug python, so the iterator bug is not a problem me. Can someone point me to some plpython.c "setof" enabled sources? Hint to build them in an ubuntu dapper environment are welcom

[HACKERS] plpython improvements on patches

2006-04-17 Thread Hannu Krosing
Hi Anyone interested in plpython, please check out/review http://archives.postgresql.org/pgsql-patches/2006-04/msg00087.php It fixes the most obvious shortcomings of plpython by making it possible to return both sets and records (and setof records) and to use named input parameters in addition to

Re: [HACKERS] plpython and bytea

2005-11-26 Thread Hannu Krosing
On Mon, 2005-11-21 at 02:11 +0200, Hannu Krosing wrote: > Hi > > It seems that plpython is unable to return bytea string when it contains > NUL bytes: > > hannu=# CREATE OR REPLACE FUNCTION get_bytea_with_nul() RETURNS bytea AS > ' > return ''aa\\0bb'' > ' LANGUAGE plpythonu SECURITY DEFINER; >

Re: [HACKERS] plpython and bytea

2005-11-21 Thread Tino Wildenhain
Am Montag, den 21.11.2005, 09:08 -0700 schrieb James William Pye: > On Mon, 2005-11-21 at 15:18 +0200, Hannu Krosing wrote: > > The project seems quite interesting, will surely take a deeper look > > ... > > The 'layout' package needs to be installed first. > > See this quick start section: > ht

Re: [HACKERS] plpython and bytea

2005-11-21 Thread James William Pye
On Mon, 2005-11-21 at 15:18 +0200, Hannu Krosing wrote: > The project seems quite interesting, will surely take a deeper look > > It looks quite alpha, seems to mess to deeply with backend, and the cvs > checkout of module be does not build, so I will probably not be able to > use it in production

Re: [HACKERS] plpython and bytea

2005-11-21 Thread Tino Wildenhain
Hannu Krosing schrieb: On Mon, 2005-11-21 at 08:37 +0100, Tino Wildenhain wrote: Am Montag, den 21.11.2005, 02:11 +0200 schrieb Hannu Krosing: Hi It seems that plpython is unable to return bytea string when it contains NUL bytes: ... Did you also try: http://python.projects.postgresql.

Re: [HACKERS] plpython and bytea

2005-11-21 Thread Hannu Krosing
On Mon, 2005-11-21 at 08:37 +0100, Tino Wildenhain wrote: > Am Montag, den 21.11.2005, 02:11 +0200 schrieb Hannu Krosing: > > Hi > > > > It seems that plpython is unable to return bytea string when it contains > > NUL bytes: > > ... > Did you also try: > > http://python.projects.postgresql.org/p

Re: [HACKERS] plpython and bytea

2005-11-20 Thread Tino Wildenhain
Am Montag, den 21.11.2005, 02:11 +0200 schrieb Hannu Krosing: > Hi > > It seems that plpython is unable to return bytea string when it contains > NUL bytes: > > hannu=# CREATE OR REPLACE FUNCTION get_bytea_with_nul() RETURNS bytea AS > ' > return ''aa\\0bb'' > ' LANGUAGE plpythonu SECURITY DEFINE

[HACKERS] plpython and bytea

2005-11-20 Thread Hannu Krosing
Hi It seems that plpython is unable to return bytea string when it contains NUL bytes: hannu=# CREATE OR REPLACE FUNCTION get_bytea_with_nul() RETURNS bytea AS ' return ''aa\\0bb'' ' LANGUAGE plpythonu SECURITY DEFINER; hannu=# select get_bytea_with_nul(); get_bytea_with_nul ---

  1   2   >