Re: [PATCHES] plpython improvements
Sven Suursoho wrote: Hi, Quoting Bruce Momjian [EMAIL PROTECTED]: Great. Please supply documentation and it will be applied. Thanks. Here it comes, including src+doc patches. Updated sources according to Michael Fuhr's comments and fixed one FIXME. Please check documentation patch thoroughly as I'm not native English speaker nor didn't manage to generate documentation from SGML sources ( openjade:postgres.sgml:3:55:W: cannot generate system identifier for public text -//OASIS//DTD DocBook V4.2//EN) Patch applied. Thanks. I improved your documentation wording, updated version attached. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/plpython.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/plpython.sgml,v retrieving revision 1.30 diff -c -c -r1.30 plpython.sgml *** doc/src/sgml/plpython.sgml 26 May 2006 19:23:09 - 1.30 --- doc/src/sgml/plpython.sgml 2 Sep 2006 12:12:40 - *** *** 46,67 titlePL/Python Functions/title para !Functions in PL/Python are declared via the usual xref linkend=sql-createfunction endterm=sql-createfunction-title !syntax. For example: programlisting ! CREATE FUNCTION myfunc(text) RETURNS text ! AS 'return args[0]' ! LANGUAGE plpythonu; /programlisting The Python code that is given as the body of the function definition !gets transformed into a Python function. !For example, the above results in programlisting ! def __plpython_procedure_myfunc_23456(): ! return args[0] /programlisting assuming that 23456 is the OID assigned to the function by --- 46,95 titlePL/Python Functions/title para !Functions in PL/Python are declared via the standard xref linkend=sql-createfunction endterm=sql-createfunction-title !syntax: ! ! programlisting ! CREATE FUNCTION replaceablefuncname/replaceable (replaceableargument-list/replaceable) ! RETURNS replaceablereturn-type/replaceable ! AS $$ ! # PL/Python function body ! $$ LANGUAGE plpythonu; ! /programlisting ! /para ! ! para !The body of a function is simply a Python script. When the function !is called, all unnamed arguments are passed as elements to the array !varnameargs[]/varname and named arguments as ordinary variables to the !Python script. The result is returned from the Python code in the usual way, !with literalreturn/literal or literalyield/literal (in case of !a resultset statement). ! /para ! ! para !For example, a function to return the greater of two integers can be !defined as: ! programlisting ! CREATE FUNCTION pymax (a integer, b integer) ! RETURNS integer ! AS $$ ! if a gt; b: ! return a ! return b ! $$ LANGUAGE plpythonu; /programlisting The Python code that is given as the body of the function definition !is transformed into a Python function. For example, the above results in programlisting ! def __plpython_procedure_pymax_23456(): ! if a gt; b: ! return a ! return b /programlisting assuming that 23456 is the OID assigned to the function by *** *** 69,74 --- 97,257 /para para +The productnamePostgreSQL/ function parameters are available in +the global varnameargs/varname list. In the +functionpymax/function example, varnameargs[0]/varname contains +whatever was passed in as the first argument and +varnameargs[1]/varname contains the second argument's value. Alternatively, +one can use named parameters as shown in the example above. This greatly simplifies +the reading and writing of applicationPL/Python/application code. + /para + + para +If an SQL null valueindextermprimarynull value/primarysecondary +sortas=PL/PythonPL/Python/secondary/indexterm is passed to a +function, the argument value will appear as symbolNone/symbol in +Python. The above function definition will return the wrong answer for null +inputs. We could add literalSTRICT/literal to the function definition +to make productnamePostgreSQL/productname do something more reasonable: +if a null value is passed, the function will not be called at all, +but will just return a null result automatically. Alternatively, +we could check for null inputs in the function body: + + programlisting + CREATE FUNCTION pymax (a integer, b integer) + RETURNS integer + AS $$ + if (a is None) or (b is None): + return None + if a b: + return a + return b + $$ LANGUAGE plpythonu; + /programlisting + +As shown above, to return an SQL null value from a PL/Python +function, return the value symbolNone/symbol. This can be done whether the +function is strict or not. + /para + + para +Composite-type arguments are passed to the
Re: [PATCHES] plpython improvements
Hi, Quoting Bruce Momjian [EMAIL PROTECTED]: Great. Please supply documentation and it will be applied. Thanks. Here it comes, including src+doc patches. Updated sources according to Michael Fuhr's comments and fixed one FIXME. Please check documentation patch thoroughly as I'm not native English speaker nor didn't manage to generate documentation from SGML sources ( openjade:postgres.sgml:3:55:W: cannot generate system identifier for public text -//OASIS//DTD DocBook V4.2//EN) -- Sven Suursoho plpython-patch.tar.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] plpython improvements
Sven Suursoho wrote: Quoting Sven Suursoho [EMAIL PROTECTED]: On Sat, 17 Jun 2006 18:47:46 +0300, Tom Lane [EMAIL PROTECTED] wrote: Obviously we should document the problem in the plpython documentation (don't try to use this feature with python versions before XXX), but I'm not any longer convinced that we have to reject this patch on security grounds. Here it is, updated against CVS HEAD. Will create documentation after acceptance of patch. And here it comes again :) Fedora has fixed bug that caused initial rejection (https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=192592). Great. Please supply documentation and it will be applied. Thanks. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] plpython improvements
Quoting Bruce Momjian pgman@candle.pha.pa.us: Added to TODO: o Allow PL/python to composite types and result sets once buggy assert-enabled versions of python can be detected I actually tried to work other way around -- get all known buggy systems fixed: 1) Debugging Python version: bug acknowledged, v2.5 fixed, v2.4 waiting (http://sourceforge.net/tracker/index.php?func=detailaid=1483133group_id=5470atid=105470) 2) Fedora: =v5 fixed, will be backported to version 4 also (https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=192592) If this approach is not satisfactionary and assertion predetection is still requested, only solution I see is to catch signal SIGABRT, set flag, longjmp back and report error. But I really don't like it. -- Sven Suursoho ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] plpython improvements
Sven [EMAIL PROTECTED] writes: If this approach is not satisfactionary and assertion predetection is still requested, only solution I see is to catch signal SIGABRT, set flag, longjmp back and report error. But I really don't like it. No, ignoring SIGABRT is surely right out. It suddenly strikes me though that we are being too picky. The reason (which I admit having forgotten) is that plpython only comes in an untrusted variant. That means that anyone writing plpython functions is already a database superuser, and has much more direct means of crashing the backend if he chooses to. So the idea that this python bug constitutes a security threat seems overblown. If there ever is a future Python release with a new sandbox mechanism, enabling resurrection of the trusted language, it would presumably contain the bug fix. Obviously we should document the problem in the plpython documentation (don't try to use this feature with python versions before XXX), but I'm not any longer convinced that we have to reject this patch on security grounds. [ Whether the patch is any good is a separate question ;-). I have not reviewed it. ] regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython improvements
Added to TODO: o Allow PL/python to composite types and result sets once buggy assert-enabled versions of python can be detected http://archives.postgresql.org/pgsql-patches/2006-04/msg00087$ --- Hannu Krosing wrote: ?hel kenal p?eval, N, 2006-05-04 kell 18:21, kirjutas Sven Suursoho: Hi, Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. Actually not bleeding-edge, but just version 2.4.x as distributed in Fedora Core (and possibly in RHAS), which have assert() enabled in python.so. The assert there is buggy (bug http://sourceforge.net/tracker/index.php?func=detailaid=1257960group_id=5470atid=105470) Did complete rewrite for SETOF functions: now accepts any python object for which iter(object) returns iterable object. In this way we don't have to deal with specific containers but can use unified python iterator API. It means that plpython is future-proof -- whenever python introduces new container, stored procedures already can use those without recompiling language handler. Also integrated with regression tests and updated existing tests to use named parameters. When using python interpreter with asserts enabled, generators still crash. But I don't think that we should drop this feature because of that. Reasons: 1) this is someone else's bug, we are using documented API correctly 2) it doesn't concern majority of users because probably there is no asserts in production packages (tested with gentoo, ubuntu, suse). This is true even for older python versions that are not patched. From reading the bug, it seems that older versions of python also don't have this bug, only 2.4. And after all, we can document using sets, lists, tuples, iterators etc and explicitly state that returning generator is undefined. I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. If you need to use generators on such a platform, compile your own python from source and make sure that configure uses your version. I think the patch should be commited so we can collect data about where else the buggy version of python exists. And if some buildfarm machines start crashing, python should be fixed there. Hannu -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] plpython improvements
Sat, 06 May 2006 20:38:48 +0300, Bruce Momjian pgman@candle.pha.pa.us: I still do not know why we can't do some kind of runtime test in python and disable this feature for 2.4 builds that have debugging enabled. Can we do a dynamic function load test from plpython? There must be some function that is only visible in debug builds. Yes, I already did research last week after discussions about that. In unmodified Python distribution, in configure: if --with-pydebug define Py_DEBUG undef NDEBUG else undef Py_DEBUG define NDEBUG fi Unfortunately, this is not case for Fedora Core 4, where assertions are used unconditionally. And to make things worse, there is no runtime symbol at all to indicate whether Python is compiled with debugging/assertions enabled (Py_DEBUG NDEBUG are preprocessor symbols) I see 3 options (from best to worst, IMHO) 1) document misbehaviour when using Python2.4 with assertions allowed with reference to bug fix. 2) new configure flag to optionally allow/disallow using generator 3) drop generators Btw, we developed returning compose types further. Basically did same as for SETOF functions -- allow to return any Python object that conforms to mapping protocol. Currently only dict as previously, though. But still, ready for new fancy unforeseen Python features without recompiling PG. Additionally, now it is possible to return compose types as Python tuples. Will create patch whenever final decision is made about generators. -- Sven Suursoho ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
Sun, 07 May 2006 12:36:59 +0300, Sven Suursoho [EMAIL PROTECTED]: Btw, we developed returning compose types further. Basically did same as for SETOF functions -- allow to return any Python object that conforms to mapping protocol. Currently only dict as previously, though. But still, ready for new fancy unforeseen Python features without recompiling PG. Additionally, now it is possible to return compose types as Python tuples. Sorry, not just Python tuples but actually any object conforming to sequence protocol (tuple, list etc) -- Sven Suursoho ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
Sven Suursoho wrote: Sat, 06 May 2006 20:38:48 +0300, Bruce Momjian pgman@candle.pha.pa.us: I still do not know why we can't do some kind of runtime test in python and disable this feature for 2.4 builds that have debugging enabled. Can we do a dynamic function load test from plpython? There must be some function that is only visible in debug builds. Yes, I already did research last week after discussions about that. In unmodified Python distribution, in configure: if --with-pydebug define Py_DEBUG undef NDEBUG else undef Py_DEBUG define NDEBUG fi Unfortunately, this is not case for Fedora Core 4, where assertions are used unconditionally. And to make things worse, there is no runtime symbol at all to indicate whether Python is compiled with debugging/assertions enabled (Py_DEBUG NDEBUG are preprocessor symbols) Can you test dynamically loading a function that is only visible in the symbol table of debug builds, and check the return code? In the Fedora Core 4 case, how did they make assertions always enabled? I see the fix: http://mail.python.org/pipermail/python-checkins/2005-August/046571.html How did Fedora Core 4 enable asserts? I see the patch calling libc's assert() directly, which should be enabled all the time. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] plpython improvements
Sun, 07 May 2006 15:58:49 +0300, Bruce Momjian pgman@candle.pha.pa.us: Unfortunately, this is not case for Fedora Core 4, where assertions are used unconditionally. And to make things worse, there is no runtime symbol at all to indicate whether Python is compiled with debugging/assertions enabled (Py_DEBUG NDEBUG are preprocessor symbols) Can you test dynamically loading a function that is only visible in the symbol table of debug builds, and check the return code? As it came out, that wouldn't help us because FC4's Python has debugging disabled but assertions are still active. See below. In the Fedora Core 4 case, how did they make assertions always enabled? Building FC4's Python rpm contains bug: When creating package rpm from spec-file, even though Python is built without --with-pydebug, rpmbuild loses -DNDEBUG from CFLAGS. Can't tell how or when it happens, I'm not so good with rpm packaging system, just looked output of compilation stage. So, we are back on square one... -- Sven Suursoho ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
Ühel kenal päeval, R, 2006-05-05 kell 09:20, kirjutas Joshua D. Drake: I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. Ubuntu ships 2.4 I don't know about SuSE. 2.4 has been out for sometime and it would be a mistake to assume that we won't run into this. Marko Kreen has tested the patch on Ubuntu and it is ok there. The problem is not using 2.4, it is using 2.4 compiled with a specific set of flags. --- Hannu ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython improvements
Sven Suursoho wrote: Fri, 05 May 2006 19:20:55 +0300, Joshua D. Drake [EMAIL PROTECTED]: I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. Ubuntu ships 2.4 I don't know about SuSE. 2.4 has been out for sometime and it would be a mistake to assume that we won't run into this. Sure, but it is known problem and there is patch for this bug. In the documentation we can clearly state that python2.4 with asserts enabled causes problem and describe how it can be tested and fixed (regardless of distribution used). As an example of absurdity of this problem: let's assume there is known distribution with buggy gethostbyname(). Fact, that we know about this, shouldn't stop us developing TCP/IP applications. Especially, if there is also patch for this bug :) It would be real shame to prevent using generator for SETOF functions because it is most natural match for plpgsql's return next For buggy distributions of gethostbyname(), we supply our own version. We don't just barrel ahead and tell them to fix it, if we can avoid it. I don't see us bundling a fixed python, however. I realize it is only Red Hat, but that is a large userbase. I still do not know why we can't do some kind of runtime test in python and disable this feature for 2.4 builds that have debugging enabled. Can we do a dynamic function load test from plpython? There must be some function that is only visible in debug builds. -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython improvements
Ühel kenal päeval, N, 2006-05-04 kell 18:21, kirjutas Sven Suursoho: Hi, Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. Actually not bleeding-edge, but just version 2.4.x as distributed in Fedora Core (and possibly in RHAS), which have assert() enabled in python.so. The assert there is buggy (bug http://sourceforge.net/tracker/index.php?func=detailaid=1257960group_id=5470atid=105470) Did complete rewrite for SETOF functions: now accepts any python object for which iter(object) returns iterable object. In this way we don't have to deal with specific containers but can use unified python iterator API. It means that plpython is future-proof -- whenever python introduces new container, stored procedures already can use those without recompiling language handler. Also integrated with regression tests and updated existing tests to use named parameters. When using python interpreter with asserts enabled, generators still crash. But I don't think that we should drop this feature because of that. Reasons: 1) this is someone else's bug, we are using documented API correctly 2) it doesn't concern majority of users because probably there is no asserts in production packages (tested with gentoo, ubuntu, suse). This is true even for older python versions that are not patched. From reading the bug, it seems that older versions of python also don't have this bug, only 2.4. And after all, we can document using sets, lists, tuples, iterators etc and explicitly state that returning generator is undefined. I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. If you need to use generators on such a platform, compile your own python from source and make sure that configure uses your version. I think the patch should be commited so we can collect data about where else the buggy version of python exists. And if some buildfarm machines start crashing, python should be fixed there. Hannu ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. Ubuntu ships 2.4 I don't know about SuSE. 2.4 has been out for sometime and it would be a mistake to assume that we won't run into this. People who are developing in Python are going to run 2.4 because it is the latest stable release of the language. Sincerely, Joshua D. Drake If you need to use generators on such a platform, compile your own python from source and make sure that configure uses your version. I think the patch should be commited so we can collect data about where else the buggy version of python exists. And if some buildfarm machines start crashing, python should be fixed there. Hannu ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] plpython improvements
Fri, 05 May 2006 19:20:55 +0300, Joshua D. Drake [EMAIL PROTECTED]: I think that a less confusing way of saying it would be : Generators crash if python version used is 2.4.x and it is compiled with asserts. Currently only known linux distributions to distibute such python.so files are Fedora and possibly other RedHat distributions, while Gentoo, Ubuntu and Suse are OK. Ubuntu ships 2.4 I don't know about SuSE. 2.4 has been out for sometime and it would be a mistake to assume that we won't run into this. Sure, but it is known problem and there is patch for this bug. In the documentation we can clearly state that python2.4 with asserts enabled causes problem and describe how it can be tested and fixed (regardless of distribution used). As an example of absurdity of this problem: let's assume there is known distribution with buggy gethostbyname(). Fact, that we know about this, shouldn't stop us developing TCP/IP applications. Especially, if there is also patch for this bug :) It would be real shame to prevent using generator for SETOF functions because it is most natural match for plpgsql's return next -- Sven Suursoho ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
As an example of absurdity of this problem: let's assume there is known distribution with buggy gethostbyname(). Fact, that we know about this, shouldn't stop us developing TCP/IP applications. Especially, if there is also patch for this bug :) It would be real shame to prevent using generator for SETOF functions because it is most natural match for plpgsql's return next All I was saying was that we need to account for the use of 2.4 :). So as long as we document (as you suggest) we should be good. Sorry if I wasn't clear. Sincerely, Joshua D. Drake --Sven Suursoho -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython improvements
Hi,
Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]:
Sven Suursoho [EMAIL PROTECTED] writes:
Unfortunately, there is still one problem when using unpatched python,
caused by too aggressive assert.
http://mail.python.org/pipermail/python-checkins/2005-August/046571.html.
I don't think we are going to be able to accept a patch that causes the
server to crash when using any but a bleeding-edge copy of Python.
Did complete rewrite for SETOF functions: now accepts any python object
for which iter(object) returns iterable object. In this way we don't have
to deal with specific containers but can use unified python iterator API.
It means that plpython is future-proof -- whenever python introduces new
container, stored procedures already can use those without recompiling
language handler.
Also integrated with regression tests and updated existing tests to use
named parameters.
When using python interpreter with asserts enabled, generators still
crash. But I don't think that we should drop this feature because of that.
Reasons:
1) this is someone else's bug, we are using documented API correctly
2) it doesn't concern majority of users because probably there is no
asserts in production packages (tested with gentoo, ubuntu, suse). This is
true even for older python versions that are not patched.
And after all, we can document using sets, lists, tuples, iterators etc
and explicitly state that returning generator is undefined.
--
Sven SuursohoIndex: plpython.c
===
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.79
diff -u -r1.79 plpython.c
--- plpython.c 27 Apr 2006 14:18:07 - 1.79
+++ plpython.c 4 May 2006 14:05:25 -
@@ -19,6 +19,7 @@
#include catalog/pg_type.h
#include commands/trigger.h
#include executor/spi.h
+#include funcapi.h
#include fmgr.h
#include nodes/makefuncs.h
#include parser/parse_type.h
@@ -108,6 +109,9 @@
bool fn_readonly;
PLyTypeInfo result; /* also used to store info for trigger tuple
* type */
+ bool is_setof; /* true, if procedure returns result set */
+ PyObject*setof; /* contents of result set. */
+ char **argnames; /* Argument names */
PLyTypeInfo args[FUNC_MAX_ARGS];
int nargs;
PyObject *code; /* compiled procedure code */
@@ -184,6 +188,7 @@
static HeapTuple PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *);
static PyObject *PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *);
+static void PLy_function_delete_args(PLyProcedure *);
static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *,
HeapTuple *);
static HeapTuple PLy_modify_tuple(PLyProcedure *, PyObject *,
@@ -218,6 +223,7 @@
static PyObject *PLyInt_FromString(const char *);
static PyObject *PLyLong_FromString(const char *);
static PyObject *PLyString_FromString(const char *);
+static HeapTuple PLyDict_ToTuple(PLyTypeInfo *, PyObject *);
/* global data */
@@ -726,11 +732,17 @@
PG_TRY();
{
- plargs = PLy_function_build_args(fcinfo, proc);
- plrv = PLy_procedure_call(proc, args, plargs);
-
- Assert(plrv != NULL);
- Assert(!PLy_error_in_progress);
+ if (!proc-is_setof || proc-setof == NULL)
+ {
+ /* python function not called yet, do it */
+ plargs = PLy_function_build_args(fcinfo, proc);
+ plrv = PLy_procedure_call(proc, args, plargs);
+ if (!proc-is_setof)
+/* SETOF function parameters are deleted when called last row is returned */
+PLy_function_delete_args(proc);
+ Assert(plrv != NULL);
+ Assert(!PLy_error_in_progress);
+ }
/*
* Disconnect from SPI manager and then create the return values datum
@@ -741,6 +753,67 @@
if (SPI_finish() != SPI_OK_FINISH)
elog(ERROR, SPI_finish failed);
+ if (proc-is_setof)
+ {
+ bool has_error = false;
+ ReturnSetInfo *rsi = (ReturnSetInfo *)fcinfo-resultinfo;
+
+ if (proc-setof == NULL)
+ {
+/* first time -- do checks and setup */
+if (!rsi || !IsA(rsi, ReturnSetInfo) ||
+ (rsi-allowedModes SFRM_ValuePerCall) == 0)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg(only value per call is allowed)));
+}
+rsi-returnMode = SFRM_ValuePerCall;
+
+/* Make iterator out of returned object */
+proc-setof = PyObject_GetIter(plrv);
+Py_DECREF(plrv);
+plrv = NULL;
+
+if (proc-setof == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg(returned object can not be iterated),
+ errdetail(SETOF must be returned as iterable object)));
+ }
+
+ /* Fetch next from iterator */
+ plrv = PyIter_Next(proc-setof);
+ if (plrv)
+rsi-isDone = ExprMultipleResult;
+ else
+ {
+rsi-isDone = ExprEndResult;
+has_error = PyErr_Occurred() != NULL;
+ }
+
+ if (rsi-isDone == ExprEndResult)
+ {
+/* Iterator is exhausted or error happened */
+
Re: [PATCHES] plpython improvements
Ühel kenal päeval, P, 2006-04-30 kell 14:43, kirjutas Tom Lane: Sven Suursoho [EMAIL PROTECTED] writes: So, what about this in configure: if --with-python test_iterator_app_crashes # errcode(FEATURE_NOT_SUPPORTED), errmsg(patch your python) disable_iterator_feature fi Testing it in configure is wrong, because there's no guarantee the same python library will be used at runtime. As it is a crash bug, I can see two other ways to test: 1) do the test in startup script (maybe have pg_ctl run something) 2) test it in postmaster by running an external testprogram and see if it crashes. How do we handle other buggy library routines, like if some system library crashes on divide-by-zero or similar ? Hannu ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] plpython improvements
Hi,
Thu, 27 Apr 2006 17:17:36 +0300, Bruce Momjian pgman@candle.pha.pa.us:
Sorry, I have to revert this patch because it is causing crashes in the
plpython regression tests. Would you please run those tests, fix the
bug, and resubmit. Thanks.
Found and fixed two problems:
1) named parameters handling if there were no parameters at all
2) didn't increment counter for borrowed reference fetched from list
Also integrated tests to plpython test suite and updated existing tests to
use named parameters.
Unfortunately, there is still one problem when using unpatched python,
caused by too aggressive assert.
See
http://mail.python.org/pipermail/python-checkins/2005-August/046571.html.
I guess there should be warning somewhere as Hannu said but didn't know
where to put it.
--
Sven SuursohoIndex: plpython.c
===
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.79
diff -u -r1.79 plpython.c
--- plpython.c 27 Apr 2006 14:18:07 - 1.79
+++ plpython.c 29 Apr 2006 10:52:22 -
@@ -19,6 +19,7 @@
#include catalog/pg_type.h
#include commands/trigger.h
#include executor/spi.h
+#include funcapi.h
#include fmgr.h
#include nodes/makefuncs.h
#include parser/parse_type.h
@@ -108,6 +109,11 @@
bool fn_readonly;
PLyTypeInfo result; /* also used to store info for trigger tuple
* type */
+ bool is_setof; /* true, if procedure returns result set */
+ PyObject*setof; /* contents of result set. */
+ int setof_count; /* numbef of items to return in result set */
+ int setof_current; /* current item in result set */
+ char **argnames; /* Argument names */
PLyTypeInfo args[FUNC_MAX_ARGS];
int nargs;
PyObject *code; /* compiled procedure code */
@@ -184,6 +190,7 @@
static HeapTuple PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *);
static PyObject *PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *);
+static void PLy_function_delete_args(PLyProcedure *);
static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *,
HeapTuple *);
static HeapTuple PLy_modify_tuple(PLyProcedure *, PyObject *,
@@ -218,6 +225,7 @@
static PyObject *PLyInt_FromString(const char *);
static PyObject *PLyLong_FromString(const char *);
static PyObject *PLyString_FromString(const char *);
+static HeapTuple PLyDict_ToTuple(PLyTypeInfo *, PyObject *);
/* global data */
@@ -726,11 +734,17 @@
PG_TRY();
{
- plargs = PLy_function_build_args(fcinfo, proc);
- plrv = PLy_procedure_call(proc, args, plargs);
-
- Assert(plrv != NULL);
- Assert(!PLy_error_in_progress);
+ if (!proc-is_setof || proc-setof_count == -1)
+ {
+ /* python function not called yet, do it */
+ plargs = PLy_function_build_args(fcinfo, proc);
+ plrv = PLy_procedure_call(proc, args, plargs);
+ if (!proc-is_setof)
+/* SETOF function parameters are deleted when called last row is returned */
+PLy_function_delete_args(proc);
+ Assert(plrv != NULL);
+ Assert(!PLy_error_in_progress);
+ }
/*
* Disconnect from SPI manager and then create the return values datum
@@ -741,6 +755,82 @@
if (SPI_finish() != SPI_OK_FINISH)
elog(ERROR, SPI_finish failed);
+ if (proc-is_setof)
+ {
+ bool is_done = false;
+ ReturnSetInfo *rsi = (ReturnSetInfo *)fcinfo-resultinfo;
+
+ if (proc-setof_current == -1)
+ {
+/* first time -- do checks and setup */
+if (!rsi || !IsA(rsi, ReturnSetInfo) ||
+ (rsi-allowedModes SFRM_ValuePerCall) == 0)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg(only value per call is allowed)));
+}
+rsi-returnMode = SFRM_ValuePerCall;
+
+/* fetch information about returned object */
+proc-setof = plrv;
+plrv = NULL;
+if (PyList_Check(proc-setof))
+ /* SETOF as list */
+ proc-setof_count = PyList_GET_SIZE(proc-setof);
+else if (PyIter_Check(proc-setof))
+ /* SETOF as iterator, unknown number of items */
+ proc-setof_current = proc-setof_count = 0;
+else
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg(SETOF must be returned as list, iterator or generator)));
+}
+ }
+
+ Assert(proc-setof != NULL);
+
+ /* Fetch next of SETOF */
+ if (PyList_Check(proc-setof))
+ {
+is_done = ++proc-setof_current == proc-setof_count;
+if (!is_done)
+{
+ plrv = PyList_GET_ITEM(proc-setof, proc-setof_current);
+
+ /* Got 'borrowed reference', must increment reference count
+ * to balance later decrement */
+ Py_INCREF(plrv);
+}
+ }
+ else if (PyIter_Check(proc-setof))
+ {
+plrv = PyIter_Next(proc-setof);
+is_done = plrv == NULL;
+ }
+
+ if (!is_done)
+ {
+rsi-isDone = ExprMultipleResult;
+ }
+ else
+ {
+rsi-isDone = ExprEndResult;
+proc-setof_count = proc-setof_current = -1;
+Py_DECREF(proc-setof);
Re: [PATCHES] plpython improvements
Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. See http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I guess there should be warning somewhere as Hannu said but didn't know where to put it. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] plpython improvements
Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. See http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I guess there should be warning somewhere as Hannu said but didn't know where to put it. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. Actually normal python installations do not cause problem, only debugging versions do. Anyway, if you think that this doesn't count as an argument, there is nothing that we can do from PG-side except drop returning SETOF as iterator/generator and only allow return SETOF as list. -- Sven Suursoho ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] plpython improvements
Sven Suursoho wrote: Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. See http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I guess there should be warning somewhere as Hannu said but didn't know where to put it. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. Actually normal python installations do not cause problem, only debugging versions do. Anyway, if you think that this doesn't count as an argument, there is nothing that we can do from PG-side except drop returning SETOF as iterator/generator and only allow return SETOF as list. Can't we detect a debug build and disable the feature? -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] plpython improvements
Sun, 30 Apr 2006 20:48:48 +0300, Bruce Momjian pgman@candle.pha.pa.us: Sun, 30 Apr 2006 19:14:28 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: Unfortunately, there is still one problem when using unpatched python, caused by too aggressive assert. http://mail.python.org/pipermail/python-checkins/2005-August/046571.html. I guess there should be warning somewhere as Hannu said but didn't know where to put it. I don't think we are going to be able to accept a patch that causes the server to crash when using any but a bleeding-edge copy of Python. Actually normal python installations do not cause problem, only debugging versions do. Anyway, if you think that this doesn't count as an argument, there is nothing that we can do from PG-side except drop returning SETOF as iterator/generator and only allow return SETOF as list. Can't we detect a debug build and disable the feature? Yes, we can, but newer python versions are already fixed. So, what about this in configure: if --with-python test_iterator_app_crashes # errcode(FEATURE_NOT_SUPPORTED), errmsg(patch your python) disable_iterator_feature fi In this way we disable feature only if it is absolutely neccessary and will give developer enough information how to fix it. -- Sven Suursoho ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] plpython improvements
Sven Suursoho [EMAIL PROTECTED] writes: So, what about this in configure: if --with-python test_iterator_app_crashes # errcode(FEATURE_NOT_SUPPORTED), errmsg(patch your python) disable_iterator_feature fi Testing it in configure is wrong, because there's no guarantee the same python library will be used at runtime. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] plpython improvements
Sun, 30 Apr 2006 21:43:03 +0300, Tom Lane [EMAIL PROTECTED]: Sven Suursoho [EMAIL PROTECTED] writes: So, what about this in configure: if --with-python test_iterator_app_crashes # errcode(FEATURE_NOT_SUPPORTED), errmsg(patch your python) disable_iterator_feature fi Testing it in configure is wrong, because there's no guarantee the same python library will be used at runtime. Yes, that's right. Ok, will do Py_DEBUG Py_GetVersion() check at runtime. -- Sven Suursoho ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] plpython improvements
Sorry, I have to revert this patch because it is causing crashes in the
plpython regression tests. Would you please run those tests, fix the
bug, and resubmit. Thanks.
---
pgman wrote:
Patch applied. Thanks.
---
Sven Suursoho wrote:
Hi,
Mon, 17 Apr 2006 19:20:38 +0300, Bruce Momjian pgman@candle.pha.pa.us:
Hannu Krosing wrote:
1) named parameters additionally to args[]
2) return composite-types from plpython as dictionary
3) return result-set from plpython as list, iterator or generator
Test script attached (patch-test.sql) but not integrated to
plpython test-suite.
If you wonder why you can't apply the patch, it is against postgres
8.0.7.
Can I apply it by merging it in manually, or are you working on a
version against CVS HEAD?
Attached patch for CVS HEAD.
Testscripts remained same (obviously :) but still not integrated.
--
Sven Suursoho
[ Attachment, skipping... ]
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDBhttp://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDBhttp://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: plpython.c
===
RCS file: /cvsroot/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.77
retrieving revision 1.78
diff -c -c -r1.77 -r1.78
*** plpython.c 4 Apr 2006 19:35:37 - 1.77
--- plpython.c 27 Apr 2006 01:05:05 - 1.78
***
*** 19,24
--- 19,25
#include catalog/pg_type.h
#include commands/trigger.h
#include executor/spi.h
+ #include funcapi.h
#include fmgr.h
#include nodes/makefuncs.h
#include parser/parse_type.h
***
*** 108,113
--- 109,119
boolfn_readonly;
PLyTypeInfo result; /* also used to store info for
trigger tuple
* type */
+ boolis_setof; /* true, if procedure returns result
set */
+ PyObject*setof; /* contents of result set. */
+ int setof_count;/* numbef of items to return in result
set */
+ int setof_current; /* current item in result set */
+ char**argnames; /* Argument names */
PLyTypeInfo args[FUNC_MAX_ARGS];
int nargs;
PyObject *code; /* compiled procedure code */
***
*** 184,189
--- 190,196
static HeapTuple PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *);
static PyObject *PLy_function_build_args(FunctionCallInfo fcinfo,
PLyProcedure *);
+ static void PLy_function_delete_args(PLyProcedure *);
static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure
*,
HeapTuple *);
static HeapTuple PLy_modify_tuple(PLyProcedure *, PyObject *,
***
*** 218,223
--- 225,231
static PyObject *PLyInt_FromString(const char *);
static PyObject *PLyLong_FromString(const char *);
static PyObject *PLyString_FromString(const char *);
+ static HeapTuple PLyDict_ToTuple(PLyTypeInfo *, PyObject *);
/* global data */
***
*** 726,736
PG_TRY();
{
! plargs = PLy_function_build_args(fcinfo, proc);
! plrv = PLy_procedure_call(proc, args, plargs);
!
! Assert(plrv != NULL);
! Assert(!PLy_error_in_progress);
/*
* Disconnect from SPI manager and then create the return
values datum
--- 734,750
PG_TRY();
{
! if (!proc-is_setof || proc-setof_count == -1)
! {
! /* python function not called yet, do it */
! plargs = PLy_function_build_args(fcinfo, proc);
! plrv = PLy_procedure_call(proc, args, plargs);
! if (!proc-is_setof)
! /* SETOF function parameters are deleted when
called last row is returned */
! PLy_function_delete_args(proc);
! Assert(plrv != NULL);
! Assert(!PLy_error_in_progress);
! }
/*
* Disconnect from SPI manager and then create the return
values datum
***
*** 741,746
--- 755,830
Re: [PATCHES] plpython improvements
Ühel kenal päeval, N, 2006-04-27 kell 10:17, kirjutas Bruce Momjian: Sorry, I have to revert this patch because it is causing crashes in the plpython regression tests. Would you please run those tests, fix the bug, and resubmit. Thanks. Where exactly does it crash ? Please tell us the version (and buildflags) of python you are using. There is a superfluous assert in some versions of python that has troubled us as well. --- Hannu ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] plpython improvements
Hannu Krosing wrote: ?hel kenal p?eval, N, 2006-04-27 kell 10:17, kirjutas Bruce Momjian: Sorry, I have to revert this patch because it is causing crashes in the plpython regression tests. Would you please run those tests, fix the bug, and resubmit. Thanks. Where exactly does it crash ? Please tell us the version (and buildflags) of python you are using. There is a superfluous assert in some versions of python that has troubled us as well. Uh, I can't test plpython here because my libraries do not support it, but I see this in the build farm: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=rookdt=2006-04-27%2010:39:02 -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] plpython improvements
Patch applied. Thanks. --- Sven Suursoho wrote: Hi, Mon, 17 Apr 2006 19:20:38 +0300, Bruce Momjian pgman@candle.pha.pa.us: Hannu Krosing wrote: 1) named parameters additionally to args[] 2) return composite-types from plpython as dictionary 3) return result-set from plpython as list, iterator or generator Test script attached (patch-test.sql) but not integrated to plpython test-suite. If you wonder why you can't apply the patch, it is against postgres 8.0.7. Can I apply it by merging it in manually, or are you working on a version against CVS HEAD? Attached patch for CVS HEAD. Testscripts remained same (obviously :) but still not integrated. -- Sven Suursoho [ Attachment, skipping... ] ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] plpython improvements
Ühel kenal päeval, L, 2006-04-15 kell 17:59, kirjutas Sven Suursoho: 1) named parameters additionally to args[] 2) return composite-types from plpython as dictionary 3) return result-set from plpython as list, iterator or generator Test script attached (patch-test.sql) but not integrated to plpython test-suite. If you wonder why you can't apply the patch, it is against postgres 8.0.7. Hannu ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] plpython improvements
Hannu Krosing wrote: ?hel kenal p?eval, L, 2006-04-15 kell 17:59, kirjutas Sven Suursoho: 1) named parameters additionally to args[] 2) return composite-types from plpython as dictionary 3) return result-set from plpython as list, iterator or generator Test script attached (patch-test.sql) but not integrated to plpython test-suite. If you wonder why you can't apply the patch, it is against postgres 8.0.7. Can I apply it by merging it in manually, or are you working on a version against CVS HEAD? -- Bruce Momjian http://candle.pha.pa.us EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] plpython improvements
Hi,
Mon, 17 Apr 2006 19:20:38 +0300, Bruce Momjian pgman@candle.pha.pa.us:
Hannu Krosing wrote:
1) named parameters additionally to args[]
2) return composite-types from plpython as dictionary
3) return result-set from plpython as list, iterator or generator
Test script attached (patch-test.sql) but not integrated to
plpython test-suite.
If you wonder why you can't apply the patch, it is against postgres
8.0.7.
Can I apply it by merging it in manually, or are you working on a
version against CVS HEAD?
Attached patch for CVS HEAD.
Testscripts remained same (obviously :) but still not integrated.
--
Sven SuursohoIndex: plpython.c
===
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.77
diff -u -r1.77 plpython.c
--- plpython.c 4 Apr 2006 19:35:37 - 1.77
+++ plpython.c 17 Apr 2006 19:16:25 -
@@ -19,6 +19,7 @@
#include catalog/pg_type.h
#include commands/trigger.h
#include executor/spi.h
+#include funcapi.h
#include fmgr.h
#include nodes/makefuncs.h
#include parser/parse_type.h
@@ -108,6 +109,11 @@
bool fn_readonly;
PLyTypeInfo result; /* also used to store info for trigger tuple
* type */
+ bool is_setof; /* true, if procedure returns result set */
+ PyObject*setof; /* contents of result set. */
+ int setof_count; /* numbef of items to return in result set */
+ int setof_current; /* current item in result set */
+ char **argnames; /* Argument names */
PLyTypeInfo args[FUNC_MAX_ARGS];
int nargs;
PyObject *code; /* compiled procedure code */
@@ -184,6 +190,7 @@
static HeapTuple PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *);
static PyObject *PLy_function_build_args(FunctionCallInfo fcinfo, PLyProcedure *);
+static void PLy_function_delete_args(PLyProcedure *);
static PyObject *PLy_trigger_build_args(FunctionCallInfo fcinfo, PLyProcedure *,
HeapTuple *);
static HeapTuple PLy_modify_tuple(PLyProcedure *, PyObject *,
@@ -218,6 +225,7 @@
static PyObject *PLyInt_FromString(const char *);
static PyObject *PLyLong_FromString(const char *);
static PyObject *PLyString_FromString(const char *);
+static HeapTuple PLyDict_ToTuple(PLyTypeInfo *, PyObject *);
/* global data */
@@ -726,11 +734,17 @@
PG_TRY();
{
- plargs = PLy_function_build_args(fcinfo, proc);
- plrv = PLy_procedure_call(proc, args, plargs);
-
- Assert(plrv != NULL);
- Assert(!PLy_error_in_progress);
+ if (!proc-is_setof || proc-setof_count == -1)
+ {
+ /* python function not called yet, do it */
+ plargs = PLy_function_build_args(fcinfo, proc);
+ plrv = PLy_procedure_call(proc, args, plargs);
+ if (!proc-is_setof)
+/* SETOF function parameters are deleted when called last row is returned */
+PLy_function_delete_args(proc);
+ Assert(plrv != NULL);
+ Assert(!PLy_error_in_progress);
+ }
/*
* Disconnect from SPI manager and then create the return values datum
@@ -741,6 +755,76 @@
if (SPI_finish() != SPI_OK_FINISH)
elog(ERROR, SPI_finish failed);
+ if (proc-is_setof)
+ {
+ bool is_done = false;
+ ReturnSetInfo *rsi = (ReturnSetInfo *)fcinfo-resultinfo;
+
+ if (proc-setof_current == -1)
+ {
+/* first time -- do checks and setup */
+if (!rsi || !IsA(rsi, ReturnSetInfo) ||
+ (rsi-allowedModes SFRM_ValuePerCall) == 0)
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg(only value per call is allowed)));
+}
+rsi-returnMode = SFRM_ValuePerCall;
+
+/* fetch information about returned object */
+proc-setof = plrv;
+plrv = NULL;
+if (PyList_Check(proc-setof))
+ /* SETOF as list */
+ proc-setof_count = PyList_GET_SIZE(proc-setof);
+else if (PyIter_Check(proc-setof))
+ /* SETOF as iterator, unknown number of items */
+ proc-setof_current = proc-setof_count = 0;
+else
+{
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
+ errmsg(SETOF must be returned as list or iterator)));
+}
+ }
+
+ Assert(proc-setof != NULL);
+
+ /* Fetch next of SETOF */
+ if (PyList_Check(proc-setof))
+ {
+is_done = ++proc-setof_current == proc-setof_count;
+if (!is_done)
+ plrv = PyList_GET_ITEM(proc-setof, proc-setof_current);
+ }
+ else if (PyIter_Check(proc-setof))
+ {
+plrv = PyIter_Next(proc-setof);
+is_done = plrv == NULL;
+ }
+
+ if (!is_done)
+ {
+rsi-isDone = ExprMultipleResult;
+ }
+ else
+ {
+rsi-isDone = ExprEndResult;
+proc-setof_count = proc-setof_current = -1;
+Py_DECREF(proc-setof);
+proc-setof = NULL;
+
+Py_XDECREF(plargs);
+Py_XDECREF(plrv);
+Py_XDECREF(plrv_so);
+
+PLy_function_delete_args(proc);
+fcinfo-isnull = true;
+return (Datum)NULL;
+ }
+ }
+
/*
* If the function is declared to return void, the Python
* return value must be None. For
