Re: [PATCHES] Developer's FAQ update
Thanks. Your changes have been merged in. --- Gavin Sherry wrote: Diff to the HTML version attached. I'm assuming that you have a script to dump the text version, so I haven't sent a diff against the text version. Note that I have made a few more changes since the last diff. Thanks, Gavin Content-Description: [ Attachment, skipping... ] -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[PATCHES] SPI function to investigate query semantics
Here's a patch containing the function SPI_iterate_query_roots(...). I'm
optimistic so it's complete with documentation :-)
I think that this function is needed so that PL/lang authors like
myself have a way to investigate the semantics of a prepared query. For
me this is essential since I want to prevent that savepoint related
statements are executed using normal SQL so that I can enforce the use
of the methods stipulated by the connection interface.
I forsee that this might be of interest for other PL/lang authors as
well. With this patch in place, it will be possible to do things like
this (returning false is rejecting in this case since false terminates
the iteration):
static bool rejectTransactionCommand(Query* query, void* clientData)
{
return !(query-commandType == CMD_UTILITY
IsA(query-utilityStmt, TransactionStmt));
}
and then use that like:
result = !SPI_iterate_query_roots(ePlan, rejectTransactionCommand,
NULL);
The patch has no side effects since it's a pure addon.
Kind regards,
Thomas Hallgren
Index: doc/src/sgml/spi.sgml
===
RCS file: /projects/cvsroot/pgsql/doc/src/sgml/spi.sgml,v
retrieving revision 1.35
diff -u -r1.35 spi.sgml
--- doc/src/sgml/spi.sgml 13 Sep 2004 20:05:25 - 1.35
+++ doc/src/sgml/spi.sgml 1 Dec 2004 19:05:28 -
@@ -1305,6 +1305,82 @@
!-- *** --
+refentry id=spi-spi-iterate-query-roots
+ refmeta
+ refentrytitleSPI_iterate_query_roots/refentrytitle
+ /refmeta
+
+ refnamediv
+ refnameSPI_iterate_query_roots/refname
+ refpurposeinvestigate the semantics of a query/refpurpose
+ /refnamediv
+
+ indextermprimarySPI_iterate_query_roots/primary/indexterm
+
+ refsynopsisdiv
+synopsis
+bool SPI_iterate_query_roots(void * parameterplan/parameter, QueryVisitor
parametercallback/parameter, void * parameterclientData/parameter)
+/synopsis
+ /refsynopsisdiv
+
+ refsect1
+ titleDescription/title
+
+ para
+The functionSPI_iterate_query_roots/function will invoke the
+symbolqueryVisitor/symbol callback once for each top level
+symbolQuery/symbol found in the supplied execution plan.
+The iteration is cancelled when a callback returns symbolfalse/symbol.
+If no callback returns symbolfalse/symbol, or if the plan is
+symbolNULL/symbol, the function returns symboltrue/symbol.
+ /para
+ /refsect1
+
+ refsect1
+ titleArguments/title
+
+ variablelist
+ varlistentry
+termliteralvoid * parameterplan/parameter/literal/term
+listitem
+ para
+ execution plan (returned by functionSPI_prepare/function)
+ /para
+/listitem
+ /varlistentry
+
+ varlistentry
+termliteralQueryVisitor
parametercallback/parameter/literal/term
+listitem
+ para
+ the callback to invoke for each query found
+ /para
+/listitem
+ /varlistentry
+
+ varlistentry
+termliteralvoid * parameterclientData/parameter/literal/term
+listitem
+ para
+ user defined data that will be passed on to the callback
+ /para
+/listitem
+ /varlistentry
+ /variablelist
+ /refsect1
+
+ refsect1
+ titleReturn Value/title
+ para
+symboltrue/symbol when all callbacks returned symboltrue/symbol or
+symbolfalse/symbol when a callback returned symbolfalse/symbol and
+thus terminated the iteration.
+ /para
+ /refsect1
+/refentry
+
+!-- *** --
+
refentry id=spi-spi-cursor-find
refmeta
refentrytitleSPI_cursor_find/refentrytitle
Index: src/backend/executor/spi.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/executor/spi.c,v
retrieving revision 1.132
diff -u -r1.132 spi.c
--- src/backend/executor/spi.c 16 Nov 2004 18:10:13 - 1.132
+++ src/backend/executor/spi.c 1 Dec 2004 19:05:29 -
@@ -1064,6 +1064,42 @@
return false;
}
+/**
+ * Invokes the queryVisitor callback for each top level Query found in an
+ * execution plan. The iteration is cancelled when a callback returns
+ * false. If no callbacks returns false, or if the plan is NULL, this
+ * function returns true.
+ *
+ * Arguments:
+ * plan An ExecutionPlan created by SPI_prepare
+ * queryVisitor The callback function
+ * clientDataUser defined data that will be passed on to the callback
+ * Returns: true if the plan is NULL or if all callback invocation returns true
+ */
+bool
+SPI_iterate_query_roots(void *plan, QueryVisitor queryVisitor, void
*clientData)
+{
+ _SPI_plan *spiplan = (_SPI_plan *) plan;
+
+ if (spiplan != NULL)
+ {
+ ListCell *query_list_list_item;
+ List *query_list_list = spiplan-qtlist;
+ foreach(query_list_list_item, query_list_list)
+ {
+ List *query_list = lfirst(query_list_list_item);
+ ListCell
Re: [PATCHES] SPI function to investigate query semantics
Tom Lane wrote: We haven't got one that will work from inside arbitrary functions --- DefineSavepoint and friends don't get it done by themselves, but expect you to call CommitTransactionCommand/StartTransactionCommand, and those functions tend to pull the rug out from under the executor. (I seem to recall trying to do it that way in the first attempt on plpgsql, and running into all kinds of memory management issues.) The existing PLs use BeginInternalSubTransaction, ReleaseCurrentSubTransaction, RollbackAndReleaseCurrentSubTransaction, but these are subset implementations only suited for exception-block-structured code. Thanks. I'll check that out and try to figure out how to use it. What are the future plans? For me it would work really well with something like Savepoint SPI_savepoint(const char* name); void SPI_releaseSavepoint(Savepoint sp); void SPI_rollbackSavepoint(Savepoint sp); The Savepoint structure could then hold information about call level etc. needed to ensure proper behaviour when nesting. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] New compile warnings
Bruce Momjian [EMAIL PROTECTED] writes: FYI, I still see compile warnings. I assume they are just related to me using Perl 5.0.5 and will ignore them: gmake[2]: Leaving directory `/usr/var/local/src/gen/pgsql/CURRENT/pgsql/src/test/regress' gmake[1]: Leaving directory `/usr/var/local/src/gen/pgsql/CURRENT/pgsql/src/test' /usr/libdata/perl5/5.00503/i386-bsdos/CORE/patchlevel.h:41: warning: `local_patches' defined but not used ppport.h:564: warning: `sv_2pv_nolen' defined but not used /usr/libdata/perl5/5.00503/i386-bsdos/CORE/patchlevel.h:41: warning: `local_patches' defined but not used ppport.h:564: warning: `sv_2pv_nolen' defined but not used SPI.c:59: warning: no previous prototype for `XS__elog' SPI.c:78: warning: no previous prototype for `XS__DEBUG' SPI.c:93: warning: no previous prototype for `XS__LOG' SPI.c:108: warning: no previous prototype for `XS__INFO' SPI.c:123: warning: no previous prototype for `XS__NOTICE' SPI.c:138: warning: no previous prototype for `XS__WARNING' SPI.c:153: warning: no previous prototype for `XS__ERROR' SPI.c:168: warning: no previous prototype for `XS__spi_exec_query' SPI.c:197: warning: no previous prototype for `boot_SPI' /usr/libdata/perl5/5.00503/i386-bsdos/CORE/patchlevel.h:41: warning: `local_patches' defined but not used ppport.h:564: warning: `sv_2pv_nolen' defined but not used Yeah, these all look like sloppy code generation by perlxs to me. I see none of them on 5.8.* Perls. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] SPI function to investigate query semantics
Thomas Hallgren [EMAIL PROTECTED] writes: What are the future plans? I haven't got any at the moment ;-). It would make sense to think about extending the SPI API along the lines you suggest, but I really am not clear on the implications. Right at the moment I'm focused on trying to push 8.0 out the door ... regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [HACKERS] plperl Safe restrictions
Uh, what was the TODO here? I forgot. --- John Hansen wrote: I think it is *way* too late in the dev cycle to be proposing this. Maybe it should be a TODO item - I at least don't have time even to think about the implications os using these pragmas. The effect of the first is achievable via an environment setting, I believe. If you need these badly enough, use plperlu where there are no restrictions to overcome - the big problem is that 'use anything' requires that we enable the 'require' op, and that is certainly not going to happen without a great deal of thought. Fair enough, was just a suggestion as they seem obviously useful, even to the non-superuser plperl programmer. TODO item would suffice :) ... John ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] plperl Safe restrictions
Bruce Momjian said: Uh, what was the TODO here? I forgot. John wanted us to allow use of the 'locale' and 'utf8' pragmas in trusted code. If there's a TODO it would be to investigate the possibility, as I am very far from certain that there is a simple way to do it safely right now. Maybe when we get plperl using GUC settings and running some interpreter initialisation it could be done. These are things on my agenda. cheers andrew --- John Hansen wrote: I think it is *way* too late in the dev cycle to be proposing this. Maybe it should be a TODO item - I at least don't have time even to think about the implications os using these pragmas. The effect of the first is achievable via an environment setting, I believe. If you need these badly enough, use plperlu where there are no restrictions to overcome - the big problem is that 'use anything' requires that we enable the 'require' op, and that is certainly not going to happen without a great deal of thought. Fair enough, was just a suggestion as they seem obviously useful, even to the non-superuser plperl programmer. TODO item would suffice :) ... John ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[PATCHES] Update for documentation on CVS
This is a little patch to correct the documentation on CVS. The URL for downloading CVS at cyclic.com site is long defunct, and I changed the text to not overtly recommend CVS 1.10, a now fairly old version. Jon -- Jon Jensen End Point Corporation http://www.endpoint.com/ Software development with Interchange, Perl, PostgreSQL, Apache, Linux, ...Index: sgml/cvs.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/cvs.sgml,v retrieving revision 1.28 diff -u -c -r1.28 cvs.sgml *** sgml/cvs.sgml 27 Sep 2004 05:54:58 - 1.28 --- sgml/cvs.sgml 2 Dec 2004 00:46:51 - *** *** 52,61 para You will need a local copy of productnameCVS/productname (Concurrent Version Control System), which you can get from ! ulink url=http://www.cyclic.com/;http://www.cyclic.com//ulink or ! any GNU software archive site. ! We currently recommend version 1.10 (the most recent at the time ! of writing). Many systems have a recent version of applicationcvs/application installed by default. /para /step --- 52,61 para You will need a local copy of productnameCVS/productname (Concurrent Version Control System), which you can get from ! ulink url=http://www.cvshome.org/;http://www.cvshome.org//ulink ! (the official site with the latest version) or any GNU software ! archive site (often somewhat outdated). We recommend version 1.10 ! or newer. Many systems have a recent version of applicationcvs/application installed by default. /para /step *** *** 167,173 For more info consult the manual that comes with productnameCVS/productname, or see the online documentation at !ulink url=http://www.cyclic.com/;http://www.cyclic.com//ulink. /para /sect1 --- 167,173 For more info consult the manual that comes with productnameCVS/productname, or see the online documentation at !ulink url=http://www.cvshome.org/;http://www.cvshome.org//ulink. /para /sect1 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] SPI function to investigate query semantics
Thomas Hallgren [EMAIL PROTECTED] writes: I think that this function is needed so that PL/lang authors like myself have a way to investigate the semantics of a prepared query. Which you will do what with? I'm not sure I see the point of treating _SPI_plan as an opaque type while assuming you know what to do with a Query. For me this is essential since I want to prevent that savepoint related statements are executed using normal SQL so that I can enforce the use of the methods stipulated by the connection interface. You do realize that SPI_execute will reject TransactionStmt anyway? The example is therefore not very compelling ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] SPI function to investigate query semantics
Tom Lane wrote: Looks pretty rejectish to me... regards, tom lane Arrghh. Forget my patch. It's not possible to set savepoints at all using SPI! Here I was, thinking that only begin/commit/rollback was rejected (I trusted the documentation and did not dive into the code). A patch is needed for the documentation to clarify this :-) So what *is* the appropriate way of starting, releasing, and rolling back savepoints then? Regards, Thomas Hallgren ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] SPI function to investigate query semantics
Tom Lane wrote: You do realize that SPI_execute will reject TransactionStmt anyway? The example is therefore not very compelling ... It won't reject savepoint related statements and that's what the example is for. I want savepoints rejected unless they go through a specific method found on my connection object. From current discussions it seems similar functionality might be needed for other PL's. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] SPI function to investigate query semantics
Thomas Hallgren [EMAIL PROTECTED] writes:
Tom Lane wrote:
You do realize that SPI_execute will reject TransactionStmt anyway?
The example is therefore not very compelling ...
It won't reject savepoint related statements and that's what the example
is for.
Really?
if (queryTree-commandType == CMD_UTILITY)
{
...
else if (IsA(queryTree-utilityStmt, TransactionStmt))
{
res = SPI_ERROR_TRANSACTION;
goto fail;
}
}
Looks pretty rejectish to me...
regards, tom lane
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] SPI function to investigate query semantics
Tom Lane wrote: Which you will do what with? I'm not sure I see the point of treating _SPI_plan as an opaque type while assuming you know what to do with a Query. What's different in that compared to the methods that use a Snapshot? The fact that I provided documentation? If so, ok remove the docs. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] SPI function to investigate query semantics
Thomas Hallgren [EMAIL PROTECTED] writes: So what *is* the appropriate way of starting, releasing, and rolling back savepoints then? We haven't got one that will work from inside arbitrary functions --- DefineSavepoint and friends don't get it done by themselves, but expect you to call CommitTransactionCommand/StartTransactionCommand, and those functions tend to pull the rug out from under the executor. (I seem to recall trying to do it that way in the first attempt on plpgsql, and running into all kinds of memory management issues.) The existing PLs use BeginInternalSubTransaction, ReleaseCurrentSubTransaction, RollbackAndReleaseCurrentSubTransaction, but these are subset implementations only suited for exception-block-structured code. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] libpq and psql not on same page about SIGPIPE
Tom Lane wrote:
Bruce Momjian [EMAIL PROTECTED] writes:
Yes, that is the logic in my patch, except that I don't check errno, I
just call sigpending().
No, that's wrong: if there is a pending SIGPIPE that belongs to the
outer app, you'd clear it.
True, but I documented that in the patch.
There are a few writes and it seemed impossible
to check them all.
Hmm? There is only one place this needs to be done, namely
pqsecure_write.
Look also in fe-print.c. Looks like a lot of popen writes in there.
I can do it but it will be harder.
BTW, have you posted the patch yet or are you still working on it?
The mail server seems a bit flaky today ...
OK, patch attached. I already sent it but who knows what happened to
it.
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: configure
===
RCS file: /cvsroot/pgsql/configure,v
retrieving revision 1.409
diff -c -c -r1.409 configure
*** configure 30 Nov 2004 06:13:03 - 1.409
--- configure 1 Dec 2004 22:53:58 -
***
*** 17431,17436
--- 17431,17448
fi
HAVE_POSIX_SIGNALS=$pgac_cv_func_posix_signals
+ if test $pgac_cv_func_posix_signals != yes -a $enable_thread_safety =
yes; then
+ { { echo $as_me:$LINENO: error:
+ *** Thread-safety requires POSIX signals, which are not supported by your
+ *** operating system.
+ 5
+ echo $as_me: error:
+ *** Thread-safety requires POSIX signals, which are not supported by your
+ *** operating system.
+ 2;}
+{ (exit 1); exit 1; }; }
+ fi
+
if test $ac_cv_func_fseeko = yes; then
# Check whether --enable-largefile or --disable-largefile was given.
if test ${enable_largefile+set} = set; then
Index: configure.in
===
RCS file: /cvsroot/pgsql/configure.in,v
retrieving revision 1.387
diff -c -c -r1.387 configure.in
*** configure.in30 Nov 2004 06:13:04 - 1.387
--- configure.in1 Dec 2004 22:54:11 -
***
*** 1174,1179
--- 1174,1186
PGAC_FUNC_POSIX_SIGNALS
+ if test $pgac_cv_func_posix_signals != yes -a $enable_thread_safety =
yes; then
+ AC_MSG_ERROR([
+ *** Thread-safety requires POSIX signals, which are not supported by your
+ *** operating system.
+ ])
+ fi
+
if test $ac_cv_func_fseeko = yes; then
AC_SYS_LARGEFILE
fi
Index: doc/src/sgml/libpq.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/libpq.sgml,v
retrieving revision 1.169
diff -c -c -r1.169 libpq.sgml
*** doc/src/sgml/libpq.sgml 27 Nov 2004 21:56:04 - 1.169
--- doc/src/sgml/libpq.sgml 1 Dec 2004 22:54:45 -
***
*** 3955,3975
/para
para
! applicationlibpq/application must ignore literalSIGPIPE/ signals
! generated internally by functionsend()/ calls to backend processes.
! When productnamePostgreSQL/ is configured without
! literal--enable-thread-safety/, applicationlibpq/ sets
! literalSIGPIPE/ to literalSIG_IGN/ before each
! functionsend()/ call and restores the original signal handler after
! completion. When literal--enable-thread-safety/ is used,
! applicationlibpq/ installs its own literalSIGPIPE/ handler
! before the first database connection. This handler uses thread-local
! storage to determine if a literalSIGPIPE/ signal has been generated
! by a libpq functionsend()/. If an application wants to install
! its own literalSIGPIPE/ signal handler, it should call
! functionPQinSend()/ to determine if it should ignore the
! literalSIGPIPE/ signal. This function is available in both
! thread-safe and non-thread-safe versions of applicationlibpq/.
/para
para
--- 3955,3964
/para
para
! applicationlibpq/application blocks and discards literalSIGPIPE/
! signals generated internally by functionsend()/ calls to the backend
! process. Therefore, applications should not expect blocked literalSIGPIPE/
! signals to remain across applicationlibpq/application function calls.
/para
para
Index: doc/src/sgml/ref/copy.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v
retrieving revision 1.60
diff -c -c -r1.60 copy.sgml
*** doc/src/sgml/ref/copy.sgml 27 Nov 2004 21:56:05 - 1.60
--- doc/src/sgml/ref/copy.sgml 1 Dec 2004 22:54:57 -
***
*** 3,8
--- 3,9
PostgreSQL documentation
--
+
refentry id=SQL-COPY
refmeta
refentrytitle id=sql-copy-titleCOPY/refentrytitle
Index: src/interfaces/libpq/fe-connect.c
===
RCS file: /cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.289
diff -c -c
Re: [PATCHES] [HACKERS] libpq and psql not on same page about SIGPIPE
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: No, that's wrong: if there is a pending SIGPIPE that belongs to the outer app, you'd clear it. True, but I documented that in the patch. A documented bug is still a bug. The entire point of this change is to not interfere with the behavior of the surrounding app, at least not more than we absolutely have to; and we do not have to clear SIGPIPEs that are certainly not ours. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] [HACKERS] libpq and psql not on same page about SIGPIPE
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: No, that's wrong: if there is a pending SIGPIPE that belongs to the outer app, you'd clear it. True, but I documented that in the patch. A documented bug is still a bug. The entire point of this change is to not interfere with the behavior of the surrounding app, at least not more than we absolutely have to; and we do not have to clear SIGPIPEs that are certainly not ours. I am working on a new version that doesn't have this problem. Will post soon. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Update for documentation on CVS
On Thu, 2004-12-02 at 00:52 +, Jon Jensen wrote: This is a little patch to correct the documentation on CVS. The URL for downloading CVS at cyclic.com site is long defunct, and I changed the text to not overtly recommend CVS 1.10, a now fairly old version. Applied. Thanks! -Neil ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
