Re: [HACKERS] Patch for 8.5, transformationHook
Hi, Robert Haas robertmh...@gmail.com writes: I don't really believe that JSON is only one use case. XML and JSON are in a class of their own; there's nothing else out there that is really comparable. You might want to hear about the UBF specs from Joe Armstrong, let me quote its page about it: UBF is a language for transporting and describing complex data structures across a network. It has three components: * UBF(A) is a data transport format, roughly equivalent to well-formed XML. * UBF(B) is a programming langauge for describing types in UBF(A) and protocols between clients and servers. UBF(B) is roughly equivalent to to Verified XML, XML-schemas, SOAP and WDSL. * UBF(C) is a meta-level protocol between used between UBF servers. While the XML series of languages had the goal of having a human readable format the UBF languages take the opposite view and provide a machine friendly format. http://www.sics.se/~joe/ubf/site/home.html It seems there's an ongoing revision to adapt this work to JSON nowadays: http://armstrongonsoftware.blogspot.com/2009/02/json-protocols-part-1.html Oh and now I'm wondering about ASN.1... Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Fwd: [BUGS] fix: plpgsql: return query and dropped columns problem
forward patch to pg_hackers There is fixed patch. Please, Jaime, can you look on it? Thank You Pavel 2009/7/30 Tom Lane t...@sss.pgh.pa.us: Jaime Casanova jcasa...@systemguards.com.ec writes: On Mon, Jul 20, 2009 at 10:09 AM, Alvaro Getting rid of the check on natts was ungood ... it needs to compare the number of undropped columns of both tupdescs. patch attached This patch is *still* introducing more bugs than it fixes. The reason is that it has modified validate_tupdesc_compat to allow the tupdescs to be somewhat different, but has fixed only one of the several call sites to deal with the consequences of that. The others will now become crash risks if we apply it as-is. What I would suggest as a suitable plan for a fix is to modify validate_tupdesc_compat so that it returns a flag indicating whether the tupdesc compatibility is exact or requires translation. If translation is required, provide another routine that does that --- probably using a mapping data structure set up by validate_tupdesc_compat, since in some of these cases we'll be processing many tuples. Then the callers just have to know enough to call the tuple-translation function when validate_tupdesc_compat tells them to. There are a number of other places in the system with similar requirements, although none of them seem to be exact matches. In particular ExecEvalConvertRowtype() provides a good template for efficient translation logic, but it's using column name matching rather than positional matching so you couldn't share the setup logic. I'm not sure if it's worth moving all this code into the core so that it can be shared with other future uses (maybe in other PLs), but it's worth considering that. access/common/heaptuple.c or tupdesc.c might be a good place for it if we decide to do that. The executor's junkfilter code is pretty nearly related as well, and perhaps the Right Thing is to make all of this stuff into applications of junkfilters with different setup routines for the different requirements. However the junkfilter code is designed to work with tuples that are in TupleTableSlots, which isn't particularly helpful for plpgsql's uses, so maybe trying to unify with that code is more trouble than it's worth. I'm marking this patch as Waiting on Author again, although perhaps Returned with Feedback would be better since my suggestions amount to wholesale rewrites. regards, tom lane *** ./src/pl/plpgsql/src/pl_exec.c.orig 2009-07-22 04:31:38.0 +0200 --- ./src/pl/plpgsql/src/pl_exec.c 2009-08-03 21:21:42.196893375 +0200 *** *** 190,196 Oid reqtype, int32 reqtypmod, bool isnull); static void exec_init_tuple_store(PLpgSQL_execstate *estate); ! static void validate_tupdesc_compat(TupleDesc expected, TupleDesc returned, const char *msg); static void exec_set_found(PLpgSQL_execstate *estate, bool state); static void plpgsql_create_econtext(PLpgSQL_execstate *estate); --- 190,200 Oid reqtype, int32 reqtypmod, bool isnull); static void exec_init_tuple_store(PLpgSQL_execstate *estate); ! static HeapTuple convert_tuple(TupleDesc dest_desc, TupleDesc src_desc, ! int *conversion_map, ! HeapTuple tuple); ! static int *validate_tupdesc_compat(TupleDesc expected, TupleDesc returned, ! bool *needs_conversion, const char *msg); static void exec_set_found(PLpgSQL_execstate *estate, bool state); static void plpgsql_create_econtext(PLpgSQL_execstate *estate); *** *** 216,221 --- 220,227 ErrorContextCallback plerrcontext; int i; int rc; + int *conversion_map = NULL; + bool needs_conversion = false; /* * Setup the execution state *** *** 388,394 { case TYPEFUNC_COMPOSITE: /* got the expected result rowtype, now check it */ ! validate_tupdesc_compat(tupdesc, estate.rettupdesc, returned record type does not match expected record type); break; case TYPEFUNC_RECORD: --- 394,400 { case TYPEFUNC_COMPOSITE: /* got the expected result rowtype, now check it */ ! conversion_map = validate_tupdesc_compat(tupdesc, estate.rettupdesc, needs_conversion, returned record type does not match expected record type); break; case TYPEFUNC_RECORD: *** *** 414,422 * Copy tuple to upper executor memory, as a tuple Datum. Make * sure it is labeled with the caller-supplied tuple type. */ ! estate.retval = ! PointerGetDatum(SPI_returntuple((HeapTuple) DatumGetPointer(estate.retval), tupdesc)); } else { --- 420,437 * Copy tuple to upper executor memory, as a tuple Datum. Make * sure it is labeled with the caller-supplied tuple type. */ ! if (!needs_conversion) ! estate.retval = !
Re: [HACKERS] SE-PostgreSQL Specifications
* KaiGai Kohei (kai...@ak.jp.nec.com) wrote: David Fetter wrote: On Mon, Aug 03, 2009 at 11:18:55PM -0400, Stephen Frost wrote: Just generally, access control is a great way to describe what's actually happening here. That people conflate access control with security has resulted in a number of disasters :P My concern is access_control_ is a bit long for prefixes, but ac_ is too short to represent what it is doing. pg_ac_? Still shorter than 'security_', uses the pg_ prefix, which we use in a number of other places, and has 'ac' in it.. Stephen signature.asc Description: Digital signature
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote: On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote: Included is a conceptual patch to use intptr_t. Comments are welcome. After closer inspection, not having a win64 box available, I have my doubts whether this patch actually does anything. Foremost, it doesn't touch the definition of the Datum type, which ought to be at the core of a change like this. Now I see that you call this a conceptual patch. Perhaps we should wait until you have developed it into a complete patch? Is there any reason to consider this patch any further during this CommitFest? It seems that this is a long way from being ready to go. I'm sorry for delaying response. This patch is needed as a base of the fix for Windows x64 in the future. There are still a lot of corrections necessary for Win x64. (typedef Datum, shared buffer, %lu messages, headers, build scripts, ...) We are trying these now, and want to offer the result to the next Commit Fest. Because we are glad if this pointer patch is confirmed at the early stage, we submitted patch to this Commit Fest. Thanks. -- Tsutomu Yamada SRA OSS, Inc. Japan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] doing %-expansion in plpgsql RAISE USING
Hi, It seems there's no way to do %-expansion in plpgsql when one is using RAISE USING: alvherre=# create or replace function f () returns void language plpgsql as $$ begin raise using message = 'hello %' || 'world'; return; end; $$; CREATE FUNCTION alvherre=# select f(); ERROR: hello %world I would like the % to be expanded to some argument, but obviously there's no way to pass the arguments that it should expand to. We could do something like RAISE USING message = 'hello %st %', args = 1, 'world' but this is obviously going to be difficult, if not impossible, to implement in the grammar. Perhaps RAISE USING message = 'brave %st %', args = (1, 'world') Thoughts? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Stephen Frost sfr...@snowman.net writes: * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: My concern is access_control_ is a bit long for prefixes, but ac_ is too short to represent what it is doing. pg_ac_? Still shorter than 'security_', uses the pg_ prefix, which we use in a number of other places, and has 'ac' in it.. I don't see anything wrong with ac_. Short is good, and there isn't any other concept in the PG internals that it would conflict with. If there were, pg_ac_ would surely not help to disambiguate. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.
Greg Stark gsst...@mit.edu writes: On Tue, Aug 4, 2009 at 5:04 AM, Tom Lanet...@postgresql.org wrote: Cause pg_proc.probin to be declared as text, not bytea. Doesn't this relate to the earlier discussion of whether to re-encode filenames and paths? What's going to happen if I have filenames which aren't valid encoded strings in the server encoding -- say UTF8 filenames but I'm using latin1 in the server or vice versa. Will my CREATE FUNCTION command end up storing an invalid encoded string? Or re-encode the filename and then fail to find the file? Right at the moment we simply aren't considering any of those cases. If you'd like to propose and implement a solution, feel free. I think the last proposal foundered on the fact that it had no idea what encoding the filesystem was expecting anyway. I'll point out though that having probin declared bytea would surely be antithetical to any attempt to treat shlib filenames in an encoding-aware fashion. Declaring it that way implies that it is *not* storing a character string that has any particular encoding. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Alvaro Herrera wrote: Joe Conway escribió: OK, how's this look? Hmm, is it possible to use OUT parameters in the function instead of declaring a new type for the result? Sure, I guess I ought to use the latest-and-greatest. Any other comments before I commit? Thanks, Joe -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iQIcBAEBCAAGBQJKeDzfAAoJEDfy90M199hl3FsP/RYXWVVlItm3jk3hKNPCqTaT maSwtl0QrFLl1pAc3ZTk16QdERrYFjfxIcxALz2am7OtStmhOz63Y4nlY8L0GHdz ZJDF7K3r5o2RJ12h7Zucpo3racxp1eJqfi4S61RCeIi+aPxtcJd7dds1588FYACc cPi6AuZyFfE6lhUjgniqOjzvTMptqWbtmuRZl1m+iyXsPE2FLlMclfugbzFKFXso cnUI31cuejg9tF1zboatopm/qAcIiCB+U2KVG7tyiI+KdL/bNJlkELRJKODxdCvB XNHzNBey1ZmbgE+H8gu1bz7DXbTxaffoQunCd4HnfOXOGrJpG+djsSCNW860sSVK icCvdvq3taHfcpLtIIjpecu9LatTFGkJ12YVIV33gnv/Fgr8pj+84VsxRfqlfNaN V743KdWtEv19gii3qIa817ZcS4tpBfIxXyt37cuZMdOv3VXR7LejeOoDr4aHGjDq SCozcQQ+Fh8diU1XMzl3rRtz3Brz1cVYE8Ue8ELcQzzuOvWT+a0cjCBii1SmZdTz DnpVEzmeOGrnHM8+IOG+h9IFRvNx3RQ2nxs71gcbgDp62fnQqb3m7c/Aj0xWv8xp mQhY2DNQg0AY1vr71nfMGbYL7xTHFD5fA2Md2l/029PA+qFhJlqnZpWScHwqafcC nkPH88kPUgy33vuNTJ3A =U88M -END PGP SIGNATURE- Index: contrib/dblink/dblink.c === RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v retrieving revision 1.82 diff -c -r1.82 dblink.c *** contrib/dblink/dblink.c 11 Jun 2009 14:48:50 - 1.82 --- contrib/dblink/dblink.c 4 Aug 2009 13:41:26 - *** *** 1635,1640 --- 1635,1723 PG_RETURN_DATUM(current_query(fcinfo)); } + /* + * Retrieve async notifications for a connection. + * + * Returns an setof record of notifications, or an empty set if none recieved. + * Can optionally take a named connection as parameter, but uses the unnamed connection per default. + * + */ + #define DBLINK_NOTIFY_COLS 3 + + PG_FUNCTION_INFO_V1(dblink_get_notify); + Datum + dblink_get_notify(PG_FUNCTION_ARGS) + { + PGconn *conn = NULL; + remoteConn *rconn = NULL; + PGnotify *notify; + ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo-resultinfo; + TupleDesc tupdesc; + Tuplestorestate *tupstore; + MemoryContext per_query_ctx; + MemoryContext oldcontext; + + DBLINK_INIT; + if (PG_NARGS() == 1) + DBLINK_GET_NAMED_CONN; + else + conn = pconn-conn; + + /* create the tuplestore */ + per_query_ctx = rsinfo-econtext-ecxt_per_query_memory; + oldcontext = MemoryContextSwitchTo(per_query_ctx); + + tupdesc = CreateTemplateTupleDesc(DBLINK_NOTIFY_COLS, false); + TupleDescInitEntry(tupdesc, (AttrNumber) 1, notify_name, + TEXTOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 2, be_pid, + INT4OID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 3, extra, + TEXTOID, -1, 0); + + tupstore = tuplestore_begin_heap(true, false, work_mem); + rsinfo-returnMode = SFRM_Materialize; + rsinfo-setResult = tupstore; + rsinfo-setDesc = tupdesc; + + MemoryContextSwitchTo(oldcontext); + + PQconsumeInput(conn); + while ((notify = PQnotifies(conn)) != NULL) + { + Datum values[DBLINK_NOTIFY_COLS]; + bool nulls[DBLINK_NOTIFY_COLS]; + + memset(values, 0, sizeof(values)); + memset(nulls, 0, sizeof(nulls)); + + if (notify-relname != NULL) + values[0] = CStringGetTextDatum(notify-relname); + else + nulls[0] = true; + + values[1] = Int32GetDatum(notify-be_pid); + + if (notify-extra != NULL) + values[2] = CStringGetTextDatum(notify-extra); + else + nulls[2] = true; + + /* switch to appropriate context while storing the tuple */ + MemoryContextSwitchTo(per_query_ctx); + tuplestore_putvalues(tupstore, tupdesc, values, nulls); + MemoryContextSwitchTo(oldcontext); + + PQfreemem(notify); + PQconsumeInput(conn); + } + + /* clean up and return the tuplestore */ + tuplestore_donestoring(tupstore); + + return (Datum) 0; + } + /* * internal functions */ Index: contrib/dblink/dblink.h === RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v retrieving revision 1.22 diff -c -r1.22 dblink.h *** contrib/dblink/dblink.h 9 Jun 2009 17:41:02 - 1.22 --- contrib/dblink/dblink.h 4 Aug 2009 13:41:26 - *** *** 57,61 --- 57,62 extern Datum dblink_build_sql_delete(PG_FUNCTION_ARGS); extern Datum dblink_build_sql_update(PG_FUNCTION_ARGS); extern Datum dblink_current_query(PG_FUNCTION_ARGS); + extern Datum dblink_get_notify(PG_FUNCTION_ARGS); #endif /* DBLINK_H */ Index: contrib/dblink/dblink.sql.in === RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.sql.in,v retrieving revision 1.18 diff -c -r1.18 dblink.sql.in *** contrib/dblink/dblink.sql.in 9 Jun 2009 17:41:02 - 1.18 ---
Re: [HACKERS] doing %-expansion in plpgsql RAISE USING
Alvaro Herrera alvhe...@commandprompt.com writes: It seems there's no way to do %-expansion in plpgsql when one is using RAISE USING: That's intentional. Just use string concatenation if you need a run-time-variable message. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] doing %-expansion in plpgsql RAISE USING
Hello 2009/8/4 Alvaro Herrera alvhe...@commandprompt.com: Hi, It seems there's no way to do %-expansion in plpgsql when one is using RAISE USING: alvherre=# create or replace function f () returns void language plpgsql as $$ begin raise using message = 'hello %' || 'world'; return; end; $$; CREATE FUNCTION alvherre=# select f(); ERROR: hello %world parameter is expression, so we could to define operator % like text % any % isn't defined for text so this is possible. This operator should be generally used, not only in RAISE attribs. I would like the % to be expanded to some argument, but obviously there's no way to pass the arguments that it should expand to. We could do something like RAISE USING message = 'hello %st %', args = 1, 'world' RAISE USING message = 'hello %st %' % (1, 'world') ?? or simple use custom variadic function RAISE USING message= subst('hello %st %', 1, 'world') good example for parser hook :) Pavel but this is obviously going to be difficult, if not impossible, to implement in the grammar. Perhaps RAISE USING message = 'brave %st %', args = (1, 'world') Thoughts? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: My concern is access_control_ is a bit long for prefixes, but ac_ is too short to represent what it is doing. pg_ac_? Still shorter than 'security_', uses the pg_ prefix, which we use in a number of other places, and has 'ac' in it.. I don't see anything wrong with ac_. Short is good, and there isn't any other concept in the PG internals that it would conflict with. If there were, pg_ac_ would surely not help to disambiguate. Works for me. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Alpha Releases: Docs?
On Monday 03 August 2009 23:08:51 Josh Berkus wrote: There's another question for alpha releases: are we going to build docs? Yes, absolutely. I'm working on making the documentation build part of the tarball build procedure. Either for www.postgresql.org, or for PGDATA/docs? The web team has to figure out whether putting the docs on the web site is worthwhile. We already have the developer docs that are rebuild on every check-in. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote: Josh Berkus wrote: I think we need some kind of docs up, otherwise we'll get little actual testing. As previously discussed, building the docs yourself from pure source involves several complicated dependancies which aren't available on all platforms. Are we going to ship ChangeLog files or some such, giving that we're not going to have release notes? There will still be release notes. They might not be as polished as the final ones, but there will be a list of new things, at least. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
Pavel Stehule pavel.steh...@gmail.com writes: I agree, so information about patch would be store in text field. But I am not sure, if your fix isn't too simply. I haven't plan to compile plpgsql to C or to binary code. But could be interesting link postgres with some virtual machine like parrot or lua vm, and translate plpgsql to p code. It's maybe far future. Early future is integration main SQL parser to plpgsql. I am not sure, but maybe we will need some persistent cache for store parametrized sql queries. I though about store it in probin column. Well, probin is the wrong place for any such thing anyhow. Regardless of datatype issues, the system is clearly built on the assumption that the value of probin is to be specified *by the user* in CREATE FUNCTION. If you want a cache for derived information it would need to be a new column. (I remain of the opinion that caching such stuff in pg_proc would be a bad design decision. Every data structure that goes to disk is another data structure that you cannot easily change. There just isn't enough gain there to justify the maintenance headaches.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
On Tuesday 04 August 2009 14:03:34 Tsutomu Yamada wrote: Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote: On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote: Included is a conceptual patch to use intptr_t. Comments are welcome. After closer inspection, not having a win64 box available, I have my doubts whether this patch actually does anything. Foremost, it doesn't touch the definition of the Datum type, which ought to be at the core of a change like this. Now I see that you call this a conceptual patch. Perhaps we should wait until you have developed it into a complete patch? Is there any reason to consider this patch any further during this CommitFest? It seems that this is a long way from being ready to go. I'm sorry for delaying response. This patch is needed as a base of the fix for Windows x64 in the future. There are still a lot of corrections necessary for Win x64. (typedef Datum, shared buffer, %lu messages, headers, build scripts, ...) We are trying these now, and want to offer the result to the next Commit Fest. Because we are glad if this pointer patch is confirmed at the early stage, we submitted patch to this Commit Fest. Well, there is nothing outright wrong with this patch, but without any measurable effect, it is too early to commit it. At least I would like to see the Datum typedef to be changed to use intptr_t and the fallout from that cleaned up. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
On Monday 03 August 2009 22:11:08 Tom Lane wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I did pose that question in my patch submission email. Unless there is overwhelming support in favor of changing, we probably shouldn't change it, at least not yet. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] doing %-expansion in plpgsql RAISE USING
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: It seems there's no way to do %-expansion in plpgsql when one is using RAISE USING: That's intentional. Just use string concatenation if you need a run-time-variable message. Yes, I can do that, but it's really ugly and gets unmaintainable quickly. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
On Tue, Aug 4, 2009 at 16:10, Peter Eisentrautpete...@gmx.net wrote: On Tuesday 04 August 2009 14:03:34 Tsutomu Yamada wrote: Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote: On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote: Included is a conceptual patch to use intptr_t. Comments are welcome. After closer inspection, not having a win64 box available, I have my doubts whether this patch actually does anything. Foremost, it doesn't touch the definition of the Datum type, which ought to be at the core of a change like this. Now I see that you call this a conceptual patch. Perhaps we should wait until you have developed it into a complete patch? Is there any reason to consider this patch any further during this CommitFest? It seems that this is a long way from being ready to go. I'm sorry for delaying response. This patch is needed as a base of the fix for Windows x64 in the future. There are still a lot of corrections necessary for Win x64. (typedef Datum, shared buffer, %lu messages, headers, build scripts, ...) We are trying these now, and want to offer the result to the next Commit Fest. Because we are glad if this pointer patch is confirmed at the early stage, we submitted patch to this Commit Fest. Well, there is nothing outright wrong with this patch, but without any measurable effect, it is too early to commit it. At least I would like to see the Datum typedef to be changed to use intptr_t and the fallout from that cleaned up. +1. I think it's good that it was posted for a quick review of the general idea, but I agree that it's too early to commit it until we can see some actual benefit. And I expect the Datum changes to be much larger than this, so we can just review/apply them as one when the time comes. -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
Joe Conway m...@joeconway.com writes: Sure, I guess I ought to use the latest-and-greatest. Any other comments before I commit? That be_pid/be_pid hack in the regression test is pretty ugly, and doesn't test anything very useful anyway seeing that it's integer division. Could you do something like be_pid = pg_backend_pid() AS is_self_notify instead, to verify that it's a self-notify? (This is not quite right because you'd need to execute pg_backend_pid() at the remote end, but I'm not awake enough to remember the dblink syntax for that. Maybe it's too complex to be worth it, but I think demonstrating how to check for self-notify would be a useful bit of doco.) Otherwise it looks ok to me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha releases: How to tag
On Monday 03 August 2009 22:52:55 David Fetter wrote: On Mon, Aug 03, 2009 at 09:22:52PM +0300, Peter Eisentraut wrote: On Monday 03 August 2009 21:07:00 David Fetter wrote: We require that people supply docs with their changes, and it is totally unreasonable to let them send in catalog changes which do not include need migration changes. That's how it works in every other RDBMS outfit that has changes on disk, and we do not need to be the exception. Well, blocker number one for that is that pg_migrator is not even in the PostgreSQL CVS repository, but is more like an endorsed third-party product. I'm not entirely sure that pg_migrator should be tied to releases of PostgreSQL, given what it does. Or did you mean that it's not been given the same scrutiny that the PostgreSQL code base has? Well, how to you expect to mandate that all patches come with upgrade support if the PostgreSQL software proper does not contain any upgrade support in the first place, because pg_migrator is maintained elsewhere. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tue, Aug 4, 2009 at 16:06, Peter Eisentrautpete...@gmx.net wrote: On Monday 03 August 2009 23:08:51 Josh Berkus wrote: The web team has to figure out whether putting the docs on the web site is worthwhile. We already have the developer docs that are rebuild on every check-in. As long as the HTML files are available, we can make this happen. We don't have the website servers set up with the full build tools, and we don't want that :-) It is probably not going to be a terrible amount of work, so if we are going to actually push for people to do serious testing on the alpha releases, we should definitely get the docs up on the website. I don't think we should keep an archive of old alphas though - that's going to leave us with insane amounts of documentation sets. But we could have a /docs/alpha/ which would hold the latest released alpha. -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 22:11:08 Tom Lane wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I did pose that question in my patch submission email. Unless there is overwhelming support in favor of changing, we probably shouldn't change it, at least not yet. While I've been poking at the pg_dump issues, it's occurred to me that changing the default would be a great forcing function for finding out any lurking problems. What I'm inclined to do now is to commit it *with* the change of default, and let it be that way at least for a few alpha-test releases. We can vote on whether to switch the default back before 8.5 final. If this seems reasonable, I can make a note of the point in the commit message, so that we won't forget when the time comes. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 22:11:08 Tom Lane wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I did pose that question in my patch submission email. Unless there is overwhelming support in favor of changing, we probably shouldn't change it, at least not yet. While I've been poking at the pg_dump issues, it's occurred to me that changing the default would be a great forcing function for finding out any lurking problems. What I'm inclined to do now is to commit it *with* the change of default, and let it be that way at least for a few alpha-test releases. We can vote on whether to switch the default back before 8.5 final. If this seems reasonable, I can make a note of the point in the commit message, so that we won't forget when the time comes. Or, what we could do is start an open items for 8.5 list similar to the one we made for 8.4. That worked pretty well, I think. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
On Tue, Aug 4, 2009 at 16:31, Robert Haasrobertmh...@gmail.com wrote: On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 22:11:08 Tom Lane wrote: I'm starting to look at this patch. I observe that it's setting the default output format to HEX. If changing the default behavior was agreed to, or even discussed, I do not remember where. Shouldn't the default stay the same? I did pose that question in my patch submission email. Unless there is overwhelming support in favor of changing, we probably shouldn't change it, at least not yet. While I've been poking at the pg_dump issues, it's occurred to me that changing the default would be a great forcing function for finding out any lurking problems. What I'm inclined to do now is to commit it *with* the change of default, and let it be that way at least for a few alpha-test releases. We can vote on whether to switch the default back before 8.5 final. If this seems reasonable, I can make a note of the point in the commit message, so that we won't forget when the time comes. Or, what we could do is start an open items for 8.5 list similar to the one we made for 8.4. That worked pretty well, I think. +1 for that solution, it seems much better than having to go back through commit messages. We might as well start it early! -- Magnus Hagander Self: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
On Mon, Aug 03, 2009 at 10:03:47AM -0500, Kevin Grittner wrote: That's about 0.52% slower with the patch. Because there was over 10% variation in the numbers with the patch, I tried leaving out the four highest outliers on both, in case it was the result of some other activity on the system (even though this machine should have been pretty quiet over the weekend) and the difference fell to 0.09%. What do people do when testing this? I think I'd look to something like Student's t-test to check for statistical significance. My working would go something like: I assume the variance is the same because it's being tested on the same machine. samples = 20 stddev = 144.26 avg1= 4783.13 avg2= 4758.46 t = 0.54 ((avg1 - avg2) / (stddev * sqrt(2/samples))) We then have to choose how certain we want to be that they're actually different, 90% is a reasonably easy level to hit (i.e. one part in ten, with 95% being more commonly quoted). For 20 samples we have 19 degrees of freedom--giving us a cut-off[1] of 1.328. 0.54 is obviously well below this allowing us to say that there's no statistical significance between the two samples at a 90% level. -- Sam http://samason.me.uk/ [1] http://en.wikipedia.org/wiki/Student's_t-distribution#Table_of_selected_values -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Robert Haas robertmh...@gmail.com writes: On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote: If this seems reasonable, I can make a note of the point in the commit message, so that we won't forget when the time comes. Or, what we could do is start an open items for 8.5 list similar to the one we made for 8.4. That worked pretty well, I think. OK. Historically we haven't made such a list until beta starts, but there's no reason we couldn't start it early. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.
On Tue, Aug 4, 2009 at 2:46 PM, Tom Lanet...@sss.pgh.pa.us wrote: I'll point out though that having probin declared bytea would surely be antithetical to any attempt to treat shlib filenames in an encoding-aware fashion. Declaring it that way implies that it is *not* storing a character string that has any particular encoding. Well that's kind of the point. Unix filesystems traditionally prohibit '/' and '\0' but otherwise allowing any series of bytes without requiring any particular encoding. If we used bytea to store filesystem paths then you could specify any arbitrary series of bytes without worrying that the server will re-encode it differently. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
Pavel Stehule escribió: I agree, so information about patch would be store in text field. But I am not sure, if your fix isn't too simply. I haven't plan to compile plpgsql to C or to binary code. But could be interesting link postgres with some virtual machine like parrot or lua vm, and translate plpgsql to p code. It's maybe far future. In this case I think it would make more sense to compile the code and keep the p-code in backend local memory. Keeping compiled code around in more permanent or global storage would only make sense if you had large amounts of code, and in that case I think we'd want to have a more concrete and complete proposal. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
Sam Mason s...@samason.me.uk writes: t = 0.54 ((avg1 - avg2) / (stddev * sqrt(2/samples))) We then have to choose how certain we want to be that they're actually different, 90% is a reasonably easy level to hit (i.e. one part in ten, with 95% being more commonly quoted). For 20 samples we have 19 degrees of freedom--giving us a cut-off[1] of 1.328. 0.54 is obviously well below this allowing us to say that there's no statistical significance between the two samples at a 90% level. Hmm, so what about 95% or 99% confidence? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote: Are we going to ship ChangeLog files or some such, giving that we're not going to have release notes? There will still be release notes. They might not be as polished as the final ones, but there will be a list of new things, at least. That would be great, but who's going to do them? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote: Are we going to ship ChangeLog files or some such, giving that we're not going to have release notes? There will still be release notes. They might not be as polished as the final ones, but there will be a list of new things, at least. That would be great, but who's going to do them? It's easy, just cvs2cl and paste that in a large pre block. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
Magnus Hagander mag...@hagander.net writes: On Tue, Aug 4, 2009 at 16:10, Peter Eisentrautpete...@gmx.net wrote: Well, there is nothing outright wrong with this patch, but without any measurable effect, it is too early to commit it. At least I would like to see the Datum typedef to be changed to use intptr_t and the fallout from that cleaned up. +1. I think it's good that it was posted for a quick review of the general idea, but I agree that it's too early to commit it until we can see some actual benefit. And I expect the Datum changes to be much larger than this, so we can just review/apply them as one when the time comes. The other thing that I would say is a non-negotiable minimum requirement is that the patch include the necessary configure pushups so it does not break machines without uintptr_t. I think we could just do a conditional typedef unsigned long uintptr_t; and proceed from there; then machines without the typedef are no worse off than before. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_proc.probin should become text?
2009/8/4 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: I agree, so information about patch would be store in text field. But I am not sure, if your fix isn't too simply. I haven't plan to compile plpgsql to C or to binary code. But could be interesting link postgres with some virtual machine like parrot or lua vm, and translate plpgsql to p code. It's maybe far future. Early future is integration main SQL parser to plpgsql. I am not sure, but maybe we will need some persistent cache for store parametrized sql queries. I though about store it in probin column. Well, probin is the wrong place for any such thing anyhow. Regardless of datatype issues, the system is clearly built on the assumption that the value of probin is to be specified *by the user* in CREATE FUNCTION. If you want a cache for derived information it would need to be a new column. (I remain of the opinion that caching such stuff in pg_proc would be a bad design decision. Every data structure that goes to disk is another data structure that you cannot easily change. There just isn't enough gain there to justify the maintenance headaches.) ook, I agree - but I am not sure, so some cache will be necessary. Pavel regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.
Greg Stark wrote: On Tue, Aug 4, 2009 at 2:46 PM, Tom Lanet...@sss.pgh.pa.us wrote: I'll point out though that having probin declared bytea would surely be antithetical to any attempt to treat shlib filenames in an encoding-aware fashion. Declaring it that way implies that it is *not* storing a character string that has any particular encoding. Well that's kind of the point. Unix filesystems traditionally prohibit '/' and '\0' but otherwise allowing any series of bytes without requiring any particular encoding. If we used bytea to store filesystem paths then you could specify any arbitrary series of bytes without worrying that the server will re-encode it differently. Is this any different from the path in COPY foo to '/path/to/file'? I suspect the probin stuff is a solution in search of a problem. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic
On Tue, Aug 04, 2009 at 10:45:52AM -0400, Tom Lane wrote: Sam Mason s...@samason.me.uk writes: t = 0.54 ((avg1 - avg2) / (stddev * sqrt(2/samples))) We then have to choose how certain we want to be that they're actually different, 90% is a reasonably easy level to hit (i.e. one part in ten, with 95% being more commonly quoted). For 20 samples we have 19 degrees of freedom--giving us a cut-off[1] of 1.328. 0.54 is obviously well below this allowing us to say that there's no statistical significance between the two samples at a 90% level. Hmm, so what about 95% or 99% confidence? The cut-off goes up to 1.729 for 95% and to 2.539 for 99%. These values are only really for a 20 samples with the above calculation, the link I gave above gives a nice table for different values. I've also realized that I did the standard deviation all wrong. I should have calculated them independently and then got the mean: stddev1 = 159.9699 stddev2 = 129.6466 stddev = 144.8083 ((stddev1+stddev2) / 2) Here it makes absolutely no difference, but when they were really different distributions it would. -- Sam http://samason.me.uk/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tuesday 04 August 2009 17:48:12 Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote: Are we going to ship ChangeLog files or some such, giving that we're not going to have release notes? There will still be release notes. They might not be as polished as the final ones, but there will be a list of new things, at least. That would be great, but who's going to do them? I can do it. Perhaps Josh wants to get involved. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
On Tuesday 04 August 2009 17:56:41 Tom Lane wrote: The other thing that I would say is a non-negotiable minimum requirement is that the patch include the necessary configure pushups so it does not break machines without uintptr_t. There is AC_TYPE_UINTPTR_T, so that should be easy. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote: Are we going to ship ChangeLog files or some such, giving that we're not going to have release notes? There will still be release notes. They might not be as polished as the final ones, but there will be a list of new things, at least. That would be great, but who's going to do them? I think the question is who are you going to allow to do them?. I sort of get the impression that you and Bruce have a love-hate relationship with the release notes. On the one hand, they're a huge time sink. On the other hand, it feels like you're not quite ready to turn them over to anyone else. I feel like there have been some previous offers of help (including by me) that either weren't an offer to do quite the right thing or the offer was made at the wrong stage of the process. I believe you told me on one occasion that they had to all be written by a single person so that they'd be consistent... but then, even though Bruce did the initial draft of the release notes, you did most of the updating as we got closer to release and more things were committed, so apparently it wasn't that critical for one person to do it all after all. For alpha, I think a dump of the CVS commit logs is fine if there's not a better option. That's what Peter originally proposed, and there's no reason to change now. But I suspect that it might be possible to find someone who would be willing to do more than that if they were treated nicely. For example, if someone volunteers to help with this now, will their work be thrown out and redone by you and Bruce the right way when the actual release arrives? Or will we just revise and extend that work throughout the release cycle? If you want to be rid of this task, then help some other people to do it and do it well. If you want to keep doing it yourself, then dump the CVS logs into the alpha release notes and call it good. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.
Andrew Dunstan and...@dunslane.net writes: Is this any different from the path in COPY foo to '/path/to/file'? I suspect the probin stuff is a solution in search of a problem. Well, the previous probin behavior is demonstrably broken. Make a shlib with backslash or non-ASCII in the name, create a function referencing it, dump and reload. Whatever your opinions are about encodings, you won't think pg_dump did the right thing. I'm not sure whether the more general pathname encoding issue is worth working on or not. In general it's a non-problem if the paths in the server filesystem are written in the database encoding. If they are not, then you have to figure out what they *are* written in, and that seems a bit tough. But anyway that problem is hardly restricted to probin, and a solution that works only for probin doesn't seem terribly interesting. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: More portable way to support 64bit platforms
Peter Eisentraut pete...@gmx.net writes: On Tuesday 04 August 2009 17:56:41 Tom Lane wrote: The other thing that I would say is a non-negotiable minimum requirement is that the patch include the necessary configure pushups so it does not break machines without uintptr_t. There is AC_TYPE_UINTPTR_T, so that should be easy. It's certainly do-able, the point is just to do it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
Robert Haas robertmh...@gmail.com writes: On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote: That would be great, but who's going to do them? I think the question is who are you going to allow to do them?. You misread that entirely. What I was pointing out was that we didn't have a volunteer to expend the nontrivial amount of time required to do nice notes. If you're volunteering, step right up. (If cvs2cl dump is the best we can do, so be it; but I predict it will put off a lot of potential testers. The commit messages have always been written by hackers for hackers, not for end users.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tue, Aug 4, 2009 at 11:44 AM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote: That would be great, but who's going to do them? I think the question is who are you going to allow to do them?. You misread that entirely. What I was pointing out was that we didn't have a volunteer to expend the nontrivial amount of time required to do nice notes. If you're volunteering, step right up. I'm willing to help if these are 8.5 release notes in process. I'm not willing to help if they are alpha release notes that will be thrown away afterwards. Which is it? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
Robert Haas robertmh...@gmail.com writes: I'm willing to help if these are 8.5 release notes in process. I'm not willing to help if they are alpha release notes that will be thrown away afterwards. Which is it? That depends largely on what they look like when we get to beta, I imagine. Are you asking for a guarantee that no one will edit your deathless prose? Traditionally one of the main time sinks involved in making the release notes has been trying to give them a uniform voice, categorizing them sensibly, weighting the space given to different topics in a way that seems to make sense in hindsight, etc. I'd be rather surprised if notes prepared piecemeal over a series of alpha releases didn't need work of that sort when we get to the end. That doesn't mean the work would be thrown away, but it does mean it's likely to get edited. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
--On Montag, August 03, 2009 23:43:08 +0200 Bernd Helmle maili...@oopsware.de wrote: Status Report: I will finish documentation and review tomorrow and will mark this patch for committer review. Here's my latest reviewed version of Pavel's patch with adjusted documentation per latest discussion. While poking a little bit with simplify_function() I realized that this patch changes the behavior of VARIADIC functions a little bit. For example: CREATE OR REPLACE FUNCTION my_test(a IN text, txt VARIADIC text[]) RETURNS text AS $$ SELECT $2[1]; $$ LANGUAGE SQL; The following doesn't work in current 8.4: SELECT my_test('abcd', ARRAY['test', 'foo']); You need to use the VARIADIC keyword to match the second argument to text[]. However, if you are going to use named notation with the patch applied, the picture changes in HEAD: SELECT my_test('abcd' AS a, ARRAY['test', 'foo'] AS txt); my_test - test (1 row) This applies also when you reverse the argument order. I don't know wether this is intended, but its conflicting with what we have currently in the docs. It's also not clear to me wether we want this at all. -- Thanks Bernd named_and_mixed_notation_review3.patch.gz Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
--On Dienstag, August 04, 2009 10:28:48 -0400 Tom Lane t...@sss.pgh.pa.us wrote: While I've been poking at the pg_dump issues, it's occurred to me that changing the default would be a great forcing function for finding out any lurking problems. +1 -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
Bernd Helmle maili...@oopsware.de writes: --On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut pete...@gmx.net wrote: OK, here is an updated patch. It has the setting as enum, completed documentation, and libpq support. I'll add it to the commit fest in the hope that someone else can look it over in detail. I've attached a slightly edited patch which fixes a compiler warning in encode.c, too. Committed with assorted corrections. I have not done anything about the issues mentioned in http://archives.postgresql.org/message-id/21837.1248215...@sss.pgh.pa.us mainly that pg_dump's treatment of large-object contents is not safe against changes of standard_conforming_strings. I think that ought to get dealt with before moving on. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tue, Aug 4, 2009 at 12:01 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm willing to help if these are 8.5 release notes in process. I'm not willing to help if they are alpha release notes that will be thrown away afterwards. Which is it? That depends largely on what they look like when we get to beta, I imagine. Are you asking for a guarantee that no one will edit your deathless prose? Not at all. Don't misunderstand me here: what I was worried about (and maybe it's without any justification) is that you and/or Bruce have ideas about how this has to be done that are so specific that it's not even worth anyone else making an attempt. If that were the case, then I wouldn't be very interested in working on it. Now, on the other hand, if doing some work now incrementally over the next couple of alpha releases will not only produce good release notes for those alpha releases but also streamline the process of putting together release notes for beta/final, then it sounds like a worthwhile investment of time. In the 8.4 release cycle, this was one of the things that held us up a bit (not a huge amount, but a bit): all the work was done at the end, and there was a lot to do. If we can make a start on this early, and if the start is good enough that it requires incremental changes rather than rm -f then that seems like a pretty good idea. Traditionally one of the main time sinks involved in making the release notes has been trying to give them a uniform voice, categorizing them sensibly, weighting the space given to different topics in a way that seems to make sense in hindsight, etc. I'd be rather surprised if notes prepared piecemeal over a series of alpha releases didn't need work of that sort when we get to the end. That doesn't mean the work would be thrown away, but it does mean it's likely to get edited. OK, great. That sounds like exactly the sort of editing that I would expect to be necessary. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bytea vs. pg_dump
I wrote: While I've been poking at the pg_dump issues, it's occurred to me that changing the default would be a great forcing function for finding out any lurking problems. What I'm inclined to do now is to commit it *with* the change of default, and let it be that way at least for a few alpha-test releases. We can vote on whether to switch the default back before 8.5 final. For the archives, attached is a patch to switch the default and then make pg_dump force use of hex mode. This is just so we won't forget what needs changing if we decide to switch the default back ... regards, tom lane *** doc/src/sgml/config.sgml.orig Tue Aug 4 12:08:35 2009 --- doc/src/sgml/config.sgmlTue Aug 4 12:40:34 2009 *** *** 4068,4078 listitem para Sets the output format for values of type typebytea/type. ! Valid values are literalhex/literal (the default) and literalescape/literal (the traditional PostgreSQL format). See xref linkend=datatype-binary for more information. The typebytea/type type always accepts both formats on input, regardless of this setting. /para /listitem /varlistentry --- 4068,4079 listitem para Sets the output format for values of type typebytea/type. ! Valid values are literalhex/literal and literalescape/literal (the traditional PostgreSQL format). See xref linkend=datatype-binary for more information. The typebytea/type type always accepts both formats on input, regardless of this setting. + The default is literalescape/literal. /para /listitem /varlistentry *** doc/src/sgml/datatype.sgml.orig Tue Aug 4 12:08:35 2009 --- doc/src/sgml/datatype.sgml Tue Aug 4 12:41:03 2009 *** *** 1196,1202 quoteescape/quote format, and quotehex/quote format. Both of these are always accepted on input. The output format depends on the configuration parameter xref linkend=guc-bytea-output; ! the default is hex. (Note that the hex format was introduced in productnamePostgreSQL/productname 8.5; earlier versions and some tools don't understand it.) /para --- 1196,1202 quoteescape/quote format, and quotehex/quote format. Both of these are always accepted on input. The output format depends on the configuration parameter xref linkend=guc-bytea-output; ! the default is escape. (Note that the hex format was introduced in productnamePostgreSQL/productname 8.5; earlier versions and some tools don't understand it.) /para *** src/backend/utils/adt/varlena.c.origTue Aug 4 12:08:36 2009 --- src/backend/utils/adt/varlena.c Tue Aug 4 12:42:36 2009 *** *** 30,36 /* GUC variable */ ! int bytea_output = BYTEA_OUTPUT_HEX; typedef struct varlena unknown; --- 30,36 /* GUC variable */ ! int bytea_output = BYTEA_OUTPUT_ESCAPE; typedef struct varlena unknown; *** src/backend/utils/misc/guc.c.orig Tue Aug 4 12:08:36 2009 --- src/backend/utils/misc/guc.cTue Aug 4 12:42:04 2009 *** *** 2553,2559 NULL }, bytea_output, ! BYTEA_OUTPUT_HEX, bytea_output_options, NULL, NULL }, { --- 2553,2559 NULL }, bytea_output, ! BYTEA_OUTPUT_ESCAPE, bytea_output_options, NULL, NULL }, { *** src/backend/utils/misc/postgresql.conf.sample.orig Mon Aug 3 15:59:39 2009 --- src/backend/utils/misc/postgresql.conf.sample Tue Aug 4 12:42:15 2009 *** *** 424,430 #statement_timeout = 0# in milliseconds, 0 is disabled #vacuum_freeze_min_age = 5000 #vacuum_freeze_table_age = 15000 ! #bytea_output = 'hex' # hex, escape #xmlbinary = 'base64' #xmloption = 'content' --- 424,430 #statement_timeout = 0# in milliseconds, 0 is disabled #vacuum_freeze_min_age = 5000 #vacuum_freeze_table_age = 15000 ! #bytea_output = 'escape' # hex, escape #xmlbinary = 'base64' #xmloption = 'content' *** src/bin/pg_dump/pg_dump.c.orig Tue Aug 4 12:08:36 2009 --- src/bin/pg_dump/pg_dump.c Tue Aug 4 12:39:39 2009 *** *** 599,604 --- 599,610 do_sql_command(g_conn, SET extra_float_digits TO 2); /* +* If supported, select hex format for bytea, for speed reasons. +*/ + if (g_fout-remoteVersion = 80500) + do_sql_command(g_conn, SET bytea_output TO hex); + + /* * If synchronized scanning is supported, disable it, to prevent * unpredictable changes in row ordering
[HACKERS] the case for machine-readable error fields
Hi, I think the case for machine-readable error fields is well made and doesn't need any further push. So, what do we need to make it happen? Here's my proposal. First we need several new error message fields: table name, function name, constraint name, and so on. One possible way to go about this would be to give each new field its own start letter (see http://www.postgresql.org/docs/8.4/static/protocol-error-fields.html); say T for table, f for function (F is taken), c for constraint (C is taken), and so on. Another possibility would be to use a single letter, say N, and add a subtype to it; so table name would be NT followed by the table name, NF for functions, etc. The documentation on the FE/BE protocol already says that frontends should ignore unrecognized type fields, so I don't think we need to bump the protocol version for this. The other part is getting the information in ErrorData. AFAICS this is just a matter of adding a few setter functions; say errtable(), errfunction(), etc. Furthermore, we can just add those to existing errcontext callbacks, which would be pretty simple. Thoughts? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.4 win32 shared memory patch
On Sat, Aug 1, 2009 at 20:30, Kevin Fieldkevinjamesfi...@gmail.com wrote: The event viewer says: The description for Event ID ( 0 ) in Source ( PostgreSQL ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: pg_ctl: could not find postgres program executable And yes, I renamed it correctly... Check permissions on it. If you moved it at some point, it may have the wrong permissions. They should be the same as for the other .EXEs in that directory. The two files (new and old exe) have identical permissions. That's just weird. It could be that the postgres executable won't work - maybe because of some DLL issue. Can you run postgres -V on the executable, or does that give you some error? It reports the version correctly. Sorry...any other ideas? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tuesday 04 August 2009 18:52:06 Robert Haas wrote: I'm willing to help if these are 8.5 release notes in process. I'm not willing to help if they are alpha release notes that will be thrown away afterwards. Which is it? I was working on the latter assumption. I have some reservations about the former approach. It would basically commit us right now to having a consistent set of volunteers available every two months within specific 1-2 day spans. Which is the sort of thing I wanted to avoid. But if we have that commitment, then go for it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] DefaultACLs
* Stephen Frost (sfr...@snowman.net) wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jul 22, 2009 at 11:26 PM, Petr Jelinekpjmo...@pjmodos.net wrote: The docs are not complete but that's up to Stephen, otherwise the patch should be finished. But I am not the reviewer :) Well, perhaps we had better prod Stephen then, since complete docs are a requirement for commit. I didn't think the docs posted were terrible, but I am working on improving them. Thanks to Joshua, there weren't really many changes I found for the docs. Here they are anyway: grant.sgml: + Replace current privileges with the ones specified using + xref linkend=sql-alterschema endterm=sql-alterschema-title. + The literalWITH GRANT OPTION/literal parameter is not applicable + because it is copied from default privileges. + Note: this can actually emphasis role=boldrevoke/emphasis some + privileges because it clears all existing privileges object has and + replaces them with the default ones for the schema in which this object + resides. How about: Replaces current privileges with the default privileges, as set using xref linkend=sql-alterschema endterm=sql-alterschema-title, for this object type in its current schema. The literalWITH GRANT OPTION/literal parameter is not applicable because it is copied from default privileges. Note: This can emphasis role=boldrevoke/emphasis privileges! This will clear all existing privileges first! If no default privilege has been set, the object will have all privileges revoked! Otherwise, I thought the docs looked pretty good. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] Alpha Releases: Docs?
I have some reservations about the former approach. It would basically commit us right now to having a consistent set of volunteers available every two months within specific 1-2 day spans. Which is the sort of thing I wanted to avoid. But if we have that commitment, then go for it. Can we hear from Bruce on this, as well? I think Robert is right to be concerned that his work will be thrown away; so am I. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] DefaultACLs
On Tue, Aug 04, 2009 at 01:28:00PM -0400, Stephen Frost wrote: Thanks to Joshua, there weren't really many changes I found for the docs. Here they are anyway: Yay, I was useful! :) How about: Replaces current privileges with the default privileges, as set using xref linkend=sql-alterschema endterm=sql-alterschema-title, for this object type in its current schema. The literalWITH GRANT OPTION/literal parameter is not applicable because it is copied from default privileges. Note: This can emphasis role=boldrevoke/emphasis privileges! This will clear all existing privileges first! If no default privilege has been set, the object will have all privileges revoked! Otherwise, I thought the docs looked pretty good. No complaints here, FWIW. - Josh signature.asc Description: Digital signature
Re: [HACKERS] Alpha Releases: Docs?
Magnus, I don't think we should keep an archive of old alphas though - that's going to leave us with insane amounts of documentation sets. But we could have a /docs/alpha/ which would hold the latest released alpha. Yes, that's perfect. For that matter, for the release notes, I wasn't planning to have a seperate set for every alpha. I was thinking of just creating a *cumulative* set, with the new items for the current alpha flagged somehow. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pg_dump vs PQescapeBytea
I mentioned in http://archives.postgresql.org/message-id/21837.1248215...@sss.pgh.pa.us that pg_dump has a bug in its handling of large objects, which is directly related to the fact that it's still using the deprecated function PQescapeBytea. The reason PQescapeBytea is deprecated is that it has to guess at the setting of standard_conforming_strings, and it is not guessing right in this one case. I speculated about fixing that by inventing a PQescapeByteaHex function, but that was pure brain fade :-(. The hex format for bytea still requires one backslash in the value, so you still have to know standard_conforming_strings to get it right. AFAICS the only way to make this work correctly and still have libpq do the work is to create a libpq function that takes an explicit parameter for the value of standard_conforming_strings to assume. (We can't pass a PQconn because we don't have one.) That seems a tad ugly, so I'm thinking we might just as well have pg_dump solve this problem for itself. Another issue related to this code is that ideally we'd like it to use hex format for speed reasons. The problem there is that when writing text output in pg_dump or pg_restore, we don't know for sure what server version the output will be loaded into. If we use hex format then it will be impossible to load the output into a pre-8.5 server. This wouldn't be the first time we've broken backwards compatibility of dump output, of course, but it's something that would be entirely impractical to fix by manual editing of the dump output. So it seems a bit worse than our average breakage. If we aren't worried about backwards compatibility then the solution is easy: have pg_dump emit LO data in the form E'\\x ... hex data here ...' which is cheap to generate and will work regardless of GUC settings. If we want it to generate backwards-compatible output then we'll essentially end up duplicating libpq's PQescapeBytea code in pg_dump. Thoughts, better ideas? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha Releases: Docs?
On Tue, Aug 4, 2009 at 1:19 PM, Peter Eisentrautpete...@gmx.net wrote: On Tuesday 04 August 2009 18:52:06 Robert Haas wrote: I'm willing to help if these are 8.5 release notes in process. I'm not willing to help if they are alpha release notes that will be thrown away afterwards. Which is it? I was working on the latter assumption. I have some reservations about the former approach. It would basically commit us right now to having a consistent set of volunteers available every two months within specific 1-2 day spans. Which is the sort of thing I wanted to avoid. But if we have that commitment, then go for it. Yeah, the timing of the alpha releases could make this tricky. I'm not sure if there's a way to work around that, or if we should just give up and keep doing it the way that we have in the past. I would like to think that the timing wouldn't need to be quite as tight as what you are supposing. For example, is it really too early to start working on the release notes for alpha1 now, or say at the end of the week? If work were started on 8/7, there would still be 8 days left before official end-of-CommitFest, and presumably the release of alpha wouldn't be until a few days after that, yet we'd have a pretty good idea what was going to be in there. Also, to be quite frank, I don't care that much about alpha releases. What I care about is production releases. Helping with the release notes for alpha is a means to an end. So it seems to me that, if necessary, we could do something quick-and-dirty to get out the door, and then we can work on patching them up over time. So the release notes for features added in the latest alpha might be a little rougher than the ones for earlier alphas (but maybe still better than, here's the CVS log, have fun). What I would like to avoid is a situation where we're basically ready to go with beta and Bruce says, Hold on, everybody, it's going to take another two weeks while I plow through 600 commit messages. I have a theory that that work can be spread out and much of it done in advance and not necessarily by Bruce. However, that theory has yet to be tested, and the committers (principally Tom and Bruce) have to be open to it for it to have any chance of success. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Alvaro Herrera alvhe...@commandprompt.com writes: First we need several new error message fields: table name, function name, constraint name, and so on. It would also help to have clear definitions of what these *mean*, which is entirely unclear from your comments --- in particular, the reference to errcontext callbacks confuses the heck out of me. I would have thought that these would be used for the referenced object name in cases like table not found, and surely using an errcontext callback for that would be the hardest possible way to implement it. ... would be to give each new field its own start letter (see http://www.postgresql.org/docs/8.4/static/protocol-error-fields.html); say T for table, f for function (F is taken), c for constraint (C is taken), and so on. Another possibility would be to use a single letter, say N, and add a subtype to it; so table name would be NT followed by the table name, NF for functions, etc. Without a pretty concrete list of what the additions are going to be, it's difficult to make any reasoned choices there. Lastly, I'm not as sure as you are that the case for these is well made. In exactly what cases would client code be able to do something useful with them? Your proposal involves a pretty huge amount of work if we are to carry it out thoroughly, and I'm 100% not convinced that there's a proportional benefit. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
2009/8/4 Bernd Helmle maili...@oopsware.de: --On Montag, August 03, 2009 23:43:08 +0200 Bernd Helmle maili...@oopsware.de wrote: Status Report: I will finish documentation and review tomorrow and will mark this patch for committer review. Here's my latest reviewed version of Pavel's patch with adjusted documentation per latest discussion. While poking a little bit with simplify_function() I realized that this patch changes the behavior of VARIADIC functions a little bit. For example: CREATE OR REPLACE FUNCTION my_test(a IN text, txt VARIADIC text[]) RETURNS text AS $$ SELECT $2[1]; $$ LANGUAGE SQL; The following doesn't work in current 8.4: SELECT my_test('abcd', ARRAY['test', 'foo']); You need to use the VARIADIC keyword to match the second argument to text[]. However, if you are going to use named notation with the patch applied, the picture changes in HEAD: SELECT my_test('abcd' AS a, ARRAY['test', 'foo'] AS txt); my_test - test (1 row) This applies also when you reverse the argument order. I don't know wether this is intended, but its conflicting with what we have currently in the docs. It's also not clear to me wether we want this at all. Named notation has different algorithm for function detection then positional notation. There are not exist variadic parameters (because these parameters hasn't individual names). So only packed variadic parameter should be there, and this parameter have to be named - so keyword VARIADIC is optional. Thank You very much Pavel -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: First we need several new error message fields: table name, function name, constraint name, and so on. It would also help to have clear definitions of what these *mean*, which is entirely unclear from your comments --- in particular, the reference to errcontext callbacks confuses the heck out of me. I would have thought that these would be used for the referenced object name in cases like table not found, and surely using an errcontext callback for that would be the hardest possible way to implement it. Huh, yeah, certainly there are cases where the errtable() call is going to be directly in the ereport() call instead of errcontext, but in some other cases (for example when reporting problems in functions) we're going to need errcontext. Lastly, I'm not as sure as you are that the case for these is well made. In exactly what cases would client code be able to do something useful with them? Your proposal involves a pretty huge amount of work if we are to carry it out thoroughly, and I'm 100% not convinced that there's a proportional benefit. Hmm, well, I skipped the rationale because it has been requested before. For example, we need to give constraint names so that applications can tell which unique key is being violated. We need table names on which they are being violated. We need column names for datatype mismatches, and so on. We frequently see people parsing the error message to extract those, but that is known to be fragile, cumbersome and error prone. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: Lastly, I'm not as sure as you are that the case for these is well made. In exactly what cases would client code be able to do something useful with them? Your proposal involves a pretty huge amount of work if we are to carry it out thoroughly, and I'm 100% not convinced that there's a proportional benefit. Hmm, well, I skipped the rationale because it has been requested before. For example, we need to give constraint names so that applications can tell which unique key is being violated. We need table names on which they are being violated. We need column names for datatype mismatches, and so on. We frequently see people parsing the error message to extract those, but that is known to be fragile, cumbersome and error prone. Frankly, I don't believe it. I've seen possibly one or two requests for such things. That's not enough interest to justify the kind of work and code-size investment you're talking about. If there are situations where this info is missing from the human-readable message, then sure, let's look into fixing that. But the use-case for automatic interpretation of the message is just a whole lot smaller than would justify the work. To take just one point, I rather doubt that SQLSTATE codes are really sufficiently fine-grained to let applications automatically determine what to do without looking at the message text. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
2009/8/4 Tom Lane t...@sss.pgh.pa.us: Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: Lastly, I'm not as sure as you are that the case for these is well made. In exactly what cases would client code be able to do something useful with them? Your proposal involves a pretty huge amount of work if we are to carry it out thoroughly, and I'm 100% not convinced that there's a proportional benefit. Hmm, well, I skipped the rationale because it has been requested before. For example, we need to give constraint names so that applications can tell which unique key is being violated. We need table names on which they are being violated. We need column names for datatype mismatches, and so on. We frequently see people parsing the error message to extract those, but that is known to be fragile, cumbersome and error prone. Frankly, I don't believe it. I've seen possibly one or two requests for such things. That's not enough interest to justify the kind of work and code-size investment you're talking about. If there are situations where this info is missing from the human-readable message, then sure, let's look into fixing that. But the use-case for automatic interpretation of the message is just a whole lot smaller than would justify the work. To take just one point, I rather doubt that SQLSTATE codes are really sufficiently fine-grained to let applications automatically determine what to do without looking at the message text. I can see sense of special err attrib for constraints, table and columns. This should to help with error procession on application level. This is language independent and +/- more stable than error messages. regards Pavel Stehule regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: It would also help to have clear definitions of what these *mean*, which is entirely unclear from your comments --- in particular, the reference to errcontext callbacks confuses the heck out of me. I would have thought that these would be used for the referenced object name in cases like table not found, and surely using an errcontext callback for that would be the hardest possible way to implement it. Huh, yeah, certainly there are cases where the errtable() call is going to be directly in the ereport() call instead of errcontext, but in some other cases (for example when reporting problems in functions) we're going to need errcontext. Also, you completely dodged the question of defining what the fields really mean, which would be 100% essential to doing anything automatic with the results. If errtable sometimes means a table that doesn't exist, and sometimes means a table that exists but doesn't contain an expected column, or sometimes a table that exists but doesn't contain an expected value, or sometimes a table that exists and contains a value that shouldn't be there, etc etc, then actually doing anything interesting with the information is going to be a matter of guess and hope rather than something that's reliably automatable. As a single example, in a foreign-key error message, is errtable going to be the referencing table or the referenced table? And how would an application writer predict that? If you don't have a clear design rule that tells the answer, there is absolutely no chance of achieving any consistency among the hundreds or thousands of ereports that will have to be consistent in order for this feature to be worth anything. Let alone whatever random effects an errcontext callback would produce; I don't think you've thought through that bit at all. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_dump vs PQescapeBytea
On Tue, 2009-08-04 at 13:35 -0400, Tom Lane wrote: I speculated about fixing that by inventing a PQescapeByteaHex function, but that was pure brain fade :-(. The hex format for bytea still requires one backslash in the value, so you still have to know standard_conforming_strings to get it right. On a related topic: right now it's impossible to safely escape (using libpq alone) bytea data for use as a text-format argument to a parameterized query. This is because PQescapeBytea returns a string that is both escaped for bytea and escaped for inclusion in a SQL string (am I wrong here?). It's a pretty minor issue, but a little inconsistent. Is it worth addressing? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 4, 2009 at 4:05 PM, Tom Lanet...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: Lastly, I'm not as sure as you are that the case for these is well made. In exactly what cases would client code be able to do something useful with them? Your proposal involves a pretty huge amount of work if we are to carry it out thoroughly, and I'm 100% not convinced that there's a proportional benefit. Hmm, well, I skipped the rationale because it has been requested before. For example, we need to give constraint names so that applications can tell which unique key is being violated. We need table names on which they are being violated. We need column names for datatype mismatches, and so on. We frequently see people parsing the error message to extract those, but that is known to be fragile, cumbersome and error prone. Frankly, I don't believe it. I've seen possibly one or two requests for such things. That's not enough interest to justify the kind of work and code-size investment you're talking about. If there are situations where this info is missing from the human-readable message, then sure, let's look into fixing that. But the use-case for automatic interpretation of the message is just a whole lot smaller than would justify the work. To take just one point, I rather doubt that SQLSTATE codes are really sufficiently fine-grained to let applications automatically determine what to do without looking at the message text. I'm not sure whether the code complexity is justified because I don't know how much code complexity is involved. But as a guy who writes a lot of web applications, I do know that the need to parse error messages is frequent. I tend to make it the sole responsibility of the database to report things like foreign key violations. For example, a user tries to delete an object via a web interface and it fails because of a foreign key violation. I need to tell him (in English) which foreign key constraint he violated, with a message something like this: This object-type may not be deleted because it is still being used by one or more other-object-type-in-plural-form. Right now, I do this like this: if ($err =~ /name_of_first_foreign_key/) { $r-error_exit('First error message.') } elsif ($err =~ /name_of_second_foreign_key/) { ... I've always found that a bit icky because it relies (for example) on their not being two constraint names such that the first is a prefix of the second, and on there not being any other data in the error string which can be confused with the constraint name. But I can't say I've ever had any serious problems as a result of this ickiness... I just make sure my constraint names are long and wordy enough to prevent confusion. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_dump vs PQescapeBytea
Jeff Davis pg...@j-davis.com writes: On a related topic: right now it's impossible to safely escape (using libpq alone) bytea data for use as a text-format argument to a parameterized query. This is because PQescapeBytea returns a string that is both escaped for bytea and escaped for inclusion in a SQL string (am I wrong here?). True, but I would think that most people would prefer to use binary format and not do any conversion/escaping at all in that situation. I doubt it's worth producing still another variant of PQescapeBytea for the people who insist on a text conversion for that case. (The hex mode already made their lives a lot easier, anyway.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Tom Lane wrote: If there are situations where this info is missing from the human-readable message, then sure, let's look into fixing that. But the use-case for automatic interpretation of the message is just a whole lot smaller than would justify the work. To take just one point, I rather doubt that SQLSTATE codes are really sufficiently fine-grained to let applications automatically determine what to do without looking at the message text. The message text is unparsable, because sometimes it is translated. If the application wants to do something particular upon receiving a particular error message, it has two choices: either it parses the string, in which case it better not be translated, or it passes it to the user untouched, in which cases it better be translated. There's no way the application can try to do both things at once. For example, say an application needs to capture a certain class of errors to take automatic action, but pass others to the user untouched for display (unexpected condition). This is not all that uncommon. However, there's no way to do this at present. What I've done is check the SQLSTATE, but then I can't get the affected table name. (Normally the application knows which is the affected table, but this is not always the case.) I don't know about finegrainedness of SQLSTATEs but I haven't had problems with that. ERRCODE_UNIQUE_VIOLATION appears twice in the backend code; ERRCODE_CHECK_VIOLATION appears four times; NOT_NULL_VIOLATION appears five times, as does ERRCODE_FOREIGN_KEY_VIOLATION. Lastly, I wonder if this could be useful to GUI tool writers? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Executor Material
Hi all, Does someone has some material which explain how the executor works? I'm looking for the internal processing of a query in the _executor_. Thanks for help Edson Ramiro
Re: [HACKERS] Executor Material
I found these docs to very helpful for understand how the backend works, but for the executor specifically, I think following the code is best. http://anoncvs.postgresql.org/cvsweb.cgi/~checkout~/pgsql/src/tools/backend/index.html -- --Dan On Tue, Aug 04, 2009 at 05:35:04PM -0300, Edson Ramiro wrote: Hi all, Does someone has some material which explain how the executor works? I'm looking for the internal processing of a query in the _executor_. Thanks for help Edson Ramiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Executor Material
Edson Ramiro erlfi...@gmail.com writes: Does someone has some material which explain how the executor works? Did you read http://developer.postgresql.org/pgdocs/postgres/overview.html and src/backend/executor/README? Once you get through those, reading the source code is the next step. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] head contrib is broken (crypto)
It seems that last contrib crypto changes broke buildfarm. You can see problem e.g. here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_mothdt=2009-08-04%2020:06:01 I don't have time to look on it now. But it should be reproducible everywhere. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tuesday 04 August 2009 23:19:24 Tom Lane wrote: Also, you completely dodged the question of defining what the fields really mean, which would be 100% essential to doing anything automatic with the results. If errtable sometimes means a table that doesn't exist, and sometimes means a table that exists but doesn't contain an expected column, or sometimes a table that exists but doesn't contain an expected value, or sometimes a table that exists and contains a value that shouldn't be there, etc etc, then actually doing anything interesting with the information is going to be a matter of guess and hope rather than something that's reliably automatable. The SQL standard contains an analogous facility that defines exactly that. Look for get diagnostics statement. It specifies what the table name etc. is in specific error situations. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Robert Haas robertmh...@gmail.com writes: Right now, I do this like this: if ($err =~ /name_of_first_foreign_key/) { $r-error_exit('First error message.') } elsif ($err =~ /name_of_second_foreign_key/) { ... I've always found that a bit icky because it relies (for example) on their not being two constraint names such that the first is a prefix of the second, and on there not being any other data in the error string which can be confused with the constraint name. As for the prefix bit, doesn't perl have a word boundary regexp constraint? However, I wonder whether we could turn this around. Instead of an open-ended project to add an ill-defined collection of fields to an ill-defined collection of error cases, maybe we could identify a very short list of cases where it's known to be useful to pull a specific bit of information out of a specific error message. And then implement just those. A minimum requirement for such a thing, in my opinion, is that *every* occurrence of one of the targeted SQLSTATE codes should be able to produce the same auxiliary fields with the same meanings. If you can't define it that way, then you haven't actually made things better than looking at the message text. The bottom line behind my complaining is that this isn't going to be helpful unless it's very clearly defined which error reports produce what auxiliary fields. The impression I got from Alvaro's comments was that he wanted to decorate everything in sight with anything he could think of, which perhaps is not what he intended. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Tom Lane escribió: However, I wonder whether we could turn this around. Instead of an open-ended project to add an ill-defined collection of fields to an ill-defined collection of error cases, maybe we could identify a very short list of cases where it's known to be useful to pull a specific bit of information out of a specific error message. And then implement just those. Hmm, yeah, it makes sense to look at the problem this way. The bottom line behind my complaining is that this isn't going to be helpful unless it's very clearly defined which error reports produce what auxiliary fields. The impression I got from Alvaro's comments was that he wanted to decorate everything in sight with anything he could think of, which perhaps is not what he intended. Right :-( -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 4, 2009 at 5:23 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Right now, I do this like this: if ($err =~ /name_of_first_foreign_key/) { $r-error_exit('First error message.') } elsif ($err =~ /name_of_second_foreign_key/) { ... I've always found that a bit icky because it relies (for example) on their not being two constraint names such that the first is a prefix of the second, and on there not being any other data in the error string which can be confused with the constraint name. As for the prefix bit, doesn't perl have a word boundary regexp constraint? Yep, and it is in fact useful for these kinds of situations. At least for me (and I obviously don't have to contend with translation issues as Alvaro does, since I have the advantage of speaking the untranslated language) the ickiness of the current situation is manageable. It's not ideal, but it is manageable. However, I wonder whether we could turn this around. Instead of an open-ended project to add an ill-defined collection of fields to an ill-defined collection of error cases, maybe we could identify a very short list of cases where it's known to be useful to pull a specific bit of information out of a specific error message. And then implement just those. A minimum requirement for such a thing, in my opinion, is that *every* occurrence of one of the targeted SQLSTATE codes should be able to produce the same auxiliary fields with the same meanings. If you can't define it that way, then you haven't actually made things better than looking at the message text. The bottom line behind my complaining is that this isn't going to be helpful unless it's very clearly defined which error reports produce what auxiliary fields. The impression I got from Alvaro's comments was that he wanted to decorate everything in sight with anything he could think of, which perhaps is not what he intended. Perhaps not. :-) Maybe a specific list of the places where he'd like to add things would be helpful. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Hmm, well, I skipped the rationale because it has been requested before. For example, we need to give constraint names so that applications can tell which unique key is being violated. We need table names on which they are being violated. We need column names for datatype mismatches, and so on. We frequently see people parsing the error message to extract those, but that is known to be fragile, cumbersome and error prone. If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] head contrib is broken (crypto)
Zdenek Kotala wrote: It seems that last contrib crypto changes broke buildfarm. You can see problem e.g. here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_mothdt=2009-08-04%2020:06:01 I don't have time to look on it now. But it should be reproducible everywhere. It's not crypto changes that's the problem -- it's bytea now being output in hex form (encrypt and decrypt both return bytea): *** /store/postgres/buildfarm/emperor_moth/HEAD/pgsql.10027/contrib/pgcrypto/expected/3des.out Tue Aug 4 22:06:20 2009 --- /store/postgres/buildfarm/emperor_moth/HEAD/pgsql.10027/contrib/pgcrypto/results/3des.out Tue Aug 4 22:13:42 2009 *** *** 37,44 -- decrypt select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des'); decrypt ! - ! foo (1 row) -- iv --- 37,44 -- decrypt select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des'); decrypt ! -- ! \x666f6f (1 row) -- iv -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. +1 to this idea in general, but *please* don't consider the use of XML. If we really need some sort of formatting, let's do CSV. Or YAML. Or JSON. Anything but XML. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200908041806 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkp4sOoACgkQvJuQZxSWSsjItACgsDtcid8Zqx9J2ehIhqbtN1l3 1jMAoNcd/6eJPBkKbdNYx6wbUl3bnxhC =H5yv -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mixed, named notation support
--On 4. August 2009 20:22:05 +0200 Pavel Stehule pavel.steh...@gmail.com wrote: Named notation has different algorithm for function detection then positional notation. There are not exist variadic parameters (because these parameters hasn't individual names). So only packed variadic parameter should be there, and this parameter have to be named - so keyword VARIADIC is optional. I wonder wether it wouldn't better to force positional notation for such functions then. I found it surprising that this works at all, but of course, someone else might enjoy this as a cool feature. To me, it feels strange and confusing that a function declared as VARIADIC suddenly accepts a sloppy argument only because you are using some other calling notation where others enforces you to use an additional keyword to match the function. At least, we need to document that both notations behaves different in this case. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Josh Berkus j...@agliodbs.com writes: If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. Huh? What he was talking about is providing additional error fields, which would presumably be made available via PQresultErrorField in libpq, or its kin in other interfaces, and would be totally invisible to any client that didn't ask for them. I can't imagine any value-add from introducing XML into the problem. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] head contrib is broken (crypto)
Alvaro Herrera alvhe...@commandprompt.com writes: It's not crypto changes that's the problem -- it's bytea now being output in hex form (encrypt and decrypt both return bytea): I was insufficiently careful about checking the non-core regression tests :-(. Mea culpa. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Tom Lane t...@sss.pgh.pa.us wrote: A minimum requirement for such a thing, in my opinion, is that *every* occurrence of one of the targeted SQLSTATE codes should be able to produce the same auxiliary fields with the same meanings. If you can't define it that way, then you haven't actually made things better than looking at the message text. I would hope that SQLSTATE *categorizes* messages rather than uniquely identifying them. If it is being used correctly (as I see it), there could well be different specific messages within the category identified by a SQLSTATE for which different identifiers are useful. I'm not so interested in using this feature, personally; but I am concerned about how the issue might affect our use of SQLSTATE, about which I do care. Many products have a sequence number to identify their messages in addition to using SQLSTATE to classify them. That seems pretty sensible to me. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Filtering dictionaries support and unaccent dictionary
On Sat, Aug 1, 2009 at 12:35 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: Teodor Sigaev wrote: As for the contrib module, I think it could use a lot more function header comments! Also, it would be great if it could be used separately from tsearch, i.e. that it provided a function unaccent(text) returns text that unaccented arbitrary strings (I guess it would use the default tsconfig). Umm? Module provides unaccent(text) and unaccent(regdictionary, text) functions. Sorry, I failed to notice. Looks good. Isn't that function leaking res pointer? Also, I'm curious why you're allocating 2*sizeof(TSLexeme) in unaccent_lexize ... So are we waiting for an updated version of this patch? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SE-PostgreSQL Specifications
Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: My concern is access_control_ is a bit long for prefixes, but ac_ is too short to represent what it is doing. pg_ac_? Still shorter than 'security_', uses the pg_ prefix, which we use in a number of other places, and has 'ac' in it.. I don't see anything wrong with ac_. Short is good, and there isn't any other concept in the PG internals that it would conflict with. If there were, pg_ac_ would surely not help to disambiguate. Works for me. OK, I'll go on with the ac_ prefix. -- OSS Platform Development Division, NEC KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 4, 2009 at 11:28 PM, Tom Lanet...@sss.pgh.pa.us wrote: Huh? What he was talking about is providing additional error fields, which would presumably be made available via PQresultErrorField in libpq, or its kin in other interfaces, and would be totally invisible to any client that didn't ask for them. I can't imagine any value-add from introducing XML into the problem. Well it could add a single field which contained a structured description of the error. That would solve the problem you described for things like foreign key references which require two sets of table and column lists and assigning meanings to them both -- essentially because a single list of fields isn't really rich enough to describe all possible errors. But I wonder if it's really that hard to parse the text errors. We have a pretty formulaic construction for our error messages. And we quote things pretty aggressively and prefixing name with the object type pretty consistently. Fixing any cases where we don't follow the rules and fixing the rules in cases where it's hard to parse seems like at least as good a solution. So an alternate proposal is to add a field in the error message which contains the untranslated string. That would let applications always look at the untranslated string for parsing and always use the translated string for user displays. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. +1 to this idea in general, but *please* don't consider the use of XML. If we really need some sort of formatting, let's do CSV. Or YAML. Or JSON. Anything but XML. +1 on the anything but XML. XML reeks of inner platform effect. http://en.wikipedia.org/wiki/Inner-platform_effect Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
Greg Stark wrote: So an alternate proposal is to add a field in the error message which contains the untranslated string. That would let applications always look at the untranslated string for parsing and always use the translated string for user displays. That's an interesting idea, but you also have to consider other stuff not related to translation, like schemas of the tables in question. Someone requested some time ago to schema-qualify the table name (or a constraint name, I don't recall offhand) in an error message, but this idea was shot down because if that person really wanted that info, what he should be looking at is including extra info in machine-readable format into errors instead of a kluge like that. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
David Fetter wrote: On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote: If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. +1 to this idea in general, I think the train left the station on this issue quite a while ago. The error messages have been like they are now for six releases. I don't have any use for changing the format. Clients can produce XML or JSON or whatever format you like already anyway. The protocol is perfectly defined already. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Use DocBook XSL stylesheets for man page building This switches
Peter Eisentraut wrote: Log Message: --- Use DocBook XSL stylesheets for man page building This switches the man page building process to use the DocBook XSL stylesheet toolchain. The previous targets for Docbook2X are removed. configure has been updated to look for the new tools. The Documentation appendix contains the new build instructions. There are also a few isolated tweaks in the documentation to improve places that came out strangely in the man pages. Excellent work, thanks. I'm wondering if dropping SPI_* manpages is really what we want. Maybe we could add them to man section 3? I know I've wanted to have them a couple of times. Not sure about dblink. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
David Fetter wrote: On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 If that's what we're trying to solve, I don't think that adding some kind of proprietary shorthand coding is a good idea. If we're do to this at all, it should be a connection-based GUC option, and use some standard formal like XML fragments. +1 to this idea in general, but *please* don't consider the use of XML. If we really need some sort of formatting, let's do CSV. Or YAML. Or JSON. Anything but XML. +1 on the anything but XML. XML reeks of inner platform effect. http://en.wikipedia.org/wiki/Inner-platform_effect So, we are just trying to whip into shape explain diagnostics which are in JSON or XML, and now you want us to exclude XML from this one because you don't like it? Can we please try for some consistency? Sorry to break it to you, but there are plenty of people and businesses who want XML. And I certainly don't want to have to master every data representation model out there. XML has far more traction than anything else that's comparable in my experience. The fact that Greg is prepared to suggest CSV, with its obvious serious deficiencies, as being *better* than XML, makes his whole argument highly suspect IMNSHO. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
So, we are just trying to whip into shape explain diagnostics which are in JSON or XML, and now you want us to exclude XML from this one because you don't like it? Can we please try for some consistency? Sorry to break it to you, but there are plenty of people and businesses who want XML. And I certainly don't want to have to master every data representation model out there. XML has far more traction than anything else that's comparable in my experience. The fact that Greg is prepared to suggest CSV, with its obvious serious deficiencies, as being *better* than XML, makes his whole argument highly suspect IMNSHO. From a business perspective, XML is the only viable option for output. Joshua D. Drake -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] DefaultACLs
I looked at this patch a bit. I haven't actually read any of the code yet, just reviewed the on-list discussions and the docs, but I think I can make a few comments at the definitional level. First: there's already been some discussion about the VIEW problem. I understand that the patch adds a GRANT ON VIEW syntax that works like GRANT ON SEQUENCE in that if you say VIEW it insists the object really is a view, but without taking away the existing laxness that allows you to still say GRANT ON TABLE for a view. This seems reasonable in isolation, but it creates a big problem for both this patch and the GRANT ON ALL patch, in that it's unclear how to treat views if they could be considered either tables or views. Unfortunately I think we have no choice but to live with the existing laxness, because (a) to do otherwise will break pretty nearly every pg_dump script, and (b) the SQL standard says we have to accept GRANT ON TABLE view. In fact GRANT ON VIEW is not in the SQL standard and there is no good reason to expect applications will adopt it at all. So my feeling is that adding GRANT ON VIEW is a bad idea. The main argument for doing it seemed to be that the author wanted to be able to grant different default privileges for tables and views, but I'm unconvinced that there's a strong use-case for that. You could very easily have one set of default privileges for all of tables, views, and sequences, simply ignoring any bits that weren't relevant for particular objects. It seems to me that that would handle common cases just fine. For complicated cases not so much, but it's not clear to me that filtering by the object subtype is all that useful for complicated cases anyway. People will want more functionality than that. Which leads into my next item. Second: both this patch and GRANT ON ALL are built on the assumption that the only way to filter/classify objects is by schema membership. Now I don't object to that as an initial implementation restriction, but I don't like hard-wiring it into the syntax. It is very clear to me that we'll want other filter rules in the future --- an immediate example is being able to say that new children of an inheritance parent table should inherit its GRANTs. So to my mind, designing the syntax around ALTER SCHEMA is right out. Maybe we could do something like ALTER DEFAULT PRIVILEGES ON TABLES IN SCHEMA foo GRANT ... where the IN SCHEMA foo part would be subject to generalization later. This also matches up a bit better with the proposed syntax for GRANT ON ALL (which also uses IN SCHEMA foo). Third: speaking of syntax, I don't like the way that this is gratituously different from GRANT/REVOKE. I don't like using ADD/DROP instead of GRANT/REVOKE, nor the unnecessary AND business. I think we should minimize confusion by using something that is spelled as nearly like GRANT/REVOKE as possible. Fourth: the system's existing treatment of default permissions is owner-dependent, that is the implied set of permissions is typically GRANT ALL TO owner (from himself, with grant option). I do not understand how schema-level default ACLs will play nicely with that, except in the special case where the schema owner also owns every contained object. If you copy the schema-level ACL directly to a contained object with a different owner it will definitely be the wrong thing, but if you try to translate the ownership it will confuse people too. And confusion in a security-related behavior is a Bad Thing. Furthermore, having the schema owner able to control the grants made for objects not owned by him is a huge security hole. What I suggest as a way to resolve this last point is that a default ACL should apply only to objects owned by the user who creates/modifies the default ACL. In this view, the question of which schema the objects are in is just an additional filter condition, not the primary determinant of which objects a default ACL applies to. Every user has his own set of default ACLs. (This is another reason not to design the syntax around ALTER SCHEMA.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 4, 2009 at 8:36 PM, Joshua D. Drakej...@commandprompt.com wrote: So, we are just trying to whip into shape explain diagnostics which are in JSON or XML, and now you want us to exclude XML from this one because you don't like it? Can we please try for some consistency? Sorry to break it to you, but there are plenty of people and businesses who want XML. And I certainly don't want to have to master every data representation model out there. XML has far more traction than anything else that's comparable in my experience. The fact that Greg is prepared to suggest CSV, with its obvious serious deficiencies, as being *better* than XML, makes his whole argument highly suspect IMNSHO. From a business perspective, XML is the only viable option for output. Wow, I feel like it's time for a bench-clearing brawl! My serialization format kicks your serialization format's butt! This doesn't have a whole lot to do with the original topic of this thread, which unless I missed something had only to do with extending the FE/BE protocol, but it definitely makes for lively conversation. Anyone want to vote ASN.1 for world domination? Can we set up some kind of cage match between the dueling standards? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] md.c should not call files relations
There's an interesting thread over here http://archives.postgresql.org/pgsql-sql/2009-08/msg00013.php in which someone mistook a filesystem-level permissions problem for a database permissions problem. It wasn't exactly his fault, I think, since the message he was presented with was ERROR: could not create relation test: Permission denied which is not all that obviously different from what you would get for a SQL-permissions violation. I am thinking that this message would be more correct and less confusing if it looked something like ERROR: could not create file 12345/67890: Permission denied ie, when reflecting an OS-level error we should call a file a file and provide its filesystem name, not the name of the table that we were hoping to map to it. This would be more likely to lead the user's mind in the right direction, and he'd need the filesystem pathname for any detailed investigation anyway. This would have the further advantage that we could make all the errors in md.c consistent --- some of them provide filesystem names rather than table names because that's all they have available. Lastly, I'm wondering why someone seems to have removed the double quotes around the filesystem name in some of these messages. Surely that's not per style guide. Comments? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
On Tue, Aug 04, 2009 at 09:16:23PM -0400, Robert Haas wrote: On Tue, Aug 4, 2009 at 8:36 PM, Joshua D. Drakej...@commandprompt.com wrote: So, we are just trying to whip into shape explain diagnostics which are in JSON or XML, and now you want us to exclude XML from this one because you don't like it? Can we please try for some consistency? Sorry to break it to you, but there are plenty of people and businesses who want XML. And I certainly don't want to have to master every data representation model out there. XML has far more traction than anything else that's comparable in my experience. The fact that Greg is prepared to suggest CSV, with its obvious serious deficiencies, as being *better* than XML, makes his whole argument highly suspect IMNSHO. From a business perspective, XML is the only viable option for output. Wow, I feel like it's time for a bench-clearing brawl! My serialization format kicks your serialization format's butt! This doesn't have a whole lot to do with the original topic of this thread, which unless I missed something had only to do with extending the FE/BE protocol, but it definitely makes for lively conversation. Anyone want to vote ASN.1 for world domination? Can we set up some kind of cage match between the dueling standards? I love standards. There are so many to choose from :) Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] the case for machine-readable error fields
David Fetter da...@fetter.org writes: On Tue, Aug 04, 2009 at 09:16:23PM -0400, Robert Haas wrote: My serialization format kicks your serialization format's butt! I love standards. There are so many to choose from :) And the funny thing is that the format we transmit this info to the client in is really the least interesting or difficult part of the whole matter ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Convert stmt back into queryString
I am currently trying to convert an insertstmt back into a const char *queryString, but I can't find an existing function to do this for the life of me. I will write one if none exits, but I figured I ask here first. Unfortunately, nodeToString is not quite right for what I'm doing. Thanks in advance. -- --Dan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Convert stmt back into queryString
Dan Colish d...@unencrypted.org writes: I am currently trying to convert an insertstmt back into a const char *queryString, but I can't find an existing function to do this for the life of me. I will write one if none exits, but I figured I ask here first. Unfortunately, nodeToString is not quite right for what I'm doing. Thanks in advance. Hmm, you mean a Query, or a raw unanalyzed InsertStmt? If the former, ruleutils.c will help. If the latter, be prepared to write a lot of code; there's nothing closer than nodeToString, and even that is pretty incomplete for raw grammar output nodes IIRC. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Convert stmt back into queryString
On Tue, Aug 04, 2009 at 10:00:24PM -0400, Tom Lane wrote: Dan Colish d...@unencrypted.org writes: I am currently trying to convert an insertstmt back into a const char *queryString, but I can't find an existing function to do this for the life of me. I will write one if none exits, but I figured I ask here first. Unfortunately, nodeToString is not quite right for what I'm doing. Thanks in advance. Hmm, you mean a Query, or a raw unanalyzed InsertStmt? If the former, ruleutils.c will help. If the latter, be prepared to write a lot of code; there's nothing closer than nodeToString, and even that is pretty incomplete for raw grammar output nodes IIRC. regards, tom lane In this case, its a raw InsertStmt. I would like to pass this back to parse_analyze, but I need to have a queryString to go with that call, so crafting a function to rate that seems to be the only way, atm. -- --Dan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] async notification patch for dblink
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Tom Lane wrote: Joe Conway m...@joeconway.com writes: Sure, I guess I ought to use the latest-and-greatest. Any other comments before I commit? That be_pid/be_pid hack in the regression test is pretty ugly, and doesn't test anything very useful anyway seeing that it's integer division. Could you do something like be_pid = pg_backend_pid() AS is_self_notify instead, to verify that it's a self-notify? (This is not quite right because you'd need to execute pg_backend_pid() at the remote end, but I'm not awake enough to remember the dblink syntax for that. Maybe it's too complex to be worth it, but I think demonstrating how to check for self-notify would be a useful bit of doco.) Otherwise it looks ok to me. OK, still not pretty, but at least more useful. Last call for comments...I hope ;-) Joe -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iQIcBAEBCAAGBQJKePNeAAoJEDfy90M199hlduYP/2UaEuEXzWnGS1RX4Be+UajS LBrS5uanAEUJcohnpMXCHjZVuLJBhXl8r/2RPXsVt5Fem3oYkxFrK3eCysMZs0jJ F6iWb/Hiyi3bVf8coHjeabm0j/TF5yYsKuE3D0kcxktwKjq9vyGOxkadcAXV/lq1 +awVp5tip1ko3R20zSv+N4dcZp6ewHZ9Fa9AT9BS/3D5vQn6Jq3aUajqZ1DdSw0W Fm5DlAl632xB8dQ9G/nNUYXMf5040OyD7foZEncS30i+diyHEvx5++5peCmlAvBj KmeULbsmGn+brlQjvMftJU5LnOzEkKAHYs90BjR/jHoivJHSBjDNbREm1lJqGU9q DU7E3NXz2/hva+/a+PPoyZjJDj+wvxzBivySB/fPCLpQFATDYHQ+1iMOKWqiGO1q N1I6bEEZTAH1/8drANR224/BVnOs10s+4uUL5L8Me+eWO0gIxyhpHyIg/MkhlJ1a Aopz3p7XEuXWmo4XGEAMVhJ2NpBiNKc69Ihno3kkY6FMo4Hs8rNsP+bdZnpBhugO JB0+EDhSaVf2O7a8zmz3fylP2cNBQTh3gZlbNzYlnYgkhkuTXda2JXjwdnAkd5DK 33vVaJ0swb7MzeBWzP5PDMXKbzVPtz7FvReVVhIZbAJk+oVlwlw+KMRo8z8g0ci8 CgX2i1YwCh0AGuk450rZ =JQmH -END PGP SIGNATURE- Index: contrib/dblink/dblink.c === RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v retrieving revision 1.82 diff -c -r1.82 dblink.c *** contrib/dblink/dblink.c 11 Jun 2009 14:48:50 - 1.82 --- contrib/dblink/dblink.c 4 Aug 2009 13:41:26 - *** *** 1635,1640 --- 1635,1723 PG_RETURN_DATUM(current_query(fcinfo)); } + /* + * Retrieve async notifications for a connection. + * + * Returns an setof record of notifications, or an empty set if none recieved. + * Can optionally take a named connection as parameter, but uses the unnamed connection per default. + * + */ + #define DBLINK_NOTIFY_COLS 3 + + PG_FUNCTION_INFO_V1(dblink_get_notify); + Datum + dblink_get_notify(PG_FUNCTION_ARGS) + { + PGconn *conn = NULL; + remoteConn *rconn = NULL; + PGnotify *notify; + ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo-resultinfo; + TupleDesc tupdesc; + Tuplestorestate *tupstore; + MemoryContext per_query_ctx; + MemoryContext oldcontext; + + DBLINK_INIT; + if (PG_NARGS() == 1) + DBLINK_GET_NAMED_CONN; + else + conn = pconn-conn; + + /* create the tuplestore */ + per_query_ctx = rsinfo-econtext-ecxt_per_query_memory; + oldcontext = MemoryContextSwitchTo(per_query_ctx); + + tupdesc = CreateTemplateTupleDesc(DBLINK_NOTIFY_COLS, false); + TupleDescInitEntry(tupdesc, (AttrNumber) 1, notify_name, + TEXTOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 2, be_pid, + INT4OID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 3, extra, + TEXTOID, -1, 0); + + tupstore = tuplestore_begin_heap(true, false, work_mem); + rsinfo-returnMode = SFRM_Materialize; + rsinfo-setResult = tupstore; + rsinfo-setDesc = tupdesc; + + MemoryContextSwitchTo(oldcontext); + + PQconsumeInput(conn); + while ((notify = PQnotifies(conn)) != NULL) + { + Datum values[DBLINK_NOTIFY_COLS]; + bool nulls[DBLINK_NOTIFY_COLS]; + + memset(values, 0, sizeof(values)); + memset(nulls, 0, sizeof(nulls)); + + if (notify-relname != NULL) + values[0] = CStringGetTextDatum(notify-relname); + else + nulls[0] = true; + + values[1] = Int32GetDatum(notify-be_pid); + + if (notify-extra != NULL) + values[2] = CStringGetTextDatum(notify-extra); + else + nulls[2] = true; + + /* switch to appropriate context while storing the tuple */ + MemoryContextSwitchTo(per_query_ctx); + tuplestore_putvalues(tupstore, tupdesc, values, nulls); + MemoryContextSwitchTo(oldcontext); + + PQfreemem(notify); + PQconsumeInput(conn); + } + + /* clean up and return the tuplestore */ + tuplestore_donestoring(tupstore); + + return (Datum) 0; + } + /* * internal functions */ Index: contrib/dblink/dblink.h === RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v retrieving revision 1.22 diff -c -r1.22 dblink.h *** contrib/dblink/dblink.h 9 Jun 2009 17:41:02 - 1.22 --- contrib/dblink/dblink.h 4 Aug 2009 13:41:26 - *** *** 57,61 --- 57,62 extern Datum