Re: [HACKERS] Patch for 8.5, transformationHook

2009-08-04 Thread Dimitri Fontaine
Hi,

Robert Haas robertmh...@gmail.com writes:
 I don't really believe that JSON is only one use case.  XML and JSON
 are in a class of their own; there's nothing else out there that is
 really comparable.

You might want to hear about the UBF specs from Joe Armstrong, let me
quote its page about it:

  UBF is a language for transporting and describing complex data
  structures across a network. It has three components:

* UBF(A) is a data transport format, roughly equivalent to
  well-formed XML.

* UBF(B) is a programming langauge for describing types in UBF(A)
  and protocols between clients and servers. UBF(B) is roughly
  equivalent to to Verified XML, XML-schemas, SOAP and WDSL.

* UBF(C) is a meta-level protocol between used between UBF servers.

  While the XML series of languages had the goal of having a human
  readable format the UBF languages take the opposite view and provide a
  machine friendly format.

  http://www.sics.se/~joe/ubf/site/home.html

It seems there's an ongoing revision to adapt this work to JSON
nowadays:

  http://armstrongonsoftware.blogspot.com/2009/02/json-protocols-part-1.html

Oh and now I'm wondering about ASN.1...

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Fwd: [BUGS] fix: plpgsql: return query and dropped columns problem

2009-08-04 Thread Pavel Stehule
forward patch to pg_hackers

There is fixed patch. Please, Jaime, can you look on it?

Thank You
Pavel

2009/7/30 Tom Lane t...@sss.pgh.pa.us:
 Jaime Casanova jcasa...@systemguards.com.ec writes:
 On Mon, Jul 20, 2009 at 10:09 AM, Alvaro
 Getting rid of the check on natts was ungood ... it needs to compare
 the number of undropped columns of both tupdescs.

 patch attached

 This patch is *still* introducing more bugs than it fixes.  The reason
 is that it has modified validate_tupdesc_compat to allow the tupdescs to
 be somewhat different, but has fixed only one of the several call sites
 to deal with the consequences of that.  The others will now become crash
 risks if we apply it as-is.

 What I would suggest as a suitable plan for a fix is to modify
 validate_tupdesc_compat so that it returns a flag indicating whether the
 tupdesc compatibility is exact or requires translation.  If translation
 is required, provide another routine that does that --- probably using a
 mapping data structure set up by validate_tupdesc_compat, since in some
 of these cases we'll be processing many tuples.  Then the callers just
 have to know enough to call the tuple-translation function when
 validate_tupdesc_compat tells them to.

 There are a number of other places in the system with similar
 requirements, although none of them seem to be exact matches.
 In particular ExecEvalConvertRowtype() provides a good template for
 efficient translation logic, but it's using column name matching
 rather than positional matching so you couldn't share the setup logic.
 I'm not sure if it's worth moving all this code into the core so that
 it can be shared with other future uses (maybe in other PLs), but it's
 worth considering that.  access/common/heaptuple.c or tupdesc.c might
 be a good place for it if we decide to do that.

 The executor's junkfilter code is pretty nearly related as well, and
 perhaps the Right Thing is to make all of this stuff into applications
 of junkfilters with different setup routines for the different
 requirements.  However the junkfilter code is designed to work with
 tuples that are in TupleTableSlots, which isn't particularly helpful for
 plpgsql's uses, so maybe trying to unify with that code is more trouble
 than it's worth.

 I'm marking this patch as Waiting on Author again, although perhaps
 Returned with Feedback would be better since my suggestions amount
 to wholesale rewrites.

                        regards, tom lane

*** ./src/pl/plpgsql/src/pl_exec.c.orig	2009-07-22 04:31:38.0 +0200
--- ./src/pl/plpgsql/src/pl_exec.c	2009-08-03 21:21:42.196893375 +0200
***
*** 190,196 
  	   Oid reqtype, int32 reqtypmod,
  	   bool isnull);
  static void exec_init_tuple_store(PLpgSQL_execstate *estate);
! static void validate_tupdesc_compat(TupleDesc expected, TupleDesc returned,
  		const char *msg);
  static void exec_set_found(PLpgSQL_execstate *estate, bool state);
  static void plpgsql_create_econtext(PLpgSQL_execstate *estate);
--- 190,200 
  	   Oid reqtype, int32 reqtypmod,
  	   bool isnull);
  static void exec_init_tuple_store(PLpgSQL_execstate *estate);
! static HeapTuple convert_tuple(TupleDesc dest_desc, TupleDesc src_desc, 
! 	int *conversion_map, 
! 	HeapTuple tuple);
! static int *validate_tupdesc_compat(TupleDesc expected, TupleDesc returned,
! 		bool *needs_conversion,
  		const char *msg);
  static void exec_set_found(PLpgSQL_execstate *estate, bool state);
  static void plpgsql_create_econtext(PLpgSQL_execstate *estate);
***
*** 216,221 
--- 220,227 
  	ErrorContextCallback plerrcontext;
  	int			i;
  	int			rc;
+ 	int	*conversion_map = NULL;
+ 	bool	needs_conversion = false;
  
  	/*
  	 * Setup the execution state
***
*** 388,394 
  			{
  case TYPEFUNC_COMPOSITE:
  	/* got the expected result rowtype, now check it */
! 	validate_tupdesc_compat(tupdesc, estate.rettupdesc,
  			returned record type does not match expected record type);
  	break;
  case TYPEFUNC_RECORD:
--- 394,400 
  			{
  case TYPEFUNC_COMPOSITE:
  	/* got the expected result rowtype, now check it */
! 	conversion_map = validate_tupdesc_compat(tupdesc, estate.rettupdesc, needs_conversion,
  			returned record type does not match expected record type);
  	break;
  case TYPEFUNC_RECORD:
***
*** 414,422 
  			 * Copy tuple to upper executor memory, as a tuple Datum. Make
  			 * sure it is labeled with the caller-supplied tuple type.
  			 */
! 			estate.retval =
! PointerGetDatum(SPI_returntuple((HeapTuple) DatumGetPointer(estate.retval),
  tupdesc));
  		}
  		else
  		{
--- 420,437 
  			 * Copy tuple to upper executor memory, as a tuple Datum. Make
  			 * sure it is labeled with the caller-supplied tuple type.
  			 */
! 			if (!needs_conversion)
! estate.retval =
! 	

Re: [HACKERS] SE-PostgreSQL Specifications

2009-08-04 Thread Stephen Frost
* KaiGai Kohei (kai...@ak.jp.nec.com) wrote:
 David Fetter wrote:
  On Mon, Aug 03, 2009 at 11:18:55PM -0400, Stephen Frost wrote:
  Just generally, access control is a great way to describe what's
  actually happening here.  That people conflate access control with
  security has resulted in a number of disasters :P
 
 My concern is access_control_ is a bit long for prefixes,
 but ac_ is too short to represent what it is doing.

pg_ac_?  Still shorter than 'security_', uses the pg_ prefix, which we
use in a number of other places, and has 'ac' in it..

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Tsutomu Yamada
Robert Haas robertmh...@gmail.com wrote:
  On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote:
   On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote:
   Included is a conceptual patch to use intptr_t. Comments are welcome.
  
   After closer inspection, not having a win64 box available, I have my doubts
   whether this patch actually does anything. Foremost, it doesn't touch the
   definition of the Datum type, which ought to be at the core of a change 
   like
   this.
  
   Now I see that you call this a conceptual patch. Perhaps we should wait
   until you have developed it into a complete patch?
  
  Is there any reason to consider this patch any further during this
  CommitFest?  It seems that this is a long way from being ready to go.

I'm sorry for delaying response.

This patch is needed as a base of the fix for Windows x64 in the future.

There are still a lot of corrections necessary for Win x64.
(typedef Datum, shared buffer, %lu messages, headers, build scripts, ...)
We are trying these now, and want to offer the result to the next Commit Fest.

Because we are glad if this pointer patch is confirmed at the early stage,
we submitted patch to this Commit Fest.

Thanks.

-- 
Tsutomu Yamada
SRA OSS, Inc. Japan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] doing %-expansion in plpgsql RAISE USING

2009-08-04 Thread Alvaro Herrera
Hi,

It seems there's no way to do %-expansion in plpgsql when one is using
RAISE USING:

alvherre=# create or replace function f () returns void language plpgsql as $$
begin
  raise using message = 'hello %' || 'world';
  return;
end;
$$;
CREATE FUNCTION
alvherre=# select f();
ERROR:  hello %world


I would like the % to be expanded to some argument, but obviously
there's no way to pass the arguments that it should expand to.  We could
do something like

RAISE USING message = 'hello %st %', args = 1, 'world'

but this is obviously going to be difficult, if not impossible, to
implement in the grammar.  Perhaps 
RAISE USING message = 'brave %st %', args = (1, 'world')

Thoughts?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SE-PostgreSQL Specifications

2009-08-04 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes:
 * KaiGai Kohei (kai...@ak.jp.nec.com) wrote:
 My concern is access_control_ is a bit long for prefixes,
 but ac_ is too short to represent what it is doing.

 pg_ac_?  Still shorter than 'security_', uses the pg_ prefix, which we
 use in a number of other places, and has 'ac' in it..

I don't see anything wrong with ac_.  Short is good, and there isn't
any other concept in the PG internals that it would conflict with.
If there were, pg_ac_ would surely not help to disambiguate.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.

2009-08-04 Thread Tom Lane
Greg Stark gsst...@mit.edu writes:
 On Tue, Aug 4, 2009 at 5:04 AM, Tom Lanet...@postgresql.org wrote:
 Cause pg_proc.probin to be declared as text, not bytea.

 Doesn't this relate to the earlier discussion of whether to re-encode
 filenames and paths?

 What's going to happen if I have filenames which aren't valid encoded
 strings in the server encoding -- say UTF8 filenames but I'm using
 latin1 in the server or vice versa. Will my CREATE FUNCTION command
 end up storing an invalid encoded string? Or re-encode the filename
 and then fail to find the file?

Right at the moment we simply aren't considering any of those cases.
If you'd like to propose and implement a solution, feel free.  I think
the last proposal foundered on the fact that it had no idea what
encoding the filesystem was expecting anyway.

I'll point out though that having probin declared bytea would surely
be antithetical to any attempt to treat shlib filenames in an
encoding-aware fashion.  Declaring it that way implies that it is
*not* storing a character string that has any particular encoding.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] async notification patch for dblink

2009-08-04 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Alvaro Herrera wrote:
 Joe Conway escribió:
 
 OK, how's this look?
 
 Hmm, is it possible to use OUT parameters in the function instead of
 declaring a new type for the result?

Sure, I guess I ought to use the latest-and-greatest. Any other comments
before I commit?

Thanks,

Joe
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iQIcBAEBCAAGBQJKeDzfAAoJEDfy90M199hl3FsP/RYXWVVlItm3jk3hKNPCqTaT
maSwtl0QrFLl1pAc3ZTk16QdERrYFjfxIcxALz2am7OtStmhOz63Y4nlY8L0GHdz
ZJDF7K3r5o2RJ12h7Zucpo3racxp1eJqfi4S61RCeIi+aPxtcJd7dds1588FYACc
cPi6AuZyFfE6lhUjgniqOjzvTMptqWbtmuRZl1m+iyXsPE2FLlMclfugbzFKFXso
cnUI31cuejg9tF1zboatopm/qAcIiCB+U2KVG7tyiI+KdL/bNJlkELRJKODxdCvB
XNHzNBey1ZmbgE+H8gu1bz7DXbTxaffoQunCd4HnfOXOGrJpG+djsSCNW860sSVK
icCvdvq3taHfcpLtIIjpecu9LatTFGkJ12YVIV33gnv/Fgr8pj+84VsxRfqlfNaN
V743KdWtEv19gii3qIa817ZcS4tpBfIxXyt37cuZMdOv3VXR7LejeOoDr4aHGjDq
SCozcQQ+Fh8diU1XMzl3rRtz3Brz1cVYE8Ue8ELcQzzuOvWT+a0cjCBii1SmZdTz
DnpVEzmeOGrnHM8+IOG+h9IFRvNx3RQ2nxs71gcbgDp62fnQqb3m7c/Aj0xWv8xp
mQhY2DNQg0AY1vr71nfMGbYL7xTHFD5fA2Md2l/029PA+qFhJlqnZpWScHwqafcC
nkPH88kPUgy33vuNTJ3A
=U88M
-END PGP SIGNATURE-
Index: contrib/dblink/dblink.c
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v
retrieving revision 1.82
diff -c -r1.82 dblink.c
*** contrib/dblink/dblink.c	11 Jun 2009 14:48:50 -	1.82
--- contrib/dblink/dblink.c	4 Aug 2009 13:41:26 -
***
*** 1635,1640 
--- 1635,1723 
  	PG_RETURN_DATUM(current_query(fcinfo));
  }
  
+ /*
+  * Retrieve async notifications for a connection. 
+  *
+  * Returns an setof record of notifications, or an empty set if none recieved.
+  * Can optionally take a named connection as parameter, but uses the unnamed connection per default.
+  *
+  */
+ #define DBLINK_NOTIFY_COLS		3
+ 
+ PG_FUNCTION_INFO_V1(dblink_get_notify);
+ Datum
+ dblink_get_notify(PG_FUNCTION_ARGS)
+ {
+ 	PGconn			   *conn = NULL;
+ 	remoteConn		   *rconn = NULL;
+ 	PGnotify		   *notify;
+ 	ReturnSetInfo	   *rsinfo = (ReturnSetInfo *) fcinfo-resultinfo;
+ 	TupleDesc			tupdesc;
+ 	Tuplestorestate	   *tupstore;
+ 	MemoryContext		per_query_ctx;
+ 	MemoryContext		oldcontext;
+ 
+ 	DBLINK_INIT;
+ 	if (PG_NARGS() == 1)
+ 		DBLINK_GET_NAMED_CONN;
+ 	else
+ 		conn = pconn-conn;
+ 
+ 	/* create the tuplestore */
+ 	per_query_ctx = rsinfo-econtext-ecxt_per_query_memory;
+ 	oldcontext = MemoryContextSwitchTo(per_query_ctx);
+ 
+ 	tupdesc = CreateTemplateTupleDesc(DBLINK_NOTIFY_COLS, false);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 1, notify_name,
+ 	   TEXTOID, -1, 0);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 2, be_pid,
+ 	   INT4OID, -1, 0);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 3, extra,
+ 	   TEXTOID, -1, 0);
+ 
+ 	tupstore = tuplestore_begin_heap(true, false, work_mem);
+ 	rsinfo-returnMode = SFRM_Materialize;
+ 	rsinfo-setResult = tupstore;
+ 	rsinfo-setDesc = tupdesc;
+ 
+ 	MemoryContextSwitchTo(oldcontext);
+ 
+ 	PQconsumeInput(conn);
+ 	while ((notify = PQnotifies(conn)) != NULL)
+ 	{
+ 		Datum		values[DBLINK_NOTIFY_COLS];
+ 		bool		nulls[DBLINK_NOTIFY_COLS];
+ 
+ 		memset(values, 0, sizeof(values));
+ 		memset(nulls, 0, sizeof(nulls));
+ 
+ 		if (notify-relname != NULL)
+ 			values[0] = CStringGetTextDatum(notify-relname);
+ 		else
+ 			nulls[0] = true;
+ 
+ 		values[1] = Int32GetDatum(notify-be_pid);
+ 
+ 		if (notify-extra != NULL)
+ 			values[2] = CStringGetTextDatum(notify-extra);
+ 		else
+ 			nulls[2] = true;
+ 
+ 		/* switch to appropriate context while storing the tuple */
+ 		MemoryContextSwitchTo(per_query_ctx);
+ 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ 		MemoryContextSwitchTo(oldcontext);
+ 
+ 		PQfreemem(notify);
+ 		PQconsumeInput(conn);
+ 	}
+ 
+ 	/* clean up and return the tuplestore */
+ 	tuplestore_donestoring(tupstore);
+ 
+ 	return (Datum) 0;
+ }
+ 
  /*
   * internal functions
   */
Index: contrib/dblink/dblink.h
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v
retrieving revision 1.22
diff -c -r1.22 dblink.h
*** contrib/dblink/dblink.h	9 Jun 2009 17:41:02 -	1.22
--- contrib/dblink/dblink.h	4 Aug 2009 13:41:26 -
***
*** 57,61 
--- 57,62 
  extern Datum dblink_build_sql_delete(PG_FUNCTION_ARGS);
  extern Datum dblink_build_sql_update(PG_FUNCTION_ARGS);
  extern Datum dblink_current_query(PG_FUNCTION_ARGS);
+ extern Datum dblink_get_notify(PG_FUNCTION_ARGS);
  
  #endif   /* DBLINK_H */
Index: contrib/dblink/dblink.sql.in
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.sql.in,v
retrieving revision 1.18
diff -c -r1.18 dblink.sql.in
*** contrib/dblink/dblink.sql.in	9 Jun 2009 17:41:02 -	1.18
--- 

Re: [HACKERS] doing %-expansion in plpgsql RAISE USING

2009-08-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 It seems there's no way to do %-expansion in plpgsql when one is using
 RAISE USING:

That's intentional.  Just use string concatenation if you need a
run-time-variable message.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] doing %-expansion in plpgsql RAISE USING

2009-08-04 Thread Pavel Stehule
Hello

2009/8/4 Alvaro Herrera alvhe...@commandprompt.com:
 Hi,

 It seems there's no way to do %-expansion in plpgsql when one is using
 RAISE USING:

 alvherre=# create or replace function f () returns void language plpgsql as $$
 begin
  raise using message = 'hello %' || 'world';
  return;
 end;
 $$;
 CREATE FUNCTION
 alvherre=# select f();
 ERROR:  hello %world



parameter is expression, so we could to define operator % like

text % any

% isn't defined for text so this is possible. This operator should be
generally used, not only in RAISE attribs.


 I would like the % to be expanded to some argument, but obviously
 there's no way to pass the arguments that it should expand to.  We could
 do something like

 RAISE USING message = 'hello %st %', args = 1, 'world'


RAISE USING message = 'hello %st %' % (1, 'world')

??

or simple use custom variadic function

RAISE USING message= subst('hello %st %', 1, 'world')

good example for parser hook :)

Pavel

 but this is obviously going to be difficult, if not impossible, to
 implement in the grammar.  Perhaps
 RAISE USING message = 'brave %st %', args = (1, 'world')

 Thoughts?

 --
 Alvaro Herrera                                http://www.CommandPrompt.com/
 The PostgreSQL Company - Command Prompt, Inc.

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SE-PostgreSQL Specifications

2009-08-04 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote:
 Stephen Frost sfr...@snowman.net writes:
  * KaiGai Kohei (kai...@ak.jp.nec.com) wrote:
  My concern is access_control_ is a bit long for prefixes,
  but ac_ is too short to represent what it is doing.
 
  pg_ac_?  Still shorter than 'security_', uses the pg_ prefix, which we
  use in a number of other places, and has 'ac' in it..
 
 I don't see anything wrong with ac_.  Short is good, and there isn't
 any other concept in the PG internals that it would conflict with.
 If there were, pg_ac_ would surely not help to disambiguate.

Works for me.

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Peter Eisentraut
On Monday 03 August 2009 23:08:51 Josh Berkus wrote:
 There's another question for alpha releases: are we going to build docs?

Yes, absolutely.  I'm working on making the documentation build part of the 
tarball build procedure.

   Either for www.postgresql.org, or for PGDATA/docs?

The web team has to figure out whether putting the docs on the web site is 
worthwhile.  We already have the developer docs that are rebuild on every 
check-in.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Peter Eisentraut
On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote:
 Josh Berkus wrote:
  I think we need some kind of docs up, otherwise we'll get little
  actual testing.  As previously discussed, building the docs yourself
  from pure source involves several complicated dependancies which
  aren't available on all platforms.

 Are we going to ship ChangeLog files or some such, giving that we're not
 going to have release notes?

There will still be release notes.  They might not be as polished as the final 
ones, but there will be a list of new things, at least.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_proc.probin should become text?

2009-08-04 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes:
 I agree, so information about patch would be store in text field. But
 I am not sure, if your fix isn't too simply. I haven't plan to compile
 plpgsql to C or to binary code. But could be interesting link postgres
 with some virtual machine like parrot or lua vm, and translate plpgsql
 to p code. It's maybe far future.

 Early future is integration main SQL parser to plpgsql. I am not sure,
 but maybe we will need some persistent cache for store parametrized
 sql queries. I though about store it in probin column.

Well, probin is the wrong place for any such thing anyhow.  Regardless
of datatype issues, the system is clearly built on the assumption that
the value of probin is to be specified *by the user* in CREATE FUNCTION.
If you want a cache for derived information it would need to be a new
column.

(I remain of the opinion that caching such stuff in pg_proc would be
a bad design decision.  Every data structure that goes to disk is
another data structure that you cannot easily change.  There just isn't
enough gain there to justify the maintenance headaches.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Peter Eisentraut
On Tuesday 04 August 2009 14:03:34 Tsutomu Yamada wrote:
 Robert Haas robertmh...@gmail.com wrote:
   On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote:
On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote:
Included is a conceptual patch to use intptr_t. Comments are welcome.
   
After closer inspection, not having a win64 box available, I have my
doubts whether this patch actually does anything. Foremost, it doesn't
touch the definition of the Datum type, which ought to be at the core
of a change like this.
   
Now I see that you call this a conceptual patch. Perhaps we should
wait until you have developed it into a complete patch?
  
   Is there any reason to consider this patch any further during this
   CommitFest?  It seems that this is a long way from being ready to go.

 I'm sorry for delaying response.

 This patch is needed as a base of the fix for Windows x64 in the future.

 There are still a lot of corrections necessary for Win x64.
 (typedef Datum, shared buffer, %lu messages, headers, build scripts, ...)
 We are trying these now, and want to offer the result to the next Commit
 Fest.

 Because we are glad if this pointer patch is confirmed at the early stage,
 we submitted patch to this Commit Fest.

Well, there is nothing outright wrong with this patch, but without any 
measurable effect, it is too early to commit it.  At least I would like to see 
the Datum typedef to be changed to use intptr_t and the fallout from that 
cleaned up.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Peter Eisentraut
On Monday 03 August 2009 22:11:08 Tom Lane wrote:
 I'm starting to look at this patch.  I observe that it's setting the
 default output format to HEX.  If changing the default behavior was
 agreed to, or even discussed, I do not remember where.  Shouldn't the
 default stay the same?

I did pose that question in my patch submission email.

Unless there is overwhelming support in favor of changing, we probably 
shouldn't change it, at least not yet.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] doing %-expansion in plpgsql RAISE USING

2009-08-04 Thread Alvaro Herrera
Tom Lane wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
  It seems there's no way to do %-expansion in plpgsql when one is using
  RAISE USING:
 
 That's intentional.  Just use string concatenation if you need a
 run-time-variable message.

Yes, I can do that, but it's really ugly and gets unmaintainable
quickly.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Magnus Hagander
On Tue, Aug 4, 2009 at 16:10, Peter Eisentrautpete...@gmx.net wrote:
 On Tuesday 04 August 2009 14:03:34 Tsutomu Yamada wrote:
 Robert Haas robertmh...@gmail.com wrote:
   On Fri, Jul 24, 2009 at 4:24 PM, Peter Eisentrautpete...@gmx.net wrote:
    On Friday 26 June 2009 12:07:24 Tsutomu Yamada wrote:
    Included is a conceptual patch to use intptr_t. Comments are welcome.
   
    After closer inspection, not having a win64 box available, I have my
    doubts whether this patch actually does anything. Foremost, it doesn't
    touch the definition of the Datum type, which ought to be at the core
    of a change like this.
   
    Now I see that you call this a conceptual patch. Perhaps we should
    wait until you have developed it into a complete patch?
  
   Is there any reason to consider this patch any further during this
   CommitFest?  It seems that this is a long way from being ready to go.

 I'm sorry for delaying response.

 This patch is needed as a base of the fix for Windows x64 in the future.

 There are still a lot of corrections necessary for Win x64.
 (typedef Datum, shared buffer, %lu messages, headers, build scripts, ...)
 We are trying these now, and want to offer the result to the next Commit
 Fest.

 Because we are glad if this pointer patch is confirmed at the early stage,
 we submitted patch to this Commit Fest.

 Well, there is nothing outright wrong with this patch, but without any
 measurable effect, it is too early to commit it.  At least I would like to see
 the Datum typedef to be changed to use intptr_t and the fallout from that
 cleaned up.

+1.

I think it's good that it was posted for a quick review of the general
idea, but I agree that it's too early to commit it until we can see
some actual benefit. And I expect the Datum changes to be much larger
than this, so we can just review/apply them as one when the time
comes.


-- 
 Magnus Hagander
 Self: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] async notification patch for dblink

2009-08-04 Thread Tom Lane
Joe Conway m...@joeconway.com writes:
 Sure, I guess I ought to use the latest-and-greatest. Any other comments
 before I commit?

That be_pid/be_pid hack in the regression test is pretty ugly, and
doesn't test anything very useful anyway seeing that it's integer
division.  Could you do something like
be_pid = pg_backend_pid() AS is_self_notify
instead, to verify that it's a self-notify?  (This is not quite right
because you'd need to execute pg_backend_pid() at the remote end, but
I'm not awake enough to remember the dblink syntax for that.  Maybe
it's too complex to be worth it, but I think demonstrating how to
check for self-notify would be a useful bit of doco.)

Otherwise it looks ok to me.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha releases: How to tag

2009-08-04 Thread Peter Eisentraut
On Monday 03 August 2009 22:52:55 David Fetter wrote:
 On Mon, Aug 03, 2009 at 09:22:52PM +0300, Peter Eisentraut wrote:
  On Monday 03 August 2009 21:07:00 David Fetter wrote:
   We require that people supply docs with their changes, and it is
   totally unreasonable to let them send in catalog changes which do
   not include need migration changes.  That's how it works in every
   other RDBMS outfit that has changes on disk, and we do not need to
   be the exception.
 
  Well, blocker number one for that is that pg_migrator is not even in
  the PostgreSQL CVS repository, but is more like an endorsed
  third-party product.

 I'm not entirely sure that pg_migrator should be tied to releases of
 PostgreSQL, given what it does.  Or did you mean that it's not been
 given the same scrutiny that the PostgreSQL code base has?

Well, how to you expect to mandate that all patches come with upgrade support 
if the PostgreSQL software proper does not contain any upgrade support in the 
first place, because pg_migrator is maintained elsewhere.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Magnus Hagander
On Tue, Aug 4, 2009 at 16:06, Peter Eisentrautpete...@gmx.net wrote:
 On Monday 03 August 2009 23:08:51 Josh Berkus wrote:
 The web team has to figure out whether putting the docs on the web site is
 worthwhile.  We already have the developer docs that are rebuild on every
 check-in.

As long as the HTML files are available, we can make this happen. We
don't have the website servers set up with the full build tools, and
we don't want that :-) It is probably not going to be a terrible
amount of work, so if we are going to actually push for people to do
serious testing on the alpha releases, we should definitely get the
docs up on the website.

I don't think we should keep an archive of old alphas though -
that's going to leave us with insane amounts of documentation sets.
But we could have a /docs/alpha/ which would hold the latest released
alpha.


-- 
 Magnus Hagander
 Self: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Monday 03 August 2009 22:11:08 Tom Lane wrote:
 I'm starting to look at this patch.  I observe that it's setting the
 default output format to HEX.  If changing the default behavior was
 agreed to, or even discussed, I do not remember where.  Shouldn't the
 default stay the same?

 I did pose that question in my patch submission email.

 Unless there is overwhelming support in favor of changing, we probably 
 shouldn't change it, at least not yet.

While I've been poking at the pg_dump issues, it's occurred to me that
changing the default would be a great forcing function for finding out
any lurking problems.  What I'm inclined to do now is to commit it
*with* the change of default, and let it be that way at least for a
few alpha-test releases.  We can vote on whether to switch the default
back before 8.5 final.

If this seems reasonable, I can make a note of the point in the commit
message, so that we won't forget when the time comes.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 Peter Eisentraut pete...@gmx.net writes:
 On Monday 03 August 2009 22:11:08 Tom Lane wrote:
 I'm starting to look at this patch.  I observe that it's setting the
 default output format to HEX.  If changing the default behavior was
 agreed to, or even discussed, I do not remember where.  Shouldn't the
 default stay the same?

 I did pose that question in my patch submission email.

 Unless there is overwhelming support in favor of changing, we probably
 shouldn't change it, at least not yet.

 While I've been poking at the pg_dump issues, it's occurred to me that
 changing the default would be a great forcing function for finding out
 any lurking problems.  What I'm inclined to do now is to commit it
 *with* the change of default, and let it be that way at least for a
 few alpha-test releases.  We can vote on whether to switch the default
 back before 8.5 final.

 If this seems reasonable, I can make a note of the point in the commit
 message, so that we won't forget when the time comes.

Or, what we could do is start an open items for 8.5 list similar to
the one we made for 8.4.  That worked pretty well, I think.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Magnus Hagander
On Tue, Aug 4, 2009 at 16:31, Robert Haasrobertmh...@gmail.com wrote:
 On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 Peter Eisentraut pete...@gmx.net writes:
 On Monday 03 August 2009 22:11:08 Tom Lane wrote:
 I'm starting to look at this patch.  I observe that it's setting the
 default output format to HEX.  If changing the default behavior was
 agreed to, or even discussed, I do not remember where.  Shouldn't the
 default stay the same?

 I did pose that question in my patch submission email.

 Unless there is overwhelming support in favor of changing, we probably
 shouldn't change it, at least not yet.

 While I've been poking at the pg_dump issues, it's occurred to me that
 changing the default would be a great forcing function for finding out
 any lurking problems.  What I'm inclined to do now is to commit it
 *with* the change of default, and let it be that way at least for a
 few alpha-test releases.  We can vote on whether to switch the default
 back before 8.5 final.

 If this seems reasonable, I can make a note of the point in the commit
 message, so that we won't forget when the time comes.

 Or, what we could do is start an open items for 8.5 list similar to
 the one we made for 8.4.  That worked pretty well, I think.

+1 for that solution, it seems much  better than having to go back
through commit messages. We might as well start it early!


-- 
 Magnus Hagander
 Self: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-04 Thread Sam Mason
On Mon, Aug 03, 2009 at 10:03:47AM -0500, Kevin Grittner wrote:
 That's about 0.52% slower with the patch.  Because there was over 10%
 variation in the numbers with the patch, I tried leaving out the four
 highest outliers on both, in case it was the result of some other
 activity on the system (even though this machine should have been
 pretty quiet over the weekend) and the difference fell to 0.09%.

What do people do when testing this?  I think I'd look to something like
Student's t-test to check for statistical significance.  My working
would go something like:

  I assume the variance is the same because it's being tested on the
  same machine.

  samples = 20
  stddev  = 144.26
  avg1= 4783.13
  avg2= 4758.46
  t   = 0.54  ((avg1 - avg2) / (stddev * sqrt(2/samples)))

We then have to choose how certain we want to be that they're actually
different, 90% is a reasonably easy level to hit (i.e. one part in ten,
with 95% being more commonly quoted).  For 20 samples we have 19 degrees
of freedom--giving us a cut-off[1] of 1.328.  0.54 is obviously well
below this allowing us to say that there's no statistical significance
between the two samples at a 90% level.

-- 
  Sam  http://samason.me.uk/

 [1] 
http://en.wikipedia.org/wiki/Student's_t-distribution#Table_of_selected_values

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Tue, Aug 4, 2009 at 10:28 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 If this seems reasonable, I can make a note of the point in the commit
 message, so that we won't forget when the time comes.

 Or, what we could do is start an open items for 8.5 list similar to
 the one we made for 8.4.  That worked pretty well, I think.

OK.  Historically we haven't made such a list until beta starts, but
there's no reason we couldn't start it early.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.

2009-08-04 Thread Greg Stark
On Tue, Aug 4, 2009 at 2:46 PM, Tom Lanet...@sss.pgh.pa.us wrote:

 I'll point out though that having probin declared bytea would surely
 be antithetical to any attempt to treat shlib filenames in an
 encoding-aware fashion.  Declaring it that way implies that it is
 *not* storing a character string that has any particular encoding.

Well that's kind of the point. Unix filesystems traditionally prohibit
 '/' and '\0' but otherwise allowing any series of bytes without
requiring any particular encoding. If we used bytea to store
filesystem paths then you could specify any arbitrary series of bytes
without worrying that the server will re-encode it differently.

-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_proc.probin should become text?

2009-08-04 Thread Alvaro Herrera
Pavel Stehule escribió:

 I agree, so information about patch would be store in text field. But
 I am not sure, if your fix isn't too simply. I haven't plan to compile
 plpgsql to C or to binary code. But could be interesting link postgres
 with some virtual machine like parrot or lua vm, and translate plpgsql
 to p code. It's maybe far future.

In this case I think it would make more sense to compile the code and
keep the p-code in backend local memory.  Keeping compiled code around
in more permanent or global storage would only make sense if you had
large amounts of code, and in that case I think we'd want to have a more
concrete and complete proposal.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-04 Thread Tom Lane
Sam Mason s...@samason.me.uk writes:
   t   = 0.54  ((avg1 - avg2) / (stddev * sqrt(2/samples)))

 We then have to choose how certain we want to be that they're actually
 different, 90% is a reasonably easy level to hit (i.e. one part in ten,
 with 95% being more commonly quoted).  For 20 samples we have 19 degrees
 of freedom--giving us a cut-off[1] of 1.328.  0.54 is obviously well
 below this allowing us to say that there's no statistical significance
 between the two samples at a 90% level.

Hmm, so what about 95% or 99% confidence?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote:
 Are we going to ship ChangeLog files or some such, giving that we're not
 going to have release notes?

 There will still be release notes.  They might not be as polished as the 
 final 
 ones, but there will be a list of new things, at least.

That would be great, but who's going to do them?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Alvaro Herrera
Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote:
  Are we going to ship ChangeLog files or some such, giving that we're not
  going to have release notes?
 
  There will still be release notes.  They might not be as polished as the 
  final 
  ones, but there will be a list of new things, at least.
 
 That would be great, but who's going to do them?

It's easy, just cvs2cl and paste that in a large pre block.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 On Tue, Aug 4, 2009 at 16:10, Peter Eisentrautpete...@gmx.net wrote:
 Well, there is nothing outright wrong with this patch, but without any
 measurable effect, it is too early to commit it.  At least I would like to 
 see
 the Datum typedef to be changed to use intptr_t and the fallout from that
 cleaned up.

 +1.

 I think it's good that it was posted for a quick review of the general
 idea, but I agree that it's too early to commit it until we can see
 some actual benefit. And I expect the Datum changes to be much larger
 than this, so we can just review/apply them as one when the time
 comes.

The other thing that I would say is a non-negotiable minimum requirement
is that the patch include the necessary configure pushups so it does not
break machines without uintptr_t.  I think we could just do a
conditional

typedef unsigned long uintptr_t;

and proceed from there; then machines without the typedef are no worse
off than before.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_proc.probin should become text?

2009-08-04 Thread Pavel Stehule
2009/8/4 Tom Lane t...@sss.pgh.pa.us:
 Pavel Stehule pavel.steh...@gmail.com writes:
 I agree, so information about patch would be store in text field. But
 I am not sure, if your fix isn't too simply. I haven't plan to compile
 plpgsql to C or to binary code. But could be interesting link postgres
 with some virtual machine like parrot or lua vm, and translate plpgsql
 to p code. It's maybe far future.

 Early future is integration main SQL parser to plpgsql. I am not sure,
 but maybe we will need some persistent cache for store parametrized
 sql queries. I though about store it in probin column.

 Well, probin is the wrong place for any such thing anyhow.  Regardless
 of datatype issues, the system is clearly built on the assumption that
 the value of probin is to be specified *by the user* in CREATE FUNCTION.
 If you want a cache for derived information it would need to be a new
 column.

 (I remain of the opinion that caching such stuff in pg_proc would be
 a bad design decision.  Every data structure that goes to disk is
 another data structure that you cannot easily change.  There just isn't
 enough gain there to justify the maintenance headaches.)

ook, I agree - but I am not sure, so some cache will be necessary.

Pavel



                        regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.

2009-08-04 Thread Andrew Dunstan



Greg Stark wrote:

On Tue, Aug 4, 2009 at 2:46 PM, Tom Lanet...@sss.pgh.pa.us wrote:
  

I'll point out though that having probin declared bytea would surely
be antithetical to any attempt to treat shlib filenames in an
encoding-aware fashion.  Declaring it that way implies that it is
*not* storing a character string that has any particular encoding.



Well that's kind of the point. Unix filesystems traditionally prohibit
 '/' and '\0' but otherwise allowing any series of bytes without
requiring any particular encoding. If we used bytea to store
filesystem paths then you could specify any arbitrary series of bytes
without worrying that the server will re-encode it differently.

  


Is this any different from the path in COPY foo to '/path/to/file'?

I suspect the probin stuff is a solution in search of a problem.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-04 Thread Sam Mason
On Tue, Aug 04, 2009 at 10:45:52AM -0400, Tom Lane wrote:
 Sam Mason s...@samason.me.uk writes:
t   = 0.54  ((avg1 - avg2) / (stddev * sqrt(2/samples)))
 
  We then have to choose how certain we want to be that they're actually
  different, 90% is a reasonably easy level to hit (i.e. one part in ten,
  with 95% being more commonly quoted).  For 20 samples we have 19 degrees
  of freedom--giving us a cut-off[1] of 1.328.  0.54 is obviously well
  below this allowing us to say that there's no statistical significance
  between the two samples at a 90% level.
 
 Hmm, so what about 95% or 99% confidence?

The cut-off goes up to 1.729 for 95% and to 2.539 for 99%.  These values
are only really for a 20 samples with the above calculation, the link I
gave above gives a nice table for different values.

I've also realized that I did the standard deviation all wrong.  I
should have calculated them independently and then got the mean:

  stddev1 = 159.9699
  stddev2 = 129.6466
  stddev  = 144.8083  ((stddev1+stddev2) / 2)

Here it makes absolutely no difference, but when they were really
different distributions it would.

-- 
  Sam  http://samason.me.uk/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Peter Eisentraut
On Tuesday 04 August 2009 17:48:12 Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote:
  Are we going to ship ChangeLog files or some such, giving that we're not
  going to have release notes?
 
  There will still be release notes.  They might not be as polished as the
  final ones, but there will be a list of new things, at least.

 That would be great, but who's going to do them?

I can do it. Perhaps Josh wants to get involved.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Peter Eisentraut
On Tuesday 04 August 2009 17:56:41 Tom Lane wrote:
 The other thing that I would say is a non-negotiable minimum requirement
 is that the patch include the necessary configure pushups so it does not
 break machines without uintptr_t.

There is AC_TYPE_UINTPTR_T, so that should be easy.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 Peter Eisentraut pete...@gmx.net writes:
 On Monday 03 August 2009 23:13:27 Alvaro Herrera wrote:
 Are we going to ship ChangeLog files or some such, giving that we're not
 going to have release notes?

 There will still be release notes.  They might not be as polished as the 
 final
 ones, but there will be a list of new things, at least.

 That would be great, but who's going to do them?

I think the question is who are you going to allow to do them?.  I
sort of get the impression that you and Bruce have a love-hate
relationship with the release notes.  On the one hand, they're a huge
time sink.  On the other hand, it feels like you're not quite ready to
turn them over to anyone else.  I feel like there have been some
previous offers of help (including by me) that either weren't an offer
to do quite the right thing or the offer was made at the wrong stage
of the process.  I believe you told me on one occasion that they had
to all be written by a single person so that they'd be consistent...
but then, even though Bruce did the initial draft of the release
notes, you did most of the updating as we got closer to release and
more things were committed, so apparently it wasn't that critical for
one person to do it all after all.

For alpha, I think a dump of the CVS commit logs is fine if there's
not a better option.  That's what Peter originally proposed, and
there's no reason to change now.  But I suspect that it might be
possible to find someone who would be willing to do more than that if
they were treated nicely.  For example, if someone volunteers to help
with this now, will their work be thrown out and redone by you and
Bruce the right way when the actual release arrives?  Or will we
just revise and extend that work throughout the release cycle?

If you want to be rid of this task, then help some other people to do
it and do it well.  If you want to keep doing it yourself, then dump
the CVS logs into the alpha release notes and call it good.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Cause pg_proc.probin to be declared as text, not bytea.

2009-08-04 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 Is this any different from the path in COPY foo to '/path/to/file'?
 I suspect the probin stuff is a solution in search of a problem.

Well, the previous probin behavior is demonstrably broken.  Make a shlib
with backslash or non-ASCII in the name, create a function referencing
it, dump and reload.  Whatever your opinions are about encodings, you
won't think pg_dump did the right thing.

I'm not sure whether the more general pathname encoding issue is worth
working on or not.  In general it's a non-problem if the paths in the
server filesystem are written in the database encoding.  If they are
not, then you have to figure out what they *are* written in, and that
seems a bit tough.  But anyway that problem is hardly restricted to
probin, and a solution that works only for probin doesn't seem terribly
interesting.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Proposal: More portable way to support 64bit platforms

2009-08-04 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On Tuesday 04 August 2009 17:56:41 Tom Lane wrote:
 The other thing that I would say is a non-negotiable minimum requirement
 is that the patch include the necessary configure pushups so it does not
 break machines without uintptr_t.

 There is AC_TYPE_UINTPTR_T, so that should be easy.

It's certainly do-able, the point is just to do it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 That would be great, but who's going to do them?

 I think the question is who are you going to allow to do them?.

You misread that entirely.  What I was pointing out was that we didn't
have a volunteer to expend the nontrivial amount of time required to
do nice notes.  If you're volunteering, step right up.

(If cvs2cl dump is the best we can do, so be it; but I predict it
will put off a lot of potential testers.  The commit messages have
always been written by hackers for hackers, not for end users.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 11:44 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Tue, Aug 4, 2009 at 10:48 AM, Tom Lanet...@sss.pgh.pa.us wrote:
 That would be great, but who's going to do them?

 I think the question is who are you going to allow to do them?.

 You misread that entirely.  What I was pointing out was that we didn't
 have a volunteer to expend the nontrivial amount of time required to
 do nice notes.  If you're volunteering, step right up.

I'm willing to help if these are 8.5 release notes in process.  I'm
not willing to help if they are alpha release notes that will be
thrown away afterwards.  Which is it?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 I'm willing to help if these are 8.5 release notes in process.  I'm
 not willing to help if they are alpha release notes that will be
 thrown away afterwards.  Which is it?

That depends largely on what they look like when we get to beta,
I imagine.  Are you asking for a guarantee that no one will edit
your deathless prose?

Traditionally one of the main time sinks involved in making the release
notes has been trying to give them a uniform voice, categorizing them
sensibly, weighting the space given to different topics in a way that
seems to make sense in hindsight, etc.  I'd be rather surprised if notes
prepared piecemeal over a series of alpha releases didn't need work of
that sort when we get to the end.  That doesn't mean the work would be
thrown away, but it does mean it's likely to get edited.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mixed, named notation support

2009-08-04 Thread Bernd Helmle
--On Montag, August 03, 2009 23:43:08 +0200 Bernd Helmle 
maili...@oopsware.de wrote:



Status Report: I will finish documentation and review tomorrow and will
mark this patch for committer review.


Here's my latest reviewed version of Pavel's patch with adjusted 
documentation per latest discussion.


While poking a little bit with simplify_function() I realized that this 
patch changes the behavior of VARIADIC functions a little bit. For example:


CREATE OR REPLACE FUNCTION my_test(a IN text, txt VARIADIC text[])
RETURNS text
AS
$$
  SELECT $2[1];
$$ LANGUAGE SQL;

The following doesn't work in current 8.4:

SELECT my_test('abcd', ARRAY['test', 'foo']);

You need to use the VARIADIC keyword to match the second argument to 
text[]. However, if you are going to use named notation with the patch 
applied, the picture changes in HEAD:


SELECT my_test('abcd' AS a, ARRAY['test', 'foo'] AS txt);
my_test
-
test
(1 row)

This applies also when you reverse the argument order. I don't know wether 
this is intended, but its conflicting with what we have currently in the 
docs. It's also not clear to me wether we want this at all.


--
 Thanks

   Bernd

named_and_mixed_notation_review3.patch.gz
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Bernd Helmle
--On Dienstag, August 04, 2009 10:28:48 -0400 Tom Lane t...@sss.pgh.pa.us 
wrote:



While I've been poking at the pg_dump issues, it's occurred to me that
changing the default would be a great forcing function for finding out
any lurking problems.


+1

--
 Thanks

   Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Tom Lane
Bernd Helmle maili...@oopsware.de writes:
 --On Samstag, Juli 11, 2009 13:40:44 +0300 Peter Eisentraut 
 pete...@gmx.net wrote:
 OK, here is an updated patch.  It has the setting as enum, completed
 documentation, and libpq support.  I'll add it to the commit fest in the
 hope  that someone else can look it over in detail.

 I've attached a slightly edited patch which fixes a compiler warning in 
 encode.c, too.

Committed with assorted corrections.  I have not done anything about
the issues mentioned in
http://archives.postgresql.org/message-id/21837.1248215...@sss.pgh.pa.us
mainly that pg_dump's treatment of large-object contents is not safe
against changes of standard_conforming_strings.  I think that ought to
get dealt with before moving on.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 12:01 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 I'm willing to help if these are 8.5 release notes in process.  I'm
 not willing to help if they are alpha release notes that will be
 thrown away afterwards.  Which is it?

 That depends largely on what they look like when we get to beta,
 I imagine.  Are you asking for a guarantee that no one will edit
 your deathless prose?

Not at all.  Don't misunderstand me here: what I was worried about
(and maybe it's without any justification) is that you and/or Bruce
have ideas about how this has to be done that are so specific that
it's not even worth anyone else making an attempt.  If that were the
case, then I wouldn't be very interested in working on it.

Now, on the other hand, if doing some work now incrementally over the
next couple of alpha releases will not only produce good release notes
for those alpha releases but also streamline the process of putting
together release notes for beta/final, then it sounds like a
worthwhile investment of time.  In the 8.4 release cycle, this was one
of the things that held us up a bit (not a huge amount, but a bit):
all the work was done at the end, and there was a lot to do.  If we
can make a start on this early, and if the start is good enough that
it requires incremental changes rather than rm -f then that seems
like a pretty good idea.

 Traditionally one of the main time sinks involved in making the release
 notes has been trying to give them a uniform voice, categorizing them
 sensibly, weighting the space given to different topics in a way that
 seems to make sense in hindsight, etc.  I'd be rather surprised if notes
 prepared piecemeal over a series of alpha releases didn't need work of
 that sort when we get to the end.  That doesn't mean the work would be
 thrown away, but it does mean it's likely to get edited.

OK, great.  That sounds like exactly the sort of editing that I would
expect to be necessary.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] bytea vs. pg_dump

2009-08-04 Thread Tom Lane
I wrote:
 While I've been poking at the pg_dump issues, it's occurred to me that
 changing the default would be a great forcing function for finding out
 any lurking problems.  What I'm inclined to do now is to commit it
 *with* the change of default, and let it be that way at least for a
 few alpha-test releases.  We can vote on whether to switch the default
 back before 8.5 final.

For the archives, attached is a patch to switch the default and then
make pg_dump force use of hex mode.  This is just so we won't forget
what needs changing if we decide to switch the default back ...

regards, tom lane

*** doc/src/sgml/config.sgml.orig   Tue Aug  4 12:08:35 2009
--- doc/src/sgml/config.sgmlTue Aug  4 12:40:34 2009
***
*** 4068,4078 
listitem
 para
  Sets the output format for values of type typebytea/type.
! Valid values are literalhex/literal (the default)
  and literalescape/literal (the traditional PostgreSQL
  format).  See xref linkend=datatype-binary for more
  information.  The typebytea/type type always
  accepts both formats on input, regardless of this setting.
 /para
/listitem
   /varlistentry
--- 4068,4079 
listitem
 para
  Sets the output format for values of type typebytea/type.
! Valid values are literalhex/literal
  and literalescape/literal (the traditional PostgreSQL
  format).  See xref linkend=datatype-binary for more
  information.  The typebytea/type type always
  accepts both formats on input, regardless of this setting.
+ The default is literalescape/literal.
 /para
/listitem
   /varlistentry
*** doc/src/sgml/datatype.sgml.orig Tue Aug  4 12:08:35 2009
--- doc/src/sgml/datatype.sgml  Tue Aug  4 12:41:03 2009
***
*** 1196,1202 
  quoteescape/quote format, and quotehex/quote format.  Both
  of these are always accepted on input.  The output format depends
  on the configuration parameter xref linkend=guc-bytea-output;
! the default is hex.  (Note that the hex format was introduced in
  productnamePostgreSQL/productname 8.5; earlier versions and some
  tools don't understand it.)
 /para
--- 1196,1202 
  quoteescape/quote format, and quotehex/quote format.  Both
  of these are always accepted on input.  The output format depends
  on the configuration parameter xref linkend=guc-bytea-output;
! the default is escape.  (Note that the hex format was introduced in
  productnamePostgreSQL/productname 8.5; earlier versions and some
  tools don't understand it.)
 /para
*** src/backend/utils/adt/varlena.c.origTue Aug  4 12:08:36 2009
--- src/backend/utils/adt/varlena.c Tue Aug  4 12:42:36 2009
***
*** 30,36 
  
  
  /* GUC variable */
! int   bytea_output = BYTEA_OUTPUT_HEX;
  
  typedef struct varlena unknown;
  
--- 30,36 
  
  
  /* GUC variable */
! int   bytea_output = BYTEA_OUTPUT_ESCAPE;
  
  typedef struct varlena unknown;
  
*** src/backend/utils/misc/guc.c.orig   Tue Aug  4 12:08:36 2009
--- src/backend/utils/misc/guc.cTue Aug  4 12:42:04 2009
***
*** 2553,2559 
NULL
},
bytea_output,
!   BYTEA_OUTPUT_HEX, bytea_output_options, NULL, NULL
},
  
{
--- 2553,2559 
NULL
},
bytea_output,
!   BYTEA_OUTPUT_ESCAPE, bytea_output_options, NULL, NULL
},
  
{
*** src/backend/utils/misc/postgresql.conf.sample.orig  Mon Aug  3 15:59:39 2009
--- src/backend/utils/misc/postgresql.conf.sample   Tue Aug  4 12:42:15 2009
***
*** 424,430 
  #statement_timeout = 0# in milliseconds, 0 is disabled
  #vacuum_freeze_min_age = 5000
  #vacuum_freeze_table_age = 15000
! #bytea_output = 'hex' # hex, escape
  #xmlbinary = 'base64'
  #xmloption = 'content'
  
--- 424,430 
  #statement_timeout = 0# in milliseconds, 0 is disabled
  #vacuum_freeze_min_age = 5000
  #vacuum_freeze_table_age = 15000
! #bytea_output = 'escape'  # hex, escape
  #xmlbinary = 'base64'
  #xmloption = 'content'
  
*** src/bin/pg_dump/pg_dump.c.orig  Tue Aug  4 12:08:36 2009
--- src/bin/pg_dump/pg_dump.c   Tue Aug  4 12:39:39 2009
***
*** 599,604 
--- 599,610 
do_sql_command(g_conn, SET extra_float_digits TO 2);
  
/*
+* If supported, select hex format for bytea, for speed reasons.
+*/
+   if (g_fout-remoteVersion = 80500)
+   do_sql_command(g_conn, SET bytea_output TO hex);
+ 
+   /*
 * If synchronized scanning is supported, disable it, to prevent
 * unpredictable changes in row ordering 

[HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
Hi,

I think the case for machine-readable error fields is well made and
doesn't need any further push.

So, what do we need to make it happen?  Here's my proposal.

First we need several new error message fields: table name, function
name, constraint name, and so on.  One possible way to go about this
would be to give each new field its own start letter (see
http://www.postgresql.org/docs/8.4/static/protocol-error-fields.html);
say T for table, f for function (F is taken), c for constraint (C
is taken), and so on.  Another possibility would be to use a single
letter, say N, and add a subtype to it; so table name would be NT
followed by the table name, NF for functions, etc.

The documentation on the FE/BE protocol already says that frontends
should ignore unrecognized type fields, so I don't think we need to bump
the protocol version for this.

The other part is getting the information in ErrorData.  AFAICS this is
just a matter of adding a few setter functions; say errtable(),
errfunction(), etc.  Furthermore, we can just add those to existing
errcontext callbacks, which would be pretty simple.

Thoughts?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 8.4 win32 shared memory patch

2009-08-04 Thread Kevin Field
 On Sat, Aug 1, 2009 at 20:30, Kevin Fieldkevinjamesfi...@gmail.com 
 wrote:
   The event viewer says:
  
   The description for Event ID ( 0 ) in Source ( PostgreSQL ) 
cannot
   be
   found. The local computer may not have the necessary registry
   information or message DLL files to display messages from a 
remote
   computer. You may be able to use the /AUXSOURCE= flag to 
retrieve
   this
   description; see Help and Support for details. The following
   information is part of the event: pg_ctl: could not find 
postgres
   program executable
  
   And yes, I renamed it correctly...
 
  Check permissions on it. If you moved it at some point, it may 
   have
  the wrong permissions. They should be the same as for the other 
   .EXEs
  in that directory.
 
  The two files (new and old exe) have identical permissions.
 
 That's just weird. It could be that the postgres executable won't 
 work
 - maybe because of some DLL issue. Can you run postgres -V on the
 executable, or does that give you some error?

It reports the version correctly.  Sorry...any other ideas?

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Peter Eisentraut
On Tuesday 04 August 2009 18:52:06 Robert Haas wrote:
 I'm willing to help if these are 8.5 release notes in process.  I'm
 not willing to help if they are alpha release notes that will be
 thrown away afterwards.  Which is it?

I was working on the latter assumption.

I have some reservations about the former approach.  It would basically commit 
us right now to having a consistent set of volunteers available every two 
months within specific 1-2 day spans.  Which is the sort of thing I wanted to 
avoid.  But if we have that commitment, then go for it.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] DefaultACLs

2009-08-04 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote:
 * Robert Haas (robertmh...@gmail.com) wrote:
  On Wed, Jul 22, 2009 at 11:26 PM, Petr Jelinekpjmo...@pjmodos.net wrote:
   The docs are not complete but that's up to Stephen, otherwise the patch
   should be finished. But I am not the reviewer :)
  
  Well, perhaps we had better prod Stephen then, since complete docs are
  a requirement for commit.
 
 I didn't think the docs posted were terrible, but I am working on
 improving them.

Thanks to Joshua, there weren't really many changes I found for the
docs.  Here they are anyway:

grant.sgml:

+ Replace current privileges with the ones specified using 
+ xref linkend=sql-alterschema endterm=sql-alterschema-title.
+ The literalWITH GRANT OPTION/literal parameter is not applicable 
+ because it is copied from default privileges.
+ Note: this can actually emphasis role=boldrevoke/emphasis some 
+ privileges because it clears all existing privileges object has and 
+ replaces them with the default ones for the schema in which this object
+ resides.

How about:

Replaces current privileges with the default privileges, as set using
xref linkend=sql-alterschema endterm=sql-alterschema-title, for
this object type in its current schema.
The literalWITH GRANT OPTION/literal parameter is not applicable
because it is copied from default privileges.
Note: This can emphasis role=boldrevoke/emphasis privileges!  This
will clear all existing privileges first!  If no default privilege has
been set, the object will have all privileges revoked!

Otherwise, I thought the docs looked pretty good.

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Josh Berkus



I have some reservations about the former approach.  It would basically commit
us right now to having a consistent set of volunteers available every two
months within specific 1-2 day spans.  Which is the sort of thing I wanted to
avoid.  But if we have that commitment, then go for it.


Can we hear from Bruce on this, as well?  I think Robert is right to be 
concerned that his work will be thrown away; so am I.


--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] DefaultACLs

2009-08-04 Thread Joshua Tolley
On Tue, Aug 04, 2009 at 01:28:00PM -0400, Stephen Frost wrote:
 Thanks to Joshua, there weren't really many changes I found for the
 docs.  Here they are anyway:

Yay, I was useful! :)

 How about:
 
 Replaces current privileges with the default privileges, as set using
 xref linkend=sql-alterschema endterm=sql-alterschema-title, for
 this object type in its current schema.
 The literalWITH GRANT OPTION/literal parameter is not applicable
 because it is copied from default privileges.
 Note: This can emphasis role=boldrevoke/emphasis privileges!  This
 will clear all existing privileges first!  If no default privilege has
 been set, the object will have all privileges revoked!
 
 Otherwise, I thought the docs looked pretty good.

No complaints here, FWIW.

- Josh


signature.asc
Description: Digital signature


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Josh Berkus

Magnus,


I don't think we should keep an archive of old alphas though -
that's going to leave us with insane amounts of documentation sets.
But we could have a /docs/alpha/ which would hold the latest released
alpha.


Yes, that's perfect.

For that matter, for the release notes, I wasn't planning to have a 
seperate set for every alpha.  I was thinking of just creating a 
*cumulative* set, with the new items for the current alpha flagged somehow.


--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] pg_dump vs PQescapeBytea

2009-08-04 Thread Tom Lane
I mentioned in
http://archives.postgresql.org/message-id/21837.1248215...@sss.pgh.pa.us
that pg_dump has a bug in its handling of large objects, which is
directly related to the fact that it's still using the deprecated
function PQescapeBytea.  The reason PQescapeBytea is deprecated is
that it has to guess at the setting of standard_conforming_strings,
and it is not guessing right in this one case.

I speculated about fixing that by inventing a PQescapeByteaHex
function, but that was pure brain fade :-(.  The hex format for
bytea still requires one backslash in the value, so you still have
to know standard_conforming_strings to get it right.

AFAICS the only way to make this work correctly and still have libpq
do the work is to create a libpq function that takes an explicit
parameter for the value of standard_conforming_strings to assume.
(We can't pass a PQconn because we don't have one.)  That seems a tad
ugly, so I'm thinking we might just as well have pg_dump solve this
problem for itself.

Another issue related to this code is that ideally we'd like it to
use hex format for speed reasons.  The problem there is that when
writing text output in pg_dump or pg_restore, we don't know for
sure what server version the output will be loaded into.  If we
use hex format then it will be impossible to load the output into
a pre-8.5 server.  This wouldn't be the first time we've broken
backwards compatibility of dump output, of course, but it's something
that would be entirely impractical to fix by manual editing of the
dump output.  So it seems a bit worse than our average breakage.

If we aren't worried about backwards compatibility then the solution
is easy: have pg_dump emit LO data in the form
E'\\x ... hex data here ...'
which is cheap to generate and will work regardless of GUC settings.
If we want it to generate backwards-compatible output then we'll
essentially end up duplicating libpq's PQescapeBytea code in pg_dump.

Thoughts, better ideas?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Alpha Releases: Docs?

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 1:19 PM, Peter Eisentrautpete...@gmx.net wrote:
 On Tuesday 04 August 2009 18:52:06 Robert Haas wrote:
 I'm willing to help if these are 8.5 release notes in process.  I'm
 not willing to help if they are alpha release notes that will be
 thrown away afterwards.  Which is it?

 I was working on the latter assumption.

 I have some reservations about the former approach.  It would basically commit
 us right now to having a consistent set of volunteers available every two
 months within specific 1-2 day spans.  Which is the sort of thing I wanted to
 avoid.  But if we have that commitment, then go for it.

Yeah, the timing of the alpha releases could make this tricky.  I'm
not sure if there's a way to work around that, or if we should just
give up and keep doing it the way that we have in the past.

I would like to think that the timing wouldn't need to be quite as
tight as what you are supposing.  For example, is it really too early
to start working on the release notes for alpha1 now, or say at the
end of the week?  If work were started on 8/7, there would still be 8
days left before official end-of-CommitFest, and presumably the
release of alpha wouldn't be until a few days after that, yet we'd
have a pretty good idea what was going to be in there.

Also, to be quite frank, I don't care that much about alpha releases.
What I care about is production releases.  Helping with the release
notes for alpha is a means to an end. So it seems to me that, if
necessary, we could do something quick-and-dirty to get out the door,
and then we can work on patching them up over time.  So the release
notes for features added in the latest alpha might be a little rougher
than the ones for earlier alphas (but maybe still better than, here's
the CVS log, have fun).

What I would like to avoid is a situation where we're basically ready
to go with beta and Bruce says, Hold on, everybody, it's going to
take another two weeks while I plow through 600 commit messages.  I
have a theory that that work can be spread out and much of it done in
advance and not necessarily by Bruce.  However, that theory has yet to
be tested, and the committers (principally Tom and Bruce) have to be
open to it for it to have any chance of success.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 First we need several new error message fields: table name, function
 name, constraint name, and so on.

It would also help to have clear definitions of what these *mean*, which
is entirely unclear from your comments --- in particular, the reference
to errcontext callbacks confuses the heck out of me.  I would have
thought that these would be used for the referenced object name in cases
like table not found, and surely using an errcontext callback for that
would be the hardest possible way to implement it.

 ... would be to give each new field its own start letter (see
 http://www.postgresql.org/docs/8.4/static/protocol-error-fields.html);
 say T for table, f for function (F is taken), c for constraint (C
 is taken), and so on.  Another possibility would be to use a single
 letter, say N, and add a subtype to it; so table name would be NT
 followed by the table name, NF for functions, etc.

Without a pretty concrete list of what the additions are going to be,
it's difficult to make any reasoned choices there.

Lastly, I'm not as sure as you are that the case for these is well made.
In exactly what cases would client code be able to do something useful
with them?  Your proposal involves a pretty huge amount of work if we
are to carry it out thoroughly, and I'm 100% not convinced that there's
a proportional benefit.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mixed, named notation support

2009-08-04 Thread Pavel Stehule
2009/8/4 Bernd Helmle maili...@oopsware.de:
 --On Montag, August 03, 2009 23:43:08 +0200 Bernd Helmle
 maili...@oopsware.de wrote:

 Status Report: I will finish documentation and review tomorrow and will
 mark this patch for committer review.

 Here's my latest reviewed version of Pavel's patch with adjusted
 documentation per latest discussion.

 While poking a little bit with simplify_function() I realized that this
 patch changes the behavior of VARIADIC functions a little bit. For example:

 CREATE OR REPLACE FUNCTION my_test(a IN text, txt VARIADIC text[])
 RETURNS text
 AS
 $$
  SELECT $2[1];
 $$ LANGUAGE SQL;

 The following doesn't work in current 8.4:

 SELECT my_test('abcd', ARRAY['test', 'foo']);

 You need to use the VARIADIC keyword to match the second argument to text[].
 However, if you are going to use named notation with the patch applied, the
 picture changes in HEAD:

 SELECT my_test('abcd' AS a, ARRAY['test', 'foo'] AS txt);
 my_test
 -
 test
 (1 row)

 This applies also when you reverse the argument order. I don't know wether
 this is intended, but its conflicting with what we have currently in the
 docs. It's also not clear to me wether we want this at all.


Named notation has different algorithm for function detection then
positional notation. There are not exist variadic parameters (because
these parameters hasn't individual names). So only packed variadic
parameter should be there, and this parameter have to be named  - so
keyword VARIADIC is optional.

Thank You very much
Pavel

 --
  Thanks

                   Bernd

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
Tom Lane wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
  First we need several new error message fields: table name, function
  name, constraint name, and so on.
 
 It would also help to have clear definitions of what these *mean*, which
 is entirely unclear from your comments --- in particular, the reference
 to errcontext callbacks confuses the heck out of me.  I would have
 thought that these would be used for the referenced object name in cases
 like table not found, and surely using an errcontext callback for that
 would be the hardest possible way to implement it.

Huh, yeah, certainly there are cases where the errtable() call is going
to be directly in the ereport() call instead of errcontext, but in some
other cases (for example when reporting problems in functions) we're
going to need errcontext.


 Lastly, I'm not as sure as you are that the case for these is well made.
 In exactly what cases would client code be able to do something useful
 with them?  Your proposal involves a pretty huge amount of work if we
 are to carry it out thoroughly, and I'm 100% not convinced that there's
 a proportional benefit.

Hmm, well, I skipped the rationale because it has been requested before.
For example, we need to give constraint names so that applications can
tell which unique key is being violated.  We need table names on which
they are being violated.  We need column names for datatype mismatches,
and so on.  We frequently see people parsing the error message to
extract those, but that is known to be fragile, cumbersome and error
prone.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Tom Lane wrote:
 Lastly, I'm not as sure as you are that the case for these is well made.
 In exactly what cases would client code be able to do something useful
 with them?  Your proposal involves a pretty huge amount of work if we
 are to carry it out thoroughly, and I'm 100% not convinced that there's
 a proportional benefit.

 Hmm, well, I skipped the rationale because it has been requested before.
 For example, we need to give constraint names so that applications can
 tell which unique key is being violated.  We need table names on which
 they are being violated.  We need column names for datatype mismatches,
 and so on.  We frequently see people parsing the error message to
 extract those, but that is known to be fragile, cumbersome and error
 prone.

Frankly, I don't believe it.  I've seen possibly one or two requests
for such things.  That's not enough interest to justify the kind of
work and code-size investment you're talking about.

If there are situations where this info is missing from the
human-readable message, then sure, let's look into fixing that.
But the use-case for automatic interpretation of the message
is just a whole lot smaller than would justify the work. 
To take just one point, I rather doubt that SQLSTATE codes are really
sufficiently fine-grained to let applications automatically determine
what to do without looking at the message text.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Pavel Stehule
2009/8/4 Tom Lane t...@sss.pgh.pa.us:
 Alvaro Herrera alvhe...@commandprompt.com writes:
 Tom Lane wrote:
 Lastly, I'm not as sure as you are that the case for these is well made.
 In exactly what cases would client code be able to do something useful
 with them?  Your proposal involves a pretty huge amount of work if we
 are to carry it out thoroughly, and I'm 100% not convinced that there's
 a proportional benefit.

 Hmm, well, I skipped the rationale because it has been requested before.
 For example, we need to give constraint names so that applications can
 tell which unique key is being violated.  We need table names on which
 they are being violated.  We need column names for datatype mismatches,
 and so on.  We frequently see people parsing the error message to
 extract those, but that is known to be fragile, cumbersome and error
 prone.

 Frankly, I don't believe it.  I've seen possibly one or two requests
 for such things.  That's not enough interest to justify the kind of
 work and code-size investment you're talking about.

 If there are situations where this info is missing from the
 human-readable message, then sure, let's look into fixing that.
 But the use-case for automatic interpretation of the message
 is just a whole lot smaller than would justify the work.
 To take just one point, I rather doubt that SQLSTATE codes are really
 sufficiently fine-grained to let applications automatically determine
 what to do without looking at the message text.

I can see sense of special err attrib for constraints, table and
columns. This should to help with error procession on application
level. This is language independent and +/- more stable than error
messages.

regards
Pavel Stehule


                        regards, tom lane

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Tom Lane wrote:
 It would also help to have clear definitions of what these *mean*, which
 is entirely unclear from your comments --- in particular, the reference
 to errcontext callbacks confuses the heck out of me.  I would have
 thought that these would be used for the referenced object name in cases
 like table not found, and surely using an errcontext callback for that
 would be the hardest possible way to implement it.

 Huh, yeah, certainly there are cases where the errtable() call is going
 to be directly in the ereport() call instead of errcontext, but in some
 other cases (for example when reporting problems in functions) we're
 going to need errcontext.

Also, you completely dodged the question of defining what the fields
really mean, which would be 100% essential to doing anything automatic
with the results.  If errtable sometimes means a table that doesn't
exist, and sometimes means a table that exists but doesn't contain an
expected column, or sometimes a table that exists but doesn't contain
an expected value, or sometimes a table that exists and contains a
value that shouldn't be there, etc etc, then actually doing anything
interesting with the information is going to be a matter of guess and
hope rather than something that's reliably automatable.  As a single
example, in a foreign-key error message, is errtable going to be
the referencing table or the referenced table?  And how would an
application writer predict that?  If you don't have a clear design
rule that tells the answer, there is absolutely no chance of achieving
any consistency among the hundreds or thousands of ereports that will
have to be consistent in order for this feature to be worth anything.
Let alone whatever random effects an errcontext callback would produce;
I don't think you've thought through that bit at all.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_dump vs PQescapeBytea

2009-08-04 Thread Jeff Davis
On Tue, 2009-08-04 at 13:35 -0400, Tom Lane wrote:
 I speculated about fixing that by inventing a PQescapeByteaHex
 function, but that was pure brain fade :-(.  The hex format for
 bytea still requires one backslash in the value, so you still have
 to know standard_conforming_strings to get it right.

On a related topic: right now it's impossible to safely escape (using
libpq alone) bytea data for use as a text-format argument to a
parameterized query. This is because PQescapeBytea returns a string that
is both escaped for bytea and escaped for inclusion in a SQL string (am
I wrong here?).

It's a pretty minor issue, but a little inconsistent. Is it worth
addressing?

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 4:05 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Alvaro Herrera alvhe...@commandprompt.com writes:
 Tom Lane wrote:
 Lastly, I'm not as sure as you are that the case for these is well made.
 In exactly what cases would client code be able to do something useful
 with them?  Your proposal involves a pretty huge amount of work if we
 are to carry it out thoroughly, and I'm 100% not convinced that there's
 a proportional benefit.

 Hmm, well, I skipped the rationale because it has been requested before.
 For example, we need to give constraint names so that applications can
 tell which unique key is being violated.  We need table names on which
 they are being violated.  We need column names for datatype mismatches,
 and so on.  We frequently see people parsing the error message to
 extract those, but that is known to be fragile, cumbersome and error
 prone.

 Frankly, I don't believe it.  I've seen possibly one or two requests
 for such things.  That's not enough interest to justify the kind of
 work and code-size investment you're talking about.

 If there are situations where this info is missing from the
 human-readable message, then sure, let's look into fixing that.
 But the use-case for automatic interpretation of the message
 is just a whole lot smaller than would justify the work.
 To take just one point, I rather doubt that SQLSTATE codes are really
 sufficiently fine-grained to let applications automatically determine
 what to do without looking at the message text.

I'm not sure whether the code complexity is justified because I don't
know how much code complexity is involved.  But as a guy who writes a
lot of web applications, I do know that the need to parse error
messages is frequent.  I tend to make it the sole responsibility of
the database to report things like foreign key violations.  For
example, a user tries to delete an object via a web interface and it
fails because of a foreign key violation.  I need to tell him (in
English) which foreign key constraint he violated, with a message
something like this:

This object-type may not be deleted because it is still being used
by one or more other-object-type-in-plural-form.

Right now, I do this like this:

if ($err =~ /name_of_first_foreign_key/) {
$r-error_exit('First error message.')
}
elsif ($err =~ /name_of_second_foreign_key/) {
...

I've always found that a bit icky because it relies (for example) on
their not being two constraint names such that the first is a prefix
of the second, and on there not being any other data in the error
string which can be confused with the constraint name.  But I can't
say I've ever had any serious problems as a result of this ickiness...
 I just make sure my constraint names are long and wordy enough to
prevent confusion.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_dump vs PQescapeBytea

2009-08-04 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes:
 On a related topic: right now it's impossible to safely escape (using
 libpq alone) bytea data for use as a text-format argument to a
 parameterized query. This is because PQescapeBytea returns a string that
 is both escaped for bytea and escaped for inclusion in a SQL string (am
 I wrong here?).

True, but I would think that most people would prefer to use binary
format and not do any conversion/escaping at all in that situation.
I doubt it's worth producing still another variant of PQescapeBytea
for the people who insist on a text conversion for that case.  (The
hex mode already made their lives a lot easier, anyway.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
Tom Lane wrote:

 If there are situations where this info is missing from the
 human-readable message, then sure, let's look into fixing that.
 But the use-case for automatic interpretation of the message
 is just a whole lot smaller than would justify the work. 
 To take just one point, I rather doubt that SQLSTATE codes are really
 sufficiently fine-grained to let applications automatically determine
 what to do without looking at the message text.

The message text is unparsable, because sometimes it is translated.  If
the application wants to do something particular upon receiving a
particular error message, it has two choices: either it parses the
string, in which case it better not be translated, or it passes it to
the user untouched, in which cases it better be translated.  There's no
way the application can try to do both things at once.

For example, say an application needs to capture a certain class of
errors to take automatic action, but pass others to the user untouched
for display (unexpected condition).  This is not all that uncommon.
However, there's no way to do this at present.  What I've done is check
the SQLSTATE, but then I can't get the affected table name.  (Normally
the application knows which is the affected table, but this is not
always the case.)

I don't know about finegrainedness of SQLSTATEs but I haven't had
problems with that.  ERRCODE_UNIQUE_VIOLATION appears twice in the
backend code; ERRCODE_CHECK_VIOLATION appears four times;
NOT_NULL_VIOLATION appears five times, as does
ERRCODE_FOREIGN_KEY_VIOLATION.


Lastly, I wonder if this could be useful to GUI tool writers?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Executor Material

2009-08-04 Thread Edson Ramiro
Hi all,

Does someone has some material which explain how the executor works?

I'm looking for the internal processing of a query in the _executor_.

Thanks for help

Edson Ramiro


Re: [HACKERS] Executor Material

2009-08-04 Thread Dan Colish

I found these docs to very helpful for understand how the backend works,
but for the executor specifically, I think following the code is best.

http://anoncvs.postgresql.org/cvsweb.cgi/~checkout~/pgsql/src/tools/backend/index.html

--
--Dan

On Tue, Aug 04, 2009 at 05:35:04PM -0300, Edson Ramiro wrote:
 Hi all,
 
 Does someone has some material which explain how the executor works?
 
 I'm looking for the internal processing of a query in the _executor_.
 
 Thanks for help
 
 Edson Ramiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Executor Material

2009-08-04 Thread Tom Lane
Edson Ramiro erlfi...@gmail.com writes:
 Does someone has some material which explain how the executor works?

Did you read
http://developer.postgresql.org/pgdocs/postgres/overview.html
and src/backend/executor/README?

Once you get through those, reading the source code is the next step.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] head contrib is broken (crypto)

2009-08-04 Thread Zdenek Kotala
It seems that last contrib crypto changes broke buildfarm. You can see
problem e.g. here:

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_mothdt=2009-08-04%2020:06:01

I don't have time to look on it now. But it should be reproducible
everywhere.


Zdenek


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Peter Eisentraut
On Tuesday 04 August 2009 23:19:24 Tom Lane wrote:
 Also, you completely dodged the question of defining what the fields
 really mean, which would be 100% essential to doing anything automatic
 with the results.  If errtable sometimes means a table that doesn't
 exist, and sometimes means a table that exists but doesn't contain an
 expected column, or sometimes a table that exists but doesn't contain
 an expected value, or sometimes a table that exists and contains a
 value that shouldn't be there, etc etc, then actually doing anything
 interesting with the information is going to be a matter of guess and
 hope rather than something that's reliably automatable.

The SQL standard contains an analogous facility that defines exactly that.  
Look for get diagnostics statement.  It specifies what the table name etc. 
is in specific error situations.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 Right now, I do this like this:

 if ($err =~ /name_of_first_foreign_key/) {
 $r-error_exit('First error message.')
 }
 elsif ($err =~ /name_of_second_foreign_key/) {
 ...

 I've always found that a bit icky because it relies (for example) on
 their not being two constraint names such that the first is a prefix
 of the second, and on there not being any other data in the error
 string which can be confused with the constraint name.

As for the prefix bit, doesn't perl have a word boundary regexp
constraint?

However, I wonder whether we could turn this around.  Instead of an
open-ended project to add an ill-defined collection of fields to an
ill-defined collection of error cases, maybe we could identify a
very short list of cases where it's known to be useful to pull a
specific bit of information out of a specific error message.  And
then implement just those.

A minimum requirement for such a thing, in my opinion, is that *every*
occurrence of one of the targeted SQLSTATE codes should be able to
produce the same auxiliary fields with the same meanings.  If you can't
define it that way, then you haven't actually made things better than
looking at the message text.

The bottom line behind my complaining is that this isn't going to be
helpful unless it's very clearly defined which error reports produce
what auxiliary fields.  The impression I got from Alvaro's comments
was that he wanted to decorate everything in sight with anything he
could think of, which perhaps is not what he intended.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
Tom Lane escribió:

 However, I wonder whether we could turn this around.  Instead of an
 open-ended project to add an ill-defined collection of fields to an
 ill-defined collection of error cases, maybe we could identify a
 very short list of cases where it's known to be useful to pull a
 specific bit of information out of a specific error message.  And
 then implement just those.

Hmm, yeah, it makes sense to look at the problem this way.

 The bottom line behind my complaining is that this isn't going to be
 helpful unless it's very clearly defined which error reports produce
 what auxiliary fields.  The impression I got from Alvaro's comments
 was that he wanted to decorate everything in sight with anything he
 could think of, which perhaps is not what he intended.

Right :-(

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 5:23 PM, Tom Lanet...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 Right now, I do this like this:

 if ($err =~ /name_of_first_foreign_key/) {
 $r-error_exit('First error message.')
 }
 elsif ($err =~ /name_of_second_foreign_key/) {
 ...

 I've always found that a bit icky because it relies (for example) on
 their not being two constraint names such that the first is a prefix
 of the second, and on there not being any other data in the error
 string which can be confused with the constraint name.

 As for the prefix bit, doesn't perl have a word boundary regexp
 constraint?

Yep, and it is in fact useful for these kinds of situations.  At least
for me (and I obviously don't have to contend with translation issues
as Alvaro does, since I have the advantage of speaking the
untranslated language) the ickiness of the current situation is
manageable.  It's not ideal, but it is manageable.

 However, I wonder whether we could turn this around.  Instead of an
 open-ended project to add an ill-defined collection of fields to an
 ill-defined collection of error cases, maybe we could identify a
 very short list of cases where it's known to be useful to pull a
 specific bit of information out of a specific error message.  And
 then implement just those.

 A minimum requirement for such a thing, in my opinion, is that *every*
 occurrence of one of the targeted SQLSTATE codes should be able to
 produce the same auxiliary fields with the same meanings.  If you can't
 define it that way, then you haven't actually made things better than
 looking at the message text.

 The bottom line behind my complaining is that this isn't going to be
 helpful unless it's very clearly defined which error reports produce
 what auxiliary fields.  The impression I got from Alvaro's comments
 was that he wanted to decorate everything in sight with anything he
 could think of, which perhaps is not what he intended.

Perhaps not.  :-)

Maybe a specific list of the places where he'd like to add things
would be helpful.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Josh Berkus

 Hmm, well, I skipped the rationale because it has been requested before.
 For example, we need to give constraint names so that applications can
 tell which unique key is being violated.  We need table names on which
 they are being violated.  We need column names for datatype mismatches,
 and so on.  We frequently see people parsing the error message to
 extract those, but that is known to be fragile, cumbersome and error
 prone.

If that's what we're trying to solve, I don't think that adding some
kind of proprietary shorthand coding is a good idea.  If we're do to
this at all, it should be a connection-based GUC option, and use some
standard formal like XML fragments.

-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] head contrib is broken (crypto)

2009-08-04 Thread Alvaro Herrera
Zdenek Kotala wrote:
 It seems that last contrib crypto changes broke buildfarm. You can see
 problem e.g. here:
 
 http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_mothdt=2009-08-04%2020:06:01
 
 I don't have time to look on it now. But it should be reproducible
 everywhere.

It's not crypto changes that's the problem -- it's bytea now
being output in hex form (encrypt and decrypt both return bytea):

*** 
/store/postgres/buildfarm/emperor_moth/HEAD/pgsql.10027/contrib/pgcrypto/expected/3des.out
  Tue Aug  4 22:06:20 2009
--- 
/store/postgres/buildfarm/emperor_moth/HEAD/pgsql.10027/contrib/pgcrypto/results/3des.out
   Tue Aug  4 22:13:42 2009
***
*** 37,44 
  -- decrypt
  select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des');
   decrypt 
! -
!  foo
  (1 row)
  
  -- iv
--- 37,44 
  -- decrypt
  select decrypt(encrypt('foo', '0123456', '3des'), '0123456', '3des');
   decrypt  
! --
!  \x666f6f
  (1 row)
  
  -- iv


-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Greg Sabino Mullane

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160


 If that's what we're trying to solve, I don't think that adding some
 kind of proprietary shorthand coding is a good idea.  If we're do to
 this at all, it should be a connection-based GUC option, and use some
 standard formal like XML fragments.

+1 to this idea in general, but *please* don't consider the use of
XML. If we really need some sort of formatting, let's do CSV. Or
YAML. Or JSON. Anything but XML.

- --
Greg Sabino Mullane g...@turnstep.com
End Point Corporation
PGP Key: 0x14964AC8 200908041806
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-BEGIN PGP SIGNATURE-

iEYEAREDAAYFAkp4sOoACgkQvJuQZxSWSsjItACgsDtcid8Zqx9J2ehIhqbtN1l3
1jMAoNcd/6eJPBkKbdNYx6wbUl3bnxhC
=H5yv
-END PGP SIGNATURE-



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mixed, named notation support

2009-08-04 Thread Bernd Helmle



--On 4. August 2009 20:22:05 +0200 Pavel Stehule pavel.steh...@gmail.com 
wrote:



Named notation has different algorithm for function detection then
positional notation. There are not exist variadic parameters (because
these parameters hasn't individual names). So only packed variadic
parameter should be there, and this parameter have to be named  - so
keyword VARIADIC is optional.


I wonder wether it wouldn't better to force positional notation for such 
functions then. I found it surprising that this works at all, but of 
course, someone else might enjoy this as a cool feature. To me, it feels 
strange and confusing that a function declared as VARIADIC suddenly accepts 
a sloppy argument only because you are using some other calling notation 
where others enforces you to use an additional keyword to match the 
function.


At least, we need to document that both notations behaves different in this 
case.




--
Thanks

Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes:
 If that's what we're trying to solve, I don't think that adding some
 kind of proprietary shorthand coding is a good idea.  If we're do to
 this at all, it should be a connection-based GUC option, and use some
 standard formal like XML fragments.

Huh?  What he was talking about is providing additional error fields,
which would presumably be made available via PQresultErrorField in
libpq, or its kin in other interfaces, and would be totally invisible to
any client that didn't ask for them.  I can't imagine any value-add
from introducing XML into the problem.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] head contrib is broken (crypto)

2009-08-04 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 It's not crypto changes that's the problem -- it's bytea now
 being output in hex form (encrypt and decrypt both return bytea):

I was insufficiently careful about checking the non-core regression
tests :-(.  Mea culpa.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote:
 
 A minimum requirement for such a thing, in my opinion, is that
 *every* occurrence of one of the targeted SQLSTATE codes should be
 able to produce the same auxiliary fields with the same meanings. 
 If you can't define it that way, then you haven't actually made
 things better than looking at the message text.
 
I would hope that SQLSTATE *categorizes* messages rather than uniquely
identifying them.  If it is being used correctly (as I see it), there
could well be different specific messages within the category
identified by a SQLSTATE for which different identifiers are useful.
 
I'm not so interested in using this feature, personally; but I am
concerned about how the issue might affect our use of SQLSTATE, about
which I do care.
 
Many products have a sequence number to identify their messages in
addition to using SQLSTATE to classify them.  That seems pretty
sensible to me.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Filtering dictionaries support and unaccent dictionary

2009-08-04 Thread Robert Haas
On Sat, Aug 1, 2009 at 12:35 AM, Alvaro
Herreraalvhe...@commandprompt.com wrote:
 Teodor Sigaev wrote:

 As for the contrib module, I think it could use a lot more function
 header comments!  Also, it would be great if it could be used separately
 from tsearch, i.e. that it provided a function unaccent(text) returns
 text that unaccented arbitrary strings (I guess it would use the default
 tsconfig).
 Umm? Module provides unaccent(text) and unaccent(regdictionary, text) 
 functions.

 Sorry, I failed to notice.  Looks good.

 Isn't that function leaking res pointer?  Also, I'm curious why you're
 allocating 2*sizeof(TSLexeme) in unaccent_lexize ...

So are we waiting for an updated version of this patch?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SE-PostgreSQL Specifications

2009-08-04 Thread KaiGai Kohei
Stephen Frost wrote:
 * Tom Lane (t...@sss.pgh.pa.us) wrote:
 Stephen Frost sfr...@snowman.net writes:
 * KaiGai Kohei (kai...@ak.jp.nec.com) wrote:
 My concern is access_control_ is a bit long for prefixes,
 but ac_ is too short to represent what it is doing.
 pg_ac_?  Still shorter than 'security_', uses the pg_ prefix, which we
 use in a number of other places, and has 'ac' in it..
 I don't see anything wrong with ac_.  Short is good, and there isn't
 any other concept in the PG internals that it would conflict with.
 If there were, pg_ac_ would surely not help to disambiguate.
 
 Works for me.

OK, I'll go on with the ac_ prefix.

-- 
OSS Platform Development Division, NEC
KaiGai Kohei kai...@ak.jp.nec.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Greg Stark
On Tue, Aug 4, 2009 at 11:28 PM, Tom Lanet...@sss.pgh.pa.us wrote:

 Huh?  What he was talking about is providing additional error fields,
 which would presumably be made available via PQresultErrorField in
 libpq, or its kin in other interfaces, and would be totally invisible to
 any client that didn't ask for them.  I can't imagine any value-add
 from introducing XML into the problem.

Well it could add a single field which contained a structured
description of the error. That would solve the problem you described
for things like foreign key references which require two sets of table
and column lists and assigning meanings to them both -- essentially
because a single list of fields isn't really rich enough to describe
all possible errors.

But I wonder if it's really that hard to parse the text errors. We
have a pretty formulaic construction for our error messages. And we
quote things pretty aggressively and prefixing name with the object
type pretty consistently. Fixing any cases where we don't follow the
rules and fixing the rules in cases where it's hard to parse seems
like at least as good a solution.

So an alternate proposal is to add a field in the error message which
contains the untranslated string. That would let applications always
look at the untranslated string for parsing and always use the
translated string for user displays.
-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread David Fetter
On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote:
 
 -BEGIN PGP SIGNED MESSAGE-
 Hash: RIPEMD160
 
 
  If that's what we're trying to solve, I don't think that adding
  some kind of proprietary shorthand coding is a good idea.  If
  we're do to this at all, it should be a connection-based GUC
  option, and use some standard formal like XML fragments.
 
 +1 to this idea in general, but *please* don't consider the use of
 XML.  If we really need some sort of formatting, let's do CSV.  Or
 YAML.  Or JSON.  Anything but XML.

+1 on the anything but XML.  XML reeks of inner platform effect.

http://en.wikipedia.org/wiki/Inner-platform_effect

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
Greg Stark wrote:

 So an alternate proposal is to add a field in the error message which
 contains the untranslated string. That would let applications always
 look at the untranslated string for parsing and always use the
 translated string for user displays.

That's an interesting idea, but you also have to consider other stuff
not related to translation, like schemas of the tables in question.
Someone requested some time ago to schema-qualify the table name (or a
constraint name, I don't recall offhand) in an error message, but this
idea was shot down because if that person really wanted that info, what
he should be looking at is including extra info in machine-readable
format into errors instead of a kluge like that.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Alvaro Herrera
David Fetter wrote:
 On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote:
  
   If that's what we're trying to solve, I don't think that adding
   some kind of proprietary shorthand coding is a good idea.  If
   we're do to this at all, it should be a connection-based GUC
   option, and use some standard formal like XML fragments.
  
  +1 to this idea in general,

I think the train left the station on this issue quite a while ago.  The
error messages have been like they are now for six releases.  I don't
have any use for changing the format.

Clients can produce XML or JSON or whatever format you like already
anyway.  The protocol is perfectly defined already.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Use DocBook XSL stylesheets for man page building This switches

2009-08-04 Thread Alvaro Herrera
Peter Eisentraut wrote:
 Log Message:
 ---
 Use DocBook XSL stylesheets for man page building
 
 This switches the man page building process to use the DocBook XSL stylesheet 
 toolchain.  The previous targets for Docbook2X are removed. configure has 
 been 
 updated to look for the new tools.  The Documentation appendix contains the 
 new build instructions.  There are also a few isolated tweaks in the 
 documentation to improve places that came out strangely in the man pages.

Excellent work, thanks.

I'm wondering if dropping SPI_* manpages is really what we want.  Maybe
we could add them to man section 3?  I know I've wanted to have them a
couple of times.

Not sure about dblink.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Andrew Dunstan



David Fetter wrote:

On Tue, Aug 04, 2009 at 10:06:37PM -, Greg Sabino Mullane wrote:
  

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160




If that's what we're trying to solve, I don't think that adding
some kind of proprietary shorthand coding is a good idea.  If
we're do to this at all, it should be a connection-based GUC
option, and use some standard formal like XML fragments.
  

+1 to this idea in general, but *please* don't consider the use of
XML.  If we really need some sort of formatting, let's do CSV.  Or
YAML.  Or JSON.  Anything but XML.



+1 on the anything but XML.  XML reeks of inner platform effect.

http://en.wikipedia.org/wiki/Inner-platform_effect


  


So, we are just trying to whip into shape explain diagnostics which are 
in JSON or XML, and now you want us to exclude XML from this one because 
you don't like it? Can we please try for some consistency?


Sorry to break it to you, but there are plenty of people and businesses 
who want XML. And I certainly don't want to have to master every data 
representation model out there. XML has far more traction than anything 
else that's comparable in my experience.


The fact that Greg is prepared to suggest CSV, with its obvious serious 
deficiencies, as being *better* than XML, makes his whole argument 
highly suspect IMNSHO.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Joshua D. Drake

 So, we are just trying to whip into shape explain diagnostics which are 
 in JSON or XML, and now you want us to exclude XML from this one because 
 you don't like it? Can we please try for some consistency?
 
 Sorry to break it to you, but there are plenty of people and businesses 
 who want XML. And I certainly don't want to have to master every data 
 representation model out there. XML has far more traction than anything 
 else that's comparable in my experience.
 
 The fact that Greg is prepared to suggest CSV, with its obvious serious 
 deficiencies, as being *better* than XML, makes his whole argument 
 highly suspect IMNSHO.

From a business perspective, XML is the only viable option for output. 

Joshua D. Drake

-- 
PostgreSQL - XMPP: jdr...@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] DefaultACLs

2009-08-04 Thread Tom Lane
I looked at this patch a bit.  I haven't actually read any of the code
yet, just reviewed the on-list discussions and the docs, but I think I can
make a few comments at the definitional level.

First: there's already been some discussion about the VIEW problem.
I understand that the patch adds a GRANT ON VIEW syntax that works
like GRANT ON SEQUENCE in that if you say VIEW it insists the object
really is a view, but without taking away the existing laxness that
allows you to still say GRANT ON TABLE for a view.  This seems reasonable
in isolation, but it creates a big problem for both this patch and the
GRANT ON ALL patch, in that it's unclear how to treat views if they
could be considered either tables or views.  Unfortunately I think
we have no choice but to live with the existing laxness, because
(a) to do otherwise will break pretty nearly every pg_dump script, and
(b) the SQL standard says we have to accept GRANT ON TABLE view.
In fact GRANT ON VIEW is not in the SQL standard and there is no good
reason to expect applications will adopt it at all.

So my feeling is that adding GRANT ON VIEW is a bad idea.  The main
argument for doing it seemed to be that the author wanted to be able
to grant different default privileges for tables and views, but I'm
unconvinced that there's a strong use-case for that.  You could very
easily have one set of default privileges for all of tables, views,
and sequences, simply ignoring any bits that weren't relevant for
particular objects.  It seems to me that that would handle common
cases just fine.  For complicated cases not so much, but it's not
clear to me that filtering by the object subtype is all that useful
for complicated cases anyway.  People will want more functionality
than that.  Which leads into my next item.

Second: both this patch and GRANT ON ALL are built on the assumption
that the only way to filter/classify objects is by schema membership.
Now I don't object to that as an initial implementation restriction,
but I don't like hard-wiring it into the syntax.  It is very clear to me
that we'll want other filter rules in the future --- an immediate example
is being able to say that new children of an inheritance parent table
should inherit its GRANTs.  So to my mind, designing the syntax around
ALTER SCHEMA is right out.  Maybe we could do something like
ALTER DEFAULT PRIVILEGES ON TABLES IN SCHEMA foo GRANT ...
where the IN SCHEMA foo part would be subject to generalization later.
This also matches up a bit better with the proposed syntax for GRANT
ON ALL (which also uses IN SCHEMA foo).

Third: speaking of syntax, I don't like the way that this is gratituously
different from GRANT/REVOKE.  I don't like using ADD/DROP instead of
GRANT/REVOKE, nor the unnecessary AND business.  I think we should
minimize confusion by using something that is spelled as nearly like
GRANT/REVOKE as possible.

Fourth: the system's existing treatment of default permissions is
owner-dependent, that is the implied set of permissions is typically
GRANT ALL TO owner (from himself, with grant option).  I do not
understand how schema-level default ACLs will play nicely with that,
except in the special case where the schema owner also owns every
contained object.  If you copy the schema-level ACL directly to a
contained object with a different owner it will definitely be the wrong
thing, but if you try to translate the ownership it will confuse people
too.  And confusion in a security-related behavior is a Bad Thing.
Furthermore, having the schema owner able to control the grants made
for objects not owned by him is a huge security hole.

What I suggest as a way to resolve this last point is that a default ACL
should apply only to objects owned by the user who creates/modifies the
default ACL.  In this view, the question of which schema the objects are
in is just an additional filter condition, not the primary determinant
of which objects a default ACL applies to.  Every user has his own set
of default ACLs.  (This is another reason not to design the syntax
around ALTER SCHEMA.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Robert Haas
On Tue, Aug 4, 2009 at 8:36 PM, Joshua D. Drakej...@commandprompt.com wrote:

 So, we are just trying to whip into shape explain diagnostics which are
 in JSON or XML, and now you want us to exclude XML from this one because
 you don't like it? Can we please try for some consistency?

 Sorry to break it to you, but there are plenty of people and businesses
 who want XML. And I certainly don't want to have to master every data
 representation model out there. XML has far more traction than anything
 else that's comparable in my experience.

 The fact that Greg is prepared to suggest CSV, with its obvious serious
 deficiencies, as being *better* than XML, makes his whole argument
 highly suspect IMNSHO.

 From a business perspective, XML is the only viable option for output.

Wow, I feel like it's time for a bench-clearing brawl!

My serialization format kicks your serialization format's butt!

This doesn't have a whole lot to do with the original topic of this
thread, which unless I missed something had only to do with extending
the FE/BE protocol, but it definitely makes for lively conversation.
Anyone want to vote ASN.1 for world domination?  Can we set up some
kind of cage match between the dueling standards?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] md.c should not call files relations

2009-08-04 Thread Tom Lane
There's an interesting thread over here
http://archives.postgresql.org/pgsql-sql/2009-08/msg00013.php
in which someone mistook a filesystem-level permissions problem
for a database permissions problem.  It wasn't exactly his fault,
I think, since the message he was presented with was

ERROR:  could not create relation test: Permission denied

which is not all that obviously different from what you would
get for a SQL-permissions violation.

I am thinking that this message would be more correct and less
confusing if it looked something like

ERROR:  could not create file 12345/67890: Permission denied

ie, when reflecting an OS-level error we should call a file a file and
provide its filesystem name, not the name of the table that we were
hoping to map to it.  This would be more likely to lead the user's
mind in the right direction, and he'd need the filesystem pathname
for any detailed investigation anyway.

This would have the further advantage that we could make all the
errors in md.c consistent --- some of them provide filesystem names
rather than table names because that's all they have available.

Lastly, I'm wondering why someone seems to have removed the double
quotes around the filesystem name in some of these messages.
Surely that's not per style guide.

Comments?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread David Fetter
On Tue, Aug 04, 2009 at 09:16:23PM -0400, Robert Haas wrote:
 On Tue, Aug 4, 2009 at 8:36 PM, Joshua D. Drakej...@commandprompt.com wrote:
 
  So, we are just trying to whip into shape explain diagnostics which are
  in JSON or XML, and now you want us to exclude XML from this one because
  you don't like it? Can we please try for some consistency?
 
  Sorry to break it to you, but there are plenty of people and businesses
  who want XML. And I certainly don't want to have to master every data
  representation model out there. XML has far more traction than anything
  else that's comparable in my experience.
 
  The fact that Greg is prepared to suggest CSV, with its obvious serious
  deficiencies, as being *better* than XML, makes his whole argument
  highly suspect IMNSHO.
 
  From a business perspective, XML is the only viable option for output.
 
 Wow, I feel like it's time for a bench-clearing brawl!
 
 My serialization format kicks your serialization format's butt!
 
 This doesn't have a whole lot to do with the original topic of this
 thread, which unless I missed something had only to do with extending
 the FE/BE protocol, but it definitely makes for lively conversation.
 Anyone want to vote ASN.1 for world domination?  Can we set up some
 kind of cage match between the dueling standards?

I love standards.  There are so many to choose from :)

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] the case for machine-readable error fields

2009-08-04 Thread Tom Lane
David Fetter da...@fetter.org writes:
 On Tue, Aug 04, 2009 at 09:16:23PM -0400, Robert Haas wrote:
 My serialization format kicks your serialization format's butt!

 I love standards.  There are so many to choose from :)

And the funny thing is that the format we transmit this info to the
client in is really the least interesting or difficult part of the
whole matter ...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Convert stmt back into queryString

2009-08-04 Thread Dan Colish
I am currently trying to convert an insertstmt back into a const
char *queryString, but I can't find an existing function to do this
for the life of me.  I will write one if none exits, but I figured I
ask here first.  Unfortunately, nodeToString is not quite right for
what I'm doing. Thanks in advance.

--
--Dan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Convert stmt back into queryString

2009-08-04 Thread Tom Lane
Dan Colish d...@unencrypted.org writes:
   I am currently trying to convert an insertstmt back into a const
   char *queryString, but I can't find an existing function to do this
   for the life of me.  I will write one if none exits, but I figured I
   ask here first.  Unfortunately, nodeToString is not quite right for
   what I'm doing. Thanks in advance.

Hmm, you mean a Query, or a raw unanalyzed InsertStmt?  If the former,
ruleutils.c will help.  If the latter, be prepared to write a lot of
code; there's nothing closer than nodeToString, and even that is pretty
incomplete for raw grammar output nodes IIRC.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Convert stmt back into queryString

2009-08-04 Thread Dan Colish

On Tue, Aug 04, 2009 at 10:00:24PM -0400, Tom Lane wrote:
 Dan Colish d...@unencrypted.org writes:
  I am currently trying to convert an insertstmt back into a const
  char *queryString, but I can't find an existing function to do this
  for the life of me.  I will write one if none exits, but I figured I
  ask here first.  Unfortunately, nodeToString is not quite right for
  what I'm doing. Thanks in advance.
 
 Hmm, you mean a Query, or a raw unanalyzed InsertStmt?  If the former,
 ruleutils.c will help.  If the latter, be prepared to write a lot of
 code; there's nothing closer than nodeToString, and even that is pretty
 incomplete for raw grammar output nodes IIRC.
 
   regards, tom lane

In this case, its a raw InsertStmt. I would like to pass this back to
parse_analyze, but I need to have a queryString to go with that call, so
crafting a function to rate that seems to be the only way, atm.

--
--Dan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] async notification patch for dblink

2009-08-04 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Tom Lane wrote:
 Joe Conway m...@joeconway.com writes:
 Sure, I guess I ought to use the latest-and-greatest. Any other comments
 before I commit?
 
 That be_pid/be_pid hack in the regression test is pretty ugly, and
 doesn't test anything very useful anyway seeing that it's integer
 division.  Could you do something like
   be_pid = pg_backend_pid() AS is_self_notify
 instead, to verify that it's a self-notify?  (This is not quite right
 because you'd need to execute pg_backend_pid() at the remote end, but
 I'm not awake enough to remember the dblink syntax for that.  Maybe
 it's too complex to be worth it, but I think demonstrating how to
 check for self-notify would be a useful bit of doco.)
 
 Otherwise it looks ok to me.

OK, still not pretty, but at least more useful. Last call for
comments...I hope ;-)

Joe
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iQIcBAEBCAAGBQJKePNeAAoJEDfy90M199hlduYP/2UaEuEXzWnGS1RX4Be+UajS
LBrS5uanAEUJcohnpMXCHjZVuLJBhXl8r/2RPXsVt5Fem3oYkxFrK3eCysMZs0jJ
F6iWb/Hiyi3bVf8coHjeabm0j/TF5yYsKuE3D0kcxktwKjq9vyGOxkadcAXV/lq1
+awVp5tip1ko3R20zSv+N4dcZp6ewHZ9Fa9AT9BS/3D5vQn6Jq3aUajqZ1DdSw0W
Fm5DlAl632xB8dQ9G/nNUYXMf5040OyD7foZEncS30i+diyHEvx5++5peCmlAvBj
KmeULbsmGn+brlQjvMftJU5LnOzEkKAHYs90BjR/jHoivJHSBjDNbREm1lJqGU9q
DU7E3NXz2/hva+/a+PPoyZjJDj+wvxzBivySB/fPCLpQFATDYHQ+1iMOKWqiGO1q
N1I6bEEZTAH1/8drANR224/BVnOs10s+4uUL5L8Me+eWO0gIxyhpHyIg/MkhlJ1a
Aopz3p7XEuXWmo4XGEAMVhJ2NpBiNKc69Ihno3kkY6FMo4Hs8rNsP+bdZnpBhugO
JB0+EDhSaVf2O7a8zmz3fylP2cNBQTh3gZlbNzYlnYgkhkuTXda2JXjwdnAkd5DK
33vVaJ0swb7MzeBWzP5PDMXKbzVPtz7FvReVVhIZbAJk+oVlwlw+KMRo8z8g0ci8
CgX2i1YwCh0AGuk450rZ
=JQmH
-END PGP SIGNATURE-
Index: contrib/dblink/dblink.c
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.c,v
retrieving revision 1.82
diff -c -r1.82 dblink.c
*** contrib/dblink/dblink.c	11 Jun 2009 14:48:50 -	1.82
--- contrib/dblink/dblink.c	4 Aug 2009 13:41:26 -
***
*** 1635,1640 
--- 1635,1723 
  	PG_RETURN_DATUM(current_query(fcinfo));
  }
  
+ /*
+  * Retrieve async notifications for a connection. 
+  *
+  * Returns an setof record of notifications, or an empty set if none recieved.
+  * Can optionally take a named connection as parameter, but uses the unnamed connection per default.
+  *
+  */
+ #define DBLINK_NOTIFY_COLS		3
+ 
+ PG_FUNCTION_INFO_V1(dblink_get_notify);
+ Datum
+ dblink_get_notify(PG_FUNCTION_ARGS)
+ {
+ 	PGconn			   *conn = NULL;
+ 	remoteConn		   *rconn = NULL;
+ 	PGnotify		   *notify;
+ 	ReturnSetInfo	   *rsinfo = (ReturnSetInfo *) fcinfo-resultinfo;
+ 	TupleDesc			tupdesc;
+ 	Tuplestorestate	   *tupstore;
+ 	MemoryContext		per_query_ctx;
+ 	MemoryContext		oldcontext;
+ 
+ 	DBLINK_INIT;
+ 	if (PG_NARGS() == 1)
+ 		DBLINK_GET_NAMED_CONN;
+ 	else
+ 		conn = pconn-conn;
+ 
+ 	/* create the tuplestore */
+ 	per_query_ctx = rsinfo-econtext-ecxt_per_query_memory;
+ 	oldcontext = MemoryContextSwitchTo(per_query_ctx);
+ 
+ 	tupdesc = CreateTemplateTupleDesc(DBLINK_NOTIFY_COLS, false);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 1, notify_name,
+ 	   TEXTOID, -1, 0);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 2, be_pid,
+ 	   INT4OID, -1, 0);
+ 	TupleDescInitEntry(tupdesc, (AttrNumber) 3, extra,
+ 	   TEXTOID, -1, 0);
+ 
+ 	tupstore = tuplestore_begin_heap(true, false, work_mem);
+ 	rsinfo-returnMode = SFRM_Materialize;
+ 	rsinfo-setResult = tupstore;
+ 	rsinfo-setDesc = tupdesc;
+ 
+ 	MemoryContextSwitchTo(oldcontext);
+ 
+ 	PQconsumeInput(conn);
+ 	while ((notify = PQnotifies(conn)) != NULL)
+ 	{
+ 		Datum		values[DBLINK_NOTIFY_COLS];
+ 		bool		nulls[DBLINK_NOTIFY_COLS];
+ 
+ 		memset(values, 0, sizeof(values));
+ 		memset(nulls, 0, sizeof(nulls));
+ 
+ 		if (notify-relname != NULL)
+ 			values[0] = CStringGetTextDatum(notify-relname);
+ 		else
+ 			nulls[0] = true;
+ 
+ 		values[1] = Int32GetDatum(notify-be_pid);
+ 
+ 		if (notify-extra != NULL)
+ 			values[2] = CStringGetTextDatum(notify-extra);
+ 		else
+ 			nulls[2] = true;
+ 
+ 		/* switch to appropriate context while storing the tuple */
+ 		MemoryContextSwitchTo(per_query_ctx);
+ 		tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ 		MemoryContextSwitchTo(oldcontext);
+ 
+ 		PQfreemem(notify);
+ 		PQconsumeInput(conn);
+ 	}
+ 
+ 	/* clean up and return the tuplestore */
+ 	tuplestore_donestoring(tupstore);
+ 
+ 	return (Datum) 0;
+ }
+ 
  /*
   * internal functions
   */
Index: contrib/dblink/dblink.h
===
RCS file: /opt/src/cvs/pgsql/contrib/dblink/dblink.h,v
retrieving revision 1.22
diff -c -r1.22 dblink.h
*** contrib/dblink/dblink.h	9 Jun 2009 17:41:02 -	1.22
--- contrib/dblink/dblink.h	4 Aug 2009 13:41:26 -
***
*** 57,61 
--- 57,62 
  extern Datum 

  1   2   >