Re: [HACKERS] Transaction-scope advisory locks

2011-02-10 Thread Itagaki Takahiro
On Thu, Feb 10, 2011 at 08:36, Marko Tiikkaja
marko.tiikk...@cs.helsinki.fi wrote:
 One issue might be in pg_locks
 Robert suggested not doing this for 9.1, and I don't have anything against
 that.

Agreed.

 Updated patch attached.

Looks good to commit. I note a few minor issues for committer:

* Functions listed in Table 9-62. Advisory Lock Functions might need
  sorted in alphabetical order.

* We could extend LockReleaseAll() to have the 3rd mode
  instead of LockReleaseSession().  Existing behavior is:
| LockReleaseAll(LOCKMETHODID lockmethodid, bool allLocks)
|   allLocks == true: release all locks including session locks.
|   allLocks == false: release all non-session locks.

* Or, we might have one subroutine for LockReleaseSession() and
  LockReleaseCurrentOwner(). They have similar codes.

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Varchar and binary protocol

2011-02-10 Thread Merlin Moncure
On Thu, Feb 10, 2011 at 2:56 AM, Radosław Smogura
rsmog...@softperience.eu wrote:
 Merlin Moncure mmonc...@gmail.com Thursday 10 February 2011 08:48:26
 On Sat, Feb 5, 2011 at 4:59 PM, Radosław Smogura

 Since there is basically zero difference in how *varchar* is handled
 in the database for the text or binary protocols (AFAIK, they use the
 same code), this is almost certainly an issue with the JDBC driver, or
 your benchmark application.

 merlin

 Actually difference is
 http://archives.postgresql.org/pgsql-hackers/2011-02/msg00415.php


ah, I stand corrected -- interesting.

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] patches that could use additional reviewers

2011-02-10 Thread Noah Misch
[Cc: trimmed]

On Wed, Feb 09, 2011 at 01:45:11PM -0500, Robert Haas wrote:
 A few other ones that could use more reviewers include:

 key locks

I'll take a look at this one.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Update docs on building for Windows to accomodate current realit

2011-02-10 Thread Andrew Dunstan



On 02/10/2011 01:04 AM, Gurjeet Singh wrote:
This commit refers to www.mingw64.org http://www.mingw64.org which 
does not exist.





Oops. URL (and name) fixed.

thanks.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add support for logging the current role

2011-02-10 Thread Itagaki Takahiro
On Mon, Feb 7, 2011 at 04:10, Stephen Frost sfr...@snowman.net wrote:
 Yeah, doesn't seem to work for me (missing '/bin/collateindex.pl',
 apparently..).

You might need yum install openjade stylesheets or similar packages
and re-configure.

 Ok, I've cleaned up that part of the documentation to be a table instead
 of the listings that were there, seems like a better approach anyway.

Yeah, that's a good job!

 I agree that it's logically good design, but we could not accept it
 as long as it breaks tools in the real world...
 If it does, I think it's pretty clear that those tools are themselves
 broken..

The word break was my wrong choice, but your new parameter still
requires very wide monitors to display SHOW ALL and pg_settings.
I'd like to solve the issue even though the feature itself is useful.
One fast and snappy solution might be to set the default value to
default, that means the compatible set of columns.
Other better ideas?

For implementation, write_csvlog() has many following lines:
 if (curr_field != num_fields) appendStringInfoChar(buf, ',');
It will be cleaner if we add first_col flag and move it out of
the switch statement.

Other questions I raised before might be matters of preference.
I'd like to here about them form third person.
 * name: log_csv_fields vs. csvlog_fields
 * when to assign: PGC_POSTMASTER vs. PGC_SIGHUP

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types - efficiency

2011-02-10 Thread Florian Weimer
* Chris Browne:

 The RangeType-based equivalent is the following:

 rangetest@localhost-  explain analyze select * from some_data where 
 '[2010-01-01,2010-02-01)'::daterange @ whensit;
QUERY PLAN
 -
  Seq Scan on some_data  (cost=0.00..634.00 rows=1 width=8) (actual 
 time=1.045..111.739 rows=390 loops=1)
Filter: ('[ 2010-01-01, 2010-02-01 )'::daterange @ whensit)
  Total runtime: 111.780 ms
 (3 rows)

 This, alas, reverts to a seq scan on the table, rather than restricting
 itself to the tuples of interest.

This is quite similar to LIKE and regexp matches.  The backend has a
kludge to use the index in such cases.  It did not seem extensible to
me last time I looked, unfortunately.

-- 
Florian Weimerfwei...@bfk.de
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SSI patch version 14

2011-02-10 Thread Heikki Linnakangas

On 09.02.2011 17:58, Kevin Grittner wrote:

Dan Portsd...@csail.mit.edu  wrote:


I think for SerializableXidHash we should probably just initially
allocate it at its maximum size. Then it'll match the PredXact
list which is allocated in full upfront, and there's no risk of
being able to allocate a transaction but not register its xid. In
fact, I believe there would be no way for starting a new
serializable transaction to fail.


To be more precise, it would prevent an out of shared memory error
during an attempt to register an xid for an active serializable
transaction.  That seems like a good thing.  Patch to remove the
hint and initially allocate that HTAB at full size attached.


Committed.

Curiously, coypu has gone green again. It's now choosing 40 connections 
and 8 MB of shared_buffers, while it used to choose 30 connections and 
24 MB of shared_buffers before the SSI patch. Looks like fixing the size 
estimation bugs helped that, but I'm not entirely sure how. Maybe it 
just failed with higher max_connections settings because of the 
misestimate. But why does it now choose a *higher* max_connections 
setting than before?


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pl/python explicit subtransactions

2011-02-10 Thread Jan Urbański
On 10/02/11 01:26, Steve Singer wrote:
 On 11-02-09 05:22 PM, Peter Eisentraut wrote:
 On tis, 2011-02-08 at 00:32 -0500, Steve Singer wrote:
 Is it necessarily a good idea that an explicit subtransaction disables
 the implicit sub-subtransactions?  It might be conceivable that you'd
 still want to do some try/catch within explicit subtransactions.

 
 I had tested nested subtransactions but not a normal try/catch within a
 subtransaction.  That sounds reasonable to allow.
 
 Unfortunately it leads to:
 
 [crash]

D'oh, I was thinking about whether it's safe to skip the internal
subxact if you're in an implicit one and somehow I always convinced
myself that since you eventually close the explicit one, it is.

Obviously my testing wasn't enough :( Attaching an updated patch with
improved docs incorporating Steve's fixes, and fixes  tests for not
statring the implicit subxact. That actually makes the patch a bit
smaller ;) OTOH I had to remove the section from the docs that claimed
performance improvement due to only starting the explicit subxact...

Cheers,
Jan
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index e05c293..87be8c2 100644
*** a/doc/src/sgml/plpython.sgml
--- b/doc/src/sgml/plpython.sgml
*** $$ LANGUAGE plpythonu;
*** 943,949 
  
/sect2
  
!   sect2
 titleTrapping Errors/title
  
 para
--- 943,949 
  
/sect2
  
!   sect2 id=plpython-trapping
 titleTrapping Errors/title
  
 para
*** $$ LANGUAGE plpythonu;
*** 968,973 
--- 968,1089 
 /para
/sect2
   /sect1
+ 
+  sect1 id=plpython-subtransaction
+   titleExplicit subtransactions/title
+   para
+ Recovering from errors caused by database access as described
+ in xref linkend=plpython-trapping can lead to an undesirable situation
+ where some operations succeed before one of them fails and after recovering
+ from that error the data is left in an inconsistent state. PL/Python offers
+ a solution to this problem in the form of explicit subtransactions.
+   /para
+ 
+   sect2
+titleSubtransaction context managers/title
+para
+  Consider a function that implements a transfer between two accounts:
+ programlisting
+ CREATE FUNCTION transfer_funds() RETURNS void AS $$
+ try:
+ plpy.execute(UPDATE accounts SET balance = balance - 100 WHERE account_name = 'joe')
+ plpy.execute(UPDATE accounts SET balance = balance + 100 WHERE account_name = 'mary')
+ except plpy.SPIError, e:
+ result = error transferring funds: %s % e.args
+ else:
+ result = funds transferred correctly
+ plpy.execute(INSERT INTO operations(result) VALUES ('%s') % result)
+ $$ LANGUAGE plpythonu;
+ /programlisting
+  If the second literalUPDATE/literal statement results in an exception
+  being raised, this function will report the error, but the result of the
+  first literalUPDATE/literal will nevertheless be committed. In other
+  words, the funds will be withdrawn from Joe's account, but will not be
+  transferred to Mary's account.
+/para
+para
+  To avoid such issues, you can wrap your literalplpy.execute/literal
+  calls in an explicit subtransaction. The literalplpy/literal module
+  provides a helper object to manage explicit subtransactions that gets
+  created with the literalplpy.subtransaction()/literal function.
+  Objects created by this function implement the
+  ulink url=http://docs.python.org/library/stdtypes.html#context-manager-types;
+  context manager interface/ulink. Using explicit subtransactions we can
+  rewrite our function as:
+ programlisting
+ CREATE FUNCTION transfer_funds2() RETURNS void AS $$
+ try:
+ with plpy.subtransaction():
+ plpy.execute(UPDATE accounts SET balance = balance - 100 WHERE account_name = 'joe')
+ plpy.execute(UPDATE accounts SET balance = balance + 100 WHERE account_name = 'mary')
+ except plpy.SPIError, e:
+ result = error transferring funds: %s % e.args
+ else:
+ result = funds transferred correctly
+ plpy.execute(INSERT INTO operations(result) VALUES ('%s') % result)
+ $$ LANGUAGE plpythonu;
+ /programlisting
+  Note that the use of literaltry/catch/literal is still
+  required. Otherwise the exception would propagate to the top of the Python
+  stack and would cause the whole function to abort with
+  a productnamePostgreSQL/productname error.
+  The literaloperations/literal table would not have any row inserted
+  into it. The subtransaction context manager does not trap errors, it only
+  assures that all database operations executed inside its scope will be
+  atomically committed or rolled back.  A rollback of the subtransaction
+  block occurrs on any kind of exception exit, not only ones caused by
+  errors originating from database access. A regular Python exception raised
+  inside an 

Re: [HACKERS] SSI patch version 14

2011-02-10 Thread Andrew Dunstan



On 02/10/2011 05:09 AM, Heikki Linnakangas wrote:

On 09.02.2011 17:58, Kevin Grittner wrote:

Dan Portsd...@csail.mit.edu  wrote:


I think for SerializableXidHash we should probably just initially
allocate it at its maximum size. Then it'll match the PredXact
list which is allocated in full upfront, and there's no risk of
being able to allocate a transaction but not register its xid. In
fact, I believe there would be no way for starting a new
serializable transaction to fail.


To be more precise, it would prevent an out of shared memory error
during an attempt to register an xid for an active serializable
transaction.  That seems like a good thing.  Patch to remove the
hint and initially allocate that HTAB at full size attached.


Committed.

Curiously, coypu has gone green again. It's now choosing 40 
connections and 8 MB of shared_buffers, while it used to choose 30 
connections and 24 MB of shared_buffers before the SSI patch. Looks 
like fixing the size estimation bugs helped that, but I'm not entirely 
sure how. Maybe it just failed with higher max_connections settings 
because of the misestimate. But why does it now choose a *higher* 
max_connections setting than before?


Rémi might have increased its available resources.

cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] psql patch: tab-complete :variables also at buffer start

2011-02-10 Thread Christoph Berg
Currently, tab-completing :variable names in psql does not work at the
beginning of the line. Fix this by moving the code block before the
empty buffer case.

(I have several sql macros in my .psqlrc like :relsize that prints
table sizes in a nicely formatted way, being able to type :tab would
be very welcome.)

diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1c9623d..a540912 100644
*** a/src/bin/psql/tab-complete.c
--- b/src/bin/psql/tab-complete.c
*** psql_completion(char *text, int start, i
*** 761,766 
--- 761,777 
if (text[0] == '\\')
COMPLETE_WITH_LIST(backslash_commands);
  
+   /* Variable interpolation */
+   else if (text[0] == ':'  text[1] != ':')
+   {
+   if (text[1] == '\'')
+   matches = complete_from_variables(text, :', ');
+   else if (text[1] == '')
+   matches = complete_from_variables(text, :\, \);
+   else
+   matches = complete_from_variables(text, :, );
+   }
+ 
/* If no previous word, suggest one of the basic sql commands */
else if (!prev_wd)
COMPLETE_WITH_LIST(sql_commands);
*** psql_completion(char *text, int start, i
*** 2767,2783 
)
matches = completion_matches(text, 
filename_completion_function);
  
- /* Variable interpolation */
-   else if (text[0] == ':'  text[1] != ':')
-   {
-   if (text[1] == '\'')
-   matches = complete_from_variables(text, :', ');
-   else if (text[1] == '')
-   matches = complete_from_variables(text, :\, \);
-   else
-   matches = complete_from_variables(text, :, );
-   }
- 
/*
 * Finally, we look through the list of things, such as TABLE, INDEX 
and
 * check if that was the previous word. If so, execute the query to get 
a
--- 2778,2783 

Christoph
-- 
c...@df7cb.de | http://www.df7cb.de/


signature.asc
Description: Digital signature


Re: [HACKERS] Re: [COMMITTERS] pgsql: Update docs on building for Windows to accomodate current realit

2011-02-10 Thread Magnus Hagander
On Thu, Feb 10, 2011 at 07:04, Gurjeet Singh singh.gurj...@gmail.com wrote:
 This commit refers to www.mingw64.org which does not exist.

Andrew fixed this alreayd.

 Also, clicking on the gitweb link below (from GMail), opens the browser
 window with an address where ';' are replaced with %3B , which leads to 404
 - no such project. Is GMail broken, or can have gitweb treat %3B as a ; ?

This has also been fixed, per previous emails. I think GMail is not
necessarily broken, but it changed behaviour to one that no other MUA
has.. And gitweb isn't really dealing with that. But the current URLs
generated are sent through a redirector that works around the gmail
issue.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread Peter Eisentraut
On ons, 2011-02-09 at 08:00 -0800, David Fetter wrote:
 On Wed, Feb 09, 2011 at 01:17:06PM +, Bruce Momjian wrote:
  Remove more SGML tabs.
 
 Perhaps we should see about putting something in .git/hooks/pre-commit
 so people can focus on more substantive matters.
 
 Is there some kind of cross-platform way to do this?  I'm thinking
 that given the fact that our build system already requires Perl, there
 should be, but I'm not quite sure how this would be accomplished.

There is make check target in doc/src/sgml/ that is supposed to catch
this.  But it's probably hard to remember to run that.

One thing I was thinking of was that we could add a global make
maintainer-check target (a name I picked up from other projects) which
would run various source code sanity checks.  Besides the SGML tabs
issue, my favourite would be duplicate_oids.  Maybe if we could find a
third use case, we'd have a quorum for implementing this.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Move WAL warning

2011-02-10 Thread Magnus Hagander
On Thu, Feb 10, 2011 at 03:20, Fujii Masao masao.fu...@gmail.com wrote:
 On Wed, Feb 9, 2011 at 7:02 PM, Magnus Hagander mag...@hagander.net wrote:
 On Thu, Feb 3, 2011 at 11:19, Magnus Hagander mag...@hagander.net wrote:
 On Wed, Feb 2, 2011 at 18:00, Magnus Hagander mag...@hagander.net wrote:
 On Wed, Feb 2, 2011 at 17:43, Heikki Linnakangas
 heikki.linnakan...@enterprisedb.com wrote:
 On 02.02.2011 16:36, Magnus Hagander wrote:

 When running pg_basebackup with -x to include all transaction log, the
 server will still throw a warning about xlog archiving if it's not
 enabled - that is completely irrelevant since pg_basebackup has
 included it already (and if it was gone, the base backup step itself
 will fail - actual error and not warning).

 This patch moves the warning from do_pg_base_backup to pg_base_backup,
 so it still shows when using the explicit function calls, but goes
 away when using pg_basebackup.

 For the sake of consistency, how about moving the pg_stop_backup 
 complete,
 all required WAL segments have been archived notice too?

 Well, it goes out as a NOTICE, so by default it doesn't show.. But
 yeah, for code-consistency it makes sense. Like so, then.

 Thinking some more about it, I realized this is not going to be enough
 - we need to be able to turn off the waiting for WAL segment as well,
 in the case when you're streaming the log. Thus, it needs to be
 controllable from the backup client, and we can't just assume the
 default is ok.

 Attached is an updated patch that adds a NOWAIT option to BASE_BACKUP,
 that turns off the waiting. If it's set, it also doesn't warn about
 not being able to wait in the case when there is nothing to wait for,
 so this is a replacement for the previous patch.

 Applied.

 Back to your original complaint. When -x option is specified, pg_basebackup
 should use NOWAIT option in BASE_BACKUP command to send to the server?

Oh yeah, I put that in the wrong patch - it's in my patch to do the
streaming wal. I'll extract it and apply it separately, in case the
other stuff isn't finished.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add support for logging the current role

2011-02-10 Thread Noah Misch
On Thu, Feb 10, 2011 at 06:56:15PM +0900, Itagaki Takahiro wrote:
 On Mon, Feb 7, 2011 at 04:10, Stephen Frost sfr...@snowman.net wrote:
  I agree that it's logically good design, but we could not accept it
  as long as it breaks tools in the real world...
  If it does, I think it's pretty clear that those tools are themselves
  broken..
 
 The word break was my wrong choice, but your new parameter still
 requires very wide monitors to display SHOW ALL and pg_settings.
 I'd like to solve the issue even though the feature itself is useful.
 One fast and snappy solution might be to set the default value to
 default, that means the compatible set of columns.
 Other better ideas?

If some tool barfs on a 330-byte GUC value, we might as well have that tool barf
early and often, not just on non-default values.

FWIW, a 330 byte boot_val doesn't seem like a big deal to me.  If it were over
_POSIX2_LINE_MAX (2048), that might be another matter.

 Other questions I raised before might be matters of preference.
 I'd like to here about them form third person.
  * name: log_csv_fields vs. csvlog_fields

+1 for csvlog_fields.  We have the precedent of syslog_* and that log_* are all
applicable to more than one log destination.

  * when to assign: PGC_POSTMASTER vs. PGC_SIGHUP

+1 for PGC_SIGHUP.  PGC_POSTMASTER is mostly for things where we have not
implemented code to instigate the change after startup (usually because the
difficulty/value ratio of doing so is too high).  There's no such problem here,
merely the risk that the DBA might not be prepared to deal with a column list
change mid-logfile.  If anything, let's have the documentation mention
pg_rotate_logfile() as potentially useful in conjunction.

nm

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] xlog functions for pg_basebackup

2011-02-10 Thread Magnus Hagander
In cleaning up the streaming part of pg_basebackup, I came across this
gem I copied from elsewherE:

/*
 * We have to use postgres.h not postgres_fe.h here, because there's so much
 * backend-only stuff in the XLOG include files we need.  But we need a
 * frontend-ish environment otherwise.  Hence this ugly hack.
 */
#define FRONTEND 1
#include postgres.h


That's obviously quite ugly. I'm getting most defines out of
xlogdefs.h, which loads fine without that hack, but I'm also using the
XLogFileName() macro, which is in xlog_internal.h. The way I see it,
my options are:

1) keep doing wha tI do and include xlog_internal.h which defines it
2) redefine the macro locally (or just duplicate the code - it's only
one line after all, but it's more a matter of principle)
3) move the definition of the macro to some place that can be accessed
externally

thoughts?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add support for logging the current role

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 6:27 AM, Noah Misch n...@leadboat.com wrote:
 FWIW, a 330 byte boot_val doesn't seem like a big deal to me.  If it were over
 _POSIX2_LINE_MAX (2048), that might be another matter.

I don't think it's entirely stupid to worry about this completely
screwing up the output of SHOW ALL on people using 80-character
terminal windows.  I haven't checked, but if it renders the output
totally unreadable then I think we should try to find an alternative
that doesn't.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PostgreSQL FDW update

2011-02-10 Thread Robert Haas
On Tue, Feb 8, 2011 at 3:52 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 I needed something to test the FDW API patch with, and didn't want to get
 involved in the COPY API changes, and also wanted to have something that
 needs real connection management and can push down quals. So I updated the
 postgresql_fdw patch to work with the latest FDW patch.

 Here. It's a bit of a mess, but it works for simple queries..

 It requires a small change to the FDW api
 (fdw-api-add-serverid-userid.patch). I added server oid and user oid fields
 to the FdwPlan - that seems like basic information that most FDW's will
 need, so it seems awkward to require the FDW to wrap them in Const nodes and
 a List.

 These are also available in my git repository at
 git://git.postgresql.org/git/users/heikki/postgres.git, branches fdw2 and
 postgresql_fdw.

So how close are we to getting the core support committed?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] psql patch: tab-complete :variables also at buffer start

2011-02-10 Thread Itagaki Takahiro
On Thu, Feb 10, 2011 at 19:37, Christoph Berg c...@df7cb.de wrote:
 Currently, tab-completing :variable names in psql does not work at the
 beginning of the line. Fix this by moving the code block before the
 empty buffer case.

Seems reasonable to me.

-- 
Itagaki Takahiro

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-02-10 Thread Alexey Klyukin

On Feb 9, 2011, at 9:28 PM, Alex Hunsaker wrote:

 On Wed, Feb 9, 2011 at 08:24, Alexey Klyukin al...@commandprompt.com wrote:
 
 What was actually broken in encode_array_literal support of composite types
 (it converted perl hashes to the literal composite-type constants, expanding
 nested arrays along the way) ? I think it would be a useful extension of the
 existing encode_array_literal.
 
 Yeah, It does not work because it did not take into account the order
 of composite columns. It always put them alphabetically by column
 name. To do it properly we would need to pass in a typid or a column
 order or something. Ideally we could expose the new
 plperl_array_to_datum() to plperl functions in some manner.

Damn, right. Each perl hash corresponds to multiple composite types, different
by the order of the type elements. Passing the typid sounds like a fair
requirement (and if it's missing we could assume that the order of columns in
composites doesn't matter to the caller).

Let me try implementing that as an XS interface to plperl_array_to_datum.

/A

--
Alexey Klyukin
The PostgreSQL Company - Command Prompt, Inc.





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Update docs on building for Windows to accomodate current realit

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 5:46 AM, Magnus Hagander mag...@hagander.net wrote:
 On Thu, Feb 10, 2011 at 07:04, Gurjeet Singh singh.gurj...@gmail.com wrote:
 This commit refers to www.mingw64.org which does not exist.

 Andrew fixed this alreayd.

 Also, clicking on the gitweb link below (from GMail), opens the browser
 window with an address where ';' are replaced with %3B , which leads to 404
 - no such project. Is GMail broken, or can have gitweb treat %3B as a ; ?

 This has also been fixed, per previous emails. I think GMail is not
 necessarily broken, but it changed behaviour to one that no other MUA
 has..

I call that broken over here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Update docs on building for Windows to accomodate current realit

2011-02-10 Thread Magnus Hagander
On Thu, Feb 10, 2011 at 14:22, Robert Haas robertmh...@gmail.com wrote:
 On Thu, Feb 10, 2011 at 5:46 AM, Magnus Hagander mag...@hagander.net wrote:
 On Thu, Feb 10, 2011 at 07:04, Gurjeet Singh singh.gurj...@gmail.com wrote:
 This commit refers to www.mingw64.org which does not exist.

 Andrew fixed this alreayd.

 Also, clicking on the gitweb link below (from GMail), opens the browser
 window with an address where ';' are replaced with %3B , which leads to 404
 - no such project. Is GMail broken, or can have gitweb treat %3B as a ; ?

 This has also been fixed, per previous emails. I think GMail is not
 necessarily broken, but it changed behaviour to one that no other MUA
 has..

 I call that broken over here.

It'd doing URL escaping. AFAIK, it's not doing anything that's not
specifically allowed for in the HTML standard.

But arguing that point is pointless, given that the workaround seems
to work fine...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] keeping a timestamp of the last stats reset (for a db, table and function)

2011-02-10 Thread Magnus Hagander
On Sun, Feb 6, 2011 at 08:17, Greg Smith g...@2ndquadrant.com wrote:
 Tomas Vondra wrote:

 Because when I create a database, the field is
 NULL - that's true. But once I connect to the database, the stats are
 updated and the field is set (thanks to the logic in pgstat.c).


 OK--so it does what I was hoping for, I just didn't test it the right way.
  Let's call that a documentation issue and move on.

 Attached is an updated patch that fixes the docs and some other random bits.
  Looks ready for committer to me now.  Make sure to adjust
 PGSTAT_FILE_FORMAT_ID, do a cat version bump, and set final OIDs for the new
 functions.

... and the regression tests expected output.


 -Fixed some tab/whitespace issues.  It looks like you had tab stops set at 8

I added a few more whitespace fixes, mainly in the whitespace at end
of line category (git diff shows it pretty clearly)


With that, applied. Thanks!

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Update docs on building for Windows to accomodate current realit

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 8:38 AM, Magnus Hagander mag...@hagander.net wrote:
 On Thu, Feb 10, 2011 at 14:22, Robert Haas robertmh...@gmail.com wrote:
 On Thu, Feb 10, 2011 at 5:46 AM, Magnus Hagander mag...@hagander.net wrote:
 On Thu, Feb 10, 2011 at 07:04, Gurjeet Singh singh.gurj...@gmail.com 
 wrote:
 This commit refers to www.mingw64.org which does not exist.

 Andrew fixed this alreayd.

 Also, clicking on the gitweb link below (from GMail), opens the browser
 window with an address where ';' are replaced with %3B , which leads to 404
 - no such project. Is GMail broken, or can have gitweb treat %3B as a ; ?

 This has also been fixed, per previous emails. I think GMail is not
 necessarily broken, but it changed behaviour to one that no other MUA
 has..

 I call that broken over here.

 It'd doing URL escaping. AFAIK, it's not doing anything that's not
 specifically allowed for in the HTML standard.

 But arguing that point is pointless, given that the workaround seems
 to work fine...

All I know is that if right-click, open-link-in-new-tab works; then
clicking on the link directly ought to work, too.  Fail.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: pg_ctl failover Re: [HACKERS] Latches, signals, and waiting

2011-02-10 Thread Magnus Hagander
On Tue, Feb 8, 2011 at 05:24, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Jan 19, 2011 at 1:01 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Thu, Jan 13, 2011 at 9:08 PM, Fujii Masao masao.fu...@gmail.com wrote:
 I did s/failover/promote. Here is the updated patch.

 I rebased the patch to current git master.

 This patch looks fine to me.  I will mark it Ready for Committer.

 (Someone else please feel free to pick it up for the actual commit, if
 you have cycles.)

I see that the docs part of the patch removes the mentioning of
reporting servers - is that intentional, or a mistake? Seems that
usecase still remains, no?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread Alvaro Herrera
Excerpts from Peter Eisentraut's message of jue feb 10 07:58:16 -0300 2011:

 One thing I was thinking of was that we could add a global make
 maintainer-check target (a name I picked up from other projects) which
 would run various source code sanity checks.  Besides the SGML tabs
 issue, my favourite would be duplicate_oids.  Maybe if we could find a
 third use case, we'd have a quorum for implementing this.

The c++ headers thing?  Or the compiles-in-isolation test for headers?

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: pg_ctl failover Re: [HACKERS] Latches, signals, and waiting

2011-02-10 Thread Magnus Hagander
On Thu, Feb 10, 2011 at 15:25, Magnus Hagander mag...@hagander.net wrote:
 On Tue, Feb 8, 2011 at 05:24, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Jan 19, 2011 at 1:01 AM, Fujii Masao masao.fu...@gmail.com wrote:
 On Thu, Jan 13, 2011 at 9:08 PM, Fujii Masao masao.fu...@gmail.com wrote:
 I did s/failover/promote. Here is the updated patch.

 I rebased the patch to current git master.

 This patch looks fine to me.  I will mark it Ready for Committer.

 (Someone else please feel free to pick it up for the actual commit, if
 you have cycles.)

 I see that the docs part of the patch removes the mentioning of
 reporting servers - is that intentional, or a mistake? Seems that
 usecase still remains, no?

Also, the patch no longer applies, since it conflicts with
faa0550572583f51dba25611ab0f1d1c31de559b.

Since you (Fujii-san) wrote both of them, feel like rebasing it
properly for current master?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Erik Rijkers
On Wed, February 9, 2011 09:35, Jeff Davis wrote:
 Updated patch.


Thanks!

I just wanted to mention that this latest patch doesn't quite apply as-is, 
because of catversion
changes.

I've removed the change to catversion.h (18 lines, starting at 4985) from the 
patch file; then it
applies cleanly.


Erik Rijkers


 As always, my repo is here:

 http://git.postgresql.org/gitweb?p=users/jdavis/postgres.git;a=log;h=refs/heads/rangetypes

 Regards,
   Jeff Davis

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] postponing some large patches to 9.2

2011-02-10 Thread Robert Haas
On Tue, Feb 8, 2011 at 7:58 PM, Jeff Davis pg...@j-davis.com wrote:
 On the flip side, if we don't provide review to WIP patches during the
 3rd commitfest, how do we expect to get anything close to committable on
 the 1st commitfest of the next cycle?

I'm not sure exactly what you're going for here, because I don't think
I've ever proposed any special treatment of patches in the third
CommitFest, and as far as I can remember everything got reviewed
except for the two Tom promised to pick up and then sat on.  But if
you were to say that WIP patches *in general* get a lot less review
than non-WIP patches, I would agree with you.

To some extent, I think that's inevitable.  It's not fun to review WIP
patches.  When you come across something that's screwed up, you say to
yourself - I could mention this in the review, but maybe the author
already knows about it.  After all, it's WIP.  In other words, it's
hard to know what you should be looking for.  I've found that it's
nearly always better to post specific questions that you want to know
the answer to, rather than a patch where people have to guess what
parts you want feedback on.

Now, once the patch is code-complete, I think we should review it.
And I think we usually do a fairly good job with that, even as late as
CF3.  It's really CF4 where I think we get a bit less excited about
reviewing patches that aren't going to make the release anyway, and
that's not a great thing, but as procedural defects go it seems better
than most.  Yeah, there won't be a lot of big patches committed in
9.2CF1; we don't have the bandwidth to review major patches and get
betas out the door at the same time, or at least we haven't in the
past. But as long as the big patches that would have made 9.2CF1 still
make it into the release, that doesn't seem like a disaster.  On the
flip side, if a few more people want to step out and help get the open
items closed, I'd be more than happy to spend the time that I would
otherwise have spent on that reviewing major feature patches for 9.2,
where there's a CommitFest in progress or not.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_hostname and pg_stat_activity

2011-02-10 Thread Robert Haas
On Tue, Feb 1, 2011 at 1:33 PM, Robert Haas robertmh...@gmail.com wrote:
 On Tue, Feb 1, 2011 at 1:09 PM, Peter Eisentraut pete...@gmx.net wrote:
 On tis, 2011-01-18 at 19:24 -0500, Steve Singer wrote:
 However if I connect with a line in pg_hba that matches on an IP
 network then my client_hostname is always null unless log_hostname is
 set to true.  This is consistent with the behavior you describe but I
 think the average user will find it a bit confusing.  Having a column
 that is always null unless a GUC is set is less than ideal but I
 understand why log_hostname isn't on by default.

 Well, we have all these track_* variables, which also control what
 appears in the statistics views.

 After thinking about this some more, I think it might be better to be
 less cute and forget about the interaction with the pg_hba.conf hostname
 behavior.  That is, the host name is set if and only if log_hostname is
 on.

 +1 for doing it that way.

I think there are no outstanding issues with this patch of any
significance, so I'm marking it Ready for Committer.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] patches that could use additional reviewers

2011-02-10 Thread Robert Haas
On Wed, Feb 9, 2011 at 1:45 PM, Robert Haas robertmh...@gmail.com wrote:
 A few other ones that could use more reviewers include:

I've just corrected the status of a few patches in the CommitFest
application.  In particular, I set the following back to Needs Review.

SQL/MED - postgresql_fdw
Self-tuning checkpoint sync spread
determining client_encoding from client locale

If anyone can jump in, that would be great.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-02-10 Thread Robert Haas
On Tue, Feb 8, 2011 at 10:54 PM, Joachim Wieland j...@mcknight.de wrote:
 On Tue, Feb 8, 2011 at 8:31 PM, Itagaki Takahiro
 itagaki.takah...@gmail.com wrote:
 On Tue, Feb 8, 2011 at 13:34, Robert Haas robertmh...@gmail.com wrote:
 So how close are we to having a committable version of this?  Should
 we push this out to 9.2?

 I think so. The feature is pretty attractive, but more works are required:
  * Re-base on synchronized snapshots patch
  * Consider to use pipe also on Windows.
  * Research libpq + fork() issue. We have a warning in docs:
 http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html
 | On Unix, forking a process with open libpq connections can lead to
 unpredictable results

 Just for the records, once the sync snapshot patch is committed, there
 is no need to do fancy libpq + fork() combinations anyway.
 Unfortunately, so far no committer has commented on the synchronized
 snapshot patch at all.

 I am not fighting for getting parallel pg_dump done in 9.1, as I don't
 really have a personal use case for the patch. However it would be the
 irony of the year if we shipped 9.1 with a synchronized snapshot patch
 but no parallel dump  :-)

True.  But it looks like there are some outstanding items from
previous reviews that you've yet to address, which makes pushing it
out seem fairly reasonable...

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_hostname and pg_stat_activity

2011-02-10 Thread Steve Singer

On 11-02-10 10:13 AM, Robert Haas wrote:

On Tue, Feb 1, 2011 at 1:33 PM, Robert Haasrobertmh...@gmail.com  wrote:

On Tue, Feb 1, 2011 at 1:09 PM, Peter Eisentrautpete...@gmx.net  wrote:

On tis, 2011-01-18 at 19:24 -0500, Steve Singer wrote:

However if I connect with a line in pg_hba that matches on an IP
network then my client_hostname is always null unless log_hostname is
set to true.  This is consistent with the behavior you describe but I
think the average user will find it a bit confusing.  Having a column
that is always null unless a GUC is set is less than ideal but I
understand why log_hostname isn't on by default.

Well, we have all these track_* variables, which also control what
appears in the statistics views.

After thinking about this some more, I think it might be better to be
less cute and forget about the interaction with the pg_hba.conf hostname
behavior.  That is, the host name is set if and only if log_hostname is
on.

+1 for doing it that way.

I think there are no outstanding issues with this patch of any
significance, so I'm marking it Ready for Committer.


Was there an uodated version of this patch I missed?

The original patch needed some sort of documentation saying that having 
something showup in the new pg_stat_activity columns is controlled by 
log_hostname.


Above Peter and you seem to agree that having the having the line 
matched in pg_hba being a controlling factor should be removed but I 
haven't seen an updated patch that implements that.



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Adding new variables into GUC

2011-02-10 Thread Θάνος Παπαπέτρου

Hi,

I am an MSc student in the department of Informatics and  
Telecommunications of the University of Athens and as part of my  
thesis I am examining a new path/plan cost model for DB optimizers. I  
have successfully changed the optimizer of PostgreSQL in order to  
implement this model, but I have stumbled upon a very little detail:  
until now I use some hardcoded values in my code which I would like to  
make accessible through GUC. After much googling the only relative  
pages I have found are about configuring existing PostgreSQL variables  
and src/backend/utils/misc/README does not mention anything about  
adding new vars. Can anybody please provide some help?


I presume I have to add new entries in ConfigureNames* arrays in src/ 
backend/utils/misc/guc.c, but I am not sure about the field values of  
config_* structs I have to enter. Also is there any standard  
location where the actual variables should be declared, or I can  
freely declare them in any source file (the source file that actually  
uses them)? Do I have to declare a config_generic struct too? How do I  
specify boot values?


Thanks in advance,

Thanos Papapetrou

Θάνος Παπαπέτρου

You insist that there is something that a machine can't do. If you  
will tell me precisely what it is that a machine cannot do, then I can  
always make a machine which will do just that.


P.S.: I initially posted this message in pgsql-generic, but after  
thinking about it more I believe this is the most appropriate list, so  
I re-posted here.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_hostname and pg_stat_activity

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 10:22 AM, Steve Singer ssinger...@sympatico.ca wrote:
 On 11-02-10 10:13 AM, Robert Haas wrote:

 On Tue, Feb 1, 2011 at 1:33 PM, Robert Haasrobertmh...@gmail.com  wrote:

 On Tue, Feb 1, 2011 at 1:09 PM, Peter Eisentrautpete...@gmx.net  wrote:

 On tis, 2011-01-18 at 19:24 -0500, Steve Singer wrote:

 However if I connect with a line in pg_hba that matches on an IP
 network then my client_hostname is always null unless log_hostname is
 set to true.  This is consistent with the behavior you describe but I
 think the average user will find it a bit confusing.  Having a column
 that is always null unless a GUC is set is less than ideal but I
 understand why log_hostname isn't on by default.

 Well, we have all these track_* variables, which also control what
 appears in the statistics views.

 After thinking about this some more, I think it might be better to be
 less cute and forget about the interaction with the pg_hba.conf hostname
 behavior.  That is, the host name is set if and only if log_hostname is
 on.

 +1 for doing it that way.

 I think there are no outstanding issues with this patch of any
 significance, so I'm marking it Ready for Committer.

 Was there an uodated version of this patch I missed?

 The original patch needed some sort of documentation saying that having
 something showup in the new pg_stat_activity columns is controlled by
 log_hostname.

 Above Peter and you seem to agree that having the having the line matched in
 pg_hba being a controlling factor should be removed but I haven't seen an
 updated patch that implements that.

I was assuming those changes were sufficiently trivial that they could
be made at commit-time, especially if Peter is committing it himself.
Of course if he'd like a re-review, he can always post an updated
patch, but I just thought that was overly pedantic in this particular
case.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_hostname and pg_stat_activity

2011-02-10 Thread Steve Singer

On 11-02-10 10:32 AM, Robert Haas wrote:


I was assuming those changes were sufficiently trivial that they could
be made at commit-time, especially if Peter is committing it himself.
Of course if he'd like a re-review, he can always post an updated
patch, but I just thought that was overly pedantic in this particular
case.



Sounds reasonable.



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions versus pg_upgrade

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Tue, Feb 8, 2011 at 9:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 In contrast, ALTER EXTENSION ADD doesn't presuppose that you couldn't
 add the object to multiple extensions; and it has a natural inverse,
 ALTER EXTENSION DROP.  I am not necessarily suggesting that we will ever
 allow either of those things, but I do suggest that we should pick a
 syntax that doesn't look like it's being forced to conform if we ever
 want to do it.  The DROP case at least seems like it might be wanted
 in the relatively near future.

 Yep.

Actually, it occurs to me that the need for ALTER EXTENSION DROP could
be upon us sooner than we think.  The cases where an extension upgrade
script would need that are
(1) you want to remove some deprecated piece of the extension's API;
(2) you want to remove some no-longer-needed internal function.
Without ALTER EXTENSION DROP it's flat out impossible to do either.

Deprecated API is not exactly far to seek in our contrib modules,
either --- the example that just reminded me of this is hstore's =
operator, which we're already going so far as to print warnings about.
We're not going to get to remove that until at least one release after
we support ALTER EXTENSION DROP.

So I'm thinking it'd be smart to expend the small amount of additional
effort needed to support DROP right off the bat.  I think that
AlterExtensionAddStmt could be extended with an add/drop boolean for
a net addition of only a few dozen lines of code, most of that being a
suitable search-and-delete function in pg_depend.c.

Any objections?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions versus pg_upgrade

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 10:41 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Tue, Feb 8, 2011 at 9:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 In contrast, ALTER EXTENSION ADD doesn't presuppose that you couldn't
 add the object to multiple extensions; and it has a natural inverse,
 ALTER EXTENSION DROP.  I am not necessarily suggesting that we will ever
 allow either of those things, but I do suggest that we should pick a
 syntax that doesn't look like it's being forced to conform if we ever
 want to do it.  The DROP case at least seems like it might be wanted
 in the relatively near future.

 Yep.

 Actually, it occurs to me that the need for ALTER EXTENSION DROP could
 be upon us sooner than we think.  The cases where an extension upgrade
 script would need that are
 (1) you want to remove some deprecated piece of the extension's API;
 (2) you want to remove some no-longer-needed internal function.
 Without ALTER EXTENSION DROP it's flat out impossible to do either.

 Deprecated API is not exactly far to seek in our contrib modules,
 either --- the example that just reminded me of this is hstore's =
 operator, which we're already going so far as to print warnings about.
 We're not going to get to remove that until at least one release after
 we support ALTER EXTENSION DROP.

 So I'm thinking it'd be smart to expend the small amount of additional
 effort needed to support DROP right off the bat.  I think that
 AlterExtensionAddStmt could be extended with an add/drop boolean for
 a net addition of only a few dozen lines of code, most of that being a
 suitable search-and-delete function in pg_depend.c.

 Any objections?

No, I was pretty much just waiting for you to arrive at the same
conclusion I'd already reached.  :-)

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions versus pg_upgrade

2011-02-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 Actually, it occurs to me that the need for ALTER EXTENSION DROP could
 be upon us sooner than we think.  The cases where an extension upgrade
 script would need that are
 (1) you want to remove some deprecated piece of the extension's API;
 (2) you want to remove some no-longer-needed internal function.
 Without ALTER EXTENSION DROP it's flat out impossible to do either.

What if you just DROP FUNCTION in the upgrade script?

That said if the function is used in some expression index or worse,
triggers, you certainly want to give users the opportunity to delay the
step where the function is no more part of the extension from the step
where you get rid of it.

But if the function is implemented in C and the newer shared object has
removed it…

 So I'm thinking it'd be smart to expend the small amount of additional
 effort needed to support DROP right off the bat.  I think that
 AlterExtensionAddStmt could be extended with an add/drop boolean for
 a net addition of only a few dozen lines of code, most of that being a
 suitable search-and-delete function in pg_depend.c.

Given your phrasing about the size of this project, I can't see any
downside here.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions versus pg_upgrade

2011-02-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 Tom Lane t...@sss.pgh.pa.us writes:
 Actually, it occurs to me that the need for ALTER EXTENSION DROP could
 be upon us sooner than we think.  The cases where an extension upgrade
 script would need that are
 (1) you want to remove some deprecated piece of the extension's API;
 (2) you want to remove some no-longer-needed internal function.
 Without ALTER EXTENSION DROP it's flat out impossible to do either.

 What if you just DROP FUNCTION in the upgrade script?

That would be rejected because you're not allowed to drop an individual
member object of an extension.  (And no, I don't want to have a kluge in
dependency.c that makes that test work differently when
creating_extension.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding new variables into GUC

2011-02-10 Thread Tom Lane
=?ISO-8859-7?B?yNzt7/Ig0OHw4fDd9PHv9Q==?= tha...@di.uoa.gr writes:
 I am an MSc student in the department of Informatics and  
 Telecommunications of the University of Athens and as part of my  
 thesis I am examining a new path/plan cost model for DB optimizers. I  
 have successfully changed the optimizer of PostgreSQL in order to  
 implement this model, but I have stumbled upon a very little detail:  
 until now I use some hardcoded values in my code which I would like to  
 make accessible through GUC. After much googling the only relative  
 pages I have found are about configuring existing PostgreSQL variables  
 and src/backend/utils/misc/README does not mention anything about  
 adding new vars. Can anybody please provide some help?

You basically need the variable, the entry in the appropriate array in
guc.c, and some documentation (at least if you'd like anyone else to
ever use the code).  Try looking at some past patches that added GUCs
similar to yours.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions versus pg_upgrade

2011-02-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 That would be rejected because you're not allowed to drop an individual
 member object of an extension.  (And no, I don't want to have a kluge in
 dependency.c that makes that test work differently when
 creating_extension.)

Fair enough, all the more as soon as we have ALTER EXTENSION DROP :)

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Tom Lane
Erik Rijkers e...@xs4all.nl writes:
 On Wed, February 9, 2011 09:35, Jeff Davis wrote:
 Updated patch.

 I just wanted to mention that this latest patch doesn't quite apply as-is, 
 because of catversion changes.

Just a note: standard practice is for submitted patches to *not* touch
catversion.h.  The committer will add that change before committing.
Otherwise, it's just guaranteed to cause merge problems such as this
one.  (It's not unreasonable to mention the need for a catversion bump
in the description of the patch, if you think the committer might not
realize it.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Jeff Davis
On Thu, 2011-02-10 at 12:04 -0500, Tom Lane wrote:
 Erik Rijkers e...@xs4all.nl writes:
  On Wed, February 9, 2011 09:35, Jeff Davis wrote:
  Updated patch.
 
  I just wanted to mention that this latest patch doesn't quite apply as-is, 
  because of catversion changes.
 
 Just a note: standard practice is for submitted patches to *not* touch
 catversion.h.  The committer will add that change before committing.
 Otherwise, it's just guaranteed to cause merge problems such as this
 one.  (It's not unreasonable to mention the need for a catversion bump
 in the description of the patch, if you think the committer might not
 realize it.)

OK, I'll remove that then.

I originally put it there so that I wouldn't mix up data directories
with a patch I'm reviewing, but I agree that it seems easier this way.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Peter Eisentraut
On tor, 2011-02-10 at 09:28 -0800, Jeff Davis wrote:
 I originally put it there so that I wouldn't mix up data directories
 with a patch I'm reviewing, but I agree that it seems easier this way.

FWIW, I disagree with Tom and do recommend putting the catversion change
in the patch.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread Robert Haas
On Wed, Feb 9, 2011 at 2:09 AM, Jeff Davis pg...@j-davis.com wrote:
 That's how arrays do it: there's a special Expr node that represents an
 array expression. Maybe the same thing could be used for range types,
 but I fear that there may be some grammar conflicts. I doubt we'd want
 to fully reserve the keyword range.

According to our documentation[1], RANGE is reserved in SQL:2008 and
SQL:2003, which makes it more imaginable to reserve it than it would
be otherwise.  I believe that in a previous email you mentioned that
you were hoping to implement RANGE JOIN, and I will just note that the
restrictions of the grammar require that any keyword that immediately
follows the previous expression and precedes JOIN must be fully
reserved.  I'm not sure if you meant that a range join would literally
use the syntax RANGE JOIN, but if so then you're going to have to
argue for fully reserving RANGE anyway, in which case there'd be no
special reason not to allow RANGE [1,10) to mean just that.  On the
other hand, if a RANGE JOIN just means a regular join on some funky
operator, and there's no other reason to reserve range, I wouldn't do
it just to get a nicer syntax here.

Have you done investigation of what RANGE is used to mean in the SQL
spec?  Is what you're implementing (a) spec, (b) similar idea, but not
the spec, or (c) something completely different?  I'm guessing (c) but
I have no idea what the spec is using it for.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

[1] http://www.postgresql.org/docs/current/static/sql-keywords-appendix.html

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Jeff Davis
On Thu, 2011-02-10 at 15:38 +0100, Erik Rijkers wrote:
 I've removed the change to catversion.h (18 lines, starting at 4985) from the 
 patch file; then it
 applies cleanly.

I should mention that the last patch changed the representation to be
more compact. So, if you have any existing test data it will need to be
reloaded to work with the latest.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Heikki Linnakangas

On 10.02.2011 20:01, Peter Eisentraut wrote:

On tor, 2011-02-10 at 09:28 -0800, Jeff Davis wrote:

I originally put it there so that I wouldn't mix up data directories
with a patch I'm reviewing, but I agree that it seems easier this way.


FWIW, I disagree with Tom and do recommend putting the catversion change
in the patch.


I'm very bad at remembering to bump it, so I also won't mind patch 
authors doing it.


The ideal reminder would be some special comment you could put on the 
catversion line that would cause git push to fail if it's still there 
when I try to push the commit to the repository. There doesn't seem to 
be a pre-push hook in git, although some googling suggests that it 
would be quite easy to write a small wrapper shell script to check that. 
I'm seriously considering to do that, given that I more often forget to 
bump catversion than not.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] postponing some large patches to 9.2

2011-02-10 Thread Jeff Davis
On Thu, 2011-02-10 at 09:46 -0500, Robert Haas wrote:
 On Tue, Feb 8, 2011 at 7:58 PM, Jeff Davis pg...@j-davis.com wrote:
  On the flip side, if we don't provide review to WIP patches during the
  3rd commitfest, how do we expect to get anything close to committable on
  the 1st commitfest of the next cycle?
 
 I'm not sure exactly what you're going for here, because I don't think
 I've ever proposed any special treatment of patches in the third
 CommitFest,

I actually meant 4th (this one). I forgot that the July one was actually
a part of the 9.1 cycle.

 But if
 you were to say that WIP patches *in general* get a lot less review
 than non-WIP patches, I would agree with you.
 
 To some extent, I think that's inevitable.  It's not fun to review WIP
 patches.

Agreed, but it doesn't really apply to this situation.

There was still a week left, and the reviewer was still reviewing. So I
found it jarring when you said that it had received enough review, and
bounced it.

In my opinion, if we're going to entertain WIP patches during a
commitfest, we shouldn't bounce them early for being WIP. We can bounce
them for other causes, like waiting on author or we couldn't find a
reviewer or we're out of time.

 I've found that it's
 nearly always better to post specific questions that you want to know
 the answer to, rather than a patch where people have to guess what
 parts you want feedback on.

Well, I've certainly posted some specific questions. I don't expect to
get an answer to all of them right away, and certainly many have been
answered -- but I didn't just throw the code out and wait.

For instance:
http://archives.postgresql.org/message-id/1297230650.27157.398.camel@jdavis


Anyway, I don't think any of this affected the patch, I was just
surprised. I'll leave it at that, because I'm sure you're busy wrapping
up this commitfest.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
I spent some time reviewing this thread.  I think the major point that's
not received adequate discussion is this: the design assumes that there's
just one current version of any extension, and that's not good enough.
David Fetter was trying to make that point upthread but didn't seem to
convince people.  I'm convinced though.  I think that one of the major
selling points of extensions could be having a controlled way of exposing
different versions of an API and letting users select which one is in use
in each database.  Look at how much effort we spend maintaining back
branches of the core code for people who don't want to, eg, update their
apps to avoid pre-8.3-style implicit casting.  (Yeah, I know that on-disk
compatibility is another major reason for staying on a back branch, but
API changes are definitely part of it.)

So I believe that it'd be a good idea if it were possible for an extension
author to distribute a package that implements, say, versions 1.0, 1.1,
and 2.0 of hstore.  Not all will choose to do the work needed for that, of
course, and that's fine.  But the extension mechanism ought to permit it.
Over time we might get to a point where somebody could be running the
latest version of the core database (with all the bug fixes and other
goodness of that) but his application compatibility problems are solved
by running back-rev versions of certain extensions.

To do this, we need to remove the concept that the control file specifies
the version of an extension; rather the version is associated with the
SQL script file.  I think we should embed the version number in the script
file name, and require one to be present (no more omitted version
numbers).  So you would distribute, say,
hstore-1.0.sql
hstore-1.1.sql
hstore-2.0.sql
representing the scripts needed to install these three versions from
scratch.  CREATE EXTENSION would have an option to select which
version to install.  If the option is omitted, there are at least two
things we could do:
1. Choose the newest available version.
2. Let the control file specify which version is the default.
I think I prefer #2 because it avoids needing a rule for comparing
version identifiers, and it caters to the possibility that the newest
version isn't yet mature enough to be a good default.

As for upgrades, let's just expect upgrade scripts to be named
extension-oldversion-newversion.sql.  ALTER EXTENSION UPGRADE knows the
relevant oldversion from pg_extension, and newversion can be handled the
same way as in CREATE, ie, either the user says which version to update to
or we use the default version from the control file.

I don't seriously expect most extension authors to bother preparing
upgrade scripts for any cases except adjacent pairs of versions.
That means that if a user comes along and wants to upgrade across several
versions of the extension, he'll have to do it in several steps:
ALTER EXTENSION hstore UPGRADE TO '1.1';
ALTER EXTENSION hstore UPGRADE TO '2.0';
ALTER EXTENSION hstore UPGRADE TO '2.1';
I don't see that as being a major problem --- how often will people have
the need to do that, anyway?  Authors who feel that it is a big deal can
expend the work to provide shortcut scripts.  I do not see adequate return
on investment from the regexp-matching complications in the currently
submitted patch.

In this scheme, all the extension scripts are independent.  We spent quite
a lot of time arguing about ways to avoid duplication of code between
scripts, but frankly I'm not convinced that that's worth troubling over.
As far as the initial-install scripts go, once you've released 1.0 it's
unlikely you'll ever change it again, so the fact that you copied and
pasted it as a starting point for 1.1 isn't really a maintenance burden.
Version upgrade scripts won't share any code at all, unless the author is
trying to provide shortcut scripts for multi-version jumps, and as I said,
I doubt that many will bother.  Also, it'll be some time before there's
much need for multi-version update scripts anyway, so I am not feeling
that it is necessary to solve that now.  We could later on add some kind
of script inclusion capability to allow authors to avoid code duplication
in multi-version update scripts, but it's just not urgent.

So, concrete proposal is to enforce the extension-version.sql and
extension-oldversion-newversion.sql naming rules for scripts, which
means getting rid of the script name parameter in control files.
(Instead, we could have a directory parameter that tells which directory
holds all the install and upgrade scripts for the extension.)  Also, the
version parameter should be renamed to something like current_version
or default_version.  We also have to be wary of whether any other
control-file parameters specify something that might be version-specific.
Looking at the current list:

comment: probably OK to consider this as a default for all versions.
We already have the 

Re: [HACKERS] Range Types (catversion.h)

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 1:23 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 On 10.02.2011 20:01, Peter Eisentraut wrote:

 On tor, 2011-02-10 at 09:28 -0800, Jeff Davis wrote:

 I originally put it there so that I wouldn't mix up data directories
 with a patch I'm reviewing, but I agree that it seems easier this way.

 FWIW, I disagree with Tom and do recommend putting the catversion change
 in the patch.

 I'm very bad at remembering to bump it, so I also won't mind patch authors
 doing it.

 The ideal reminder would be some special comment you could put on the
 catversion line that would cause git push to fail if it's still there when
 I try to push the commit to the repository. There doesn't seem to be a
 pre-push hook in git, although some googling suggests that it would be
 quite easy to write a small wrapper shell script to check that. I'm
 seriously considering to do that, given that I more often forget to bump
 catversion than not.

And I share Tom's preference, which is to not include it, because I
usually apply patches using patch, and when diff hunks fail it's a
nuisance for me.

So basically, do whatever you want, someone won't like it no matter what.  :-)

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread David Fetter
On Thu, Feb 10, 2011 at 12:58:16PM +0200, Peter Eisentraut wrote:
 On ons, 2011-02-09 at 08:00 -0800, David Fetter wrote:
  On Wed, Feb 09, 2011 at 01:17:06PM +, Bruce Momjian wrote:
   Remove more SGML tabs.
  
  Perhaps we should see about putting something in .git/hooks/pre-commit
  so people can focus on more substantive matters.
  
  Is there some kind of cross-platform way to do this?  I'm thinking
  that given the fact that our build system already requires Perl, there
  should be, but I'm not quite sure how this would be accomplished.
 
 There is make check target in doc/src/sgml/ that is supposed to catch
 this.  But it's probably hard to remember to run that.

It's check-tabs, and if you hadn't mentioned it, I'd never have seen it.

 One thing I was thinking of was that we could add a global make
 maintainer-check target (a name I picked up from other projects) which
 would run various source code sanity checks.  Besides the SGML tabs
 issue, my favourite would be duplicate_oids.  Maybe if we could find a
 third use case, we'd have a quorum for implementing this.

I think all such checks belong in .git/hooks/pre-commit, and need to
be as cross-platform as needed for committers.  Would a *n*x-based
version do for a start?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread Jeff Davis
On Thu, 2011-02-10 at 13:07 -0500, Robert Haas wrote:
 According to our documentation[1], RANGE is reserved in SQL:2008 and
 SQL:2003, which makes it more imaginable to reserve it than it would
 be otherwise.

Oh, interesting.

 I believe that in a previous email you mentioned that
 you were hoping to implement RANGE JOIN, and I will just note that the
 restrictions of the grammar require that any keyword that immediately
 follows the previous expression and precedes JOIN must be fully
 reserved.  I'm not sure if you meant that a range join would literally
 use the syntax RANGE JOIN, but if so then you're going to have to
 argue for fully reserving RANGE anyway, in which case there'd be no
 special reason not to allow RANGE [1,10) to mean just that.  On the
 other hand, if a RANGE JOIN just means a regular join on some funky
 operator, and there's no other reason to reserve range, I wouldn't do
 it just to get a nicer syntax here.

It's mostly just a regular join on a funky operator. We may want that
operator to allow a new plan (range merge join); but I think we can
determine that it's a range join from the use of the operator. I'll have
to look into that more.

 Have you done investigation of what RANGE is used to mean in the SQL
 spec?  Is what you're implementing (a) spec, (b) similar idea, but not
 the spec, or (c) something completely different?  I'm guessing (c) but
 I have no idea what the spec is using it for.

(c) was my intention. I did take a brief look at the spec a while back,
but I'll take a more detailed look. I think it only has to do with
window specifications.

This might solve the constructor problem nicely if we could do things
like:
  RANGE[10,20)
But I have a feeling that will either cause a bizarre problem with the
grammar, or someone will think it's not very SQL-like.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 1:41 PM, Jeff Davis pg...@j-davis.com wrote:
 This might solve the constructor problem nicely if we could do things
 like:
  RANGE[10,20)
 But I have a feeling that will either cause a bizarre problem with the
 grammar, or someone will think it's not very SQL-like.

I think won't cause any problem at all if RANGE is fully reserved, but
like you say we probably don't want to do that unless it's absolutely
necessary, and if you don't actually need to be able to type in foo
RANGE JOIN bar then it probably isn't.

I think your proposed naming schema for constructors is pretty
reasonable, except I might use o for open and c for closed rather
than i and _, i.e. range_oo(), range_oc(), range_co(), range_cc().
 If that'll get us by without fully reserving RANGE then I'd certainly
be in favor of doing it that way.  I was just saying - if we were
inevitably going to have to reserve RANGE, then we could try to
squeeze a little more out of it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 10:26 AM, Tom Lane wrote:

   1. Choose the newest available version.
   2. Let the control file specify which version is the default.
 I think I prefer #2 because it avoids needing a rule for comparing
 version identifiers, and it caters to the possibility that the newest
 version isn't yet mature enough to be a good default.

+1. I assume there will be some way to build versioned shared object libraries 
too, then?

 In this scheme, all the extension scripts are independent.  We spent quite
 a lot of time arguing about ways to avoid duplication of code between
 scripts, but frankly I'm not convinced that that's worth troubling over.
 As far as the initial-install scripts go, once you've released 1.0 it's
 unlikely you'll ever change it again, so the fact that you copied and
 pasted it as a starting point for 1.1 isn't really a maintenance burden.

I disagree with this. A lot of dynamic language libraries never get to 1.0, and 
even if they do can go through periods of extensive development with major 
changes from version to version. Just have a look at the pgTAP changes file for 
an example:

  https://github.com/theory/pgtap/blob/master/Changes

I already do a *lot* of work in the Makefile to patch things so that it works 
all the way back to 8.0. And I'm adding stuff now to generate other files that 
will contain a subset of the pgTAP functionality. I don't think I'd ever write 
upgrade scripts for pgTAP, but I've worked with a lot of Perl modules that have 
followed similar aggressive development, and can imagine times when I'd need to 
write upgrade scripts for aggressively-developed PostgreSQL extensions. And I 
quail at the idea. Lord help me if I'd need to also write create patches for my 
upgrade scripts to support older versions of PostgreSQL.

 Version upgrade scripts won't share any code at all, unless the author is
 trying to provide shortcut scripts for multi-version jumps, and as I said,
 I doubt that many will bother.  Also, it'll be some time before there's
 much need for multi-version update scripts anyway, so I am not feeling
 that it is necessary to solve that now.  We could later on add some kind
 of script inclusion capability to allow authors to avoid code duplication
 in multi-version update scripts, but it's just not urgent.

Okay, that would be a big help. And I'm fine with it being something to maybe 
be added later. We'll see then what cow paths develop, and demands for pasture 
fences to be cut down. Or something.

 So, concrete proposal is to enforce the extension-version.sql and
 extension-oldversion-newversion.sql naming rules for scripts, which
 means getting rid of the script name parameter in control files.
 (Instead, we could have a directory parameter that tells which directory
 holds all the install and upgrade scripts for the extension.)

+1 I like this idea. I'm already putting all my scripts into an sql/ directory 
for PGXN distributions:

  https://github.com/theory/pg-semver

 encoding: I don't see any big problem with insisting that all scripts for
 a given extension be in the same encoding.

+1. Also, can't one set client_encoding in the scripts anyway?

 requires, relocatable and schema: These are problematic, because it's not
 out of the question that someone might want to change these properties
 from one version to another.  But as things are currently set up, we must
 know these things before we start to run the extension script, because
 they are needed to set up the search_path correctly.
 
 Perhaps for now it's sufficient to say that these properties can't change
 across versions.  Alternatively, we could allow there to be a secondary
 version-specific control file that can override the main control file.
 IOW, we'd read extension.control to get the directory and
 default_version values, then determine the version we are installing or
 upgrading to, then see if there's an extension-version.control file
 in the extension's directory, and if so read that and let it replace
 the remaining parameters' values.

+1.

I'll need to play around with some of this stuff to see how it affects PGXN 
distributions. My main concern will be allowing an extension distribution to 
somehow work both on 9.1 with EXTENSIONs and in  9.0 as PGXS-installed modules 
currently work, without too much pain to the developer to support previous 
versions of PostgreSQL.

Best,

David



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding new variables into GUC

2011-02-10 Thread Josh Berkus

 You basically need the variable, the entry in the appropriate array in
 guc.c, and some documentation (at least if you'd like anyone else to
 ever use the code).  Try looking at some past patches that added GUCs
 similar to yours.

For completeness, it would also be good to add rows to the pg_settings
system catalog, but that's not necessary for testing.

-- 
  -- Josh Berkus
 PostgreSQL Experts Inc.
 http://www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 1:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 requires, relocatable and schema: These are problematic, because it's not
 out of the question that someone might want to change these properties
 from one version to another.  But as things are currently set up, we must
 know these things before we start to run the extension script, because
 they are needed to set up the search_path correctly.

My biggest concern with this extensions work is that these variables
are poorly designed.  The extension mechanism is basically the
equivalent of RPM for inside the database.  And while in theory there
is such a thing as a relocatable RPM, I don't know that I've ever used
it, at least not successfully.  I'm worried this is going to be a
pretty serious rough edge that's difficult to file down later.
Forcing everything into a single schema (like pg_extension) seems a
bit too draconian, but this idea that you can install things wherever
you like and somehow it's gonna just work seems pretty optimistic.

However, that's a side point.  The overall design you propose seems
reasonable to me.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding new variables into GUC

2011-02-10 Thread Robert Haas
2011/2/10 Josh Berkus j...@agliodbs.com:

 You basically need the variable, the entry in the appropriate array in
 guc.c, and some documentation (at least if you'd like anyone else to
 ever use the code).  Try looking at some past patches that added GUCs
 similar to yours.

 For completeness, it would also be good to add rows to the pg_settings
 system catalog, but that's not necessary for testing.

pg_settings doesn't need to be separately updated.  It's just a view.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Sync Rep for 2011CF1

2011-02-10 Thread Heikki Linnakangas

On 08.02.2011 20:53, Robert Haas wrote:

That having been said, there is at least one part of this patch which
looks to be in pretty good shape and seems independently useful
regardless of what happens to the rest of it, and that is the code
that sends replies from the standby back to the primary.  This allows
pg_stat_replication to display the write/flush/apply log positions on
the standby next to the sent position on the primary, which as far as
I am concerned is pure gold.  Simon had this set up to happen only
when synchronous replication or XID feedback in use, but I think
people are going to want it even with plain old asynchronous
replication, because it provides a FAR easier way to monitor standby
lag than anything we have today.  I've extracted this portion of the
patch, cleaned it up a bit, written docs, and attached it here.


Thanks!


I wasn't too sure how to control the timing of the replies.  It's
worth noting that you have to send them pretty frequently for the
distinction between xlog written and xlog flushed to have any value.
What I've done here is made it so that every time we read all
available data on the socket, we send a reply.  After flushing, we
send another reply.  And then just for the heck of it we send a reply
at least every 10 seconds (configurable), which causes the
last-known-apply position to eventually get updated on the master.
This means the apply position can lag reality by a bit.


Seems reasonable. As the patch stands, however, the standby doesn't send 
any status updates if its busy receiving, writing, and flushing the 
incoming WAL. That would happen if you have a fast network, and slow 
disk, and the standby is catching up, e.g after restoring a base backup.


I added a XLogWalRcvSendReply() call into XLogWalRcvFlush() so that it 
also sends a status update every time the WAL is flushed. If the 
walreceiver is busy receiving and flushing, that would happen once per 
WAL segment, which seems sensible.


The comment above StandbyReplyMessage said that its message type is 'r'. 
However, no message type was actually sent for the replies. A message 
type byte seems like a good idea, for the sake of extensibility, so I 
made the code match that comment. I also added documentation of this new 
message type in the manual section about the streaming replication protocol.


I committed the patch with those changes, and some minor comment tweaks 
and other kibitzing.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread Florian Pflug
 This might solve the constructor problem nicely if we could do things
 like:
  RANGE[10,20)
 But I have a feeling that will either cause a bizarre problem with the
 grammar, or someone will think it's not very SQL-like.


It will certainly mess up syntax highlighting and matching bracket detection
in pretty much all text editors...

best regards,
Florian Pflug


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread Peter Eisentraut
On tor, 2011-02-10 at 10:40 -0800, David Fetter wrote:
 I think all such checks belong in .git/hooks/pre-commit, and need to
 be as cross-platform as needed for committers.  Would a *n*x-based
 version do for a start?

I think as a matter of principle, the only things that belongs into git
hooks are things that relate to the repository itself, such as commit
metadata (author checks, commit message spell checks, etc.).  Anything
that relates to the source belongs into the source.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread Tom Lane
Florian Pflug f...@phlo.org writes:
 This might solve the constructor problem nicely if we could do things
 like:
 RANGE[10,20)
 But I have a feeling that will either cause a bizarre problem with the
 grammar, or someone will think it's not very SQL-like.

 It will certainly mess up syntax highlighting and matching bracket detection
 in pretty much all text editors...

Yeah.  It's a cute-looking notation but surely it will cause many more
problems than it's worth.  I agree with Robert's suggestion of plain
functions named like range_co() etc.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-10 Thread Peter Eisentraut
On sön, 2011-02-06 at 20:44 +0100, Jan Urbański wrote:
 On 27/01/11 23:24, Jan Urbański wrote:
  On 11/01/11 12:20, Jan Urbański wrote:
  On 11/01/11 01:27, Tom Lane wrote:
  Hannu Krosing ha...@2ndquadrant.com writes:
  On 10.1.2011 17:20, Jan Urbański wrote:
  I changed that patch to use Perl instead of sed to generate the
  exceptions, which should be a more portable.
  
  Updated as an incremental patch on to of the recently sent version of
  explicit-subxacts.
 
 Updated again.

Why do the error messages print spiexceptions.SyntaxError instead of
plpy.spiexceptions.SyntaxError?  Is this intentional or just the way it
comes out of Python?

Please add some documentation.  Not a list of all exceptions, but at
least a paragraph that various kinds of specific exceptions may be
generated, what package and module they are in, and how they relate.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes:
 +1. I assume there will be some way to build versioned shared object 
 libraries too, then?

I'm not really addressing that in this proposal.  You could imagine
supporting all the extension versions in one .so, or you could have one
per version (meaning the upgrade scripts would have to CREATE OR REPLACE
all the C functions to re-point them at a different .so), or mixed
cases.  Right now the PGXS infrastructure would favor the first because
it has only limited ability to build multiple .so's in one directory;
but we could think about improving that if there's demand.

Note that you can version a function even within a single .so, for
example if hstore 1.0 defines foo() one way and hstore 1.1 defines
it another, you could make the latter point to the C function name
foo_1_1 while C function foo continues to provide the old behavior.
You have to at least provide a stub foo (that could just throw error
if called) for as long as you want to support upgrading from 1.0.

 In this scheme, all the extension scripts are independent.  We spent quite
 a lot of time arguing about ways to avoid duplication of code between
 scripts, but frankly I'm not convinced that that's worth troubling over.
 As far as the initial-install scripts go, once you've released 1.0 it's
 unlikely you'll ever change it again, so the fact that you copied and
 pasted it as a starting point for 1.1 isn't really a maintenance burden.

 I disagree with this. A lot of dynamic language libraries never get to
 1.0, and even if they do can go through periods of extensive development
 with major changes from version to version.

I don't see how that affects my point?  You can spell 1.0 as 0.1
and 1.1 as 0.2 if you like that kind of numbering, but I don't
see that that has any real impact.  At the end of the day an author is
going to crank out a series of releases, and if he cares about people
using those releases for production, he's going to have to provide at
least a upgrade script to move an existing database from release N to
release N+1.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Adding new variables into GUC

2011-02-10 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes:
 You basically need the variable, the entry in the appropriate array in
 guc.c, and some documentation (at least if you'd like anyone else to
 ever use the code).  Try looking at some past patches that added GUCs
 similar to yours.

 For completeness, it would also be good to add rows to the pg_settings
 system catalog, but that's not necessary for testing.

Huh?  pg_settings is a dynamically generated view.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 11:31 AM, Tom Lane wrote:

 I'm not really addressing that in this proposal.  You could imagine
 supporting all the extension versions in one .so, or you could have one
 per version (meaning the upgrade scripts would have to CREATE OR REPLACE
 all the C functions to re-point them at a different .so), or mixed
 cases.  Right now the PGXS infrastructure would favor the first because
 it has only limited ability to build multiple .so's in one directory;
 but we could think about improving that if there's demand.
 
 Note that you can version a function even within a single .so, for
 example if hstore 1.0 defines foo() one way and hstore 1.1 defines
 it another, you could make the latter point to the C function name
 foo_1_1 while C function foo continues to provide the old behavior.
 You have to at least provide a stub foo (that could just throw error
 if called) for as long as you want to support upgrading from 1.0.

Good enough for me.

 I don't see how that affects my point?  You can spell 1.0 as 0.1
 and 1.1 as 0.2 if you like that kind of numbering, but I don't
 see that that has any real impact.  At the end of the day an author is
 going to crank out a series of releases, and if he cares about people
 using those releases for production, he's going to have to provide at
 least a upgrade script to move an existing database from release N to
 release N+1.

Yeah, but given a rapidly-developing extension, that could create a lot of 
extra work. I don't know that there's much of a way around that, other than 
concatenating files to build migration scripts from parts (perhaps via `Make` 
as dim suggested). But it can get complicated pretty fast. My desire here is to 
keep the barrier to creating PostgreSQL extensions as low as is reasonably 
possible.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 10:41 AM, Jeff Davis wrote:

 This might solve the constructor problem nicely if we could do things
 like:
  RANGE[10,20)
 But I have a feeling that will either cause a bizarre problem with the
 grammar, or someone will think it's not very SQL-like.

I like it a lot better than the funkily-named functions you posted yesterday.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-02-10 Thread Andrew Dunstan



On 02/10/2011 08:15 AM, Alexey Klyukin wrote:

On Feb 9, 2011, at 9:28 PM, Alex Hunsaker wrote:


On Wed, Feb 9, 2011 at 08:24, Alexey Klyukinal...@commandprompt.com  wrote:

What was actually broken in encode_array_literal support of composite types
(it converted perl hashes to the literal composite-type constants, expanding
nested arrays along the way) ? I think it would be a useful extension of the
existing encode_array_literal.

Yeah, It does not work because it did not take into account the order
of composite columns. It always put them alphabetically by column
name. To do it properly we would need to pass in a typid or a column
order or something. Ideally we could expose the new
plperl_array_to_datum() to plperl functions in some manner.

Damn, right. Each perl hash corresponds to multiple composite types, different
by the order of the type elements. Passing the typid sounds like a fair
requirement (and if it's missing we could assume that the order of columns in
composites doesn't matter to the caller).

Let me try implementing that as an XS interface to plperl_array_to_datum.



Are you intending this as a completion of the current patch or as 9.2 
work? If the former you need to send it in real fast.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 1:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 requires, relocatable and schema: These are problematic, because it's not
 out of the question that someone might want to change these properties
 from one version to another.  But as things are currently set up, we must
 know these things before we start to run the extension script, because
 they are needed to set up the search_path correctly.

 My biggest concern with this extensions work is that these variables
 are poorly designed.

Yeah, I didn't especially like relocatable/schema either.  I thought for
awhile about redefining relocatable as a three-way switch, corresponding
to the three use cases (relocatable after the fact, relocatable only at
initial install, no relocation) but didn't pull the trigger.  It is
advantageous to have an explicit notion of a particular schema
containing the extension's exported stuff, so that we can add that
schema into the search path for dependent extensions.  That means that
you can't easily remove the explicit schema value for the third case,
so it's not that easy to make it look cleaner.

 The extension mechanism is basically the
 equivalent of RPM for inside the database.  And while in theory there
 is such a thing as a relocatable RPM, I don't know that I've ever used
 it, at least not successfully.

General opinion around Red Hat is relocatable RPMs don't work.  But
pushing a set of functions from one schema to another is a very much
narrower problem than what an RPM has to deal with, so I'm not convinced
that the analogy holds.

Now, if you want to argue that moving an extension after the fact (ALTER
EXTENSION SET SCHEMA) is so dangerous as to be useless, I wouldn't
argue very hard.  Do you want to propose ripping that out?  But
relocating at first install doesn't seem horrible.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Range Type constructors

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 11:21 AM, Tom Lane wrote:

 It will certainly mess up syntax highlighting and matching bracket detection
 in pretty much all text editors...
 
 Yeah.  It's a cute-looking notation but surely it will cause many more
 problems than it's worth.  I agree with Robert's suggestion of plain
 functions named like range_co() etc.

I could see myself using ranges in ways similar to using arrays. It should 
would be nice to have the equivalent of ARRAY[] and/or ARRAY() to use for 
ranges…

Best,

David
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes:
 On Feb 10, 2011, at 11:31 AM, Tom Lane wrote:
 I don't see how that affects my point?  You can spell 1.0 as 0.1
 and 1.1 as 0.2 if you like that kind of numbering, but I don't
 see that that has any real impact.  At the end of the day an author is
 going to crank out a series of releases, and if he cares about people
 using those releases for production, he's going to have to provide at
 least a upgrade script to move an existing database from release N to
 release N+1.

 Yeah, but given a rapidly-developing extension, that could create a lot of 
 extra work. I don't know that there's much of a way around that, other than 
 concatenating files to build migration scripts from parts (perhaps via `Make` 
 as dim suggested). But it can get complicated pretty fast. My desire here is 
 to keep the barrier to creating PostgreSQL extensions as low as is reasonably 
 possible.

Oh, I see, you're just saying that it's not unlikely somebody could find
himself with dozens of minor releases all being supported.  Yeah, he'd
then really need to provide shortcut upgrade scripts, and
building/maintaining those would be a pain.

The design as I sketched it didn't need to make any assumptions at all
about the meaning of the version identifiers.  But if you were willing
to assume that the identifiers are comparable/sortable by some rule,
then it wouldn't be that hard for ALTER EXTENSION UPGRADE to figure out
how to chain a series of upgrade scripts together to get from A to B,
and then there would be no need for manual maintenance of shortcut
scripts.  IIRC the main objection to doing it that way was that the
underlying .so has to be compatible (at least to the extent of allowing
CREATE OR REPLACE FUNCTION) with all the intermediate versions --- but
if you believe the use-case I'm arguing for, that would be wanted
anyway, because all the intermediate versions would be considered
potentially useful stopping points.

I'm not philosophically opposed to requiring the version numbers to be
sortable, I just didn't want to introduce the concept if we didn't have
to.  But maybe automatic application of a series of upgrade scripts is
enough reason.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 2:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 General opinion around Red Hat is relocatable RPMs don't work.  But
 pushing a set of functions from one schema to another is a very much
 narrower problem than what an RPM has to deal with, so I'm not convinced
 that the analogy holds.

 Now, if you want to argue that moving an extension after the fact (ALTER
 EXTENSION SET SCHEMA) is so dangerous as to be useless, I wouldn't
 argue very hard.  Do you want to propose ripping that out?  But
 relocating at first install doesn't seem horrible.

I'm not very concerned about letting people set the schema after the
fact.  If we think it's OK for them to whack the location around at
first install, I don't know why we shouldn't also let them whack it
around later.  The question I have is whether it's really reasonable
to let extension-owned objects be moved around at all.  It'll probably
work fine as long as there are no other extensions depending on the
one that's getting moved, but it doesn't pay to design for the trivial
case.  The real issue is what happens when you want to install
extension A, which depends on extensions B, C, and D, and B, C, and D
are all in non-standard locations.  Does that have any chance of
working under the system we're proposing?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 3:02 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 David E. Wheeler da...@kineticode.com writes:
 On Feb 10, 2011, at 11:31 AM, Tom Lane wrote:
 I don't see how that affects my point?  You can spell 1.0 as 0.1
 and 1.1 as 0.2 if you like that kind of numbering, but I don't
 see that that has any real impact.  At the end of the day an author is
 going to crank out a series of releases, and if he cares about people
 using those releases for production, he's going to have to provide at
 least a upgrade script to move an existing database from release N to
 release N+1.

 Yeah, but given a rapidly-developing extension, that could create a lot of 
 extra work. I don't know that there's much of a way around that, other than 
 concatenating files to build migration scripts from parts (perhaps via 
 `Make` as dim suggested). But it can get complicated pretty fast. My desire 
 here is to keep the barrier to creating PostgreSQL extensions as low as is 
 reasonably possible.

 Oh, I see, you're just saying that it's not unlikely somebody could find
 himself with dozens of minor releases all being supported.  Yeah, he'd
 then really need to provide shortcut upgrade scripts, and
 building/maintaining those would be a pain.

 The design as I sketched it didn't need to make any assumptions at all
 about the meaning of the version identifiers.  But if you were willing
 to assume that the identifiers are comparable/sortable by some rule,
 then it wouldn't be that hard for ALTER EXTENSION UPGRADE to figure out
 how to chain a series of upgrade scripts together to get from A to B,
 and then there would be no need for manual maintenance of shortcut
 scripts.  IIRC the main objection to doing it that way was that the
 underlying .so has to be compatible (at least to the extent of allowing
 CREATE OR REPLACE FUNCTION) with all the intermediate versions --- but
 if you believe the use-case I'm arguing for, that would be wanted
 anyway, because all the intermediate versions would be considered
 potentially useful stopping points.

 I'm not philosophically opposed to requiring the version numbers to be
 sortable, I just didn't want to introduce the concept if we didn't have
 to.  But maybe automatic application of a series of upgrade scripts is
 enough reason.

You don't need them to be sortable.  You just need them to be
comparable, and equality seems like a plenty good enough comparison
rule.  You can compute the shortest chain of upgrade scripts that can
take you from the current version to the target version.

But I'd be happy to leave that for 9.2.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-10 Thread Jan Urbański
On 10/02/11 20:24, Peter Eisentraut wrote:
 On sön, 2011-02-06 at 20:44 +0100, Jan Urbański wrote:
 On 27/01/11 23:24, Jan Urbański wrote:
 On 11/01/11 12:20, Jan Urbański wrote:
 On 11/01/11 01:27, Tom Lane wrote:
 Hannu Krosing ha...@2ndquadrant.com writes:
 On 10.1.2011 17:20, Jan Urbański wrote:
 I changed that patch to use Perl instead of sed to generate the
 exceptions, which should be a more portable.

 Updated as an incremental patch on to of the recently sent version of
 explicit-subxacts.

 Updated again.
 
 Why do the error messages print spiexceptions.SyntaxError instead of
 plpy.spiexceptions.SyntaxError?  Is this intentional or just the way it
 comes out of Python?

That's how traceback.format_exception() works IIRC, which is what the
Python interpreter uses and what PL/Python mimicks in PLy_traceback.

 Please add some documentation.  Not a list of all exceptions, but at
 least a paragraph that various kinds of specific exceptions may be
 generated, what package and module they are in, and how they relate.

Sure, Steve already asked for docs in another thread, and I'm writing them.

Jan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 Now, if you want to argue that moving an extension after the fact (ALTER
 EXTENSION SET SCHEMA) is so dangerous as to be useless, I wouldn't
 argue very hard.  Do you want to propose ripping that out?  But
 relocating at first install doesn't seem horrible.

Either an extension is relocatable or you have to deal with what Josh
Berkus the search_path hell.  Lots of databases are using a host of
schema for their own objects already, and will want to have extensions
either all in the same place or scattered around each in its own schema.

I don't think we are in a position to impose a choice to our users here.

 I'm not very concerned about letting people set the schema after the
 fact.  If we think it's OK for them to whack the location around at
 first install, I don't know why we shouldn't also let them whack it
 around later.  The question I have is whether it's really reasonable
 to let extension-owned objects be moved around at all.  It'll probably
 work fine as long as there are no other extensions depending on the
 one that's getting moved, but it doesn't pay to design for the trivial

If your extension depends on some others and your scripts are not
prepared to deal with those being moved around, you just setup your
extension as not relocatable.  That's it.

 case.  The real issue is what happens when you want to install
 extension A, which depends on extensions B, C, and D, and B, C, and D
 are all in non-standard locations.  Does that have any chance of
 working under the system we're proposing?

Yes.  It all depends on what's in the extension and what exactly the
dependency is.  You have more problem when calling another extension's
function relying on the search_path that you have when using another
extension's data type.  But it boils down to which way the dependency is
setup.  And if moving objects breaks the install, you move them back
then fill a bug, and the extension's author changes relocatable to false
in the next version, or fix the bug in another way.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 3:33 PM, Dimitri Fontaine
dimi...@2ndquadrant.fr wrote:
 Robert Haas robertmh...@gmail.com writes:
 Now, if you want to argue that moving an extension after the fact (ALTER
 EXTENSION SET SCHEMA) is so dangerous as to be useless, I wouldn't
 argue very hard.  Do you want to propose ripping that out?  But
 relocating at first install doesn't seem horrible.

 Either an extension is relocatable or you have to deal with what Josh
 Berkus the search_path hell.  Lots of databases are using a host of
 schema for their own objects already, and will want to have extensions
 either all in the same place or scattered around each in its own schema.

 I don't think we are in a position to impose a choice to our users here.

Well, for that matter, the user could want to install the same SQL
objects in more than one schema, in effect installing the same
extension twice.

 I'm not very concerned about letting people set the schema after the
 fact.  If we think it's OK for them to whack the location around at
 first install, I don't know why we shouldn't also let them whack it
 around later.  The question I have is whether it's really reasonable
 to let extension-owned objects be moved around at all.  It'll probably
 work fine as long as there are no other extensions depending on the
 one that's getting moved, but it doesn't pay to design for the trivial

 If your extension depends on some others and your scripts are not
 prepared to deal with those being moved around, you just setup your
 extension as not relocatable.  That's it.

No, you have to get *those other module authors* to make *their*
extensions not relocatable so that you can depend on them.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 12:02 PM, Tom Lane wrote:

 Oh, I see, you're just saying that it's not unlikely somebody could find
 himself with dozens of minor releases all being supported.  Yeah, he'd
 then really need to provide shortcut upgrade scripts, and
 building/maintaining those would be a pain.

Yes, exactly.

 The design as I sketched it didn't need to make any assumptions at all
 about the meaning of the version identifiers.  But if you were willing
 to assume that the identifiers are comparable/sortable by some rule,
 then it wouldn't be that hard for ALTER EXTENSION UPGRADE to figure out
 how to chain a series of upgrade scripts together to get from A to B,
 and then there would be no need for manual maintenance of shortcut
 scripts.  IIRC the main objection to doing it that way was that the
 underlying .so has to be compatible (at least to the extent of allowing
 CREATE OR REPLACE FUNCTION) with all the intermediate versions --- but
 if you believe the use-case I'm arguing for, that would be wanted
 anyway, because all the intermediate versions would be considered
 potentially useful stopping points.

And that was essentially my original proposal.

 I'm not philosophically opposed to requiring the version numbers to be
 sortable, I just didn't want to introduce the concept if we didn't have
 to.  But maybe automatic application of a series of upgrade scripts is
 enough reason.

I always thought it was.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 12:07 PM, Robert Haas wrote:

 You don't need them to be sortable.  You just need them to be
 comparable, and equality seems like a plenty good enough comparison
 rule.  You can compute the shortest chain of upgrade scripts that can
 take you from the current version to the target version.

You have to be able to apply them in order. Unless I'm missing something, that 
means you need to be able to sort them.

 But I'd be happy to leave that for 9.2.

Yeah, if necessary. The only downside to that is, if we do indeed need them to 
be sortable, then we'd have to mandate a versioning format. And if there were 
existing extensions before 9.2, that might mess with them.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 2:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Now, if you want to argue that moving an extension after the fact (ALTER
 EXTENSION SET SCHEMA) is so dangerous as to be useless, I wouldn't
 argue very hard.  Do you want to propose ripping that out?  But
 relocating at first install doesn't seem horrible.

 I'm not very concerned about letting people set the schema after the
 fact.  If we think it's OK for them to whack the location around at
 first install, I don't know why we shouldn't also let them whack it
 around later.

The argument was that whether it's safe to move it during initial
install is strictly a property of the extension's own internals.  Once
it's been in the database for awhile, moving it safely depends not only
on the extension's internals but also on whether you have created any
*other* objects that depend on where the extension is; for example,
functions that have its schema name embedded in a SET search_path
property or even hardwired in their code.

However, this risk isn't really any different from when you do ALTER foo
SET SCHEMA on a loose object, so on reflection it's not clear to me
that we should refuse this case when we allow the latter.  We're merely
allowing people to shoot themselves in the foot with a machine-gun
instead of a revolver, by providing a command that encapsulates a whole
lot of SET SCHEMA commands in one action.

 The real issue is what happens when you want to install
 extension A, which depends on extensions B, C, and D, and B, C, and D
 are all in non-standard locations.  Does that have any chance of
 working under the system we're proposing?

Again, it's not really any different from the case where the dependent
objects are loose rather than members of an extension.  It's pretty
much up to the user to be aware of the consequences.  If we had a way to
mark individual functions as safe or unsafe for renames to happen, it'd
be reasonable to extend that notion to whole extensions.  But we don't
have that and I don't think it's appropriate to hold extensions to a
higher standard than we do loose objects --- especially when it takes
superuser privileges to break things by moving an extension but not to
break them by moving loose objects.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 3:46 PM, David E. Wheeler da...@kineticode.com wrote:
 On Feb 10, 2011, at 12:07 PM, Robert Haas wrote:

 You don't need them to be sortable.  You just need them to be
 comparable, and equality seems like a plenty good enough comparison
 rule.  You can compute the shortest chain of upgrade scripts that can
 take you from the current version to the target version.

 You have to be able to apply them in order. Unless I'm missing something, 
 that means you need to be able to sort them.

Not at all.  Say the currently installed version of the dungeon
extension is kobold and you want to upgrade to bugbear.  You have
the following scripts:

dungeon-goblin-orc.sql
dungeon-hobgoblin-bugbear.sql
dungeon-kobold-goblin.sql
dungeon-orc-hobgoblin.sql

Now, it's pretty clear that the only way to get to bugbear is to come
from hobgoblin, and the only way to get to hobgoblin is to come from
orc.  orc can be reached only from goblin, which can be reached only
from kobold.  So it's 100% clear that you have to apply the scripts in
the following order:

dungeon-kobold-goblin.sql
dungeon-goblin-orc.sql
dungeon-orc-hobgoblin.sql
dungeon-hobgoblin-bugbear.sql

Note that this even works if the versions aren't totally ordered.  For
example, suppose you release version 0.1 of a module and later you
release a 1.0, which unfortunately is incompatible: there's no upgrade
path from 0.1 to 1.0.  In time, 1.0 is superseded by 1.1.  And then
you make some improvements to the old 0.1 code base and release that
as 0.2.  Finally, you come up with an idea for unifying the two and
release a 1.2 version, which supports upgrades from all the previous
versions.  You just ship:

foo-0.1-0.2.sql
foo-0.2-1.2.sql
foo-1.0-1.1.sql
foo-1.1-1.2.sql

If the user asks to upgrade to version 1.2, we'll observe that you can
get to 1.2 from 1.1 or from 0.2.  Not knowing what the version numbers
mean, we'll look a bit further and see that you can get from 1.0 to
1.1 or from 0.1 to 0.2.  Thus you can get to 1.2 like this:

0.1 - 0.2 - 1.2
0.2 - 1.2
1.0 - 1.1 - 1.2
1.1 - 1.2

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 3:02 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 The design as I sketched it didn't need to make any assumptions at all
 about the meaning of the version identifiers.  But if you were willing
 to assume that the identifiers are comparable/sortable by some rule,

 You don't need them to be sortable.  You just need them to be
 comparable, and equality seems like a plenty good enough comparison
 rule.  You can compute the shortest chain of upgrade scripts that can
 take you from the current version to the target version.

Hmm.  The problem with that is that once there are large numbers of
intermediate versions, the number of potential paths grows
exponentially.  I was envisioning an algorithm like this:

1.  Scan directory for upgrade scripts with oldversion = version we
have, and take the one with largest newversion = version we want.

2.  Apply this script (or more likely, just remember it until we've
verified there is a chain leading to version we want).

3.  If now the version is not what we want, return to step 1.

I don't see an equally efficient method if we only have equality.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
Hi,

Tom Lane t...@sss.pgh.pa.us writes:
 I spent some time reviewing this thread.  I think the major point that's

Thanks for doing that, we badly needed someone without an horse in this
race to do that and finish the design.

 So I believe that it'd be a good idea if it were possible for an extension
 author to distribute a package that implements, say, versions 1.0, 1.1,
 and 2.0 of hstore.  Not all will choose to do the work needed for that, of
 course, and that's fine.  But the extension mechanism ought to permit it.

Agreed.  I've been weary of being told I'm trying to put too much into
the first PostgreSQL release with extensions, but I'm happy to see you
want to take it this far.  So well, here follows some ideas I've been
trying hard not to push too soon :)

 To do this, we need to remove the concept that the control file specifies
 the version of an extension; rather the version is associated with the
 SQL script file.  I think we should embed the version number in the script
 file name, and require one to be present (no more omitted version
 numbers).  So you would distribute, say,
   hstore-1.0.sql
   hstore-1.1.sql
   hstore-2.0.sql
 representing the scripts needed to install these three versions from

What I don't like in that is that this restrict what the version strings
can look like.  In debian for example it's pretty common to use the ~
separator, because 1.0~alpha1  1.0~beta  1.0 with their sorting rules.
And this trick won't work on windows filenames, AFAIK.  That's one
reason why I've wanted to stay away from having the version number
strings encoded into the filename in the first place.

But if you get to sorting rules of version strings, you have to define
them properly and impose them to users.  That's both a good thing and a
bad thing, but IMO requires that we provide a proper data type for that.

So my opinion here is that we should not only go with your design here
with the version string in the filename, but also imposes how to spell
out version strings in a way that we know will work for PostgreSQL on
every supported system.

 scratch.  CREATE EXTENSION would have an option to select which
 version to install.  If the option is omitted, there are at least two
 things we could do:
   1. Choose the newest available version.
   2. Let the control file specify which version is the default.
 I think I prefer #2 because it avoids needing a rule for comparing
 version identifiers, and it caters to the possibility that the newest
 version isn't yet mature enough to be a good default.

I like this idea.  +1 for having the default version to install in the
control file.  See below for some more details about that, though.

 As for upgrades, let's just expect upgrade scripts to be named
 extension-oldversion-newversion.sql.  ALTER EXTENSION UPGRADE knows the
 relevant oldversion from pg_extension, and newversion can be handled the
 same way as in CREATE, ie, either the user says which version to update to
 or we use the default version from the control file.

Again, I like the idea and how simple it make things look, but I think
if we should then bite the bullet and restrict what a version string is
expected to be and offer a data type with proper sorting while at it.
And of course use that as the pg_extension.extversion column type.

That way a SQL query can check if there's a new version available on
your system.  That's useful in some places to use as a monitoring alert
coupled with nagios.  The sysadmin team does the apt-get install part of
the job and then the DBA team is paged to go upgrade the extensions in
the databases, or shut the alarm somehow.

 I don't seriously expect most extension authors to bother preparing
 upgrade scripts for any cases except adjacent pairs of versions.
 That means that if a user comes along and wants to upgrade across several
 versions of the extension, he'll have to do it in several steps:
   ALTER EXTENSION hstore UPGRADE TO '1.1';
   ALTER EXTENSION hstore UPGRADE TO '2.0';
   ALTER EXTENSION hstore UPGRADE TO '2.1';
 I don't see that as being a major problem --- how often will people have
 the need to do that, anyway?  Authors who feel that it is a big deal can
 expend the work to provide shortcut scripts.  I do not see adequate return
 on investment from the regexp-matching complications in the currently
 submitted patch.

The regexp matching reason to live is so that we don't have to know
anything about version strings at all.  If you're saying that a version
string can not contain a dash and must be a valid filesystem name (often
enough, for all systems supported by PostgreSQL), and you're now saying
that ALTER EXTENSION UPGRADE could automate multi-steps upgrade, then I
think we have to provide the version (or pgversion) data type and
all that jazz.

If we get to somehow, even lightly, depend on some rules, better offer
them in code and documentation rather than have them implicit.

 that it is necessary to 

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 No, you have to get *those other module authors* to make *their*
 extensions not relocatable so that you can depend on them.

Just tell me exactly in which world an extension's author is setting up
the dependencies in the 'required' property and yet fails to realise
that those dependencies mean his extension is not relocatable?  And
will refuse to fix the problem when bugs are filled?

I'm not following your reasonning…
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 3:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 3:02 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 The design as I sketched it didn't need to make any assumptions at all
 about the meaning of the version identifiers.  But if you were willing
 to assume that the identifiers are comparable/sortable by some rule,

 You don't need them to be sortable.  You just need them to be
 comparable, and equality seems like a plenty good enough comparison
 rule.  You can compute the shortest chain of upgrade scripts that can
 take you from the current version to the target version.

 Hmm.  The problem with that is that once there are large numbers of
 intermediate versions, the number of potential paths grows
 exponentially.  I was envisioning an algorithm like this:

 1.  Scan directory for upgrade scripts with oldversion = version we
 have, and take the one with largest newversion = version we want.

 2.  Apply this script (or more likely, just remember it until we've
 verified there is a chain leading to version we want).

 3.  If now the version is not what we want, return to step 1.

 I don't see an equally efficient method if we only have equality.

It's certainly not exponential i.e. O(2^n) or something of that form.
Even a naive application of Dijkstra's algorithm is only going to be
O(n^2) in the number of versions, which is likely tolerable even if
upgrades are supported for dozens of old versions.  It might break
down if there are hundreds of old versions, but that doesn't seem
likely to be a real problem in practice.  But if you're concerned
about it, you can replace the linked list that the naive algorithm
uses with a binary heap or (if you really want to go nuts) a fibonacci
heap.  The latter approach has a runtime of O(n + m lg m), where n is
the number of versions and m is the number of upgrade scripts.  You
need one heck of a lot of backward compatibility before that algorithm
breaks a sweat.  Even the binary heap is only O((n + m) lg m), which
pretty darn fast.

Personally, I think we'll be lucky if people support ten back revs,
let alone three hundred, but it's a simple matter of programming - and
an afternoon with an introductory algorithms textbook - to make it as
efficient as we could ever want it to be.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 I don't think it's appropriate to hold extensions to a
 higher standard than we do loose objects --- especially when it takes
 superuser privileges to break things by moving an extension but not to
 break them by moving loose objects.

FWIW, +1.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 4:14 PM, Dimitri Fontaine
dimi...@2ndquadrant.fr wrote:
 Robert Haas robertmh...@gmail.com writes:
 No, you have to get *those other module authors* to make *their*
 extensions not relocatable so that you can depend on them.

 Just tell me exactly in which world an extension's author is setting up
 the dependencies in the 'required' property and yet fails to realise
 that those dependencies mean his extension is not relocatable?  And
 will refuse to fix the problem when bugs are filled?

No, the problem is this.  I write an extension called foo.  By
default, it installs in schema foo.

You write an extension called bar.  By default, it installs in schema
bar.  It also depends on foo.

Now Alice wants to install foo and bar.  But she already has a schema
called foo, so she installs the extension foo in foo2.  Now she tries
to install bar, but it doesn't work, because it is looking for objects
in schema foo, and on this system they are in foo2.

There's no way for you, as the author of bar, to fix this problem,
other than to persuade me, as the author of foo, that I should make my
extension not relocatable.  I might not want to do that.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 3:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Hmm.  The problem with that is that once there are large numbers of
 intermediate versions, the number of potential paths grows
 exponentially.

 It's certainly not exponential i.e. O(2^n) or something of that form.
 Even a naive application of Dijkstra's algorithm is only going to be
 O(n^2) in the number of versions, which is likely tolerable even if
 upgrades are supported for dozens of old versions.

Well, okay, let's go with that plan then.  If we don't need to assume
anything more than equality of version names being meaningful, I think
chaining update scripts automatically should solve most of the
complaints here.  People who really want to maintain shortcut scripts
still could, but I think it'd be an unusual case.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Robert Haas
On Thu, Feb 10, 2011 at 3:52 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 The real issue is what happens when you want to install
 extension A, which depends on extensions B, C, and D, and B, C, and D
 are all in non-standard locations.  Does that have any chance of
 working under the system we're proposing?

 Again, it's not really any different from the case where the dependent
 objects are loose rather than members of an extension.  It's pretty
 much up to the user to be aware of the consequences.  If we had a way to
 mark individual functions as safe or unsafe for renames to happen, it'd
 be reasonable to extend that notion to whole extensions.  But we don't
 have that and I don't think it's appropriate to hold extensions to a
 higher standard than we do loose objects --- especially when it takes
 superuser privileges to break things by moving an extension but not to
 break them by moving loose objects.

Well, the difference is that loose objects are just on my system,
whereas extensions are supposed to work on anybody's system.  I'm not
clear that it's possible to write an extension that depends on a
relocatable extension in a sensible way.  If it is, objection
withdrawn.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Revised patches to add table function support to PL/Tcl (TODO item)

2011-02-10 Thread Andrew Dunstan



On 02/08/2011 08:37 PM, Andrew Dunstan wrote:



On 02/07/2011 11:30 PM, Robert Haas wrote:

On Tue, Dec 28, 2010 at 9:23 PM, Karl Lehenbauer
karllehenba...@gmail.com  wrote:

On Dec 28, 2010, at 7:29 PM, Tom Lane wrote:

This patch appears to be changing a whole lot of stuff that in fact
pg_indent has never changed, so there's something wrong with the 
way you

are doing it.  It looks like a bad typedef list from here.
You were right, Tom.  The problem was that typedefs 
pltcl_interp_desc, pltcl_proc_key, and pltcl_proc_ptr weren't 
in src/tools/pgindent/typedefs.list.  After adding them (and 
building and installing the netbsd-based, patched indent), pgindent 
only changes a handful of lines.


pltcl-karl-try3-1-of-3-pgindent.patch patches typedefs.list with the 
three missing typedefs and pltcl.c with the small changes made by 
pgindent (it shifted some embedded comments left within their lines, 
mainly).


As before, but try3 now, pltcl-karl-try3-2-of-3-objects.patch 
converts pltcl.c to use the Tcl objects C API.


And as before, but try3 now, pltcl-karl-try3-3-of-3-setof.patch 
adds returning record and SETOF record.

This patch did not get reviewed, because the person who originally
planned to review it had a hardware failure that prevented him from
doing so.  Can anyone pick this up?


I will have a look at it.





As promised I have had a look. The first point is that it doesn't have 
any documentation at all.


The second is that it doesn't appear from a my admittedly short look to 
support nested composites, or perhaps more importantly composites with 
array fields. I think if we're going to add support for composites to 
pltcl, we should make sure we support these from the start rather than 
store up for ourselves the sorts of trouble that we're now grappling 
with in plperl-land. We shouldn't start to make pltcl users pass back 
composed array or record literals, if possible.


As for the API changes, I'd like to have that piece reviewed by someone 
more familiar with the Tcl API than I am. I'm not sure who if anyone we 
have that has that familiarity, now Jan is no longer active.


I know this has been on the table for six weeks, and an earlier review 
might have given Karl more chance to remedy these matters in time. I'm 
sorry about that, it's a pity the original reviewer ran into issues.  
But for now I'm inclined to mark this as Returned with Feedbnack.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 1:22 PM, Tom Lane wrote:

 Well, okay, let's go with that plan then.  If we don't need to assume
 anything more than equality of version names being meaningful, I think
 chaining update scripts automatically should solve most of the
 complaints here.  People who really want to maintain shortcut scripts
 still could, but I think it'd be an unusual case.

Yes, I think that this is a great solution. I only have to create on upgrade 
script for each release, and I don't have to worry about concatenating anything 
or be required to change my versioning algorithm.

+1

Finally, a solution!

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pl/python custom exceptions for SPI

2011-02-10 Thread Steve Singer

On 11-02-10 03:13 PM, Jan Urbański wrote:

On 10/02/11 20:24, Peter Eisentraut wrote:


Here is the rest of my review.


Submission Review
---
Patch applies cleanly.
Documentation is still outstanding but Jan has promised it soon.

Usability Review
---
We don't have this for plpython,  that we have a similar idea with 
plpgsql.  I think this feature is useful and worth having.


The CamelCase naming of the exceptions is consistent with how the 
built-in python exceptions are named (camel case).




Feature Test
---
I did basic testing of the feature (catching a few exception types 
thrown from both direct SQL and prepared statements) and the feature 
worked as expected.


Performance Impact

The impact of mapping error codes to exception types shouldn't come into 
play unless an SPI error is returned and with the hash it should still 
be minimal.




Code Review
-

Ideally char * members of ExceptionMap would be const, but since many 
versions of python take a non-const value to PyErr_NewException that 
won't work :(


After you search the for an exception in the hash you have:

/* We really should find it, but just in case have a fallback */
Assert(entry != NULL);
exc = entry ? entry-exc : PLy_exc_spi_error;

I'm not sure the assert is needed here.  Just falling back to the 
exception type seems reasonable and more desirable than an assert if 
showhow a new exception gets missed from the list. I don't feel that 
strongly on this.



line 3575:  PLy_elog(ERROR, Failed to add the spiexceptions module);
Failed should be failed

Other than that the patch looks fine to me.





Updated again.


Why do the error messages print spiexceptions.SyntaxError instead of
plpy.spiexceptions.SyntaxError?  Is this intentional or just the way it
comes out of Python?


That's how traceback.format_exception() works IIRC, which is what the
Python interpreter uses and what PL/Python mimicks in PLy_traceback.


Please add some documentation.  Not a list of all exceptions, but at
least a paragraph that various kinds of specific exceptions may be
generated, what package and module they are in, and how they relate.


Sure, Steve already asked for docs in another thread, and I'm writing them.

Jan




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-02-10 Thread Alexey Klyukin

On Feb 10, 2011, at 9:44 PM, Andrew Dunstan wrote:

 
 
 On 02/10/2011 08:15 AM, Alexey Klyukin wrote:
 On Feb 9, 2011, at 9:28 PM, Alex Hunsaker wrote:
 
 On Wed, Feb 9, 2011 at 08:24, Alexey Klyukinal...@commandprompt.com  
 wrote:
 What was actually broken in encode_array_literal support of composite types
 (it converted perl hashes to the literal composite-type constants, 
 expanding
 nested arrays along the way) ? I think it would be a useful extension of 
 the
 existing encode_array_literal.
 Yeah, It does not work because it did not take into account the order
 of composite columns. It always put them alphabetically by column
 name. To do it properly we would need to pass in a typid or a column
 order or something. Ideally we could expose the new
 plperl_array_to_datum() to plperl functions in some manner.
 Damn, right. Each perl hash corresponds to multiple composite types, 
 different
 by the order of the type elements. Passing the typid sounds like a fair
 requirement (and if it's missing we could assume that the order of columns in
 composites doesn't matter to the caller).
 
 Let me try implementing that as an XS interface to plperl_array_to_datum.
 
 
 Are you intending this as a completion of the current patch or as 9.2 work? 
 If the former you need to send it in real fast.

I'd like to extend the current patch, going to post the update by tomorrow. 

/A

--
Alexey Klyukin
The PostgreSQL Company - Command Prompt, Inc.





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 Well, the difference is that loose objects are just on my system,
 whereas extensions are supposed to work on anybody's system.  I'm not
 clear that it's possible to write an extension that depends on a
 relocatable extension in a sensible way.  If it is, objection
 withdrawn.

I proposed that in this case, we bypass the relocatable property and
just have the system work out that reverse dependencies make all those
extensions not relocatable.  Tom said that he does not see the point in
trying to limit this foot gun power.

-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Dimitri Fontaine
David E. Wheeler da...@kineticode.com writes:
 Yes, I think that this is a great solution. I only have to create on
 upgrade script for each release, and I don't have to worry about
 concatenating anything or be required to change my versioning
 algorithm.

You still have to make sure that the C code remains compatible with any
intermediate release, for the whole life of your extension.  But I agree
that it's way better than what we had before.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Feb 10, 2011 at 3:52 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Again, it's not really any different from the case where the dependent
 objects are loose rather than members of an extension.

 Well, the difference is that loose objects are just on my system,
 whereas extensions are supposed to work on anybody's system.  I'm not
 clear that it's possible to write an extension that depends on a
 relocatable extension in a sensible way.  If it is, objection
 withdrawn.

I don't deny that there are risks here.  But I think the value of being
able to move an extension when it is safe outweighs the difficulty that
sometimes it isn't safe.  I think we can leave making it safer as a
topic for future investigation.

Dimitri did suggest treating an extension as nonrelocatable if there is
any other extension installed that depends on it.  But that seems like
more of a kluge than a nice solution, primarily because it does nothing
for the loose-object risks.  I'd rather just document that moving an
extension post-installation might break things, and leave it at that for
now.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 1:38 PM, Tom Lane wrote:

 I don't deny that there are risks here.  But I think the value of being
 able to move an extension when it is safe outweighs the difficulty that
 sometimes it isn't safe.  I think we can leave making it safer as a
 topic for future investigation.
 
 Dimitri did suggest treating an extension as nonrelocatable if there is
 any other extension installed that depends on it.  But that seems like
 more of a kluge than a nice solution, primarily because it does nothing
 for the loose-object risks.  I'd rather just document that moving an
 extension post-installation might break things, and leave it at that for
 now.

+1

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Aidan Van Dyk
On Thu, Feb 10, 2011 at 9:38 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 Well, the difference is that loose objects are just on my system,
 whereas extensions are supposed to work on anybody's system.  I'm not
 clear that it's possible to write an extension that depends on a
 relocatable extension in a sensible way.  If it is, objection
 withdrawn.

 I don't deny that there are risks here.  But I think the value of being
 able to move an extension when it is safe outweighs the difficulty that
 sometimes it isn't safe.  I think we can leave making it safer as a
 topic for future investigation.

Personally, I'ld rather be able to install the *same*
extension/version in different schemas at the same time then move an
extension from 1 schema to another, although I have no problems with
extensions moving out under a function's foot (just like loose
objects).

a.



-- 
Aidan Van Dyk                                             Create like a god,
ai...@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 David E. Wheeler da...@kineticode.com writes:
 Yes, I think that this is a great solution. I only have to create on
 upgrade script for each release, and I don't have to worry about
 concatenating anything or be required to change my versioning
 algorithm.

 You still have to make sure that the C code remains compatible with any
 intermediate release, for the whole life of your extension.  But I agree
 that it's way better than what we had before.

What you have to do is make sure the C code remains compatible with any
version you are shipping an install or upgrade script for.  Once you
decide that versions before, say, 2.0 are dead as doornails, you remove
all the older scripts, and you can delete the .so infrastructure for
them too.

For example, suppose I have foobar 1.5 installed on my system.  If you
are shipping a package that includes foobar-1.5-2.0.sql, I should
reasonably expect that I can install that package first and upgrade the
extension afterwards.  If you aren't shipping any script that claims it
can upgrade from 1.5, you don't need to provide .so compatibility either
--- it's clear that I have to upgrade first and install your newer
package after.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread David Fetter
On Thu, Feb 10, 2011 at 09:16:09PM +0200, Peter Eisentraut wrote:
 On tor, 2011-02-10 at 10:40 -0800, David Fetter wrote:
  I think all such checks belong in .git/hooks/pre-commit, and need
  to be as cross-platform as needed for committers.  Would a
  *n*x-based version do for a start?
 
 I think as a matter of principle, the only things that belongs into
 git hooks are things that relate to the repository itself, such as
 commit metadata (author checks, commit message spell checks, etc.).
 Anything that relates to the source belongs into the source.

So on that principle, should make check do these checks?

Just to put another viewpoint out there, the reason that such hooks
exist is precisely to enable people to do deeper checks than commit
metadata.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Remove more SGML tabs.

2011-02-10 Thread David Fetter
On Thu, Feb 10, 2011 at 11:29:43AM -0300, Alvaro Herrera wrote:
 Excerpts from Peter Eisentraut's message of jue feb 10 07:58:16 -0300 2011:
 
  One thing I was thinking of was that we could add a global make
  maintainer-check target (a name I picked up from other projects)
  which would run various source code sanity checks.  Besides the
  SGML tabs issue, my favourite would be duplicate_oids.  Maybe if
  we could find a third use case, we'd have a quorum for
  implementing this.
 
 The c++ headers thing?  Or the compiles-in-isolation test for
 headers?

Both sound reasonable, given the number of times they come up and the
ease of checking them mechanically.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SPI_exec doesn't return proc context (on 9.1)

2011-02-10 Thread Alvaro Herrera
Excerpts from Pavel Stehule's message of sáb ene 29 16:56:40 -0300 2011:
 2011/1/29 Tom Lane t...@sss.pgh.pa.us:

  The less crocky way to do that is to use SPI_palloc() for something that
  should be allocated in the outer context.
 
 I understand. Is there some way, where I can use a cstring_to_text
 function? There isn't simple way to get a saveCtx.
 
 some like SPI_copyDatum ... ?

I wrote such a patch some time ago but never applied it ...
http://archives.postgresql.org/message-id/20091116162531.ga3...@alvh.no-ip.org
I still wonder if it's useful enough to be applied.  Would it solve your
use case?

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


  1   2   >