Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread Bruce Momjian
Oleg Bartunov wrote:
 Hi there,
 
 it's clear we need versions, probably, major.minor would be enough. The 
 problem
 I see is how to keep .so in sync with .sql ? Should we store .sql in database 
 ?
 
 Also, we need permissions for extension, since we have open/closed 
 extensions.
 

Don't people normally define the version number in the Makefile and pass
the version string into the C code and perhaps a psql variable?

---


 
 Oleg
 
 On Sat, 11 Dec 2010, David E. Wheeler wrote:
 
  On Dec 11, 2010, at 1:09 PM, David Fetter wrote:
 
  Why is it in the makefile at all?  If the makefile does need to know it,
  why don't we have it scrape the number out of the control file?  Or even
  more to the point, since when do we need version numbers in extensions?
 
  We *absolutely* need version numbers in extensions.  People will want
  to have a certain version, or a certain minimum version, etc., etc.,
  etc., just as they do for any other software.
 
  Seriously, are you OK?
 
  One of the biggest mistakes in the creation of CPAN was allowing modules 
  without extensions. It makes figuring out what to upgrade extremely 
  difficult. Learning from that, PGXN requires version numbers for all 
  extensions.
 
  Best,
 
  David
 
 
 
 
   Regards,
   Oleg
 _
 Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
 Sternberg Astronomical Institute, Moscow University, Russia
 Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
 phone: +007(495)939-16-83, +007(495)939-23-83
 
 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread David E. Wheeler
On Dec 29, 2010, at 12:00 PM, Bruce Momjian wrote:

 Don't people normally define the version number in the Makefile and pass
 the version string into the C code and perhaps a psql variable?

There is no standard pattern AFAIK. A best practice would be welcome here.

David

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Oleg Bartunov wrote:
 it's clear we need versions, probably, major.minor would be enough. The 
 problem
 I see is how to keep .so in sync with .sql ? Should we store .sql in 
 database ?

 Don't people normally define the version number in the Makefile and pass
 the version string into the C code and perhaps a psql variable?

We had a long discussion upthread of what version numbers to keep where.
IMHO the Makefile is about the *least* useful place to put a version
number; the more so if you want more than one.  What we seem to need is
a version number in the .sql file itself (so that we can tell whether we
need to take action to update the extension's catalog entries).  I'm not
convinced yet whether there needs to be another version number embedded
in the .so file --- it may well be that the PG major version number
embedded with PG_MODULE_MAGIC is sufficient.

Personally I'd forget the notion of major.minor numbers here; all that
will accomplish is to complicate storage and comparison of the numbers.
We just need a simple integer that gets bumped whenever the extension's
SQL script changes.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread David E. Wheeler
On Dec 29, 2010, at 12:23 PM, Tom Lane wrote:

 We had a long discussion upthread of what version numbers to keep where.
 IMHO the Makefile is about the *least* useful place to put a version
 number; the more so if you want more than one.  What we seem to need is
 a version number in the .sql file itself (so that we can tell whether we
 need to take action to update the extension's catalog entries).  I'm not
 convinced yet whether there needs to be another version number embedded
 in the .so file --- it may well be that the PG major version number
 embedded with PG_MODULE_MAGIC is sufficient.

For contrib maybe, but not 3rd-party extensions.

 Personally I'd forget the notion of major.minor numbers here; all that
 will accomplish is to complicate storage and comparison of the numbers.
 We just need a simple integer that gets bumped whenever the extension's
 SQL script changes.

That won't be very flexible for third-party extensions. FWIW, for PGXN I 
mandated symantic version numbers (http://semver.org/), mainly because they're 
quite close to Pg core version numbers. I also created a basic data type for 
them:

  https://github.com/theory/pgxn-manager/blob/master/sql/02-types.sql#L70

Best,

David



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread Robert Haas
On Wed, Dec 29, 2010 at 3:23 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Bruce Momjian br...@momjian.us writes:
 Oleg Bartunov wrote:
 it's clear we need versions, probably, major.minor would be enough. The 
 problem
 I see is how to keep .so in sync with .sql ? Should we store .sql in 
 database ?

 Don't people normally define the version number in the Makefile and pass
 the version string into the C code and perhaps a psql variable?

 We had a long discussion upthread of what version numbers to keep where.
 IMHO the Makefile is about the *least* useful place to put a version
 number; the more so if you want more than one.  What we seem to need is
 a version number in the .sql file itself (so that we can tell whether we
 need to take action to update the extension's catalog entries).  I'm not
 convinced yet whether there needs to be another version number embedded
 in the .so file --- it may well be that the PG major version number
 embedded with PG_MODULE_MAGIC is sufficient.

 Personally I'd forget the notion of major.minor numbers here; all that
 will accomplish is to complicate storage and comparison of the numbers.
 We just need a simple integer that gets bumped whenever the extension's
 SQL script changes.

I think there are really two tasks here:

1. Identify whether a newer set of SQL definitions than the one
installed is available.  If so, the extension is a candidate for an
upgrade.

2. Identify whether the installed version of the SQL definitions is
compatible with the installed shared object.  If it's not, we'd like
the shared library load (or at a minimum, any use of the shared
library) to fail when attempted, rather than attempting to plunge
blindly onward and then crashing.

As to point #2, what an extension author would typically do (I hope)
is publish a new shared object is make it backward-compatible with
some number of previous versions of the SQL definitions, but not
necessarily all the way to the beginning of time.  So the typical
upgrade sequence would be to install the new .so, and then upgrade the
SQL definitions.  You'd want an interlock, though, in case someone
tried to use a set of SQL definitions that were either too new or too
old for the corresponding shared library.  The too new case could
occur if someone installed a new version of the shared library,
upgraded the SQL definitions, and then put the old shared library file
back.  The too old case could occur if the extension were upgraded
through many releases in a single step, such that whatever
backward-compatibility support existed in the shared library didn't
reach back far enough to cater to the ancient SQL definitions.  In
either case, you want to chunk an error when someone tries to use the
module, rather than soldiering on blindly and crashing.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-29 Thread David E. Wheeler
On Dec 29, 2010, at 1:27 PM, Robert Haas wrote:

 I think there are really two tasks here:
 
 1. Identify whether a newer set of SQL definitions than the one
 installed is available.  If so, the extension is a candidate for an
 upgrade.
 
 2. Identify whether the installed version of the SQL definitions is
 compatible with the installed shared object.  If it's not, we'd like
 the shared library load (or at a minimum, any use of the shared
 library) to fail when attempted, rather than attempting to plunge
 blindly onward and then crashing.

3. Check dependencies for one extension on other extensions.

 As to point #2, what an extension author would typically do (I hope)
 is publish a new shared object is make it backward-compatible with
 some number of previous versions of the SQL definitions, but not
 necessarily all the way to the beginning of time.  So the typical
 upgrade sequence would be to install the new .so, and then upgrade the
 SQL definitions.  You'd want an interlock, though, in case someone
 tried to use a set of SQL definitions that were either too new or too
 old for the corresponding shared library.  The too new case could
 occur if someone installed a new version of the shared library,
 upgraded the SQL definitions, and then put the old shared library file
 back.  The too old case could occur if the extension were upgraded
 through many releases in a single step, such that whatever
 backward-compatibility support existed in the shared library didn't
 reach back far enough to cater to the ancient SQL definitions.  In
 either case, you want to chunk an error when someone tries to use the
 module, rather than soldiering on blindly and crashing.

Yeah, makes sense.

Best,

David



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-12 Thread Oleg Bartunov

Hi there,

it's clear we need versions, probably, major.minor would be enough. The problem
I see is how to keep .so in sync with .sql ? Should we store .sql in database ?

Also, we need permissions for extension, since we have open/closed 
extensions.



Oleg

On Sat, 11 Dec 2010, David E. Wheeler wrote:


On Dec 11, 2010, at 1:09 PM, David Fetter wrote:


Why is it in the makefile at all?  If the makefile does need to know it,
why don't we have it scrape the number out of the control file?  Or even
more to the point, since when do we need version numbers in extensions?


We *absolutely* need version numbers in extensions.  People will want
to have a certain version, or a certain minimum version, etc., etc.,
etc., just as they do for any other software.

Seriously, are you OK?


One of the biggest mistakes in the creation of CPAN was allowing modules 
without extensions. It makes figuring out what to upgrade extremely difficult. 
Learning from that, PGXN requires version numbers for all extensions.

Best,

David





Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-11 Thread David Fetter
On Fri, Dec 10, 2010 at 11:24:27AM -0500, Tom Lane wrote:
 Dimitri Fontaine dimi...@2ndquadrant.fr writes:
  Tom Lane t...@sss.pgh.pa.us writes:
  Are there any actual remaining use-cases for that sed step?
 
  The goal here is to allow extension authors to maintain their version
  number in the Makefile rather than in the Makefile and in the control
  file separately. Having the same version number in more than one place
  never eases maintenance.
 
 Why is it in the makefile at all?  If the makefile does need to know it,
 why don't we have it scrape the number out of the control file?  Or even
 more to the point, since when do we need version numbers in extensions?

We *absolutely* need version numbers in extensions.  People will want
to have a certain version, or a certain minimum version, etc., etc.,
etc., just as they do for any other software.

Seriously, are you OK?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-11 Thread David E. Wheeler
On Dec 11, 2010, at 1:09 PM, David Fetter wrote:

 Why is it in the makefile at all?  If the makefile does need to know it,
 why don't we have it scrape the number out of the control file?  Or even
 more to the point, since when do we need version numbers in extensions?
 
 We *absolutely* need version numbers in extensions.  People will want
 to have a certain version, or a certain minimum version, etc., etc.,
 etc., just as they do for any other software.
 
 Seriously, are you OK?

One of the biggest mistakes in the creation of CPAN was allowing modules 
without extensions. It makes figuring out what to upgrade extremely difficult. 
Learning from that, PGXN requires version numbers for all extensions.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Dimitri Fontaine
David E. Wheeler da...@kineticode.com writes:
 On Dec 9, 2010, at 12:34 PM, Dimitri Fontaine wrote:
 - add support for 'relocatable' boolean property in the control file,
   as discussed on list

 This still isn't ideal, but I think it's a big improvement. Thanks.

Glad you like it :) If you see any way to manage that better, please do
tell. Just be sure to review the past 18 months of on-list discussion
about the topic before to go thinking extension vs search_path is easy
to solve, or even possible to solve.

   \dx lists only installed extensions
   \dx+ extension lists the objects, calling pg_extension_objects()
   \dX lists available extensions (and installed too)

 +1 I think that's much more like existing psql commands.

Good, I'll have that in the next patch version, waiting for until your
review of the new one :)

 So:

 * If $extension.control.in exists, use that
 * If it doesn't, generate $extension.control from the Makefile variables

What if $extension.control exists? Is it a byproduct of the .in file
from previous `make` run or a user file? What if we have both the .in
and the make variable because people are confused? Or both the make
variables and a .control and not .control.in? Etc...

 * Always remove $extension.control in the `clean` targets

Hell no, as you can bypass the .in mechanism and provide directly the
.control file.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 David E. Wheeler da...@kineticode.com writes:
 What if $extension.control exists? Is it a byproduct of the .in file
 from previous `make` run or a user file? What if we have both the .in
 and the make variable because people are confused? Or both the make
 variables and a .control and not .control.in? Etc...

 * Always remove $extension.control in the `clean` targets

 Hell no, as you can bypass the .in mechanism and provide directly the
 .control file.

Are there any actual remaining use-cases for that sed step?  It's
certainly vestigial as far as the contrib modules are concerned:
it would be simpler and more readable to replace MODULE_PATHNAME with
$libdir in the sources.  Unless somebody can point to a real-world
use-case, I'd just as soon get rid of the .in files altogether while
we're having this flag day.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 Are there any actual remaining use-cases for that sed step?

The goal here is to allow extension authors to maintain their version
number in the Makefile rather than in the Makefile and in the control
file separately. Having the same version number in more than one place
never eases maintenance.

Oh and in PostgreSQL sources cases, that would add like 36 spots where
to manually maintain our major version string. I'm not eager to do that.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 Tom Lane t...@sss.pgh.pa.us writes:
 Are there any actual remaining use-cases for that sed step?

 The goal here is to allow extension authors to maintain their version
 number in the Makefile rather than in the Makefile and in the control
 file separately. Having the same version number in more than one place
 never eases maintenance.

Why is it in the makefile at all?  If the makefile does need to know it,
why don't we have it scrape the number out of the control file?  Or even
more to the point, since when do we need version numbers in extensions?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 Why is it in the makefile at all?  If the makefile does need to know it,
 why don't we have it scrape the number out of the control file?  Or even
 more to the point, since when do we need version numbers in extensions?

It's in the Makefile so that you find it in the control file later, then
in the extension catalog. We need the version number just because I'm
not able to name a single software that's not letting you know about its
version number once installed.

Well in fact I know about one, and I wish the situation would be quite
different there.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 Tom Lane t...@sss.pgh.pa.us writes:
 Why is it in the makefile at all?  If the makefile does need to know it,
 why don't we have it scrape the number out of the control file?  Or even
 more to the point, since when do we need version numbers in extensions?

 It's in the Makefile so that you find it in the control file later, then
 in the extension catalog.

This doesn't answer my question of why it couldn't be done the other
way.  Why does the makefile need to know it?  If it does need to know
it, couldn't it get it out of the control file instead of vice versa?

 We need the version number just because I'm
 not able to name a single software that's not letting you know about its
 version number once installed.

I'm not convinced that this is actually a requirement, or that doing it
this specific way is a good solution.  In particular, keeping the
version number in the system catalogs seems pretty dubious.  The common
method for upgrading an already-installed contrib module just involves
dropping in a new .so --- that's not going to change the system
catalogs.  It would likely be better to keep the version ID inside the
.so file.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread David E. Wheeler
On Dec 10, 2010, at 12:26 AM, Dimitri Fontaine wrote:

 What if $extension.control exists? Is it a byproduct of the .in file
 from previous `make` run or a user file? What if we have both the .in
 and the make variable because people are confused? Or both the make
 variables and a .control and not .control.in? Etc...

There are ways to deal with those issue, I'm sure.

 * Always remove $extension.control in the `clean` targets
 
 Hell no, as you can bypass the .in mechanism and provide directly the
 .control file.

I'm saying disallow the .control file, only allow the control.in file.

David



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread David E. Wheeler
On Dec 10, 2010, at 7:32 AM, Tom Lane wrote:

 Are there any actual remaining use-cases for that sed step?  It's
 certainly vestigial as far as the contrib modules are concerned:
 it would be simpler and more readable to replace MODULE_PATHNAME with
 $libdir in the sources.  Unless somebody can point to a real-world
 use-case, I'd just as soon get rid of the .in files altogether while
 we're having this flag day.

I've made extensive use of them in pgTAP, but they don't depend on PGXS's doing 
its bit. So no. Unless we require control.in and not .control.

Best,

David
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Robert Haas
On Fri, Dec 10, 2010 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 I'm not convinced that this is actually a requirement, or that doing it
 this specific way is a good solution.  In particular, keeping the
 version number in the system catalogs seems pretty dubious.  The common
 method for upgrading an already-installed contrib module just involves
 dropping in a new .so --- that's not going to change the system
 catalogs.  It would likely be better to keep the version ID inside the
 .so file.

This is an interesting point.  There are really two things here: the
.so version, and the version of the system catalog entries.  For
example, imagine that an extension provides a single function, called
foo().  So we load up the .so and CREATE FUNCTION statement to match.
Later, the extension is so successful that the author writes a second
function, bar().  The new .so can (at least possibly) be used with the
old schema definitions, but the new schema definitions aren't
compatible with the old .so.  The logical upgrade process is to swap
out the .so first, and then add update the catalog definitions.

On the other hand, if you were dropping a deprecated function, you'd
need to do the steps in reverse order.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Fri, Dec 10, 2010 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 ... In particular, keeping the
 version number in the system catalogs seems pretty dubious.  The common
 method for upgrading an already-installed contrib module just involves
 dropping in a new .so --- that's not going to change the system
 catalogs.  It would likely be better to keep the version ID inside the
 .so file.

 This is an interesting point.  There are really two things here: the
 .so version, and the version of the system catalog entries.

True.  Consider a situation like an RPM upgrade: it's going to drop in a
new .so version, *and nothing else*.  It's pure fantasy to imagine that
the RPM script is going to find all your databases and execute some SQL
commands against them.  Since a large number of bug-fix cases do require
only a .so update, not being able to track the .so version seems like
it's missing most of the argument for having version tracking at all.

(In the RPM case, the RPM infrastructure would be able to tell you
which version you had installed, so I'm not sold that PG needs to
duplicate that.)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread David E. Wheeler
On Dec 10, 2010, at 10:20 AM, Tom Lane wrote:

 True.  Consider a situation like an RPM upgrade: it's going to drop in a
 new .so version, *and nothing else*.  It's pure fantasy to imagine that
 the RPM script is going to find all your databases and execute some SQL
 commands against them.  Since a large number of bug-fix cases do require
 only a .so update, not being able to track the .so version seems like
 it's missing most of the argument for having version tracking at all.

Sometimes there will be changes to the SQL, too. How does that work with CREATE 
EXTENSION? Do I install the upgrade, then run CREATE EXTENSION to get the 
latest SQL script to run? But then all the objects already exist…

Best,

David
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes:
 This doesn't answer my question of why it couldn't be done the other
 way.  Why does the makefile need to know it?  If it does need to know
 it, couldn't it get it out of the control file instead of vice versa?

Well the Makefile support is just a facility to fill in the control file
automatically for you, on the grounds that you're probably already
maintaining your version number in the Makefile. Or that it's easy to
get it there, as in:

EXTVERSION = $(shell dpkg-parsechangelog | awk -F '[:-]' '/^Version:/ { print 
substr($$2, 2) }')

That comes from a real world example that's yet to be adapted to being
an extension in 9.1, but still:

  https://github.com/dimitri/pgfincore/blob/debian/Makefile

 I'm not convinced that this is actually a requirement, or that doing it
 this specific way is a good solution.  In particular, keeping the
 version number in the system catalogs seems pretty dubious.  The common
 method for upgrading an already-installed contrib module just involves
 dropping in a new .so --- that's not going to change the system
 catalogs.  It would likely be better to keep the version ID inside the
 .so file.

Upgrade are left for a future patch, did we decide. Still, it seems to
me that we will support some upgrade scripts so that author can decide
what to do knowing current and next version, and yes, knowing that the
module has already been taken care of by the OS-level packaging.

That means some extensions upgrades will break the database between the
OS-level package upgrade and the sql upgrade (support to come), but in
my experience that's seldom the case. And not by chance.

So in the case that only the module (.so) needs upgrading, we would
still provide for an upgrade path in the script / sql support so that
the version number has a chance of being upgraded too.  As you say in
another mail, of course, the OS packaging system will not forcibly be
willing to care for that all by itself. I can imagine debian offering
the choice to the users and acting accordingly, though.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread David E. Wheeler
On Dec 10, 2010, at 11:28 AM, Dimitri Fontaine wrote:

 Well the Makefile support is just a facility to fill in the control file
 automatically for you, on the grounds that you're probably already
 maintaining your version number in the Makefile. Or that it's easy to
 get it there, as in:
 
 EXTVERSION = $(shell dpkg-parsechangelog | awk -F '[:-]' '/^Version:/ { print 
 substr($$2, 2) }')
 
 That comes from a real world example that's yet to be adapted to being
 an extension in 9.1, but still:
 
  https://github.com/dimitri/pgfincore/blob/debian/Makefile

I use that in pgTAP, too (line 23):

   https://github.com/theory/pgtap/blob/master/Makefile

But I don't need core to support that. Frankly, if we're not going to generate 
the control file from Makefile variables, then I'd rather not have any control 
file Makefile variables at all.

 Upgrade are left for a future patch, did we decide. Still, it seems to
 me that we will support some upgrade scripts so that author can decide
 what to do knowing current and next version, and yes, knowing that the
 module has already been taken care of by the OS-level packaging.

Yeah, this will be needed ASAP.

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes:
 On Dec 10, 2010, at 11:28 AM, Dimitri Fontaine wrote:
 Upgrade are left for a future patch, did we decide. Still, it seems to
 me that we will support some upgrade scripts so that author can decide
 what to do knowing current and next version, and yes, knowing that the
 module has already been taken care of by the OS-level packaging.

 Yeah, this will be needed ASAP.

I don't mind if we don't have an implementation of upgrade cases in
hand.  But we had better have a design in hand, to make sure what we're
doing now doesn't foreclose upgrade cases.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
 Tom Lane t...@sss.pgh.pa.us writes:
 This doesn't answer my question of why it couldn't be done the other
 way.  Why does the makefile need to know it?  If it does need to know
 it, couldn't it get it out of the control file instead of vice versa?

 Well the Makefile support is just a facility to fill in the control file
 automatically for you, on the grounds that you're probably already
 maintaining your version number in the Makefile.

Why would you choose to maintain it in the Makefile?  In most cases
makefiles are the least likely thing to be changing during a minor
update.  I would think that the right place for it is in the C code
(if we're trying to version .so files) or the .sql file, if we're trying
to version the SQL objects.  In particular, if the only reason the
makefile needs to know it is to inject it into the control file, it
seems completely silly to not just maintain it in the control file
instead.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-10 Thread David E. Wheeler
On Dec 10, 2010, at 11:47 AM, Tom Lane wrote:

 Why would you choose to maintain it in the Makefile?  In most cases
 makefiles are the least likely thing to be changing during a minor
 update.  I would think that the right place for it is in the C code
 (if we're trying to version .so files) or the .sql file, if we're trying
 to version the SQL objects.  In particular, if the only reason the
 makefile needs to know it is to inject it into the control file, it
 seems completely silly to not just maintain it in the control file
 instead.

+1

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Extensions, patch v16

2010-12-09 Thread David E. Wheeler
On Dec 9, 2010, at 12:34 PM, Dimitri Fontaine wrote:

 - add support for 'relocatable' boolean property in the control file,
   as discussed on list
 
   this controls what happens at create extension time, by doing a
   relocation of the extension objects when the extension is relocatable
   and the asked schema isn't the first of the search_path (you can't
   relocate an object to a schema where it already is)
 
   when the extension is not relocatable, the mechanism used is the
   @extschema@ replacement in the script so that the user still has a
   say, but at create time only

This still isn't ideal, but I think it's a big improvement. Thanks.

 - nothing is done for the psql commands \dx and \dx+, here's an idea:
 
   \dx lists only installed extensions
   \dx+ extension lists the objects, calling pg_extension_objects()
   \dX lists available extensions (and installed too)

+1 I think that's much more like existing psql commands.

 - we still depend on extension authors providing a control file. Do we
   want to spend some efforts on trying to get rid of this file? I know
   David desperately want to, but that's at the cost of making it much
   harder to manage more than one extension in a single directory, for
   once, and the Makefile mechanisms to make than happen (include a rule
   depending on the presence of some variables, keep track of it for the
   cleaning, etc) doesn't seem to me to worth it.

I don't think it makes it any harder to manage multiple extension in a single 
directory because one can create the control file explicitly (or perhaps rely 
on .control.in for that), just as they do now. Everyone else can do less work.

So:

* If $extension.control.in exists, use that
* If it doesn't, generate $extension.control from the Makefile variables
* Always remove $extension.control in the `clean` targets

Best,

David


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers