Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread David E. Wheeler
On Feb 4, 2011, at 8:54 AM, Robert Haas wrote: > Yeah, I understand. I believe Tom said he was going to look at the > basic functionality with an eye toward commit, and I hope to look at > it as well, either before or after it gets committed. Time > permitting, I'd then like to look at this, but

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 11:48 AM, David E. Wheeler wrote: > Robert, I think that the core extension if pretty uncontroversial, modulo > some minor issues. It's the upgrade process that's more controversial. I > think the case can be made to accept even that part as Dim has written it, > because

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread David E. Wheeler
On Feb 4, 2011, at 7:48 AM, Robert Haas wrote: > I'd like to answer this question, but I have not had enough time to > read through this patch in detail, because there are 97 patches in > this CommitFest. The point I'm trying to make, however, is > procedural. We shouldn't commit anything at the

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Fri, Feb 4, 2011 at 10:13 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> I don't think we should commit something that for 9.1 that we may need >> to change incompatibly for 9.2.  If we're not completely happy with >> it, it gets booted.  Whatever we put in place here is going to be with

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Dimitri Fontaine
Robert Haas writes: > I don't think we should commit something that for 9.1 that we may need > to change incompatibly for 9.2. If we're not completely happy with > it, it gets booted. Whatever we put in place here is going to be with > us for a long, long time. So, what is it specifically here

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Robert Haas
On Thu, Feb 3, 2011 at 4:24 PM, Josh Berkus wrote: > So every package would include a script called upgrade.sql ( or > upgrade.c? ) which is supposed to handle the upgrade, and it's up to the > module author to power that, at least until 9.2?  Seem like the most > reasonable course for February ..

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-04 Thread Dimitri Fontaine
Josh Berkus writes: > How *are* we detecting which version is installed, anyway? Is that in > the pg_extenstions table? The installed version is in the pg_extenstion catalog, the version we're upgrading to is in the control file and can be seen in the system view pg_available_extensions or from

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Josh Berkus
> FWIW, I think that last objection is bogus. There's no reason that an > extension author can't leave dummy C functions in his code to support > obsolete CREATE FUNCTION calls. (As an example, I added one to > Alexander Korotkov's recent pg_trgm patch. It consists of 10 lines of > boilerplate

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread David E. Wheeler
On Feb 3, 2011, at 11:35 AM, Josh Berkus wrote: > (D) Requires a series of ordered upgrade scripts in sortable version > numbering, each of which gets applied in order between the two versions. > This initially seems like the most attractive option -- and is the one > used by dozens of popular ope

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Tom Lane
Josh Berkus writes: > (D) Requires a series of ordered upgrade scripts in sortable version > numbering, each of which gets applied in order between the two versions. > This initially seems like the most attractive option -- and is the one > used by dozens of popular open source web applications -

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Josh Berkus
> Well if you want to support upgrades between each two versions, that > means you have users and you don't know what they currently have > installed. Then you have this problem to solve, and it's complex the > same no matter what tools are offered. How *are* we detecting which version is instal

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Josh Berkus writes: > Let me summarize the prospective solutions based on some chatting with > some potential extension authors (that is, folks who maintain in-house > stuff they're thinking of offering as extensions). Especially since I > think at this point the majority of -hackers has lost tra

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Josh Berkus
All, Let me summarize the prospective solutions based on some chatting with some potential extension authors (that is, folks who maintain in-house stuff they're thinking of offering as extensions). Especially since I think at this point the majority of -hackers has lost track of the argument: (A

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 2:02 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Feb 3, 2011 at 1:10 PM, David E. Wheeler >> wrote: >>> On Feb 3, 2011, at 10:07 AM, Tom Lane wrote: Well, pg_upgrade is designed to work within a major-version series, eg you could do a 9.1-to-9.1 upgrade

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread David E. Wheeler
On Feb 3, 2011, at 11:02 AM, Tom Lane wrote: >> That's putting it mildly. It's more like sending a rocket into outer >> space to tweak the orbit of a comet so that it crashes into your >> barbecue grill to light a fire so you can roast marshmallows. LOL. > No, it's more like using a sledgehamme

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 3, 2011 at 1:10 PM, David E. Wheeler wrote: >> On Feb 3, 2011, at 10:07 AM, Tom Lane wrote: >>> Well, pg_upgrade is designed to work within a major-version series, eg >>> you could do a 9.1-to-9.1 upgrade if you needed to install a newer >>> version of an extensi

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 1:10 PM, David E. Wheeler wrote: > On Feb 3, 2011, at 10:07 AM, Tom Lane wrote: > >> Well, pg_upgrade is designed to work within a major-version series, eg >> you could do a 9.1-to-9.1 upgrade if you needed to install a newer >> version of an extension.  Admittedly, this is

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Tom Lane writes: > Now having said that, it does occur to me that there is an upgrade-ish > scenario that every user is going to hit immediately, which is how to > get from an existing installation with a pile of "loose" objects created > by one or more contrib modules to a state where those objec

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Tom Lane writes: > Maybe somewhere around here we should stop and ask why we are bothering > with any of this. The original idea for an extension concept was that > (1) some collection of objects could be designated as a module > (2) pg_dump would be taught to dump "LOAD MODULE foo" instead of th

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread David E. Wheeler
On Feb 3, 2011, at 10:07 AM, Tom Lane wrote: > Well, pg_upgrade is designed to work within a major-version series, eg > you could do a 9.1-to-9.1 upgrade if you needed to install a newer > version of an extension. Admittedly, this is swinging a rather larger > hammer than "apply an upgrade script

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Tom Lane
"David E. Wheeler" writes: > I think we will need to come back to it before, long, however, because many > extensions are released far more often than major versions of PostgreSQL. So > while one might run pg_upgrade, at most, about once every 12-18 months, they > will often want to take advant

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread David E. Wheeler
On Feb 3, 2011, at 9:54 AM, Robert Haas wrote: > I think you can pretty much take it to the bank that there will be > demand. This is an important, real-world problem. > > That having been said, I'm not 100% convinced that the main extensions > patch is ready for prime-time, and I'm even less co

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 12:38 PM, Tom Lane wrote: > Florian Pflug writes: >> I fully agree. The extension infrastructure should provide basic support >> for upgrades, not every kind of bell and whistle one could possible think of. > > Maybe somewhere around here we should stop and ask why we are b

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread David E. Wheeler
On Feb 3, 2011, at 9:38 AM, Tom Lane wrote: > Given that pg_upgrade is now considered a supported piece of the system, > ISTM that most real-world upgrade scenarios will be accomplished with > pg_upgrade relying on provision (3). It looks to me like we're talking > about adding a large amount of

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Tom Lane
Florian Pflug writes: > I fully agree. The extension infrastructure should provide basic support > for upgrades, not every kind of bell and whistle one could possible think of. Maybe somewhere around here we should stop and ask why we are bothering with any of this. The original idea for an exte

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 11:53 AM, Dimitri Fontaine wrote: > Robert Haas writes: >>> Yes, from the backend-developer's perspective. But not from the >>> extension-developer's perspective :-) And seriously, make is one of >>> those things that is supremely capable of doing lots of stuff, but is so >

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Florian Pflug
On Feb3, 2011, at 16:31 , Dimitri Fontaine wrote: > "Ross J. Reedstrom" writes: >> Hmm, how about allowing a list of files to execute? That allows the > > Sure. I still don't see why doing it in the control file is better than > in the Makefile, even if it's already better than in the SQL script

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Robert Haas writes: >> Yes, from the backend-developer's perspective. But not from the >> extension-developer's perspective :-) And seriously, make is one of >> those things that is supremely capable of doing lots of stuff, but is so >> difficult to use correctly that everyone keeps reinventing ne

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Tom Lane
=?iso-8859-2?Q?=AEiga_Kranjec?= writes: > This might be a silly idea, but I'm not sure if it was really > discussed with respect to extensions here before... yes, it was ... > Why not use PL/pgSQL as install/upgrade script language, > specially now when it's included in the core by default? "I

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 11:25 AM, Ross J. Reedstrom wrote: >> Speaking about which, using Make rules to prepare your upgrade files >> from other pieces means no development at all on the backend side.  You >> can hardly beat that. > > Yes, from the backend-developer's perspective. But not from the

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Ross J. Reedstrom
On Thu, Feb 03, 2011 at 04:31:08PM +0100, Dimitri Fontaine wrote: > "Ross J. Reedstrom" writes: > > Hmm, how about allowing a list of files to execute? That allows the > > Sure. I still don't see why doing it in the control file is better than > in the Makefile, even if it's already better than

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
"Ross J. Reedstrom" writes: > Hmm, how about allowing a list of files to execute? That allows the Sure. I still don't see why doing it in the control file is better than in the Makefile, even if it's already better than in the SQL script, at least in terms of code to write to support the idea.

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Robert Haas
On Thu, Feb 3, 2011 at 9:53 AM, Ross J. Reedstrom wrote: >> Example: >> >> upgrade_from_1_0 = '1.0 => upgrade_from_1.0.sql' >> upgrade_from_2_0 = '2.0 => upgrade_from_2.0.sql' >> upgrade_from_3_0 = '3.0 => upgrade_from_3.0.sql' > > Hmm, how about allowing a list of files to execute? That allows th

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Ross J. Reedstrom
On Thu, Feb 03, 2011 at 10:21:28AM +0200, Anssi Kääriäinen wrote: > On 02/02/2011 08:22 PM, Dimitri Fontaine wrote: >> Either one line in the Makefile or a new file with the \i equivalent >> lines, that would maybe look like: >> >>SELECT pg_execute_sql_file('upgrade.v14.sql'); >>SELECT pg_e

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Žiga Kranjec
This might be a silly idea, but I'm not sure if it was really discussed with respect to extensions here before... Why not use PL/pgSQL as install/upgrade script language, specially now when it's included in the core by default? This would allow for relatively straightforward inclusion of condit

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Anssi Kääriäinen writes: > upgrade_from_2.0.sql contents: > alter table foobar add column id3 integer; > pg_execute_extension_file('upgrade_from_3.0.sql'); > > ... > > So, when creating a new version you would need to update the main .sql file, > create a new upgrade file, and alter the upgrade_fr

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Dimitri Fontaine
Robert Haas writes: > Stepping back from the implementation details and file naming > conventions a bit, it seems to me that when you do schema upgrades, > there are basically three possible things you might want to put in the > upgrade script: [...] > I've managed schema upgrades that went throug

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Anssi Kääriäinen
On 02/03/2011 12:23 AM, Robert Haas wrote: [..] -- unconditional stuff [..6] -- stuff to do if coming from pre-7 [..] -- some more unconditional stuff [6..12] -- stuff to do if coming from between 6 and 12 [..] -- a few more unconditional things This might be a stupid idea, but how about intr

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-03 Thread Anssi Kääriäinen
On 02/02/2011 08:22 PM, Dimitri Fontaine wrote: Either one line in the Makefile or a new file with the \i equivalent lines, that would maybe look like: SELECT pg_execute_sql_file('upgrade.v14.sql'); SELECT pg_execute_sql_file('upgrade.v15.sql'); So well… I don't see how you've made it les

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Tom Lane
Aidan Van Dyk writes: > My concern with this approach (upgrade is forced through all > intermetiary versions) is that the shared libray now for version 15 > *has* to have all the "intermediary" compatibility for *all* versions > in it. So it has to have functions with all symbols so the "CREATE >

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Robert Haas
On Wed, Feb 2, 2011 at 12:31 PM, David E. Wheeler wrote: > They are identical except for the extra line in the second one. If I had, say > 15 different versions of an extension, then I'd have 15 upgrade scripts. > That's fine. But in your plan, the script to upgrade from version 1 to > version

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Aidan Van Dyk
On Wed, Feb 2, 2011 at 12:31 PM, David E. Wheeler wrote: > > They are identical except for the extra line in the second one. If I had, say > 15 different versions of an extension, then I'd have 15 upgrade scripts. > That's fine. But in your plan, the script to upgrade from version 1 to > versi

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
"David E. Wheeler" writes: > On Feb 2, 2011, at 10:14 AM, Aidan Van Dyk wrote: > >> My concern with this approach (upgrade is forced through all >> intermetiary versions) is that the shared libray now for version 15 >> *has* to have all the "intermediary" compatibility for *all* versions >> in it

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 10:22 AM, Dimitri Fontaine wrote: > Either one line in the Makefile or a new file with the \i equivalent > lines, that would maybe look like: > > SELECT pg_execute_sql_file('upgrade.v14.sql'); > SELECT pg_execute_sql_file('upgrade.v15.sql'); > > So well… I don't see how you'

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 10:14 AM, Aidan Van Dyk wrote: > My concern with this approach (upgrade is forced through all > intermetiary versions) is that the shared libray now for version 15 > *has* to have all the "intermediary" compatibility for *all* versions > in it. So it has to have functions with

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
"David E. Wheeler" writes: >> upgrade.null-v15.sql: upgrade.v14.sql upgrade.v15.sql >> cat upgrade.v14.sql upgrade.v15.sql > $@ > > Sure, if you know Make really well. But then I need to add a line to > the Makefile for every bloody upgrade script. Gross. Either one line in the Makefile or a

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 10:04 AM, Dimitri Fontaine wrote: > "David E. Wheeler" writes: >> They are identical except for the extra line in the second one. If I >> had, say 15 different versions of an extension, then I'd have 15 >> upgrade scripts. That's fine. But in your plan, the script to upgrade >>

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
"David E. Wheeler" writes: > They are identical except for the extra line in the second one. If I > had, say 15 different versions of an extension, then I'd have 15 > upgrade scripts. That's fine. But in your plan, the script to upgrade > from version 1 to version 15 would have all the same code a

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 9:19 AM, Dimitri Fontaine wrote: > I see there no solution to your reaction here. Please take the time to > tell us more about what exactly it is that you hated, and how to make it > lovely. We won't make any progress with your current commenting style. Here is your example o

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
"David E. Wheeler" writes: > On Feb 2, 2011, at 9:03 AM, Dimitri Fontaine wrote: > >> Well, fair enough I suppose. Or it would be if you gave me an >> alternative that provides a simpler way to support those 3 upgrades. > > I did: a naming convention with upgrade scripts that have the version >

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 9:03 AM, Dimitri Fontaine wrote: > Well, fair enough I suppose. Or it would be if you gave me an > alternative that provides a simpler way to support those 3 upgrades. I did: a naming convention with upgrade scripts that have the version number in them. You rejected it. > Of

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
"David E. Wheeler" writes: > On Feb 2, 2011, at 6:45 AM, Dimitri Fontaine wrote: > >> Well, you write 3 scripts. >> >> Let's consider an example, the lo contrib, with its 3 objects: >> >> CREATE DOMAIN lo AS pg_catalog.oid; >> CREATE OR REPLACE FUNCTION lo_oid(lo) … >> CREATE OR REPLACE FUNCT

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread David E. Wheeler
On Feb 2, 2011, at 6:45 AM, Dimitri Fontaine wrote: > Well, you write 3 scripts. > > Let's consider an example, the lo contrib, with its 3 objects: > > CREATE DOMAIN lo AS pg_catalog.oid; > CREATE OR REPLACE FUNCTION lo_oid(lo) … > CREATE OR REPLACE FUNCTION lo_manage() … > > Now, the upgrad

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
Itagaki Takahiro writes: > The latest extension might drop some functions. Then the upgrade script contains the DROP commands. > I'm still not clear what "upgrade" means. if module authors wrote > functions with C, they can just replace .so to upgrade. If with > SQL or PL/pgSQL, they should exec

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Itagaki Takahiro
On Wed, Feb 2, 2011 at 20:29, Dimitri Fontaine wrote: > I didn't think about "detach", I'm not sure I see the use case… The latest extension might drop some functions. > It's not about upgrading major versions, it's about upgrading > extensions.  The only time you will need to run the scripts in

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Dimitri Fontaine
Itagaki Takahiro writes: > Excuse me for asking, but could you explain what is the purpose? > Which is true, "upgrade to 9.1 from past versions" or "upgrade > from 9.1 to future versions"? Also, how much advantage will we > have compared with uninstall_MODULE.sql + CREATE EXTENSION? Both are "tr

Re: [HACKERS] ALTER EXTENSION UPGRADE, v3

2011-02-02 Thread Itagaki Takahiro
On Wed, Feb 2, 2011 at 03:21, Dimitri Fontaine wrote: > PFA version 3 of the ALTER EXTENSION PATCH, cleaned and merged against > recent HEAD and extension's branch from which I just produced the v30 > patch. Excuse me for asking, but could you explain what is the purpose? Which is true, "upgrade

<    1   2