Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-19 Thread Robert Haas
On Wed, Dec 18, 2013 at 10:05 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Stephen Frost escribió: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Basically with building `UNIT` we realise with hindsight that we failed to build a proper `EXTENSION` system, and we send that message

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-19 Thread Cédric Villemain
Le jeudi 19 décembre 2013 14:01:17, Robert Haas a écrit : On Wed, Dec 18, 2013 at 10:05 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Stephen Frost escribió: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Basically with building `UNIT` we realise with hindsight that we

SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: On 17 December 2013 23:42, Tom Lane t...@sss.pgh.pa.us wrote: We aim to have the simplest implementation that meets the stated need and reasonable extrapolations of that. Text in a catalog table is the simplest implementation. That is not a reason to

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I keep telling you this, and it keeps not sinking in. How can you say that? I've been spending a couple of years on designing and implementing and arguing for a complete feature set where dealing with modules is avoided at all costs. The problem we have now

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Here's my attempt: # Inline Extension, Extension Templates The problem with *Inline Extension* is the dump and restore policy. The contents of an extensions are not be found in a `pg_dump` script, ever. You keep coming back to this and I

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Alvaro Herrera
Stephen Frost escribió: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Basically with building `UNIT` we realise with hindsight that we failed to build a proper `EXTENSION` system, and we send that message to our users. Little difficult to draw conclusions about what out 'hindsight'

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Greg Stark
Yeah I think this whole discussion is happening at the wrong level. The problem you're having, despite appearances, is not that people disagree about the best way to accomplish your goals. The problem is that not everyone is convinced your goals are a good idea. Either they just don't understand

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Right. I think a lot of the tension comes from people being unconvinced that the existing extension feature is an ideal model for this sort of use-case. Extensions were mainly designed around the notion of a .so The effort here is all about extending the

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: OTOH, for a set of pure-SQL objects, it's not necessary that there be a canonical text file somewhere, and we have in principle complete knowledge of the objects' semantics as well as the ability to dump-and-restore into newer PG versions.

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: My thinking is that if we invent a new mechanism for extensions that are not managed like contribs, we will find out that only contribs are going to be using extensions. That's only accurate if the new mechanism supports .so's, which seems unlikely to

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: My thinking is that if we invent a new mechanism for extensions that are not managed like contribs, we will find out that only contribs are going to be using extensions. That's only accurate if

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Stephen Frost sfr...@snowman.net writes: That's only accurate if the new mechanism supports .so's, which seems unlikely to be the case. Really? Yes, really. So now, you don't need anymore to have file system write privileges into a central

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Josh Berkus
On 12/16/2013 11:44 AM, Tom Lane wrote: Right. I think a lot of the tension comes from people being unconvinced that the existing extension feature is an ideal model for this sort of use-case. Extensions were mainly designed around the notion of a .so with some SQL datatype/function/etc

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Simon Riggs
On 13 December 2013 18:42, Stephen Frost sfr...@snowman.net wrote: * Jeff Davis (pg...@j-davis.com) wrote: For what it's worth, I think the idea of extension templates has good conceptual integrity. Extensions are external blobs. To make them work more smoothly in several ways, we move them

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: I keep seeing people repeat I don't like blobs as if that were an objection. There is no danger or damage from doing this. I can't see any higher beauty that we're striving for by holding out. Why not allow the user to choose XML, JSON, YAML, or

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Heikki Linnakangas
On 12/17/2013 08:32 PM, Stephen Frost wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: My only personal interest in this is to stimulate the writing of further extensions, which is fairly clearly hampered by the overhead required for packaging. I'm not convinced of that but I agree that we

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Simon Riggs
On 17 December 2013 17:54, Tom Lane t...@sss.pgh.pa.us wrote: So now, you don't need anymore to have file system write privileges into a central place owned by root, it can be anywhere else, Modern OSes have security checks that can prevent loading libraries from random places. This is

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Simon Riggs
On 17 December 2013 18:32, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: I keep seeing people repeat I don't like blobs as if that were an objection. There is no danger or damage from doing this. I can't see any higher beauty that we're striving for by

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 17 December 2013 18:32, Stephen Frost sfr...@snowman.net wrote: I have no idea where you're going with this, but I *do* object to sticking an SQL script which defines a bunch of objects into a catalog table *right next to where they are properly

Re: [HACKERS] Extension Templates S03E11

2013-12-17 Thread Simon Riggs
On 17 December 2013 23:42, Tom Lane t...@sss.pgh.pa.us wrote: We aim to have the simplest implementation that meets the stated need and reasonable extrapolations of that. Text in a catalog table is the simplest implementation. That is not a reason to reject it, especially when we aren't

Re: [HACKERS] Extension Templates S03E11

2013-12-16 Thread Josh Berkus
On 12/15/2013 10:47 PM, Jeff Davis wrote: The patch offers an alternative to dropping files on the filesystem before doing CREATE EXTENSION. Instead, if the extension has no C code, you can put it in the catalog using ordinary SQL access, and execute the same kind of CREATE EXTENSION. Aside

Re: [HACKERS] Extension Templates S03E11

2013-12-16 Thread Josh Berkus
On 12/16/2013 10:53 AM, Josh Berkus wrote: Some PostgreSQL shops with lots of servers have large internal libraries of functions, views, and similar code that they've written to support their applications, which don't comprise a complete database. This feature would allow them to package

Re: [HACKERS] Extension Templates S03E11

2013-12-16 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: So, the reason I was confused by this feature -- and the reason Stephen hates it, I think -- is that I thought it was solving the Extensions don't follow replication, and they are complicated to install if your OS doesn't have good packages problem. It's

Re: [HACKERS] Extension Templates S03E11

2013-12-16 Thread Jim Nasby
On 12/16/13, 1:00 PM, Josh Berkus wrote: On 12/16/2013 10:53 AM, Josh Berkus wrote: Some PostgreSQL shops with lots of servers have large internal libraries of functions, views, and similar code that they've written to support their applications, which don't comprise a complete database. This

Re: [HACKERS] Extension Templates S03E11

2013-12-15 Thread Jeff Davis
On Sat, 2013-12-14 at 13:46 -0800, Josh Berkus wrote: All: Can someone summarize the issues with this patch for those of us who haven't been following it closely? I was just chatting with a couple other contributors, and at this point none of just know what it implements, what it doesn't

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Jeff Davis
On Fri, 2013-12-13 at 13:42 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: For what it's worth, I think the idea of extension templates has good conceptual integrity. Extensions are external blobs. To make them work more smoothly in several ways, we move them into the

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: To make much more progress, it seems like we either need an ingenious idea about how to change existing extensions to work for all purposes, or we need to invent a new concept. I'm still in favor of supporting SQL/trusted-language only 'extensions' and

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Josh Berkus
All: Can someone summarize the issues with this patch for those of us who haven't been following it closely? I was just chatting with a couple other contributors, and at this point none of just know what it implements, what it doesn't implement, what the plans are for expanding its feature set

Re: [HACKERS] Extension Templates S03E11

2013-12-13 Thread Jeff Davis
On Wed, 2013-12-11 at 20:49 +0100, Dimitri Fontaine wrote: Robert Haas robertmh...@gmail.com writes: I strongly agree. PostgreSQL has succeeded because we try not to do things at all until we're sure we know how to do them right. I still agree to the principle, or I wouldn't even try. Not

Re: [HACKERS] Extension Templates S03E11

2013-12-13 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: For what it's worth, I think the idea of extension templates has good conceptual integrity. Extensions are external blobs. To make them work more smoothly in several ways, we move them into the catalog. They have pretty much the same upsides and

Re: [HACKERS] Extension Templates S03E11

2013-12-13 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: Stephen had some legitimate concerns. I don't entirely agree, but they are legitimate concerns, and we don't want to just override them. The main disturbing concern for me is to do with pg_restore and major upgrades, where the blob we have in the

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Dimitri Fontaine
Hi, Stephen Frost sfr...@snowman.net writes: * Jeff Davis (pg...@j-davis.com) wrote: What is stopping Extension Templates, as proposed, from being this special extension creation mode? What would be a better design? The extra catalog tables which store SQL scripts in text columns is one of

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Stephen Frost sfr...@snowman.net writes: The extra catalog tables which store SQL scripts in text columns is one of my main objections to the as-proposed Extension Templates. I view those scripts as a poor man's definition of database objects

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 10:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: So while I hear your objection to the script in catalog idea Stephen, I think we should move forward. We don't have the luxury of only applying patches where no compromise has to be made, where everyone is fully happy with

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: You've got that backwards. We do have the luxury of rejecting new features until people are generally satisfied that the basic design is right. There's no overlord decreeing that this must be in 9.4. I strongly agree. PostgreSQL has succeeded

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Stephen Frost
Dimitri, * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: The extra catalog tables which store SQL scripts in text columns is one of my main objections to the as-proposed Extension Templates. I view those scripts as a poor man's definition of

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:49 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: You've got that backwards. We do have the luxury of rejecting new features until people are generally satisfied that the basic design is right. There's no overlord

Re: [HACKERS] Extension Templates S03E11

2013-12-09 Thread Robert Haas
On Sat, Dec 7, 2013 at 3:12 AM, Jeff Davis pg...@j-davis.com wrote: So if we do it this way, then we should pick a new name, like package. That was my first reaction as well, when I looked at this a few years ago, but I've since backed away from that position. You're certainly correct that it's

Re: [HACKERS] Extension Templates S03E11

2013-12-09 Thread Jeff Davis
On Mon, 2013-12-09 at 12:17 -0500, Robert Haas wrote: On Sat, Dec 7, 2013 at 3:12 AM, Jeff Davis pg...@j-davis.com wrote: So if we do it this way, then we should pick a new name, like package. That was my first reaction as well, when I looked at this a few years ago, but I've since backed

Re: [HACKERS] Extension Templates S03E11

2013-12-07 Thread Jeff Davis
On Wed, 2013-12-04 at 14:54 -0500, Tom Lane wrote: I think Stephen has already argued why it could be a good idea here. But in a nutshell: it seems like there are two use-cases to be supported, one where you want CREATE EXTENSION hstore to give you some appropriate version of hstore, and one

Re: [HACKERS] Extension Templates S03E11

2013-12-07 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: On Wed, 2013-12-04 at 14:54 -0500, Tom Lane wrote: I think Stephen has already argued why it could be a good idea here. But in a nutshell: it seems like there are two use-cases to be supported, one where you want CREATE EXTENSION hstore to give you

Re: [HACKERS] Extension Templates S03E11

2013-12-07 Thread Jeff Davis
On Sat, 2013-12-07 at 12:27 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: The behavior of an extension should not depend on how it was installed. The kind of extension being described by Stephen will: * Not be updatable by doing ALTER EXTENSION foo UPDATE TO

Re: [HACKERS] Extension Templates S03E11

2013-12-07 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: I understand there are reasons, but I'm having a hard time getting past the idea that I have extension foo v1.2 now needs to be qualified with installed using SQL or installed using the filesystem to know what you actually have and how it will behave. I

Re: [HACKERS] Extension Templates S03E11

2013-12-05 Thread Peter Eisentraut
On 12/1/13, 10:47 PM, Stephen Frost wrote: Having a management system for sets of objects is a *great* idea- and one which we already have through schemas. What we don't have is any kind of versioning system built-in or other metadata about it, nor do we have good tooling which leverages such

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Tue, 2013-12-03 at 08:44 -0500, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: On 3 December 2013 02:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: ISTM that the real solution to this particular problem is to decouple the extensions that are currently in contrib from

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Mon, 2013-12-02 at 15:44 -0500, Stephen Frost wrote: How are we going to handle new keywords being added in new major versions? A pg_dump of the extension template script is then going to be loaded into the new major version but will not actually be able to be run because it'll error

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Tue, 2013-12-03 at 10:23 -0500, Robert Haas wrote: In more normal cases, however, the system can (and probably should) figure out what was intended by choosing the *shortest* path to get to the intended version. For example, if someone ships 1.0, 1.0--1.1, 1.1, and 1.1--1.2, the system

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: On Tue, 2013-12-03 at 14:31 -0500, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: On Mon, 2013-12-02 at 15:44 -0500, Stephen Frost wrote: How are we going to handle new keywords being added in new major versions? A pg_dump of the extension template script is then going to be loaded into the new major version but will not

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Wed, 2013-12-04 at 09:50 -0500, Stephen Frost wrote: I still don't see that Extension Templates are all bad: * They preserve the fact that two instances of the same extension (e.g. in different databases) were created from the same template. This is only true if we change the

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Tue, 2013-12-03 at 14:31 -0500, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to implement the dump/reload than a text blob which is

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Tue, 2013-12-03 at 14:31 -0500, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to implement the

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Robert Haas
On Tue, Dec 3, 2013 at 11:44 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: We should also consider the possibility of a user trying to deliberately install and older release. For example, if the user has 1.0, 1.0--1.1, 1.1, 1.1--1.2, and 1.2--1.0 (a downgrade script) with

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 3:39 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2013-12-03 at 10:23 -0500, Robert Haas wrote: In more normal cases, however, the system can (and probably should) figure out what was intended by choosing the *shortest* path to get to the intended version. For

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Peter Eisentraut
On 12/2/13, 9:14 AM, Dimitri Fontaine wrote: What I want to build is an “extension distribution” software that knows how to prepare anything from PGXN (and other places) so that it's fully ready for being used in the database. Then the main client would run as a CREATE EXTENSION

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Peter Eisentraut
On 12/2/13, 2:33 PM, Greg Stark wrote: Just tossing an idea out there. What if you could install an extension by specifying not a local file name but a URL. Obviously there's a security issue but for example we could allow only https URLs with verified domain names that are in a list of

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Peter Eisentraut
On 12/3/13, 9:20 AM, Stephen Frost wrote: Another option, which I generally like better, is to have a new package format for PGXN that contains the results of make install, more-or-less, synonymous to Debian source vs. .deb packages. Perhaps we could even have psql understand that format and

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 12/3/13, 9:20 AM, Stephen Frost wrote: Another option, which I generally like better, is to have a new package format for PGXN that contains the results of make install, more-or-less, synonymous to Debian source vs. .deb packages. Perhaps

Re: [HACKERS] Extension Templates S03E11

2013-12-04 Thread Jeff Davis
On Wed, 2013-12-04 at 15:28 -0500, Robert Haas wrote: My experience with software upgrades is that cases like this, and even weirder things, happen pretty routinely, so I think more control is good. There would still be control: just use full SQL scripts appropriately. I'm sure there's still

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Heikki Linnakangas
On 12/03/2013 09:25 AM, Jeff Davis wrote: On Mon, 2013-12-02 at 11:07 +0200, Heikki Linnakangas wrote: There should be no difference between file-based extensions and catalog-based extensions. It's just two different ways to install the same extension. The extension author doesn't need to care

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 1:31 AM, Jeff Davis pg...@j-davis.com wrote: On Sun, 2013-12-01 at 15:48 +0100, Dimitri Fontaine wrote: Jeff Davis pg...@j-davis.com writes: I don't see why we are trying to accommodate a case where the author doesn't offer enough full SQL scripts and offers broken

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Greg Stark
On Mon, Dec 2, 2013 at 7:46 PM, Robert Haas robertmh...@gmail.com wrote: Just tossing an idea out there. What if you could install an extension by specifying not a local file name but a URL. Obviously there's a security issue but for example we could allow only https URLs with verified domain

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: On 3 December 2013 02:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: ISTM that the real solution to this particular problem is to decouple the extensions that are currently in contrib from a specific postgres version. Problem? It's not a bug

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Can we separate this feature out? It's an issue with extensions today, and I'm eager to make some progress after the explosion of differing opinions today. +1 for separating that part out. I thought it was separated, at some point.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Tom Dunstan (pg...@tomd.cc) wrote: Extensions in contrib live in a weird place. Totally builtin stuff should obviously be dumped without versions, and stuff which is completely separate and follows its own release schedule should obviously be versioned. I guess we consider all modules in

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN today. Not at all- and that'd be the point. An external tool could take the PGXN extension, run 'make',

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Greg Stark (st...@mit.edu) wrote: I thought the fundamental problem the in-catalog extensions were trying to solve were the issue with not having access to the filesystem. If that's the case then being able to say create extension from http://... would solve that. That's not really 'solved'

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 8:44 AM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: On 3 December 2013 02:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: ISTM that the real solution to this particular problem is to decouple the extensions that are currently in

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Robert Haas
On Tue, Dec 3, 2013 at 8:43 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: What Jeff is proposing is to simplify that down and have PostgreSQL auto discover the upgrade cycle when the version asked for isn't directly available with a creation script. We would keep the behavior depicted

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: That's not really 'solved' unless you feel we can depend on that create extension from URL to work at pg_restore time... I wouldn't have guessed that people would accept that, but I've already been wrong about such things in this thread once.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: That's not really 'solved' unless you feel we can depend on that create extension from URL to work at pg_restore time... I wouldn't have guessed that people would accept that, but I've already been

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: In more normal cases, however, the system can (and probably should) figure out what was intended by choosing the *shortest* path to get to the intended version. For example, if someone ships 1.0, 1.0--1.1, 1.1, and 1.1--1.2, the system should choose

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: I understand that you once proposed that and it was shot down but I think we need to move past that now that we've seen what the alternative is.. That isn't to say anything about the code or about you specifically, but, for my part, I really don't like

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread David E. Wheeler
On Dec 3, 2013, at 9:14 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I understand that it can happen, it still really sucks when it does. delusionnal paragraph, censored for lack of humour (incl. sarcasm) I have not followed this project closely, Dimitri, but I for one have

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 10:08 +0200, Heikki Linnakangas wrote: Another perspective is that that's already a situation we'd rather not have. Let's not make it worse by introducing a third way to install an extension, which again requires the extension author to package the extension

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 09:20 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN today. Not at all- and that'd be the point.

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: This sounds like Inline Extensions to me, which was previously proposed. I've not looked at that proposal very carefully, but I agree that what we're describing is a lot closer to 'inline extensions' than 'extension templates'. If I recall, that proposal

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to implement the dump/reload than a text blob which is carried forward from major version to major version

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: On Tue, 2013-12-03 at 09:20 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: Stephen mentioned using external tools and/or metadata, but to me that sounds like it would require porting the extension away from what's on PGXN

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 14:41 -0500, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: The extra catalog tables which store SQL scripts in text columns is one of my main objections to the as-proposed Extension Templates. OK, that's what I thought. This seems like the root of your

Re: [HACKERS] Extension Templates S03E11

2013-12-03 Thread Jeff Davis
On Tue, 2013-12-03 at 14:31 -0500, Tom Lane wrote: Stephen Frost sfr...@snowman.net writes: When it comes to dump/reload, I'd much rather see a mechanism which uses our deep understanding of the extension's objects (as database objects) to implement the dump/reload than a text blob which is

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Heikki Linnakangas
On 12/02/2013 05:34 AM, Stephen Frost wrote: * Jeff Davis (pg...@j-davis.com) wrote: I see where you're coming from, but after some thought, and looking at the patch, I think we really do want a catalog representation for (at least some) extensions. Perhaps I'm missing something- but we

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Andres Freund
On 2013-12-02 11:07:28 +0200, Heikki Linnakangas wrote: Perhaps I'm missing something- but we already *have* a catalog representation for every extension that's ever installed into a given database. A representation that's a heck of a lot better than a big text blob. Right. I think Jeff

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Heikki Linnakangas hlinnakan...@vmware.com writes: Right. I think Jeff was thinking of a catalog representation for extensions that haven't been installed yet, but are available in the system and could be installed with CREATE EXTENSION foo. I wouldn't mind having a catalog like that. Even

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: Having a versioning notion (and whatever other meta data we, or an extension author, feels is useful) for what are otherwise simple containers (aka the schematic we already have..) makes sense and it would be great to provide support around that,

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Heikki Linnakangas (hlinnakan...@vmware.com) wrote: Right. I think Jeff was thinking of a catalog representation for extensions that haven't been installed yet, but are available in the system and could be installed with CREATE EXTENSION foo. I really don't particularly see value in this

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: There should be no difference between file-based extensions and catalog-based extensions. It's just two different ways to install the same extension. The extension author doesn't need to care about that, it's the DBA that decides which

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: What is the next step to allow an extension pulled down from pgxn to be installed, unchanged, into a given database? An extension packaging system. Unchanged is not a goal, and not possible even today. PGXN is a *source based* packaging system. You

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: *That doesn't make this approach the right one*. If anything, I'm afraid we've ended up building ourselves a rube goldberg machine because of this constant struggle to fit a square peg into a round

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: What is the next step to allow an extension pulled down from pgxn to be installed, unchanged, into a given database? An extension packaging system. Unchanged is not a goal, and not possible even

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Heikki Linnakangas
On 12/02/2013 04:14 PM, Dimitri Fontaine wrote: Stephen Frost sfr...@snowman.net writes: What is the next step to allow an extension pulled down from pgxn to be installed, unchanged, into a given database? An extension packaging system. Unchanged is not a goal, and not possible even today.

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: Clearly we need the information from the extension package (the scripts which are on the PG server's filesystem today, but need not be in the future) but that doesn't mean we need to keep those text blobs in the catalog. So, I guess it would have been

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: Fine- so we need a step that goes from 'source' to 'built'. I don't see that step being done in or by a PG backend process. Adding a new option which can take a pgxn source and build a script from it which can be run against PG via libpq is what I'd

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Heikki Linnakangas hlinnakan...@vmware.com writes: I fear we're wandering off the point again. So let me repeat: It must be possible to install the same extension the way you do today, and using the new mechanism. The way you do today is running make install or apt-get install or something

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Then as soon as we are able to CREATE EXTENSION mystuff; without ever pre-installing files on the file system as root, then we would like to be able to do just that even with binary modules. I really

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Stephen Frost sfr...@snowman.net writes: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Then as soon as we are able to CREATE EXTENSION mystuff; without ever pre-installing files on the file system as root, then we would like to be able to do just that

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: Clearly we need the information from the extension package (the scripts which are on the PG server's filesystem today, but need not be in the future) but that doesn't mean we need to keep those text

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: Fine- so we need a step that goes from 'source' to 'built'. I don't see that step being done in or by a PG backend process. Adding a new option which can take a pgxn source and build a script from

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: On the other hand, I can appreciate the concern that we don't really want a dump/restore to include the extension definition when it's already on the filesystem. That said, it amazes me that we don't include the version # of the extension in pg_dump's

Re: [HACKERS] Extension Templates S03E11

2013-12-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Tom Lane t...@sss.pgh.pa.us writes: Yes, exactly. What's more, you're going to face huge push-back from vendors who are concerned about security (which is most of them). Last time I talked with vendors, they were working in the Open Shift

  1   2   >