Re: [HACKERS] Extensions Dependency Checking
Aidan Van Dyk ai...@highrise.ca writes: I think the general movement is toward *feature* dependancies. So for intstance, an extension can specify what *feature* it requires, and difference versions of an extension can provide different features. That sounds like what Emacs is doing too. But checking http://developer.postgresql.org/pgdocs/postgres/extend-extensions.html, I don't see any provides mechanism. That might be something actually needed if we are trying to avoid version comparisons and want to be describing actual dependencies... The 'provides' mechanism can be added later I think. It's like saying that in 9.1 an extension 'foo' provides the feature 'foo' and you can't control that, whereas in future version you will be able to control what your extension (and its specific version) provides. Also we will be able to list what the PostgreSQL server provides, maybe, so that compile time options can be depended on by extensions. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Apr 4, 2011, at 3:57 PM, Tom Lane wrote: I think the general movement is toward *feature* dependancies. So for intstance, an extension can specify what *feature* it requires, and difference versions of an extension can provide different features. Right. Sounds like a book-keeping nightmare for extension developers. It will discourage large or rapidly-evolving extensions like pgTAP because it will be a PITA to specify features. But checking http://developer.postgresql.org/pgdocs/postgres/extend-extensions.html, I don't see any provides mechanism. Yes, some sort of manual Provides: (in addition to automatically extracted Provides:) would likely be part of any serious solution. shed type=bikeI'd like to request Features: instead of Provides:./shed We're not there yet, and we're not going to get there in time for 9.1. But in any case, mechanisms that involve version ordering comparisons seem to be on their way out for deciding whether package A is compatible with package B. This is news to me, frankly, and the bookkeeping requirements seem potentially awful. If it's possible that it won't work out this way, that those arguing for version dependency resolution end up getting the consensus, not having a version string format is going to be a nightmare. On the other hand, if we added one now, and feature dependency tracking won the day, well, a version string format could always be loosened later. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Tue, Apr 5, 2011 at 4:20 PM, David E. Wheeler da...@kineticode.com wrote:
On Apr 4, 2011, at 3:57 PM, Tom Lane wrote:
I think the general movement is toward *feature* dependancies. So for
intstance, an extension can specify what *feature* it requires, and
difference versions of an extension can provide different
features.
Right.
Sounds like a book-keeping nightmare for extension developers. It will
discourage large or rapidly-evolving extensions like pgTAP because it will be
a PITA to specify features.
Sure, but if you want, the feature you can provide can be something like:
pgtap-1.0 (or any of pgtap-0.2{0,1,2,3,4}).
And if your package is backwards compatable, it could even provide:
pgtap-0.25
pgtap-0.24
pgtap-0.23
And that also means that you don't have to screw every body over when
some future pgtap-123.45 is no longer compatible, and the extensions
have relied on $VERSION 0.23 meaning they'll work with it.
I mean, PG itself is an example. Does pg 8.4 mean your code will
work with all future (or even past, but 8.4) PG versions?
We're not there yet, and we're not going to get there in time for 9.1.
But in any case, mechanisms that involve version ordering comparisons
seem to be on their way out for deciding whether package A is
compatible with package B.
This is news to me, frankly, and the bookkeeping requirements seem
potentially awful.
If it's possible that it won't work out this way, that those arguing for
version dependency resolution end up getting the consensus, not having a
version string format is going to be a nightmare. On the other hand, if we
added one now, and feature dependency tracking won the day, well, a version
string format could always be loosened later.
As someone who has had to try and deal with package versions for
dependencies in RPM and DEB, and been through the hell that is open
source package variants, all with the ability to turn on/off features
at configure/compile time, a just versions even with , =, =, =,
all mapped correctly isn't good enough.
Of course, I'ld love for extension in 9.1 to provide a basic
provides/features for my extension to give, but if that train has
already left the station, I don't have much choice ;-(
a.
--
Aidan Van Dyk Create like a god,
ai...@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Apr 5, 2011, at 1:42 PM, Aidan Van Dyk wrote:
Sure, but if you want, the feature you can provide can be something like:
pgtap-1.0 (or any of pgtap-0.2{0,1,2,3,4}).
And if your package is backwards compatable, it could even provide:
pgtap-0.25
pgtap-0.24
pgtap-0.23
I see, I get it.
And that also means that you don't have to screw every body over when
some future pgtap-123.45 is no longer compatible, and the extensions
have relied on $VERSION 0.23 meaning they'll work with it.
I see.
I mean, PG itself is an example. Does pg 8.4 mean your code will
work with all future (or even past, but 8.4) PG versions?
I see. So the extension author can more easily tell users when compatibility
has been dropped for something. That makes sense.
As someone who has had to try and deal with package versions for
dependencies in RPM and DEB, and been through the hell that is open
source package variants, all with the ability to turn on/off features
at configure/compile time, a just versions even with , =, =, =,
all mapped correctly isn't good enough.
Yeah. The use of an implicit = in CPAN modules has been a decent 90% solution,
but it does cause headaches for people that they can't express things better.
The ability to do so would require a mini-language with more operators,
precedence, grouping, etc.
Of course, I'ld love for extension in 9.1 to provide a basic
provides/features for my extension to give, but if that train has
already left the station, I don't have much choice ;-(
Yeah, but the way it is doesn't break the ability to do it later. I suspect
that Dim and Tom will be thinking about it for 9.2.
Anyway, your post helps me to understand things better, and so I'm less
insistent about imposing a version numbering scheme now (though I still think
it would be more useful to have one than not).
Best,
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Tue, Apr 5, 2011 at 4:51 PM, David E. Wheeler da...@kineticode.com wrote: Of course, I'ld love for extension in 9.1 to provide a basic provides/features for my extension to give, but if that train has already left the station, I don't have much choice ;-( Yeah, but the way it is doesn't break the ability to do it later. I suspect that Dim and Tom will be thinking about it for 9.2. Anyway, your post helps me to understand things better, and so I'm less insistent about imposing a version numbering scheme now (though I still think it would be more useful to have one than not). Versions are useful for figuring out if I should upgrade packages or not. But I believe the extension framework has explicitly made the upgrade problem a manual one at this point, either taking destination versions from the control, or the alter command. So for PGXN's problem, I see the point of versions being required. But for installation the dependancy graph, provides/features rather than versions are much more useful.And automatic feature/provides (like library so, and symbol versions in the OS package world, objects in PG world) would definitely be nice, but my Makefile can build those for me for now until 9.2 (or 9.3, 9.3, etc), if only I had a way to track them with my installed extension ;-) /stop begging a. -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Fri, Apr 1, 2011 at 11:45 AM, David E. Wheeler da...@kineticode.com wrote: But I'm assuming that at some point there's going to be something a bit more robust: specifically, requiring a minimum version, perhaps something like: requires = 'foo 1.0, bar 0.31.4' Or maybe: requires = 'foo = 1.0, bar = 0.31.4' * I think we're going to need a formal version string spec for extensions. I agree. * If that's true, I think it should be specified *now*, before extensions are in the wild, so that we don't end up with the legacy version string nightmares that Perl modules suffer from. I think I agree with this, too. * If we do adopt a spec, I think it should reflect the PostgreSQL core version strings as closely as possible, and should be fully compatible with them. I am less sure about this one. * So it might be worth looking at semver or something similar to integrate. No. It's too late to be monkeying with this. I think for 9.1 we will need to content ourselves with setting a good precedent, rather than enforcing it programatically. It's not going to work to insist on all numeric version strings anyway, because we've already got this 'FROM unpackaged' bit floating around. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
Robert Haas robertmh...@gmail.com writes: On Fri, Apr 1, 2011 at 11:45 AM, David E. Wheeler da...@kineticode.com wrote: * I think we're going to need a formal version string spec for extensions. I agree. I don't. We deliberately decided *not* to have any wired-in interpretation of extension numbers, and I don't think that decision needs to be reversed. David can choose to enforce something for stuff distributed through PGXN if he wishes, but that's no concern of the core server's. In particular I'm really skeptical of the theory that we need or should want version restrictions in Requires references. The equivalent feature in RPM is deprecated for Fedora/RedHat packaging use, and I see no reason why we'd need it more than they do. * So it might be worth looking at semver or something similar to integrate. No. It's too late to be monkeying with this. I think for 9.1 we will need to content ourselves with setting a good precedent, rather than enforcing it programatically. It's not going to work to insist on all numeric version strings anyway, because we've already got this 'FROM unpackaged' bit floating around. Once 9.1 is out, it'll probably be too late to dictate any semantics for version numbers, because somebody will have done something incompatible with it before 9.2 is released. If we are going to try to insist on this, now is the time. But I don't agree with that position. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Mon, Apr 4, 2011 at 5:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Apr 1, 2011 at 11:45 AM, David E. Wheeler da...@kineticode.com wrote: * I think we're going to need a formal version string spec for extensions. I agree. I don't. We deliberately decided *not* to have any wired-in interpretation of extension numbers, and I don't think that decision needs to be reversed. David can choose to enforce something for stuff distributed through PGXN if he wishes, but that's no concern of the core server's. In particular I'm really skeptical of the theory that we need or should want version restrictions in Requires references. The equivalent feature in RPM is deprecated for Fedora/RedHat packaging use, and I see no reason why we'd need it more than they do. Oh, really? How can you possibly get by without it? Dependencies of this type are all over the place. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Apr 4, 2011, at 2:48 PM, Tom Lane wrote: Once 9.1 is out, it'll probably be too late to dictate any semantics for version numbers, because somebody will have done something incompatible with it before 9.2 is released. If we are going to try to insist on this, now is the time. Yes, exactly my point. If it's going to be done, it should be now. But I don't agree with that position. Why not? Best. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
On Mon, Apr 4, 2011 at 6:06 PM, Robert Haas robertmh...@gmail.com wrote: I don't. We deliberately decided *not* to have any wired-in interpretation of extension numbers, and I don't think that decision needs to be reversed. David can choose to enforce something for stuff distributed through PGXN if he wishes, but that's no concern of the core server's. In particular I'm really skeptical of the theory that we need or should want version restrictions in Requires references. The equivalent feature in RPM is deprecated for Fedora/RedHat packaging use, and I see no reason why we'd need it more than they do. Oh, really? How can you possibly get by without it? Dependencies of this type are all over the place. I think the general movement is toward *feature* dependancies. So for intstance, an extension can specify what *feature* it requires, and difference versions of an extension can provide different features. In that case, you don't really need extenson foo 2.1, you need the feature that foo 2.1.x provides, maybe foo-api-2 (note that 2 would be part of a name, not any comparison aware version. I'm already going to be naming my extensions with major versions as part of the name (like all the distro postgresql packages) so my versions will only ever be simple integers of exactly compatable objects. But checking http://developer.postgresql.org/pgdocs/postgres/extend-extensions.html, I don't see any provides mechanism. That might be something actually needed if we are trying to avoid version comparisons and want to be describing actual dependencies... -- Aidan Van Dyk Create like a god, ai...@highrise.ca command like a king, http://www.highrise.ca/ work like a slave. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
Robert Haas robertmh...@gmail.com writes: On Mon, Apr 4, 2011 at 5:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... In particular I'm really skeptical of the theory that we need or should want version restrictions in Requires references. The equivalent feature in RPM is deprecated for Fedora/RedHat packaging use, and I see no reason why we'd need it more than they do. Oh, really? How can you possibly get by without it? Dependencies of this type are all over the place. Yeah, but the proposed solution doesn't actually fix the problem. There are various issues, but the fundamental objection is that when you really need it the information is usually wrong/obsolete, especially if it's maintained by hand. As an example, if we have foo depending on bar and it says Requires: bar = 2.0, that's all well and good, but it tells us nothing whatsoever about whether this version of foo will work with bar 3.0, especially if 3.0 postdates the time when the foo author put that into his specfile. Maybe we should really interpret that as Requires: bar = 2.0 3.0, or maybe not. It's also very easy for foo's author to start using some nice_bar_function() without realizing that that was actually only added in bar 2.1. If you want automatic dependency checking to actually do something useful, you need something very substantially finer-grained and more complex than this. In the Fedora world they're mostly depending on per-shared-library sonames and per-exported-symbol symbol versioning, and even then it doesn't always do what you wished. But I'll note that neither of those mechanisms bothers to infer any version ordering: either you have a match, or you don't. The equivalent thing of what's now considered good packaging practice would probably be to extract lists of specific objects provided or required by a module, preferably automatically. The requires side of that might be doable just by inspecting pg_depend. The provides side would be harder, mainly because you'd want some way to omit objects that are considered purely internal to a module. Anyway, I'm uneager to adopt a practice that was discredited years ago among real packaging systems. I think it'll just delay somebody deciding to fix it properly. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Extensions Dependency Checking
Aidan Van Dyk ai...@highrise.ca writes: On Mon, Apr 4, 2011 at 6:06 PM, Robert Haas robertmh...@gmail.com wrote: Oh, really? How can you possibly get by without it? Dependencies of this type are all over the place. I think the general movement is toward *feature* dependancies. So for intstance, an extension can specify what *feature* it requires, and difference versions of an extension can provide different features. Right. But checking http://developer.postgresql.org/pgdocs/postgres/extend-extensions.html, I don't see any provides mechanism. Yes, some sort of manual Provides: (in addition to automatically extracted Provides:) would likely be part of any serious solution. We're not there yet, and we're not going to get there in time for 9.1. But in any case, mechanisms that involve version ordering comparisons seem to be on their way out for deciding whether package A is compatible with package B. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
