Re: [Pkg-javascript-devel] parallel installation
On Tue, 19 Sep 2017 22:27:34 +0200, Paul Gevers said: > Hi Hubert, Thanks for writing this down. > On 19-09-17 17:58, Hubert Chathi wrote: >> https://wiki.debian.org/HubertChathi/JavaScriptPolicy >> >> Please let me know what you all think of it, and suggest >> improvements/fixes/etc. > I was wondering how you envision the following: "The source package > name may also include the APIVERSION in its name." I mean, if the > binary package should contain it, and there are multiple versions, > shouldn't the source package also contain it? Or is this related to > the last paragraph of that chapter? In general, I went for more permissive language rather than mandating what *has* do be done, partly in order to account for the status quo. So you are right that if two source packages provide different APIVERSIONs, then they need to have different names. But what I didn't want to do was to automatically make every JavaScript package that's currently in Debian non-compliant, which is what would happen if I we made it a "MUST". What I was thinking of was, to pick a random example, libjs-jquery is currently built from the jquery source package. If a new jQuery comes out which breaks the API, then the new source package would be named, say jquery4 or jquery3.3 or whatever, while the old source package would remain plain "jquery". I would be fine changing that to a "should", if other people would prefer that. > Also, I think we want some more words when the package with APIVERSION > is "updated" with a newer version. I.e. I think it shouldn't be > updated when you still want to provide the old APIVERSION, but instead > there should be a new package. Or are you envisioning to provide > multiple APIVERSIONs from the same source (I would recommend against > that, although since the latest uscan version less so). I'm not sure what kind of situation you're referring to. Maybe you can explain a bit more? If a library still provides an old APIVERSION, then it can still keep the same APIVERSION. Unless you're talking about the situation where a library provides an old APIVERSION but it's deprecated and will be removed in a future version. So, say, foo1 uses the old API, foo2 has both the old API and the new API, and foo3 has just the new API. In that case, I guess foo2 could be packaged as libjs-foo2, with a "Provides: libjs-foo1". Would this solve the situation you're thinking of? > Last time we discussed a target for how many APIVERSIONs are > allowed. I don't remember quickly what came out of that discussion, > but maybe having the discussion (or different opinions) reflected in > the policy as point to consider may be appropriate. I don't think there was a consensus on how many APIVERSIONS to allow, or whether there should be any limit at all. I could probably add in some language saying something about trying to limit the number of versions packaged to a "reasonable" number. BTW, I seem to have omitted the "Exclude auto-generated files from source" section entirely, which was not intentional. So until I add that back in, then pretend that it's at the bottom of my proposal. I did remove the bit about the "source package name should be called foo.js" on purpose, since it seems not many packages actually follow that. But I can re-add that if others think it should go back in. -- Hubert Chathi -- https://www.uhoreg.ca/ Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368 -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Hi Hubert, Thanks for writing this down. On 19-09-17 17:58, Hubert Chathi wrote: > https://wiki.debian.org/HubertChathi/JavaScriptPolicy > > Please let me know what you all think of it, and suggest > improvements/fixes/etc. I was wondering how you envision the following: "The source package name may also include the APIVERSION in its name." I mean, if the binary package should contain it, and there are multiple versions, shouldn't the source package also contain it? Or is this related to the last paragraph of that chapter? Also, I think we want some more words when the package with APIVERSION is "updated" with a newer version. I.e. I think it shouldn't be updated when you still want to provide the old APIVERSION, but instead there should be a new package. Or are you envisioning to provide multiple APIVERSIONs from the same source (I would recommend against that, although since the latest uscan version less so). Last time we discussed a target for how many APIVERSIONs are allowed. I don't remember quickly what came out of that discussion, but maybe having the discussion (or different opinions) reflected in the policy as point to consider may be appropriate. Paul signature.asc Description: OpenPGP digital signature -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
I've had a draft sitting on my computer for a while, and I haven't had much time recently to improve it, so here is a draft policy that includes stuff about parallel installation. It also changes the structure of the policy, because I was having a hard time fitting it into the existing structure. https://wiki.debian.org/HubertChathi/JavaScriptPolicy Please let me know what you all think of it, and suggest improvements/fixes/etc. I've only done the plain JavaScript policy for now. I haven't attempted the Node policy yet. -- Hubert Chathi -- https://www.uhoreg.ca/ Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368 -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
On Mon, 21 Aug 2017 18:03:02 +0200, Ross Gammon
said:
> Anyway - if we instead separately packaged a different version of a
> node module, where would it be installed? Do we have to patch all
> reverse dependencies of the module so it uses the right version at
> runtime (when both versions might be installed), or can we rely on
> some PATH trickery?
That's what I was trying to get opinions on in my original message. I
think the sanest method is to use symlinks within the node_modules
directory. So, for example, if foo (which is in /usr/lib/nodejs/foo)
uses bar API version 2 (which is in /usr/lib/nodejs/bar2), then
/usr/lib/nodejs/foo/node_modules/bar would be a symlink to
/usr/lib/nodejs/bar2. Then when foo does a "require("bar");" then Node
will find /usr/lib/nodejs/foo/node_modules/bar, follow the symlink, and
use the right version.
I can understand the need in some certain situations to embed code
copies, but I think that those should be treated as the exception rather
than the rule. (FWIW, I maintain a package in C/C++-land that does
embed code (because at the time it was originally written, a proper
library wasn't written yet, though some day I'd like to push a patch
upstream to make it use the now-written library), and I can say that
security support for it is a huge pain. So I can sympathize with
maintainers who are placed in such unpleasant situations.)
--
Hubert Chathi -- https://www.uhoreg.ca/
Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org
PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368
--
Pkg-javascript-devel mailing list
Pkg-javascript-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
On 08/18/2017 08:53 PM, Jonas Smedegaard wrote: > Quoting Ross Gammon (2017-08-18 19:54:50) >> On 08/18/2017 06:09 PM, Jonas Smedegaard wrote: >>> Quoting Ross Gammon (2017-08-18 16:47:10) On 08/18/2017 04:50 AM, Hubert Chathi wrote: > On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon > said: > >> For node-* stuff however, upstream handle this by bundling a >> particular version of a module in node_modules. If it is "really >> difficult" to patch a node module/app to use the Debian version >> of a library (because the versions have changed too much), then >> shouldn't we bundle the node_module and install it as upstream do >> it (avoiding all the relative path issues)? It could be followed >> up with a bug (severity wishlist/normal?) to remove the bundled >> module once Debian and upstream are more aligned. > Embedding copies of libraries should be highly discouraged. For > one thing, it is agaist policy[1], but it also it makes security > support much harder, you may end up with multiple buggy versions > of a library on your system, and have a bunch of duplication. It > may make initial packaging easier, but it usually makes > maintenance harder. > > [1] > https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles The title of this section of policy is actually "Convenience copies of code". It is definitely not a convenience to heavily patch a package just to use a "way out of date" dependency, when it is out of date because many other packages still require that old dependency. >>> It is an _upstream_ antipattern, and is indeed a convenience for >>> them. >>> >>> I agree that it should be discouraged though, except in rare cases. It is just a normal transition (like in C/C++) after all. >>> Not sure if you agree with policy or try argue against it. >>> >>> Whether it is bundled, or several versions of the same upstream are packaged separately, you still have the issue of code duplication, >>> No, several versions of a library is not convenience code copies. >>> >>> and the possibility that a security update might be required in several places at the same time. >>> It is certainly _possible_ for a security bug to exist in multiple >>> upstream releases of a code project, but when tracked on its own it >>> is easier to hunt down such bugs, and when each version exist only >>> at one place in Debian then each version needs only be fixed once >>> for a security bug to get squashed. >>> >>> Of course, bundled copies are harder to find. But we can manage that in the team (via a transition bug, and/or a list on the wiki?) while we push all the unwilling upstreams to align on the same version (and nodejs upstreams are REALLY unwilling on this - believe me). I still think it is better to manage multiple copies in the same way that upstream do. It will give a lot less friction upstream. >>> It is not adequate to agree in this team about bundling code: As an >>> example, the security team tracks security bugs and will also need >>> to agree on such deviation from Debian Policy. >>> >>> Personally I do *not* find bundling easier manageable than >>> separately tracking each true upstream project. You are welcome to >>> explore argue for that, but you will need to convince not only this >>> team but Debian in general. >>> >>> >>> - Jonas >>> >> From the policy: "Debian packages should not make use of these >> convenience copies unless the included package is explicitly intended >> to be used in this way.". >> >> "Should" does not equal "must", it means something closer to "maybe". >> This gave me some trouble in my Danish language classes :-) > And also from policy: "Non-conformance with guidelines denoted by should > (or recommended) will generally be considered a bug, but will not > necessarily render a package unsuitable for distribution.". > > So yes, it does not equal "must" (and I believe I didn't claim it did), > but it does not equal "maybe" either (arguably you didn't claim it did - > but your choice of words had a real risk of being perceived that way). > > >> I believe upstream "intended to be used in this way". > I also believe that upstream intended the code they released to be used > the way they prepared it. > > But also I believe that the Debian project considers it an antipattern > a.k.a. a bug. > > >> But I don't really want to argue about it any more, and I am happy to >> go with what the team agrees. > We do not have any agreement yet, but thanks also to your input we may > find one. I really appreciated your input (even if I disagree with it) > and hope you will continue to provide input to this discussion if you > have additional points or perhaps feel that some of the points you've > made already is being misunderstood or misrepresented. :-) > > > - Jonas
Re: [Pkg-javascript-devel] parallel installation
On Sat, 19 Aug 2017 12:32:39 +0200, Mathias Behrle said: > * Hubert Chathi: " [Pkg-javascript-devel] parallel installation" (Mon, > 14 Aug 2017 14:38:34 -0400): > I appreciate very much the initiative to find a common procedure for > making different versions of JS libs available while still being > compliant to policy. The very special use case for me is tryton-sao > [1], which was already in NEW but which I had to withdraw, because it > didn't (and still does not) support the jquery version in stretch. >> At the BoF at DebConf, we were talking about parallel installation of >> different versions of JS libraries. In order to do parallel >> installation, we'd need differently named packages for different >> versions, and it seems like the obvious way to do that is to have >> packages called something like libjs-fooVER and node-fooVER, where >> VER some sort of the API version, similar to the way that C/C++ >> library packages are named after the library SONAME. If upstream >> follows semver properly, then VER would be the major version number. > I wonder why we should not also support MAJOR.MINOR as VER, if needed > in special cases. I am not that familiar with the JS world, but I > think to remember, that e.g. jquery didn't always follow properly > semver and in such cases it could be necessary to have a special MINOR > available in the archive. There isn't really a problem with MAJOR.MINOR as VER. The reason that I say that "VER would be the major version number" for libraries that follow semver is that it's what semver says. What's really important is that VER indicates that the API is stable with newer releases of that version. That is, a package that depends on libjs-fooVER >= x won't break due to API changes in versions after x; if the library breaks API compatibility, then VER needs to be bumped in some way or another. So yes, if upstream breaks API compatibility with some minor version numbers, then the minor version number will be part of VER. (In the C/C++ world, the SONAME (and hence the version embedded in the package name) is just the major version number, but one counterexample seems to be libssl, which includes minor and even sub-minor version numbers in the package name). I'll try to draft a policy with some accompanying documentation over the next few days. -- Hubert Chathi -- https://www.uhoreg.ca/ Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368 -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
* Paul Gevers: " Re: [Pkg-javascript-devel] parallel installation" (Tue, 15 Aug 2017 21:20:49 +0200): > Hi, > > On 14-08-17 20:38, Hubert Chathi wrote: > > At the BoF at DebConf, we were talking about parallel installation of > > different versions of JS libraries. In order to do parallel > > installation, we'd need differently named packages for different > > versions, and it seems like the obvious way to do that is to have > > packages called something like libjs-fooVER and node-fooVER, where VER > > some sort of the API version, similar to the way that C/C++ library > > packages are named after the library SONAME. If upstream follows semver > > properly, then VER would be the major version number. > > This sounds like a reasonable approach, but what I am missing is a > recommendation on what to do to keep the number of packages limited. I > think it is not such a great idea to replace one problem (not the right > version) with another one (too many packages to support). > > One approach could be for example the following. Try to have only two > versions per release, but really limit the number to a maximum of 3. I think instead of hardcoding a number of allowed versions it would be more adequate to check, if a library is still needed and remove it as soon as any package no more depends on it. -- Mathias Behrle PGP/GnuPG key availabable from any keyserver, ID: 0xD6D09BE48405BBF6 AC29 7E5C 46B9 D0B6 1C71 7681 D6D0 9BE4 8405 BBF6 -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
* Hubert Chathi: " [Pkg-javascript-devel] parallel installation" (Mon, 14 Aug
2017 14:38:34 -0400):
I appreciate very much the initiative to find a common procedure for making
different versions of JS libs available while still being compliant to policy.
The very special use case for me is tryton-sao [1], which was already in NEW but
which I had to withdraw, because it didn't (and still does not) support the
jquery version in stretch.
> At the BoF at DebConf, we were talking about parallel installation of
> different versions of JS libraries. In order to do parallel
> installation, we'd need differently named packages for different
> versions, and it seems like the obvious way to do that is to have
> packages called something like libjs-fooVER and node-fooVER, where VER
> some sort of the API version, similar to the way that C/C++ library
> packages are named after the library SONAME. If upstream follows semver
> properly, then VER would be the major version number.
I wonder why we should not also support MAJOR.MINOR as VER, if needed in
special cases. I am not that familiar with the JS world, but I think to
remember, that e.g. jquery didn't always follow properly semver and in such
cases it could be necessary to have a special MINOR available in the archive.
>
> For libjs-fooVER, the JavaScript files would probably go in
> /usr/share/javascript/fooVER/ rather than /usr/share/javascript/foo.
> Obviously everything that previously pointed to the old place would have
> to now point to the new place, but we could create a libjs-foo
> transitional package libjs-foo that symlinks /usr/share/javascript/foo
> to /usr/share/javascript/fooVER so that existing packages don't break.
>
> For node-fooVER, the logical analogue would be to put the files in
> /usr/lib/nodejs/fooVER. However, anything that uses the library does
> "require("foo")", and so would not be able to find it there. There are
> a couple possible options for fixing that I can think of. The first is
> to change all "require("foo")" to "require("fooVER")", but that sounds
> painful to do. The second option is to add a symlink from
> node_modules/foo to /usr/lib/nodejs/fooVER in the root directory of the
> thing that requires foo. I think the second option is the best,
> especially since Node upstream seems to prefer searching for things in
> node_modules rather than in /usr/lib/nodejs. And again, we may want to
> create a transitional package node-foo that symlinks /usr/lib/nodejs/foo
> to /usr/lib/nodejs/fooVER.
This second option sounds most reasonable to me.
[1] https://anonscm.debian.org/cgit/tryton/tryton-sao.git/
--
Mathias Behrle
PGP/GnuPG key availabable from any keyserver, ID: 0xD6D09BE48405BBF6
AC29 7E5C 46B9 D0B6 1C71 7681 D6D0 9BE4 8405 BBF6
--
Pkg-javascript-devel mailing list
Pkg-javascript-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Quoting Ross Gammon (2017-08-18 19:54:50) > On 08/18/2017 06:09 PM, Jonas Smedegaard wrote: > > Quoting Ross Gammon (2017-08-18 16:47:10) > >> On 08/18/2017 04:50 AM, Hubert Chathi wrote: > >>> On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon > >>> said: > >>> > For node-* stuff however, upstream handle this by bundling a > particular version of a module in node_modules. If it is "really > difficult" to patch a node module/app to use the Debian version > of a library (because the versions have changed too much), then > shouldn't we bundle the node_module and install it as upstream do > it (avoiding all the relative path issues)? It could be followed > up with a bug (severity wishlist/normal?) to remove the bundled > module once Debian and upstream are more aligned. > >>> Embedding copies of libraries should be highly discouraged. For > >>> one thing, it is agaist policy[1], but it also it makes security > >>> support much harder, you may end up with multiple buggy versions > >>> of a library on your system, and have a bunch of duplication. It > >>> may make initial packaging easier, but it usually makes > >>> maintenance harder. > >>> > >>> [1] > >>> https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles > >> The title of this section of policy is actually "Convenience copies > >> of code". It is definitely not a convenience to heavily patch a > >> package just to use a "way out of date" dependency, when it is out > >> of date because many other packages still require that old > >> dependency. > > It is an _upstream_ antipattern, and is indeed a convenience for > > them. > > > > > >> I agree that it should be discouraged though, except in rare cases. > >> It is just a normal transition (like in C/C++) after all. > > Not sure if you agree with policy or try argue against it. > > > > > >> Whether it is bundled, or several versions of the same upstream are > >> packaged separately, you still have the issue of code duplication, > > No, several versions of a library is not convenience code copies. > > > > > >> and the possibility that a security update might be required in > >> several places at the same time. > > It is certainly _possible_ for a security bug to exist in multiple > > upstream releases of a code project, but when tracked on its own it > > is easier to hunt down such bugs, and when each version exist only > > at one place in Debian then each version needs only be fixed once > > for a security bug to get squashed. > > > > > >> Of course, bundled copies are harder to find. But we can manage > >> that in the team (via a transition bug, and/or a list on the wiki?) > >> while we push all the unwilling upstreams to align on the same > >> version (and nodejs upstreams are REALLY unwilling on this - > >> believe me). I still think it is better to manage multiple copies > >> in the same way that upstream do. It will give a lot less friction > >> upstream. > > It is not adequate to agree in this team about bundling code: As an > > example, the security team tracks security bugs and will also need > > to agree on such deviation from Debian Policy. > > > > Personally I do *not* find bundling easier manageable than > > separately tracking each true upstream project. You are welcome to > > explore argue for that, but you will need to convince not only this > > team but Debian in general. > > > > > > - Jonas > > > From the policy: "Debian packages should not make use of these > convenience copies unless the included package is explicitly intended > to be used in this way.". > > "Should" does not equal "must", it means something closer to "maybe". > This gave me some trouble in my Danish language classes :-) And also from policy: "Non-conformance with guidelines denoted by should (or recommended) will generally be considered a bug, but will not necessarily render a package unsuitable for distribution.". So yes, it does not equal "must" (and I believe I didn't claim it did), but it does not equal "maybe" either (arguably you didn't claim it did - but your choice of words had a real risk of being perceived that way). > I believe upstream "intended to be used in this way". I also believe that upstream intended the code they released to be used the way they prepared it. But also I believe that the Debian project considers it an antipattern a.k.a. a bug. > But I don't really want to argue about it any more, and I am happy to > go with what the team agrees. We do not have any agreement yet, but thanks also to your input we may find one. I really appreciated your input (even if I disagree with it) and hope you will continue to provide input to this discussion if you have additional points or perhaps feel that some of the points you've made already is being misunderstood or misrepresented. :-) - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://d
Re: [Pkg-javascript-devel] parallel installation
On 08/18/2017 06:09 PM, Jonas Smedegaard wrote: > Quoting Ross Gammon (2017-08-18 16:47:10) >> On 08/18/2017 04:50 AM, Hubert Chathi wrote: >>> [meant to reply to the list, so sending again] >>> >>> On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon >>> said: >>> For node-* stuff however, upstream handle this by bundling a particular version of a module in node_modules. If it is "really difficult" to patch a node module/app to use the Debian version of a library (because the versions have changed too much), then shouldn't we bundle the node_module and install it as upstream do it (avoiding all the relative path issues)? It could be followed up with a bug (severity wishlist/normal?) to remove the bundled module once Debian and upstream are more aligned. >>> Embedding copies of libraries should be highly discouraged. For one >>> thing, it is agaist policy[1], but it also it makes security support >>> much harder, you may end up with multiple buggy versions of a library on >>> your system, and have a bunch of duplication. It may make initial >>> packaging easier, but it usually makes maintenance harder. >>> >>> [1] https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles >> The title of this section of policy is actually "Convenience copies of >> code". It is definitely not a convenience to heavily patch a package >> just to use a "way out of date" dependency, when it is out of date >> because many other packages still require that old dependency. > It is an _upstream_ antipattern, and is indeed a convenience for them. > > >> I agree that it should be discouraged though, except in rare cases. It >> is just a normal transition (like in C/C++) after all. > Not sure if you agree with policy or try argue against it. > > >> Whether it is bundled, or several versions of the same upstream are >> packaged separately, you still have the issue of code duplication, > No, several versions of a library is not convenience code copies. > > >> and the possibility that a security update might be required in >> several places at the same time. > It is certainly _possible_ for a security bug to exist in multiple > upstream releases of a code project, but when tracked on its own it is > easier to hunt down such bugs, and when each version exist only at one > place in Debian then each version needs only be fixed once for a > security bug to get squashed. > > >> Of course, bundled copies are harder to find. But we can manage that >> in the team (via a transition bug, and/or a list on the wiki?) while >> we push all the unwilling upstreams to align on the same version (and >> nodejs upstreams are REALLY unwilling on this - believe me). I still >> think it is better to manage multiple copies in the same way that >> upstream do. It will give a lot less friction upstream. > It is not adequate to agree in this team about bundling code: As an > example, the security team tracks security bugs and will also need to > agree on such deviation from Debian Policy. > > Personally I do *not* find bundling easier manageable than separately > tracking each true upstream project. You are welcome to explore argue > for that, but you will need to convince not only this team but Debian in > general. > > > - Jonas > >From the policy: "Debian packages should not make use of these convenience copies unless the included package is explicitly intended to be used in this way.". "Should" does not equal "must", it means something closer to "maybe". This gave me some trouble in my Danish language classes :-) I believe upstream "intended to be used in this way". But I don't really want to argue about it any more, and I am happy to go with what the team agrees. Ross -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Quoting Ross Gammon (2017-08-18 16:47:10) > On 08/18/2017 04:50 AM, Hubert Chathi wrote: > > [meant to reply to the list, so sending again] > > > > On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon > > said: > > > >> For node-* stuff however, upstream handle this by bundling a > >> particular version of a module in node_modules. If it is "really > >> difficult" to patch a node module/app to use the Debian version of a > >> library (because the versions have changed too much), then shouldn't > >> we bundle the node_module and install it as upstream do it (avoiding > >> all the relative path issues)? It could be followed up with a bug > >> (severity wishlist/normal?) to remove the bundled module once Debian > >> and upstream are more aligned. > > Embedding copies of libraries should be highly discouraged. For one > > thing, it is agaist policy[1], but it also it makes security support > > much harder, you may end up with multiple buggy versions of a library on > > your system, and have a bunch of duplication. It may make initial > > packaging easier, but it usually makes maintenance harder. > > > > [1] https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles > > The title of this section of policy is actually "Convenience copies of > code". It is definitely not a convenience to heavily patch a package > just to use a "way out of date" dependency, when it is out of date > because many other packages still require that old dependency. It is an _upstream_ antipattern, and is indeed a convenience for them. > I agree that it should be discouraged though, except in rare cases. It > is just a normal transition (like in C/C++) after all. Not sure if you agree with policy or try argue against it. > Whether it is bundled, or several versions of the same upstream are > packaged separately, you still have the issue of code duplication, No, several versions of a library is not convenience code copies. > and the possibility that a security update might be required in > several places at the same time. It is certainly _possible_ for a security bug to exist in multiple upstream releases of a code project, but when tracked on its own it is easier to hunt down such bugs, and when each version exist only at one place in Debian then each version needs only be fixed once for a security bug to get squashed. > Of course, bundled copies are harder to find. But we can manage that > in the team (via a transition bug, and/or a list on the wiki?) while > we push all the unwilling upstreams to align on the same version (and > nodejs upstreams are REALLY unwilling on this - believe me). I still > think it is better to manage multiple copies in the same way that > upstream do. It will give a lot less friction upstream. It is not adequate to agree in this team about bundling code: As an example, the security team tracks security bugs and will also need to agree on such deviation from Debian Policy. Personally I do *not* find bundling easier manageable than separately tracking each true upstream project. You are welcome to explore argue for that, but you will need to convince not only this team but Debian in general. - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
On 08/18/2017 04:50 AM, Hubert Chathi wrote: > [meant to reply to the list, so sending again] > > On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon > said: > >> For node-* stuff however, upstream handle this by bundling a >> particular version of a module in node_modules. If it is "really >> difficult" to patch a node module/app to use the Debian version of a >> library (because the versions have changed too much), then shouldn't >> we bundle the node_module and install it as upstream do it (avoiding >> all the relative path issues)? It could be followed up with a bug >> (severity wishlist/normal?) to remove the bundled module once Debian >> and upstream are more aligned. > Embedding copies of libraries should be highly discouraged. For one > thing, it is agaist policy[1], but it also it makes security support > much harder, you may end up with multiple buggy versions of a library on > your system, and have a bunch of duplication. It may make initial > packaging easier, but it usually makes maintenance harder. > > [1] https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles The title of this section of policy is actually "Convenience copies of code". It is definitely not a convenience to heavily patch a package just to use a "way out of date" dependency, when it is out of date because many other packages still require that old dependency. I agree that it should be discouraged though, except in rare cases. It is just a normal transition (like in C/C++) after all. Whether it is bundled, or several versions of the same upstream are packaged separately, you still have the issue of code duplication, and the possibility that a security update might be required in several places at the same time. Of course, bundled copies are harder to find. But we can manage that in the team (via a transition bug, and/or a list on the wiki?) while we push all the unwilling upstreams to align on the same version (and nodejs upstreams are REALLY unwilling on this - believe me). I still think it is better to manage multiple copies in the same way that upstream do. It will give a lot less friction upstream. Cheers, Ross -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Quoting Paul Gevers (2017-08-15 21:20:49) > On 14-08-17 20:38, Hubert Chathi wrote: > > At the BoF at DebConf, we were talking about parallel installation > > of different versions of JS libraries. In order to do parallel > > installation, we'd need differently named packages for different > > versions, and it seems like the obvious way to do that is to have > > packages called something like libjs-fooVER and node-fooVER, where > > VER some sort of the API version, similar to the way that C/C++ > > library packages are named after the library SONAME. If upstream > > follows semver properly, then VER would be the major version number. > > This sounds like a reasonable approach, I agree above sounds sensible. > but what I am missing is a recommendation on what to do to keep the > number of packages limited. I think it is not such a great idea to > replace one problem (not the right version) with another one (too many > packages to support). > > One approach could be for example the following. Try to have only two > versions per release, but really limit the number to a maximum of 3. I see no need for a team policy specifically for parallel versions of same project: Some libraries may have 4 well-maintained concurrent releases, whereas others may have *zero* well-maintained releases. We need to ensure that our packages are properly *maintained*. Period. If an upstream project releases foo1 and foo2 and foo3, and other packages depend on all three, then that is fine if all three are properly maintained - both upstream and in Debian. If an upstream project releases foo1 and only that one which other packages depend on, then that is bad if that that one is poorly maintained - either upstream and in Debian. I believe we cannot make a simple rule like "only 2 parallel releases", but I believe we can easier identify potential problems if we have a policy on "at least two team members must care specifically", as is practiced in the Multimedia team. - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
[meant to reply to the list, so sending again] On Wed, 16 Aug 2017 09:27:49 +0200, Ross Gammon said: > For node-* stuff however, upstream handle this by bundling a > particular version of a module in node_modules. If it is "really > difficult" to patch a node module/app to use the Debian version of a > library (because the versions have changed too much), then shouldn't > we bundle the node_module and install it as upstream do it (avoiding > all the relative path issues)? It could be followed up with a bug > (severity wishlist/normal?) to remove the bundled module once Debian > and upstream are more aligned. Embedding copies of libraries should be highly discouraged. For one thing, it is agaist policy[1], but it also it makes security support much harder, you may end up with multiple buggy versions of a library on your system, and have a bunch of duplication. It may make initial packaging easier, but it usually makes maintenance harder. [1] https://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles If upstream follows semver properly, then it shouldn't be that hard -- we'd just need one package for each major version that is actually being used (though as Paul suggested, it may be a good idea to limit the number of versions that we actually have). If upstream keeps breaking their API, then it may be a good idea for the package maintainers to talk to upstream about that. I know that several DDs have had to educate their upstreams about how to use SONAMEs in C/C++ libraries, resulting in more usable libraries. -- Hubert Chathi -- https://www.uhoreg.ca/ Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368 -- Pkg-javascript-devel mailing list Pkg-javascript-devel@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Hi,
On 08/15/2017 09:20 PM, Paul Gevers wrote:
> Hi,
>
> On 14-08-17 20:38, Hubert Chathi wrote:
>> At the BoF at DebConf, we were talking about parallel installation of
>> different versions of JS libraries. In order to do parallel
>> installation, we'd need differently named packages for different
>> versions, and it seems like the obvious way to do that is to have
>> packages called something like libjs-fooVER and node-fooVER, where VER
>> some sort of the API version, similar to the way that C/C++ library
>> packages are named after the library SONAME. If upstream follows semver
>> properly, then VER would be the major version number.
> This sounds like a reasonable approach, but what I am missing is a
> recommendation on what to do to keep the number of packages limited. I
> think it is not such a great idea to replace one problem (not the right
> version) with another one (too many packages to support).
>
> One approach could be for example the following. Try to have only two
> versions per release, but really limit the number to a maximum of 3.
>
>> For libjs-fooVER, the JavaScript files would probably go in
>> /usr/share/javascript/fooVER/ rather than /usr/share/javascript/foo.
>> Obviously everything that previously pointed to the old place would have
>> to now point to the new place, but we could create a libjs-foo
>> transitional package libjs-foo that symlinks /usr/share/javascript/foo
>> to /usr/share/javascript/fooVER so that existing packages don't break.
For libjs stuff, this is generally aligned with how things are managed
upstream. I have seen many apps looking for a particular version of js
stuff on an online CDN anyway.
>> For node-fooVER, the logical analogue would be to put the files in
>> /usr/lib/nodejs/fooVER. However, anything that uses the library does
>> "require("foo")", and so would not be able to find it there. There are
>> a couple possible options for fixing that I can think of. The first is
>> to change all "require("foo")" to "require("fooVER")", but that sounds
>> painful to do. The second option is to add a symlink from
>> node_modules/foo to /usr/lib/nodejs/fooVER in the root directory of the
>> thing that requires foo. I think the second option is the best,
>> especially since Node upstream seems to prefer searching for things in
>> node_modules rather than in /usr/lib/nodejs. And again, we may want to
>> create a transitional package node-foo that symlinks /usr/lib/nodejs/foo
>> to /usr/lib/nodejs/fooVER.
>>
>> What are peoples' thoughts on this?
> For packages that are somewhat forgiving and/or want to migrate when
> required, it may make sense to not think about the *-foo packages as
> transitional packages, but as the default, which will always point to
> the latest version. Honestly, I rather depend on such a package and fix
> bugs as they come than having to keep an eye out for updates and update
> the dependency to the latest version.
>
> Paul
>
>
>
For node-* stuff however, upstream handle this by bundling a particular
version of a module in node_modules. If it is "really difficult" to
patch a node module/app to use the Debian version of a library (because
the versions have changed too much), then shouldn't we bundle the
node_module and install it as upstream do it (avoiding all the relative
path issues)? It could be followed up with a bug (severity
wishlist/normal?) to remove the bundled module once Debian and upstream
are more aligned.
Cheers,
Ross
--
Pkg-javascript-devel mailing list
Pkg-javascript-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
Re: [Pkg-javascript-devel] parallel installation
Hi,
On 14-08-17 20:38, Hubert Chathi wrote:
> At the BoF at DebConf, we were talking about parallel installation of
> different versions of JS libraries. In order to do parallel
> installation, we'd need differently named packages for different
> versions, and it seems like the obvious way to do that is to have
> packages called something like libjs-fooVER and node-fooVER, where VER
> some sort of the API version, similar to the way that C/C++ library
> packages are named after the library SONAME. If upstream follows semver
> properly, then VER would be the major version number.
This sounds like a reasonable approach, but what I am missing is a
recommendation on what to do to keep the number of packages limited. I
think it is not such a great idea to replace one problem (not the right
version) with another one (too many packages to support).
One approach could be for example the following. Try to have only two
versions per release, but really limit the number to a maximum of 3.
> For libjs-fooVER, the JavaScript files would probably go in
> /usr/share/javascript/fooVER/ rather than /usr/share/javascript/foo.
> Obviously everything that previously pointed to the old place would have
> to now point to the new place, but we could create a libjs-foo
> transitional package libjs-foo that symlinks /usr/share/javascript/foo
> to /usr/share/javascript/fooVER so that existing packages don't break.
>
> For node-fooVER, the logical analogue would be to put the files in
> /usr/lib/nodejs/fooVER. However, anything that uses the library does
> "require("foo")", and so would not be able to find it there. There are
> a couple possible options for fixing that I can think of. The first is
> to change all "require("foo")" to "require("fooVER")", but that sounds
> painful to do. The second option is to add a symlink from
> node_modules/foo to /usr/lib/nodejs/fooVER in the root directory of the
> thing that requires foo. I think the second option is the best,
> especially since Node upstream seems to prefer searching for things in
> node_modules rather than in /usr/lib/nodejs. And again, we may want to
> create a transitional package node-foo that symlinks /usr/lib/nodejs/foo
> to /usr/lib/nodejs/fooVER.
>
> What are peoples' thoughts on this?
For packages that are somewhat forgiving and/or want to migrate when
required, it may make sense to not think about the *-foo packages as
transitional packages, but as the default, which will always point to
the latest version. Honestly, I rather depend on such a package and fix
bugs as they come than having to keep an eye out for updates and update
the dependency to the latest version.
Paul
signature.asc
Description: OpenPGP digital signature
--
Pkg-javascript-devel mailing list
Pkg-javascript-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
[Pkg-javascript-devel] parallel installation
At the BoF at DebConf, we were talking about parallel installation of
different versions of JS libraries. In order to do parallel
installation, we'd need differently named packages for different
versions, and it seems like the obvious way to do that is to have
packages called something like libjs-fooVER and node-fooVER, where VER
some sort of the API version, similar to the way that C/C++ library
packages are named after the library SONAME. If upstream follows semver
properly, then VER would be the major version number.
For libjs-fooVER, the JavaScript files would probably go in
/usr/share/javascript/fooVER/ rather than /usr/share/javascript/foo.
Obviously everything that previously pointed to the old place would have
to now point to the new place, but we could create a libjs-foo
transitional package libjs-foo that symlinks /usr/share/javascript/foo
to /usr/share/javascript/fooVER so that existing packages don't break.
For node-fooVER, the logical analogue would be to put the files in
/usr/lib/nodejs/fooVER. However, anything that uses the library does
"require("foo")", and so would not be able to find it there. There are
a couple possible options for fixing that I can think of. The first is
to change all "require("foo")" to "require("fooVER")", but that sounds
painful to do. The second option is to add a symlink from
node_modules/foo to /usr/lib/nodejs/fooVER in the root directory of the
thing that requires foo. I think the second option is the best,
especially since Node upstream seems to prefer searching for things in
node_modules rather than in /usr/lib/nodejs. And again, we may want to
create a transitional package node-foo that symlinks /usr/lib/nodejs/foo
to /usr/lib/nodejs/fooVER.
What are peoples' thoughts on this?
--
Hubert Chathi -- https://www.uhoreg.ca/
Jabber: hub...@uhoreg.ca -- Matrix: @uhoreg:matrix.org
PGP/GnuPG key: 4096R/F24C F749 6C73 DDB8 DCB8 72DE B2DE 88D3 113A 1368
--
Pkg-javascript-devel mailing list
Pkg-javascript-devel@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-javascript-devel
