Re: Should fast-evolving packages be backports-only?
On Tue, Nov 11, 2014 at 2:35 PM, Rogério Brito rogerbr...@uol.com.br wrote: On 2014-11-11 15:30, Henrique de Moraes Holschuh wrote: However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. Does this mean that I should ask for the removal of youtube-dl from testing? It will certainly bitrot in a stable release, as it supports downloading from many sites, the target sites are moving too fast (that's the nature of the web) and there's no chance that I will be hunting minimal patches to fix breakage of multiple sites, as upstream generally refactors the whole thing constantly and as multiple sites may get broken, the pile of patches would sometimes be larger than the code to extract data from some simpler sites. Maybe a decent release goal for Jessie+1: what to do with these packages that require changes that aren't fit for stable-updates to remain useful. CUT [1,2] I believe, was the most recent stab that this idea, but as Daniel Pocock pointed out there are an increasing number of cloud/web/networking/communications packages that require larger changes than stable-updates would reasonably accept. Such packages, if released in stable, would be unusable, or at worst dangerous, to users. As such, some are blocked from testing for now. The problem is that those packages can't be in backports because they can't be in testing because they can't be in stable because they can't be updated by stable-updates and need to be updated in backports which they can't be in because then they would have to be in testing. There's a chicken-and-egg game that prevents supporting those packages in Debian, users can't even say I'll just run testing since those packages aren't allowed in testing. I don't think backports being enabled by default, on its own, is the right answer (stable systems should be stable) - but some other mechanism might be appropriate for users to choose which packages they want continuously updatable. Rebecca's suggestion might be a clever way of obtaining such a feature: (1) blocking migration to testing, (2) maintaining in backports, and (3) incorporating some easy way for users to choose to pin the backports version or install from backports if not available via stable. [1] http://cut.debian.net/ [2] http://lwn.net/Articles/406301/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CANg8-dBv1vs-L1Zt+nUM�d-ks4btjz397_jxzbz0dc_j1...@mail.gmail.com
Re: Should fast-evolving packages be backports-only?
On 12/11/14 17:42, Scott Howard wrote: On Tue, Nov 11, 2014 at 2:35 PM, Rogério Brito rogerbr...@uol.com.br wrote: On 2014-11-11 15:30, Henrique de Moraes Holschuh wrote: However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. Does this mean that I should ask for the removal of youtube-dl from testing? It will certainly bitrot in a stable release, as it supports downloading from many sites, the target sites are moving too fast (that's the nature of the web) and there's no chance that I will be hunting minimal patches to fix breakage of multiple sites, as upstream generally refactors the whole thing constantly and as multiple sites may get broken, the pile of patches would sometimes be larger than the code to extract data from some simpler sites. Maybe a decent release goal for Jessie+1: what to do with these packages that require changes that aren't fit for stable-updates to remain useful. CUT [1,2] I believe, was the most recent stab that this idea, but as Daniel Pocock pointed out there are an increasing number of cloud/web/networking/communications packages that require larger changes than stable-updates would reasonably accept. Such packages, if released in stable, would be unusable, or at worst dangerous, to users. As such, some are blocked from testing for now. The problem is that those packages can't be in backports because they can't be in testing because they can't be in stable because they can't be updated by stable-updates and need to be updated in backports which they can't be in because then they would have to be in testing. There's a chicken-and-egg game that prevents supporting those packages in Debian, users can't even say I'll just run testing since those packages aren't allowed in testing. I don't think backports being enabled by default, on its own, is the right answer (stable systems should be stable) - but some other mechanism might be appropriate for users to choose which packages they want continuously updatable. Rebecca's suggestion might be a clever way of obtaining such a feature: (1) blocking migration to testing, (2) maintaining in backports, and (3) incorporating some easy way for users to choose to pin the backports version or install from backports if not available via stable. [1] http://cut.debian.net/ [2] http://lwn.net/Articles/406301/ Looking at it from the user perspective: users expect Debian packages to be stable So if we have some packages that will never be stable, as long as we have a way to signal that to users when they choose the package, I think we should make it as easy as possible to make them available in a stable release. backports already does some of that. Security team is using security updates to do it with browsers. The only question is whether to have packages that are always essentially like a backport. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5463956b.5090...@pocock.pro
Re: Should fast-evolving packages be backports-only?
On Tue, 11 Nov 2014, Rogério Brito wrote: On 2014-11-11 15:30, Henrique de Moraes Holschuh wrote: However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. Does this mean that I should ask for the removal of youtube-dl from testing? I don't think youtube-dl is in reason (c), if anything it would be in (b). from many sites, the target sites are moving too fast (that's the nature of the web) and there's no chance that I will be hunting minimal patches to fix breakage of multiple sites, as upstream generally refactors the whole thing constantly and as multiple sites may get broken, the pile of patches would sometimes be larger than the code to extract data from some simpler sites. Well, yeah, that could be a problem, as that much change can easily introduce regressions. I'd ask the release managers if they're OK with the heavy use of stable-updates for this. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20141112215130.gb20...@khazad-dum.debian.net
Should fast-evolving packages be backports-only?
It has been recently stated [0-1] that backports is enabled by default in Jessie. 1. Does that mean that if pkgX is in jessie-backports but not jessie, apt-get install pkgX will install it from -backports? 2. If so, when (if ever) is it appropriate to deliberately invoke that behaviour by removing pkgX from jessie? Possible candidates: a. Packages that work closely with hardware, where old versions don't work with new hardware (example: beignet) b. Packages that implement fast-evolving file formats or network protocols, where you need the same version as the people you are communicating with (possible example: jscommunicator [2]) c. Packages that are generally rapidly improving, and are typically used where this improvement is more important than stability The advantage of doing so (over having both the old version in jessie and the new one in jessie-backports) is that non-technical users (who may not know that backports exists) get the new version they probably want; the disadvantage is that users who explicitly want stability can no longer choose it (except by pinning or using snapshot.debian.org, which also block security updates of that package). In the long run it may be a better idea to have these packages suggest upgrading to -backports in their this hardware/protocol version/option not supported error message, or on startup if there is no easy way to identify attempts to use the newer features, but it is too late to do this for jessie. (Release team have already ruled that a. (#767961) and b. (#768933) are not valid reasons for freeze exceptions; I guess this would also forbid stable updates) [0] https://lists.debian.org/debian-devel/2014/11/msg00339.html [1] My own sources.list has # jessie-backports, previously on backports.debian.org # Line commented out by installer because it failed to verify: #deb http://ftp.uk.debian.org/debian/ jessie-backports main but https://lists.debian.org/debian-user/2014/09/msg01174.html reports getting one with that line uncommented [2] https://lists.debian.org/debian-release/2014/11/msg00866.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/54620f78.4040...@zoho.com
Re: Should fast-evolving packages be backports-only?
On 11/11/14 14:30, Rebecca N. Palmer wrote: It has been recently stated [0-1] that backports is enabled by default in Jessie. 1. Does that mean that if pkgX is in jessie-backports but not jessie, apt-get install pkgX will install it from -backports? 2. If so, when (if ever) is it appropriate to deliberately invoke that behaviour by removing pkgX from jessie? Possible candidates: a. Packages that work closely with hardware, where old versions don't work with new hardware (example: beignet) b. Packages that implement fast-evolving file formats or network protocols, where you need the same version as the people you are communicating with (possible example: jscommunicator [2]) c. Packages that are generally rapidly improving, and are typically used where this improvement is more important than stability Maybe also d. packages that the security team decide to support by using upstream releases (in other words, should the browsers be distributed through backports only?) One big question that arises then (and what I asked in a separate thread about the browser-related packages but it is relevant to other classes of package too) is compatibility - if package foo is allowed to change, do all packages broken by the change (e.g. browser plugins) get to be uploaded again too? - if some package hides the complexity of the change and the maintainer has kept the API stable so that dependent packages don't break should it be looked on more favorably and allowed to be updated in stable too? Personally, I feel that having backports enabled by default is only part of the solution to the challenges with volatile packages but it may be a step in the right direction. I also feel that this is something that impacts each maintainer and each user differently. Some people are working in parts of the system where the freeze concept really is the most important thing. Other people are working on applications where network compatibility is the most important thing (as it is with communications) and people simply won't use the package or won't be able to use it successfully if is not updated. Ultimately, with more and more packages being in this category as the world becomes more networked/cloudified, this impacts Debian's relevance for whole groups of users. The advantage of doing so (over having both the old version in jessie and the new one in jessie-backports) is that non-technical users (who may not know that backports exists) get the new version they probably want; the disadvantage is that users who explicitly want stability can no longer choose it (except by pinning or using snapshot.debian.org, which also block security updates of that package). In the long run it may be a better idea to have these packages suggest upgrading to -backports in their this hardware/protocol version/option not supported error message, or on startup if there is no easy way to identify attempts to use the newer features, but it is too late to do this for jessie. (Release team have already ruled that a. (#767961) and b. (#768933) are not valid reasons for freeze exceptions; I guess this would also forbid stable updates) [0] https://lists.debian.org/debian-devel/2014/11/msg00339.html [1] My own sources.list has # jessie-backports, previously on backports.debian.org # Line commented out by installer because it failed to verify: #deb http://ftp.uk.debian.org/debian/ jessie-backports main but https://lists.debian.org/debian-user/2014/09/msg01174.html reports getting one with that line uncommented [2] https://lists.debian.org/debian-release/2014/11/msg00866.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/54621b37.4040...@pocock.pro
Re: Should fast-evolving packages be backports-only?
On Tue, Nov 11, 2014 at 9:20 AM, Daniel Pocock dan...@pocock.pro wrote:
On 11/11/14 14:30, Rebecca N. Palmer wrote:
It has been recently stated [0-1] that backports is enabled by default
in Jessie.
1. Does that mean that if pkgX is in jessie-backports but not jessie,
apt-get install pkgX will install it from -backports?
2. If so, when (if ever) is it appropriate to deliberately invoke that
behaviour by removing pkgX from jessie?
One big question that arises then (and what I asked in a separate thread
about the browser-related packages but it is relevant to other classes
of package too) is compatibility
- if package foo is allowed to change, do all packages broken by the
change (e.g. browser plugins) get to be uploaded again too?
- if some package hides the complexity of the change and the maintainer
has kept the API stable so that dependent packages don't break should it
be looked on more favorably and allowed to be updated in stable too?
maybe a crazy idea, but maybe build in some easy way to apt-pin
package (and dependencies) to testing in apt/dpkg? This way we can
leverage all of the existing transition infrastructure and essentially
provide backports for all packages with no extra work?
possibly lots of unintended consequences, and someone will mess up
their machine by pulling in tons of libraries from the future, but it
will essentially perform the same function for those users that need
the up-to-date leaf packages while keeping the stable core. testing is
pinned by default to 100
$ apt-get install-updated ${PACKAGE}
where install-updated will pin ${PACKAGE} to testing and do $ apt-get
install ${PACKAGE} -t testing This will pull in the package from
testing and dependencies from testing that are missing from stable.
Not a good idea for jessie, maybe something to think about for future.
Either way, I think you have excellent points in your final paragraph,
thank you Rebecca and Daniel for bringing this up.
On Tue, Nov 11, 2014 at 9:20 AM, Daniel Pocock dan...@pocock.pro wrote:
I also feel that this is something that impacts each maintainer and each
user differently. Some people are working in parts of the system where
the freeze concept really is the most important thing. Other people are
working on applications where network compatibility is the most
important thing (as it is with communications) and people simply won't
use the package or won't be able to use it successfully if is not
updated. Ultimately, with more and more packages being in this category
as the world becomes more networked/cloudified, this impacts Debian's
relevance for whole groups of users.
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/cang8-dbkbdzghr-vvkc7x9g0f6c1quaz9j3kgvbsdsusent...@mail.gmail.com
Re: Should fast-evolving packages be backports-only?
Rebecca N. Palmer rebecca_pal...@zoho.com (2014-11-11): It has been recently stated [0-1] that backports is enabled by default in Jessie. Yes, and that's a bug. See #764982. 1. Does that mean that if pkgX is in jessie-backports but not jessie, apt-get install pkgX will install it from -backports? Yes. And that's the reason why I'm going to disable jessie-backports by default when I process my todo list. Mraw, KiBi. signature.asc Description: Digital signature
Re: Should fast-evolving packages be backports-only?
On Tue, 11 Nov 2014, Rebecca N. Palmer wrote: Possible candidates: a. Packages that work closely with hardware, where old versions don't work with new hardware (example: beignet) b. Packages that implement fast-evolving file formats or network protocols, where you need the same version as the people you are communicating with (possible example: jscommunicator [2]) c. Packages that are generally rapidly improving, and are typically used where this improvement is more important than stability We used to have the volatile archive for software that absolutely needs to be updated very often (like virus scanners). We now have the stable-updates archive for this. https://wiki.debian.org/StableUpdates If packages are taking too long to go from stable-proposed-updates to stable-updates, that's something we could talk to the release managers about. Although, I am *sure* lack of widespread use (and therefore testing) of stable-proposed-updates by users and developers (HINT HINT HINT HINT) is one of the reasons. However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. backports seem like a better solution for this case. However, we'd need to add further requirements: packages not built from the same source package cannot depend on such never-for-stable packages, and we must tag them somehow so that they never get released to stable... -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2014173015.gb2...@khazad-dum.debian.net
Re: Should fast-evolving packages be backports-only?
On November 11, 2014 12:22:57 PM EST, Cyril Brulebois k...@debian.org wrote: Rebecca N. Palmer rebecca_pal...@zoho.com (2014-11-11): It has been recently stated [0-1] that backports is enabled by default in Jessie. Yes, and that's a bug. See #764982. 1. Does that mean that if pkgX is in jessie-backports but not jessie, apt-get install pkgX will install it from -backports? Yes. And that's the reason why I'm going to disable jessie-backports by default when I process my todo list. As long as apt prefers a version from stable over a version from backports when both are available (unless instructed to install from backports) why is this a problem? It seems more user friendly to me for a package that's been specifically ask for to end up installed rather than not. Scott K -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/e8f87680-99dc-4fa5-8fc2-754d48d57...@kitterman.com
Re: Should fast-evolving packages be backports-only?
Scott Kitterman deb...@kitterman.com (2014-11-11): As long as apt prefers a version from stable over a version from backports when both are available (unless instructed to install from backports) why is this a problem? It seems more user friendly to me for a package that's been specifically ask for to end up installed rather than not. As I probably wrote in the mentioned bug report, it seems more friendly, safe, and honest to stick to stable packages on a stable system instead of silently stuff from backports. It's not like uncommenting a line in sources.list should be a huge burden anyway (having to figure out which line to add, depending on the target suite, wasn't too trivial when backports moved from backports.d.o to the archive; but we're way past that now). Mraw, KiBi. signature.asc Description: Digital signature
Re: Should fast-evolving packages be backports-only?
On 11/11/14 18:30, Henrique de Moraes Holschuh wrote: On Tue, 11 Nov 2014, Rebecca N. Palmer wrote: Possible candidates: a. Packages that work closely with hardware, where old versions don't work with new hardware (example: beignet) b. Packages that implement fast-evolving file formats or network protocols, where you need the same version as the people you are communicating with (possible example: jscommunicator [2]) c. Packages that are generally rapidly improving, and are typically used where this improvement is more important than stability We used to have the volatile archive for software that absolutely needs to be updated very often (like virus scanners). We now have the stable-updates archive for this. https://wiki.debian.org/StableUpdates If packages are taking too long to go from stable-proposed-updates to stable-updates, that's something we could talk to the release managers about. Although, I am *sure* lack of widespread use (and therefore testing) of stable-proposed-updates by users and developers (HINT HINT HINT HINT) is one of the reasons. However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. backports seem like a better solution for this case. However, we'd need to add further requirements: packages not built from the same source package cannot depend on such never-for-stable packages, and we must tag them somehow so that they never get released to stable... That is not so easy though or it may have side effects For example, if a library changes implementation details but the public API and ABI does not change and no other dependent packages need to be recompiled then it should be OK for those dependent packages to live in stable. Does that mean the maintainer has to put their libfoo-dev in stable while keeping their volatile libfoo1 in backports? -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/54624ca0.80...@pocock.pro
Re: Should fast-evolving packages be backports-only?
On Tue, 11 Nov 2014, Daniel Pocock wrote: On 11/11/14 18:30, Henrique de Moraes Holschuh wrote: On Tue, 11 Nov 2014, Rebecca N. Palmer wrote: Possible candidates: a. Packages that work closely with hardware, where old versions don't work with new hardware (example: beignet) b. Packages that implement fast-evolving file formats or network protocols, where you need the same version as the people you are communicating with (possible example: jscommunicator [2]) c. Packages that are generally rapidly improving, and are typically used where this improvement is more important than stability We used to have the volatile archive for software that absolutely needs to be updated very often (like virus scanners). We now have the stable-updates archive for this. https://wiki.debian.org/StableUpdates If packages are taking too long to go from stable-proposed-updates to stable-updates, that's something we could talk to the release managers about. Although, I am *sure* lack of widespread use (and therefore testing) of stable-proposed-updates by users and developers (HINT HINT HINT HINT) is one of the reasons. However, candidate packages due to reason (c) above really are a problem, IMHO they shouldn't be in stable in the first place. backports seem like a better solution for this case. However, we'd need to add further requirements: packages not built from the same source package cannot depend on such never-for-stable packages, and we must tag them somehow so that they never get released to stable... That is not so easy though or it may have side effects For example, if a library changes implementation details but the public API and ABI does not change and no other dependent packages need to be recompiled then it should be OK for those dependent packages to live in stable. Does that mean the maintainer has to put their libfoo-dev in stable while keeping their volatile libfoo1 in backports? Looks like a let's not go there kind of deal. Make it simple, so that it has a non-zero chance of flying and all that. After we have some experience with it in practice, we revisit the issue. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/2014184244.gd2...@khazad-dum.debian.net
Re: Should fast-evolving packages be backports-only?
❦ 11 novembre 2014 12:29 -0500, Scott Kitterman deb...@kitterman.com : As long as apt prefers a version from stable over a version from backports when both are available (unless instructed to install from backports) why is this a problem? The user may expect the same characteristics than for packages in stable, like stability (not upgrading to a new upstream version each month). -- Let the data structure the program. - The Elements of Programming Style (Kernighan Plauger) signature.asc Description: PGP signature
Re: Should fast-evolving packages be backports-only?
On 2014-11-11 15:30, Henrique de Moraes Holschuh wrote:
However, candidate packages due to reason (c) above really are a problem,
IMHO they shouldn't be in stable in the first place.
Does this mean that I should ask for the removal of youtube-dl from testing?
It will certainly bitrot in a stable release, as it supports downloading
from many sites, the target sites are moving too fast (that's the nature
of the web) and there's no chance that I will be hunting minimal patches
to fix breakage of multiple sites, as upstream generally refactors the
whole thing constantly and as multiple sites may get broken, the pile of
patches would sometimes be larger than the code to extract data from
some simpler sites.
I am, though, very happy to upload newer upstream versions.
Cheers,
--
Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFC
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/m3toeh$ja3$1...@ger.gmane.org
