Re: Proposal: Repository for fast-paced package backports
On 2019, ജൂലൈ 28 6:36:21 PM IST, Phil Morrell wrote: >On Tue, Jan 01, 2019 at 05:49:37PM +0530, Pirate Praveen wrote: >> I think STS (Short term support) will fit nicely with LTS. If there >is >> no serious objections, I'd go with this. > >As debconf is finishing, though I don't know if either of you attended >this year, has there been any progress on this idea? Is there an >evergreen/sts/fasttrack destination I can put in my dput.cf to support >normally unsuitable packages like jenkins/virtualbox/firefox/gitlab? Hi Phil, It stalled for a long time and we restarted work on it recently. We are in the process of getting server space to setup dak. Praveen -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Proposal: Repository for fast-paced package backports
On Tue, Jan 01, 2019 at 05:49:37PM +0530, Pirate Praveen wrote: > I think STS (Short term support) will fit nicely with LTS. If there is > no serious objections, I'd go with this. As debconf is finishing, though I don't know if either of you attended this year, has there been any progress on this idea? Is there an evergreen/sts/fasttrack destination I can put in my dput.cf to support normally unsuitable packages like jenkins/virtualbox/firefox/gitlab? signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Hi Dominik, On 26/12/18 2:16 am, Dominik George wrote: > Heisann, alle sammen, > > as announced in the recent thread about maintaining, I hereby propose a > repository that allows making “backports” of packages available to users > of the stable distribution, if those packages cannot be maintained in > testing and backported in the usual way. If you are interested in what > lead up to that, please see bug #915050. I will give a short summary of > it here. > > > Reasons for having a special place for some packages > > > (You may want to skip this part if you are familiar with the situation.) > > As all developers know (but passers-by may not), for software to enter > the Debian archive, it is always uploaded to the unstable distribution, > then migrates to testing (hopefully ;)), which is at some point snapshot > and made the new stable release. From there on, maintainers have two > obligations: Firstly, keep the package in stable good and secure, e.g. > by uploading security fixes for it once they become available upstream, > or even backport fixes themselves. Secondly, provide the package in > unstable with updates and ensure its migration, to keep it ready for the > next stable release. > > Now, for some software packages, this process is problematic, because > upstream may have another idea about software lifecycles. Concerning the > GitLab example, upstream provides security fixes for three months for > their stable releases. Backporting fixes from newer versions is very > hard or impossible because the massive amounts of changes to the > software in every new versions. This is something that also affects > other packages, like Mozilla Firefox, which has a firefox package in > unstable, and a separate firefox-esr package, with the ESR version of > Firefox. Only the latter migrates to testing. > > Users of Debian honour it for its stability, but as an agile software > lifecycle is adapted by more and more very popular software packages, > not being able to install these packages in the trusted, well-known > fashion through the official apt repositories is becoming more and more > of a drawback. > > It can easily be assumed that the normal release and maintenance cycle > of Debian stable will not change, which is very good, so we should find > a way to still provide such software as described above to users. > > > Why backports is not enough > === > > This also is well-known, but for completeness: Formal backports in > stable-backports are required to be direct backports from testing, and > are a stepping stone within the upgrade from stable to stable+1. Thus, a > version of a package that is not in testing can never be in > stable-backports. > > > > Implications for the situation at hand (gitlab) > === > > As there were quite a few concerns raised (some of which I share, and > some I don’t): Of course, if a software intended for volatile has a ton > of dependencies (intended to go into backports), all backports rules and > powers of the ftp-masters apply. Repeating myself: volatile is not meant > to ease the life of maintainers. Did you get a chance to work on it? This is mostly in reference to #915050. Since GitLab is in good shape now (people have their own perception of good, though), we'd like to fast track this and take this forward. Let us know about the same :) Best, Utkarsh signature.asc Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
Hi Dominik,
happy new year to you and everybody.
I read your proposal, and the whole discussion with great interest.
In brief, I think that it would be wise to uncouple the fast-paced
("fast-forwards" ?) repository that you propose from the stable backports,
and I hope that you will be able to get support from the Debian
infrastructure for making it happen.
Thomas sent the link to the 2013 proposal on "Developer repositories for
Debian" (https://lists.debian.org/debian-devel/2013/05/msg00131.html),
which has a great potential. If you have the resources for doing so,
how about sending a patch to the FTP team that would implement a
solution in line with this proposal ? Debian support for this endeavor
might be possible though sposored developments such as the Google Summer
of Code or the Outreachy programs, for instance. If the resulting work
is integrated, you would get an easy way to use the Debian
infrastructure for your goals, and if it is rejected, well, at least you
will be able to run it on your side, and it may be an asset for
experimenting or for operating through stable releases in the long run.
Regarding the details of your proposal, I have not seen keywords
regarding regression checks and reproducible builds. These are powerful
tools that did not exist when the Debian Backports were invented. Have
you considered utilising them ? Perhaps it is too naive, but on my
side, for the leaf packages I maintain, I have always been dreaming of a
repository where I can just migrate a package from unstable (instead of
rebuilding) if there is sufficient evidence that it works flawlessly on
a Stable background. Which may be why I am trying to convince you to
implemente "Developer repositories" (aka "PPAs" or "Bikesheds") for
solving your own problem :)
Have a nice day,
Charles
--
Charles Plessy Tsurumi, Kanagawa, Japan
Debian Med packaging team http://www.debian.org/devel/debian-med
Tooting from work, https://mastodon.technology/@charles_plessy
Tooting from home, https://framapiaf.org/@charles_plessy
Re: Proposal: Repository for fast-paced package backports
On Mon, Dec 31, 2018 at 1:06 PM Ben Hutchings wrote: > At the risk of bikeshedding, some alternate names that might be less > confusing: > > - fresh-apps > - evergreen > - rolling-apps At further risk of bikeshedding, how about "sideports"? 1. Uses a metaphor rather similar to backports. 2. Sorts lexicographically after bpo, although its not clear to me which should take precedence. 3. The sideports.org domain name could be registered and used to address the concern backports maintainers keep bringing up about starting this outside of debian first. Less arguing required, more doing. Negatives: 1. Sorts lexicographically after security updates, "debXuY" Best wishes, Mike
Re: Proposal: Repository for fast-paced package backports
On 1/1/19 3:42 PM, Scott Leggett wrote: > To continue the bikeshedding, what about "express"? Seems quite fitting > as the packages don't make the usual stop through testing... I think STS (Short term support) will fit nicely with LTS. If there is no serious objections, I'd go with this. signature.asc Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
On 2018-12-31.18:06, Ben Hutchings wrote:
> On Mon, 2018-12-31 at 18:31 +0100, Jonas Meurer wrote:
> > Pirate Praveen:
> > > On 2018, ഡിസംബർ 31 5:19:22 PM IST, Jonas Meurer
> > > wrote:
> > > >
> > > > Please don't name it 'rolling'. This term is used a lot in the sense of
> > > > 'rolling releases' by other distributions, and also in discussions
> > > > about
> > > > constantly usable testing.
> > >
> > > Well, it only makes things clear as the packages in this repo will be
> > > rolling.
> >
> > ... as packages do in unstable, testing and ${stable}-backports. So it's
> > not a particularly good term to describe the unique feature of the new
> > repo either. In my eyes, 'fastpaced' makes the point far better.
> >
> > But as said, the main argument against calling it 'rolling' is that it
> > would create confusion due to the name already being used in other
> > (Debian-related) contexts.
>
> At the risk of bikeshedding, some alternate names that might be less
> confusing:
>
> - fresh-apps
> - evergreen
> - rolling-apps
To continue the bikeshedding, what about "express"? Seems quite fitting
as the packages don't make the usual stop through testing...
--
Regards,
Scott Leggett.
signature.asc
Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On Mon, 2018-12-31 at 18:31 +0100, Jonas Meurer wrote:
> Pirate Praveen:
> > On 2018, ഡിസംബർ 31 5:19:22 PM IST, Jonas Meurer
> > wrote:
> > > Pirate Praveen:
> > > > On 12/28/18 11:06 AM, Thomas Goirand wrote:
> > > > > If the problem is hardware and connectivity, then IMO you can easily
> > > > > find a sponsor for it. My company could well offer it for example
> > > > > (hosted in Geneva with very nice connectivity to almost everywhere).
> > > > >
> > > > > Setting-up a repository isn't hard. And for a start, I don't think
> > > you
> > > > > really need a buildd network, just amd64 is ok-ish.
> > > >
> > > > I'd like go ahead with this offer and create rolling.debian.net (as
> > > > someone suggested already to avoid reusing volatile). I think we can
> > > > take the setup discussions offlist.
> > >
> > > Please don't name it 'rolling'. This term is used a lot in the sense of
> > > 'rolling releases' by other distributions, and also in discussions
> > > about
> > > constantly usable testing.
> >
> > Well, it only makes things clear as the packages in this repo will be
> > rolling.
>
> ... as packages do in unstable, testing and ${stable}-backports. So it's
> not a particularly good term to describe the unique feature of the new
> repo either. In my eyes, 'fastpaced' makes the point far better.
>
> But as said, the main argument against calling it 'rolling' is that it
> would create confusion due to the name already being used in other
> (Debian-related) contexts.
At the risk of bikeshedding, some alternate names that might be less
confusing:
- fresh-apps
- evergreen
- rolling-apps
Ben.
--
Ben Hutchings
Power corrupts. Absolute power is kind of neat. - John Lehman
signature.asc
Description: This is a digitally signed message part
Re: Proposal: Repository for fast-paced package backports
Pirate Praveen:
> On 2018, ഡിസംബർ 31 5:19:22 PM IST, Jonas Meurer wrote:
>> Pirate Praveen:
>>> On 12/28/18 11:06 AM, Thomas Goirand wrote:
If the problem is hardware and connectivity, then IMO you can easily
find a sponsor for it. My company could well offer it for example
(hosted in Geneva with very nice connectivity to almost everywhere).
Setting-up a repository isn't hard. And for a start, I don't think
>> you
really need a buildd network, just amd64 is ok-ish.
>>>
>>> I'd like go ahead with this offer and create rolling.debian.net (as
>>> someone suggested already to avoid reusing volatile). I think we can
>>> take the setup discussions offlist.
>>
>> Please don't name it 'rolling'. This term is used a lot in the sense of
>> 'rolling releases' by other distributions, and also in discussions
>> about
>> constantly usable testing.
>
> Well, it only makes things clear as the packages in this repo will be rolling.
... as packages do in unstable, testing and ${stable}-backports. So it's
not a particularly good term to describe the unique feature of the new
repo either. In my eyes, 'fastpaced' makes the point far better.
But as said, the main argument against calling it 'rolling' is that it
would create confusion due to the name already being used in other
(Debian-related) contexts.
Cheers
jonas
signature.asc
Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
On 2018, ഡിസംബർ 31 5:19:22 PM IST, Jonas Meurer wrote: >Pirate Praveen: >> On 12/28/18 11:06 AM, Thomas Goirand wrote: >>> If the problem is hardware and connectivity, then IMO you can easily >>> find a sponsor for it. My company could well offer it for example >>> (hosted in Geneva with very nice connectivity to almost everywhere). >>> >>> Setting-up a repository isn't hard. And for a start, I don't think >you >>> really need a buildd network, just amd64 is ok-ish. >> >> I'd like go ahead with this offer and create rolling.debian.net (as >> someone suggested already to avoid reusing volatile). I think we can >> take the setup discussions offlist. > >Please don't name it 'rolling'. This term is used a lot in the sense of >'rolling releases' by other distributions, and also in discussions >about >constantly usable testing. Well, it only makes things clear as the packages in this repo will be rolling. >From all names that got suggested so far, 'fastpaced' is the only one >that doesn't have a special meaning in Debian context yet, at least to >my knowledge. That's why I would prefer that one. > >Thanks for pushing that effort, Nik and Pirate! I really like the idea >of having a repository that provides newer releases of fast-moving >software built for stable. > >Cheers, > jonas -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Proposal: Repository for fast-paced package backports
Pirate Praveen: > On 12/28/18 11:06 AM, Thomas Goirand wrote: >> If the problem is hardware and connectivity, then IMO you can easily >> find a sponsor for it. My company could well offer it for example >> (hosted in Geneva with very nice connectivity to almost everywhere). >> >> Setting-up a repository isn't hard. And for a start, I don't think you >> really need a buildd network, just amd64 is ok-ish. > > I'd like go ahead with this offer and create rolling.debian.net (as > someone suggested already to avoid reusing volatile). I think we can > take the setup discussions offlist. Please don't name it 'rolling'. This term is used a lot in the sense of 'rolling releases' by other distributions, and also in discussions about constantly usable testing. From all names that got suggested so far, 'fastpaced' is the only one that doesn't have a special meaning in Debian context yet, at least to my knowledge. That's why I would prefer that one. Thanks for pushing that effort, Nik and Pirate! I really like the idea of having a repository that provides newer releases of fast-moving software built for stable. Cheers, jonas signature.asc Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
On 12/30/18 8:02 AM, Pirate Praveen wrote:
> On 12/28/18 11:06 AM, Thomas Goirand wrote:
>>> If you know how to start with a new service at
>>> {volatile,fastpaced,whatever}.debian.net without having to reinvent the
>>> wheel for acceptign uploads, getting packages built, etc., please
>>> enlighten me.
>>
>> Setting-up reprepro, or even Dak, isn't that hard.
>
> I already use reprepro
> https://people.debian.org/~praveen/gitlab/
>
> and will continue to use the same (as the number of packages would be
> small and it is pretty simple to setup).
For such service, and because your goal is to later integrate with the
rest of Debian, I would recommend Dak, rather than reprepro. Can someone
of the FTP master team confirm this?
Cheers,
Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
On 12/30/18 12:47 AM, Wouter Verhelst wrote: > On Wed, Dec 26, 2018 at 07:19:02PM +0100, Dominik George wrote: >> - It costs a lot time that could better be used elsewhere. >> - It costs extra money, which I for one do not have to spare. > > So ask someone who does and who would care about this proposed new > suite? Alternatively, ask for a VM on debian.org infrastructure? > Personally I don't know whether you'll get that, but IMHO this seems > like something that could use debian.org infrastructure, even if it > should use a debian.net domain for the time being. If you can't get a VM on the Debian infra, get in touch with me, and I'll get one (sponsored) for you. Cheers, Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
On 12/28/18 11:06 AM, Thomas Goirand wrote:
> If the problem is hardware and connectivity, then IMO you can easily
> find a sponsor for it. My company could well offer it for example
> (hosted in Geneva with very nice connectivity to almost everywhere).
>
> Setting-up a repository isn't hard. And for a start, I don't think you
> really need a buildd network, just amd64 is ok-ish.
I'd like go ahead with this offer and create rolling.debian.net (as
someone suggested already to avoid reusing volatile). I think we can
take the setup discussions offlist.
>> If you know how to start with a new service at
>> {volatile,fastpaced,whatever}.debian.net without having to reinvent the
>> wheel for acceptign uploads, getting packages built, etc., please
>> enlighten me.
>
> Setting-up reprepro, or even Dak, isn't that hard.
I already use reprepro
https://people.debian.org/~praveen/gitlab/
and will continue to use the same (as the number of packages would be
small and it is pretty simple to setup).
signature.asc
Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
On Wed, Dec 26, 2018 at 07:19:02PM +0100, Dominik George wrote: > > If you don't see obstacles, why not start today? > > I think I already made those obstacles clear: Starting outside means > buying, (or getting donated) > installing and operating at least a server vor volatile.debian.net (or > whatever you call it), setting up and maintaining an upload queue, the > queued, and everything around it, building from source for at least the most > important architectures on hardware that needs to be there and maintained for > that, etc. There are several issues with that: > > - It costs a lot time that could better be used elsewhere. > - It costs extra money, which I for one do not have to spare. So ask someone who does and who would care about this proposed new suite? Alternatively, ask for a VM on debian.org infrastructure? Personally I don't know whether you'll get that, but IMHO this seems like something that could use debian.org infrastructure, even if it should use a debian.net domain for the time being. > - I do not sure I can do it right, because I do not know all the >technical details. I'm sure that if you ask nicely, ftpmaster people would be happy to help you out in setting up things, if you get stuck. After all, they did that for backports initially, too, if I'm not mistaken... [...] > Don't get me wrong - I would not hesitate to go through it if it were > for anything that could break things, or make life harder for others, or > something like that. I am just putting the impact of the change and the > resources needed for seperate infrastructure in relation. Everything > about this proposal ahs already been tested when -backports was young > (thanks for doing the work!). This proposal contains nothing new to > learn, neither technically nor policy-wise. Nothing you can see right now, perhaps. However, since your proposed ruleset is different from that of -backports as well as that of the official debian.org infrastructure, I can imagine that you might want to change the setup a few times when you're still experimenting with the whole thing. Doing that on the same infrastructure as the official Debian archive network carries with it a significant risk to the proper operation of the base Debian distribution (imagine you ask DSA to please make a destructive change to your suite and they accidentally make it to more than just your suite...). I'm sure you don't want that. Even ignoring that, having your own infrastructure allows you to easily experiment with things, without having to ask someone with the necessary root login to make the changes that you need. For that reason, it seems entirely reasonable to request that, at least initially, new suites are installed on separate infrastructure from the official debian.org infrastructure. -- To the thief who stole my anti-depressants: I hope you're happy -- seen somewhere on the Internet on a photo of a billboard
Re: Proposal: Repository for fast-paced package backports
On 12/26/18 8:16 PM, Dominik George wrote: >> For backports the general supportability assumption is that you provide a >> sane upgrade path from stable to the backports and from the backport to the >> next stable (optimally the same package). Once you take the presence of the >> stable package out of the mix, it becomes weird. How long do you need to >> preserve compatibility code? How does an agile package that does not fit >> Debian's release cycle cater to these requirements? > > This is wrong, at least in a big part. It's completely, fully, right. I very much subscribe to all of what you've quoted from Philipp Kern. > The stable release cycle does not apply for -backports. It very much does. Backports are opened when Stable is released, and closed when Stable is EOL (and there's no backports for LTS, ATM). So backports are exactly following the lifecycle of a given stable release. Or do you mean something else here? > A package in -backports can be udpated an > arbitrary number of times during the development window of stable+1 And if there's any problem during any of the upgrades, it can be considered a bug and shall be fixed, by all means! And any upgrades you may think of. Like stable -> backports, or backports -> testing, or even from one version of the backport to the next. > If you take into account edge cases, where a package is removed from > testing during the freeze, is removed from -backports as a result, is > not released with stable+1, then gets fixed and reintroduced to testing Normally, this does not happen. If a package is removed from testing during the freeze, then it's not re-introduced. BTW, could you *please* stop calling what you want as "volatile"? Each time I see it, I think of stable-updates, that's *very* confusing. Cheers, Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
On 12/27/18 10:49 AM, L.P.H. van Belle wrote: > > Hai, > > A very interesting thread this, since im doing this already for samba, my > comments.. > If i may .. > > Im running a samba repo now for jessie and stretch. ( and ubuntu 18.04 ) > I really needed newer samba packages and i was not able to get them uploaded > to unstable. > So i decided to build them myself and share them. > > And now people are more and more using my samba package over the official > debian package. > Because the newer version are build against debian stable or oldstable, and > people can choose there upgrade. > > If the might be a fast-lane repo, why not per package version. > This way we can keep the changes to other packages small and limited. > > What i now now do. > I have 4 repo's for jessie, jessie-samba45 jessie-samba46 jessie-samba47 > jessie-samba48 > I have 4 repo's for stretch, stretch-samba46 stretch-samba47 stretch-samba48 > stretch-samba49 > (And for the ubuntu supporters a samba49 in amd64 only.) > > Why 4? > https://wiki.samba.org/index.php/Samba_Release_Planning > Debian version 4.5 (in stable) And the 3 maintanted samba versions. > > Currely Debian samba is 4.5.12, which is fine, but if you want a more > advanced samba, you really need to upgrade. > The difference between 4.5.12 and 4.5.16, is major in winbind fixed already. > > What i also do, at least try to, keep 2 versions of samba in sync, above > shows 3 but i need 2 at least. > I do this so the OS upgrade wont affect a samba upgrade. > Users choose a samba version and stay in that version, untill they dicede to > upgrade samba, or get new when new debian stable has a higher release. > Im doing this since samba 4.1.x debian Wheezy, and main reason is the fast > samba pace and slow debian packages. > Not that i mind that, i do love debian and its stability so, keep it slow, > yes, but an option for fast moving packages would be nice. > > So how about something like this. > deb http://deb.debian.org/debian fastmove/stretch-samba48 main contrib > non-free" > > And if one want a samba 4.9 that does not exist withing debian, you create > the stretch-samba49 line. > deb http://deb.debian.org/debian fastmove/stretch-samba49 main contrib > non-free" > > And if the stable version gets thats a higher version then the fastmove, its > automaticly picked up again. > > So if a contibuter wants a higher version, he can build it, and upload it. > And the original maintains should get a ping of a higher release version, so > if needed they can adopt it to experimental before it goes to unstable. > > This is a bit what i do for samba ( and debian ) > > Just my suggest as community helper. > > > Greetz, > > Louis If you care so much for Samba, why don't you maintain it in Debian backports yourself then? Especially considering the current maintainer wrote he doesn't have time to do so, it'd be nice to see your contribution there. Cheers, Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
On 12/27/18 3:06 PM, Dominik George wrote: >> Basically you would like to build an architecture equivalent to Ubuntu's >> PPA > > No. Please at least try reading the thread first - it was explicitly > explained several times that this has absolutely nothing to do with the > idea of PPAs. Well, I have read, and taken into account all of the thread. However, I feel like what you need is the Debian Bikesheds repositories (note: absolutely *not* the Ubuntu-PPA-like, but rather, Ganneff proposal). How about contributing to Dak so that we get the Bikesheds creation API that is still lacking? Cheers, Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
Thomas Goirand schrieb: > And for a start, I don't think you really need a buildd network, just amd64 > is ok-ish. Agreed. If there's actual demand for further architecture support, it will appear naturally by people offering to do the necessary setup. Cheers, Moritz
Re: Proposal: Repository for fast-paced package backports
(Please reply to pkg-samba-maint only) Le jeu. 27 déc. 2018 à 11:00, L.P.H. van Belle a écrit : > > > Hai, Hi, > A very interesting thread this, since im doing this already for samba, my > comments.. > If i may .. > > Im running a samba repo now for jessie and stretch. ( and ubuntu 18.04 ) > I really needed newer samba packages and i was not able to get them uploaded > to unstable. > So i decided to build them myself and share them. This is different here. Samba is not in backport because of lack of time from my side (or other members of the team). I think that Samba perfectly fits in backport, as the version in testing is already the latest upstream. > And now people are more and more using my samba package over the official > debian package. > Because the newer version are build against debian stable or oldstable, and > people can choose there upgrade. Do you have any stats here? How many download each month? How many different source IPs? per dist, per samba version? > If the might be a fast-lane repo, why not per package version. > This way we can keep the changes to other packages small and limited. > > What i now now do. > I have 4 repo's for jessie, jessie-samba45 jessie-samba46 jessie-samba47 > jessie-samba48 > I have 4 repo's for stretch, stretch-samba46 stretch-samba47 stretch-samba48 > stretch-samba49 > (And for the ubuntu supporters a samba49 in amd64 only.) So, you have 9 repos. How long does it takes to update all those when a security fix comes? Regards -- Mathieu Parent
Re: Proposal: Repository for fast-paced package backports
On 12/26/18 7:19 PM, Dominik George wrote:
> Hi,
>
>> 2. I am happy with the current charter of backports and I think it's
>> possible to move forward with fastpaced without having to change
>> that charter.
>
> Yep. That's exactly why the proposal changes nothing about -backports. I
> am still confused why Alex and you keep insisting that anything would be
> changing there.
>
>> 3. formerer is speaking from experience when he says that it's
>> possible to make this kind of change unofficially first, learn
>> from it, and thus set the groundwork for making it official.
>>
>> If you foresee obstacles to that, can you say more about where
>> they lie? Maybe we can help address them, or maybe we can find
>> another way forward.
>>
>> If you don't see obstacles, why not start today?
>
> I think I already made those obstacles clear: Starting outside means
> buying, installing and operating at least a server vor
> volatile.debian.net (or whatever you call it), setting up and
> maintaining an upload queue, the queued, and everything around it,
> building from source for at least the most important architectures on
> hardware that needs to be there and maintained for that, etc. There are
> several issues with that:
>
> - It costs a lot time that could better be used elsewhere.
> - It costs extra money, which I for one do not have to spare.
> - I do not sure I can do it right, because I do not know all the
>technical details.
If the problem is hardware and connectivity, then IMO you can easily
find a sponsor for it. My company could well offer it for example
(hosted in Geneva with very nice connectivity to almost everywhere).
Setting-up a repository isn't hard. And for a start, I don't think you
really need a buildd network, just amd64 is ok-ish.
> If you know how to start with a new service at
> {volatile,fastpaced,whatever}.debian.net without having to reinvent the
> wheel for acceptign uploads, getting packages built, etc., please
> enlighten me.
Setting-up reprepro, or even Dak, isn't that hard.
Cheers,
Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
On 12/26/18 5:45 PM, Dominik George wrote: > On Wed, Dec 26, 2018 at 03:05:55PM +0100, gregor herrmann wrote: >> And besides that, I think the more universal answer is >> bikesheds/PPAs/you-name-it instead of yet-another-suite. > > Absolutely not. It might be an answer, but to an entirely different > question. This proposal is about providing packages under the same > rules, policies and QA as any other package in Debian, built in the same > trustworthy manner. This is something a PPA does not do. You probably want to read the definition of the Debian Bikesheds as per the proposal of Ganneff. They are *NOT* the same as PPA, and they *DO* include "rules, policies and QA as any other package in Debian". > [...] Debian telling users to add a PPA to > their trusted entities that is managed by some person alone, be they a > DD or not, defeats this entirely. This is *NOT* what the Debian Bikeshed proposal is about. They would *not* be managed "by some person alone". Please do your homework, and read the proposal, especially this one: https://lists.debian.org/debian-devel/2013/05/msg00131.html > No. The dpendencies of gitlab not being accepted into backports right > now is an entirely different issue. I am repeating myself: This proposal > is not intended to ease the life of maintainers whose packages qulify > for -backports. The only difference between -backports and -volatile in > this draft proposal is that -volatile can take packages that are not in > testing due to the exact one reason that hey have a shorter lifespan. To my experience working on the OpenStack packages, upstream lifespan is not a reason good enough. You can still do the work the regular way. Though Debian Bikesheds would be much nicer to deal with the upstream fast pace, because you could have multiple backport repositories. If I had it available, I would create one per upstream release (every 6 months in my case...). Your proposal doesn't address this, which is a major concern for upgrades. > Alexander, please don't get me wrong, but have you read the full > proposal by now and considered it, independent of the gitlab story? I am > pretty certain you did not did that yesterday before starting to object > it - not because of your argumentation, but because reading, > understanding, considering and challenging it and then writing your > reply is simply not physically possible within the 4½ minutes it took > you to object to it ☺. I don't think using this kind of wording will be of any help. Contrary to what you may think, this shows your lack of understanding of Alex's point of view, or your will to consider it, rather than him being stubborn. So, I'll ask for him again, as he must be both busy and (understandably) annoyed by your reply. Why don't *you* consider what Alex told you: have a try with your proposal outside of debian.org (for example on a .net), and see how this works. That's how backports started, and that's probably how your idea should too. > Therefore, I ask you to bring up the points you think are against your > vision of backports. In fact, the proposal is laid out in a way that > explicitly does *not* contradict it, and I am wondering what makes you > think it does, let alone "completely". Let's have a try, no? > I still got the impression you are also confusing me with Praveen, to > the views of whom I do bject as well to some extent (see above). I still have the impression you aren't considering Alex's proposal that you do an attempt external to Debian first, then we see how it goes... How about "fastlane.debian.net" or something? Use that on your own server, and we see what happens, no? > Thus, > please let us discuss this in a well-founded, argumentative manner > instead of just ruling it out from the start. Last time I write it: Alex has *not* ruled it out. Cheers, Thomas Goirand (zigo)
Re: Proposal: Repository for fast-paced package backports
> Basically you would like to build an architecture equivalent to Ubuntu's > PPA No. Please at least try reading the thread first - it was explicitly explained several times that this has absolutely nothing to do with the idea of PPAs. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Hi guys, Trying to make sure I understand clearly this long thread here. Basically you would like to build an architecture equivalent to Ubuntu's PPA that users could add to their apt sources to get newer versions faster instead of dealing with their apt preferences? I personally like the current idea of having a full distro with packages battle-tested with eachother and I'm a bit worried of the added time spent dealing with bugs between a package and a dependency that would be a bit too old, introducing unstable/unsecure behaviors into stable. Maybe another way to do would be to have more frequent full releases of the entire distro to stable? More frequent releases could make the freeze needs shorter as problems wouldn't have as much time to accumulate maybe? We do have more and more packages tested through debci, so making the need of autopkgtest more strict to up to testing could make testing an easier ground for more frequent releases maybe? Thanks, Joseph
Re: Proposal: Repository for fast-paced package backports
Den 27. desember 2018 12:30:33 CET, skrev Holger Levsen : >On Wed, Dec 26, 2018 at 11:37:21PM +0100, Dominik George wrote: >> As long as the package is in -volatile > >FYI: as long as this proposal is been presented with the name -volatile >it's dead to me and saves me from reading mails in this thread. > >Please try to listen if you start a discussion. If you confuse "discussion" with "immediately do what I say before anyone else gets the chance to say something", it's probably better if you ignore it :). I will respect your opinion and incorporate it in the next overhaul, but I have no obligation to "listen", as I am not in a lecture or otherwise under your education. -nik
Re: Proposal: Repository for fast-paced package backports
On Wed, Dec 26, 2018 at 11:37:21PM +0100, Dominik George wrote: > As long as the package is in -volatile FYI: as long as this proposal is been presented with the name -volatile it's dead to me and saves me from reading mails in this thread. Please try to listen if you start a discussion. -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
RE: Proposal: Repository for fast-paced package backports
Hai, A very interesting thread this, since im doing this already for samba, my comments.. If i may .. Im running a samba repo now for jessie and stretch. ( and ubuntu 18.04 ) I really needed newer samba packages and i was not able to get them uploaded to unstable. So i decided to build them myself and share them. And now people are more and more using my samba package over the official debian package. Because the newer version are build against debian stable or oldstable, and people can choose there upgrade. If the might be a fast-lane repo, why not per package version. This way we can keep the changes to other packages small and limited. What i now now do. I have 4 repo's for jessie, jessie-samba45 jessie-samba46 jessie-samba47 jessie-samba48 I have 4 repo's for stretch, stretch-samba46 stretch-samba47 stretch-samba48 stretch-samba49 (And for the ubuntu supporters a samba49 in amd64 only.) Why 4? https://wiki.samba.org/index.php/Samba_Release_Planning Debian version 4.5 (in stable) And the 3 maintanted samba versions. Currely Debian samba is 4.5.12, which is fine, but if you want a more advanced samba, you really need to upgrade. The difference between 4.5.12 and 4.5.16, is major in winbind fixed already. What i also do, at least try to, keep 2 versions of samba in sync, above shows 3 but i need 2 at least. I do this so the OS upgrade wont affect a samba upgrade. Users choose a samba version and stay in that version, untill they dicede to upgrade samba, or get new when new debian stable has a higher release. Im doing this since samba 4.1.x debian Wheezy, and main reason is the fast samba pace and slow debian packages. Not that i mind that, i do love debian and its stability so, keep it slow, yes, but an option for fast moving packages would be nice. So how about something like this. deb http://deb.debian.org/debian fastmove/stretch-samba48 main contrib non-free" And if one want a samba 4.9 that does not exist withing debian, you create the stretch-samba49 line. deb http://deb.debian.org/debian fastmove/stretch-samba49 main contrib non-free" And if the stable version gets thats a higher version then the fastmove, its automaticly picked up again. So if a contibuter wants a higher version, he can build it, and upload it. And the original maintains should get a ping of a higher release version, so if needed they can adopt it to experimental before it goes to unstable. This is a bit what i do for samba ( and debian ) Just my suggest as community helper. Greetz, Louis > -Oorspronkelijk bericht- > Van: Dominik George [mailto:naturesha...@debian.org] > Verzonden: dinsdag 25 december 2018 21:46 > Aan: debian-devel@lists.debian.org; > debian-backpo...@lists.debian.org; debian-rele...@lists.debian.org > Onderwerp: Proposal: Repository for fast-paced package backports > > Heisann, alle sammen, > > as announced in the recent thread about maintaining, I hereby > propose a > repository that allows making “backports” of packages > available to users > of the stable distribution, if those packages cannot be maintained in > testing and backported in the usual way. If you are interested in what > lead up to that, please see bug #915050. I will give a short > summary of > it here. > > > Reasons for having a special place for some packages > > > (You may want to skip this part if you are familiar with the > situation.) > > As all developers know (but passers-by may not), for software to enter > the Debian archive, it is always uploaded to the unstable > distribution, > then migrates to testing (hopefully ;)), which is at some > point snapshot > and made the new stable release. From there on, maintainers have two > obligations: Firstly, keep the package in stable good and secure, e.g. > by uploading security fixes for it once they become available > upstream, > or even backport fixes themselves. Secondly, provide the package in > unstable with updates and ensure its migration, to keep it > ready for the > next stable release. > > Now, for some software packages, this process is problematic, because > upstream may have another idea about software lifecycles. > Concerning the > GitLab example, upstream provides security fixes for three months for > their stable releases. Backporting fixes from newer versions is very > hard or impossible because the massive amounts of changes to the > software in every new versions. This is something that also affects > other packages, like Mozilla Firefox, which has a firefox package in > unstable, and a separate firefox-esr package, with the ESR version of > Firefox. Only the latter migrates to testing. > > Users of Debian honour it for its stability, but as an agile software > lifec
Re: Proposal: Repository for fast-paced package backports
On 12/25/18 3:46 PM, Dominik George wrote: [...] > > Name of the new repository > == > > In the past, the name “volatile” was used for a similar repository, but > with a different scope (limited to data packages for things like virus > scanners). I will thus use the working title volatile throughout this > proposal, although this may change. > > Other ideas: fastlane, unsupported > "rolling" sounds like a good fit [...] > > Building packages and package dependencies > == > > Packages for volatile are built the same way as formal backports, only > that the source is taken from unstable rather than testing. In > particular: > > - Changes shall be kept as small as possible. > - The package is rebuilt against stable. > - The package may depend on packages in stable, stable-backports or > stable-volatile. > stable-rolling sounds more appropriate than stable-volatile. > Dependencies on other packages in volatile should be avoided if > possible. How to handle upgrades from stable to stable+1. Packages from backports upgrade with no issues as stable+1 contains the same packages already compiled for the stable+1. How about LTS? As stable-rolling repository would be usable in conjunction with stable-backports and stable, would then oldstable-rolling continue to roll or just freeze in place at the moment when the stable becomes oldstable? [...] > > > Implications for the situation at hand (gitlab) > === > > As there were quite a few concerns raised (some of which I share, and > some I don’t): Of course, if a software intended for volatile has a ton > of dependencies (intended to go into backports), all backports rules and > powers of the ftp-masters apply. Repeating myself: volatile is not meant > to ease the life of maintainers. > Continuous delivery development model based upstream applications are not quite a good fit for a stable release distribution. Milan signature.asc Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
Hi, > How to handle upgrades from stable to stable+1. Packages from backports > upgrade with no issues as stable+1 contains the same packages already > compiled for the stable+1. As long as the package is in -volatile, it is not in stable+1, and upgrades are ensured by the volatile maintainer. If the package is to go into stable+1 again, ist must move to -backports (see original proposal for details on that). > How about LTS? As stable-rolling repository would be usable in > conjunction with stable-backports and stable, would then > oldstable-rolling continue to roll or just freeze in place at the moment > when the stable becomes oldstable? I think oldstable-volatile could keep rolling if the maintainer wishes to do so, but must never be newer than stable-volatile, of course. Upgrades between oldstable-volatile and stable-volatile must be ensured by the maintainer. > Continuous delivery development model based upstream applications are > not quite a good fit for a stable release distribution. Maybe that's why we are drafting a mechanism to support them outside the stable release distribution ;). -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
> For backports the general supportability assumption is that you provide a > sane upgrade path from stable to the backports and from the backport to the > next stable (optimally the same package). Once you take the presence of the > stable package out of the mix, it becomes weird. How long do you need to > preserve compatibility code? How does an agile package that does not fit > Debian's release cycle cater to these requirements? This is wrong, at least in a big part. The stable release cycle does not apply for -backports. A package in -backports can be udpated an arbitrary number of times during the development window of stable+1, leaving us with the exact same issue as you are describing for -volatile. If you take into account edge cases, where a package is removed from testing during the freeze, is removed from -backports as a result, is not released with stable+1, then gets fixed and reintroduced to testing and -backports, it gets even closer. In short: This is a situation every maintainer has to take measures for, be it in -backports or -volatile. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On 26/12/2018 19:31, Dominik George wrote: - The package must not be in testing, and care must be taken for the package not to migrate to testing. So what would a user of testing do? Will there be a $codename-volatile[1] suite for testing users? Or would they directly install unstable with no other pre-release staging ground? (Which seems like a bad idea.) The testing distribution and this concept contradict each other. The testing distribution is the least supported stage in the Debian release cycle, and if used, shall only be used for testing the next Debian release. Especially, there are no timely security updates in testing, neither by the security nor by the maintainer. So using it for anything other than testing the next Debian release in a laboratory is a bad idea. This proposal, however, is about providing fast-paced software as an exception on an otherwise stable system. So if anyone wants to test fast-paced software within a Debian testing system, they should use the package from unstable, yes. (Having testing on a system without the sid zources is close to impossible outside the freeze anyway.) Of course you can use testing without having sid available, that's the point of testing[1]. I think there's a misunderstand about testing lurking here, too. And the requirement of backports for the package to be in testing is also to get a sane upgrade path from stable to next-stable. So you have to support your thing alongside testing, too. At least during the freeze, but optimally shortly after the release has been cut. Similarly what are the constraints you set for upgrading, if any? How far back will upgrades work and how current do users need to keep their system in order to still be able to upgrade? For one, I think you will need to set expectations here towards the maintainers if a package is never included in a stable release, as they get very muddy otherwise. Plus you need to set expectations for the users as the next package (maybe not gitlab) might come up with requirements that upgrades need to go through every version on the way to, say, update the database properly. And that's hardly supportable unless everyone knows to update quickly. That certainly is something maintainers need to care about. (I consider a software not supporting skipping versions on upgrade a bug, anyway.) But most importantly, this is nothing that is specific to the -volatile proposal - it is the same for regular backports. So discussing this general backporting issue should not be mixed with the discussion of this proposal. For backports the general supportability assumption is that you provide a sane upgrade path from stable to the backports and from the backport to the next stable (optimally the same package). Once you take the presence of the stable package out of the mix, it becomes weird. How long do you need to preserve compatibility code? How does an agile package that does not fit Debian's release cycle cater to these requirements? Just discounting that on the grounds of "that's normal for backporting" if it's unique to your proposal is not quite satisfactory to me. Kind regards Philipp Kern [1] You can make the argument that there's a problem with security update. But that's why the urgency system exists and maintainers can declare that a certain package needs to migrate with minimal waiting time. And most of the time (not always) the exploits start to materialize later.
Re: Proposal: Repository for fast-paced package backports
> - Should the package begin to migrate to testing again, it must >be moved to stable-backports. > > - Using the same ~bpo version namespace Both of these poitns are there to *not* change anything about backports. If a package stops qualifying for -volatile, and starts qualifying for -backports, it's under the backports realm again. I consider this very important so it is very clear for maintainers what -volatile is for - in particular, *not* for bypassing -backports limitations. The sharing of the version namespace is partially a direct consequence of the previous point. > - "treat it as part of backports", which I assume means that >backports users would automatically consume this repo No. I see where the misunderstanding comes from - that's not what I was intending to say. -colatile is intended to be a compelte separate suite, that users can add to their sources.list separately (if they do, they also need to add the regular -backports, however). The rest of what I meant as "treat as part of" is adhering to the same rules, standards, etc., and re-using existing infrastructure like the NEW queue due to that. Also to ensure that the qualification of packages for either -backports or -volatile is clear and inforced. > > - new binary uploads to volatile have to undergo the >same NEW queue as backports This as about sharing resources and enforcing the same rules (except for source and target suites). The proposal is still possible without sharing the same NEW queue, but the first two points are a major concept ensuring that it will work. It will not work as well when removing them. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Dominik George wrote:
> Jonathan Nieder wrote:
>> 2. I am happy with the current charter of backports and I think it's
>> possible to move forward with fastpaced without having to change
>> that charter.
>
> Yep. That's exactly why the proposal changes nothing about -backports. I
> am still confused why Alex and you keep insisting that anything would be
> changing there.
It has a few points of intersection:
- Should the package begin to migrate to testing again, it must
be moved to stable-backports.
- Using the same ~bpo version namespace
- "treat it as part of backports", which I assume means that
backports users would automatically consume this repo
- new binary uploads to volatile have to undergo the
same NEW queue as backports
I don't think these are deep, inherent things (it's possible to
preserve the spirit of the proposal while removing them), but please
don't accuse me of pulling them out of thin air.
[...]
>> 3. formerer is speaking from experience when he says that it's
>> possible to make this kind of change unofficially first, learn
>> from it, and thus set the groundwork for making it official.
>>
>> If you foresee obstacles to that, can you say more about where
>> they lie? Maybe we can help address them, or maybe we can find
>> another way forward.
>>
>> If you don't see obstacles, why not start today?
>
> I think I already made those obstacles clear: Starting outside means
> buying, installing and operating at least a server vor
> volatile.debian.net (or whatever you call it), setting up and
> maintaining an upload queue, the queued, and everything around it,
> building from source for at least the most important architectures on
> hardware that needs to be there and maintained for that, etc.
Thanks. That points to who you may want to get help from:
- DSA, for hosting
- ftpmasters, in case you'd share their DAK instance
- porters, to find out what level of port + buildd support they want
to maintain
[...]
> - I do not sure I can do it right, because I do not know all the
>technical details.
That's fine. There's no time like the present to learn!
> Thus, because the change as it is proposed has such a low impact on
> anything else, I consider doing all that over again unnecessary.
>
> Don't get me wrong - I would not hesitate to go through it if it were
> for anything that could break things, or make life harder for others, or
> something like that.
I think you're underestimating the impact on other teams. That's
fine: it's probably worth it, but you will need to get buy in.
[...]
> If you know how to start with a new service at
> {volatile,fastpaced,whatever}.debian.net without having to reinvent the
> wheel for acceptign uploads, getting packages built, etc., please
> enlighten me.
backports maintainers, debian-ports maintainers, and others may have
experience with this. I don't know the best place to get advice from
them --- you may already be in the right place. :)
Sincerely,
Jonathan
Re: Proposal: Repository for fast-paced package backports
> > - The package must not be in testing, and care must be taken for the > > package not to migrate to testing. > So what would a user of testing do? Will there be a $codename-volatile[1] > suite for testing users? Or would they directly install unstable with no > other pre-release staging ground? (Which seems like a bad idea.) The testing distribution and this concept contradict each other. The testing distribution is the least supported stage in the Debian release cycle, and if used, shall only be used for testing the next Debian release. Especially, there are no timely security updates in testing, neither by the security nor by the maintainer. So using it for anything other than testing the next Debian release in a laboratory is a bad idea. This proposal, however, is about providing fast-paced software as an exception on an otherwise stable system. So if anyone wants to test fast-paced software within a Debian testing system, they should use the package from unstable, yes. (Having testing on a system without the sid zources is close to impossible outside the freeze anyway.) > > Similarly what are the constraints you set for upgrading, if any? How far > back will upgrades work and how current do users need to keep their system > in order to still be able to upgrade? For one, I think you will need to set > expectations here towards the maintainers if a package is never included in > a stable release, as they get very muddy otherwise. Plus you need to set > expectations for the users as the next package (maybe not gitlab) might come > up with requirements that upgrades need to go through every version on the > way to, say, update the database properly. And that's hardly supportable > unless everyone knows to update quickly. That certainly is something maintainers need to care about. (I consider a software not supporting skipping versions on upgrade a bug, anyway.) But most importantly, this is nothing that is specific to the -volatile proposal - it is the same for regular backports. So discussing this general backporting issue should not be mixed with the discussion of this proposal. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Hi,
> 2. I am happy with the current charter of backports and I think it's
> possible to move forward with fastpaced without having to change
> that charter.
Yep. That's exactly why the proposal changes nothing about -backports. I
am still confused why Alex and you keep insisting that anything would be
changing there.
> 3. formerer is speaking from experience when he says that it's
> possible to make this kind of change unofficially first, learn
> from it, and thus set the groundwork for making it official.
>
> If you foresee obstacles to that, can you say more about where
> they lie? Maybe we can help address them, or maybe we can find
> another way forward.
>
> If you don't see obstacles, why not start today?
I think I already made those obstacles clear: Starting outside means
buying, installing and operating at least a server vor
volatile.debian.net (or whatever you call it), setting up and
maintaining an upload queue, the queued, and everything around it,
building from source for at least the most important architectures on
hardware that needs to be there and maintained for that, etc. There are
several issues with that:
- It costs a lot time that could better be used elsewhere.
- It costs extra money, which I for one do not have to spare.
- I do not sure I can do it right, because I do not know all the
technical details.
Thus, because the change as it is proposed has such a low impact on
anything else, I consider doing all that over again unnecessary.
Don't get me wrong - I would not hesitate to go through it if it were
for anything that could break things, or make life harder for others, or
something like that. I am just putting the impact of the change and the
resources needed for seperate infrastructure in relation. Everything
about this proposal ahs already been tested when -backports was young
(thanks for doing the work!). This proposal contains nothing new to
learn, neither technically nor policy-wise. It works the same way
backports do, with the same considerations, except for the source and
target suites of the packages.
If you know how to start with a new service at
{volatile,fastpaced,whatever}.debian.net without having to reinvent the
wheel for acceptign uploads, getting packages built, etc., please
enlighten me.
-nik
signature.asc
Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On 25/12/2018 21:46, Dominik George wrote: Requirements for a package to go into stable-volatile = The new volatile proposal is not intended to ease life for package maintainers who want to bypass the migration and QA requirements of the regular stable lifecycle, so special need must be taken to ensure only packages that need it go into volatile. I want to summarise the requirements like so: - The package must be maintained in unstable, like every other package. - The package must not be in testing, and care must be taken for the package not to migrate to testing. So what would a user of testing do? Will there be a $codename-volatile[1] suite for testing users? Or would they directly install unstable with no other pre-release staging ground? (Which seems like a bad idea.) Similarly what are the constraints you set for upgrading, if any? How far back will upgrades work and how current do users need to keep their system in order to still be able to upgrade? For one, I think you will need to set expectations here towards the maintainers if a package is never included in a stable release, as they get very muddy otherwise. Plus you need to set expectations for the users as the next package (maybe not gitlab) might come up with requirements that upgrades need to go through every version on the way to, say, update the database properly. And that's hardly supportable unless everyone knows to update quickly. Kind regards Philipp Kern [1] I would like to re-register my objection to that name for the same reason Holger stated: it is confusing to reuse an older name (which, by the way, started outside of Debian, too and was then merged in) with a new concept.
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018, Dominik George wrote: > > >If there are other issues to solve than the lifespan of the package > > >version, they must be solved in another way. > > > > I agree with you, it is the best outcome. But when people with power > > (-backports ftp masters) are not willing to consider it, we have to go > > with plan B, which is less than ideal, but can move things forward. > > Plan B in this case are PPAs. If you want to engage in that idea, please > do separately from the -volatile idea. > > > >> As I said, gitlab was not about manpower. This new repo is completly > > >against > > >> our vision of what backports is. Therefore we don't want it within > > >the > > >> backports suite. > > > > > > If people argue both ways, how can we answer? Either it adds more work > > for -backports team or it does not. Some people say its not fair to > > add more load while ftp masters say its not about load. > > As Alex laid out, it's mostly just the -backports team handling the NEW > queue. So all of this really is independent from -backports, if another > NEW queue is added (which I do not think is the best idea, but still > possible). > > But, I do not think it is possible to start -volatile completely > independently. I am pretty certain there is enough man power to handle > it as a new suite, but on the other hand I am also certain there is not > enough manpower to operate a compelte set of seperate services for it. as said, we are also guests on the ftpmaster services. They are the people to ask. The NEW queue is just a minor detail of a suite. Alex signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Hi, Pirate Praveen wrote: > I agree with you, it is the best outcome. But when people with power > (-backports ftp masters) are not willing to consider it, we have to > go with plan B, which is less than ideal, but can move things > forward. Just to avoid this being thought of as an idiosyncrasy of backports ftpmasters, I suppose I should put my own views forward. 1. Nik, I think you're onto something with this fastpaced proposal. I would be happy to see some suite to make it easier for users to consume packages that lack long-term support, like non-ESR firefox. 2. I am happy with the current charter of backports and I think it's possible to move forward with fastpaced without having to change that charter. 3. formerer is speaking from experience when he says that it's possible to make this kind of change unofficially first, learn from it, and thus set the groundwork for making it official. If you foresee obstacles to that, can you say more about where they lie? Maybe we can help address them, or maybe we can find another way forward. If you don't see obstacles, why not start today? 4. Just to reiterate about (2), just like I think it's completely reasonable for release team to exercise discretion about what goes in stable and testing, it's completely reasonable for backports team to exercise discretion about what goes into backports. Please don't take it personally. It's an important part of what they do to make the suite sustainable, and I am very grateful for it. Thanks and hope that helps, Jonathan
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018, Pirate Praveen wrote: > > > On 2018, ഡിസംബർ 26 10:15:35 PM IST, Dominik George > wrote: > >No. The dpendencies of gitlab not being accepted into backports right > >now is an entirely different issue. I am repeating myself: This > >proposal > >is not intended to ease the life of maintainers whose packages qulify > >for -backports. The only difference between -backports and -volatile in > >this draft proposal is that -volatile can take packages that are not in > >testing due to the exact one reason that hey have a shorter lifespan. > >No > >single other thing qualifies a package for -volatile if it is not > >qualified for -backports. > > > >If there are other issues to solve than the lifespan of the package > >version, they must be solved in another way. > > I agree with you, it is the best outcome. But when people with power > (-backports ftp masters) are not willing to consider it, we have to go with > plan B, which is less than ideal, but can move things forward. > > >On Wed, Dec 26, 2018 at 04:32:28PM +0100, Alexander Wirt wrote: > >> As I said, gitlab was not about manpower. This new repo is completly > >against > >> our vision of what backports is. Therefore we don't want it within > >the > >> backports suite. > > > > If people argue both ways, how can we answer? Either it adds more work for > -backports team or it does not. Some people say its not fair to add more load > while ftp masters say its not about load. > > If it does not add extra load, then I don't see why it can be kept out of > -backports when it qualifies except for being a dependency of -volatile. They > say it does not match with their vision. So what option is left for us? We > have to take their word for it and move forward without inconveniencing them. > I don't think -backports ftp masters is going to accept this proposal (from > the responses we already received), so I can live with all dependencies in > -volatile. Jftr, we never said anything about dependencies. If it qualifies for backports, I don't see any reason for not having those deps in backports. We never questioned that topic. If it qualifies: great. If not, please don't upload it. A package qualifies for backports if: - it adds new features - wasn't available before - is in testing - will receive support for the lifetime of the backports suite so things like npm are perfectly fine for backports. And we never said anything for libs, even if they are "standalone" (nothing in backports is using those libs). The main problem with gitlab was: from our perspective several hundred packages only had one uploader (to backports, other suites don't count) and we wanted to minimize the risk of being alone with hundreds of unmaintained backports. We solved that problem (by getting more people responsible for the backports) and accepted it. Alex
Re: Proposal: Repository for fast-paced package backports
> >If there are other issues to solve than the lifespan of the package > >version, they must be solved in another way. > > I agree with you, it is the best outcome. But when people with power > (-backports ftp masters) are not willing to consider it, we have to go > with plan B, which is less than ideal, but can move things forward. Plan B in this case are PPAs. If you want to engage in that idea, please do separately from the -volatile idea. > >> As I said, gitlab was not about manpower. This new repo is completly > >against > >> our vision of what backports is. Therefore we don't want it within > >the > >> backports suite. > > > If people argue both ways, how can we answer? Either it adds more work > for -backports team or it does not. Some people say its not fair to > add more load while ftp masters say its not about load. As Alex laid out, it's mostly just the -backports team handling the NEW queue. So all of this really is independent from -backports, if another NEW queue is added (which I do not think is the best idea, but still possible). But, I do not think it is possible to start -volatile completely independently. I am pretty certain there is enough man power to handle it as a new suite, but on the other hand I am also certain there is not enough manpower to operate a compelte set of seperate services for it. In any case, I propose we stop discussing the who and where questions for a while and concentrate on the what and how. I will collect the opinions on that, and in a week or two, incorporate them into the proposal, along with the different possibilities for implementation. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On 2018, ഡിസംബർ 26 10:15:35 PM IST, Dominik George wrote: >No. The dpendencies of gitlab not being accepted into backports right >now is an entirely different issue. I am repeating myself: This >proposal >is not intended to ease the life of maintainers whose packages qulify >for -backports. The only difference between -backports and -volatile in >this draft proposal is that -volatile can take packages that are not in >testing due to the exact one reason that hey have a shorter lifespan. >No >single other thing qualifies a package for -volatile if it is not >qualified for -backports. > >If there are other issues to solve than the lifespan of the package >version, they must be solved in another way. I agree with you, it is the best outcome. But when people with power (-backports ftp masters) are not willing to consider it, we have to go with plan B, which is less than ideal, but can move things forward. >On Wed, Dec 26, 2018 at 04:32:28PM +0100, Alexander Wirt wrote: >> As I said, gitlab was not about manpower. This new repo is completly >against >> our vision of what backports is. Therefore we don't want it within >the >> backports suite. > If people argue both ways, how can we answer? Either it adds more work for -backports team or it does not. Some people say its not fair to add more load while ftp masters say its not about load. If it does not add extra load, then I don't see why it can be kept out of -backports when it qualifies except for being a dependency of -volatile. They say it does not match with their vision. So what option is left for us? We have to take their word for it and move forward without inconveniencing them. I don't think -backports ftp masters is going to accept this proposal (from the responses we already received), so I can live with all dependencies in -volatile. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018, Dominik George wrote: > > I don't want backports to contain things are are not suited for a > > release. > > That's why we are doing all this. It is NOT about anything to backports. > It is about adding something new that uses the same RULES as backports, > with a slight diversion, and thus can also make use of infrastructure > already there for backports. Neither being economic with manpower and > machines nor trying to be a good neighbour by adhering to the same rules > means to change or add anything to -backports. And I said I think it should start independently to prove its worth the work. And it isn't backports infrastructure. It is ftpmaster's, we are just users. We can't even add anything, even if we want to. The only things we control are the NEW queue of the backports-suites ( a new suite with have its own queue) and the website (which also doesn't fit for the new suite). So imho addressing this as a backports topic is just wrong. Alex signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
> I don't want backports to contain things are are not suited for a > release. That's why we are doing all this. It is NOT about anything to backports. It is about adding something new that uses the same RULES as backports, with a slight diversion, and thus can also make use of infrastructure already there for backports. Neither being economic with manpower and machines nor trying to be a good neighbour by adhering to the same rules means to change or add anything to -backports. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018, Dominik George wrote: > Hi, > > On Wed, Dec 26, 2018 at 03:05:55PM +0100, gregor herrmann wrote: > > (Can we keep this on one mailing list, please? /me restricts this to > > -devel) > > No. This has the potential of keeping people who are directly impacted > by this proposal out of the loop. > > > And besides that, I think the more universal answer is > > bikesheds/PPAs/you-name-it instead of yet-another-suite. > > Absolutely not. It might be an answer, but to an entirely different > question. This proposal is about providing packages under the same > rules, policies and QA as any other package in Debian, built in the same > trustworthy manner. This is something a PPA does not do. > > To stay with the gitlab example: I would very much like to see some > people (including the company I work at, two organisations I am > otherwise involved with,…) use packages from Debian. This is mostly > about trust - it is a very useful policy to limit the entities to trust > for software distribution if you run production systems, especially when > they handle third-party data. Debian is such an entity - while there are > many people working in it, it is a body with defined procedures and > standards that can be relied upon. Debian telling users to add a PPA to > their trusted entities that is managed by some person alone, be they a > DD or not, defeats this entirely. > > On Wed, Dec 26, 2018 at 08:29:17PM +0530, Pirate Praveen wrote: > > The -backports team does not want the dependencies of gitlab to be in > > -backports even though it meets the criteria for backports. So we will > > end up adding it to volatile. Now if some one else wants the same in > > -backports, they will have to repeat the process. > > > > Take nodejs or npm for example, which I backported now. In buster the > > -backports team does not want it in backports if I'm doing it for > > gitlab, even though they satisfy the requirement for -backports. So we > > will end up uploading these to volatile, if someone else wants it in > > -backports, they will have to do it again. > > > > It is one way (volatile can use -backports, but -backports can't use > > volatile). I'm fine with that if people don't want our work for volatile > > not added to -backports. > > > > Dominik, > > > > I think we can go ahead with volatile as separate suite and take > > packages from -backports if exist but add all new dependencies to -volatile. > > > > This, > > > > "Dependencies on other packages in volatile should be avoided if > > possible. Especially, dependencies of the package that also need > > backporting must not be added to volatile just because they are > > dependencies — every dependency that is needed to be backported to > > support the volatile package must be considered on its own and in all > > but unprobable edge cases be maintained as a formal backport. Obviously, > > the unprobable edge case occurs when the package depends on another > > package that also fully qualifies for volatile, as described above." > > > > should be changed to, > > > > "Dependencies of the package that also need backporting must be added to > > volatile." > > No. The dpendencies of gitlab not being accepted into backports right > now is an entirely different issue. I am repeating myself: This proposal > is not intended to ease the life of maintainers whose packages qulify > for -backports. The only difference between -backports and -volatile in > this draft proposal is that -volatile can take packages that are not in > testing due to the exact one reason that hey have a shorter lifespan. No > single other thing qualifies a package for -volatile if it is not > qualified for -backports. And this is also solved. I emptied the NEW queue two or three days ago. If there are dependencies missing the backports wasn't tested, which sucks. > If there are other issues to solve than the lifespan of the package > version, they must be solved in another way. > > On Wed, Dec 26, 2018 at 04:32:28PM +0100, Alexander Wirt wrote: > > As I said, gitlab was not about manpower. This new repo is completly against > > our vision of what backports is. Therefore we don't want it within the > > backports suite. > > Alexander, please don't get me wrong, but have you read the full > proposal by now and considered it, independent of the gitlab story? I am > pretty certain you did not did that yesterday before starting to object > it - not because of your argumentation, but because reading, > understanding, considering and challenging it and then writing your > reply is simply not physically possible within the 4½ minutes it took > you to object to it ☺. Yes. Nothing changed til then. > Therefore, I ask you to bring up the points you think are against your > vision of backports. In fact, the proposal is laid out in a way that > explicitly does *not* contradict it, and I am wondering what makes you > think it does, let alone "completely". > > I still got the impression
Re: Proposal: Repository for fast-paced package backports
Hi, On Wed, Dec 26, 2018 at 03:05:55PM +0100, gregor herrmann wrote: > (Can we keep this on one mailing list, please? /me restricts this to > -devel) No. This has the potential of keeping people who are directly impacted by this proposal out of the loop. > And besides that, I think the more universal answer is > bikesheds/PPAs/you-name-it instead of yet-another-suite. Absolutely not. It might be an answer, but to an entirely different question. This proposal is about providing packages under the same rules, policies and QA as any other package in Debian, built in the same trustworthy manner. This is something a PPA does not do. To stay with the gitlab example: I would very much like to see some people (including the company I work at, two organisations I am otherwise involved with,…) use packages from Debian. This is mostly about trust - it is a very useful policy to limit the entities to trust for software distribution if you run production systems, especially when they handle third-party data. Debian is such an entity - while there are many people working in it, it is a body with defined procedures and standards that can be relied upon. Debian telling users to add a PPA to their trusted entities that is managed by some person alone, be they a DD or not, defeats this entirely. On Wed, Dec 26, 2018 at 08:29:17PM +0530, Pirate Praveen wrote: > The -backports team does not want the dependencies of gitlab to be in > -backports even though it meets the criteria for backports. So we will > end up adding it to volatile. Now if some one else wants the same in > -backports, they will have to repeat the process. > > Take nodejs or npm for example, which I backported now. In buster the > -backports team does not want it in backports if I'm doing it for > gitlab, even though they satisfy the requirement for -backports. So we > will end up uploading these to volatile, if someone else wants it in > -backports, they will have to do it again. > > It is one way (volatile can use -backports, but -backports can't use > volatile). I'm fine with that if people don't want our work for volatile > not added to -backports. > > Dominik, > > I think we can go ahead with volatile as separate suite and take > packages from -backports if exist but add all new dependencies to -volatile. > > This, > > "Dependencies on other packages in volatile should be avoided if > possible. Especially, dependencies of the package that also need > backporting must not be added to volatile just because they are > dependencies — every dependency that is needed to be backported to > support the volatile package must be considered on its own and in all > but unprobable edge cases be maintained as a formal backport. Obviously, > the unprobable edge case occurs when the package depends on another > package that also fully qualifies for volatile, as described above." > > should be changed to, > > "Dependencies of the package that also need backporting must be added to > volatile." No. The dpendencies of gitlab not being accepted into backports right now is an entirely different issue. I am repeating myself: This proposal is not intended to ease the life of maintainers whose packages qulify for -backports. The only difference between -backports and -volatile in this draft proposal is that -volatile can take packages that are not in testing due to the exact one reason that hey have a shorter lifespan. No single other thing qualifies a package for -volatile if it is not qualified for -backports. If there are other issues to solve than the lifespan of the package version, they must be solved in another way. On Wed, Dec 26, 2018 at 04:32:28PM +0100, Alexander Wirt wrote: > As I said, gitlab was not about manpower. This new repo is completly against > our vision of what backports is. Therefore we don't want it within the > backports suite. Alexander, please don't get me wrong, but have you read the full proposal by now and considered it, independent of the gitlab story? I am pretty certain you did not did that yesterday before starting to object it - not because of your argumentation, but because reading, understanding, considering and challenging it and then writing your reply is simply not physically possible within the 4½ minutes it took you to object to it ☺. Therefore, I ask you to bring up the points you think are against your vision of backports. In fact, the proposal is laid out in a way that explicitly does *not* contradict it, and I am wondering what makes you think it does, let alone "completely". I still got the impression you are also confusing me with Praveen, to the views of whom I do bject as well to some extent (see above). So, this proposal is about extending -backports, but without getting in its way, and following all its ideas except for the source suite. Thus, please let us discuss this in a well-founded, argumentative manner instead of just ruling it out from the start. Thanks, Nik signature.asc Description: PGP
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018, Pirate Praveen wrote: > > > On 2018, ഡിസംബർ 26 2:16:07 AM IST, Dominik George > wrote: > >Heisann, alle sammen, > > > >as announced in the recent thread about maintaining, I hereby propose a > >repository that allows making “backports” of packages available to > >users > >of the stable distribution, if those packages cannot be maintained in > >testing and backported in the usual way. If you are interested in what > >lead up to that, please see bug #915050. I will give a short summary of > >it here. > > > > > >Reasons for having a special place for some packages > > > > > >(You may want to skip this part if you are familiar with the > >situation.) > > > >As all developers know (but passers-by may not), for software to enter > >the Debian archive, it is always uploaded to the unstable distribution, > >then migrates to testing (hopefully ;)), which is at some point > >snapshot > >and made the new stable release. From there on, maintainers have two > >obligations: Firstly, keep the package in stable good and secure, e.g. > >by uploading security fixes for it once they become available upstream, > >or even backport fixes themselves. Secondly, provide the package in > >unstable with updates and ensure its migration, to keep it ready for > >the > >next stable release. > > > >Now, for some software packages, this process is problematic, because > >upstream may have another idea about software lifecycles. Concerning > >the > >GitLab example, upstream provides security fixes for three months for > >their stable releases. Backporting fixes from newer versions is very > >hard or impossible because the massive amounts of changes to the > >software in every new versions. This is something that also affects > >other packages, like Mozilla Firefox, which has a firefox package in > >unstable, and a separate firefox-esr package, with the ESR version of > >Firefox. Only the latter migrates to testing. > > > >Users of Debian honour it for its stability, but as an agile software > >lifecycle is adapted by more and more very popular software packages, > >not being able to install these packages in the trusted, well-known > >fashion through the official apt repositories is becoming more and more > >of a drawback. > > > >It can easily be assumed that the normal release and maintenance cycle > >of Debian stable will not change, which is very good, so we should find > >a way to still provide such software as described above to users. > > > > > >Why backports is not enough > >=== > > > >This also is well-known, but for completeness: Formal backports in > >stable-backports are required to be direct backports from testing, and > >are a stepping stone within the upgrade from stable to stable+1. Thus, > >a > >version of a package that is not in testing can never be in > >stable-backports. > > > > > >Name of the new repository > >== > > > >In the past, the name “volatile” was used for a similar repository, but > >with a different scope (limited to data packages for things like virus > >scanners). I will thus use the working title volatile throughout this > >proposal, although this may change. > > > >Other ideas: fastlane, unsupported > > > >(Please feel free to add other ideas.) > > > > > >Requirements for a package to go into stable-volatile > >= > > > >The new volatile proposal is not intended to ease life for package > >maintainers who want to bypass the migration and QA requirements of the > >regular stable lifecycle, so special need must be taken to ensure only > >packages that need it go into volatile. I want to summarise the > >requirements like so: > > > >- The package must be maintained in unstable, like every other package. > > - The package must not be in testing, and care must be taken for the > > package not to migrate to testing. > > - Regular maintenance for the lifetime of stable must be impossible > > or unnecessarily hard, and this requirement should be assessed in > > a verifiable manner, e.g. referring to upstream’s lifecycle model. > > - There must be notable need for the package. Like for backports, user > > requests might be an indicator. > > - Should the package be removed from unstable, it must also be removed > > from volatile. > > - Should the package begin to migrate to testing again, it must > > be moved to stable-backports. > > > >Before starting to maintain a volatile package, the maintainer shall > >seek consent (or doubt) on debian-devel. > > > > > >Building packages and package dependencies > >== > > > >Packages for volatile are built the same way as formal backports, only > >that the source is taken from unstable rather than testing. In > >particular: > > > > - Changes shall be kept as small as possible. > > - The package is rebuilt against stable. > >- The package may depend
Re: Proposal: Repository for fast-paced package backports
On Wed, Dec 26, 2018 at 02:35:19PM +0100, Dominik George wrote: > >volatile is a very bad name for this because we've used it already for > >something else. > Well, I consider it more or less the same basic idea. The old and new ideas > have more in common than not, with the only difference being that previously, > volatile packages also had versions in stable. that *you* understand this naming was out of the question and is besides my point. (and this is absolutly not ment in any hostile way, just stating a fact.) -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
[As requested, keeping it to -devel only] On 12/26/18 7:35 PM, Antonio Terceiro wrote: > On Wed, Dec 26, 2018 at 01:04:44PM +0530, Pirate Praveen wrote: >> If it has to be completely separate from -backports, it means some packages >> will need to be maintained twice, even when they meet the criteria for >> backports fully, just because a package in volatile declare a dependency on >> them. > > There is nothing that stops you, or whoever wants to maintain this newn > repository from doing it in a way that 1) reuses what's already in > backports, even automatically and 2) adds the bits that are not deemed > appropriate for backports. > The -backports team does not want the dependencies of gitlab to be in -backports even though it meets the criteria for backports. So we will end up adding it to volatile. Now if some one else wants the same in -backports, they will have to repeat the process. Take nodejs or npm for example, which I backported now. In buster the -backports team does not want it in backports if I'm doing it for gitlab, even though they satisfy the requirement for -backports. So we will end up uploading these to volatile, if someone else wants it in -backports, they will have to do it again. It is one way (volatile can use -backports, but -backports can't use volatile). I'm fine with that if people don't want our work for volatile not added to -backports. Dominik, I think we can go ahead with volatile as separate suite and take packages from -backports if exist but add all new dependencies to -volatile. This, "Dependencies on other packages in volatile should be avoided if possible. Especially, dependencies of the package that also need backporting must not be added to volatile just because they are dependencies — every dependency that is needed to be backported to support the volatile package must be considered on its own and in all but unprobable edge cases be maintained as a formal backport. Obviously, the unprobable edge case occurs when the package depends on another package that also fully qualifies for volatile, as described above." should be changed to, "Dependencies of the package that also need backporting must be added to volatile." signature.asc Description: OpenPGP digital signature
Re: Proposal: Repository for fast-paced package backports
On 2018-12-26 15:05, gregor herrmann wrote: > (Can we keep this on one mailing list, please? /me restricts this to > -devel) Agreed. > And besides that, I think the more universal answer is > bikesheds/PPAs/you-name-it instead of yet-another-suite. IMHO, this is not the same. Both "volatile/fastlane/whatever" and "PPAs/bikeshed" have legitimate use cases.
Re: Proposal: Repository for fast-paced package backports
On Wed, 26 Dec 2018 13:07:07 +, Holger Levsen wrote: (Can we keep this on one mailing list, please? /me restricts this to -devel) > On Wed, Dec 26, 2018 at 12:07:42AM +0100, Dominik George wrote: > > I actually think volatile is a good name. After all, it's not so far from > > the previous volatile. > volatile is a very bad name for this because we've used it already for > something else. Agreed. And besides that, I think the more universal answer is bikesheds/PPAs/you-name-it instead of yet-another-suite. IIRC, there's even a draft specification … What I found now is https://lists.debian.org/debian-devel/2013/05/msg00131.html [0] https://lists.debian.org/debian-dak/2015/09/msg00023.html ff. Since then bikesheds keep being mentioned every now and then for handling fast-moving software; e.g. https://lists.debian.org/debian-devel/2016/01/msg00696.html https://lists.debian.org/debian-devel/2016/03/msg00363.html https://lists.debian.org/debian-devel/2016/04/msg00059.html https://lists.debian.org/debian-devel/2018/10/msg00072.html Cheers, gregor [0] "Aggressive Backport: This is the "dotdeb.org" scenario. For whatever reason, people need whatever the latest version of php/mysql/apache/nginx/selectyourpoison is, compiled for (old)stable, in order to run on their otherwise stable servers." -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: Rolling Stones: Moonlight-mile-live signature.asc Description: Digital Signature
Re: Proposal: Repository for fast-paced package backports
On Wed, Dec 26, 2018 at 01:04:44PM +0530, Pirate Praveen wrote: > If it has to be completely separate from -backports, it means some packages > will need to be maintained twice, even when they meet the criteria for > backports fully, just because a package in volatile declare a dependency on > them. There is nothing that stops you, or whoever wants to maintain this newn repository from doing it in a way that 1) reuses what's already in backports, even automatically and 2) adds the bits that are not deemed appropriate for backports. signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
>> I actually think volatile is a good name. After all, it's not so far >from the previous volatile. > >volatile is a very bad name for this because we've used it already for >something else. Well, I consider it more or less the same basic idea. The old and new ideas have more in common than not, with the only difference being that previously, volatile packages also had versions in stable. -nik
Re: Proposal: Repository for fast-paced package backports
On Wed, Dec 26, 2018 at 12:07:42AM +0100, Dominik George wrote: > I actually think volatile is a good name. After all, it's not so far from the > previous volatile. volatile is a very bad name for this because we've used it already for something else. -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On 2018, ഡിസംബർ 26 2:16:07 AM IST, Dominik George wrote: >Heisann, alle sammen, > >as announced in the recent thread about maintaining, I hereby propose a >repository that allows making “backports” of packages available to >users >of the stable distribution, if those packages cannot be maintained in >testing and backported in the usual way. If you are interested in what >lead up to that, please see bug #915050. I will give a short summary of >it here. > > >Reasons for having a special place for some packages > > >(You may want to skip this part if you are familiar with the >situation.) > >As all developers know (but passers-by may not), for software to enter >the Debian archive, it is always uploaded to the unstable distribution, >then migrates to testing (hopefully ;)), which is at some point >snapshot >and made the new stable release. From there on, maintainers have two >obligations: Firstly, keep the package in stable good and secure, e.g. >by uploading security fixes for it once they become available upstream, >or even backport fixes themselves. Secondly, provide the package in >unstable with updates and ensure its migration, to keep it ready for >the >next stable release. > >Now, for some software packages, this process is problematic, because >upstream may have another idea about software lifecycles. Concerning >the >GitLab example, upstream provides security fixes for three months for >their stable releases. Backporting fixes from newer versions is very >hard or impossible because the massive amounts of changes to the >software in every new versions. This is something that also affects >other packages, like Mozilla Firefox, which has a firefox package in >unstable, and a separate firefox-esr package, with the ESR version of >Firefox. Only the latter migrates to testing. > >Users of Debian honour it for its stability, but as an agile software >lifecycle is adapted by more and more very popular software packages, >not being able to install these packages in the trusted, well-known >fashion through the official apt repositories is becoming more and more >of a drawback. > >It can easily be assumed that the normal release and maintenance cycle >of Debian stable will not change, which is very good, so we should find >a way to still provide such software as described above to users. > > >Why backports is not enough >=== > >This also is well-known, but for completeness: Formal backports in >stable-backports are required to be direct backports from testing, and >are a stepping stone within the upgrade from stable to stable+1. Thus, >a >version of a package that is not in testing can never be in >stable-backports. > > >Name of the new repository >== > >In the past, the name “volatile” was used for a similar repository, but >with a different scope (limited to data packages for things like virus >scanners). I will thus use the working title volatile throughout this >proposal, although this may change. > >Other ideas: fastlane, unsupported > >(Please feel free to add other ideas.) > > >Requirements for a package to go into stable-volatile >= > >The new volatile proposal is not intended to ease life for package >maintainers who want to bypass the migration and QA requirements of the >regular stable lifecycle, so special need must be taken to ensure only >packages that need it go into volatile. I want to summarise the >requirements like so: > >- The package must be maintained in unstable, like every other package. > - The package must not be in testing, and care must be taken for the > package not to migrate to testing. > - Regular maintenance for the lifetime of stable must be impossible > or unnecessarily hard, and this requirement should be assessed in > a verifiable manner, e.g. referring to upstream’s lifecycle model. > - There must be notable need for the package. Like for backports, user > requests might be an indicator. > - Should the package be removed from unstable, it must also be removed > from volatile. > - Should the package begin to migrate to testing again, it must > be moved to stable-backports. > >Before starting to maintain a volatile package, the maintainer shall >seek consent (or doubt) on debian-devel. > > >Building packages and package dependencies >== > >Packages for volatile are built the same way as formal backports, only >that the source is taken from unstable rather than testing. In >particular: > > - Changes shall be kept as small as possible. > - The package is rebuilt against stable. >- The package may depend on packages in stable, stable-backports or >stable-volatile. > >Dependencies on other packages in volatile should be avoided if >possible. Especially, dependencies of the package that also need >backporting must not be added to volatile just because they are >dependencies
Re: Proposal: Repository for fast-paced package backports
> - no need to keep a volatile package out of testing Oh, and yes. Having a package in testing means it will be supported for a stable lifecycle - a full contradiction to volatile! -nik
Re: Proposal: Repository for fast-paced package backports
Hi, >I would, however, completely separate it from backports. I.e. > > - separate NEW queue > - different suffix > - no need to keep a volatile package out of testing > >Why? > > - volatile is a different beast from backports, this should be > very clear to both package maintainers and our users The idea is to have them separated, but fully interoperable. I.e. the proposal ensures such things as: - foo is not supportable for the buster release cycle. It goes to volatile. - foo becomes supportable for buster+2. - foo is backported (as in -backports) to buster+1 This will work properly, among other such scenari. > - volatile must not put any burden on the backports team, which > e.g. a common NEW queue would probably impose The whole point is that it is not new work or a new burden. This is one reason for the rules being almost the same and the clear decision path and movement between -backports and -volatile. A -volatile package is handled exactly the same, except it comes from unstable. The workload is the same as if the package had migrated to testing and was being uploaded to -backports. The defined preconditions ensure this is not abused for a ton of packages. -nik
Re: Proposal: Repository for fast-paced package backports
Hi all, I like the idea of having a volatile archive and I agree with almost all what Dominik wrote about the motivation. I would, however, completely separate it from backports. I.e. - separate NEW queue - different suffix - no need to keep a volatile package out of testing Why? - volatile is a different beast from backports, this should be very clear to both package maintainers and our users - in volatile we can give less guarantees about future upgradability than backport provides - volatile must not put any burden on the backports team, which e.g. a common NEW queue would probably impose Just my 6¢, Cheers
Re: Proposal: Repository for fast-paced package backports
>Just to make things a bit clearer for people who may not have followed >some of the discussions on d-bp-users lately: the point is to be able >to >support fast-moving software with not-so-fast moving dependencies; >the dependencies may easily be backported without too large a burden >(their versions will not come too often, so they will be able to >migrate > to testing and thus fulfil the criteria for being in backports), while >the main piece of software moves too fast, including across major >versions and with incompatible changes, so that it is not suitable for >being included in a stable release (thus the part in the proposal about >blocking its migration to testing). > >The maintainers of the stack will first package the dependencies, wait >for them to migrate to testing, then backport them, and then they will >upload the main piece of software first to unstable and then to the new >suite under discussion. Exactly. And the result shall still have the same quality as any package in -backports, technically, as far as it can. Thus the requirements for version, etc. Volatile is not to become a place to dump packages to bypass -backports. On the contrary. -nik
Re: Proposal: Repository for fast-paced package backports
On Tue, Dec 25, 2018 at 11:52:07PM +0100, Dominik George wrote:
> Hi,
>
> >having read the whole Gitlab discussion, I still don't get how/why the
> >new repository depends or relates to backports. Instead it could be
> >self-contained, except for stuff already available in stable. Couldn't
> >you roll the new repository entirely independent of any backports? Even
> >if you say there won't be any additional work for the backport policy
> >owners, letting a new repo depend on backports will implicitly have an
> >impact, which doesn't sound fully thought through yet.
>
> This is answered in the proposal. The reason is to not have volatile
> abused to ease backporting, and to allow packages to easily move back to
> backports again.
Just to make things a bit clearer for people who may not have followed
some of the discussions on d-bp-users lately: the point is to be able to
support fast-moving software with not-so-fast moving dependencies;
the dependencies may easily be backported without too large a burden
(their versions will not come too often, so they will be able to migrate
to testing and thus fulfil the criteria for being in backports), while
the main piece of software moves too fast, including across major
versions and with incompatible changes, so that it is not suitable for
being included in a stable release (thus the part in the proposal about
blocking its migration to testing).
The maintainers of the stack will first package the dependencies, wait
for them to migrate to testing, then backport them, and then they will
upload the main piece of software first to unstable and then to the new
suite under discussion.
G'luck,
Peter
--
Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} p...@storpool.com
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13
signature.asc
Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
Hi, I like the general direction, but there are some aspects of your >proposal >which should be improved. Thanks! >> Other ideas: fastlane, unsupported > >Or maybe something like "fastpaced", after all this repo would not be >unsupported at all, the very point is to provide actual support after >all. I actually think volatile is a good name. After all, it's not so far from the previous volatile. >> - The package must be maintained in unstable, like every other >package. > >Given the nature of the packages in "fastpaced", it's counterproductive >to mandate the same standards as for the standard archive, it rather >makes >sense to relax some aspects. > >E.g. we usually try to avoid embedded code copies. But for a package >like Gitlab that doesn't really add any value, if an embedded Ruby >package is affected, Gitlab upstream fixes it in their weekly release >anyway. And if not using the embedded code copies you'll end up with >plenty of >dependencies which can no longer be fulfilled from stable as upstream >moves forward. The intention is to keep the way open to have a real backport again should the situation change. I find that very important for compatibility and assuring upgrade paths. >> I propose to add the volatile repository next to the backports >> repository, and treat it as part of backports. > >I wouldn't tie this to backports at all, rather make it a separate >section of the archive and have some ACL mechanism to allow the DDs >maintaining a fastpaced package to grant access to it (similar to >#817285). I am open to this, as long as the goals to have full compatibility with backports stay the same. -nik
Re: Proposal: Repository for fast-paced package backports
Hi all, having read the whole Gitlab discussion, I still don't get how/why the new repository depends or relates to backports. Instead it could be self-contained, except for stuff already available in stable. Couldn't you roll the new repository entirely independent of any backports? Even if you say there won't be any additional work for the backport policy owners, letting a new repo depend on backports will implicitly have an impact, which doesn't sound fully thought through yet. I consider especially copying parts of the version scheme fairly confusing. This gives your concept a bad touch of just trying to work around established rules (i.e. backports rules). Instead of defining such minor facets I would recommend you to work on clarity about what rules you want to establish in the new repo instead. Also, as Alex suggested, I would prefer if such experiments could be started outside the official Debian archive, like backports once successfully did. Given how much efforts it took to get backports integrated officially, I don't consider adding a new repo a minor change. Did you discuss your idea with ftp masters, dak maintainers, and buildd admins before? I acknowledge that Debian needs a solution to support fast moving projects like Gitlab better than now. Yet, without a *proof* of concept how this could work out in the long run (i.e. across more than one Debian release cycle), I don't think it is the right time to ask for such a big change now. I consider Debian open enough to support such concepts outside the official archive first. Kind regards, Micha
Re: Proposal: Repository for fast-paced package backports
Hi, >having read the whole Gitlab discussion, I still don't get how/why the >new repository depends or relates to backports. Instead it could be >self-contained, except for stuff already available in stable. Couldn't >you roll the new repository entirely independent of any backports? Even >if you say there won't be any additional work for the backport policy >owners, letting a new repo depend on backports will implicitly have an >impact, which doesn't sound fully thought through yet. This is answered in the proposal. The reason is to not have volatile abused to ease backporting, and to allow packages to easily move back to backports again. >I consider especially copying parts of the version scheme fairly >confusing. This gives your concept a bad touch of just trying to work >around established rules (i.e. backports rules). Instead of defining >such minor facets I would recommend you to work on clarity about what >rules you want to establish in the new repo instead. I am a bit disappointed that my efforts to emphasize good compatibility with established processes is interpreted that way. As I already laid out several times during the last days, I am in fact disappointed that assuming bad or egoistic intentions seems to have become normal in Debian. That said, the version numbering is a way to ensure work *with* established rules, not around. >Also, as Alex suggested, I would prefer if such experiments could be >started outside the official Debian archive, like backports once >successfully did. Given how much efforts it took to get backports >integrated officially, I don't consider adding a new repo a minor >change. Did you discuss your idea with ftp masters, dak maintainers, >and buildd admins before? I did not discuss this proposal before discussing this proposal, no. That's why I am discussing this proposal :). If you read it properly, you will find that does not add anything really new, but extends something existing - yet without interfering with it much. >I acknowledge that Debian needs a solution to support fast moving >projects like Gitlab better than now. Yet, without a *proof* of concept >how this could work out in the long run (i.e. across more than one >Debian release cycle), I don't think it is the right time to ask for >such a big change now. Again, the change is not new - it is an extension of backports, using the exact same concepts and rules, apart from the source distribution and the target directory. It is an extension designed to play very nicely with backports. > I consider Debian open enough to support such >concepts outside the official archive first. I hope that e.g. official buildds will not grab code from my private machine and build it, for example. -nik
Re: Proposal: Repository for fast-paced package backports
> In short: This proposal addresses the exact concerns you raised before > )although I am not the person you expressed them towards). Well, sure, I was involved in that thread, but only in the way that I announced a proposal (this one). Not in any of the stuff concerning adding something to -backports. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On Tue, Dec 25, 2018 at 10:11:43PM +0100, Alexander Wirt wrote: > https://lists.debian.org/debian-backports/2018/12/msg00028.html > > This wasn't about gitlab. Oh. I must have misread the "gitlab" in the subject, along withthe mail being sent to the gitlab maintainer, a gitlab bugreport in the BTS, and concerning a request to accept gitlab into backports ;). Still, there's a big difference: * The thread you refer to is about uploading to backports. This proposal ia about *not* uploading to backports. The newly-proposed section is only intended to co-exist with backports, and interact nicely with backports. (Mind the difference between backport as a general term for a package made available for an older distribution, and the name backports for a section in the Debian repository). * Your mail you are referring to talks about "backports" from unstable being a different workflow - this proposal proposes such a workflow. * Your mail refers to packages being indistinguishable in -backports - this proposal is all about having a new section in the repository to distinguish them. In short: This proposal addresses the exact concerns you raised before )although I am not the person you expressed them towards). -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On Tue, 25 Dec 2018, Dominik George wrote: > > We already told you to build your own repo. > > You should probably start with identifying the senders of mail > correctly ☺. I am not the gitlab maintainer (and will never be). https://lists.debian.org/debian-backports/2018/12/msg00028.html This wasn't about gitlab. > > > Imho you should start the same way backports started - outside of > > debian. > > Prove that it works and integrate into Debian later. > > I would agree with you if it were a big change - however, the proposal > has a very low impact, if not none at all, on existing stuff. In > contrast to what you seem to believe (accuse people of…), this proposal > is about helping Debian as a whole, not forcing a certain package into > the distribution. gitlab only serves as an example of why it is useful. > The Debian infrastructure already supports everything that is needed to > implement this, and starting with parallel infrastructure would probably > mean that it will fail because this requires a single person spending > time and money to maintain the infrastructure (which is otherwise > already there), and to make it really work, this is a low (think of > buildds, etc.). > > In any case, I do not see why you would fight the fact that someone > makes a detailed proposal. A proposal can be accepted or denied, of > course, but your tone implies you think noone should have made the > proposal i nthe first place. > > Please don't fight people wanting to help based on your opinion about a > prior case around gitlab. I don't fight against it. I just want to keep it away from backports, thats not the same. Alex signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
> We already told you to build your own repo. You should probably start with identifying the senders of mail correctly ☺. I am not the gitlab maintainer (and will never be). > Imho you should start the same way backports started - outside of > debian. > Prove that it works and integrate into Debian later. I would agree with you if it were a big change - however, the proposal has a very low impact, if not none at all, on existing stuff. In contrast to what you seem to believe (accuse people of…), this proposal is about helping Debian as a whole, not forcing a certain package into the distribution. gitlab only serves as an example of why it is useful. The Debian infrastructure already supports everything that is needed to implement this, and starting with parallel infrastructure would probably mean that it will fail because this requires a single person spending time and money to maintain the infrastructure (which is otherwise already there), and to make it really work, this is a low (think of buildds, etc.). In any case, I do not see why you would fight the fact that someone makes a detailed proposal. A proposal can be accepted or denied, of course, but your tone implies you think noone should have made the proposal i nthe first place. Please don't fight people wanting to help based on your opinion about a prior case around gitlab. -nik signature.asc Description: PGP signature
Re: Proposal: Repository for fast-paced package backports
On Tue, 25 Dec 2018, Dominik George wrote: > Heisann, alle sammen, > > as announced in the recent thread about maintaining, I hereby propose a > repository that allows making “backports” of packages available to users > of the stable distribution, if those packages cannot be maintained in > testing and backported in the usual way. If you are interested in what > lead up to that, please see bug #915050. I will give a short summary of > it here. > > > Reasons for having a special place for some packages > > > (You may want to skip this part if you are familiar with the situation.) > > As all developers know (but passers-by may not), for software to enter > the Debian archive, it is always uploaded to the unstable distribution, > then migrates to testing (hopefully ;)), which is at some point snapshot > and made the new stable release. From there on, maintainers have two > obligations: Firstly, keep the package in stable good and secure, e.g. > by uploading security fixes for it once they become available upstream, > or even backport fixes themselves. Secondly, provide the package in > unstable with updates and ensure its migration, to keep it ready for the > next stable release. > > Now, for some software packages, this process is problematic, because > upstream may have another idea about software lifecycles. Concerning the > GitLab example, upstream provides security fixes for three months for > their stable releases. Backporting fixes from newer versions is very > hard or impossible because the massive amounts of changes to the > software in every new versions. This is something that also affects > other packages, like Mozilla Firefox, which has a firefox package in > unstable, and a separate firefox-esr package, with the ESR version of > Firefox. Only the latter migrates to testing. > > Users of Debian honour it for its stability, but as an agile software > lifecycle is adapted by more and more very popular software packages, > not being able to install these packages in the trusted, well-known > fashion through the official apt repositories is becoming more and more > of a drawback. > > It can easily be assumed that the normal release and maintenance cycle > of Debian stable will not change, which is very good, so we should find > a way to still provide such software as described above to users. > > > Why backports is not enough > === > > This also is well-known, but for completeness: Formal backports in > stable-backports are required to be direct backports from testing, and > are a stepping stone within the upgrade from stable to stable+1. Thus, a > version of a package that is not in testing can never be in > stable-backports. > > > Name of the new repository > == > > In the past, the name “volatile” was used for a similar repository, but > with a different scope (limited to data packages for things like virus > scanners). I will thus use the working title volatile throughout this > proposal, although this may change. > > Other ideas: fastlane, unsupported > > (Please feel free to add other ideas.) > > > Requirements for a package to go into stable-volatile > = > > The new volatile proposal is not intended to ease life for package > maintainers who want to bypass the migration and QA requirements of the > regular stable lifecycle, so special need must be taken to ensure only > packages that need it go into volatile. I want to summarise the > requirements like so: > > - The package must be maintained in unstable, like every other package. > - The package must not be in testing, and care must be taken for the >package not to migrate to testing. > - Regular maintenance for the lifetime of stable must be impossible >or unnecessarily hard, and this requirement should be assessed in >a verifiable manner, e.g. referring to upstream’s lifecycle model. > - There must be notable need for the package. Like for backports, user >requests might be an indicator. > - Should the package be removed from unstable, it must also be removed >from volatile. > - Should the package begin to migrate to testing again, it must >be moved to stable-backports. > > Before starting to maintain a volatile package, the maintainer shall > seek consent (or doubt) on debian-devel. > > > Building packages and package dependencies > == > > Packages for volatile are built the same way as formal backports, only > that the source is taken from unstable rather than testing. In > particular: > > - Changes shall be kept as small as possible. > - The package is rebuilt against stable. > - The package may depend on packages in stable, stable-backports or > stable-volatile. > > Dependencies on other packages in volatile should be avoided if > possible. Especially, dependencies of the package that
Proposal: Repository for fast-paced package backports
Heisann, alle sammen, as announced in the recent thread about maintaining, I hereby propose a repository that allows making “backports” of packages available to users of the stable distribution, if those packages cannot be maintained in testing and backported in the usual way. If you are interested in what lead up to that, please see bug #915050. I will give a short summary of it here. Reasons for having a special place for some packages (You may want to skip this part if you are familiar with the situation.) As all developers know (but passers-by may not), for software to enter the Debian archive, it is always uploaded to the unstable distribution, then migrates to testing (hopefully ;)), which is at some point snapshot and made the new stable release. From there on, maintainers have two obligations: Firstly, keep the package in stable good and secure, e.g. by uploading security fixes for it once they become available upstream, or even backport fixes themselves. Secondly, provide the package in unstable with updates and ensure its migration, to keep it ready for the next stable release. Now, for some software packages, this process is problematic, because upstream may have another idea about software lifecycles. Concerning the GitLab example, upstream provides security fixes for three months for their stable releases. Backporting fixes from newer versions is very hard or impossible because the massive amounts of changes to the software in every new versions. This is something that also affects other packages, like Mozilla Firefox, which has a firefox package in unstable, and a separate firefox-esr package, with the ESR version of Firefox. Only the latter migrates to testing. Users of Debian honour it for its stability, but as an agile software lifecycle is adapted by more and more very popular software packages, not being able to install these packages in the trusted, well-known fashion through the official apt repositories is becoming more and more of a drawback. It can easily be assumed that the normal release and maintenance cycle of Debian stable will not change, which is very good, so we should find a way to still provide such software as described above to users. Why backports is not enough === This also is well-known, but for completeness: Formal backports in stable-backports are required to be direct backports from testing, and are a stepping stone within the upgrade from stable to stable+1. Thus, a version of a package that is not in testing can never be in stable-backports. Name of the new repository == In the past, the name “volatile” was used for a similar repository, but with a different scope (limited to data packages for things like virus scanners). I will thus use the working title volatile throughout this proposal, although this may change. Other ideas: fastlane, unsupported (Please feel free to add other ideas.) Requirements for a package to go into stable-volatile = The new volatile proposal is not intended to ease life for package maintainers who want to bypass the migration and QA requirements of the regular stable lifecycle, so special need must be taken to ensure only packages that need it go into volatile. I want to summarise the requirements like so: - The package must be maintained in unstable, like every other package. - The package must not be in testing, and care must be taken for the package not to migrate to testing. - Regular maintenance for the lifetime of stable must be impossible or unnecessarily hard, and this requirement should be assessed in a verifiable manner, e.g. referring to upstream’s lifecycle model. - There must be notable need for the package. Like for backports, user requests might be an indicator. - Should the package be removed from unstable, it must also be removed from volatile. - Should the package begin to migrate to testing again, it must be moved to stable-backports. Before starting to maintain a volatile package, the maintainer shall seek consent (or doubt) on debian-devel. Building packages and package dependencies == Packages for volatile are built the same way as formal backports, only that the source is taken from unstable rather than testing. In particular: - Changes shall be kept as small as possible. - The package is rebuilt against stable. - The package may depend on packages in stable, stable-backports or stable-volatile. Dependencies on other packages in volatile should be avoided if possible. Especially, dependencies of the package that also need backporting must not be added to volatile just because they are dependencies — every dependency that is needed to be backported to support the volatile package must be considered on its own and in all but unprobable edge cases be maintained as a formal
