Re: Summary: Git Packaging Round 2
Hello, On Wed 06 Nov 2019 at 11:10AM -05, Sam Hartman wrote: > * If you were involved enough that you can read the summary and say > "Yeah, that's more or less what happened," please please do that. (If > you think I got something wrong in the summary, then please say that too) I read almost all the mail from the discussion and when I read your summary I did not think that anything was incorrect. HTH! -- Sean Whitton signature.asc Description: PGP signature
Re: Summary: Git Packaging Round 2
TL;DR: I'd feel a lot more comfortable if a couple of people would explicitly review wether I correctly captured the discussion in the summary. So, we've received a number of comments on aspects of the discussion. That plus the original discussion leads me to believe we really are interested in this issue and that there's community interest in moving forward. But many of the comments have explicitly said they didn't follow the discussion. The summaries are half the check that the facilitator is doing a good job in a consensus discussion. The other half is having some people involved sanity check the summary. It would really help me have the confidence to move forward if a couple of people did that: * If you were involved enough that you can read the summary and say "Yeah, that's more or less what happened," please please do that. (If you think I got something wrong in the summary, then please say that too) * If you're willing to read the summary and compare against the actual discussion, I'd greatly appreciate that, especially if we cannot get people involved to confirm from their memory. If you do that, I recommend using a threaded mail reader and being prepared to skip reading subthreads that get too repetative or off-topic. When I did my re-read to prepare the summary, I trimmed out most of the discussion of non-free services from my re-read. With that trimmed, it took me an hour and a half to take notes on the entire discussion. I do understand that's a significant time investment to ask others to make. * If you aren't able to spend that time or have personal recollections, but think the level of review I and others have already done is adequate, please say that. Thanks, --Sam
Git Branch Names / DEP-14 (Was: Re: Summary: Git Packaging Round 2 [comments by 11/05/2019])
On 11/5/19 9:51 PM, Nicholas D Steeves wrote: > Richard Laager writes: >> I'd love to see more information about a recommended branch >> structure. FWIW, I've been using branches named for each release >> (e.g. "sid" is the default, but I also have "buster" for a (proposed) >> stable update, will likely soon have "buster-backports"). This works >> really well, and also scales to also having branches for Ubuntu >> (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also >> keeps "master" available for the upstream master branch. This seems so >> obvious and wonderful to me, but I'm not sure how popular it is. As I moved my first package to Salsa and was rethinking everything, I switched from "sid" to "unstable", for reasons that will come up below. > https://dep-team.pages.debian.net/deps/dep14 > > It sounds like you're already using something similar :-) I had forgotten about DEP-14 until it was mentioned earlier today by Feri . I'll likely rename my branches again to add the debian/ prefix, to be in compliance with DEP-14. DEP-14 specifies three options for development releases: debian/master (the recommended default) debian/sid debian/unstable Having three names increases inconsistency between packages, which seems to me contrary to the goal of DEP-14. If someone is working with both unstable and experimental, then they must use two branches to differentiate them. DEP-14 says to do so with debian/experimental for experimental. So far, so good. For unstable, DEP-14 says to use debian/sid or debian/unstable. Why not A) pick *one* of those, and B) always use it, never using debian/master? As for _which_ one (debian/sid or debian/unstable)... The DEP-14 recommended branch names for stable release are debian/CODENAME. The recommended branch name for experimental is debian/experimental. These, plus debian/unstable, but not debian/master or debian/sid, are consistent with distribution names from debian/changelog and the .changes file. [0] Therefore, it seems like one should use debian/unstable. [0] https://www.debian.org/doc/debian-policy/ch-controlfields.html#id25 -- Richard signature.asc Description: OpenPGP digital signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
Richard Laager writes:
> This all looks very good.
>
> Presumably the repository / Salsa project name should match the source
> package name? If so, that might be good to note, at least as the
> default.
>
I'm curious what other people think about this point, because it could
cause a lot of churn either in source package renames or salsa project
renames and uploads to fix VCS links.
> I'd love to see more information about a recommended branch
> structure. FWIW, I've been using branches named for each release
> (e.g. "sid" is the default, but I also have "buster" for a (proposed)
> stable update, will likely soon have "buster-backports"). This works
> really well, and also scales to also having branches for Ubuntu
> (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also
> keeps "master" available for the upstream master branch. This seems so
> obvious and wonderful to me, but I'm not sure how popular it is.
>
https://dep-team.pages.debian.net/deps/dep14
It sounds like you're already using something similar :-)
[1] One modification to DEP14 that I'd like to see is a requirement to set
the Debian development branch as the default branch on Salsa, so that a
debcheckout will put the user in the right place to get work done. Eg:
"debian/next", "debian/sid", "debian/devel", etc. This also makes it
easier for new contributors to submit MRs based on the correct branch.
[snip]
> If this message is threaded wrong, my apologies. While I am now, I was
> not subscribed to debian-devel at the time. I have tried to set
> In-Reply-To directly, but this is my first attempt at doing so. In
> these situations, I would normally download the mbox archive, import
> it, and reply to the message that way, but mbox archives are seemingly
> unavailable (at least to non-DDs?) for Debian lists.
>
I didn't notice a threading problem; however your email had long lines
that I had to manually reflow.
wf...@debian.org writes:
> Sam Hartman writes:
>
>> General Recommendation
>> ==
>> [...]
>> You should document the branch format (such as patches unapplied (gbp
>> pq)) that your repository uses. Best practice for documenting this is
>> still evolving. The best current option is to document your branch
>> format in debian/README.source.
>
> I'm not sure this best current option (README.source) is something I'd
> actually recommend. Granted, it's annoying to have to figure the git
> workflows, and a freeform documentation text like README.source helps if
> you have to do it a couple of times, but it still isn't a thing you can
> automate. Thus I think that the overall burden it creates outweighs its
> value. Also, from the Policy: "maintainers are encouraged to document
> in a debian/README.source file any source package with a particularly
> complex or unintuitive source layout or build system" -- do we intend to
> recommend something "particularly complex or unintuitive"?
>
[2] I wonder if the investigative work Ian did for tag2upload can be
recuperated for the development of an extension of the source format
specification. Namely, a shorthand git-project format. One would then
consult the table for repos that declare a style using a shorthand key
term that one isn't familiar with. "other" would need to be an option
for a while. I'm not sure if it should go in L2 of "source/format" or
somewhere else.
> I have to admit I haven't read the full discussion. Have we got a list
> of items we need to know about the "branch format"? Your text mentions
> patches applied or unapplied, which dpkg-source (and, according to you,
> dgit) already handles, so a utility could provide the answer if there's
> at least one patch in the package; why document it then? If the
> heuristic is deemed too fragile or we want to store this information
> even in the absence of patches, perhaps we could invent a new (local)
> dpkg-source option to override it and require using that (just an
> idea).
The utility would parse [2]. I'm not sure about using heuristics and
suspect that the "branch format" list will need to be manually
maintained. It might also benefit from codification in Policy for the
following case: a decade from now someone adopts a package, but the
"branch format" document/section has substantially changed since the
last upload. Codification in Policy would allow the person adopting the
package to reference a historical copy of Policy to more easily decipher
the branch format.
> Another obvious item is the main packaging branch, which is declared by
> Vcs-Git ("ideally", says the Policy -- shouldn't we make this stronger?)
>
See [1]. I'd prefer for the general case to be solved on salsa rather
than using extra Vcs-Git metadata. To get the affect of "debcheckout
foo && cd foo" and be ready for work ISTM that debcheckout (and other
tools) would need to parse this and switch to the correct branch. How
can this approach solve the problem of getting the wrong working branch
after a salsa
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
Sam Hartman writes:
> General Recommendation
> ==
> [...]
> You should document the branch format (such as patches unapplied (gbp
> pq)) that your repository uses. Best practice for documenting this is
> still evolving. The best current option is to document your branch
> format in debian/README.source.
I'm not sure this best current option (README.source) is something I'd
actually recommend. Granted, it's annoying to have to figure the git
workflows, and a freeform documentation text like README.source helps if
you have to do it a couple of times, but it still isn't a thing you can
automate. Thus I think that the overall burden it creates outweighs its
value. Also, from the Policy: "maintainers are encouraged to document
in a debian/README.source file any source package with a particularly
complex or unintuitive source layout or build system" -- do we intend to
recommend something "particularly complex or unintuitive"?
I have to admit I haven't read the full discussion. Have we got a list
of items we need to know about the "branch format"? Your text mentions
patches applied or unapplied, which dpkg-source (and, according to you,
dgit) already handles, so a utility could provide the answer if there's
at least one patch in the package; why document it then? If the
heuristic is deemed too fragile or we want to store this information
even in the absence of patches, perhaps we could invent a new (local)
dpkg-source option to override it and require using that (just an idea).
Another obvious item is the main packaging branch, which is declared by
Vcs-Git ("ideally", says the Policy -- shouldn't we make this stronger?)
If we really want to go beyond these, the next step would be discovering
the actual names of the DEP-14 entities, I guess. This would probably
start with identifying the tooling (gbp, git-debrebase, git-dpm, etc.)
then reading their configuration (if any). This would give answers for
the previous items as well. But do we target mass operations on this
scale of variability at all?
Anyway: I think we should recommend setups which allow reliable
determination of the needed information, thus not requiring manual
textual documentation.
--
Thanks,
Feri
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
This all looks very good. Presumably the repository / Salsa project name should match the source package name? If so, that might be good to note, at least as the default. I'd love to see more information about a recommended branch structure. FWIW, I've been using branches named for each release (e.g. "sid" is the default, but I also have "buster" for a (proposed) stable update, will likely soon have "buster-backports"). This works really well, and also scales to also having branches for Ubuntu (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also keeps "master" available for the upstream master branch. This seems so obvious and wonderful to me, but I'm not sure how popular it is. I'm currently maintaining my packages on GitHub, but I'm going to move them to Salsa shortly to follow the consensus recommendation. I'm not a DD (and not a DM either, though I'm finally getting around to applying). Unfortunately, this means I have the, IMHO obnoxious, -guest suffix on my Salsa account. I certainly don't want to create things under that namespace, and it's doubly painful because if I ever become a DD, my username would change. So I'm asking my sponsor to create projects in the debian namespace. Of course, that's the recommendation anyway, so the -guest might be an inadvertent feature because it's encouraging me to use the debian team. :) I understand that DMs are quite literally second-class citizens (and non-DM package maintainers are third-class), but I really wish that things like Salsa usernames didn't call that out. If this message is threaded wrong, my apologies. While I am now, I was not subscribed to debian-devel at the time. I have tried to set In-Reply-To directly, but this is my first attempt at doing so. In these situations, I would normally download the mbox archive, import it, and reply to the message that way, but mbox archives are seemingly unavailable (at least to non-DDs?) for Debian lists. -- Richard signature.asc Description: OpenPGP digital signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
Sam Hartman dixit: >depend on non-free software. Github is a common example of a web >service that uses non-free software. So is Salsa. It does not use the packaged version of GitLab, which is in a sorry state anyway, and not suitable for a stable release, but the proprietary “open core” versioin from the GitLab people. I prefer to not host my packages on Salsa; for this reason, and because of the experience of having to change all the VCS headers multiple times, move repos, etc. when using the Debian-provided hosting platform. I can host very well on my own, using a Free Software platform. bye, //mirabilos -- I believe no one can invent an algorithm. One just happens to hit upon it when God enlightens him. Or only God invents algorithms, we merely copy them. If you don't believe in God, just consider God as Nature if you won't deny existence. -- Coywolf Qi Hunt
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
On Tue, Oct 22, 2019 at 09:58:00PM -0400, Sam Hartman wrote: > I think we have about two weeks left in the review period. > Just as a reminder we do need comments. > Even if people generally agree, we do need at least a few comments to > that effect. I like the current proposal for a default suggestion for how to lay out repositories on salsa when one is not working with a team and one has no better ideas. I think it would both ease the path of people[1] (re)learning debian packaging, and of people[1] who can do packaging and, having no particular needs, like to stick to standards to ease further work through uniformity. I liked Ansgar's concern about the standard risking to become obsolete. As a recommendation for a default layout, I think there should be a way to keep the recommendations aligned to current project expectations, and I'd like some upgrading-checklist-style document to assist in keeping things aligned. Hopefully the recommendations won't change too often, or the effort saved by not having to explore multiple options while setting up a packaging workflow, would be spent in keeping the workflow up to date. I have no particular ideas about how to keep a recommendation up to date, though I like the idea of having in Debian the possibility of doing that a lot, as I see it valuable for a broader range of things. While over-normating things in Debian would stifle its valuable diversity, I think that being able to recommend as much standards as possible would make it much easier to navigate that diversity. Enrico [1] like often me -- GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini signature.asc Description: PGP signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
I think we have about two weeks left in the review period. Just as a reminder we do need comments. Even if people generally agree, we do need at least a few comments to that effect.
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
Hi Sam & Debianistas, this is far TLDR for me. That is not meant as a critique, but as a feedback so you have a data point from some random Debianer's available CPU resources. (in general I'm fine to declare best practices for whatever issue so that people can orient themselves on where to head to, provided that those "best practices" aren't "too far out") Am 08.10.19 um 03:22 schrieb Sam Hartman: > > [...]
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
TL;DR: These are only guidelines. You're free to ignore them. So I think the issue you bring up isn't a big deal in this case. In more detail: > "Theodore" == Theodore Y Ts'o writes: Theodore> One thing that is been left unclear is what does it mean Theodore> to "use salsa"? Use Salsa" as a term doesn't seem to appear in the recommendation text. I'd assume that your evaluation of the recommendation might go like this: * No existing team * Hmm, I'm a Debian developer packaging for debian. So, let's look at the general recommendation. Wonder along. Get to notes and limitations and decide probably that you don't want all developers to have push access because you don't want to force push when you disagree with changes. * So you decide that the general recommendation is not for you and get to the "Packages not on Salsa" recommendation. Perhaps you decide to mirror to salsa as suggested there. I'm fairly sure you're already following the rest of the guidelines in that section. Theodore> So I could create a Salsa repo for e2fsprogs and add it to Theodore> the list; but what does that actually mean? What does it Theodore> mean to have a Vcs-Git line pointing at git.kernel.org Theodore> versus salsa.debian.org? It surely doesn't mean anything Theodore> about access rights, whether it's "any random Debian Theodore> person can check in arbitrary things to the repo --- Mostly it makes things more consistent about where to find your code for people thinking more about a lot of Debian packages than about e2fsprogs. Does it have huge value? It might some day if we get to a point where we do require everyone to at least mirror to salsa. Today it makes it easier for us to do things like that in the future. Is it worth you doing given how many repos you already have? Don't know; perhaps you have enough that it is easy. Perhaps adding one more is more effort and you don't see the point. Theodore> And suppose I did create a Salsa repo for e2fsprogs, which Theodore> could be changed by anyone in the debian group. And Theodore> suppose someone adds something to the git repo which is Theodore> totally wrong, and which bypassed any kind of code review. Theodore> No problem! I'd just do a force push and the commit in Theodore> Salsa would Go Away. Or is that sort of thing frowned Theodore> upon with having a git repository on Salsa? I bet we don't have a consensus. I personally wouldn't put a repo that commonly had master force-pushed in the debian group on Salsa. I wouldn't mind force-pushing rarely say if I changed branch formats or there was a significant oops. I would prefer not to have to develop such a consensus now. I agree that it's a great thing to converge on in the future. Theodore> As a result, I'd argue that when we talk about "forcing" Theodore> people to use Salsa, it's actually kind of underspecified Theodore> what might be meant by that. I agree. But since we're not talking about forcing people to use Salsa, I think it is OK. We're giving people best practice guidelines to use as a starting point. Unlike the dh discussion where there are some behaviors we consider buggy, these are guidelines you can use if they help you or ignore if they do not. I think it's OK to be less specified for such guidelines. We may get more specific in the future. But I think if we try to do that all in one step, it will be too big to ever take. --Sam
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
One thing that is been left unclear is what does it mean to "use salsa"? For example, the e2fsprogs git repository is hosted at multiple locations: * https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git * https://github.com/tytso/e2fsprogs.git * https://git.code.sf.net/p/e2fsprogs/code * https://repo.or.cz/e2fsprogs.git I push changes to all of them when do updates from my development system, and they all have my debian packaging branches. Which one is the "master" repo? There's really no such thing. I suppose we could call git.kernel.org the "master" because it was the first but technically, the bitkeeper repository predates them all. :-) So I could create a Salsa repo for e2fsprogs and add it to the list; but what does that actually mean? What does it mean to have a Vcs-Git line pointing at git.kernel.org versus salsa.debian.org? It surely doesn't mean anything about access rights, whether it's "any random Debian person can check in arbitrary things to the repo --- there are some packages that are in groups that have very tight access controls, and that's probably a good thing. I'm much more comfortable knowing that stealing some random Debian maintainer's git credentials is not enough to install trojan horses into the openssh package! And suppose I did create a Salsa repo for e2fsprogs, which could be changed by anyone in the debian group. And suppose someone adds something to the git repo which is totally wrong, and which bypassed any kind of code review. No problem! I'd just do a force push and the commit in Salsa would Go Away. Or is that sort of thing frowned upon with having a git repository on Salsa? As a result, I'd argue that when we talk about "forcing" people to use Salsa, it's actually kind of underspecified what might be meant by that. If a developer has their git repository on github, or git.kernel.org, or on their own private server, what value does it add to have another copy on Salsa? As far as I'm concerned, it neither adds much value, nor does it cost much. It's when you start saying that it must be the *canonical* repository, and it doesn't matter what random DD's push to it; once they've pushed to it, it must be preserved ***forever*** without any forced pushes or rewinds, that it starts to make more of a difference. Cheers, - Ted
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
On Wed, 09 Oct 2019 12:40:12 -0400, Sam Hartman wrote: > > "gregor" == gregor herrmann writes: > > >> General Recommendation == > > This entire section effectively only applies to individual packages that > are not in a team. > > Would "When there is no Team to use" or similar be a better headline > and address your concern? Yes, something like this would make it clearer IMO. > That recommendation technically applies to: > > * debian group > > * Things created in personal namespaces by non-DDs who don't want to > wait for someone to create a debian-group repo for them. Maybe add this as an intro? > but yes, I was explicitly not wanting to make teams document source > package format in README.source. Great, thanks for the clarification. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: Janis Joplin: Cry Baby signature.asc Description: Digital Signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
> "gregor" == gregor herrmann writes: >> General Recommendation == This entire section effectively only applies to individual packages that are not in a team. Would "When there is no Team to use" or similar be a better headline and address your concern? >> This recommendation is intended for people who are simply looking >> for an approach that will work well with approaches others are >> using. If you are packaging something in a large team, follow >> their existing practices. gregor> […] >> You should document the branch format (such as patches unapplied >> (gbp pq)) that your repository uses. Best practice for >> documenting this is still evolving. The best current option is >> to document your branch format in debian/README.source. >> Alternatively if you use `dgit push-source`, your source format >> is documented in maintainer tags. gregor> Does this paragraph refer to repos in the Debian group or to gregor> all repos, i.e. team-maintained ones? As implied by above, effectively non-team stuff. That recommendation technically applies to: * debian group * Things created in personal namespaces by non-DDs who don't want to wait for someone to create a debian-group repo for them. but yes, I was explicitly not wanting to make teams document source package format in README.source.
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
On Mon, 07 Oct 2019 21:22:46 -0400, Sam Hartman wrote: > Ansgar argued that documenting the branch format is unnecessary given > all the work you need to do to contribute to a package. Several > disagreed arguing that it helped them do their work. I have an idea where Ansgar might be coming from; maybe I'm wrong but in any case I have the following reservation or request for clarification: > Conclusiony Thing > - […] > * If you are using debian group you should document your branch format. > Update to say that. Ok, here it says that the branch format should be documented for the Debian group (which reads like "not for teams if teams have different policies / general documentation somewhere"). > General Recommendation > == > > This recommendation is intended for people who are simply looking for an > approach that will work well with approaches others are using. If you > are packaging something in a large team, follow their existing > practices. […] > You should document the branch format (such as patches unapplied (gbp > pq)) that your repository uses. Best practice for documenting this is > still evolving. The best current option is to document your branch > format in debian/README.source. Alternatively if you use `dgit > push-source`, your source format is documented in maintainer tags. Does this paragraph refer to repos in the Debian group or to all repos, i.e. team-maintained ones? The reason I'm asking is that I have very bad memories of the times when Policy required to document the use of quilt in debian/README.source (although quilt was very visible in both debian/control and debian/rules at that time); and literally thousands of boilerplate debian/README.source files were added; and some years later removed after a Policy change. I'm pretty sure there are still quite a few left in the archive. So if this recommendation is targetted at repos in the Debian group only (and teams can ignore it) I'm not enthusiastic by the prospect of outdated documentation there, but well … If it's meant to apply to team-maintained repos as well I'd consider this a mistake. (Maybe a sentence like "The following paragraphs refer to the Debian group, for teams see below under 'Recommendations for Teams'." in/after the first paragraph would help to disambiguate the scope at least.) Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: Janis Joplin: Cry Baby signature.asc Description: Digital Signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
On Tue, 08 Oct 2019, Holger Levsen wrote: > On Mon, Oct 07, 2019 at 09:22:46PM -0400, Sam Hartman wrote: > > [...] as a last opportunity for > > others to comment. > > what's the deadline to grok this 20k and respond? It's in the subject: [comments by 11/05/2019] November 5th. Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/ signature.asc Description: PGP signature
Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]
On Mon, Oct 07, 2019 at 09:22:46PM -0400, Sam Hartman wrote: > [...] as a last opportunity for > others to comment. what's the deadline to grok this 20k and respond? -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
