Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, 2021-06-30 at 14:31 +0200, Tomas Tomecek wrote: > After reading your explanation of how gentoo does packaging, it > indeed makes a lot of sense and feels like that most of the > concerns I pointed out have solutions or could be mitigated. > > The biggest problem I personally see is the effort which would be > needed to pull this off: it would take years to accomplish, a ton of > teams would be involved and all the maintainers would need to > cooperate - and then we'd need to do the same thing for CentOS Stream > and RHEL. > > Such a change to the development workflow would be massive and the > level of coordination would be immense. That's all I can say. I agree, it would be a lot to change. > > Looks like the gentoo-bot is a critical piece of the development > process so that people are able to progress with their contributions > efficiently. It certainly does feel pretty helpful. Even if we don't go for a monorepo here, one could imagine a bot like that being useful here too. > > Thanks Randy for such a great write-up. Now that I think about it, > the monorepo proposal is orthogonal to the source-git proposal as the > two workflows serve different purposes: one is meant for development > (source-git: write a patch), the other one for integration into the > operating system (tinker with a spec file or the build system). Yeah I do think they are orthogonal concerns. Enjoy your weekend Tomas (and everyone!) signature.asc Description: This is a digitally signed message part ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Zbyszek, thank you for your detailed feedback. I agree with you that the step to serialize patches into dist-git complicates things heavily and is error-prone. I say this as a person who wrote a ton of code [1] in packit to achieve this. Would be awesome if we could build from the git checkout instead of juggle with patches. After reading your response I feel like we agree on the fact that sources should be maintained as code in a git repo, we just don't have an agreement on how the code should be compiled into an SRPM and built in the build system. As you see in this thread, maintainers have their own preferences so there won't ever be "one way fits all". The bigger the changes we propose, the longer it will take to implement, integrate and adopt. I'm wondering if what you're proposing is affected by Packaging:SourceURL -> Git Hosting Services [2], namely "If the upstream does create tarballs you should use them as tarballs provide an easier trail for people auditing the packages." - would that imply that the "fork" is suddenly being packaged instead of the actual upstream release? [1] https://github.com/packit/packit/blob/main/packit/patches.py [2] https://fedoraproject.org/wiki/Packaging:SourceURL#Git_Hosting_Services On Wed, Jun 30, 2021 at 3:46 PM Zbigniew Jędrzejewski-Szmek wrote: > > On Wed, Jun 30, 2021 at 02:03:22PM +0200, Tomas Tomecek wrote: > > On Wed, Jun 30, 2021 at 10:40 AM Zbigniew Jędrzejewski-Szmek > > wrote: > > > > > > > Our definition of source-git is: > > > > * upstream release tarball > > > > * downstream code changes applied as patches during %prep > > > > * additional configs for sake of building and testing > > > > > > But that describes dist-git *exactly*. Where is the difference? > > > > Upstream git history and downstream patches tracked as git commits > > instead of files. > > Hmm, so debian does something like this, with quilt for patches… > Debian-style packaging needs to do this, because they still need to > support non-git upstreams, even upstreams without version control, and > generally cater to the lowest possible denominator. > > Since source-git would be the the biggest change in Fedora packaging > in a decade, I hope we could do something better. The "source-git" > proposal as is right now tracks the patches as git commits, but > serialized. This means that the patches must be a serial sequence of > commits, so any history with merges needs to be turned into a fake > linear history. > > The current proposal also keeps the "upstream tarball" as the central > object. In the past a tarball could be a custom-crafted thing with > some pre-generated or pre-compiled stuff and files added and removed, > and the pristine checkout of a repo wouldn't build. Those times are > thankfully behind us. Tarballs are fine, but nowadays pretty much > everything is just a snapshot generated from a git repo. So we > shouldn't treat this one blob as special. A tarball generated from a > release commit is just like a tarball generated from non-tagged > commit, except that upstream designated it as special. But if the > maintainer wants/needs to build from a non-release commit, then we > should just do it. > > If we are using git, we shouldn't need to deal with applying and > removing textified patches. > > In particular, requiring a linear history quickly breaks down for > real-life repos. For example, let's say that upstream has released > version 248, does a few immediate bugfix commits, and then applies a > few spelling patches on top, and another CVE bugfix _merged_ as commit > 248+15. We want to build this as quickly as possible. > > We have two choices now: use upstream tarball and pick patches by hand > and create a linearized history. This is complex work, fraught with > potential breakage. If the merge commit had some conflicts, the > original patch from the bugfix branch will not apply. Even if there is > _no confict_, the maintainer might have applied some fixes in the > merge. So the packager needs to do a rebase of the patch and possibly > find subsequent tweaks that are not part of the patch. > > I maintain the stable branches from systemd [1], and I know firsthand > that trying to rebase non-trivial patches is complex and you just > screw it up every once in a while. And this is for a project which I > know extremely well. For a project where I'm just a packager, > non-trivial rebases become guesswork. > > The other choice is to take the appropriate upstream commit and just > build the tarball from that. Just do it, without guesswork or manual > steps. > > It feels very much like the current "source-git" proposal is trying > to recreate the existing workflows with some minimal changes. Those > existing workflows evolved to handle idiosyncrasies which are not > needed anymore. Those old projects will not switch to source-git anyway. > I think it's much better to handle those modern git upstreams > idiomatically. > > In the proposed workflow [2], 'git reset --hard' is
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021, 8:52 AM Neal Gompa wrote: > On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > > > > > > > > > On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" > wrote: > > > >On 24. 06. 21 23:07, Miroslav Suchý wrote: > > > >> Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > > > One thing to consider is that the upstream tarballs might be > > > >cryptographically > > > signed and packages should verify the signature in %prep. > > > >>> This is a very good point - in such a case, we should always pull > > > >the > > > >>> official upstream tarball instead of generating a new one > downstream > > > >> > > > >> Does it matter? If you are able to generate byte2byte identical > > > >tarball then > > > >> you can choose any of them. > > > > > > > >AFAIK git does not grantee to produce byte2byte identical archives > > > >across > > > >different versions of git, zlib, gzip etc. So even if upstream signs > > > >the git > > > >generated archive, generating a byte2byte identical one might be > > > >tricky. > > > > > > Especially with xz, which iirc has reproducibility issues in parallel > mode. > > > > I think we should try to push upstream to sign git tags, instead or in > > addition to tarballs. For upstreams, this is actually much easier > > (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > > a tarball on github which requires some interaction with the web service. > > > > As an upstream, I would literally *never* GPG sign git tags. If you > ask me to do that, I won't. It's far too annoying to deal with for me > to be willing to suffer through that. > I don't get it. I sign every commit and every tag for the upstream linux-firmware repo. It is not onerous. What issues do you have? josh > I'm not going to ask people to do something I would be unwilling to do > myself. > > > > -- > 真実はいつも一つ！/ Always, there's only one truth! > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, Jun 30, 2021 at 08:11:38AM -0500, Justin Forbes wrote: > On Wed, Jun 30, 2021 at 3:40 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Tue, Jun 29, 2021 at 03:13:10PM +0200, Tomas Tomecek wrote: > > > On Thu, Jun 24, 2021 at 8:09 PM Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > > > > > On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: > > > > > On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok > > > > > wrote: > > > > > > > > > > > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > > > > > > ## Choosing git forge to host source-git repositories > > > > > > > We need to find a home for all the source-git repositories. This > > > > > > > is > > > > > > > actually a really hard task because we have many options > > > > > > > (github.com, > > > > > > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > > > > > > on-premise) and different expectations: some projects already have > > > > > > > repos set up on different platforms while Pagure is the primary > > > > > > > forge > > > > > > > now. Since the CPE team is investigating GitLab as a forge, it's > > > > > > > even > > > > > > > harder for us to figure out the primary forge. We may end up > > > > > > > supporting both actually: pagure.io and gitlab.com. What are your > > > > > > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > > > > > > More info: > > > > > > > * https://pagure.io/fedora-source-git/sig/issue/1 > > > > > > > * https://pagure.io/fedora-source-git/sig/issue/7 > > > > > > > > > > > > I would expect that since the soruce git is just an intermediate > > > > > > thing, > > > > > > supporting "any forge" is nice to have, but hard. I'd start with > > > > > > some common > > > > > > options (Pagure + 1 other) and see where you'll get. > > > > > > > > > > Yep, that's exactly what I'd like us to do. As soon as we have the > > > > > service which accepts processes events up and running, it shouldn't be > > > > > that hard to teach it new fedora-messaging messages or webhook > > > > > payloads. > > > > > > > > > > > > ## High-level workflow proposal up for review > > > > > > > Hunor proposed a high-level workflow linked below and I strongly > > > > > > > recommend reading it. We have also started discussing many > > > > > > > details in > > > > > > > the process, such as getting archives: should we generate one > > > > > > > from the > > > > > > > source-git repo or use the official release archive from upstream? > > > > > > > > > > > > One thing to consider is that the upstream tarballs might be > > > > > > cryptographically > > > > > > signed and packages should verify the signature in %prep. > > > > > > > > > > This is a very good point - in such a case, we should always pull the > > > > > official upstream tarball instead of generating a new one downstream. > > > > > > > > Hmm, but how would that work? I thought that the whole point of > > > > source-git is to build from a commit that contains some upstream > > > > version + downstream patches + downstream distro config like the spec > > > > file, > > > > and that the build step of applying downstream patches just doesn't > > > > exist. > > > > If you reuse the upstream tarball, then by necessity those downstream > > > > patches need to be serialized and applied on top like in dist-git. So > > > > you lose the property that the checkout from version control is *the* > > > > source you build, but you also lose the property that the source you > > > > build is what was signed (since patches are applied). > > > > > > What you just described I understand as an (upstream) maintenance > > > branch and is not what we call source-git. > > > > It's an "upstream maintenance branch" PLUS the .spec bits that specify > > how to build a package. > > > > > Our definition of source-git is: > > > * upstream release tarball > > > * downstream code changes applied as patches during %prep > > > * additional configs for sake of building and testing > > > > But that describes dist-git *exactly*. Where is the difference? > > > > In the spirit of openness that Fedora is based upon, it must be very > easy to determine what patches we carry as opposed to upstream, and > exactly how we build things. I very much agree with this. > This does not go away even if we get to > the point where dist-git no longer exists, and we can build directly > from source-git. In the meantime, dist-git is the "canonical source > of truth" for fedora packages, so there is a middle step to convert > from source-git to dist-git. With both, the difference in using > source-git means there is an exploded tree there, with patches > applied. This makes debug and development much easier. There is also > git history, which hopefully contains a lot more background than > currently exists in dist-git. Finally, it makes bisecting a much more > simple process, whether the offending patch is in upstream or > downstream changes. [See my long reply to the parent message...] The
Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, Jun 30, 2021 at 02:03:22PM +0200, Tomas Tomecek wrote: > On Wed, Jun 30, 2021 at 10:40 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > > Our definition of source-git is: > > > * upstream release tarball > > > * downstream code changes applied as patches during %prep > > > * additional configs for sake of building and testing > > > > But that describes dist-git *exactly*. Where is the difference? > > Upstream git history and downstream patches tracked as git commits > instead of files. Hmm, so debian does something like this, with quilt for patches… Debian-style packaging needs to do this, because they still need to support non-git upstreams, even upstreams without version control, and generally cater to the lowest possible denominator. Since source-git would be the the biggest change in Fedora packaging in a decade, I hope we could do something better. The "source-git" proposal as is right now tracks the patches as git commits, but serialized. This means that the patches must be a serial sequence of commits, so any history with merges needs to be turned into a fake linear history. The current proposal also keeps the "upstream tarball" as the central object. In the past a tarball could be a custom-crafted thing with some pre-generated or pre-compiled stuff and files added and removed, and the pristine checkout of a repo wouldn't build. Those times are thankfully behind us. Tarballs are fine, but nowadays pretty much everything is just a snapshot generated from a git repo. So we shouldn't treat this one blob as special. A tarball generated from a release commit is just like a tarball generated from non-tagged commit, except that upstream designated it as special. But if the maintainer wants/needs to build from a non-release commit, then we should just do it. If we are using git, we shouldn't need to deal with applying and removing textified patches. In particular, requiring a linear history quickly breaks down for real-life repos. For example, let's say that upstream has released version 248, does a few immediate bugfix commits, and then applies a few spelling patches on top, and another CVE bugfix _merged_ as commit 248+15. We want to build this as quickly as possible. We have two choices now: use upstream tarball and pick patches by hand and create a linearized history. This is complex work, fraught with potential breakage. If the merge commit had some conflicts, the original patch from the bugfix branch will not apply. Even if there is _no confict_, the maintainer might have applied some fixes in the merge. So the packager needs to do a rebase of the patch and possibly find subsequent tweaks that are not part of the patch. I maintain the stable branches from systemd [1], and I know firsthand that trying to rebase non-trivial patches is complex and you just screw it up every once in a while. And this is for a project which I know extremely well. For a project where I'm just a packager, non-trivial rebases become guesswork. The other choice is to take the appropriate upstream commit and just build the tarball from that. Just do it, without guesswork or manual steps. It feels very much like the current "source-git" proposal is trying to recreate the existing workflows with some minimal changes. Those existing workflows evolved to handle idiosyncrasies which are not needed anymore. Those old projects will not switch to source-git anyway. I think it's much better to handle those modern git upstreams idiomatically. In the proposed workflow [2], 'git reset --hard' is an integral part of the maintainer steps. I proposed how not to do that in a comment there [3]. The important part is that how we reach the commit from which we will build, i.e. the details of history, should be an implementation detail. (Similarly to how normal git repos work: some upstreams use github's "squash", others use "merge" always, and so on. We only care about the end result, i.e. the tree content of the commit, not its parents.) In very short detail, the workflow IMO should be: - take an upstream repo - create a branch that contains selected upstream commit + possibly some patches + downstream packaging stuff (.spec, etc). The tree object attached to that commit is what will be built. (In the common case, this branch will be a child of the upstream release and the previous packaging branch. After all, most of the time packaging changes when updating to a new release are very small. This is important to be able to see (using appropriate 'git diff' and 'git log' invocations), what changed wrt. to the upstream, previous packaging build, latest upstream release, etc.) - to build: 'git archive', unpack in the build system, go to %build. [1] https://github.com/systemd/systemd-stable [2] https://pagure.io/fedora-source-git/docs/pull-request/2#request_diff [3] https://pagure.io/fedora-source-git/docs/pull-request/2#comment-152304 Zbyszek ___ devel mailing
Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, Jun 30, 2021 at 3:40 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Tue, Jun 29, 2021 at 03:13:10PM +0200, Tomas Tomecek wrote: > > On Thu, Jun 24, 2021 at 8:09 PM Zbigniew Jędrzejewski-Szmek > > wrote: > > > > > > On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: > > > > On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok > > > > wrote: > > > > > > > > > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > > > > > ## Choosing git forge to host source-git repositories > > > > > > We need to find a home for all the source-git repositories. This is > > > > > > actually a really hard task because we have many options > > > > > > (github.com, > > > > > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > > > > > on-premise) and different expectations: some projects already have > > > > > > repos set up on different platforms while Pagure is the primary > > > > > > forge > > > > > > now. Since the CPE team is investigating GitLab as a forge, it's > > > > > > even > > > > > > harder for us to figure out the primary forge. We may end up > > > > > > supporting both actually: pagure.io and gitlab.com. What are your > > > > > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > > > > > More info: > > > > > > * https://pagure.io/fedora-source-git/sig/issue/1 > > > > > > * https://pagure.io/fedora-source-git/sig/issue/7 > > > > > > > > > > I would expect that since the soruce git is just an intermediate > > > > > thing, > > > > > supporting "any forge" is nice to have, but hard. I'd start with some > > > > > common > > > > > options (Pagure + 1 other) and see where you'll get. > > > > > > > > Yep, that's exactly what I'd like us to do. As soon as we have the > > > > service which accepts processes events up and running, it shouldn't be > > > > that hard to teach it new fedora-messaging messages or webhook > > > > payloads. > > > > > > > > > > ## High-level workflow proposal up for review > > > > > > Hunor proposed a high-level workflow linked below and I strongly > > > > > > recommend reading it. We have also started discussing many details > > > > > > in > > > > > > the process, such as getting archives: should we generate one from > > > > > > the > > > > > > source-git repo or use the official release archive from upstream? > > > > > > > > > > One thing to consider is that the upstream tarballs might be > > > > > cryptographically > > > > > signed and packages should verify the signature in %prep. > > > > > > > > This is a very good point - in such a case, we should always pull the > > > > official upstream tarball instead of generating a new one downstream. > > > > > > Hmm, but how would that work? I thought that the whole point of > > > source-git is to build from a commit that contains some upstream > > > version + downstream patches + downstream distro config like the spec > > > file, > > > and that the build step of applying downstream patches just doesn't exist. > > > If you reuse the upstream tarball, then by necessity those downstream > > > patches need to be serialized and applied on top like in dist-git. So > > > you lose the property that the checkout from version control is *the* > > > source you build, but you also lose the property that the source you > > > build is what was signed (since patches are applied). > > > > What you just described I understand as an (upstream) maintenance > > branch and is not what we call source-git. > > It's an "upstream maintenance branch" PLUS the .spec bits that specify > how to build a package. > > > Our definition of source-git is: > > * upstream release tarball > > * downstream code changes applied as patches during %prep > > * additional configs for sake of building and testing > > But that describes dist-git *exactly*. Where is the difference? > In the spirit of openness that Fedora is based upon, it must be very easy to determine what patches we carry as opposed to upstream, and exactly how we build things. This does not go away even if we get to the point where dist-git no longer exists, and we can build directly from source-git. In the meantime, dist-git is the "canonical source of truth" for fedora packages, so there is a middle step to convert from source-git to dist-git. With both, the difference in using source-git means there is an exploded tree there, with patches applied. This makes debug and development much easier. There is also git history, which hopefully contains a lot more background than currently exists in dist-git. Finally, it makes bisecting a much more simple process, whether the offending patch is in upstream or downstream changes. Justin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
Re: Fedora Source-git SIG report #1 (June 2021)
Randy, thank you for providing such detailed information! A few comments inline. On Wed, Jun 30, 2021 at 3:01 AM Randy Barlow via devel < devel@lists.fedoraproject.org> wrote: > On Tue, 2021-06-29 at 15:36 +0200, Tomas Tomecek wrote: > > > On the "uni-repo" counter proposal: there's a bunch of real-world > > > examples here of distributions using this successfully: > > > > > https://github.com/projectatomic/rpmdistro-gitoverlay/blob/master/doc/reworking-fedora-releng.md#unified-source-and-pr-driven-workflow > > Hey Tomas! > > As Colin noted above, Gentoo uses a single repo (well, technically, > they also have overlays, but the main distro is just one git repo is > what I mean…) I'll add some comments to your thoughts below: > After reading your explanation of how gentoo does packaging, it indeed makes a lot of sense and feels like that most of the concerns I pointed out have solutions or could be mitigated. The biggest problem I personally see is the effort which would be needed to pull this off: it would take years to accomplish, a ton of teams would be involved and all the maintainers would need to cooperate - and then we'd need to do the same thing for CentOS Stream and RHEL. Such a change to the development workflow would be massive and the level of coordination would be immense. That's all I can say. * On a system with an ssd I hadn't pulled for two days, and a git > pull took 0m3.357s. > * On a system with a spinny disk I hadn't pulled since January 17, > and a git pull took 1m27.878s. I'd say that this is probably close > to a "worst case" scenario, except that I do have a 200 Mbps > internet connection. Most of the time was spent doing stuff with > the disk and not on network. I could imagine a slow network making > this even longer. > Since we all would work in a single git repo, I'm assuming the pulls would be cheap most of the time. CI and builders would probably need a caching mechanism so the repo wouldn't need to be fetched from scratch all the time. > * How would CI function? > > Due to the atomic commit, I'd say CI could function better because you > know what changes need to be tested together. > > Gentoo doesn't have per-package CI that I'm aware of, but they do have > general checks that they do on all packages. They've got a QA bot that > check PRs on GitHub and marks them as pass/fail. > > > * Where and how would tests be stored? > > As said, I don't think Gentoo does this, but I could imagine having a > tests/ subfolder in the package's folder, not too different from what > we do now. > Well, tests could be stored alongside packaging files as they are right now. > * How would contributions be handled? It would be hard to detect > > stalled PRs, maintainers would be swarmed with changes not related to > > their packages. > > Here's an example of a Gentoo PR I worked on recently: > > https://github.com/gentoo/gentoo/pull/20975 > > Note that there are two bots that have commented on it. The interesting > one for this question is gentoo-bot - it came and marked the PR with > some metadata, helpful links into the bug tracker, CoC, and other > stuff, and most usefully, labeled the PR with some special labels. > > If Fedora had such a bot you could imagine it doing things like > assigning it to the right person (or otherwise notifying them), pinging > long-open PRs, or other things like that. > > The other bot on that PR is the Gentoo QA bot, and it does some basic > checks on your commit to make sure you followed some basic > rules/guidelines. It's not the same thing as what we have in Fedora. > > We do have PRs in Fedora that stay open for a long time too, so I don't > think that's a monorepo vs. lots-of-repos problem ☺ > Looks like the gentoo-bot is a critical piece of the development process so that people are able to progress with their contributions efficiently. Oh, and one other thought, Gentoo is not doing what we call source-git > either. Like us, the packages operate on a tarball, typically from > upstream, and then they apply patches to them as needed. I think it > would be probably too extreme to do a source-git thing with a monorepo > (like, if the kernel source code and the Firefox source code and the > GNOME source code were all in the same repo, that would probably be a > bit much ☺) > Monorepo source-git wouldn't work, our SSDs are not that big, yet :D Thanks Randy for such a great write-up. Now that I think about it, the monorepo proposal is orthogonal to the source-git proposal as the two workflows serve different purposes: one is meant for development (source-git: write a patch), the other one for integration into the operating system (tinker with a spec file or the build system). Tomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct:
Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, Jun 30, 2021 at 10:40 AM Zbigniew Jędrzejewski-Szmek wrote: > > > Our definition of source-git is: > > * upstream release tarball > > * downstream code changes applied as patches during %prep > > * additional configs for sake of building and testing > > But that describes dist-git *exactly*. Where is the difference? Upstream git history and downstream patches tracked as git commits instead of files. > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Tue, Jun 29, 2021 at 3:38 PM Tomas Tomecek wrote: > * Can you imagine maintaining Fedora's 30k+ packages in a single repo? > Without some git-fetch magic it would be unbearable to perform a > git-pull. I cannot imagine *not* doing this, maintaining a distro and preserving any sanity in the process. I cannot imagine having all your updates based on a moving target. I cannot imagine why have side-tags, why decouple building from committing, why make mass-rebuilds a big thing instead of a topical branch they are and why make composes a big thing and not just a commit in a repo of 'the state of Fedora'. Composing this email takes about the same time (3m) as cloning [1], the second largest package collection on repology, at full depth. The tip of it, which the users actually need to download when they update, weights <25M. --depth=1 clone takes 10 seconds. [1] time git clone g...@github.com:NixOS/nixpkgs ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Tue, Jun 29, 2021 at 03:13:10PM +0200, Tomas Tomecek wrote: > On Thu, Jun 24, 2021 at 8:09 PM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: > > > On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok wrote: > > > > > > > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > > > > ## Choosing git forge to host source-git repositories > > > > > We need to find a home for all the source-git repositories. This is > > > > > actually a really hard task because we have many options (github.com, > > > > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > > > > on-premise) and different expectations: some projects already have > > > > > repos set up on different platforms while Pagure is the primary forge > > > > > now. Since the CPE team is investigating GitLab as a forge, it's even > > > > > harder for us to figure out the primary forge. We may end up > > > > > supporting both actually: pagure.io and gitlab.com. What are your > > > > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > > > > More info: > > > > > * https://pagure.io/fedora-source-git/sig/issue/1 > > > > > * https://pagure.io/fedora-source-git/sig/issue/7 > > > > > > > > I would expect that since the soruce git is just an intermediate thing, > > > > supporting "any forge" is nice to have, but hard. I'd start with some > > > > common > > > > options (Pagure + 1 other) and see where you'll get. > > > > > > Yep, that's exactly what I'd like us to do. As soon as we have the > > > service which accepts processes events up and running, it shouldn't be > > > that hard to teach it new fedora-messaging messages or webhook > > > payloads. > > > > > > > > ## High-level workflow proposal up for review > > > > > Hunor proposed a high-level workflow linked below and I strongly > > > > > recommend reading it. We have also started discussing many details in > > > > > the process, such as getting archives: should we generate one from the > > > > > source-git repo or use the official release archive from upstream? > > > > > > > > One thing to consider is that the upstream tarballs might be > > > > cryptographically > > > > signed and packages should verify the signature in %prep. > > > > > > This is a very good point - in such a case, we should always pull the > > > official upstream tarball instead of generating a new one downstream. > > > > Hmm, but how would that work? I thought that the whole point of > > source-git is to build from a commit that contains some upstream > > version + downstream patches + downstream distro config like the spec file, > > and that the build step of applying downstream patches just doesn't exist. > > If you reuse the upstream tarball, then by necessity those downstream > > patches need to be serialized and applied on top like in dist-git. So > > you lose the property that the checkout from version control is *the* > > source you build, but you also lose the property that the source you > > build is what was signed (since patches are applied). > > What you just described I understand as an (upstream) maintenance > branch and is not what we call source-git. It's an "upstream maintenance branch" PLUS the .spec bits that specify how to build a package. > Our definition of source-git is: > * upstream release tarball > * downstream code changes applied as patches during %prep > * additional configs for sake of building and testing But that describes dist-git *exactly*. Where is the difference? Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
This is a super update report, thanks for sharing CPE have activated an open source license for Gitlab to trial out some of the workflows and features. This is their top level SaaS offering which they grant to all open source projects so it has all the features we would need for testing. Stephen (on CC) and I have admin access. I'd love to grant folks in the SIG access, if someone can ping me a list of emails that are associated with a gitlab.com account (you need to register!) I can make that happen. Leigh On Thu 24 Jun 2021, 10:17 Tomas Tomecek, wrote: > Greetings from the Fedora source-git SIG! We are planning to start > publishing reports of what we are working on so everyone can easily > pay attention and get involved if interested. If you have any ideas, > comments or requests, don’t be shy and let us know :) > > Here’s a short list of things which we are working on. > > ## Choosing git forge to host source-git repositories > We need to find a home for all the source-git repositories. This is > actually a really hard task because we have many options (github.com, > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > on-premise) and different expectations: some projects already have > repos set up on different platforms while Pagure is the primary forge > now. Since the CPE team is investigating GitLab as a forge, it's even > harder for us to figure out the primary forge. We may end up > supporting both actually: pagure.io and gitlab.com. What are your > thoughts on this topic? Would you prefer pagure.io or gitlab.com > More info: > * https://pagure.io/fedora-source-git/sig/issue/1 > * https://pagure.io/fedora-source-git/sig/issue/7 > > ## High-level workflow proposal up for review > Hunor proposed a high-level workflow linked below and I strongly > recommend reading it. We have also started discussing many details in > the process, such as getting archives: should we generate one from the > source-git repo or use the official release archive from upstream? > > Another big topic in terms of workflows are rebases (= updates to the > latest upstream release, which are very common in Rawhide). Rebases > are straightforward in dist-git, but when your source-git repo has > complete upstream git history, they are no longer trivial, especially > if one wants to get a review of a rebase. > > More info: > * https://pagure.io/fedora-source-git/sig/issue/2 > * Workflow proposal: > https://pagure.io/fedora-source-git/docs/pull-request/2 > * > https://pagure.io/fedora-source-git/docs/blob/main/f/resources/CommitRules.pdf > * https://pagure.io/fedora-source-git/sig/issue/8 > > ## Tooling > Packit is the tooling which will be used to work with source-git > repos. No surprise there I assume :D > * https://packit.dev/ > > We've done a lot of work here lately, mainly to polish the process of > creating source-git repos and doing updates of dist-git repositories > based on the source-git content. > * https://packit.dev/docs/source-git/work-with-source-git/ > * https://github.com/packit/packit/releases > > ## Interested? > We meet biweekly on Wednesdays via gmeet, 2:30 - 3:30 UTC, next one is > scheduled for July 7th. > * https://calendar.fedoraproject.org/SIGs/2021/7/5/#m9982 > > Everyone is welcome to join the SIG or provide any feedback on the > issues and PRs above. > > You can always find the latest information over here: > * https://fedoraproject.org/wiki/SIGs/Source-git > * https://pagure.io/fedora-source-git/sig/issues > > I'd like to thank all the SIG members for being so active, so happy to > work with all of you! > > Cheers, > Tomas > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Wed, 2021-06-30 at 01:01 +, Randy Barlow via devel wrote: > An example of this is gstreamer - in Gentoo I just have > gstreamer installed, and the various plugin packs like good bad and > ugly are just USE variables on that one package. Ooops, I was wrong on this example. They do package good, bad, ugly as separate packages. signature.asc Description: This is a digitally signed message part ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Tue, 2021-06-29 at 15:36 +0200, Tomas Tomecek wrote: > > On the "uni-repo" counter proposal: there's a bunch of real-world > > examples here of distributions using this successfully: > > https://github.com/projectatomic/rpmdistro-gitoverlay/blob/master/doc/reworking-fedora-releng.md#unified-source-and-pr-driven-workflow Hey Tomas! As Colin noted above, Gentoo uses a single repo (well, technically, they also have overlays, but the main distro is just one git repo is what I mean…) I'll add some comments to your thoughts below: > * Can you imagine maintaining Fedora's 30k+ packages in a single repo? > Without some git-fetch magic it would be unbearable to perform a git-pull. Gentoo has 19,302 packages according to their packages web app[0]. Some Gentoo packages map to more than one Fedora package, since Gentoo has slots and uses USE flags. For example, there's just one package called "python", but you can get various Python versions from the same name (vs how we have a separate package for each Python version in Fedora). Programs with lots of plugins end up being one package that has USE flags for the plugins, where in Fedora the plugins might be packages separately. An example of this is gstreamer - in Gentoo I just have gstreamer installed, and the various plugin packs like good bad and ugly are just USE variables on that one package. I mention these differences to say that it's probably fair enough to compare the set to Fedora's 30k. I will say that a git pull does take some time, especially if you haven't run it very recently. It's not unusable, but you do notice it for sure. I just ran a quick test on two different systems with wildly different results: * On a system with an ssd I hadn't pulled for two days, and a git pull took 0m3.357s. * On a system with a spinny disk I hadn't pulled since January 17, and a git pull took 1m27.878s. I'd say that this is probably close to a "worst case" scenario, except that I do have a 200 Mbps internet connection. Most of the time was spent doing stuff with the disk and not on network. I could imagine a slow network making this even longer. I would say this is the top downside to the monorepo approach that I personally experience. But I also tend to use the SSD system for this, and also tend to pull often enough that it's not a big deal. > * Git history would be immensely bloated (though `git log $path` > would work well). Yeah I always just use git log $path myself, and it works fine. I also think it's kinda cool that I can tail a single git log and see what people have been up to across the project, but we can also do that with the message broker in Fedora. > * On the other hand, I like that changes could be done "atomically" > in a single commit, even for multiple packages. I *love* this, and to me it's the best thing about the design. If you have a change to one package that requires a corresponding change in others, you just do it in a single commit. In Fedora we don't have an easy automatic process for this. We *could* build one, but it would be harder to do. With a monorepo, it's just something you get for free. > * How would CI function? Due to the atomic commit, I'd say CI could function better because you know what changes need to be tested together. Gentoo doesn't have per-package CI that I'm aware of, but they do have general checks that they do on all packages. They've got a QA bot that check PRs on GitHub and marks them as pass/fail. > * Where and how would tests be stored? As said, I don't think Gentoo does this, but I could imagine having a tests/ subfolder in the package's folder, not too different from what we do now. > * As Neal pointed out, ACL mechanics would need to be thought > through. GitHub does have a codeowners feature, which I think could be adapted for this purpose. One could imagine a server-side git push hook that checks an ACL rule list against the paths that were altered. I think such a thing could probably be implemented in not GitHub as well. I wouldn't say it's trivial to do so, but not extremely difficult either. > * src.fp.o and koji would need to be updated, significantly. Agreed. > * How would contributions be handled? It would be hard to detect > stalled PRs, maintainers would be swarmed with changes not related to > their packages. Here's an example of a Gentoo PR I worked on recently: https://github.com/gentoo/gentoo/pull/20975 Note that there are two bots that have commented on it. The interesting one for this question is gentoo-bot - it came and marked the PR with some metadata, helpful links into the bug tracker, CoC, and other stuff, and most usefully, labeled the PR with some special labels. If Fedora had such a bot you could imagine it doing things like assigning it to the right person (or otherwise notifying them), pinging long-open PRs, or other things like that. The other bot on that PR is the Gentoo QA bot, and it does some basic checks on your
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 7:52 AM Neal Gompa wrote: > > On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > > > > > > > > > On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" > > > wrote: > > > >On 24. 06. 21 23:07, Miroslav Suchý wrote: > > > >> Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > > > One thing to consider is that the upstream tarballs might be > > > >cryptographically > > > signed and packages should verify the signature in %prep. > > > >>> This is a very good point - in such a case, we should always pull > > > >the > > > >>> official upstream tarball instead of generating a new one downstream > > > >> > > > >> Does it matter? If you are able to generate byte2byte identical > > > >tarball then > > > >> you can choose any of them. > > > > > > > >AFAIK git does not grantee to produce byte2byte identical archives > > > >across > > > >different versions of git, zlib, gzip etc. So even if upstream signs > > > >the git > > > >generated archive, generating a byte2byte identical one might be > > > >tricky. > > > > > > Especially with xz, which iirc has reproducibility issues in parallel > > > mode. > > > > I think we should try to push upstream to sign git tags, instead or in > > addition to tarballs. For upstreams, this is actually much easier > > (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > > a tarball on github which requires some interaction with the web service. > > > > As an upstream, I would literally *never* GPG sign git tags. If you > ask me to do that, I won't. It's far too annoying to deal with for me > to be willing to suffer through that. > > I'm not going to ask people to do something I would be unwilling to do myself. I am not sure what method you are using to sign things, but git GPG integration is easy, and generally gets out of the way. Justin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Tue, Jun 29, 2021 at 12:48 AM Colin Walters wrote: > > > > On Thu, Jun 24, 2021, at 5:16 AM, Tomas Tomecek wrote: > > Greetings from the Fedora source-git SIG! We are planning to start > > publishing reports of what we are working on so everyone can easily > > pay attention and get involved if interested. If you have any ideas, > > comments or requests, don’t be shy and let us know :) > > I really appreciate your efforts here. I think most of us know > that the current RPM workflow is very antiquated. > > However, two things: > > - The representation for how we maintain source code is a > very long-term commitment. It's hard to maintain multiple > of them. Related to that, the details of how this works > will quickly become "frozen" and hard to change, so > it's important to get it right from the start. Related > to that: Yes, we can see that happening right now: doing some fundamental changes to the current development process takes months/years/Ø. > - I think this proposal only benefits very few packages, > mainly kernel/grub where the upstream is large > and we carry nontrivial deltas. For most others, > it will either be neutral or worse. So...my counter > proposal is to iterate closer to a NixOS/OpenEmbedded style "uni-repo" > model, leaving these special cases to basically become > forked upstream repositories. Instead of carrying 208+ > patches to grub, we'd have a separate git repo that gets rebased. > Which...is how it already works at > https://github.com/rhboot/grub2/tree/fedora-35 > it looks like. > > On the "uni-repo" counter proposal: there's a bunch of real-world examples > here of distributions using this successfully: > https://github.com/projectatomic/rpmdistro-gitoverlay/blob/master/doc/reworking-fedora-releng.md#unified-source-and-pr-driven-workflow This is a nice write-up and the idea is definitely appealing but I can't personally imagine how would this scale in Fedora: * Can you imagine maintaining Fedora's 30k+ packages in a single repo? Without some git-fetch magic it would be unbearable to perform a git-pull. * Git history would be immensely bloated (though `git log $path` would work well). * On the other hand, I like that changes could be done "atomically" in a single commit, even for multiple packages. * How would CI function? * Where and how would tests be stored? * As Neal pointed out, ACL mechanics would need to be thought through. * src.fp.o and koji would need to be updated, significantly. * How would contributions be handled? It would be hard to detect stalled PRs, maintainers would be swarmed with changes not related to their packages. Would be awesome to get a PoC of this uni-repo approach. Tomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 8:09 PM Zbigniew Jędrzejewski-Szmek wrote: > > On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: > > On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok wrote: > > > > > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > > > ## Choosing git forge to host source-git repositories > > > > We need to find a home for all the source-git repositories. This is > > > > actually a really hard task because we have many options (github.com, > > > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > > > on-premise) and different expectations: some projects already have > > > > repos set up on different platforms while Pagure is the primary forge > > > > now. Since the CPE team is investigating GitLab as a forge, it's even > > > > harder for us to figure out the primary forge. We may end up > > > > supporting both actually: pagure.io and gitlab.com. What are your > > > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > > > More info: > > > > * https://pagure.io/fedora-source-git/sig/issue/1 > > > > * https://pagure.io/fedora-source-git/sig/issue/7 > > > > > > I would expect that since the soruce git is just an intermediate thing, > > > supporting "any forge" is nice to have, but hard. I'd start with some > > > common > > > options (Pagure + 1 other) and see where you'll get. > > > > Yep, that's exactly what I'd like us to do. As soon as we have the > > service which accepts processes events up and running, it shouldn't be > > that hard to teach it new fedora-messaging messages or webhook > > payloads. > > > > > > ## High-level workflow proposal up for review > > > > Hunor proposed a high-level workflow linked below and I strongly > > > > recommend reading it. We have also started discussing many details in > > > > the process, such as getting archives: should we generate one from the > > > > source-git repo or use the official release archive from upstream? > > > > > > One thing to consider is that the upstream tarballs might be > > > cryptographically > > > signed and packages should verify the signature in %prep. > > > > This is a very good point - in such a case, we should always pull the > > official upstream tarball instead of generating a new one downstream. > > Hmm, but how would that work? I thought that the whole point of > source-git is to build from a commit that contains some upstream > version + downstream patches + downstream distro config like the spec file, > and that the build step of applying downstream patches just doesn't exist. > If you reuse the upstream tarball, then by necessity those downstream > patches need to be serialized and applied on top like in dist-git. So > you lose the property that the checkout from version control is *the* > source you build, but you also lose the property that the source you > build is what was signed (since patches are applied). What you just described I understand as an (upstream) maintenance branch and is not what we call source-git. Our definition of source-git is: * upstream release tarball * downstream code changes applied as patches during %prep * additional configs for sake of building and testing It's not that one workflow is better than the other: teams tend to pick one and use it to maintain their project based on their preference and ease of use. I hope we'd be able to support both workflows but right now we focus on the source-git part. HTH, Tomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Mon, Jun 28, 2021 at 6:48 PM Colin Walters wrote: > > > > On Thu, Jun 24, 2021, at 5:16 AM, Tomas Tomecek wrote: > > Greetings from the Fedora source-git SIG! We are planning to start > > publishing reports of what we are working on so everyone can easily > > pay attention and get involved if interested. If you have any ideas, > > comments or requests, don’t be shy and let us know :) > > I really appreciate your efforts here. I think most of us know > that the current RPM workflow is very antiquated. > > However, two things: > > - The representation for how we maintain source code is a > very long-term commitment. It's hard to maintain multiple > of them. Related to that, the details of how this works > will quickly become "frozen" and hard to change, so > it's important to get it right from the start. Related > to that: > - I think this proposal only benefits very few packages, > mainly kernel/grub where the upstream is large > and we carry nontrivial deltas. For most others, > it will either be neutral or worse. So...my counter > proposal is to iterate closer to a NixOS/OpenEmbedded style "uni-repo" > model, leaving these special cases to basically become > forked upstream repositories. Instead of carrying 208+ > patches to grub, we'd have a separate git repo that gets rebased. > Which...is how it already works at > https://github.com/rhboot/grub2/tree/fedora-35 > it looks like. > > On the "uni-repo" counter proposal: there's a bunch of real-world examples > here of distributions using this successfully: > https://github.com/projectatomic/rpmdistro-gitoverlay/blob/master/doc/reworking-fedora-releng.md#unified-source-and-pr-driven-workflow The monorepo model scales very badly once you have a huge number of contributors. And having per-package ACLs is basically toast in a monorepo. Note that AUR doesn't use a monorepo, they map each SVN folder to a Git branch instead. It's effectively thousands of Git repos smushed into one super-repo with thousands of branches. Having experienced monorepo models for packaging before, it's rife with issues and makes tracking history super-hard. In general, I think a monorepo model is a horrible idea and should never be considered for Fedora. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021, at 5:16 AM, Tomas Tomecek wrote: > Greetings from the Fedora source-git SIG! We are planning to start > publishing reports of what we are working on so everyone can easily > pay attention and get involved if interested. If you have any ideas, > comments or requests, don’t be shy and let us know :) I really appreciate your efforts here. I think most of us know that the current RPM workflow is very antiquated. However, two things: - The representation for how we maintain source code is a very long-term commitment. It's hard to maintain multiple of them. Related to that, the details of how this works will quickly become "frozen" and hard to change, so it's important to get it right from the start. Related to that: - I think this proposal only benefits very few packages, mainly kernel/grub where the upstream is large and we carry nontrivial deltas. For most others, it will either be neutral or worse. So...my counter proposal is to iterate closer to a NixOS/OpenEmbedded style "uni-repo" model, leaving these special cases to basically become forked upstream repositories. Instead of carrying 208+ patches to grub, we'd have a separate git repo that gets rebased. Which...is how it already works at https://github.com/rhboot/grub2/tree/fedora-35 it looks like. On the "uni-repo" counter proposal: there's a bunch of real-world examples here of distributions using this successfully: https://github.com/projectatomic/rpmdistro-gitoverlay/blob/master/doc/reworking-fedora-releng.md#unified-source-and-pr-driven-workflow ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Dne 25. 06. 21 v 16:19 Peter Pentchev napsal(a): FWIW, pristine-tar (http://joeyh.name/code/pristine-tar/) can handle almost all upstream tarballs, and it also has support for storing detached signatures alongside its metadata. I keep hearing people say that there are cases when it fails, but it has worked for me for dozens of packages. Of course, it does have its own expectations about the structure of the Git repository, but those are mostly limited to "give me a branch to play in, I'll take care of the rest". https://www.preining.info/blog/2014/06/debian-pristine-tar-packaging/ How well it is maintained nowadays? I cannot see recent commit in past few years. Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 10:43:17AM -0400, Neal Gompa wrote: > On Fri, Jun 25, 2021 at 9:04 AM Frédéric Pierret > wrote: > > > > > > > > Le 6/25/21 à 2:51 PM, Neal Gompa a écrit : > > > On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek > > > wrote: > > >> > > >> On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > > >>> > > >>> > > >>> On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" > > >>> wrote: > > On 24. 06. 21 23:07, Miroslav Suchý wrote: > > > Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > > >>> One thing to consider is that the upstream tarballs might be > > cryptographically > > >>> signed and packages should verify the signature in %prep. > > >> This is a very good point - in such a case, we should always pull > > the > > >> official upstream tarball instead of generating a new one downstream > > > > > > Does it matter? If you are able to generate byte2byte identical > > tarball then > > > you can choose any of them. > > > > AFAIK git does not grantee to produce byte2byte identical archives > > across > > different versions of git, zlib, gzip etc. So even if upstream signs > > the git > > generated archive, generating a byte2byte identical one might be > > tricky. > > >>> > > >>> Especially with xz, which iirc has reproducibility issues in parallel > > >>> mode. > > >> > > >> I think we should try to push upstream to sign git tags, instead or in > > >> addition to tarballs. For upstreams, this is actually much easier > > >> (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > > >> a tarball on github which requires some interaction with the web service. > > >> > > > > > > As an upstream, I would literally *never* GPG sign git tags. If you > > > ask me to do that, I won't. It's far too annoying to deal with for me > > > to be willing to suffer through that. > > > > > > I'm not going to ask people to do something I would be unwilling to do > > > myself. > > > > What about only version tags? You could do some git/bash alias to create > > commit version + signed tag at once. For example, we do that on Qubes OS > > and that's not more work that just committing the version. > > > > The problem is that the workflow for tag signatures sucks for Git. And > I'd need to get it registered in the forges or whatever systems are > used to consume and verify signatures. That's Herculean in a way that > I'm unwilling to deal with. I guess I'm missing something. What is hard about doing signed tags? Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 9:04 AM Frédéric Pierret wrote: > > > > Le 6/25/21 à 2:51 PM, Neal Gompa a écrit : > > On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek > > wrote: > >> > >> On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > >>> > >>> > >>> On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" > >>> wrote: > On 24. 06. 21 23:07, Miroslav Suchý wrote: > > Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > >>> One thing to consider is that the upstream tarballs might be > cryptographically > >>> signed and packages should verify the signature in %prep. > >> This is a very good point - in such a case, we should always pull > the > >> official upstream tarball instead of generating a new one downstream > > > > Does it matter? If you are able to generate byte2byte identical > tarball then > > you can choose any of them. > > AFAIK git does not grantee to produce byte2byte identical archives > across > different versions of git, zlib, gzip etc. So even if upstream signs > the git > generated archive, generating a byte2byte identical one might be > tricky. > >>> > >>> Especially with xz, which iirc has reproducibility issues in parallel > >>> mode. > >> > >> I think we should try to push upstream to sign git tags, instead or in > >> addition to tarballs. For upstreams, this is actually much easier > >> (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > >> a tarball on github which requires some interaction with the web service. > >> > > > > As an upstream, I would literally *never* GPG sign git tags. If you > > ask me to do that, I won't. It's far too annoying to deal with for me > > to be willing to suffer through that. > > > > I'm not going to ask people to do something I would be unwilling to do > > myself. > > What about only version tags? You could do some git/bash alias to create > commit version + signed tag at once. For example, we do that on Qubes OS and > that's not more work that just committing the version. > The problem is that the workflow for tag signatures sucks for Git. And I'd need to get it registered in the forges or whatever systems are used to consume and verify signatures. That's Herculean in a way that I'm unwilling to deal with. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 02:40:22PM +0200, Miroslav Suchý wrote: > Dne 24. 06. 21 v 23:22 Miro Hrončok napsal(a): > > AFAIK git does not grantee to produce byte2byte identical archives > > across different versions of git, zlib, gzip etc. So even if upstream > > signs the git generated archive, generating a byte2byte identical one > > might be tricky. > > Neither git nor tar can do that. But it is not impossible. E.g. Tito [1] has > some hacks on top of git-archive which produces identical tar-balls. > > [1] https://github.com/rpm-software-management/tito/ FWIW, pristine-tar (http://joeyh.name/code/pristine-tar/) can handle almost all upstream tarballs, and it also has support for storing detached signatures alongside its metadata. I keep hearing people say that there are cases when it fails, but it has worked for me for dozens of packages. Of course, it does have its own expectations about the structure of the Git repository, but those are mostly limited to "give me a branch to play in, I'll take care of the rest". G'luck, Peter -- Peter Pentchev r...@ringlet.net r...@debian.org p...@storpool.com PGP key:http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13 signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Le 6/25/21 à 2:51 PM, Neal Gompa a écrit : On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek wrote: On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" wrote: On 24. 06. 21 23:07, Miroslav Suchý wrote: Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): One thing to consider is that the upstream tarballs might be cryptographically signed and packages should verify the signature in %prep. This is a very good point - in such a case, we should always pull the official upstream tarball instead of generating a new one downstream Does it matter? If you are able to generate byte2byte identical tarball then you can choose any of them. AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. Especially with xz, which iirc has reproducibility issues in parallel mode. I think we should try to push upstream to sign git tags, instead or in addition to tarballs. For upstreams, this is actually much easier (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing a tarball on github which requires some interaction with the web service. As an upstream, I would literally *never* GPG sign git tags. If you ask me to do that, I won't. It's far too annoying to deal with for me to be willing to suffer through that. I'm not going to ask people to do something I would be unwilling to do myself. What about only version tags? You could do some git/bash alias to create commit version + signed tag at once. For example, we do that on Qubes OS and that's not more work that just committing the version. Best, Frédéric OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On 25. 06. 21 14:54, Miroslav Suchý wrote: Dne 25. 06. 21 v 14:50 Miro Hrončok napsal(a): AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. Neither git nor tar can do that. But it is not impossible. E.g. Tito [1] has some hacks on top of git-archive which produces identical tar-balls. That would require our upstreams to use tito (or similar), which we are in many case unlikely to affect. I do not expect that. I rather meant that it is technically possible and Packit can "steal" that code from Tito. What I meant is that even if Packit has an ability to create reproducible git tarballs, upstream would need to sign Packit-generated tarballs, which would require much higher level of Fedora's involvement to the upstream release process than is usual for independent upstreams. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Dne 25. 06. 21 v 14:50 Miro Hrončok napsal(a): AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. Neither git nor tar can do that. But it is not impossible. E.g. Tito [1] has some hacks on top of git-archive which produces identical tar-balls. That would require our upstreams to use tito (or similar), which we are in many case unlikely to affect. I do not expect that. I rather meant that it is technically possible and Packit can "steal" that code from Tito. Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 3:43 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > > > > > > On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" wrote: > > >On 24. 06. 21 23:07, Miroslav Suchý wrote: > > >> Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > > One thing to consider is that the upstream tarballs might be > > >cryptographically > > signed and packages should verify the signature in %prep. > > >>> This is a very good point - in such a case, we should always pull > > >the > > >>> official upstream tarball instead of generating a new one downstream > > >> > > >> Does it matter? If you are able to generate byte2byte identical > > >tarball then > > >> you can choose any of them. > > > > > >AFAIK git does not grantee to produce byte2byte identical archives > > >across > > >different versions of git, zlib, gzip etc. So even if upstream signs > > >the git > > >generated archive, generating a byte2byte identical one might be > > >tricky. > > > > Especially with xz, which iirc has reproducibility issues in parallel mode. > > I think we should try to push upstream to sign git tags, instead or in > addition to tarballs. For upstreams, this is actually much easier > (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > a tarball on github which requires some interaction with the web service. > As an upstream, I would literally *never* GPG sign git tags. If you ask me to do that, I won't. It's far too annoying to deal with for me to be willing to suffer through that. I'm not going to ask people to do something I would be unwilling to do myself. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On 25. 06. 21 14:40, Miroslav Suchý wrote: Dne 24. 06. 21 v 23:22 Miro Hrončok napsal(a): AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. Neither git nor tar can do that. But it is not impossible. E.g. Tito [1] has some hacks on top of git-archive which produces identical tar-balls. That would require our upstreams to use tito (or similar), which we are in many case unlikely to affect. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Dne 24. 06. 21 v 23:22 Miro Hrončok napsal(a): AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. Neither git nor tar can do that. But it is not impossible. E.g. Tito [1] has some hacks on top of git-archive which produces identical tar-balls. [1] https://github.com/rpm-software-management/tito/ Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 10:51:11AM +0200, Florian Weimer wrote: > * Zbigniew Jędrzejewski-Szmek: > > > I think we should try to push upstream to sign git tags, instead or in > > addition to tarballs. For upstreams, this is actually much easier > > (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > > a tarball on github which requires some interaction with the web service. > > Would anyone recognize these signatures today, given that they are > essentially SHA-1 based? git has a reasonably recent sha1 implementation… It's time to move on, but afaik it isn't currently compromised. The switch to sha256 is ongoing. It seems … stalled (?), but it'll happen sooner or later. So I think we can tell people to sign commits, and then at some point in the future this will magically work with sha256. > Fedora has already disabled SHA-1 signatures by default elsewhere, so > such an encouragement would just result in wasted work. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
* Zbigniew Jędrzejewski-Szmek: > I think we should try to push upstream to sign git tags, instead or in > addition to tarballs. For upstreams, this is actually much easier > (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing > a tarball on github which requires some interaction with the web service. Would anyone recognize these signatures today, given that they are essentially SHA-1 based? Fedora has already disabled SHA-1 signatures by default elsewhere, so such an encouragement would just result in wasted work. Thanks, Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 11:07:45PM +0200, Miroslav Suchý wrote: > Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > > > One thing to consider is that the upstream tarballs might be > > > cryptographically > > > signed and packages should verify the signature in %prep. > > This is a very good point - in such a case, we should always pull the > > official upstream tarball instead of generating a new one downstream > > Does it matter? If you are able to generate byte2byte identical > tarball then you can choose any of them. If the upstream tarball contains generated files, chances of re-creating bytewise identical tarball is low, and for autotools based projects it is essentially zero. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Fri, Jun 25, 2021 at 03:49:23AM +, Dan Čermák wrote: > > > On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" wrote: > >On 24. 06. 21 23:07, Miroslav Suchý wrote: > >> Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > One thing to consider is that the upstream tarballs might be > >cryptographically > signed and packages should verify the signature in %prep. > >>> This is a very good point - in such a case, we should always pull > >the > >>> official upstream tarball instead of generating a new one downstream > >> > >> Does it matter? If you are able to generate byte2byte identical > >tarball then > >> you can choose any of them. > > > >AFAIK git does not grantee to produce byte2byte identical archives > >across > >different versions of git, zlib, gzip etc. So even if upstream signs > >the git > >generated archive, generating a byte2byte identical one might be > >tricky. > > Especially with xz, which iirc has reproducibility issues in parallel mode. I think we should try to push upstream to sign git tags, instead or in addition to tarballs. For upstreams, this is actually much easier (just 'git tag' → 'git tag -s' and you're done) compared to e.g. signing a tarball on github which requires some interaction with the web service. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On June 24, 2021 6:08:17 PM UTC, "Zbigniew Jędrzejewski-Szmek" wrote: >On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: >> On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok >wrote: >> > >> > On 24. 06. 21 11:16, Tomas Tomecek wrote: >> > > ## Choosing git forge to host source-git repositories >> > > We need to find a home for all the source-git repositories. This >is >> > > actually a really hard task because we have many options >(github.com, >> > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or >> > > on-premise) and different expectations: some projects already >have >> > > repos set up on different platforms while Pagure is the primary >forge >> > > now. Since the CPE team is investigating GitLab as a forge, it's >even >> > > harder for us to figure out the primary forge. We may end up >> > > supporting both actually: pagure.io and gitlab.com. What are your >> > > thoughts on this topic? Would you prefer pagure.io or gitlab.com >> > > More info: >> > > * https://pagure.io/fedora-source-git/sig/issue/1 >> > > * https://pagure.io/fedora-source-git/sig/issue/7 >> > >> > I would expect that since the soruce git is just an intermediate >thing, >> > supporting "any forge" is nice to have, but hard. I'd start with >some common >> > options (Pagure + 1 other) and see where you'll get. >> >> Yep, that's exactly what I'd like us to do. As soon as we have the >> service which accepts processes events up and running, it shouldn't >be >> that hard to teach it new fedora-messaging messages or webhook >> payloads. >> >> > > ## High-level workflow proposal up for review >> > > Hunor proposed a high-level workflow linked below and I strongly >> > > recommend reading it. We have also started discussing many >details in >> > > the process, such as getting archives: should we generate one >from the >> > > source-git repo or use the official release archive from >upstream? >> > >> > One thing to consider is that the upstream tarballs might be >cryptographically >> > signed and packages should verify the signature in %prep. >> >> This is a very good point - in such a case, we should always pull the >> official upstream tarball instead of generating a new one downstream. > >Hmm, but how would that work? I thought that the whole point of >source-git is to build from a commit that contains some upstream >version + downstream patches + downstream distro config like the spec >file, >and that the build step of applying downstream patches just doesn't >exist. >If you reuse the upstream tarball, then by necessity those downstream >patches need to be serialized and applied on top like in dist-git. So >you lose the property that the checkout from version control is *the* >source you build, but you also lose the property that the source you >build is what was signed (since patches are applied). But if the tooling automatically creates the "correct" srpm for you (e.g. either upstream tarball + patches or a git tarball), then I'd consider that a valuable improvement already. Sure, it might not be perfect, but it would help. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On June 24, 2021 9:22:51 PM UTC, "Miro Hrončok" wrote: >On 24. 06. 21 23:07, Miroslav Suchý wrote: >> Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): One thing to consider is that the upstream tarballs might be >cryptographically signed and packages should verify the signature in %prep. >>> This is a very good point - in such a case, we should always pull >the >>> official upstream tarball instead of generating a new one downstream >> >> Does it matter? If you are able to generate byte2byte identical >tarball then >> you can choose any of them. > >AFAIK git does not grantee to produce byte2byte identical archives >across >different versions of git, zlib, gzip etc. So even if upstream signs >the git >generated archive, generating a byte2byte identical one might be >tricky. Especially with xz, which iirc has reproducibility issues in parallel mode. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021, at 5:22 PM, Miro Hrončok wrote: > On 24. 06. 21 23:07, Miroslav Suchý wrote: > > Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): > >>> One thing to consider is that the upstream tarballs might be > >>> cryptographically > >>> signed and packages should verify the signature in %prep. > >> This is a very good point - in such a case, we should always pull the > >> official upstream tarball instead of generating a new one downstream > > > > Does it matter? If you are able to generate byte2byte identical tarball > > then > > you can choose any of them. > > AFAIK git does not grantee to produce byte2byte identical archives across > different versions of git, zlib, gzip etc. So even if upstream signs the git > generated archive, generating a byte2byte identical one might be tricky. See also https://github.com/cgwalters/git-evtag ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On 24. 06. 21 23:07, Miroslav Suchý wrote: Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): One thing to consider is that the upstream tarballs might be cryptographically signed and packages should verify the signature in %prep. This is a very good point - in such a case, we should always pull the official upstream tarball instead of generating a new one downstream Does it matter? If you are able to generate byte2byte identical tarball then you can choose any of them. AFAIK git does not grantee to produce byte2byte identical archives across different versions of git, zlib, gzip etc. So even if upstream signs the git generated archive, generating a byte2byte identical one might be tricky. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Dne 24. 06. 21 v 15:48 Tomas Tomecek napsal(a): One thing to consider is that the upstream tarballs might be cryptographically signed and packages should verify the signature in %prep. This is a very good point - in such a case, we should always pull the official upstream tarball instead of generating a new one downstream Does it matter? If you are able to generate byte2byte identical tarball then you can choose any of them. Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 03:48:54PM +0200, Tomas Tomecek wrote: > On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok wrote: > > > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > > ## Choosing git forge to host source-git repositories > > > We need to find a home for all the source-git repositories. This is > > > actually a really hard task because we have many options (github.com, > > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > > on-premise) and different expectations: some projects already have > > > repos set up on different platforms while Pagure is the primary forge > > > now. Since the CPE team is investigating GitLab as a forge, it's even > > > harder for us to figure out the primary forge. We may end up > > > supporting both actually: pagure.io and gitlab.com. What are your > > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > > More info: > > > * https://pagure.io/fedora-source-git/sig/issue/1 > > > * https://pagure.io/fedora-source-git/sig/issue/7 > > > > I would expect that since the soruce git is just an intermediate thing, > > supporting "any forge" is nice to have, but hard. I'd start with some common > > options (Pagure + 1 other) and see where you'll get. > > Yep, that's exactly what I'd like us to do. As soon as we have the > service which accepts processes events up and running, it shouldn't be > that hard to teach it new fedora-messaging messages or webhook > payloads. > > > > ## High-level workflow proposal up for review > > > Hunor proposed a high-level workflow linked below and I strongly > > > recommend reading it. We have also started discussing many details in > > > the process, such as getting archives: should we generate one from the > > > source-git repo or use the official release archive from upstream? > > > > One thing to consider is that the upstream tarballs might be > > cryptographically > > signed and packages should verify the signature in %prep. > > This is a very good point - in such a case, we should always pull the > official upstream tarball instead of generating a new one downstream. Hmm, but how would that work? I thought that the whole point of source-git is to build from a commit that contains some upstream version + downstream patches + downstream distro config like the spec file, and that the build step of applying downstream patches just doesn't exist. If you reuse the upstream tarball, then by necessity those downstream patches need to be serialized and applied on top like in dist-git. So you lose the property that the checkout from version control is *the* source you build, but you also lose the property that the source you build is what was signed (since patches are applied). Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 1:01 PM PGNet Dev wrote: > > On 6/24/21 6:40 AM, Miro Hrončok wrote: > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > >> ## Choosing git forge to host source-git repositories > >> We need to find a home for all the source-git repositories. This is > >> actually a really hard task because we have many options (github.com, > >> gitlab.com, pagure.io, src.fedoraproject.org, something custom or > >> on-premise) and different expectations: some projects already have > >> repos set up on different platforms while Pagure is the primary forge > >> now. Since the CPE team is investigating GitLab as a forge, it's even > >> harder for us to figure out the primary forge. We may end up > >> supporting both actually: pagure.io and gitlab.com. What are your > >> thoughts on this topic? Would you prefer pagure.io or gitlab.com > >> More info: > >> * https://pagure.io/fedora-source-git/sig/issue/1 > >> * https://pagure.io/fedora-source-git/sig/issue/7 > > Indirectly related: > > my scm-sourced COPR projects often pull from git repos (upstream @ Fedora src > projects), and use forgemeta macros in rpm config. > > ime, forgemeta had lots of issues in the past with gitlab source matching > when pulling specific tags/commits, requiring customized source strings -- > usually after a bunch of trial-n-error. > > github & pagure had no such problems. > > to work around the challenges, I 1st mirrored gitlab repos to pagure, then > pulled from there in my COPR specs, originally specifying commits/tags. > > I haven't revisited gitlab for a fairly long while to check again. > > Neither have I tested forgemeta's newer support for packaging a branch state > > > https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/#_branch_example > > which is now available, stable & terribly convenient -- for github & pagure. > > TL;DR in this particular case: as long as it plays nicely with COPR forgemeta > source-reference macros, no preference Thank you for your suggestion! I don't have any experience with those macros myself so can't comment. I created an issue for us to investigate further: https://pagure.io/fedora-source-git/sig/issue/12 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 12:41 PM Miro Hrončok wrote: > > On 24. 06. 21 11:16, Tomas Tomecek wrote: > > ## Choosing git forge to host source-git repositories > > We need to find a home for all the source-git repositories. This is > > actually a really hard task because we have many options (github.com, > > gitlab.com, pagure.io, src.fedoraproject.org, something custom or > > on-premise) and different expectations: some projects already have > > repos set up on different platforms while Pagure is the primary forge > > now. Since the CPE team is investigating GitLab as a forge, it's even > > harder for us to figure out the primary forge. We may end up > > supporting both actually: pagure.io and gitlab.com. What are your > > thoughts on this topic? Would you prefer pagure.io or gitlab.com > > More info: > > * https://pagure.io/fedora-source-git/sig/issue/1 > > * https://pagure.io/fedora-source-git/sig/issue/7 > > I would expect that since the soruce git is just an intermediate thing, > supporting "any forge" is nice to have, but hard. I'd start with some common > options (Pagure + 1 other) and see where you'll get. Yep, that's exactly what I'd like us to do. As soon as we have the service which accepts processes events up and running, it shouldn't be that hard to teach it new fedora-messaging messages or webhook payloads. > > ## High-level workflow proposal up for review > > Hunor proposed a high-level workflow linked below and I strongly > > recommend reading it. We have also started discussing many details in > > the process, such as getting archives: should we generate one from the > > source-git repo or use the official release archive from upstream? > > One thing to consider is that the upstream tarballs might be cryptographically > signed and packages should verify the signature in %prep. This is a very good point - in such a case, we should always pull the official upstream tarball instead of generating a new one downstream. Miro, thank you for your insights! Tomas > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 3:39 PM Ben Cotton wrote: > > Hi Tomas, > > This is great. Do you mind if I republish this in the Fedora Community > Blog? https://communityblog.fedoraproject.org Go for it, Ben :) My inspiration for the report comes from the Q1 update of the CentOS Hyperscale SIG: https://blog.centos.org/2021/04/centos-hyperscale-sig-quarterly-report/ I also like these condensed reports where one knows what the project is up to within a few minutes. Thank you for reaching out! Tomas > (Replying on-list to encourage others to submit this kind of content > to the CommBlog and to read it, too) > > -- > Ben Cotton > He / Him / His > Fedora Program Manager > Red Hat > TZ=America/Indiana/Indianapolis > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
Hi Tomas, This is great. Do you mind if I republish this in the Fedora Community Blog? https://communityblog.fedoraproject.org (Replying on-list to encourage others to submit this kind of content to the CommBlog and to read it, too) -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On Thu, Jun 24, 2021 at 11:16:11AM +0200, Tomas Tomecek wrote: > Greetings from the Fedora source-git SIG! We are planning to start > publishing reports of what we are working on so everyone can easily > pay attention and get involved if interested. If you have any ideas, > comments or requests, don’t be shy and let us know :) Awesome -- I appreciate summary reports like this _so much_. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On 6/24/21 6:40 AM, Miro Hrončok wrote: On 24. 06. 21 11:16, Tomas Tomecek wrote: ## Choosing git forge to host source-git repositories We need to find a home for all the source-git repositories. This is actually a really hard task because we have many options (github.com, gitlab.com, pagure.io, src.fedoraproject.org, something custom or on-premise) and different expectations: some projects already have repos set up on different platforms while Pagure is the primary forge now. Since the CPE team is investigating GitLab as a forge, it's even harder for us to figure out the primary forge. We may end up supporting both actually: pagure.io and gitlab.com. What are your thoughts on this topic? Would you prefer pagure.io or gitlab.com More info: * https://pagure.io/fedora-source-git/sig/issue/1 * https://pagure.io/fedora-source-git/sig/issue/7 Indirectly related: my scm-sourced COPR projects often pull from git repos (upstream @ Fedora src projects), and use forgemeta macros in rpm config. ime, forgemeta had lots of issues in the past with gitlab source matching when pulling specific tags/commits, requiring customized source strings -- usually after a bunch of trial-n-error. github & pagure had no such problems. to work around the challenges, I 1st mirrored gitlab repos to pagure, then pulled from there in my COPR specs, originally specifying commits/tags. I haven't revisited gitlab for a fairly long while to check again. Neither have I tested forgemeta's newer support for packaging a branch state https://docs.fedoraproject.org/en-US/packaging-guidelines/SourceURL/#_branch_example which is now available, stable & terribly convenient -- for github & pagure. TL;DR in this particular case: as long as it plays nicely with COPR forgemeta source-reference macros, no preference ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Fedora Source-git SIG report #1 (June 2021)
On 24. 06. 21 11:16, Tomas Tomecek wrote: ## Choosing git forge to host source-git repositories We need to find a home for all the source-git repositories. This is actually a really hard task because we have many options (github.com, gitlab.com, pagure.io, src.fedoraproject.org, something custom or on-premise) and different expectations: some projects already have repos set up on different platforms while Pagure is the primary forge now. Since the CPE team is investigating GitLab as a forge, it's even harder for us to figure out the primary forge. We may end up supporting both actually: pagure.io and gitlab.com. What are your thoughts on this topic? Would you prefer pagure.io or gitlab.com More info: * https://pagure.io/fedora-source-git/sig/issue/1 * https://pagure.io/fedora-source-git/sig/issue/7 I would expect that since the soruce git is just an intermediate thing, supporting "any forge" is nice to have, but hard. I'd start with some common options (Pagure + 1 other) and see where you'll get. ## High-level workflow proposal up for review Hunor proposed a high-level workflow linked below and I strongly recommend reading it. We have also started discussing many details in the process, such as getting archives: should we generate one from the source-git repo or use the official release archive from upstream? One thing to consider is that the upstream tarballs might be cryptographically signed and packages should verify the signature in %prep. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Fedora Source-git SIG report #1 (June 2021)
Greetings from the Fedora source-git SIG! We are planning to start publishing reports of what we are working on so everyone can easily pay attention and get involved if interested. If you have any ideas, comments or requests, don’t be shy and let us know :) Here’s a short list of things which we are working on. ## Choosing git forge to host source-git repositories We need to find a home for all the source-git repositories. This is actually a really hard task because we have many options (github.com, gitlab.com, pagure.io, src.fedoraproject.org, something custom or on-premise) and different expectations: some projects already have repos set up on different platforms while Pagure is the primary forge now. Since the CPE team is investigating GitLab as a forge, it's even harder for us to figure out the primary forge. We may end up supporting both actually: pagure.io and gitlab.com. What are your thoughts on this topic? Would you prefer pagure.io or gitlab.com More info: * https://pagure.io/fedora-source-git/sig/issue/1 * https://pagure.io/fedora-source-git/sig/issue/7 ## High-level workflow proposal up for review Hunor proposed a high-level workflow linked below and I strongly recommend reading it. We have also started discussing many details in the process, such as getting archives: should we generate one from the source-git repo or use the official release archive from upstream? Another big topic in terms of workflows are rebases (= updates to the latest upstream release, which are very common in Rawhide). Rebases are straightforward in dist-git, but when your source-git repo has complete upstream git history, they are no longer trivial, especially if one wants to get a review of a rebase. More info: * https://pagure.io/fedora-source-git/sig/issue/2 * Workflow proposal: https://pagure.io/fedora-source-git/docs/pull-request/2 * https://pagure.io/fedora-source-git/docs/blob/main/f/resources/CommitRules.pdf * https://pagure.io/fedora-source-git/sig/issue/8 ## Tooling Packit is the tooling which will be used to work with source-git repos. No surprise there I assume :D * https://packit.dev/ We've done a lot of work here lately, mainly to polish the process of creating source-git repos and doing updates of dist-git repositories based on the source-git content. * https://packit.dev/docs/source-git/work-with-source-git/ * https://github.com/packit/packit/releases ## Interested? We meet biweekly on Wednesdays via gmeet, 2:30 - 3:30 UTC, next one is scheduled for July 7th. * https://calendar.fedoraproject.org/SIGs/2021/7/5/#m9982 Everyone is welcome to join the SIG or provide any feedback on the issues and PRs above. You can always find the latest information over here: * https://fedoraproject.org/wiki/SIGs/Source-git * https://pagure.io/fedora-source-git/sig/issues I'd like to thank all the SIG members for being so active, so happy to work with all of you! Cheers, Tomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
