Re: fedpkg sources - downloading unused source files: opt-in/opt-out
> Are you manually editing the "sources" file? No, why would I? A.FI. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
Neal Gompa kirjoitti 10.5.2022 klo 2.10: On Mon, May 9, 2022 at 7:00 PM Kevin Fenzi wrote: On Mon, May 09, 2022 at 01:21:53PM -0400, Neal Gompa wrote: On Mon, May 9, 2022 at 1:13 PM Kevin Fenzi wrote: On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). Author of the patch under discussion here. The premise was that "specfile sources" equal "sources file sources". Since there is a request like this, that is apparently not always the case. From that perspective, the patch is wrong and opt-in would be the correct way. I think opt-in will be useless and make the entire option pointless. Most maintainers won't be aware it exists. Why would someone want to opt-out of this? I need to when working on ffmpeg updates, since it clobbers my regenerated tarballs when I'm working normally. I had no idea about this until someone pointed it out to me. I have difficulties understanding this. If the problem is that downloads clobber locally modified files, how can "opting out of avoiding downloading" help? I would think "opting in to avoid downloading" or, equivalently, "opting out of downloading" would help. So you mean where you have modified the source, but the name is the same as in spec and it overwrites your local changes by downloading the lookaside one over it? Yes. Such problem is not related to the original post. The discussion here is if a file listed in the sources file, but *not* as Source in the specfile, should be downloaded. Fedpkg also has the feature that is avoids downloading sources that are locally available with matching hash. So, as already suggested in other replies, to avoid clobbering, after local changes, update the hash in the sources file. 'fedpkg new-sources --offline' will do that for you. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
A ter, 10-05-2022 às 10:22 +0200, Vít Ondruch escreveu: > Ok, now I see commits such as: > > https://src.fedoraproject.org/rpms/ffmpeg/c/70ecae14df6b89cbd269778fc6808eb6e51e141e?branch=rawhide > > which is awful that we need something like this. But @Neal, wouldn't > it > be better if your `ffmpeg_gen_free_tarball.sh` simply updated the > hashes > in `sources` file? The `fedpkg new-sources --offline` could help with > that I guess. +1 @Neal I think you can solve your problem with: fedpkg new-sources --offline ffmpeg-free-5.0.1.tar.xz --offline is yet another feature, but what is asking here is the opposite, is the old behavior, `fedpkg srpm` download all sources in sources files, even if they aren't use anymore, which I don't see in what can be useful . Best regards > Vít > > > Dne 10. 05. 22 v 10:10 Vít Ondruch napsal(a): > > I somehow don't understand why there should be anything like > > "unused > > source files". Why is something like this even possible? It seems > > strange that this was not questioned originally and it seems still > > strange nobody questions this in this thread. > > > > > > Vít > > > > > > Dne 04. 05. 22 v 17:01 Ondrej Nosek napsal(a): > > > Hi all, > > > > > > A few months ago fedpkg introduced a change which avoids > > > downloading > > > source files (from dist-git) that are not used in the specfile > > > and > > > therefore downloading them would be wasting of resources and > > > time. > > > The original request was opened here [1] and implemented here > > > [2]. > > > The logic is part of the command "fedpkg sources" and currently > > > can't > > > be disabled manually. The logic parses specfile, but doesn't do a > > > deep analysis, so it is doesn't always right. > > > > > > Recently we got a request for opt-in implementation of this. It > > > means > > > you should actively use some argument (ie. --skip-unused) to > > > avoid > > > downloading unused sources. The requestor points out that it > > > broke > > > the original functionality and it is not possible to add any > > > extra > > > arguments into the complicated release process (RHEL kernel). > > > > > > On the other hand, opt-out (--download-unused) has (I think) a > > > significantly higher impact on saving resources (time and network > > > capacity). Of course, it doesn't have to be implemented as an > > > extra > > > argument, there might be a different (maybe not so clean) > > > solution > > > for such projects. > > > > > > What do you think about it? > > > > > > I added into a loop (bcc) names who already were involved in this > > > in > > > a past (on the Fedora devel mailing list) > > > > > > Thanks, Ondrej > > > > > > [1] https://pagure.io/rpkg/issue/559 > > > [2] https://pagure.io/rpkg/pull-request/564 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure -- Sérgio M. B. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On 10/05/2022 10:23, Artur Frenszek-Iwicki wrote: 2. Edit the spec file in a way that changes which sources are used (say, update to a new version) 3. Do not run "fedpkg new-sources" to upload the new tarballs 4. The "sources" file now lists files that are not actually used by the spec Are you manually editing the "sources" file? -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
V Tue, May 10, 2022 at 08:23:17AM -, Artur Frenszek-Iwicki napsal(a): > > I somehow don't understand why there should be anything like "unused > > source files". Why is something like this even possible? > 1. Grab a package > 2. Edit the spec file in a way that changes which sources are used (say, > update to a new version) > 3. Do not run "fedpkg new-sources" to upload the new tarballs > 4. The "sources" file now lists files that are not actually used by the spec Here I do these two additional steps: 4.1. Review thew new sources for new licenses, usually by comparing them to the old sources. 4.2. Run "fedpkg new-sources" because now I know the sources are license-acceptable and because it prevents me from forgetting to run "fedpkg new-sources" before pushing commits to dist-git. > 5. Run "fedpkg mockbuild" > 6. Mockbuild attempts to download package sources > Here I do rsync to a virtual machine and "fedpkg local" there. But I guess it does not differ much from "fedpkg mockbuild". -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
> I very likely have the files listed in sources > around from previous attempts Well, yeah, but it's also likely that someone: 1. Has multiple machines, hadn't done any work on this package on the current machine, and did a fresh "fedpkg clone" 2. Got fed up with clutter in the directory and started their work with "rm *.tar.gz *.src.rpm" In both these cases the old sources wouldn't be around. A.FI. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
Dne 10. 05. 22 v 10:23 Artur Frenszek-Iwicki napsal(a): I somehow don't understand why there should be anything like "unused source files". Why is something like this even possible? 1. Grab a package 2. Edit the spec file in a way that changes which sources are used (say, update to a new version) 3. Do not run "fedpkg new-sources" to upload the new tarballs 4. The "sources" file now lists files that are not actually used by the spec 5. Run "fedpkg mockbuild" 6. Mockbuild attempts to download package sources But in this scenario, I very likely have the files listed in sources around from previous attempts and once they were around, they were never downloaded again. Vít A.FI. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
> I somehow don't understand why there should be anything like "unused > source files". Why is something like this even possible? 1. Grab a package 2. Edit the spec file in a way that changes which sources are used (say, update to a new version) 3. Do not run "fedpkg new-sources" to upload the new tarballs 4. The "sources" file now lists files that are not actually used by the spec 5. Run "fedpkg mockbuild" 6. Mockbuild attempts to download package sources A.FI. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
Ok, now I see commits such as: https://src.fedoraproject.org/rpms/ffmpeg/c/70ecae14df6b89cbd269778fc6808eb6e51e141e?branch=rawhide which is awful that we need something like this. But @Neal, wouldn't it be better if your `ffmpeg_gen_free_tarball.sh` simply updated the hashes in `sources` file? The `fedpkg new-sources --offline` could help with that I guess. Vít Dne 10. 05. 22 v 10:10 Vít Ondruch napsal(a): I somehow don't understand why there should be anything like "unused source files". Why is something like this even possible? It seems strange that this was not questioned originally and it seems still strange nobody questions this in this thread. Vít Dne 04. 05. 22 v 17:01 Ondrej Nosek napsal(a): Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). On the other hand, opt-out (--download-unused) has (I think) a significantly higher impact on saving resources (time and network capacity). Of course, it doesn't have to be implemented as an extra argument, there might be a different (maybe not so clean) solution for such projects. What do you think about it? I added into a loop (bcc) names who already were involved in this in a past (on the Fedora devel mailing list) Thanks, Ondrej [1] https://pagure.io/rpkg/issue/559 [2] https://pagure.io/rpkg/pull-request/564 OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
I somehow don't understand why there should be anything like "unused source files". Why is something like this even possible? It seems strange that this was not questioned originally and it seems still strange nobody questions this in this thread. Vít Dne 04. 05. 22 v 17:01 Ondrej Nosek napsal(a): Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). On the other hand, opt-out (--download-unused) has (I think) a significantly higher impact on saving resources (time and network capacity). Of course, it doesn't have to be implemented as an extra argument, there might be a different (maybe not so clean) solution for such projects. What do you think about it? I added into a loop (bcc) names who already were involved in this in a past (on the Fedora devel mailing list) Thanks, Ondrej [1] https://pagure.io/rpkg/issue/559 [2] https://pagure.io/rpkg/pull-request/564 OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Mon, May 9, 2022 at 7:00 PM Kevin Fenzi wrote: > > On Mon, May 09, 2022 at 01:21:53PM -0400, Neal Gompa wrote: > > On Mon, May 9, 2022 at 1:13 PM Kevin Fenzi wrote: > > > > > > On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > > > > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > > > > Hi all, > > > > > > > > > > A few months ago fedpkg introduced a change which avoids downloading > > > > > source > > > > > files (from dist-git) that are not used in the specfile and therefore > > > > > downloading them would be wasting of resources and time. > > > > > The original request was opened here [1] and implemented here [2]. The > > > > > logic is part of the command "fedpkg sources" and currently can't be > > > > > disabled manually. The logic parses specfile, but doesn't do a deep > > > > > analysis, so it is doesn't always right. > > > > > > > > > > Recently we got a request for opt-in implementation of this. It means > > > > > you > > > > > should actively use some argument (ie. --skip-unused) to avoid > > > > > downloading > > > > > unused sources. The requestor points out that it broke the original > > > > > functionality and it is not possible to add any extra arguments into > > > > > the > > > > > complicated release process (RHEL kernel). > > > > > > > > Author of the patch under discussion here. > > > > > > > > The premise was that "specfile sources" equal "sources file sources". > > > > Since > > > > there is a request like this, that is apparently not always the case. > > > > From > > > > that perspective, the patch is wrong and opt-in would be the correct > > > > way. > > > > > > I think opt-in will be useless and make the entire option pointless. > > > Most maintainers won't be aware it exists. > > > > > > Why would someone want to opt-out of this? > > > > > > > I need to when working on ffmpeg updates, since it clobbers my > > regenerated tarballs when I'm working normally. I had no idea about > > this until someone pointed it out to me. > > So you mean where you have modified the source, but the name is the same > as in spec and it overwrites your local changes by downloading > the lookaside one over it? > Yes. > I can see that being an issue early on, but after initial packaging > wouldn't changes always also include the version and thus be different > from whats in the spec/sources? > Nope. If you look at how I've been changing ffmpeg, the majority of changes are within the same version: https://src.fedoraproject.org/rpms/ffmpeg/commits/rawhide > I was pleasently surprised when it didn't uselessly download the old > source after I locally updated a spec. > For a lot of things, it's very useful, for sure. Just not for packages like ffmpeg. :) -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Mon, May 09, 2022 at 01:21:53PM -0400, Neal Gompa wrote: > On Mon, May 9, 2022 at 1:13 PM Kevin Fenzi wrote: > > > > On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > > > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > > > Hi all, > > > > > > > > A few months ago fedpkg introduced a change which avoids downloading > > > > source > > > > files (from dist-git) that are not used in the specfile and therefore > > > > downloading them would be wasting of resources and time. > > > > The original request was opened here [1] and implemented here [2]. The > > > > logic is part of the command "fedpkg sources" and currently can't be > > > > disabled manually. The logic parses specfile, but doesn't do a deep > > > > analysis, so it is doesn't always right. > > > > > > > > Recently we got a request for opt-in implementation of this. It means > > > > you > > > > should actively use some argument (ie. --skip-unused) to avoid > > > > downloading > > > > unused sources. The requestor points out that it broke the original > > > > functionality and it is not possible to add any extra arguments into the > > > > complicated release process (RHEL kernel). > > > > > > Author of the patch under discussion here. > > > > > > The premise was that "specfile sources" equal "sources file sources". > > > Since > > > there is a request like this, that is apparently not always the case. From > > > that perspective, the patch is wrong and opt-in would be the correct way. > > > > I think opt-in will be useless and make the entire option pointless. > > Most maintainers won't be aware it exists. > > > > Why would someone want to opt-out of this? > > > > I need to when working on ffmpeg updates, since it clobbers my > regenerated tarballs when I'm working normally. I had no idea about > this until someone pointed it out to me. So you mean where you have modified the source, but the name is the same as in spec and it overwrites your local changes by downloading the lookaside one over it? I can see that being an issue early on, but after initial packaging wouldn't changes always also include the version and thus be different from whats in the spec/sources? I was pleasently surprised when it didn't uselessly download the old source after I locally updated a spec. kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Mon, May 9, 2022 at 1:13 PM Kevin Fenzi wrote: > > On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > > Hi all, > > > > > > A few months ago fedpkg introduced a change which avoids downloading > > > source > > > files (from dist-git) that are not used in the specfile and therefore > > > downloading them would be wasting of resources and time. > > > The original request was opened here [1] and implemented here [2]. The > > > logic is part of the command "fedpkg sources" and currently can't be > > > disabled manually. The logic parses specfile, but doesn't do a deep > > > analysis, so it is doesn't always right. > > > > > > Recently we got a request for opt-in implementation of this. It means you > > > should actively use some argument (ie. --skip-unused) to avoid downloading > > > unused sources. The requestor points out that it broke the original > > > functionality and it is not possible to add any extra arguments into the > > > complicated release process (RHEL kernel). > > > > Author of the patch under discussion here. > > > > The premise was that "specfile sources" equal "sources file sources". Since > > there is a request like this, that is apparently not always the case. From > > that perspective, the patch is wrong and opt-in would be the correct way. > > I think opt-in will be useless and make the entire option pointless. > Most maintainers won't be aware it exists. > > Why would someone want to opt-out of this? > I need to when working on ffmpeg updates, since it clobbers my regenerated tarballs when I'm working normally. I had no idea about this until someone pointed it out to me. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Wed, May 04, 2022 at 09:45:55PM +0300, Otto Urpelainen wrote: > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > Hi all, > > > > A few months ago fedpkg introduced a change which avoids downloading source > > files (from dist-git) that are not used in the specfile and therefore > > downloading them would be wasting of resources and time. > > The original request was opened here [1] and implemented here [2]. The > > logic is part of the command "fedpkg sources" and currently can't be > > disabled manually. The logic parses specfile, but doesn't do a deep > > analysis, so it is doesn't always right. > > > > Recently we got a request for opt-in implementation of this. It means you > > should actively use some argument (ie. --skip-unused) to avoid downloading > > unused sources. The requestor points out that it broke the original > > functionality and it is not possible to add any extra arguments into the > > complicated release process (RHEL kernel). > > Author of the patch under discussion here. > > The premise was that "specfile sources" equal "sources file sources". Since > there is a request like this, that is apparently not always the case. From > that perspective, the patch is wrong and opt-in would be the correct way. I think opt-in will be useless and make the entire option pointless. Most maintainers won't be aware it exists. Why would someone want to opt-out of this? > The suggestion to also allow configuring this in fedpkg.conf is good, > because for the majority of users who do not encounter these special > packages could avoid the effort to adding an extra parameter every time. I suppose maintainers could set a conf option, but I suspect the vast majority wouldn't even know it exists. > It is also good to keep in mind that the original reason why unused sources > were bothering packagers was that they easily happen during package version > updates, when test builds are done with 'fedokg mockbuild' after specfile > has been updated to the new package version, but before lookaside cache has > been updated with new sources. At the same time wiht #564, I also wrote > another path #561 [1] that enabled 'fedpkg new-sources --offline'. That > allows a package update workflow that also avoids unnecessary downloads: > > $ rpm-bumpspec -n 1.2.3 *.spec > $ spectool -g *.spec > Downloading: https://example.com/downloads/package-1.2.3.tar.gz > Downloaded: package-1.2.3.tar.gz > $ fedpkg new-sources --offline package-1.2.3.tar.gz > Uploading: package-1.2.3.tar.gz > *Upload disabled* > Source upload succeeded. Don't forget to commit the sources file > $ fedpkg mockbuild > Not downloading already downloaded package-1.2.3.tar.gz > > Given availability of this method, if there are no other, major cases where > unused sources appear, I do not think opt-in is a bad solution. I suspect most maintainers won't use that either. If you are going to be doing a PR you _do_ want to upload the source (so the tests on the PR work). But you likely want to build/test locally first before uploading in case there's some big problem with the source... Just my 2cents. kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
Sérgio Basto kirjoitti 8.5.2022 klo 7.52: On Wed, 2022-05-04 at 21:45 +0300, Otto Urpelainen wrote: Author of the patch under discussion here. The premise was that "specfile sources" equal "sources file sources". Since there is a request like this, that is apparently not always the case. Hi, Again where is the request ? I'd like to see a bit better why it is needed and if we really need that. But if he want download sources that aren't use in spec file, I prefer opt-in in sources : fedpkg sources --force Just to be clear, all I know about the request is what has been said in this thread. Also I would be interested in more details, what is being stored in the lookaside cache and why it cannot/should not be listed as Source or Patch in the specfile? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Wed, 2022-05-04 at 21:45 +0300, Otto Urpelainen wrote: > Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: > > Hi all, > > > > A few months ago fedpkg introduced a change which avoids > > downloading source > > files (from dist-git) that are not used in the specfile and > > therefore > > downloading them would be wasting of resources and time. > > The original request was opened here [1] and implemented here [2]. > > The > > logic is part of the command "fedpkg sources" and currently can't > > be > > disabled manually. The logic parses specfile, but doesn't do a deep > > analysis, so it is doesn't always right. > > > > Recently we got a request for opt-in implementation of this. It > > means you > > should actively use some argument (ie. --skip-unused) to avoid > > downloading > > unused sources. The requestor points out that it broke the original > > functionality and it is not possible to add any extra arguments > > into the > > complicated release process (RHEL kernel). > > Author of the patch under discussion here. > > The premise was that "specfile sources" equal "sources file sources". > Since there is a request like this, that is apparently not always the > case. Hi, Again where is the request ? I'd like to see a bit better why it is needed and if we really need that. But if he want download sources that aren't use in spec file, I prefer opt-in in sources : fedpkg sources --force > From that perspective, the patch is wrong and opt-in would be the > correct way. > > The suggestion to also allow configuring this in fedpkg.conf is good, > because for the majority of users who do not encounter these special > packages could avoid the effort to adding an extra parameter every > time. > > It is also good to keep in mind that the original reason why unused > sources were bothering packagers was that they easily happen during > package version updates, when test builds are done with 'fedokg > mockbuild' after specfile has been updated to the new package > version, > but before lookaside cache has been updated with new sources. At the > same time wiht #564, I also wrote another path #561 [1] that enabled > 'fedpkg new-sources --offline'. That allows a package update workflow > that also avoids unnecessary downloads: > > $ rpm-bumpspec -n 1.2.3 *.spec > $ spectool -g *.spec > Downloading: https://example.com/downloads/package-1.2.3.tar.gz > Downloaded: package-1.2.3.tar.gz > $ fedpkg new-sources --offline package-1.2.3.tar.gz > Uploading: package-1.2.3.tar.gz > *Upload disabled* > Source upload succeeded. Don't forget to commit the sources file > $ fedpkg mockbuild > Not downloading already downloaded package-1.2.3.tar.gz > > Given availability of this method, if there are no other, major cases > where unused sources appear, I do not think opt-in is a bad solution. > > [1]: https://pagure.io/rpkg/pull-request/561 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure -- Sérgio M. B. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
Ondrej Nosek kirjoitti 4.5.2022 klo 18.01: Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). Author of the patch under discussion here. The premise was that "specfile sources" equal "sources file sources". Since there is a request like this, that is apparently not always the case. From that perspective, the patch is wrong and opt-in would be the correct way. The suggestion to also allow configuring this in fedpkg.conf is good, because for the majority of users who do not encounter these special packages could avoid the effort to adding an extra parameter every time. It is also good to keep in mind that the original reason why unused sources were bothering packagers was that they easily happen during package version updates, when test builds are done with 'fedokg mockbuild' after specfile has been updated to the new package version, but before lookaside cache has been updated with new sources. At the same time wiht #564, I also wrote another path #561 [1] that enabled 'fedpkg new-sources --offline'. That allows a package update workflow that also avoids unnecessary downloads: $ rpm-bumpspec -n 1.2.3 *.spec $ spectool -g *.spec Downloading: https://example.com/downloads/package-1.2.3.tar.gz Downloaded: package-1.2.3.tar.gz $ fedpkg new-sources --offline package-1.2.3.tar.gz Uploading: package-1.2.3.tar.gz *Upload disabled* Source upload succeeded. Don't forget to commit the sources file $ fedpkg mockbuild Not downloading already downloaded package-1.2.3.tar.gz Given availability of this method, if there are no other, major cases where unused sources appear, I do not think opt-in is a bad solution. [1]: https://pagure.io/rpkg/pull-request/561 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On 04. 05. 22 17:21, Petr Pisar wrote: V Wed, May 04, 2022 at 03:14:24PM -, Artur Frenszek-Iwicki napsal(a): For me personally, opt-out would be better, since the logic is right most of the time. And doesn't break it, in the rare cases, building packages in Koji? When a source package is built in Koji, sources are downloaded from lookaside cache with "fedpkg sources". Using fedpkg-minimal, not fedpkg. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
On Wed, 2022-05-04 at 17:01 +0200, Ondrej Nosek wrote: > Hi all, > > A few months ago fedpkg introduced a change which avoids downloading > source files (from dist-git) that are not used in the specfile and > therefore downloading them would be wasting of resources and time. > The original request was opened here [1] and implemented here [2]. > The logic is part of the command "fedpkg sources" and currently can't > be disabled manually. The logic parses specfile, but doesn't do a > deep analysis, so it is doesn't always right. > > Recently we got a request for opt-in implementation of this. It means > you should actively use some argument (ie. --skip-unused) to avoid > downloading unused sources. The requestor points out that it broke > the original functionality and it is not possible to add any extra > arguments into the complicated release process (RHEL kernel). where is this request ? I can't find it Thank you > > On the other hand, opt-out (--download-unused) has (I think) a > significantly higher impact on saving resources (time and network > capacity). Of course, it doesn't have to be implemented as an extra > argument, there might be a different (maybe not so clean) solution > for such projects. > > What do you think about it? > > I added into a loop (bcc) names who already were involved in this in > a past (on the Fedora devel mailing list) > > Thanks, Ondrej > > [1] https://pagure.io/rpkg/issue/559 > [2] https://pagure.io/rpkg/pull-request/564 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure -- Sérgio M. B. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
V Wed, May 04, 2022 at 03:14:24PM -, Artur Frenszek-Iwicki napsal(a): > For me personally, opt-out would be better, since the logic is right most of > the time. > And doesn't break it, in the rare cases, building packages in Koji? When a source package is built in Koji, sources are downloaded from lookaside cache with "fedpkg sources". -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: fedpkg sources - downloading unused source files: opt-in/opt-out
For me personally, opt-out would be better, since the logic is right most of the time. That being said, how about adding a value for this in $HOME/.config/rpkg/fedpkg.conf? That way, you have some default value, which can be overridden by the config, which in turn can be overridden by the opt-in/opt-out command-line arg. A.FI. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
fedpkg sources - downloading unused source files: opt-in/opt-out
Hi all, A few months ago fedpkg introduced a change which avoids downloading source files (from dist-git) that are not used in the specfile and therefore downloading them would be wasting of resources and time. The original request was opened here [1] and implemented here [2]. The logic is part of the command "fedpkg sources" and currently can't be disabled manually. The logic parses specfile, but doesn't do a deep analysis, so it is doesn't always right. Recently we got a request for opt-in implementation of this. It means you should actively use some argument (ie. --skip-unused) to avoid downloading unused sources. The requestor points out that it broke the original functionality and it is not possible to add any extra arguments into the complicated release process (RHEL kernel). On the other hand, opt-out (--download-unused) has (I think) a significantly higher impact on saving resources (time and network capacity). Of course, it doesn't have to be implemented as an extra argument, there might be a different (maybe not so clean) solution for such projects. What do you think about it? I added into a loop (bcc) names who already were involved in this in a past (on the Fedora devel mailing list) Thanks, Ondrej [1] https://pagure.io/rpkg/issue/559 [2] https://pagure.io/rpkg/pull-request/564 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
