Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
@pmatilai converted this issue into discussion #2654. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#event-10342508518 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
To your comment: > in some workflows people receive an srpm from somewhere and build that. my answer is precisely that. **When I try to rebuild the single package the process fails at the end of a 3-hour build run with an extremely obscure error message.** Yet somehow it seems to work with the upstream packager as the rpms are built and shared publicly. The upstream packager's build environment is not public and I've seen in some cases that to make the build work in some cases some "munging" of the build environment is needed. That's very messy. However, I think you understand my point of view. The intent of me creating this issue was to bring it up. It seems you're aware of the problem space and have shared that others also experience similar issues. Is there anything that can be done now? What should happen next? I do not think I can do anything right now and clearly any changes would be a long term effort. Can any further progress be made? -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#issuecomment-1667726799 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
ok, so it depends on your point of view, that's clear. For downstream "repackagers" there's clearly internal macro "mangling" and perhaps build environment differences specifically due to that, and while that's clearly a very important use case it's not the same as mine. It may be that some of this behaviour needs to be optional, but again if the current rpm build process does not allow you to complete a rebuild correctly or it requires a large amount of investigation to work out how to achieve the end goal of reproducibly building packages then to some extent I think it's fragile. The link you provide just goes to show how fragile the current process is if you look at it in any detail, even if general building it seems to work. I suspect/know that things are more complex now than they were in the initial rpm days. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#issuecomment-1667718170 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
> I'm also aware that if you run `rpmbuild -bs .spec` there's no binary > build process going on so I'd guess it's quite possible that macro > definitions and installed package lists may not actually be useful. However, > if you build source and binary packages together, which I'd assume is the > correct thing to do, e.g. `rpmbuild -ba > whatever.spec` then you will have the information needed and could save this > inside the `src.rpm`. "correct thing to do" is a matter of opinion. An easier question is whether this is what always happens, and it's easy to answer: no, in some workflows people receive an srpm from somewhere and build that. See for example the (now obsolete) workflow for CentOS: RH would publish rebranded srpm and various projects would rebuild them, treating the srpm as the initial input. In general, the macros that are defined during srpm build and the macros that are defined during binary build can be completely different. See https://pagure.io/koji/issue/3878 for a slightly different approach to this problem. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#issuecomment-1667459110 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
Also related to your comments about how rpm works with dependencies: I'm aware of rpm dependencies. I've been building rpm packages since RedHat 3.0.3 (that's in 1996). Also I'm not the owner of the rpms I want to rebuild so it's not a matter of me rebuilding my rpms it's also a matter of me figuring out how to rebuild others' rpms. I could certainly suggest the upstream packagers make changes to their packages but that's a longer term discussion and it requires them doing this explicitly. The nature of this issue I have created is have rpm do this for us automatically, so I don't have to figure out the details myself. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#issuecomment-1667348812 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] Make rpm builds more reproducible (Issue #2590)
In the end **as a first step** I'd like to be able to reproduce the builds as close to what the original builder did. So I'd like that build to be reproducible in the way referenced in the URL. - One thing is to record the rpm macro values used during the build process. - another simple thing would be to record the full list of name / version of rpms installed on the build system. "rpm builds" are a bit of a pain as often we may use multiple repos. The build environment does not explicitly say where the `BuildRequires:` packages come from, so `rpm` itself is unable to pull down the needed packages in order to trigger the builds. `yum` or `dnf` could do that but it `rpm` doesn't know about this and the lack of integration between the two sets of tools has always somewhat surprised me though I guess that's a different story and it's water under the bridge now. I have changed the issue title to **make rpm builds more reproducible** as I think that's what I want and I think that by solving the 2 points above this would help a lot. I'm also aware that if you run `rpmbuild -bs .spec` there's no binary build process going on so I'd guess it's quite possible that macro definitions and installed package lists may not actually be useful. However, if you build source and binary packages together, which I'd assume is the correct thing to do, e.g. `rpmbuild -ba whatever.spec` then you will have the information needed and could save this inside the `src.rpm`. I think this would pretty much help solve my problem. My second desire, **which is NOT relevant to this issue**, is to then be able to modify the build config or sources or add patches so that the finally built packages provide extra features compared to the originally built packaging, yet if done correctly the result will be compatible with the original packages. You could think of this along the lines of building extra kernel modules in a separate rpm provided as part of the build process which can be installed and used on the upstream running base kernel. It's much easier to do this if you can reproduce the upstream build process first. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2590#issuecomment-1667282383 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
