Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: > * In principle, you can apply a patch from a phase. However, this causes the > result of "guix build --source" to not correspond to the actual source code > anymore (i.e., it doesn't act as corresponding source anymore), so consider > this a last resort for situations such as avoiding causing a world-rebuild > for a patch fixing a target-specific bug by making the patching conditional > upon target-foo?. If you apply a patch from a phase, make sure that the patch > appears in the inputs or native-inputs, such that "guix build --source=all" > will include the patch. Should we have an option for "guix build --source=all" to also include the Guix "scripts used to control compilation and installation"? -Philip
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: > * Patches must not be used to remove non-free files, because a patch by > construction contains the non-free file itself so the patch would be > non-free, which would not be acceptable to Guix. Likewise, patches should not > be used to remove bundled libraries, to avoid large space usage, but this is > not an absolute rule unlike as for non-free files. It is possible to create patches that do not contain the deleted file, e.g. with `git format-patch --irreversible-delete`. That said, I don't know if the version of `patch` we use to patch origins is able to apply such patches—but maybe it would be a useful feature? -Philip
Re: Strategy for Zig packages
FWIW I was commenting on the impossibility of dynamically linking Zig libraries that uses comptime. It doesn't make a difference, AFAICT my explanation on why it can work holds equally for static as for shared libraries. On 03-08-2022 05:35, mcsi...@disroot.org wrote: Compile-time execution must be done at, well, compilation time, thus macros in shared objects are just glorified embedded source code that are not used at runtime. My point was that macros and dynamic (or static) linking is compatible with generics and macros. Of course, if your Zig code consists purely of generics and macros, it's kind of pointless, but presumably there's non-generic code too and non-generic library code that chooses a particular instance of a generic (monomorphisation?) that could benefit from compiling only once instead of being recompiled for every leaf (+- application). That being said, there may be a Zig ABI in the future, with the cost of missing certain features (like comptime) and optimizations: https://github.com/ziglang/zig/issues/3786 A fixed, stable, ABI or documented ABI is not required for Guix because we have the store and RPATH (or RUNPATH, I always forget which one we use) and the store model instead of putting things in /usr/lib and searching for them there, so occasional ABI breaks are not a problem for Guix. Distros like, say, Debian, are another matter of course, though they could resort to build-from-scratch like done with Rust currently. You are referring to optimisations, but at least LTO is perfectly combinable with static libraries (see, e.g., Rust). Shared libraries should be equally possible if it is accepted that it won't be always possible anymore to simply replace the shared library with another (because of things like inlining). Greetings, Maxime OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: The case for moving raw binaries
Am Donnerstag, dem 04.08.2022 um 10:54 +0200 schrieb Ludovic Courtès: > Hi, > > Liliana Marie Prikler skribis: > > > Am Freitag, dem 29.07.2022 um 17:20 -0400 schrieb Philip McGrath: > > > Hi, > > > > > > [...] > > > I was mildly annoyed recently with several programs that use the > > > ".foo-real" name in their `--help` output, for example: > > > > > > ``` > > > $ guix shell --pure reuse -- reuse -h > > > usage: .reuse-real [-h] [--debug] [--include-submodules] > > > ``` > > > > > > I wondered about just changing `wrap-program` to put the real > > > program > > > at `.real/foo` instead of `.foo-real`. One advantage is that it > > > wouldn't need any special cooperation like setting up an output > > > or an > > > environment variable. > > Even as the one who made the suggestion that issue has an easier > > workaround: Use exec -a to pass the 0th argument unchanged. > > That’s already happening: > > --8<---cut here---start->8--- > $ head -3 < $(guix build reuse)/bin/reuse | tail -1 > exec -a "$0" "/gnu/store/iwsddc43xqxz4ibncrd7cgv4qjdy0jjd-reuse- > 1.0.0/bin/.reuse-real" "$@" > --8<---cut here---end--->8--- > > I’m not sure why it doesn’t have the desired effect though. I found out: it's because reuse is a python script and the exec -a hack stops working because python initializes sys.args using the filename. Cheers
Re: Why is greetd greeter user in so many groups?
Hi, Here you are: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=56971 I added explanations and proposed solution there as well. Thanks in advance, muradm Ludovic Courtès writes: Hi, muradm skribis: This change broke greetd on my side as I am using it with gtkgreet. Could you open an issue so we keep track of this? While greetd is running as root, specific greeter started by it is not. This many groups was required to: - make it able starting graphical greeters with user greeter - access to seatd's socket file That sounds surprising to me. If other distros don’t do that, there must be something we’re missing? Anyway, let’s see how to move forward. Thanks, Ludo’. signature.asc Description: PGP signature
Re: Why is greetd greeter user in so many groups?
Hi, muradm skribis: > This change broke greetd on my side as I am using it with gtkgreet. Could you open an issue so we keep track of this? > While greetd is running as root, specific greeter started by it is > not. > > This many groups was required to: > - make it able starting graphical greeters with user greeter > - access to seatd's socket file That sounds surprising to me. If other distros don’t do that, there must be something we’re missing? Anyway, let’s see how to move forward. Thanks, Ludo’.
Re: The case for moving raw binaries
Hi, Liliana Marie Prikler skribis: > Am Freitag, dem 29.07.2022 um 17:20 -0400 schrieb Philip McGrath: >> Hi, >> >> [...] >> I was mildly annoyed recently with several programs that use the >> ".foo-real" name in their `--help` output, for example: >> >> ``` >> $ guix shell --pure reuse -- reuse -h >> usage: .reuse-real [-h] [--debug] [--include-submodules] >> ``` >> >> I wondered about just changing `wrap-program` to put the real program >> at `.real/foo` instead of `.foo-real`. One advantage is that it >> wouldn't need any special cooperation like setting up an output or an >> environment variable. > Even as the one who made the suggestion that issue has an easier > workaround: Use exec -a to pass the 0th argument unchanged. That’s already happening: --8<---cut here---start->8--- $ head -3 < $(guix build reuse)/bin/reuse | tail -1 exec -a "$0" "/gnu/store/iwsddc43xqxz4ibncrd7cgv4qjdy0jjd-reuse-1.0.0/bin/.reuse-real" "$@" --8<---cut here---end--->8--- I’m not sure why it doesn’t have the desired effect though. Ludo’.
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Hello! Maxime Devos skribis: > Context: it's currently a mess:, and at times contradictory As a rule of thumb, I’d suggest avoiding denigrating wording like this, in an effort to keep communication smooth and effective. > * There is policy involving those three, as can be seen from the >shepherd mess. What is “the shepherd mess”? Realize also that not everyone may agree that there is “a mess” in the first place. The ‘shepherd’ package uses a snippet to fix a bug. I think that’s akin to applying a patch: the intent is that ‘guix build -S’ gives you the code that’s actually built, with patches applied. > * This policy is partially secret, as can be seen by some people >treating some things as policy even if it's not in the manual. There’s no secret, but there might be unwritten rules. I think what we need to do is improve the “Snippets” section of the manual, as you propose, so we don’t have unwritten rules and misunderstandings based on hearsay. [...] > 20.4.5 Snippets, phases and patches > > Snippets, phases and patches at times serve overlapping purposes. To > decide between the three, there are several considerations to keep in > mind: > > * Patches must not be used to remove non-free files, because a patch >by construction contains the non-free file itself so the patch would >be non-free, which would not be acceptable to Guix. Likewise, >patches should not be used to remove bundled libraries, to avoid >large space usage, but this is not an absolute rule unlike as for >non-free files. > * Snippets are often convenient for removing unwanted files such as >bundled libraries, non-free sources and binaries. It is technically >also possible to use phases for this, albeit slightly less >convenient at times. However, phases must not be used to remove >non-free sources, as then the output of "guix build --source" would >still contain the non-free sources, which is incompatible with Guix' >stance on free software. Likewise, phases should not be used to >remove binaries; however, this is not strictly forbidden. > * Snippets must not embed store items in the source, as this is >incompatible with cross-compilation and prevents effectively sharing >the source code produced with "guix build --source" with people >using non-Guix systems. > * In principle, you can apply a patch from a phase. However, this >causes the result of "guix build --source" to not correspond to the >actual source code anymore (i.e., it doesn't act as corresponding >source anymore), so consider this a last resort for situations such >as avoiding causing a world-rebuild for a patch fixing a >target-specific bug by making the patching conditional upon >target-foo?. If you apply a patch from a phase, make sure that the >patch appears in the inputs or native-inputs, such that "guix build >--source=all" will include the patch. > >@c this relaxes the old rule a little > > * Ideally, the source derived from the origin should be usable for >building on any system that the upstream package supports (even if >Guix does not support that system), as a courtesy to the people that >the source code is shared with. However, this is not an absolute >rule, most important is that it is usable on Guix and it is allowed >to neglect this recommendation when it is tricky to follow or a >large amount of work. For example, if some Windows-specific source >files are non-free, you can simply remove them without replacing >them by a free implementation, even if that would reduce the set of >systems the package can be built on. > > Sometimes, there remains more than one acceptable way to accomplish > the goal. In that case, choose whatever appears to be most convenient. I kinda agree with what Julien wrote. I’d suggest starting with a patch against that section to address one specific point that you think is the most pressing one. From there we can continue the discussion. WDYT? Thanks, Ludo’.
A real-life test of long-term reproducibility
Hi everyone, One of our claims is that Guix can rebuild code identically as long as we have a machine with a Linux kernel and a POSIX filesystem. This week I had an occasion to put this to a real-life test. So far it's a failure. I can guess reasons for my failed attempts, but I don't think they were unreasonable to try. So I'd like to document something that works, to avoid others falling into the same traps. I just don't know yet what the Right Way To Do It is! The package I want to rebuild and use is "nmoldyn" from Guix commit f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced in my notes, obtained via "guix describe" in early 2018. I am pretty sure it worked fine back then. First attempt: $ guix time-machine --commit=f250a868d8c687df08559682fa68fb4ea2a1ea69 -- build nmoldyn Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'... Backtrace: In guix/store.scm: 672:3 19 (_) In ice-9/boot-9.scm: 1752:10 18 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In guix/store.scm: 659:37 17 (thunk) In guix/status.scm: 815:4 16 (call-with-status-report _ _) In guix/store.scm: 1298:8 15 (call-with-build-handler # …) 2168:25 14 (run-with-store # _ # _ # _ # _) In guix/inferior.scm: 903:8 13 (_ _) In guix/channels.scm: 944:2 12 (_ _) 891:2 11 (_ _) In ./guix/monads.scm: 487:9 10 (_ _) In guix/store.scm: 1996:8 9 (_ _) In guix/channels.scm: 642:36 8 (_ #) 703:11 7 (_) In ice-9/eval.scm: 619:8 6 (_ #(#(#(#) "/gnu/store/…" …) …)) 626:19 5 (_ #(#(#(#) "/gnu/store/…" …) …)) 155:9 4 (_ #(#(#(#) "/gnu/store/…" …) …)) 223:20 3 (proc #(#(#(#) "/gnu/sto…" …) …)) In unknown file: 2 (%resolve-variable (7 . %guix-register-program) #) In ice-9/boot-9.scm: 1685:16 1 (raise-exception _ #:continuable? _) 1685:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1685:16: In procedure raise-exception: error: %guix-register-program: unbound variable I don't understand what is going wrong here, but it may be related to the fact that the commit I am trying to go back to is older than "guix time-machine". If that's the explanation, it would help if Guix showed some clear error message instead of crashing. Next I tried checking out the source code for commit f250a868d8c687df08559682fa68fb4ea2a1ea69, and building it from source. This is a bit tricky because 2018 Guix cannot be built in today's Guix build environment. For example, today we have Guile 3, but back then we had Guile 2.2. So I need to do "guix environment guix" in an older Guix, before the Guile 3 transition, but later than the introduction of time-machine. I picked one somewhat at random: $ guix time-machine --commit=e2293cbbe0cd20ddeb932e6f5616565ab468c087 -- environment –pure guix Then I did "bootstrap", "configure –localstatedir=/var", "make check". The latter shows 15 failures, some of which look important: FAIL: tests/builders.scm FAIL: tests/derivations.scm FAIL: tests/packages.scm FAIL: tests/guix-environment.sh FAIL: tests/guix-daemon.sh And indeed I cannot build much with my compiled guix: $ ./pre-inst-env guix build nmoldyn hangs after a while, running a binary called "test-lock" for hours. Given the time lapse, I suppose there have been incompatible changes in the build daemon, making the old Guix incompatible with the rather recent build daemon running on my machine. But is there a way around this, other than installing an old Guix in a fully isolated VM? And if installing the old Guix in a VM is the only solution, what would be the best way to do that? I can't think of much else than starting from another distribution (e.g. Debian) and following the installation instructions. That's already a lot of work, but it's made worse by the installation instructions being hidden inside the manual of that old commit, which I cannot easily consult. I'd be grateful for any suggestions! Cheers, Konrad
