What should "guix build --source" produce? (was Re: Dependency cycle issues when using a Gexp-based snippet)
Hi Maxim,
maxim.courno...@gmail.com writes:
> While trying to move some of the patching done to qtbase into a snippet,
> with the goal of having at least the ./configure script runnable in a
> guix environment without having to manually run patching phases:
[...]
> + (snippet
> + (with-imported-modules '((guix build utils))
> + #~(begin
> + (use-modules (guix build utils))
> + ;; corelib uses bundled harfbuzz, md4, md5, sha3
> + (with-directory-excursion "src/3rdparty"
> + (for-each delete-file-recursively
> + (list "double-conversion" "freetype"
> "harfbuzz-ng"
> + "libpng" "libjpeg" "pcre2" "sqlite"
> "xcb"
> + "zlib")))
> +
> + (let ((coreutils #+(canonical-package coreutils)))
> + (substitute* "configure"
> + (("/bin/pwd")
> + (string-append coreutils "/bin/pwd")))
> + (substitute* "src/corelib/global/global.pri"
> + (("/bin/ls")
> + (string-append coreutils "/bin/ls"
> + #t)
Apart from the technical difficulties with cyclic modules, I'd like to
raise another issue.
In my opinion, "guix build --source PACKAGE" should produce sources that
can be used to build the package on any system that the upstream package
supports, not just on Guix systems.
Alternatively, Guix should at least have *some* command to do this.
Such a command would be especially useful for packages that we clean for
FSDG compliance. For example, I've made sure that "guix build --source
icecat" produces a tarball that's suitable for any system that IceCat
supports, and incidentally I intend to use Guix to generate the official
IceCat source tarballs.
Such a command would be useful for 'ungoogled-chromium' as well, and for
many of our other packages that include snippets to remove
non-FSDG-compliant code.
The snippet that you proposed above would produce "sources" that can
only be built on Guix systems, and moreover, only on the same
architecture and core-updates cycle that produced it.
I think that we ought to think about what "corresponding sources" should
be, and put some care into making sure that "guix build --source"
produces something worthy of that name.
What do you think?
Mark
More checks for Makefile.am:assert-no-store-file-names ?
When running "make dist" there are some checks run, such as checking for
hard-coded store paths.
Would it be a good idea to add this or a similar check to
etc/git/pre-push and/or guix lint?
Would it make sense to set up a job to run "make dist" on the build farm
to catch these problems?
# Make sure we're not shipping a file that embeds a local /gnu/store file name.
assert-no-store-file-names:
$(AM_V_at)if grep -r --exclude=*.texi --exclude=*.info
\
--exclude=*.info-[0-9] --exclude=*.dot
\
--exclude=*.eps --exclude-dir=bootstrap
\
--exclude=guix-manual.pot --exclude=guix-manual.*.po
\
--exclude=guix-cookbook.pot --exclude=guix-cookbook.*.po
\
--exclude=guix-prettify.el
\
--exclude=ChangeLog*
\
--exclude=binutils-boot-2.20*.patch
\
-E "$(storedir)/[a-z0-9]{32}-" $(distdir) ;
\
then
\
echo "error: store file names embedded in the distribution" >&2 ;
\
exit 1 ;
\
fi
Checking this more often could prevent:
bug#43005: make dist fails: "store file names embedded in the distribution"
It would be nice to catch these bugs earlier, especially when they are
low down on dependency chain!
live well,
vagrant
signature.asc
Description: PGP signature
IceCat-78.2 preview on 'wip-icecat-78' branch; need icedove-78.
Hi Jonathan, and other fellow Guix! I've pushed a preview of IceCat-78.2 to the 'wip-icecat-78' branch on Savannah. It's ready for early testing by interested parties. It works well enough that I've switched to it as my primary browser. This version also supports Wayland natively. However, in case the IceCat 78 preview doesn't (yet) work well for your use case, early testers might want to make a backup of ~/.mozilla before running it, in case you need to switch back to 68. I don't know off-hand whether IceCat 68 would cope with a profile that 78 has been run on. There's one thing that needs to be done before this can be pushed to 'master': IceDove needs to be updated to version 78 as well. For now, IceDove is almost certainly broken on the 'wip-icecat-78' branch. Jonathan: you seem to be the defacto maintainer of our IceDove package. Would you like to work on updating it to 78 on the 'wip-icecat-78' branch? We have about 2 weeks before IceCat 78 must be pushed to 'master'. Best regards, Mark
Re: [PATCH] hydra: Use the new 'systems' field for build-machine definitions.
Hello Mathieu, Mathieu Othacehe writes: > Hello Maxim, > >> The reason I've kept those in is that they are used to dial the speed >> field of the emulated build machines down, to prefer native hardware. >> I'm not sure if this is still useful / worth the additional complexity? > > Oh, I missed that detail. Then I guess you can feel free to proceed :). Pushed! Thank you, Maxim
Re: Dependency cycle issues when using a Gexp-based snippet
Hi Ludovic, Ludovic Courtès writes: > Indeed: the problem is that when loading this module, we try to resolve > one of the variables referenced in the snippet, but that variable is not > defined yet because it comes from a module that’s in a dependency circle > with the one at hand. [...] > As Marius noted before, the snippets for ungoogled-chromium and > linux-libre are contrived because of this limitation. (Perhaps we can > use ‘delayed’ instead of ‘thunked’.) I can't speak for ungoogled-chromium, but for Linux-libre and IceCat, there's at least one other limitation in snippets that are preventing us from using them: the names of the pre- and post-snippet directories must be the same. In the case of IceCat, that would force us to either (1) mislabel the upstream firefox source as "icecat", or (2) mislabel the transformed icecat source as "firefox". Ditto for "linux-libre" vs "linux". Thanks, Mark
Re: Service implementation for LXQt desktop
Hi Ludovic, On Mon, 07 Sep 2020 11:45:01 +0200 Ludovic Courtès wrote: > Instead of accessing each users’s home directory, I strongly recommend > adding a new “skeleton”: a set of files that are automatically added > to new home directories when a new user account shows up. as I know, skeletons only apply on new home directories, and for example if an existing user wants to switch from XFCE to LXQt desktop, related skeleton files wont be applied on their home directory. > To do that, the lxqt service can extend ‘account-service-type’ with > new skeletons. I can’t find an example of that but let us know if > it’s harder than it seems! as I understand from the documents, `account-service-type` extension returns a list of `user-account` and `user-group` records. I also didn't find any reference related to ad skeletons to `user-account`. a solution that I was thinking of was to add these configurations, to the related `$XDG_CONFIG_DIRS` path in system profile located in `/run/current-system/profile/etc/xdg/...` just don't know which service extension I should extend for this purpose. Thanks, Reza -- Reza Alizadeh Majd PantherX Team https://www.pantherx.org/
Re: Packaging pd
Andreas Enge writes: > I knew that three letter software names are bad, but there is worse - > two letter software names! I would like to package > pd - Primal-Dual Methods for Vertex and Facet Enumeration > http://www.cs.unb.ca/~bremner/software/pd/ > > Now there already is a "pd" in Guix; are there any precedents on how to > name a second pd? "pd-polytope"? "primal-dual"? "pd-primal-dual"? “pd” in Guix is “Pure Data”, the visual data flow language that’s often used for music. It is an option to rename it to “pure-data”. -- Ricardo
Re: [Guix Website] A Search Page for Packages
Hi Danjela! Daniela Lura skribis: > I'm sending this email to follow up on the search functionality for > packages. > With Chris's help I managed to make some changes that make the query for > searching packages within the Guix Data Service faster. > I don't know if the search is fast enough to be included on the Guix > website, but I thought it would be good to share. > > Search packages page in the test version of the website that Chris > deployed: guix-website-test.cbaines.net/packages/search It seems to work well and it’s pretty fast! > If you'd want to have a look at the code changes: > https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=ee613cdb305cc1e443135d311aead6c799902b8a > this is the commit where the changes start. Nice. What would it take to integrate it on the web site? That’d be really cool. :-) Thanks for following up, great work! Ludo’.
Re: Packaging pd
Hi, Andreas Enge skribis: > I knew that three letter software names are bad, but there is worse - > two letter software names! I would like to package > pd - Primal-Dual Methods for Vertex and Facet Enumeration > http://www.cs.unb.ca/~bremner/software/pd/ > > Now there already is a "pd" in Guix; are there any precedents on how to > name a second pd? "pd-polytope"? "primal-dual"? "pd-primal-dual"? I’d say “primal-dual” and mention “pd” in the description. > Then is there a point in packaging it at all? This is more or less a > one-shot research software, I think, with no versioning. So maybe not > relevant enough? (Let us have this Wikipedia discussion!) In that case, > I would like to add it to the guix-past channel, since I need it for > the 10 years reproducibility challenge, so the naming question still > stands. Your call! If you think Guix-Past is a better fit, go for it! > By the way, how are naming conflicts between channels resolved? The same as when you type, say, “guix build guile@2”: a warning is printed. Ludo’.
Re: Messed up with (guix ssh)
Hi! Jan Nieuwenhuizen skribis: >> So I wanted to give a heads-up and apologize. It got me thinking; this >> patch hadn’t gone through guix-patches, which could have prevented this, >> maybe. > > Thanks for the heads-up and quick fix. I'm often amazed how well > guix-patches work, notably also for catching/preventing silly mistakes. > > I think it's a trade-off and not all black and white, to get a feel > for where the boundary is we have to cross it now and then. Dunno, > very happy with what you're doing :-) Heh, thanks for the kind words. It reminded me of things like “goto fail;” though, so I thought like giving a heads-up was the least I could do. A similar mistake in git-authenticate.scm or substitute.scm would have a very different effect… Ludo’.
Re: Service implementation for LXQt desktop
Hi Reza, Reza Alizadeh Majd skribis: > working on a service definition for LXQt desktop, I need to perform a > series of default configurations. for example to set the window manager, > prepare default panel, set the default theme, etc. > > these configurations should be located $XDG_CONFIG_DIRS so the default > paths are: > > /run/current-system/profile/etc/xdg > /home/$USER/.guix-profile/etc/xdg > /home/$USER/.config/ > > I wanted to use `activation-service-type` to prepare default > configurations in users home directory, but don't know how to access > each user's home directory. Instead of accessing each users’s home directory, I strongly recommend adding a new “skeleton”: a set of files that are automatically added to new home directories when a new user account shows up. To do that, the lxqt service can extend ‘account-service-type’ with new skeletons. I can’t find an example of that but let us know if it’s harder than it seems! Thanks, Ludo’.
Re: Dependency cycle issues when using a Gexp-based snippet
Hi Maxim,
Maxim Cournoyer skribis:
>>> Attempting a suggested fix by Ludovic in that same conversation [0],
>>> namely, making the snippet field of the record a thunked one:
>>>
>>> modified guix/packages.scm
>>> @@ -250,7 +250,8 @@ as base32. Otherwise, it must be a bytevector."
>>>(patches origin-patches ; list of file names
>>> (default '()) (delayed))
>>>
>>> - (snippet origin-snippet (default #f)) ; sexp or #f
>>> + (snippet origin-snippet
>>> + (default #f) (thunked)) ; sexp or #f
>>>(patch-flags origin-patch-flags; list of strings
>>> (default '("-p1")))
>>
>> We should check what this change costs in CPU and memory, but it’s
>> probably worth it. As Marius noted before, the snippets for
>> ungoogled-chromium and linux-libre are contrived because of this
>> limitation. (Perhaps we can use ‘delayed’ instead of ‘thunked’.)
>
> What is the difference between delayed and thunked? Would a thunked
> capture the closure of its environment while delayed not? Is the
> closure useful to access record-bound values such as the version field
> of a package?
‘Thunk’ uses an actual thunk (zero-argument procedure) that’s called
each time the field is accessed; ‘delayed’ uses a promise, which is
similar except that the result is memoized (info "(guile) Delayed
Evaluation").
> I checked the usage at compilation and run time, using the 'time'
> command (aliased to time+ on my system), and didn't find any meaningful
> difference whether the snippet is made a thunked or delayed field, or
> none (current situation):
>
> On current master:
>
> time+ make -j8
> 2436.29user 56.47system 14:29.36elapsed 286%CPU (0avgtext+0avgdata
> 870828maxresident)k
> 5480inputs+405952outputs (71major+320522minor)pagefaults 0swaps
>
> time+ ./pre-inst-env guix package -A | wc -l
> 9.87user 0.24system 0:06.51elapsed 155%CPU (0avgtext+0avgdata
> 281564maxresident)k
> 0inputs+0outputs (0major+25636minor)pagefaults 0swaps
> 14702
What would be interesting is a comparison of the performance of
‘package-derivation’, which can be done with something like:
time guix build -d --no-grafts libreoffice pandoc
For memory consumption, try:
GUIX_PROFILING=gc guix build -d --no-grafts libreoffice pandoc
>> + (snippet
>> + (with-imported-modules '((guix build utils))
>> + #~(begin
>> + (use-modules (guix build utils))
>> + ;; corelib uses bundled harfbuzz, md4, md5, sha3
>> + (with-directory-excursion "src/3rdparty"
>> + (for-each delete-file-recursively
>> + (list "double-conversion" "freetype"
>> "harfbuzz-ng"
>> + "libpng" "libjpeg" "pcre2" "sqlite"
>> "xcb"
>> + "zlib")))
>> +
>> + (let ((coreutils #+(canonical-package coreutils)))
>> + (substitute* "configure"
>> + (("/bin/pwd")
>> + (string-append coreutils "/bin/pwd")))
>> + (substitute* "src/corelib/global/global.pri"
>> + (("/bin/ls")
>> + (string-append coreutils "/bin/ls"
>> + #t)
>>
>> Such substitutions are system-dependent; thus, they should be made in a
>> phase, not in a snippet. Perhaps we’ll sidestep the issue altogether?
>> :-)
>
> Indeed. I didn't consider this aspect well. Apart from being
> inefficient (the sources of a package would be different for each
> system) it would still technically work, no?
It would work, but it’s “not the right place” for that, aesthetically.
(Note that when there’s a snippet, we get different derivations for each
system anyway.)
Thanks,
Ludo’.
Re: Outreachy 2020-2021 round participation
Hi Gábor, Gábor Boskovits skribis: > Guix is now signed up to participate in this round of Outreachy. > > Important information for prospective applicants: initial application > deadline is 20th Sept. 2020 at 1600UTC. Yay, thank you! > Important information for prospective mentors: the proposal submission > deadline is 24th Sept. 2020 at 1600UTC. If you submitted a proposal for a > previous round, and no intern was accepted for the proposal and you are > still willing to mentor that project, then the proposal should be > resubmitted for this round. If you have a new idea, please feel free to > start a discussion thread on guix-devel, so that we can find out if that is > a good candidate for Outreachy and we can refine the proposal. Please tag > discussion threads with [OUTREACHY] on the subject line, so that my mail > filters can pick them up. Thanks. If this round’s mentors and interns have something to share about their experience, good or bad, now’s a good time! Ludo’.
Re: Hello, new committer here!
Hello! Pierre Langlois skribis: > So, as indicated in our contributing process on the manual, I'm happy to > announce maintainers have agreed to give me commit access, thank you > everybody! Woohoo, welcome, and thanks for all your work! Ludo’.
Re: [bug#42738] [PATCH v4] gnu: emacs: Update to 27.1.
Hi Mark,
Mark H Weaver skribis:
> Ludovic Courtès wrote:
>> Mark H Weaver skribis:
>>
>>> (define-public emacs-next
>>> (let ((commit "c36c5a3dedbb2e0349be1b6c3b7567ea7b594f1c")
>>> (revision "0")
>>> (emacs-version "27.0.91"))
>>> (package
>>> (inherit emacs)
>>> (name "emacs-next")
>>> (version (git-version emacs-version revision commit))
>>> (source
>>>(origin
>>> (inherit (package-source emacs))
>>> (method git-fetch)
>>> (uri (git-reference
>>>(url "https://git.savannah.gnu.org/git/emacs.git;)
>>>(commit commit)))
>>
>> This can be handled with ‘--with-git-url’.
>
> I think that wouldn't work in this case, because we also need to
> preserve the existing 'patches' and 'snippet' fields, which I arranged
> to inherit above via (inherit (package-source emacs)). That probably
> deserves a comment, since it's easily overlooked.
>
>>> (sha256
>>> (base32 "0mlrg2npy1r79laahkgzhxd1qassfcdz8qk1cpw7mqgf6y5x505h"))
>>> (file-name (git-file-name name version
>>> (native-inputs
>>>`(("autoconf" ,autoconf) ; needed when building from trunk
>>> ,@(package-native-inputs emacs)))
>>
>> For this, we’d need a new ‘--with-extra-input’ package transformation
>> option or similar. That way, we wouldn’t even need an ‘emacs-next’
>> package: people would just run
>>
>> guix install emacs --with-git-url=… --with-extra-input=autoconf
>
> There's also the 'native-search-paths' field, which cannot simply be
> inherited because of the version number embedded within EMACSLOADPATH.
> This particular issue could be avoided if the 'native-search-paths'
> field were a function of the version number, but that raises migration
> issues and I'm not sure it's worth it.
>
> What do you think?
Ah yes, both good points that I had overlooked.
Thanks,
Ludo’.
