Maintaining Jami #3
Hello everyone, I have a problem with running Jami on an forein distribution - when I try running Jami, I get the following message: Locale not supported by C library. Using the fallback 'C' locale. terminate called after throwing an instance of 'std::runtime_error' what(): locale::facet::_S_create_c_locale name not valid /home/user/.guix-profile/bin/jami: line 46: 3874 Aborted (core dumped) ami-gnome $* Guix nagged me about some locales and profiles and profiles, so I added the commands it asket for to .bashrc. It fixed for example icecat having no characters displayed, but didn't fork for Jami. What am I missing? I need to try running it on a foreign distro to check if my hardware is not an issue, since Jami devs can't reproduce the issue I wrote about some mails ago. Jan Wielkiewicz
Re: Speeding up “guix pull”: splitting modules
zimoun writes: > On Wed, 8 Jan 2020 at 22:50, Ludovic Courtès wrote: >> Ricardo Wurmus skribis: > >> > On the other hand: this would need to be an ongoing effort. Newly >> > introduced packages or even new features might create complex module >> > cycles. It sounds tedious to keep track of this and to enforce >> > boundaries. >> >> Yes, I think this is a dead end: glibc could well end up become on >> Haskell (hi, Pandoc!), and then the whole module split effort collapses. > > What kind of metrics could help to detect which modules are going to > the wrong way? > For example, would some DAG post-processings help? I don’t think it’s feasible to clean up the module graph in a way that can be kept clean long enough. Ideally we wouldn’t need to think about Guile modules at all, so the best screw to turn appears to be in the Guile compiler and not in Guix. -- Ricardo
Re: Proposal for a blog contribution on reproducible computations
Hello, kudos for the great article! Ludovic Courtès writes: [...] > The format we use is Markdown fed to Haunt: > > https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/posts > > (which is sad because your Org file with Babel sessions is much nicer…). > I think Pierre had something to convert Org to Markdown. you could try pandoc or emacs-ox-hugo, both in Guix I can help convert/adapt if needed HTH! [...] -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: Proposal for a blog contribution on reproducible computations
Hi Ludo, On Fri, 10 Jan 2020 at 17:59, Ludovic Courtès wrote: > • In the ‘guix pack’ example, you could perhaps omit all the -S flags > except for /bin, and mention ‘--save-provenance’. I am the culprit. The invocation of "guix pack -f docker" is not clear to me. So basically, I copied/pasted the lines here [1] :-) because it works all the time. [1] http://bioinformatics.mdc-berlin.de/pigx/supplementary-materials.html That said, I also find interesting the command-line and hashes comparisons: --8<---cut here---start->8--- /usr/bin/gcc pi.c -o pi-debian-gcc8 docker run -v `pwd`:`pwd` -w `pwd` -ti gcc-toolchain gcc pi.c -o pi-docker guix environment --container --ad-hoc gcc-toolchain -- gcc pi.c -o pi-guix md5sum pi-* b268af34d62763a2a707944403bf7b0b pi-debian-gcc8 1be3c1b5d1e065017e4c56f725b1a692 pi-docker 1be3c1b5d1e065017e4c56f725b1a692 pi-guix --8<---cut here---end--->8--- Anyway! :-) > • Would it make sense to mention MPFR in the paragraph about IEEE 754? And MPFI? ;-) All the best, simon
Re: Scicloj web meeting about Guix-Jupyter today
Hello! zimoun skribis: > Thank you! > It was interesting. :-) Glad you liked it. There were bumps on our Jitsi road in the middle, but apart from that, I enjoyed chatting with everyone! Now we should keep in touch with the Clojure folks to work on an importer (I learned about “tools.deps”) and to get Clojupyter packaged… Ludo’.
Re: Hacking ideas from the Reproducible Builds Summit
Hello, Efraim Flashner skribis: > I wish 'guix challenge' by default challenged all the servers in the > substitute-url list without needing to specify it with a flag and not > just the default one (berlin) All the servers in which list? The default list currently used by guix-daemon? (Bad news: that list is not available to clients.) Ludo’.
Re: Proposal for a blog contribution on reproducible computations
Another thing that comes to mind: would it make sense to mention ‘guix graph’ in the part where you pipe the output of ‘guix show’ to ‘recsel’, etc.? Ludo’.
Re: Proposal for a blog contribution on reproducible computations
Hi Konrad, Konrad Hinsen skribis: > Here is a first complete draft: > > > https://github.com/khinsen/reproducibility-with-guix/blob/master/reproducibility-with-guix.org > > Feedback welcome, be it by mail or as issues on GitHub. I’ve read it entirely and I think it’s perfect. It’s a pleasant read, it covers many aspects in a pedagogical way (if I’m able to judge that!), and it always shows how these nitty-gritty details relate to reproducible computations. I like how you explain that it’s human interpretation that leads us to split “inputs” and “outputs” into more specific categories (I had already enjoyed that in one of your talks). Minor comments: • You write “Build systems are packages as well”. This could be slightly misleading: build systems are (1) a set of packages, and (2) a build procedure. Dunno if it makes sense to clarify that. • In the ‘guix pack’ example, you could perhaps omit all the -S flags except for /bin, and mention ‘--save-provenance’. • Would it make sense to mention MPFR in the paragraph about IEEE 754? • Regarding ‘--container’, you write that namespaces “may not be present on your system, or may be disabled by default”, which is a bit strong; “may be present on your system, but perhaps disabled by default” would be more accurate. :-) > Also, what is the procedure for submitting blog posts? What are the > right formats for text and graphics? The format we use is Markdown fed to Haunt: https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/posts (which is sad because your Org file with Babel sessions is much nicer…). I think Pierre had something to convert Org to Markdown. To syntax-highlight Scheme code, you must start Scheme blocks with “```scheme” in Markdown. PNGs for graphics are good. You can post a patch against the guix-artwork.git repo here when you’re ready. If you want we can publish it next Tuesday or Thursday. We could have it on both hpc.guix.info and guix.gnu.org, with one saying that it’s a re-post of the other. Thank you for the great article! Ludo’.
Re: Parameterized packages
Hello!
The way I see it, we’re still toying with the idea and its pros and
cons—discussions about CLI syntax can come later. ;-)
The added flexibility of package parameters is definitely nice, but
really, maintainability is a big concern. The example Tobias gave (a
parameter to enable/disable X11 support) is interesting because it shows
that parameters can quickly become ubiquitous and get “out of control”,
from a maintenance viewpoint.
Pierre Neidhardt skribis:
> 7. Dependency management.
>Also known as the USE flag nightmare among Gentoo users...
>This is where hell breaks loose! :p
>
>The problem: the user wants to specify a parameter to use globally where it
>applies, on all installed packages and all their inputs, recursively.
>
>For instance, use guile-2.2.4 instead of guile for all guile libraries, or
>use pulseaudio everywhere, including in dependencies that are not
> explicitly
>installed to the user profile.
>
>The obstacle: A package may require inputs with a specific parameter.
>For instance, BAR depends on a FOO package built with ":audio
>pulseaudio". What happens if the user seta ":audio alsa" globally
>and installs BAR? BAR needs to specify explicitly that its FOO input
>requires pulseaudio. Otherwise BAR would fail to build.
>
>To specify that a package input depends on a specific parameter, we could
>extend the syntax of the (inputs ...) and (native-inputs ...) fields like
> so:
>
>(input
> `(("foo" ,foo "(:audio pulseaudio)")))
>
>A bigger problem is that the parameter compatibility issue is
> combinatorial:
>"Which parameter combination does BAR support?" It's hard to know it in
>advance. Any idea how to tackle this?
That’s another good example of a problem that would arise. :-/
It doesn’t seem reasonable to me to add complex logic in (guix packages)
to deal with this issue; I would very much prefer to leave input
handling unchanged.
I’m worried about the maintenance cost of parameters. Having the
feature is one thing, but being able to guarantee that the package
combinations we offer all work is another one.
We could have the feature and use it very seldom in Guix itself, but I’m
guessing that that’s not what you have in mind.
Also, for CI purposes, we would need a way to enumerate all the possible
combinations…
Apologies for spoiling the party :-), but I think these concerns need
all our attention!
Thanks,
Ludo’.
Re: Another update on the Guix Data Service
Pierre Neidhardt writes: > Christopher Baines writes: > >> Is there a specific thing you're interested in/unsure about? > > Well, mostly the patch continuous integration that you mentioned. Cool, I'm hoping to have something more concrete by the upcoming Guix Days! > Regarding the documentation, it'd be nice to explain _what_ can be done > with Guix Data Service, since it is rather large already. Indeed, I'll try to get around to improving this. Thanks, Chris signature.asc Description: PGP signature
Re: Package file indexing
Pierre Neidhardt writes: > Christopher Baines writes: > >> So, to elaborate a bit more on the architecture I've had in mind for >> dealing with the actual nars… >> >> I see the scope of the Guix Data Service extending as far as what nars >> are available for outputs, and what outputs are associated with each >> revision, but I don't think it should store the actual nar files. >> >> What you could have is another service, which subscribes to the Guix >> Data Service to find out about new revisions and nars (from build >> servers). When this new service finds out about Guix revisions, it would >> ask this Guix Data Service for all the outputs, and store this away in a >> database. When it finds out about nars, it would download them, and >> maybe extract out the list of files. >> >> I think this setup would allow this new service to construct a file >> containing information about all files in all the outputs for a >> revision, which it has nars available for. This file could then be >> downloaded, and searched through when you want to find which output >> contains a file. > > Tell me if I understood you correctly: in this scenario we would modify > the Guix derivation process to store the file list in the nars. Is this > correct? Not quite. As Ludo mentioned, you can trivially extract out the file list from nar files already (like guix archive -t). So this new service I'm thinking about which stores the nar files, would be able to read the list of files from the nar. > Question about the Guix Data Service: I suppose that the information about > the outputs > of a given revision is built incrementally, i.e. as they get published > by the build farm. Is this correct? So the information about what outputs the derivations produce is completely available once the revision has been loaded. However, nar files for those outputs become available as build farms build them, and the Guix Data Service hopefully finds out about these soon after. Also, since the build could be non-deterministic, it's possible for multiple different nar files to be generated for the same output (maybe even containing different files in the extreme case!). > If so, then the file index service needs to update the database > incrementally as well. So we need some entry point to fetch the > information delta between now and the last time we fetch the information. Yeah, you might fetch the file list database, and then another derivation is built, revealing which files are in the associated outputs. A new database can then be constructed containing that additional information. In the trivial case, the new file could be downloaded to replace the old one. This could maybe be optimised by just downloading the changes, maybe by using something like xdelta. > Please correct me if I got it all wrong! :D All great questions, hopefully I've managed to clear things up! signature.asc Description: PGP signature
Re: Speeding up “guix pull”: splitting modules
Hi Gábor, On Fri, 10 Jan 2020 at 13:42, Gábor Boskovits wrote: > > The modules graph (DAG) is already available. :-) > > The main problem here is that the modules do not form a DAG. > There are circular dependencies between the modules. > If those were not, then modular build would be possible, but because of > the spaghetti we are forced to build these together. Maybe we have a naming problem. :-) I agree that it is not an Acyclic graph and if I understand you correctly it is because there are cycles that the mess starts. Using the Directed properties (but not required in fact, whatever :-), traversing the graph detects the cycle. It is more or less what it is done in the function `guix/import/utils.scm (topological-sort)`. So knowing where the cycles are could help to transform the DaG (not fully acyclic yet) to a DAG. :-) All the best, simon
Re: Speeding up “guix pull”: splitting modules
Hello zimoun, zimoun ezt írta (időpont: 2020. jan. 10., P, 13:10): > > Hi Gábor, > > Thank you for the explanations. > > Below, I am thinking loudly. :-) > > On Tue, 7 Jan 2020 at 21:14, Gábor Boskovits wrote: > > > > > Gábor once suggested an iterative approach of identifying the most > > > > important nodes in the package graph that should be moved to their own > > > > modules, so that we would end up with a package graph that looks less > > > > like a hair ball. I think it would useful to get a better view on the > > > > relationships between modules. > > [...] > > > I was suggesting the following idea: > > The underlying package graph is actually a DAG, so by splitting modules it > > is > > possible to achieve a state where the module graph also becomes a DAG. > > The exact heuristic or algorithm of splitting was not defined. > > The modules graph (DAG) is already available. :-) The main problem here is that the modules do not form a DAG. There are circular dependencies between the modules. If those were not, then modular build would be possible, but because of the spaghetti we are forced to build these together. > Modulo some glue code / tweaks, the tools are already in > guix/graph.scm and guix/scripts/graph.scm, if I understand correctly. > And note that the commit ddd59159004ca73c9449a27945116ff5069c3743 > introduces topological sort -- for another topic: recursive import -- > but could be reused /adapted to detect cycles, IMHO. > > Would a weighted DAG help to detect which modules are in "bad shape"? > Other said, mix somehow the packages DAG and the modules DAG? > > For example, the edge between the module m1 and the module m2 should > be weighted by the ratio between the number of packages defined in the > module m2 required in the module m1. > So then, sorting this edges by weight value, it would give a better > view of the relationship between modules. > > What do you think? > > We need to detect which "big" modules are imported for "few" packages, right? > > > Aside, a naive question: does '#:select' improve the situation? I don't know, but it is an interesting question. > > > > All the best, > simon Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
Re: Another update on the Guix Data Service
Hi Christopher, Thanks for the details! Christopher Baines writes: > Is there a specific thing you're interested in/unsure about? Well, mostly the patch continuous integration that you mentioned. Regarding the documentation, it'd be nice to explain _what_ can be done with Guix Data Service, since it is rather large already. Good luck! -- Pierre Neidhardt https://ambrevar.xyz/ signature.asc Description: PGP signature
Re: Speeding up “guix pull”: splitting modules
Hi Ludo, On Wed, 8 Jan 2020 at 22:50, Ludovic Courtès wrote: > Ricardo Wurmus skribis: > > On the other hand: this would need to be an ongoing effort. Newly > > introduced packages or even new features might create complex module > > cycles. It sounds tedious to keep track of this and to enforce > > boundaries. > > Yes, I think this is a dead end: glibc could well end up become on > Haskell (hi, Pandoc!), and then the whole module split effort collapses. What kind of metrics could help to detect which modules are going to the wrong way? For example, would some DAG post-processings help? All the best, simon
Re: Speeding up “guix pull”: splitting modules
Hi Gábor, Thank you for the explanations. Below, I am thinking loudly. :-) On Tue, 7 Jan 2020 at 21:14, Gábor Boskovits wrote: > > > Gábor once suggested an iterative approach of identifying the most > > > important nodes in the package graph that should be moved to their own > > > modules, so that we would end up with a package graph that looks less > > > like a hair ball. I think it would useful to get a better view on the > > > relationships between modules. [...] > I was suggesting the following idea: > The underlying package graph is actually a DAG, so by splitting modules it is > possible to achieve a state where the module graph also becomes a DAG. > The exact heuristic or algorithm of splitting was not defined. The modules graph (DAG) is already available. :-) Modulo some glue code / tweaks, the tools are already in guix/graph.scm and guix/scripts/graph.scm, if I understand correctly. And note that the commit ddd59159004ca73c9449a27945116ff5069c3743 introduces topological sort -- for another topic: recursive import -- but could be reused /adapted to detect cycles, IMHO. Would a weighted DAG help to detect which modules are in "bad shape"? Other said, mix somehow the packages DAG and the modules DAG? For example, the edge between the module m1 and the module m2 should be weighted by the ratio between the number of packages defined in the module m2 required in the module m1. So then, sorting this edges by weight value, it would give a better view of the relationship between modules. What do you think? We need to detect which "big" modules are imported for "few" packages, right? Aside, a naive question: does '#:select' improve the situation? All the best, simon
