Re: New ‘guix pull’
Ludovico, I think it is a good idea to add roll-back to the guix pull with a flag of some sort. However, I would be wary of people unintentionally rolling-back packages for not understanding the usecase of such a feature. However, I suppose that we can not "idiot-proof" everything because they did not read the documentation. Brett Gilio bre...@posteo.net | b...@member.fsf.org Free Software -- Free Society! On 07/12/2018 11:01 AM, Ludovic Courtès wrote: Hi Chris, Thanks for your feedback (and apologies for the hiccups!). Chris Marusich skribis: Thank you for your hard work on this improvement! One of the reasons I have always been a little scared to run "guix pull" frequently is because it was difficult to roll back. Now that the user-installed Guix is managed in a profile of sorts, it will probably be easy to add something like "guix pull --roll-back" to help people recover easily. The manual currently recommends: guix package -p ~/.config/guix/current --roll-back but we could (should?) add a roll-back option directly in ‘guix pull’. Ludo’.
Re: [PATCH] profiles: Let canonicalize-profile return an absolute path.
Ludovic Courtès writes: > Hi Roel, > > Roel Janssen skribis: > >> I'd like to change the way the symlinks to custom profiles are created. >> Here's what currently happens: >> >> $ guixr package -i hello -p guix-profiles/test >> $ ls -l guix-profiles >> lrwxrwxrwx. 1 user group 25 Jul 3 19:53 test -> guix-profiles/test-1-link >> lrwxrwxrwx. 1 user group 51 Jul 3 19:53 test-1-link -> >> /gnu/store/...6qbaps-profile >> >> Now, that symlink is broken. >> Instead, I'd like to have it always use absolute paths: > > How about instead making the link to the generation file (“test-1-link”) > always a relative symlink? Like this: > > --8<---cut here---start->8--- > $ ./pre-inst-env guix package -p foo/x -i sed > > [...] > > $ ls -l foo/* > lrwxrwxrwx 1 ludo users 8 Jul 11 13:03 foo/x -> x-1-link > lrwxrwxrwx 1 ludo users 51 Jul 11 13:03 foo/x-1-link -> > /gnu/store/qp6dqlbsf0pw9p9fwc3gzdcaxx40rn9v-profile > --8<---cut here---end--->8--- > > Patch below. > > FWIW I prefer avoiding ‘canonicalize-path’ in general because it’s > inefficient and because it can surprise the user: you can end up with a > long file name that you didn’t type in, or you can have ENOENT errors > because ‘canonicalize-path’ requires the given file to exist. > > WDYT? > > Thanks, > Ludo’. I like your patch a lot better than mine! It fixes the issue I run into, so it'd be great to apply your patch soon. There's one other thing I also run into that is somewhat related to this: On a multi-user system, where ‘root’ cannot see what's in a user's directory, it's impossible to keep track of custom profiles. However, the default user profiles are fine, because they are actually stored in the local state dir, and symlinked outside. Could we do the same with custom profiles? The functionality stays the same, it might even be cleaner in the user's directory because it only needs a single symlink to the latest generation of a profile, and we might be able to do garbage collection again on our cluster! I'd image something like this: --8<---cut here---start->8--- $ echo $HOME /home/roel $ guix package -i hello teeworlds -p ~/my/custom/profile --> /var/guix/profiles/per-user/roel/home/roel/my/custom/profile -> ... --> /var/guix/profiles/per-user/roel/home/roel/my/custom/profile-1-link $ ls -l ~/my/custom drwxrwxrwx ... profile -> /var/guix/profiles/per-user/roel/home/roel/my/custom/profile --8<---cut here---end--->8--- That way, if root cannot look into ‘/home/roel’, it can still keep track of the profile because it can look into ‘/var/guix’. Kind regards, Roel Janssen
Firefox 52's end of life, packaging Icecat 60
Hello, As this blog article[1] says, Firefox 52's end of life will happen on August 28, 2018. That is, in 47 days. I imagine that by that time Icecat 60 will be released, but it seems that we are pretty far from being able to package it, because of the Rust packages that are needed. There might be other technical difficulties that I'm not aware of, though. Is there any plan in this regard? Maybe, as a first step, we should list the required dependencies so that people can pick them up? My understanding of that article[2] is that the top level dependencies are listed in Cargo.toml[3]. What is the current state of the Rust build system (cargo-build-system)? Is there anything blocking that would prevent its use? I'm asking because I see very few packages in gnu/packages/rust.scm. Thank you, Clément [1]: https://blog.mozilla.org/futurereleases/2018/01/11/announcing-esr60-policy-engine/ [2]: https://developer.mozilla.org/en-US/Firefox/Building_Firefox_with_Rust_code [3]: https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml
Re: bootstrap integration strategies
Orians, Jeremiah (DTMB) writes: >> Gash seems to be a low-hanging fruit and a relatively easy thing, >> because it's architecture-independent. How >> far is it from being able to run typical 'configure' scripts? > Well we would have to replace the parser at a bare minimum Yes, the parser is being refactored. It "parses" a lot of SH code, but it proved that handling variables properly was getting really tricky. >> This could allow us to remove quite a lot of MiBs from our binary seeds. > FTFY > > At this point, we effectively have a rope bridge to full bootstrappability Hear, hear! > But we still have a lot of details to hammer out, like getting basic > ARM support and having the ARM and x86 binaries verify each other's > bootstrap; > Hammer out cross-platform build details for MesCC and M2-Planet These I certainly agree with! Some other archs can come later? janneke -- Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
Re: bootstrap integration strategies
Ludovic Courtès writes: >>> I think that's the main difficulty. I think we'd rather not have >>> separate bootstrap paths for Intel GNU/Linux on one hand, and everything >>> else on the other hand. >> >> Well, due to the design of mescc-tools; the bootstrap paths only have to be >> divergent up to the M1-macro level. >> After that, we could simply use flags make the source work on different >> platforms > > Sounds nice. I wonder if Jan was referring to something else then? I was looking at a narrower, or more short-term perspective. M1 is solving *a lot* of problems for us. The only platform-dependency that Mes then has, is: lib/x86-mes/x86.M1: About 200 M1 macros such as DEFINE add$i32,%eax 05 DEFINE add$i32,%ecx 81c1 So, technically it's solved. In practice, MesCC doesn't do proper register mapping. So, we could hard-wire a mapping for AArch64 but we'd probably want to introduce abstraction. That's a significant effort. Also, we'll have to solve all tiny details/bugs that we encounter in MesCC. There's no real limitation, just some hard choices and work. Then there is some assembly in Mes lib/linux-*.c as well, much less of a problem, could still take weeks of work. Then we have TCC; i'm not sure at their support for AArch64, and of course we currently use gcc-2.95 which also doesn't do AArch64. GCC 3 depends on a richer C library. Of course we can make that (and this is one of the things we will most probably do anyway) but still: work. So what I was saying is probably: we have x86 NOW, can we use it and do we want that somehow? OR do we plan some of the work above, and go that route? > There’s still the question of GNU/Hurd, though, which requires a vastly > different libc. Yes. I "almost" have a hello world there. TBH, after spending about half a week on it, I think x86/Hurd may be the easiest second architecture...maybe amd64. > So, problem solved? Or am I missing something? :-) > > I think the ‘wip-bootstrap’ branch does not use M1 at this point, does > it? Sure, wip-bootstrap uses M1. Currently the %mes-seed consists of mes.M1; M1 is then used to produce a binary. We are working to remove this seed by translating mes.c into mes.M2. Also, MesCC produces M1 output; so tcc is compiled to M1, and then to binary. >>> there's also another option you didn't mention: ditching the 2.0 >>> bootstrap Guile in favor of Mes. That can be done in several steps: >> >>> 1. Replace the guile-2.0.*.xz binary tarballs with Mes, and add a step >>> that builds Guile 2.x using our big bootstrap GCC binary. >> Slow but possible Yes, performance is really the thing here. Currently, mes is about 30x slower than Guile. It will definately not work if mes has to interpret all of gnu/packages/*.scm, it may work if we can do something smart. Also, Mes resembles Guile just enough to run Nyacc, but it's not on par with Guile by far. >>> 2. Same, but build Guile 2.x, libgc, etc. using MesCC. >> MesCC can't directly build Guile yet but I do enjoy that ambition ;-) > > I wonder what it would take to fix that. After all, compiling libguile > must not be much harder than compiling tcc, no? tcc is ~20,000LOC (1.5h to build), libguile is ~100,000LOC, libgc is ~30,000LOC. Unless we find a way to compile a fraction of libguile #if !BOOTSTRAP ..., compilation could take a day. The good news is that MesCC also runs on Guile, so development would suffer less from this. I *really* like taking the early Mes route, or early Guile route (1 or 2). I don't see how it these would work out--it just feels right. > IMO what matters most at this point is to come up with a plan that allow > us to incrementally reduce the size of our binary seeds. A port of > M1/stage0 to Z80 can wait. ;-) So we really need a list of actionable > items in the short term to start taking advantage of all the work that’s > been done on Mes, MesCC, M1, stage0, Gash, etc. Hmm, yes! My idea was to remove gcc, glibc and binutils from the x86 bootstrap binaries (possibly one at a time) and replace them by M1+Mes+Tcc-built ones. Possibly by adding an alternative %bootstrap-binaries package without them. Then build the x86 system on top of that. Then, or meanwhile, start thinking about x86_64 or AArch64. I'm not saying this because I think it's better than an early Guile route, but it's only because I think I know this can be done. Greetings, janneke -- Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
RE: bootstrap integration strategies
> Sounds nice. I wonder if Jan was referring to something else then? Probably alternate operating systems like Hurd is my guess but I'm probably wrong. > There’s still the question of GNU/Hurd, though, which requires a vastly > different libc. Fortunately Janneke has done a good job making that selectable > So far the initial ports of Guix to non-x86 were done through > cross-compilation (info "(guix) Porting"). So in a way, the binary > seeds for these platforms were built from source; we just “cut” the > source-to-binary connection by making those binaries the root of the > dependency graph on these platforms. Thank you for clarifying > Maybe that’s something we’ll have to live with on new architectures. Unless you want to make qemu a root dependency > So, problem solved? Or am I missing something? :-) Mescc-tools can build valid binaries for all instruction sets with sane immediate representations (RISC-V is the only exception here; hopefully they fix that) But Definition files need to be written and tests generated > I think the ‘wip-bootstrap’ branch does not use M1 at this point, does it? M1 and Hex2 are core pieces of mescc-tools and are required for MesCC to produce binaries. As MesCC outputs M1-macro files (the .S files) > I wonder what it would take to fix that. After all, compiling libguile > must not be much harder than compiling tcc, no? Janneke know far better than me on this one > One thing at a time. :-) But this is lots of fun :D > IMO what matters most at this point is to come up with a plan that allow > us to incrementally reduce the size of our binary seeds. A port of > M1/stage0 to Z80 can wait. ;-) So we really need a list of actionable > items in the short term to start taking advantage of all the work that’s Ok, how does this sound: We walk the bootstrap binaries towards MesCC (This will add more bootstrap binaries in the short term) Then we eliminate them one at a time until guile and mescc-tools are the only binaries that remain Mescc is a scheme program which is interpreted by either guile or mes.c and thus could leverage guile until mes.c is in a state that it can replace guile for : guix, mescc and GASH Thus the making of the bootstrap binaries can follow 2 possible paths: 1) Fast via standard C compilers: just build mes.c and mescc-tools and be done 2) Platform specific Stage0, which starts with hex0 (`200B) -> hex1 (~500B) -> hex2 (`1KB) -> M0 (~2KB) -> [Everything here on is platform neutral] ->M2-Planet (~16KB) -> mes.c + mescc-tools (M1 and Hex2) The best part is all of the binary seeds of all of the platforms will be able to build the binary seeds for all of the other platforms with bit for bit identical results (Which eliminates hardware based Trusting Trust attacks avoiding detection) Jeremiah Orians Cell phone: (517) 896-2948
Re: New ‘guix pull’
On Thu, Jul 12, 2018 at 04:29:02PM +0200, Ricardo Wurmus wrote: > > Pjotr Prins writes: > > > On Thu, Jul 12, 2018 at 04:17:32PM +0200, Ricardo Wurmus wrote: > >> “guix pull” updates its own profile only. It updates > >> ~/.config/guix/current, so you need to use Guix from > >> ~/.config/guix/current/bin. > > > > And it says so after guix pull. It is easy to forget however. > > > > Why don't we (also) add it to the default guix profile? To most users > > this would make sense. No point in running guix pull if you don't want > > to update guix itself - right? > > It does update Guix itself. The “guix” package, however, should > generally not be installed into a user profile, because it is always > necessarily *older* than the version of Guix that you are using to > install it (unless you’re using “--with-source” to use a more recent > tarball). Hmmm. I don't understand. I would think it the other way round: the guix package you install is newer than the running daemon. But, no matter. Maybe we should just accentuate that guix gets its own independent profile. That I can understand ;). So it should *never* be in the user profile. Not even by accident. Pj.
Re: gnumaint changes
On Thu, Jul 12, 2018 at 17:57:01 +0200, Ludovic Courtès wrote: > Hello, > > Mike Gerwitz skribis: [...] >> Do you have a couple examples of what you think would be beneficial to >> pull form Guix? I'm certainly open to the idea where it makes sense; >> there's no sense in us duplicating effort within GNU unnecessarily. > > I realize that Guix doesn’t have all GNU packages yet so in fact there’s > not so much to pull from at this point. I was suspecting blurbs are > likely to be more up-to-date in Guix, but that’s very subjective, I > don’t know if this is the case. It seems like the blurbs in Guix may be slightly different: in Womb they are provided by the package author for use here: https://www.gnu.org/manual/blurbs.html In Guix they may be augmented with additional information that the Guix package author finds useful, and may deviate from what the GNU package author provided. Is that true? It makes sense to me, though, that Guix and that page would be in sync. But if the intent is to have the blurbs be written by the package authors, syncing them would mean that Guix would forefit the ability to manage its own package descriptions. I'm not sure if that's something Guix would want to do. I'm also unaware of how many GNU package maintainers even remember that the blurbs page even exists. So it's possible that such descriptions could be updated. It'd be worth maintainers@ occasionally asking package maintainers to review our records. >> I'm also working on automating parts of our recordkeeping: in the next >> few weeks, Womb will have up-to-date version information automatically >> pulled from info-gnu release announcements; the FTP server; and a couple >> websites where necessary, though I'll be manually committing it for the >> first few months to verify that it is all working properly. So Guix >> might also be able to depend on rec/gnupackages.rec for checking for new >> releases as well, since unfortunately GNU doesn't mandate the use of the >> FTP server, or even info-gnu (so releases are all over the place). > > The (guix gnu-maintenance) modules are tools to retrieve the latest > version of a GNU package by traversing its ftp.gnu.org (or similar) > directory. That’s something you might find useful. Here’s an example: Thanks---I was going to reference Guix's implementation. But do note that many GNU packages don't make use of GNU's FTP server, so this doesn't work on its own as a comprehensive version check tool for GNU software. But if this hasn't been a practical problem for Guix yet, then there's no need to change that. -- Mike Gerwitz Free Software Hacker+Activist | GNU Maintainer & Volunteer GPG: D6E9 B930 028A 6C38 F43B 2388 FEF6 3574 5E6F 6D05 https://mikegerwitz.com signature.asc Description: PGP signature
Re: bootstrap integration strategies
Hello OriansJ, "Orians, Jeremiah (DTMB)" skribis: >> I think that's the main difficulty. I think we'd rather not have >> separate bootstrap paths for Intel GNU/Linux on one hand, and everything >> else on the other hand. > > Well, due to the design of mescc-tools; the bootstrap paths only have to be > divergent up to the M1-macro level. > After that, we could simply use flags make the source work on different > platforms Sounds nice. I wonder if Jan was referring to something else then? There’s still the question of GNU/Hurd, though, which requires a vastly different libc. >> Yet, we know that porting what you already did on x86-linux-gnu to >> GNU/Hurd and ARMv7 and AArch64 etc. is going to be a lot of non-trivial >> work (especially since historical versions of the GNU toolchain did not >> support AArch64, for instance.) > Nor RISC-V but that is likely to be a much bigger issue in terms of > bootstrapping So far the initial ports of Guix to non-x86 were done through cross-compilation (info "(guix) Porting"). So in a way, the binary seeds for these platforms were built from source; we just “cut” the source-to-binary connection by making those binaries the root of the dependency graph on these platforms. Maybe that’s something we’ll have to live with on new architectures. >> Waiting for this to be "solved" (and we don't even know how) would >> equate to a status quo. But obviously, it'd be sad to have all this >> work already done on Intel and not be able to benefit from it. > Actually the work for the stage0 bootstrap steps have already been done on > non-x86 hardware (Knight platform to be precise) > And the engineering decisions involved where explicitly selected to minimize > porting and cross-platform bootstrapping effort. > M1-macro and hex2-linker only need flags to be set to build for all of the > different supported platforms So, problem solved? Or am I missing something? :-) I think the ‘wip-bootstrap’ branch does not use M1 at this point, does it? [...] >> There's also another option you didn't mention: ditching the 2.0 >> bootstrap Guile in favor of Mes. That can be done in several steps: > >> 1. Replace the guile-2.0.*.xz binary tarballs with Mes, and add a step >> that builds Guile 2.x using our big bootstrap GCC binary. > Slow but possible > >> 2. Same, but build Guile 2.x, libgc, etc. using MesCC. > MesCC can't directly build Guile yet but I do enjoy that ambition ;-) I wonder what it would take to fix that. After all, compiling libguile must not be much harder than compiling tcc, no? > At this point, we effectively have a rope bridge to full bootstrappability > But we still have a lot of details to hammer out, like getting basic ARM > support and having the ARM and x86 binaries verify each other's bootstrap; > Finding 6502, z80, 8051, 68K, VAX, pdp11, Alpha, MIPS, SPARC and > PowerPC/Power Developer(s) to do stage0 work for their platforms and perform > the cross verify steps. > Hammer out cross-platform build details for MesCC and M2-Planet One thing at a time. :-) IMO what matters most at this point is to come up with a plan that allow us to incrementally reduce the size of our binary seeds. A port of M1/stage0 to Z80 can wait. ;-) So we really need a list of actionable items in the short term to start taking advantage of all the work that’s been done on Mes, MesCC, M1, stage0, Gash, etc. Thanks for your feedback! Ludo’.
Re: New ‘guix pull’
Hi Chris, Thanks for your feedback (and apologies for the hiccups!). Chris Marusich skribis: > Thank you for your hard work on this improvement! One of the reasons I > have always been a little scared to run "guix pull" frequently is > because it was difficult to roll back. Now that the user-installed Guix > is managed in a profile of sorts, it will probably be easy to add > something like "guix pull --roll-back" to help people recover easily. The manual currently recommends: guix package -p ~/.config/guix/current --roll-back but we could (should?) add a roll-back option directly in ‘guix pull’. Ludo’.
Re: gnumaint changes
Hello,
Mike Gerwitz skribis:
>> It’d be nice if synopses and descriptions in the Womb could contain
>> Texinfo markup.
>>
>> In fact, perhaps it’d make sense to reverse the roles, i.e., have the
>> Womb take (some of its) descriptions from Guix?
>
> `blub' in pkgblurbs (which is what `official-description' uses) is
> provided by package authors after they've been dubbed by rms. That is
> in turn used on gnu.org. Consequently, I think it's best to have such
> blurbs maintained independently of guix.
I see, that makes sense.
> What sort of Texinfo markup are you looking for, and are we talking
> about the same field? What field does guix use for the synopsis?
> Everything in rec/gnupackages.rec is handled by us at maintainers@, so
> we can do whatever we want there.
For packages we occasionally use Texinfo markup, typically ornaments
like @code or @itemize bullet lists. Not every synopsis/description
needs it, but it’s nice to be able to use it.
> Do you have a couple examples of what you think would be beneficial to
> pull form Guix? I'm certainly open to the idea where it makes sense;
> there's no sense in us duplicating effort within GNU unnecessarily.
I realize that Guix doesn’t have all GNU packages yet so in fact there’s
not so much to pull from at this point. I was suspecting blurbs are
likely to be more up-to-date in Guix, but that’s very subjective, I
don’t know if this is the case.
> I'm also working on automating parts of our recordkeeping: in the next
> few weeks, Womb will have up-to-date version information automatically
> pulled from info-gnu release announcements; the FTP server; and a couple
> websites where necessary, though I'll be manually committing it for the
> first few months to verify that it is all working properly. So Guix
> might also be able to depend on rec/gnupackages.rec for checking for new
> releases as well, since unfortunately GNU doesn't mandate the use of the
> FTP server, or even info-gnu (so releases are all over the place).
The (guix gnu-maintenance) modules are tools to retrieve the latest
version of a GNU package by traversing its ftp.gnu.org (or similar)
directory. That’s something you might find useful. Here’s an example:
--8<---cut here---start->8---
scheme@(guile-user)> ,use(gnu packages gcc)
scheme@(guile-user)> ,use(guix upstream)
scheme@(guile-user)> (package-latest-release gcc (force %updaters))
$4 = #< package: "gcc" version: "8.1.0" urls:
("mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.xz"
"mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.gz") signature-urls:
("mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.xz.sig"
"mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.gz.sig")>
--8<---cut here---end--->8---
Or simply:
--8<---cut here---start->8---
scheme@(guile-user)> ,use(guix gnu-maintenance)
scheme@(guile-user)> (latest-release "emacs")
$5 = #< package: "emacs" version: "26.1" urls:
("ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.xz;
"ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.gz;) signature-urls:
("ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.xz.sig;
"ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.gz.sig;)>
--8<---cut here---end--->8---
This relies primarily on .
Packages not hosted on gnu.org are typically annotated with the download
URL such that the update-checking code does the right thing.
Thanks,
Ludo’.
Re: Note on Outreachy internship
Hello, Ricardo Wurmus skribis: > after reviewing the current state of the project “Improve the user > experience for the "guix package" command line tool”, for which Gábor > and I mentored an Outreachy intern, we have decided to terminate the > internship at the midterm evaluations. This is a sad outcome of course, but I’d like to thank Sahithi for joining us on this internship, and a big thanks to you Ricardo and Gábor for your tireless efforts as mentors and overseers of the internship. Ludo’.
Re: New ‘guix pull’
Pjotr Prins writes: > On Thu, Jul 12, 2018 at 04:17:32PM +0200, Ricardo Wurmus wrote: >> “guix pull” updates its own profile only. It updates >> ~/.config/guix/current, so you need to use Guix from >> ~/.config/guix/current/bin. > > And it says so after guix pull. It is easy to forget however. > > Why don't we (also) add it to the default guix profile? To most users > this would make sense. No point in running guix pull if you don't want > to update guix itself - right? It does update Guix itself. The “guix” package, however, should generally not be installed into a user profile, because it is always necessarily *older* than the version of Guix that you are using to install it (unless you’re using “--with-source” to use a more recent tarball). -- Ricardo
Re: New ‘guix pull’
On Thu, Jul 12, 2018 at 04:17:32PM +0200, Ricardo Wurmus wrote: > “guix pull” updates its own profile only. It updates > ~/.config/guix/current, so you need to use Guix from > ~/.config/guix/current/bin. And it says so after guix pull. It is easy to forget however. Why don't we (also) add it to the default guix profile? To most users this would make sense. No point in running guix pull if you don't want to update guix itself - right? Pj.
Re: New ‘guix pull’
Pjotr Prins writes: > On Thu, Jul 12, 2018 at 02:35:13AM -0700, Chris Marusich wrote: >> l...@gnu.org (Ludovic Courtès) writes: >> >> > I have just pushed a new ‘guix pull’. >> > >> > Feedback welcome! >> >> The biggest hiccup for me was that I didn't realize at first that I had >> to run "guix pull" twice to upgrade from the old style "guix pull". >> Anyway, after I finally did it, things have been working well. > > Yes. The other thing is that guix pull did not update my profile. So, > I had to figure out which guix was installed in the store! “guix pull” updates its own profile only. It updates ~/.config/guix/current, so you need to use Guix from ~/.config/guix/current/bin. -- Ricardo
Re: New ‘guix pull’ - guile-sqlite3 problem
After adding the paths it is fine. So environment is not picking them up. On Thu, Jul 12, 2018 at 02:35:29PM +0200, Pjotr Prins wrote: > So I updated guix pull multiple times :) and both daemon and client > are at > > guix (GNU Guix) ecba50bb79a49b317c4b1e718f4732b36438227f > > Now I am creating an environment to build the source tree with > > env -i /bin/bash --login --noprofile --norc > guix environment guix --ad-hoc help2man git strace \ > pkg-config less vim binutils coreutils grep guile guile-git guile-json > gcc po4a guile-sqlite3 > > Then I run the full monty after a git pull guix master: > > gcc --version > guile --version > rm -rf autom4te.cache/ # to be sure > make clean > ./bootstrap > ./configure --localstatedir=/var > make clean# to be really sure > make clean-go # to be even surer > time make > > which stops in the configure step with > > checking for pkg-config... > /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/pkg-config > checking pkg-config is at least version 0.9.0... yes > configure: checking for guile 2.2 > configure: found guile 2.2 > checking for guile-2.2... no > checking for guile2.2... no > checking for guile-2... no > checking for guile2... no > checking for guile... > /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guile > checking for Guile version >= 2.2... 2.2.3 > checking for guild... > /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guild > checking for guile-config... > /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guile-config > checking if (gnutls) is available... yes > checking if (git) is available... yes > checking if (json) is available... yes > checking whether Guile-Sqlite3 is available and recent enough... no > configure: error: A recent Guile-SQLite3 could not be found; please > install it. > > Erm. It is in the environment! What am I missing here? The PATHs may > point at the problem > > > GUILE_LOAD_COMPILED_PATH=/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/lib/guile/2.2/site-ccache:/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/share/guile/site/2.2:/gnu/store/bcmf06k2n1pfwqkzpclvvc3w9jdfi71a-guile-json-0.6.0/lib/guile/2.2/site-ccache:/gnu/store/jvsgj5l4lgdknari34rv29wv253n8jcl-gnutls-3.5.13/lib/guile/2.2/site-ccache:/gnu/store/x4c2a0z0lf9zp9kgx96k5aplw4ihnak1-guile-git-0.0-4.951a32c/lib/guile/2.2/site-ccache:/gnu/store/x8cwg5bm9y2dk1mrhisr63b5q98mbhhz-guile-bytestructures-20170402.91d042e/lib/guile/2.2/site-ccache:/gnu/store/8n8wwvbkarxqxkn0kc6p5iin0ywiidb0-guile-ssh-0.11.2/lib/guile/2.2/site-ccache > > GUILE_LOAD_PATH=/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/share/guile/site/2.2:/gnu/store/bcmf06k2n1pfwqkzpclvvc3w9jdfi71a-guile-json-0.6.0/share/guile/site/2.2:/gnu/store/jvsgj5l4lgdknari34rv29wv253n8jcl-gnutls-3.5.13/share/guile/site/2.2:/gnu/store/x4c2a0z0lf9zp9kgx96k5aplw4ihnak1-guile-git-0.0-4.951a32c/share/guile/site/2.2:/gnu/store/x8cwg5bm9y2dk1mrhisr63b5q98mbhhz-guile-bytestructures-20170402.91d042e/share/guile/site/2.2:/gnu/store/8n8wwvbkarxqxkn0kc6p5iin0ywiidb0-guile-ssh-0.11.2/share/guile/site/2.2 > > Something missing for sure. > > So close to building again - I can almost feel it! > > Pj >
Re: New ‘guix pull’ - guile-sqlite3 problem
So I updated guix pull multiple times :) and both daemon and client are at guix (GNU Guix) ecba50bb79a49b317c4b1e718f4732b36438227f Now I am creating an environment to build the source tree with env -i /bin/bash --login --noprofile --norc guix environment guix --ad-hoc help2man git strace \ pkg-config less vim binutils coreutils grep guile guile-git guile-json gcc po4a guile-sqlite3 Then I run the full monty after a git pull guix master: gcc --version guile --version rm -rf autom4te.cache/ # to be sure make clean ./bootstrap ./configure --localstatedir=/var make clean# to be really sure make clean-go # to be even surer time make which stops in the configure step with checking for pkg-config... /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/pkg-config checking pkg-config is at least version 0.9.0... yes configure: checking for guile 2.2 configure: found guile 2.2 checking for guile-2.2... no checking for guile2.2... no checking for guile-2... no checking for guile2... no checking for guile... /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guile checking for Guile version >= 2.2... 2.2.3 checking for guild... /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guild checking for guile-config... /gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/bin/guile-config checking if (gnutls) is available... yes checking if (git) is available... yes checking if (json) is available... yes checking whether Guile-Sqlite3 is available and recent enough... no configure: error: A recent Guile-SQLite3 could not be found; please install it. Erm. It is in the environment! What am I missing here? The PATHs may point at the problem GUILE_LOAD_COMPILED_PATH=/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/lib/guile/2.2/site-ccache:/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/share/guile/site/2.2:/gnu/store/bcmf06k2n1pfwqkzpclvvc3w9jdfi71a-guile-json-0.6.0/lib/guile/2.2/site-ccache:/gnu/store/jvsgj5l4lgdknari34rv29wv253n8jcl-gnutls-3.5.13/lib/guile/2.2/site-ccache:/gnu/store/x4c2a0z0lf9zp9kgx96k5aplw4ihnak1-guile-git-0.0-4.951a32c/lib/guile/2.2/site-ccache:/gnu/store/x8cwg5bm9y2dk1mrhisr63b5q98mbhhz-guile-bytestructures-20170402.91d042e/lib/guile/2.2/site-ccache:/gnu/store/8n8wwvbkarxqxkn0kc6p5iin0ywiidb0-guile-ssh-0.11.2/lib/guile/2.2/site-ccache GUILE_LOAD_PATH=/gnu/store/k6s1r0pamx0gih2d4mg87ypyz23h7rzq-profile/share/guile/site/2.2:/gnu/store/bcmf06k2n1pfwqkzpclvvc3w9jdfi71a-guile-json-0.6.0/share/guile/site/2.2:/gnu/store/jvsgj5l4lgdknari34rv29wv253n8jcl-gnutls-3.5.13/share/guile/site/2.2:/gnu/store/x4c2a0z0lf9zp9kgx96k5aplw4ihnak1-guile-git-0.0-4.951a32c/share/guile/site/2.2:/gnu/store/x8cwg5bm9y2dk1mrhisr63b5q98mbhhz-guile-bytestructures-20170402.91d042e/share/guile/site/2.2:/gnu/store/8n8wwvbkarxqxkn0kc6p5iin0ywiidb0-guile-ssh-0.11.2/share/guile/site/2.2 Something missing for sure. So close to building again - I can almost feel it! Pj
Re: bootstrap integration strategies
> I think that's the main difficulty. I think we'd rather not have > separate bootstrap paths for Intel GNU/Linux on one hand, and everything > else on the other hand. Well, due to the design of mescc-tools; the bootstrap paths only have to be divergent up to the M1-macro level. After that, we could simply use flags make the source work on different platforms > Yet, we know that porting what you already did on x86-linux-gnu to > GNU/Hurd and ARMv7 and AArch64 etc. is going to be a lot of non-trivial > work (especially since historical versions of the GNU toolchain did not > support AArch64, for instance.) Nor RISC-V but that is likely to be a much bigger issue in terms of bootstrapping > Waiting for this to be "solved" (and we don't even know how) would > equate to a status quo. But obviously, it'd be sad to have all this > work already done on Intel and not be able to benefit from it. Actually the work for the stage0 bootstrap steps have already been done on non-x86 hardware (Knight platform to be precise) And the engineering decisions involved where explicitly selected to minimize porting and cross-platform bootstrapping effort. M1-macro and hex2-linker only need flags to be set to build for all of the different supported platforms > So perhaps we'll have to get over it and have a different bootstrap path > on x86-linux-gnu. A multiway bootstrap path that exceeds the requirements of DDC actually > (BTW, I suspect we can get away with using 32-bit bootstrap binaries on > both i686/x86_64 and armv7/aarch64, no?) For AMD64, absolutely, ARM however I am not familiar enough to say > Gash seems to be a low-hanging fruit and a relatively easy thing, > because it's architecture-independent. How > far is it from being able to run typical 'configure' scripts? Well we would have to replace the parser at a bare minimum > I think the day it's able to run 'configure' scripts, we can switch to > it right away without further ado, and then incrementally improve it as > we stumble upon limitations and bugs. Well we only need Gash to get to the build make and bash level, after that its scope can be limited. In theory, someone could hand replace the make build script with a custom version that gash can use right now instead of us enhancing gash > There's also another option you didn't mention: ditching the 2.0 > bootstrap Guile in favor of Mes. That can be done in several steps: > 1. Replace the guile-2.0.*.xz binary tarballs with Mes, and add a step > that builds Guile 2.x using our big bootstrap GCC binary. Slow but possible > 2. Same, but build Guile 2.x, libgc, etc. using MesCC. MesCC can't directly build Guile yet but I do enjoy that ambition ;-) > This could allow us to remove quite a lot of MiBs from our binary seeds. FTFY At this point, we effectively have a rope bridge to full bootstrappability But we still have a lot of details to hammer out, like getting basic ARM support and having the ARM and x86 binaries verify each other's bootstrap; Finding 6502, z80, 8051, 68K, VAX, pdp11, Alpha, MIPS, SPARC and PowerPC/Power Developer(s) to do stage0 work for their platforms and perform the cross verify steps. Hammer out cross-platform build details for MesCC and M2-Planet Jeremiah Orians Cell phone: (517) 896-2948
Re: GSoC update
> Could you expound a bit? That’s a very short summary for all the sweat
> you’ve put in it. :-)
My apologies, at the time I sent the mail in a hurry.
Basically now instead of converting unit files to services individually it
happens in bulk so that it can check if there is a corresponding .socket file
per service file (and the reverse). If there is a corresponding .socket file
then the input and output of the .service will be redirected to the result of
(accept) on the socket corresponding to the .socket file.
It also makes the select that waits for the commands from herd to also wait for
the sockets.
> Also, what is the patch against? It’s not against ‘master’; I suppose
> it’s against the previous state of your own branch, do you have a copy
> of your repo on-line?
It's against the previous patch that I sent, but I can put the branch online.
> It’s OK that the thing doesn’t quite work—we knew it was not an easy
> task. What’s disappointing though is that you didn’t come to us to
> discuss the issues until now. GSoC is not about working in all
> loneliness separately from the rest of the group; it’s about becoming
> part of the group.
>
> On IRC Jelle and I (and possibly others) offered help on the ‘signalfd’
> issue; I also outlined reasonable milestones (first, only use
> signalfd(2) instead of SIGCHLD, then discuss together what it would take
> to Fiberize the whole thing.) It’s sad that you remained stuck instead
> of taking this opportunity to discuss it with us.
Until now (in general, not only during the gsoc) I tried to solve any issues
that had arisen when I was programming by myself, so it was a bit difficult to
change that mindset - I will try to be more communicative after this however.
> The patch changes lots of things and unfortunately, without
> explanations, I do not understand what to do with it. Like what’s the
> new feature? How is it used? What implementation choices were made?
> What’s left to be done?…
The new feature is initial support of .socket unit files. It is used like:
(let* ((port1 (open-input-file "/systemd/test.service"))
(port2 (open-input-file "/systemd/test.socket")))
(apply register-services (unit-files->services `(("/systemd/test.service"
"test" service ,(read-unit-file port1))
("/systemd/test.socket"
"test" socket ,(read-unit-file port2)
(close-port port1)
(close-port port2))
The things that are left are supporting more systemd options, and making it work
properly.
Re: New ‘guix pull’
On Thu, Jul 12, 2018 at 01:04:38PM +0200, Pjotr Prins wrote: > On Thu, Jul 12, 2018 at 02:35:13AM -0700, Chris Marusich wrote: > > l...@gnu.org (Ludovic Courtès) writes: > > > > > I have just pushed a new ‘guix pull’. > > > > > > Feedback welcome! > > > > The biggest hiccup for me was that I didn't realize at first that I had > > to run "guix pull" twice to upgrade from the old style "guix pull". > > Anyway, after I finally did it, things have been working well. > > Yes. The other thing is that guix pull did not update my profile. So, > I had to figure out which guix was installed in the store! One has to > capture the git commit when running guix pull and look for that in the > store path, e.g. > > ecba50bb79a49b317c4b1e718f4732b36438227f > > mapped to > > /gnu/store/a5cixasz02ynhppna33zxymxahp3vl13-guix-ecba50bb7/bin/guix > > Next restart the daemon with the new path and all should be well. Another weirdness wash when using above guix daemon and client: /gnu/store/a5cixasz02ynhppna33zxymxahp3vl13-guix-ecba50bb7/bin/guix pull /gnu/store/8ahlc84zaj0iw9jpsdpa9wx4ms0s9cma-guix-daemon-0.15.0-1.4876bc8/libexec/guix/offload: line 8: /root/.config/current/bin/guix: No such file or directory guix pull: error: build failed: unexpected EOF reading a line now where did that come from ;). This got not cleaned up by Ricardo's suggestion guix gc --verify=repair,contents and it was probably caused by some manual copy of the binary tar installation - you know I try things, it is evil. ln -s /gnu/store/a5cixasz02ynhppna33zxymxahp3vl13-guix-ecba50bb7/ /root/.config/current got it going again. For posterity. Pj.
Re: New ‘guix pull’
On Thu, Jul 12, 2018 at 02:35:13AM -0700, Chris Marusich wrote: > l...@gnu.org (Ludovic Courtès) writes: > > > I have just pushed a new ‘guix pull’. > > > > Feedback welcome! > > The biggest hiccup for me was that I didn't realize at first that I had > to run "guix pull" twice to upgrade from the old style "guix pull". > Anyway, after I finally did it, things have been working well. Yes. The other thing is that guix pull did not update my profile. So, I had to figure out which guix was installed in the store! One has to capture the git commit when running guix pull and look for that in the store path, e.g. ecba50bb79a49b317c4b1e718f4732b36438227f mapped to /gnu/store/a5cixasz02ynhppna33zxymxahp3vl13-guix-ecba50bb7/bin/guix Next restart the daemon with the new path and all should be well. Pj.
Re: New ‘guix pull’
l...@gnu.org (Ludovic Courtès) writes: > I have just pushed a new ‘guix pull’. > > Feedback welcome! The biggest hiccup for me was that I didn't realize at first that I had to run "guix pull" twice to upgrade from the old style "guix pull". Anyway, after I finally did it, things have been working well. Thank you for your hard work on this improvement! One of the reasons I have always been a little scared to run "guix pull" frequently is because it was difficult to roll back. Now that the user-installed Guix is managed in a profile of sorts, it will probably be easy to add something like "guix pull --roll-back" to help people recover easily. -- Chris signature.asc Description: PGP signature
Re: shortening the git test suite
Hi Ricardo, Mark, and Ludo, Ricardo Wurmus writes: > What do you think of pushing some package updates only to feature > branches that follow a certain naming convention (e.g. “_update-foo” for > updating the “foo” package), which causes Cuirass to build them and > merge the branch into “master” (semi-)automatically when the build is > successful? If the goal is to ensure that the substitutes for a newly pulled Guix are available when "guix pull" runs, then wouldn't channels accomplish that goal? I realize we don't have channels yet, though. I think your proposal would be a reasonable improvement. In the case of a branch like "_update-foo", Cuirass would probably need to build not only the package "foo" but also any packages that depend on it, right? Since we're talking about updates to the master branch, this shouldn't cause too many rebuilds, so it shouldn't add much load to the build farm. If we're going to allow Cuirass to merge an update branch into master automatically, it might also make sense to allow Cuirass to automatically rebase the update branch onto master before merging it (to ensure the merge is a fast-forward merge), and delete the update branch after merging it (to prevent clutter in commands like "git branch -av"). Alternatively, we could leave those steps to a human and just have Cuirass send a summary email or make the results available via its web interface. I think that with the right automated tests to protect against silly bugs (like accidentally deleting the master branch), it would probably be safer to have Cuirass automatically merge and clean up after itself, rather than rely on humans to do it. Humans are prone to make mistakes, especially when performing tedious tasks. Mark H Weaver writes: >> Personally, I think that the SVN tests are non-essential (after all, >> we’re building Git here, we keep running the individual test suites of >> Git and Subversion, and git-svn interop seems like a thing that only >> upstream need to worry about), which is why I made this proposal. > > Why do you say that only upstream needs to worry about it? I would > think that you could say the same thing about almost any test suite, but > there's always the possibility that our particular combination of input > versions, or the unusual aspects of Guix, might break something that > upstream doesn't know about. > > I would think that git-svn interop is something that any user of the > git/svn integration needs to worry about. I agree: just because I don't use a feature doesn't mean that we shouldn't test that feature if we can. We ought to be able to run all the tests. I think Ricardo's proposal would help increase the availability of substitutes, which would make long-running tests more tolerable. l...@gnu.org (Ludovic Courtès) writes: > Mark H Weaver skribis: > >> l...@gnu.org (Ludovic Courtès) writes: > > [...] > >>> Currently if test fails, the whole derivation fails, and you can’t >>> install your package. If tests were run separately, this would no >>> longer hold: you could get your package regardless of whether tests >>> fail. >> >> Indeed, and I agree that we would need to address this. >> >>> How would you address this? I guess that calls for a new build >>> model, no? >> >> I'm not sure what you mean by "build model" [...] > > [...] > > IOW, this looks like a radical change with lots of potential pitfalls, > one I’d rather not make before 1.0. I agree that long test suites can > be a problem, but for now I think we should focus on package-specific > solutions, to the extent possible, such as what we discussed for Git, as > well as improvements to our build farm infrastructure. > > Thoughts? I agree that changes like those seem especially tricky to get right. While I'm here, I'll suggest another far-fetched idea. Perhaps we could rethink the relationship between packages, phases (of build systems), and derivations. Currently, I believe one package corresponds to one derivation, and the build phases are just procedures executed during that derivation. Perhaps we could change this so that a package corresponds to multiple derivations: one derivation per phase. If that were possible, then a package that runs all phases plus the test phase would be able to easily re-use the results of a package that runs the same phases but not the test phase. However, I'm not sure if this really makes sense, and in any case, the details would probably be difficult to design and implement. -- Chris signature.asc Description: PGP signature
