Re: GSoC: Porting Guix to Hurd week 3+4 report.
Thomas Schwinge tho...@codesourcery.com skribis: On Thu, 04 Jun 2015 22:48:48 +0200, l...@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) wrote: Manolis Ragkousis manolis...@gmail.com skribis: [...] autoreconf ./configure --localstatedir=/var \ --with-libgcrypt-prefix=/gnu/store/... make (Not relevant right now, but why is the libgcrypt path not communicated via the environment variables set up with guix environment? Is that just because there are no appropriate environment variables, or something else?) Also, I wanted to note that I, very guixy, computed that path using: $ guix build libgcrypt warning: failed to install locale: Invalid argument /gnu/store/r16v30hlw2d6n232rm37p53qy8rpr7f2-libgcrypt-1.6.3 /gnu/store/42ls5n7k6lai1k6xfa8v8wks7nn9g3gn-libgcrypt-1.6.3-debug Yes, that’s fine. Next, I ran: $ ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K -c 8 After it fails ..., and eventually reproduced the problem. However, that took a series of steps that was much longer that I had anticipated: It turns out that hydra.gnu.org is not (yet) serving pre-built binaries for this branch, so you ended up rebuilding the world, including doing a complete bootstrap of the distro (info (guix) Bootstrapping). I realize this is terribly inconvenient, so apologies for that! I hope we can soon merge the changes to the “core” packages in the ‘core-updates’ branch, and have ‘wip-hurd’ contain only very specific patches. That way, most of the things will have pre-built binaries available on hydra.gnu.org. Assuming I need to patch the GCC sources, do I just do that in /tmp/nix-build-gcc-4.7.4.drv-0/gcc-4.7.4/, and then, to continue the build, just run the same guix build command again? (I guess I'll also have to preserve my changes elsewhere, as that temporary working tree will be removed upon a successful build?) The simplest way is to use the --with-source option of ‘guix build’, which allows you to specify an alternate source tarball (info (guix) Invoking guix build). Now, from a discussion we had on IRC, I think the problem reported at the beginning of this thread is fixed. Manolis, can you confirm? Also, I think the target should be 4.9 or 4.8, but definitely not 4.7. See also http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00052.html. Thank you! Ludo’.
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Hello Thomas Actually I did a mistake last time, I should have told you to build gcc-4.8, not gcc-4.7, because that's what make-bootstrap.scm use. Btw I managed to build the bootstrap-tarballs. Per Ludo's suggestion I removed --disable-decimal-float flag from %gcc-static and we got the binaries. :-) Did that. From there: git clone git://git.savannah.gnu.org/guix.git (Per Manolis' suggestion, I'm using his GitHub repository.) I am cleaning up my local patches so we can just use wip-hurd from Savannah. cd guix git checkout wip-hurd guix environment guix (Very pedantically, is that last commant completely correct? My understanding is that it sets up an environment for building the version of the guix package that is current known to Guix, and not the version From the checkout of the wip-hurd branch. Of course, the underlying assumption must be that these two versions have the same dependencies, so their environment setup will be the same.) You are correct, these two versions have the same dependencies, so their environment setup will be the same. autoreconf ./configure --localstatedir=/var \ --with-libgcrypt-prefix=/gnu/store/... make (Not relevant right now, but why is the libgcrypt path not communicated via the environment variables set up with guix environment? Is that just because there are no appropriate environment variables, or something else?) Also, I wanted to note that I, very guixy, computed that path using: $ guix build libgcrypt warning: failed to install locale: Invalid argument /gnu/store/r16v30hlw2d6n232rm37p53qy8rpr7f2-libgcrypt-1.6.3 /gnu/store/42ls5n7k6lai1k6xfa8v8wks7nn9g3gn-libgcrypt-1.6.3-debug It will be better if Ludo answers that. Next, I ran: $ ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K -c 8 After it fails ..., and eventually reproduced the problem. However, that took a series of steps that was much longer that I had anticipated: $ ls -lrt /var/log/guix/drvs/*/* I have not yet made an attempt to understand all of that. ;-) Keep in mind that ls -lrt /var/log/guix/drvs/*/* will show you all the logs. You will probably only need to check the last ones. go to /tmp/nix-build-gcc-4.7... and there you can find the failed build. Everything will be there. Assuming I need to patch the GCC sources, do I just do that in /tmp/nix-build-gcc-4.7.4.drv-0/gcc-4.7.4/, and then, to continue the build, just run the same guix build command again? (I guess I'll also have to preserve my changes elsewhere, as that temporary working tree will be removed upon a successful build?) You will need to patch the source at gcc-4.7 in gcc.scm.To add a new patch, first place the new patch in gnu/packages/patches/a-new-patch.patch, then add it in gnu-system.am in the base directory under patchdir and the add it in the source field of the package you want. Search for patches inside gnu/packages/ in files *.scm on how to do that. There are plenty of examples. Also, in case it should be necessary to restart this build step from scratch, do I just remove the /tmp/nix-build-gcc-4.7.4.drv-0/build directory, and then run the same guix build command again? You just rerun the same command. It will just create a new one (drv-1, 2, 3 and so on) in /tmp. No need to delete anything. In case I need to modify the build instructions of an earlier build step, I guess I need to do this the Guix way, that is, edit the respective *.scm file, and re-run guix build? Do I need to invalidate the earlier build, or will that happen automatically? How do I patch the sources of an earlier build step, that is, rebuild a package using the same build instructions but with a different set of sources (say, from a local Git checkout)? It will happen automatically. You patch the source of each package in the way I described you above. If you want to change the source, you just change the source field of the package. Search for git-fetch in gnu/packages and you will understand how to make it use a remote git (I am not sure how to make it use a local repo, Ludo please help :-)). Keep in mind that everytime you change the source you should update the hash of those sources. You can get the hash of a source tarball or a git repo with guix hash. Manolis
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Hello Now, from a discussion we had on IRC, I think the problem reported at the beginning of this thread is fixed. Manolis, can you confirm? Yes, I am doing a local cleanup and I am sending the patches to the list. Also, I think the target should be 4.9 or 4.8, but definitely not 4.7. See also http://lists.gnu.org/archive/html/guix-devel/2015-06/msg00052.html. Yes it was a mistake from my part. Manolis
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Hi! On Thu, 04 Jun 2015 22:48:48 +0200, l...@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) wrote: Manolis Ragkousis manolis...@gmail.com skribis: Hey Thomas, thank you for looking into this. Many thanks to you as well as Ludo for your very helpful description and instructions! I've also spent an hour or two on the Guix manual. So let's start with the easy one. [...] Alternately, there’s a really easy way: grab the binary tarball and follow the steps at http://www.gnu.org/software/guix/download/ [...] Did that. From there: git clone git://git.savannah.gnu.org/guix.git (Per Manolis' suggestion, I'm using his GitHub repository.) cd guix git checkout wip-hurd guix environment guix (Very pedantically, is that last commant completely correct? My understanding is that it sets up an environment for building the version of the guix package that is current known to Guix, and not the version From the checkout of the wip-hurd branch. Of course, the underlying assumption must be that these two versions have the same dependencies, so their environment setup will be the same.) autoreconf ./configure --localstatedir=/var \ --with-libgcrypt-prefix=/gnu/store/... make (Not relevant right now, but why is the libgcrypt path not communicated via the environment variables set up with guix environment? Is that just because there are no appropriate environment variables, or something else?) Also, I wanted to note that I, very guixy, computed that path using: $ guix build libgcrypt warning: failed to install locale: Invalid argument /gnu/store/r16v30hlw2d6n232rm37p53qy8rpr7f2-libgcrypt-1.6.3 /gnu/store/42ls5n7k6lai1k6xfa8v8wks7nn9g3gn-libgcrypt-1.6.3-debug Next, I ran: $ ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K -c 8 After it fails ..., and eventually reproduced the problem. However, that took a series of steps that was much longer that I had anticipated: $ ls -lrt /var/log/guix/drvs/*/* -rw-r--r-- 1 root root 14 Jun 7 18:41 /var/log/guix/drvs/j1/lh997i7bzdkp9p41rxz3qbxim6wi5h-module-import.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 18:41 /var/log/guix/drvs/sl/8ib3bv1mg2cfnr5yar3aac0y8v0vfc-module-import.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 18:41 /var/log/guix/drvs/5x/a7hclvqnbmflx45k9l7lfiwr8af15a-module-import.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 18:43 /var/log/guix/drvs/cx/ghd2z5f3s8is7dkd45xf7k0d5m6fmh-module-import-compiled.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 18:43 /var/log/guix/drvs/ni/ybxnzzf3v97i8qkma1wsvcl7dss5cd-module-import-compiled.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 18:43 /var/log/guix/drvs/px/m5fvm4hyd45jz957h0ygpf16wwmril-module-import-compiled.drv.bz2 -rw-r--r-- 1 root root 678428 Jun 7 18:46 /var/log/guix/drvs/ws/mrcw363m1a1fyrl90v6lw7l9ps6x1g-gcc-4.8.4.tar.xz.drv.bz2 -rw-r--r-- 1 root root 6912 Jun 7 18:47 /var/log/guix/drvs/4i/qpmrg8fp9skrlzqwh6sx10hl7nqmzx-findutils-4.4.2.tar.xz.drv.bz2 -rw-r--r-- 1 root root 1784 Jun 7 18:47 /var/log/guix/drvs/g5/0yphc79id0q893cyj96w0fs7g4vwqh-acl-2.2.52.src.tar.xz.drv.bz2 -rw-r--r-- 1 root root 16872 Jun 7 18:47 /var/log/guix/drvs/da/m2vq48wja8mfllyffv63l20z0nk3fy-findutils-4.4.2.drv.bz2 -rw-r--r-- 1 root root 92676 Jun 7 18:48 /var/log/guix/drvs/8q/gkslsq79mj5yiqlyxsgiqff94gdasr-binutils-cross-boot0-2.25.drv.bz2 -rw-r--r-- 1 root root 99108 Jun 7 18:50 /var/log/guix/drvs/m4/4gzq6psik91wpg641ciwiqdf4ws8dn-perl-5.16.1.drv.bz2 -rw-r--r-- 1 root root 13651 Jun 7 18:50 /var/log/guix/drvs/gi/irgv2f3n6mkpv0fxvz0632abb5s4a0-xz-5.0.4.drv.bz2 -rw-r--r-- 1 root root 453999 Jun 7 18:53 /var/log/guix/drvs/bd/ir2kw05i27qvb4cacj23s6ly0i10ki-gcc-cross-boot0-4.8.4.drv.bz2 -rw-r--r-- 1 root root 225864 Jun 7 18:54 /var/log/guix/drvs/k2/33n6h7i9yk7xrjx2v8jnlpacy34pyj-linux-libre-headers-3.14.37.drv.bz2 -rw-r--r-- 1 root root 37213 Jun 7 18:55 /var/log/guix/drvs/ay/s1lxgxkfwm4v4d7f5g9yq35q2frnin-texinfo-5.2.drv.bz2 -rw-r--r-- 1 root root 64985 Jun 7 18:57 /var/log/guix/drvs/5y/nkn97vzsh25d0v5nlkmk0n1h63fqgl-gettext-boot0-0.19.4.drv.bz2 -rw-r--r-- 1 root root 245674 Jun 7 19:07 /var/log/guix/drvs/5d/ki8gm79l8i8rnbmaj8nwqnfmk57v0q-glibc-intermediate-2.21.drv.bz2 -rw-r--r-- 1 root root 14 Jun 7 19:07 /var/log/guix/drvs/lp/ps9by4yx1k74ndqmcr5s4izd1w2hlp-gcc-cross-boot0-wrapped-4.8.4.drv.bz2 -rw-r--r-- 1 root root 21686 Jun 7 19:08 /var/log/guix/drvs/ly/qs87clgx12m3qs8s29w9704iifs1xm-m4-1.4.17.drv.bz2 -rw-r--r-- 1 root root 97293 Jun 7 19:09 /var/log/guix/drvs/s8/lv0cl9rsnhybgirrjnxkdc4mx2a5rq-perl-5.16.1.drv.bz2 -rw-r--r-- 1 root root 12327 Jun 7 19:10 /var/log/guix/drvs/2r/08d0n4jarsgkr6w7g6iz9lhycx4pq6-bison-3.0.4.drv.bz2 -rw-r--r-- 1 root root 16573 Jun 7 19:11 /var/log/guix/drvs/4f/n2jvxqclaysfsy2lycnbd7inm558nr-bash-light-4.3.33.drv.bz2 -rw-r--r-- 1 root root 242768 Jun 7 19:21
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Manolis Ragkousis manolis...@gmail.com skribis: Hey Thomas, thank you for looking into this. On 2 June 2015 at 18:55, Thomas Schwinge tho...@codesourcery.com wrote: Shame on me, but I've never actively used/built Guix before. I do know about https://github.com/Phant0mas/Guix-on-Hurd, and that there must be a Guix manual existing -- but can you help me get started, please? (Pointers to specific parts of documentation are appreciated, of course.) So let's start with the easy one. Currently Guix can be used only of top of a GNU/Linux system. So with that in mind, clone the repo (the one from savannah git://git.savannah.gnu.org/guix.git ) on one such system and make sure you have the dependencies mentioned at http://www.gnu.org/software/guix/manual/guix.html#Requirements installed. Alternately, there’s a really easy way: grab the binary tarball and follow the steps at http://www.gnu.org/software/guix/download/ (make sure to check the ownership of /var and /root after unpacking it; the extracting the current tarball will incorrectly change the owner.) From there: git clone git://git.savannah.gnu.org/guix.git cd guix git checkout wip-hurd guix environment guix autoreconf ./configure --localstatedir=/var \ --with-libgcrypt-prefix=/gnu/store/... make After it fails go to /tmp/nix-build-gcc-4.7... and there you can find the failed build. Everything will be there. As for the the logs, they can be found with ls -ltr /usr/local/var/log/guix/drvs/*/*. .. or “guix build gcc-4.7 --target=i686-pc-gnu --log-file”. Ludo’.
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Hey Thomas, thank you for looking into this. On 2 June 2015 at 18:55, Thomas Schwinge tho...@codesourcery.com wrote: Shame on me, but I've never actively used/built Guix before. I do know about https://github.com/Phant0mas/Guix-on-Hurd, and that there must be a Guix manual existing -- but can you help me get started, please? (Pointers to specific parts of documentation are appreciated, of course.) So let's start with the easy one. Currently Guix can be used only of top of a GNU/Linux system. So with that in mind, clone the repo (the one from savannah git://git.savannah.gnu.org/guix.git ) on one such system and make sure you have the dependencies mentioned at http://www.gnu.org/software/guix/manual/guix.html#Requirements installed. Then you just need the usual ./configure make make install on the master branch to build and install guix. After that follow http://www.gnu.org/software/guix/manual/guix.html#Setting-Up-the-Daemon on how to setup the builder accounts and how to run the daemon. Congrats you are a proud user of Guix :-). Now just checkout the wip-hurd branch from my github repo. I have some changes there not yet present or ready for the branch in savannah. Run make again and you are ready. Is there a way to have the Guix build process create a shell script (or some other linearized log of command invocations)? At this point, the latter would be earier for me to debug (evem reproduce?), compared to really learning Guix -- which I'll be happy to do (and have been meaning to do for a looong time...), but then it's probably going to take longer for me to really get started. Alternatively/additionally, I'd also appreciate a high-level (coarse-grained) overview about the steps you're executing right now. (Maybe that already documented somewhere, or is obvious from the Guix build description files -- which I have not yet looked at.) That is, something like: 1) install GNU Mach headers: configure [...] make install [...], 2) install Hurd headers: [...], 3) build minimal cross-GCC configured with [...] options: [...], and so on. Now the hard part. Ludovic if I do any mistakes on this part, please correct me. :-) I will mostly point you on the right places in the code to look, because I believe the package definitions are quite simple to understand and they will be much more helpful in understanding what's going on. The command we want to use is ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K The flag -K keeps the failed build directory in /tmp. This commands tries to cross build gcc for --target=i686-pc-gnu. Let's start on how we get the cross-toolchain in gnu/packages/cross-base.scm. The building sequence 1) Build cross-binutils for target system. 2) Build minimal cross-gcc with the flags from cross-gcc-arguments 3) Build xgnumach-headers, xmig, xhurd-headers, xglibc/hurd-headers, xhurd-minimal and finally cross-libc 4) Build cross-gcc with the now present cross-libc. 5) We now have a cross toolchain for our needs. This part works as expected. The part that is failing is the Canadian cross. That is, cross-compiling GCC. To reproduce, in the guix directory, run ./pre-inst-env guix build --target=i686-pc-gnu gcc-4.7 -K gcc-4.7 package definition is located in gnu/packages/gcc.scm and the configure flags are defined in configure-flags in the same file. We build this packages using the cross-toolchain from above. After it fails go to /tmp/nix-build-gcc-4.7... and there you can find the failed build. Everything will be there. As for the the logs, they can be found with ls -ltr /usr/local/var/log/guix/drvs/*/*. I hope this explains most of the things on how to reproduce the error. Thank you again for helping me on this :-) Manolis
Re: GSoC: Porting Guix to Hurd week 3+4 report.
Hi! On Tue, 2 Jun 2015 17:06:48 +0300, Manolis Ragkousis manolis...@gmail.com wrote: [GCC build problem] I am attaching the complete build log and any related log. Thanks, and I tried, but it's too cumbersome to handle that massive amount of data in this way. It'll be much easier if I can reproduce the problem locally. Shame on me, but I've never actively used/built Guix before. I do know about https://github.com/Phant0mas/Guix-on-Hurd, and that there must be a Guix manual existing -- but can you help me get started, please? (Pointers to specific parts of documentation are appreciated, of course.) Is there a way to have the Guix build process create a shell script (or some other linearized log of command invocations)? At this point, the latter would be earier for me to debug (evem reproduce?), compared to really learning Guix -- which I'll be happy to do (and have been meaning to do for a looong time...), but then it's probably going to take longer for me to really get started. Alternatively/additionally, I'd also appreciate a high-level (coarse-grained) overview about the steps you're executing right now. (Maybe that already documented somewhere, or is obvious from the Guix build description files -- which I have not yet looked at.) That is, something like: 1) install GNU Mach headers: configure [...] make install [...], 2) install Hurd headers: [...], 3) build minimal cross-GCC configured with [...] options: [...], and so on. Thanks for all the uneasy frustrating work! You are welcome, I really enjoy working on it. :-). Muahahaha! ;-P The GNU toolchain can be a pain. (Same for every other non-trivial piece of software, heh...) If you suffer through enough of it, you might, one day, make an honest living out of what you've learned. ;-D Grüße, Thomas signature.asc Description: PGP signature
