Re: Hurd substitute availability (27.5%) and next steps?
On Mon, 2021-03-08 at 22:47 +, Christopher Baines wrote: > Vincent Legoll writes: > > > often I'll be unable to SSH in > > > > Couldn't you get a console from a virtual serial port from the VM ? > > Maybe, I also want to look at getting the serial port output logged > to a file (if that's even possible). Hello, this runs a qemu image with serial output in a log file: Add to grub.cfg in the hurd image: + serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1 + terminal_input serial + terminal_output serial - multiboot /boot/gnumach.gz root=device:hd0s1 -s --crash-debug + multiboot /boot/gnumach.gz root=device:hd0s1 console=com0 -s --crash- debug Running with graphic and serial console, log file in serial.log: qemu-system-x86_64 -chardev stdio,id=char0,logfile=serial.log,signal=off -serial chardev:char0 -- enable-kvm -m 2048 hurd-cross-serial.img Note: The mailer wraps the command lines. Thanks!
Re: Hurd substitute availability (27.5%) and next steps?
On Tue, Mar 09, 2021 at 07:57:33AM +, Christopher Baines wrote: > > jbra...@dismail.de writes: > > > I'd be happy to reformat this as a guix blog post, unless you'd rather > > I not. > > I think another blog post on the Hurd would be nice, although I'm not > sure what the main takeaway should be. The substitute availability stat > I led this email thread with doesn't really convey to a wider audience, > since they're substitutes from my test substitute server setup. At a minimum it keeps everyone knowing that we're still working on it. I had a blog post which was mostly a list of packages which still weren't building on aarch64-linux a few years ago. It'd also be a good way to group together some of the know-how on how to configure a childhurd with swap and other stuff. -- Efraim Flashner אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted signature.asc Description: PGP signature
Re: Hurd substitute availability (27.5%) and next steps?
jbra...@dismail.de writes: > I'd be happy to reformat this as a guix blog post, unless you'd rather > I not. I think another blog post on the Hurd would be nice, although I'm not sure what the main takeaway should be. The substitute availability stat I led this email thread with doesn't really convey to a wider audience, since they're substitutes from my test substitute server setup. signature.asc Description: PGP signature
Re: Hurd substitute availability (27.5%) and next steps?
I'd be happy to reformat this as a guix blog post, unless you'd rather I not. Thanks, Joshua March 8, 2021 4:57 PM, "Christopher Baines" wrote: > Hey, > > So I finally got around to trying the Guix Build Coordinator agent on > the Hurd [1], and builds have been happening on the guix.cbaines.net > build farm that I've been using to test the Guix Build Coordinator when > building things for substitutes. > > 1: https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00223.html > > The tooling I've got for looking at how far things have progressed isn't > great, but either there are blocking builds or some that have got stuck, > as the agents aren't particularly active currently. Anyway, currently > for one recent revision (1462a11dbb3d2256c8693e56a583cfd100e27609) I > tried, guix weather reports 27.5% of substitutes are available. [2] > > → guix weather --system=i586-gnu --substitute-urls=https://guix.cbaines.net > computing 10,415 package derivations for i586-gnu... > looking for 10,811 store items on https://guix.cbaines.net... > updating substitutes from 'https://guix.cbaines.net'... 100.0% > https://guix.cbaines.net > 27.5% substitutes available (2,977 out of 10,811) > 4,589.5 MiB of nars (compressed) > 18,975.5 MiB on disk (uncompressed) > 0.214 seconds per request (2,313.6 seconds in total) > 4.7 requests per second > (continuous integration information unavailable) > > The Guix Build Coordinator does have a rudimentary feature to find > "blocking" builds, and it reports these as the top derivations that are > failing, and blocking the highest number of builds (in order from top to > bottom), there are more failures than this, but this is just the top > few. > > /gnu/store/xrinbhmmsyyxbr755f4b9xiwi1x4lna8-util-linux-2.35.1.drv > /gnu/store/sgs5vq0y49xhjpc3avi3zpvawzda2j55-tcl-8.6.10.drv > /gnu/store/nzn4iq4pj65ll0k9c1hkmq83v6czxb3n-zstd-1.4.4.drv > /gnu/store/cvcra95jcy42s9z99r9mkv6kiqkhr8s3-tcsh-6.22.02.drv > /gnu/store/qpd0sa86slxz6gb8lzlsybsk8fcd8wzm-valgrind-3.16.1.drv > /gnu/store/23pw46xhzvnghi6ycpcmj7jz01sz7vaz-apr-1.6.5.drv > /gnu/store/cq1kpfb8f76rss3kcc5jkaqagcmrf832-ruby-2.6.5.drv > /gnu/store/jrmm3v34rnsynh59fxjgpi6vkip3slx4-jemalloc-5.2.1.drv > /gnu/store/lry6i5280aryvmr92jalb5drfd7bj912-libcap-2.31.drv > /gnu/store/qbf27f3ymxli367nl82cq2z3xh71frm0-elfutils-0.182.drv > /gnu/store/zfz9hglpn87hwwdf3n3izrrwksb35kl0-kmod-27.drv > /gnu/store/0lgxyqiaxc5hxyfmkyzzfb2sa5ac9g93-alsa-lib-1.2.4.drv > /gnu/store/s3zn4wzyi88vyj0cjd8vvpph1ldzg361-libxshmfence-1.3.drv > /gnu/store/b5awhhqsvgf4c2mxanq2p06nmq1vsgf0-gsl-2.6.drv > /gnu/store/zqbpy5668wmhpjx6pcdyj1vc6w5iz4n1-linux-pam-1.5.1.drv > /gnu/store/h0h1yvq4crcbi9rf9cpn95hkdxaigb7h-openblas-0.3.9.drv > /gnu/store/72b2k49d4kqmnrf0dbjf3bp78nj038d2-jemalloc-4.5.0.drv > /gnu/store/gvnhx587vsh8l32gq0gfpr8czh2fvzqm-nspr-4.29.drv > /gnu/store/qfcgw634621gjwswpcq0c8vi5y2rn435-libcap-2.45.drv > /gnu/store/dzfhypl0vdil3d6kp0w0zcarijbr6shi-flac-1.3.3.drv > /gnu/store/5cv7sbimd774j6zck18v0pazlh200fb6-libaio-0.3.112.drv > /gnu/store/0n6fxczd0asf50daznzm4c8m2rw9yk1n-mrustc-0.9.drv > /gnu/store/3x86mmnf8l2hj1rf74hm8ljbzihj7br0-mtdev-1.1.6.drv > /gnu/store/l4c3jw3225nwd1q66snhxszgjqa8ir35-libtirpc-1.2.5.drv > /gnu/store/gcf220qpz1pjk56qp6pxvi302mg7l185-cdparanoia-10.2.drv > > You can find details about packages and builds from this Guix Data > Service instance [2], the builds can be found on this page for example > [3]. > > 2: https://data.guix-patches.cbaines.net > 3: > https://data.guix-patches.cbaines.net/revision/1462a11dbb3d2256c8693e56a583cfd100e27609/builds?build > server=3=i586-gnu=none_results=50 > > The util-linux and tcl failures are both test related, and I believe > building without the tests works. With this data, given a package which > isn't build for the Hurd, say git as an example, it's possible to find > out which relevant builds are failing [4]. > > 4: > https://data.guix-patches.cbaines.net/build-server/3/build?build_server_build_id=4ebf32a7-5b1e-425f- > 9e9-cd3933875da3 > > So, what are the next steps for Hurd stuff in Guix? > > In terms of getting more packages building, and substitute availability, > personally I think it would be useful to disable tests for packages for > i586-gnu if that gets packages building. It's not ideal to not run the > tests, but it's also difficult to investigate the failures and develop > patches if you don't have substitutes for the software you need to do > that (like Git for example). > > More generally, I also know of a few important Hurd related issues: > > guix gc support: > https://issues.guix.gnu.org/42047 > > chroot for builds: > https://issues.guix.gnu.org/43857 > > decisions around what's in the build environment: > https://guix.gnu.org/en/blog/2020/childhurds-and-substitutes > > I had a go at getting a childhurd with a swap partition, but didn't get > that far: > > https://issues.guix.gnu.org/46726 > > I have reliability issues with the childhurd VMs running Guix Build >
Re: Hurd substitute availability (27.5%) and next steps?
Vincent Legoll writes: > Hello, > > On Mon, Mar 8, 2021 at 10:57 PM Christopher Baines wrote: >> In terms of getting more packages building, and substitute availability, >> personally I think it would be useful to disable tests for packages for >> i586-gnu if that gets packages building. It's not ideal to not run the >> tests, but it's also difficult to investigate the failures and develop >> patches if you don't have substitutes for the software you need to do >> that (like Git for example). > > Is disabling tests for a whole platform possible without patching all the > build recipes ? Probably, although I still think it's worth considering every package on a case by case basis, and looking at fixing the tests. I just think it will be more effective to disable tests for some packages, so that it becomes easier to debug things and work on the Hurd in general. >> often I'll be unable to SSH in > > Couldn't you get a console from a virtual serial port from the VM ? Maybe, I also want to look at getting the serial port output logged to a file (if that's even possible). signature.asc Description: PGP signature
Re: Hurd substitute availability (27.5%) and next steps?
Hello, On Mon, Mar 8, 2021 at 10:57 PM Christopher Baines wrote: > In terms of getting more packages building, and substitute availability, > personally I think it would be useful to disable tests for packages for > i586-gnu if that gets packages building. It's not ideal to not run the > tests, but it's also difficult to investigate the failures and develop > patches if you don't have substitutes for the software you need to do > that (like Git for example). Is disabling tests for a whole platform possible without patching all the build recipes ? > often I'll be unable to SSH in Couldn't you get a console from a virtual serial port from the VM ? BTW, looks like you're doing excellent work ! Thanks -- Vincent Legoll
Hurd substitute availability (27.5%) and next steps?
Hey, So I finally got around to trying the Guix Build Coordinator agent on the Hurd [1], and builds have been happening on the guix.cbaines.net build farm that I've been using to test the Guix Build Coordinator when building things for substitutes. 1: https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00223.html The tooling I've got for looking at how far things have progressed isn't great, but either there are blocking builds or some that have got stuck, as the agents aren't particularly active currently. Anyway, currently for one recent revision (1462a11dbb3d2256c8693e56a583cfd100e27609) I tried, guix weather reports 27.5% of substitutes are available. [2] → guix weather --system=i586-gnu --substitute-urls=https://guix.cbaines.net computing 10,415 package derivations for i586-gnu... looking for 10,811 store items on https://guix.cbaines.net... updating substitutes from 'https://guix.cbaines.net'... 100.0% https://guix.cbaines.net 27.5% substitutes available (2,977 out of 10,811) 4,589.5 MiB of nars (compressed) 18,975.5 MiB on disk (uncompressed) 0.214 seconds per request (2,313.6 seconds in total) 4.7 requests per second (continuous integration information unavailable) The Guix Build Coordinator does have a rudimentary feature to find "blocking" builds, and it reports these as the top derivations that are failing, and blocking the highest number of builds (in order from top to bottom), there are more failures than this, but this is just the top few. /gnu/store/xrinbhmmsyyxbr755f4b9xiwi1x4lna8-util-linux-2.35.1.drv /gnu/store/sgs5vq0y49xhjpc3avi3zpvawzda2j55-tcl-8.6.10.drv /gnu/store/nzn4iq4pj65ll0k9c1hkmq83v6czxb3n-zstd-1.4.4.drv /gnu/store/cvcra95jcy42s9z99r9mkv6kiqkhr8s3-tcsh-6.22.02.drv /gnu/store/qpd0sa86slxz6gb8lzlsybsk8fcd8wzm-valgrind-3.16.1.drv /gnu/store/23pw46xhzvnghi6ycpcmj7jz01sz7vaz-apr-1.6.5.drv /gnu/store/cq1kpfb8f76rss3kcc5jkaqagcmrf832-ruby-2.6.5.drv /gnu/store/jrmm3v34rnsynh59fxjgpi6vkip3slx4-jemalloc-5.2.1.drv /gnu/store/lry6i5280aryvmr92jalb5drfd7bj912-libcap-2.31.drv /gnu/store/qbf27f3ymxli367nl82cq2z3xh71frm0-elfutils-0.182.drv /gnu/store/zfz9hglpn87hwwdf3n3izrrwksb35kl0-kmod-27.drv /gnu/store/0lgxyqiaxc5hxyfmkyzzfb2sa5ac9g93-alsa-lib-1.2.4.drv /gnu/store/s3zn4wzyi88vyj0cjd8vvpph1ldzg361-libxshmfence-1.3.drv /gnu/store/b5awhhqsvgf4c2mxanq2p06nmq1vsgf0-gsl-2.6.drv /gnu/store/zqbpy5668wmhpjx6pcdyj1vc6w5iz4n1-linux-pam-1.5.1.drv /gnu/store/h0h1yvq4crcbi9rf9cpn95hkdxaigb7h-openblas-0.3.9.drv /gnu/store/72b2k49d4kqmnrf0dbjf3bp78nj038d2-jemalloc-4.5.0.drv /gnu/store/gvnhx587vsh8l32gq0gfpr8czh2fvzqm-nspr-4.29.drv /gnu/store/qfcgw634621gjwswpcq0c8vi5y2rn435-libcap-2.45.drv /gnu/store/dzfhypl0vdil3d6kp0w0zcarijbr6shi-flac-1.3.3.drv /gnu/store/5cv7sbimd774j6zck18v0pazlh200fb6-libaio-0.3.112.drv /gnu/store/0n6fxczd0asf50daznzm4c8m2rw9yk1n-mrustc-0.9.drv /gnu/store/3x86mmnf8l2hj1rf74hm8ljbzihj7br0-mtdev-1.1.6.drv /gnu/store/l4c3jw3225nwd1q66snhxszgjqa8ir35-libtirpc-1.2.5.drv /gnu/store/gcf220qpz1pjk56qp6pxvi302mg7l185-cdparanoia-10.2.drv You can find details about packages and builds from this Guix Data Service instance [2], the builds can be found on this page for example [3]. 2: https://data.guix-patches.cbaines.net/ 3: https://data.guix-patches.cbaines.net/revision/1462a11dbb3d2256c8693e56a583cfd100e27609/builds?build_server=3=i586-gnu=none_results=50 The util-linux and tcl failures are both test related, and I believe building without the tests works. With this data, given a package which isn't build for the Hurd, say git as an example, it's possible to find out which relevant builds are failing [4]. 4: https://data.guix-patches.cbaines.net/build-server/3/build?build_server_build_id=4ebf32a7-5b1e-425f-89e9-cd3933875da3 So, what are the next steps for Hurd stuff in Guix? In terms of getting more packages building, and substitute availability, personally I think it would be useful to disable tests for packages for i586-gnu if that gets packages building. It's not ideal to not run the tests, but it's also difficult to investigate the failures and develop patches if you don't have substitutes for the software you need to do that (like Git for example). More generally, I also know of a few important Hurd related issues: guix gc support: https://issues.guix.gnu.org/42047 chroot for builds: https://issues.guix.gnu.org/43857 decisions around what's in the build environment: https://guix.gnu.org/en/blog/2020/childhurds-and-substitutes/ I had a go at getting a childhurd with a swap partition, but didn't get that far: https://issues.guix.gnu.org/46726 I have reliability issues with the childhurd VMs running Guix Build Coordinator agents, often I'll be unable to SSH in, and I'm not quite sure how to debug this. It would be nice to run childhurd VMs for the guix-patches build coordinator, but ideally they would run reliably and wouldn't get stuck. I'm quite new to
