Re: Merging ‘staging’?
Hello, Ludovic Courtès writes: > ‘guix weather -s i686-linux’ says 89% (which is underestimated, because > it wrongfully checks for all the packages, including unsupported > packages), which sounds good. > > We have to check for AArch64 & co. Any takers? Sorry for the delay. I've built some packages from the staging branch on powerpc64le-linux (including Emacs, which brings in a lot of stuff) and it seems good. -- Thanks Thiago
Re: Merging ‘staging’?
On June 11, 2022 9:53:14 AM UTC, "Ludovic Courtès" wrote: >Hi, > >Efraim Flashner skribis: > >> My main concern is that so few of the missing items are queued to be >> built. > >I wonder if that info is accurate. > >For instance, https://ci.guix.gnu.org/build/716980/details shows the >derivation for /gnu/store/2smylcjq5cppxcrj2mn229hlmh6bf7w4-libxft-2.3.3. >It is queued, just not being processed (yet). > >> I've restarted some of them manually, but we're going to need to tell >> cuirass to rebuild those packages. > >It went from 17.1% to 17.3% in two days, even though the AArch64 >machines have been busy all along it seems. Maybe they’ve been >processing the backlog that had accumulated on ‘master’ rather than the >things we care about. > >--8<---cut here---start->8--- >$ ./pre-inst-env guix weather -s aarch64-linux -c200 >computing 18,932 package derivations for aarch64-linux... >looking for 19,704 store items on https://ci.guix.gnu.org... >https://ci.guix.gnu.org > 17.3% substitutes available (3,410 out of 19,704) > at least 22,359.4 MiB of nars (compressed) > 24,158.2 MiB on disk (uncompressed) > 0.008 seconds per request (127.0 seconds in total) > 128.7 requests per second > > 0.0% (0 out of 16,294) of the missing items are queued > at least 1,000 queued builds > powerpc64le-linux: 987 (98.7%) > aarch64-linux: 12 (1.2%) > x86_64-linux: 1 (.1%) > build rate: 40.03 builds per hour > x86_64-linux: 16.83 builds per hour > aarch64-linux: 11.23 builds per hour > i686-linux: 9.03 builds per hour > powerpc64le-linux: 3.04 builds per hour >1496 packages are missing from 'https://ci.guix.gnu.org' for 'aarch64-linux', >among which: > 14236libxft@2.3.3 > /gnu/store/2smylcjq5cppxcrj2mn229hlmh6bf7w4-libxft-2.3.3 > 12379nghttp2@1.44.0 > /gnu/store/4cs553aj8x1xbavfzfdh4gn5idf5ij81-nghttp2-1.44.0-lib > /gnu/store/4phvjrp12n8bfpncyp89414q2pa4d46q-nghttp2-1.44.0 > 10066icu4c@69.1 > /gnu/store/n9nx5rmnhdf3hdswhvns31diayq9vfq2-icu4c-69.1 > 7723 jbig2dec@0.19 > /gnu/store/lrpczm2m0agx0x5rp6kmn8c46mc85l35-jbig2dec-0.19 > 6989 ninja@1.10.2 > /gnu/store/jdq3xrcj0am4j698xr81hwgj8skcn7xq-ninja-1.10.2 > 6927 python-libxml2@2.9.12 > /gnu/store/swgvpb6pb6nf84a08m8flmy7fjhxwvni-python-libxml2-2.9.12 > 6924 mallard-ducktype@1.0.2 > /gnu/store/662lari1slhz56q8333sdczq1av9f799-mallard-ducktype-1.0.2 > 6552 libxt@1.2.1 > /gnu/store/rqqkla9kqjq3182gdqynkq2jb4jf2bl5-libxt-1.2.1-doc > /gnu/store/82pzzkny0gwdfkfa5zvnyy4vbzxqnwyy-libxt-1.2.1 > 6503 python-fonttools@4.28.5 > /gnu/store/vp3g1ddrv067gay8zk1xd7c0kcgq1cv5-python-fonttools-4.28.5 > 5120 python-wheel@0.37.0 > /gnu/store/0rmkfrmrjsvf2hvl9px7h3g91zypmycz-python-wheel-0.37.0 > 5113 python-flit-core-bootstrap@3.5.1 > /gnu/store/pcwmq56r3wb8wdv8blj8ajfsmvcj6wm1-python-flit-core-bootstrap-3.5.1 > 4555 openblas@0.3.20 > /gnu/store/wc8cbkcvgqkdf65fj7drb82plkf8igyw-openblas-0.3.20 > 4231 libxfixes@6.0.0 > /gnu/store/h70i9gyvif5kzpw7a8bdaxhw7sqiv7wz-libxfixes-6.0.0 > 4041 python-markupsafe@2.0.1 > /gnu/store/1k63s700a1sy3pvcms0kax7csqsjzxdr-python-markupsafe-2.0.1 > 3788 libxrandr@1.5.2 > /gnu/store/9gqkb75sr60lxic9a3nx543a6wxz0866-libxrandr-1.5.2 > 3563 eudev@3.2.11 > /gnu/store/nvdlxv30l6f06nclls7j6zi4s8cm1jnc-eudev-3.2.11 > /gnu/store/5ja9bj947iir15ypmqzrwxk04w6vhiyz-eudev-3.2.11-static > 3360 libpsl@0.21.1 > /gnu/store/5jyxcfsgk0hsj6rzfsz84wnap6bn57ka-libpsl-0.21.1 > 3329 libevent@2.1.12 > /gnu/store/28iivyxgqvnp2alx2fyqqx18md7pwja8-libevent-2.1.12-bin > /gnu/store/cvvsdqkcb8z78bh7rhzirv2n4sb4svgg-libevent-2.1.12 > 3320 libxkbfile@1.1.0 > /gnu/store/kk0xk8j8ffgxvfs5kz1vxp2ipz28xwh4-libxkbfile-1.1.0 > 3297 xcb-util@0.4.0 > /gnu/store/n2x5kvy5zxbnh28ak7rbvw6brp97z1c9-xcb-util-0.4.0 > 3279 xcb-util-wm@0.4.1 > /gnu/store/hhasmz0l5f10f9nfg0nx378c7bcgf9v0-xcb-util-wm-0.4.1 > 3263 libxres@1.2.1 > /gnu/store/5mpjhsyq2bbri006g7lh6h82239s8fb8-libxres-1.2.1 > 3215 python-flit-core@3.5.1 > /gnu/store/zldar3i40q3l40fypm8n6fh3kpddxhzq-python-flit-core-3.5.1 > 3213 python-pytz@2022.1 > /gnu/store/byyn7jzs85r2n11hdbk98dk2ikb74z9n-python-pytz-2022.1 > 3205 python-pycparser@2.21 > /gnu/store/4a5l67ama5msmd2qi8w7mvlq64b04znb-python-pycparser-2.21-doc > /gnu/store/98g8chr9pcxyz0czp9fq2w3q8rm2sn14-python-pycparser-2.21 > 3201 python-idna@3.3 > /gnu/store/fj5sc68s4g0rh5gycnzvx2nkmviphfyb-python-idna-3.3 > 3170 python-pretend@1.0.9 > /gnu/store/ksyw3ffmmhizh25ql0jp7yfv5llg55km-python-pretend-1.0.9 > 3132 python-semantic-version@2.8.5 > /gnu/store/0g95jbn19fwr52f5l5j1ibv7h9g7dsyl-python-semantic-version-2.8.5 > 3127 python-asn1crypto@1.4.0 > /gnu/store/12xs95mcz9v9swwyinsagfdn0qa74rgw-python-asn1crypto-1.4.0 > 3126 python-cryptography-vectors@3.4.8 > /gnu/store/5ww2a12dzyq8fbazwkqhn26wv5ahk1xj-python-cryptography-vectors-3.4.8 > 2754
Re: U.S. Midwest based build farm
On 2022-06-11, Maxime Devos wrote: > jbra...@dismail.de schreef op za 11-06-2022 om 16:06 [+]: >> What's good and/or bad about this idea? > > A positive point: extra resources, could be useful for reproducibility > testing, ...? > > A negative point: extra points through with malware can be introduced > (->compromises). Can be solved by reproducible builds and variation of > "guix challenge". Unfortunately, "guix challenge" is inherently racy. > "guix substitute" currently only checks that the narinfo has a _single_ > authorised signature, maybe it can be adjusted to allow the user to > ask: ‘only consider a substitute to be authorised if the same hash is > signed by N different authorised keys’? Even without "signed by N" reproducible builds and guix substitute servers have some very interesting qualities! It's been a while since I've tested, but I seem to recall setting up a situation where I had a untrusted substitute server locally (e.g. I didn't add that server's keys to guix's trusted keys), and also configured my guix machine to use the default guix substitute servers (which were in the guix acl for authorized keys). Roughly approximated as: guix COMMAND --substitute-urls='https://untrusted.example.com https://ci.guix.gnu.org https://bordeaux.guix.gnu.org' For packages that build reproducibly, you could actually download the signatures (which are fairly small) from the "trusted" substitute servers, but download the actual packages (which can be quite large) from the "untrusted" substitute server... I've actually been wondering if one couldn't make this behavior more explicit, e.g. have substitute servers that *only* served signatures, and substitute servers that *only* served (unsigned?) builds. I guess you can more-or-less create this effect by never publishing the key that packages are signed with for the untrusted/untrustable substitute server? Anything that you can download from the "unstrusted" server is demonstratably reproducible, because a "trusted" server also built it. presuming, of course, both servers are actually performing the builds, but worst case you still get the bit-for-bit identical packages as the "trusted" substitutes. It's an awesome way to be able to distribute the downloads for that 80% and growing number of packages that are reproducible away from the default substitute servers, without actually having to even place much trust an arbitrary third party, other than metadata about what you've downloaded... I remember this being one of my favorite features of guix that I learned about early on, but haven't really done much with it! live well, vagrant signature.asc Description: PGP signature
Re: U.S. Midwest based build farm
June 11, 2022 4:00 PM, "Maxime Devos" wrote: > jbra...@dismail.de schreef op za 11-06-2022 om 16:06 [+]: > >> What's good and/or bad about this idea? > > A positive point: extra resources, could be useful for reproducibility > testing, ...? That's actually a good idea. I could give limited ssh access to a few guix developers. Those guix developers could use my old and hopefully more powerful machines to quickly compile software. Rust takes ages to compile... > > A negative point: extra points through with malware can be introduced > (->compromises). Can be solved by reproducible builds and variation of > "guix challenge". Unfortunately, "guix challenge" is inherently racy. > "guix substitute" currently only checks that the narinfo has a _single_ > authorised signature, maybe it can be adjusted to allow the user to > ask: ‘only consider a substitute to be authorised if the same hash is > signed by N different authorised keys’? > Thanks for the feedback. We could also use the machines as a mirror or an additional substitute server. > Other points: ...? > > Greetings, > Maxime.
Re: U.S. Midwest based build farm
jbra...@dismail.de schreef op za 11-06-2022 om 16:06 [+]: > What's good and/or bad about this idea? A positive point: extra resources, could be useful for reproducibility testing, ...? A negative point: extra points through with malware can be introduced (->compromises). Can be solved by reproducible builds and variation of "guix challenge". Unfortunately, "guix challenge" is inherently racy. "guix substitute" currently only checks that the narinfo has a _single_ authorised signature, maybe it can be adjusted to allow the user to ask: ‘only consider a substitute to be authorised if the same hash is signed by N different authorised keys’? Other points: ...? Greetings, Maxime. signature.asc Description: This is a digitally signed message part
U.S. Midwest based build farm
Hey guix, I live near a big university that sells old Dell 7020 optiplex machines. So each desktop machine costs about $200 - $250, depending on how the current market rate is for hard drive and RAM. My current landlord has an unused basement. It should be somewhat easy to get an ethernet cord to the basement, plugged up to 2+ desktop machines. My ISP is metronet, which is usually pretty friendly to self-hosting things. If guix is interested in paying for some of the ISP bills, electric bills, and/or renting my landlord's basement, I think it would be pretty cool to try to set up another build farm. Why I am the best candidate for this role: I'm not. I have a pretty bad track record for being lazy. I have still not finished my opensmtpd configuration for the opensmtpd service. What do you all think? What's good and/or bad about this idea? Thanks, Joshua
Re: Merging ‘staging’?
Ludovic Courtès writes: > It went from 17.1% to 17.3% in two days, even though the AArch64 > machines have been busy all along it seems. Maybe they’ve been > processing the backlog that had accumulated on ‘master’ rather than the > things we care about. Per https://issues.guix.gnu.org/55848, it looks like grunewald/kreuzberg/pankow are busy processing jobs, but each time failing after 15 minutes with the same network-level error. > Objections? SGTM!
Re: Merging ‘staging’?
Hi, Efraim Flashner skribis: > My main concern is that so few of the missing items are queued to be > built. I wonder if that info is accurate. For instance, https://ci.guix.gnu.org/build/716980/details shows the derivation for /gnu/store/2smylcjq5cppxcrj2mn229hlmh6bf7w4-libxft-2.3.3. It is queued, just not being processed (yet). > I've restarted some of them manually, but we're going to need to tell > cuirass to rebuild those packages. It went from 17.1% to 17.3% in two days, even though the AArch64 machines have been busy all along it seems. Maybe they’ve been processing the backlog that had accumulated on ‘master’ rather than the things we care about. --8<---cut here---start->8--- $ ./pre-inst-env guix weather -s aarch64-linux -c200 computing 18,932 package derivations for aarch64-linux... looking for 19,704 store items on https://ci.guix.gnu.org... https://ci.guix.gnu.org 17.3% substitutes available (3,410 out of 19,704) at least 22,359.4 MiB of nars (compressed) 24,158.2 MiB on disk (uncompressed) 0.008 seconds per request (127.0 seconds in total) 128.7 requests per second 0.0% (0 out of 16,294) of the missing items are queued at least 1,000 queued builds powerpc64le-linux: 987 (98.7%) aarch64-linux: 12 (1.2%) x86_64-linux: 1 (.1%) build rate: 40.03 builds per hour x86_64-linux: 16.83 builds per hour aarch64-linux: 11.23 builds per hour i686-linux: 9.03 builds per hour powerpc64le-linux: 3.04 builds per hour 1496 packages are missing from 'https://ci.guix.gnu.org' for 'aarch64-linux', among which: 14236 libxft@2.3.3 /gnu/store/2smylcjq5cppxcrj2mn229hlmh6bf7w4-libxft-2.3.3 12379 nghttp2@1.44.0 /gnu/store/4cs553aj8x1xbavfzfdh4gn5idf5ij81-nghttp2-1.44.0-lib /gnu/store/4phvjrp12n8bfpncyp89414q2pa4d46q-nghttp2-1.44.0 10066 icu4c@69.1 /gnu/store/n9nx5rmnhdf3hdswhvns31diayq9vfq2-icu4c-69.1 7723 jbig2dec@0.19 /gnu/store/lrpczm2m0agx0x5rp6kmn8c46mc85l35-jbig2dec-0.19 6989 ninja@1.10.2 /gnu/store/jdq3xrcj0am4j698xr81hwgj8skcn7xq-ninja-1.10.2 6927 python-libxml2@2.9.12 /gnu/store/swgvpb6pb6nf84a08m8flmy7fjhxwvni-python-libxml2-2.9.12 6924 mallard-ducktype@1.0.2 /gnu/store/662lari1slhz56q8333sdczq1av9f799-mallard-ducktype-1.0.2 6552 libxt@1.2.1 /gnu/store/rqqkla9kqjq3182gdqynkq2jb4jf2bl5-libxt-1.2.1-doc /gnu/store/82pzzkny0gwdfkfa5zvnyy4vbzxqnwyy-libxt-1.2.1 6503 python-fonttools@4.28.5 /gnu/store/vp3g1ddrv067gay8zk1xd7c0kcgq1cv5-python-fonttools-4.28.5 5120 python-wheel@0.37.0 /gnu/store/0rmkfrmrjsvf2hvl9px7h3g91zypmycz-python-wheel-0.37.0 5113 python-flit-core-bootstrap@3.5.1 /gnu/store/pcwmq56r3wb8wdv8blj8ajfsmvcj6wm1-python-flit-core-bootstrap-3.5.1 4555 openblas@0.3.20 /gnu/store/wc8cbkcvgqkdf65fj7drb82plkf8igyw-openblas-0.3.20 4231 libxfixes@6.0.0 /gnu/store/h70i9gyvif5kzpw7a8bdaxhw7sqiv7wz-libxfixes-6.0.0 4041 python-markupsafe@2.0.1 /gnu/store/1k63s700a1sy3pvcms0kax7csqsjzxdr-python-markupsafe-2.0.1 3788 libxrandr@1.5.2 /gnu/store/9gqkb75sr60lxic9a3nx543a6wxz0866-libxrandr-1.5.2 3563 eudev@3.2.11 /gnu/store/nvdlxv30l6f06nclls7j6zi4s8cm1jnc-eudev-3.2.11 /gnu/store/5ja9bj947iir15ypmqzrwxk04w6vhiyz-eudev-3.2.11-static 3360 libpsl@0.21.1 /gnu/store/5jyxcfsgk0hsj6rzfsz84wnap6bn57ka-libpsl-0.21.1 3329 libevent@2.1.12 /gnu/store/28iivyxgqvnp2alx2fyqqx18md7pwja8-libevent-2.1.12-bin /gnu/store/cvvsdqkcb8z78bh7rhzirv2n4sb4svgg-libevent-2.1.12 3320 libxkbfile@1.1.0 /gnu/store/kk0xk8j8ffgxvfs5kz1vxp2ipz28xwh4-libxkbfile-1.1.0 3297 xcb-util@0.4.0 /gnu/store/n2x5kvy5zxbnh28ak7rbvw6brp97z1c9-xcb-util-0.4.0 3279 xcb-util-wm@0.4.1 /gnu/store/hhasmz0l5f10f9nfg0nx378c7bcgf9v0-xcb-util-wm-0.4.1 3263 libxres@1.2.1 /gnu/store/5mpjhsyq2bbri006g7lh6h82239s8fb8-libxres-1.2.1 3215 python-flit-core@3.5.1 /gnu/store/zldar3i40q3l40fypm8n6fh3kpddxhzq-python-flit-core-3.5.1 3213 python-pytz@2022.1 /gnu/store/byyn7jzs85r2n11hdbk98dk2ikb74z9n-python-pytz-2022.1 3205 python-pycparser@2.21 /gnu/store/4a5l67ama5msmd2qi8w7mvlq64b04znb-python-pycparser-2.21-doc /gnu/store/98g8chr9pcxyz0czp9fq2w3q8rm2sn14-python-pycparser-2.21 3201 python-idna@3.3 /gnu/store/fj5sc68s4g0rh5gycnzvx2nkmviphfyb-python-idna-3.3 3170 python-pretend@1.0.9 /gnu/store/ksyw3ffmmhizh25ql0jp7yfv5llg55km-python-pretend-1.0.9 3132 python-semantic-version@2.8.5 /gnu/store/0g95jbn19fwr52f5l5j1ibv7h9g7dsyl-python-semantic-version-2.8.5 3127 python-asn1crypto@1.4.0 /gnu/store/12xs95mcz9v9swwyinsagfdn0qa74rgw-python-asn1crypto-1.4.0 3126 python-cryptography-vectors@3.4.8 /gnu/store/5ww2a12dzyq8fbazwkqhn26wv5ahk1xj-python-cryptography-vectors-3.4.8 2754 python-pyyaml@6.0 /gnu/store/pxxnkbrljhizypjpfdl27djjzsyflnic-python-pyyaml-6.0 2542 python-dnspython@2.1.0 /gnu/store/m66znr3nwqb7w132yiw9iznzqif2z84r-python-dnspython-2.1.0 2539
Re: On commit access, patch review, and remaining healthy
Hi, Thiago Jung Bauermann skribis: > The binutils-gdb repo has a Python script to generate a skeleton > ChangeLog. I don't know how well it would work for Scheme patches: > > https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=contrib/mklog.py;hb=HEAD I believe the ‘etc/committer.scm’ scripts plays a similar role (it’s even more sophisticated!), but perhaps it’s too little known. Ludo’.
Re: Merging ‘staging’?
On Thu, Jun 09, 2022 at 09:02:14PM +0200, pelzflorian (Florian Pelz) wrote: > I will try again with staging at 091eb323ba27. But it will take a > long time; feel free to proceed regardless. All my manifest built and runs on rock64 aarch64. Thank you all! Regards, Florian
