While trying to update my GuixSD system in the last hour, I found that every attempt by the substituter to download NARs resulted in a 500 "Internal Server Error":
--8<---------------cut here---------------start------------->8--- mhw@jojen ~$ guix package -u substitute: ^Msubstitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 0.0%^Msubstitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0% Downloading https://mirror.hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz (2.3MiB installed)... guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz' failed: 500, "Internal Server Error" guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l58cq0l2j2ikyimjvxyv1dakha2ai6rx-gnome-calendar-3.22.2.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source mhw@jojen ~$ guix package -u Downloading https://mirror.hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz (2.3MiB installed)... guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz' failed: 500, "Internal Server Error" guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l58cq0l2j2ikyimjvxyv1dakha2ai6rx-gnome-calendar-3.22.2.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source --8<---------------cut here---------------end--------------->8--- Bypassing the mirror didn't help: --8<---------------cut here---------------start------------->8--- mhw@jojen ~$ guix package -u . --substitute-urls=https://hydra.gnu.org substitute: ^Msubstitute: updating list of substitutes from 'https://hydra.gnu.org'... 0.0%^Msubstitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0% substitute: ^Msubstitute: updating list of substitutes from 'https://hydra.gnu.org'... 0.0%^Msubstitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0% Downloading https://hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz (2.3MiB installed)... guix substitute: error: download from 'https://hydra.gnu.org/guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz' failed: 500, "Internal Server Error" guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l58cq0l2j2ikyimjvxyv1dakha2ai6rx-gnome-calendar-3.22.2.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source --8<---------------cut here---------------end--------------->8--- Here's what I found in the recent output of the 'guix publish' server on Hydra (running in root's screen session): --8<---------------cut here---------------start------------->8--- GET /74ch6nvjfkj3i56nygwijnaghlpi01d4.narinfo In guix/scripts/publish.scm: 393:2 2 (render-narinfo/cached #<build-daemon 256.96 211f7c0> ...) In guix/store.scm: 663:9 1 (query-path-from-hash-part #<build-daemon 256.96 2...> #) In unknown file: 0 (put-bytevector #<input-output: socket 12> #vu8(# ...) ...) ERROR: In procedure fport_write: Broken pipe GET /guix/nar/qz4mg7sid6avdav158lhr6wziqswpjmx-gnome-calendar-3.22.2.tar.xz In guix/scripts/publish.scm: 491:8 2 (render-nar #<build-daemon 256.96 211f7c0> #<<requ...> ...) In guix/store.scm: 648:0 1 (valid-path? #<build-daemon 256.96 211f7c0> "/gnu/sto...") In unknown file: 0 (put-bytevector #<input-output: socket 12> #vu8(1 ...) ...) ERROR: In procedure fport_write: Broken pipe GET /zhxab8rkbbalgrlhg86q8pzpqi3s6q36.narinfo In guix/scripts/publish.scm: 393:2 2 (render-narinfo/cached #<build-daemon 256.96 211f7c0> ...) In guix/store.scm: 663:9 1 (query-path-from-hash-part #<build-daemon 256.96 2...> #) In unknown file: 0 (put-bytevector #<input-output: socket 12> #vu8(# ...) ...) ERROR: In procedure fport_write: Broken pipe GET /z2rygpahxbpkwx8k164bi5lr5lann357.narinfo In guix/scripts/publish.scm: 393:2 2 (render-narinfo/cached #<build-daemon 256.96 211f7c0> ...) In guix/store.scm: 663:9 1 (query-path-from-hash-part #<build-daemon 256.96 2...> #) In unknown file: 0 (put-bytevector #<input-output: socket 12> #vu8(# ...) ...) ERROR: In procedure fport_write: Broken pipe GET /yw635wxpm4ac40khzd9w6j9v3n72a708.narinfo In guix/scripts/publish.scm: 393:2 2 (render-narinfo/cached #<build-daemon 256.96 211f7c0> ...) In guix/store.scm: 663:9 1 (query-path-from-hash-part #<build-daemon 256.96 2...> #) In unknown file: 0 (put-bytevector #<input-output: socket 12> #vu8(# ...) ...) ERROR: In procedure fport_write: Broken pipe --8<---------------cut here---------------end--------------->8--- I killed the process, restarted it, and now things seem to be working again, but it would be good to know what went wrong here. Mark
