Hi,
Ludovic Courtès <l...@gnu.org> writes:
> Hi,
>
> Maxim Cournoyer <maxim.courno...@gmail.com> skribis:
>
>> It's something I've been observing for a while, but substitutes are very
>> IO intensive (as can be seen in iotop, the substitute process is waiting
>> on IO > 99% of the time) and is much slower than expected (3 minutes to
>> transfer 100 MiB uncompressed over a 50 mbps downstream link):
>>
>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
>> 13934 be/4 root 1033.09 K/s 1485.06 K/s 0.00 % 93.36 % guile \
>> /gnu/store/vphx2839xv0qj9xwcwrb95592lzrrnx7-guix-1.3.0-3.50dfbbf/bin/guix
>> substitute --substitute
>>
>>
>> The publisher (remote machine) is has its guix-daemon configured via:
>>
>> (service guix-publish-service-type
>> (guix-publish-configuration
>> (advertise? #t)
>> (compression '(("zstd" 3)))
>> (host "0.0.0.0"))) ;listen on all interfaces
>
> Note that in this case nars are built and compressed on the fly on the
> server side, which puts an upper bound on the bandwidth you can achieve.
>
> I showed earlier how I profiled these things:
>
> https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/
>
> If the client is I/O-bound, that’s good: it means we can’t do any better
> (unless we skip unpacking as demonstrated by distri).
>
> If you can provide detailed profiles of either the server side or the
> client side (but in that case, make sure the server is caching things),
> that’d be great!
>
> Otherwise I’m afraid this is not actionable. :-)
Since moving from HDDs to SSDs, I haven't seen this problem, so I
suspect the poor IO of the HDDs was really the culprit rather than
something to do with guile-zstd (and we had also benchmarked the late
some when I experimented with using zstd-compressed man pages).
Closing!
--
Thanks,
Maxim
