Hi, Maxim Cournoyer <maxim.courno...@gmail.com> skribis:
>> I agree. The ‘guix publish’ TTL¹ at ci.guix was increased to 180 days >> following <https://issues.guix.gnu.org/48926> in 2021. That’s still not >> that much and these days and right now we have 84 TiB free at ci.guix. >> >> I guess we can afford increasing the TTL, probably starting with, say, >> 300 days, and monitoring disk usage. >> >> WDYT? > > While the 84 TiB we have at our disposal is indeed lot, I'd rather we > keep the TTL at 180 days, to keep things more manageable for backup/sync > purposes. Our current TTL currently yields 7 TiB of compressed NARs, > which fits nicely into the hydra-guix-129 10 TiB slice available for > local/simple redundancy (it's still on my TODO, missing the copy bit). > > I've been meaning to document an easy mirroring setup for that > /var/cache/guix/publish directory, and having 14 TiB instead of 7 TiB > there would hurt such setups. Maybe we should learn from what Chris has been doing with the Nar-Herder, too. Ideally, the build farm front-end (‘berlin’ in this case) would be merely a cache for recently-built artifacts, and we’d have long-term storage elsewhere where we could keep nars for several years. The important thing being: we need to decouple the build farm from (long-term) nar provision. > Perhaps a compromise we could do is drop yet another compression format? > We carry both Zstd and LZMA for Berlin, which I see little value in; if > we carried only ZSTD archives we could probably continue having < 10 TiB > of NARs for a TTL of 360 days (although having only 3.5 TiB of NARs to > sync around for mirrors would be great too!). > > What do you think? For compatibility reasons¹ and performance reasons², I would refrain from removing lzip or zstd substitutes, at least for “current” substitutes. For long-term storage though, we could choose to keep lzip only (because it compresses better). Not something we can really do with the current ‘guix publish’ setup though. Thoughts? Ludo’. ¹ Zstd support was added relatively recently. Older daemons may support lzip but not zstd. ² https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/