Re: [gentoo-dev] Profile 23.0 testing with stages and binhost (part 2 of 2)
"Andreas K. Huettel" writes: > Hi all, > > the 23.0 profiles are ready for testing, including stage downloads, > binary packages, and update instructions for existing installations, > for all arches. > > [...] > > Note 2: While there are 23.0 split-usr profiles, the *stage* downloads > are *all* of the merged-usr type. Why? Not because I'm a big fan of that, > but because we should try to unify and standardize a bit again - to > avoid too many different build configurations leading to too many Heisenbugs. I don't think this is a good idea. We've promised people that they can keep unmerged-usr if they want, but not having stages means new installs aren't doable, and it also makes testing a pain because you can't easily unmerge. You can easily merge, but you can't easily unmerge. What you can do is provide a limited number of non-merged-usr variants given it's just about saving people rebuilds. (I also think it's the wrong way to do such a change anyway - the releng part should be last after decision-making, not first.) > [...] > Cheers & have fun, thanks, sam signature.asc Description: PGP signature
Re: [gentoo-dev] mirror storage growth rate
Hi Jaco, * we have more stages * the binary packages have to go somewhere * and, temporarily, things are duplicated due to the 17.x / 23.0 profile transition The third point will eventually go away. However, I'm not sure how much it actually contributes. https://www.akhuettel.de/~huettel/plots/mirrors.php If you look at the plots, the distfiles part is surprisingly large. Binary packages (17.x and 23.0) and 17.x stages are under "releases". The 23.0 stages for testing are under "experimental". Lastly, I'm still working on an automated cleanup for outdated "small arches" binary packages (i.e. not arm64 and amd64, these are cleaned automatically already). This just wasn't a priority so far. Hope this helps. -a Am Freitag, 15. März 2024, 09:06:36 CET schrieb Jaco Kroon: > Hi All, > > I was messing with some storage related caching on some of our hosts > this morning when I wondered about how much storage the gentoo mirrors > were consuming. I'm not too worried about the current storage, but I am > noticing that the storage requirements are creeping quite a bit (as per > attached), and if that growth rate continues it may become a problem > *eventually*. > > Can this growth be explained? > > Is it expected to continue at this rate? > > Kind regards, > Jaco > -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, comrel, toolchain, base-system, perl, libreoffice) https://wiki.gentoo.org/wiki/User:Dilfridge signature.asc Description: This is a digitally signed message part.
[gentoo-dev] Profile 23.0 testing with stages and binhost (part 2 of 2)
Hi all, the 23.0 profiles are ready for testing, including stage downloads, binary packages, and update instructions for existing installations, for all arches. IMPORTANT Exception IMPORTANT ** musl on (32bit) arm and x86 does NOT work yet (gcc build failure) ** IMPORTANT Update instructions IMPORTANT https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_update_instructions Stage downloads (temporarily, for all arches): [preferably] https://distfiles.gentoo.org/experimental/x86/23.0_stages/ [direct/osu] https://gentoo.osuosl.org/experimental/x86/23.0_stages/ The changes can be seen here https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_transition and the timeline so far here https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_timeline The update instructions also double as the news item that should be published max. 1-2 weeks from now. They are mostly unchanged compared to my last e-mail, just some wording has been clarified. Note 1: The next steps are, now really in 1-2 weeks max: * make 23.0 profiles the same stability level as 17.x profiles, * degrade 17.x profiles all to exp (so the CI doesn't explode) * publish news item * replace stage downloads with 23.0 version (in situ) Note 2: While there are 23.0 split-usr profiles, the *stage* downloads are *all* of the merged-usr type. Why? Not because I'm a big fan of that, but because we should try to unify and standardize a bit again - to avoid too many different build configurations leading to too many Heisenbugs. Note 3: amd64 now has CET turned on by default. https://docs.kernel.org/next/x86/shstk.html If you have already used the unannounced 23.0 profiles, you should wipe your package cache and emerge -ev world now. Note 4: arm64 does *not* have its equivalent turned on yet since we encountered last-minute problems (guess what, gcc build failure). Note 5: There are no hppa builds yet since our machine is still busy. One gcc build takes about a week there. Cheers & have fun, Andreas -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] mirror storage growth rate
Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian is also packaging Rust this way[1]. I think this was tried long time ago in rust-overlay and failed at the end because the dependency graph was incredibly big. In fact you can see it on the wiki, this is larger than _the bigger_ Haskell packages. I guess the simplest explanation is that software is growing larger, This is not only the case of Rust, but Go, JAVA and .NET and maybe some other projects. Self-bootstrap anyone? :) Can this growth be explained? Is it expected to continue at this rate? Graph is just showing the overall growth, if we associate distfiles to packages we will get the answers. W dniu 15.03.2024 o 16:40, Hoël Bézier pisze: I guess the simplest explanation is that software is growing larger, and in the end we should be expecting to adding new packages faster than removing dead ones. Add to that the grotesque inefficiency of modern programming languages such as Go and Rust. Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian is also packaging Rust this way[1]. [0]: https://wiki.gentoo.org/wiki/Project:Rust/rust-dev [1]: https://wiki.debian.org/Rust Hoël -- Have a great day! ~ Maciej XGQT Barć x...@gentoo.org Gentoo Linux developer (dotnet, emacs, math, ml, nim, scheme, sci) https://wiki.gentoo.org/wiki/User:Xgqt 9B0A 4C5D 02A3 B43C 9D6F D6B1 14D7 4A1F 43A6 AC3C OpenPGP_0x14D74A1F43A6AC3C.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] mirror storage growth rate
I guess the simplest explanation is that software is growing larger, and in the end we should be expecting to adding new packages faster than removing dead ones. Add to that the grotesque inefficiency of modern programming languages such as Go and Rust. Wouldn’t initiatives like rust-dev[0] help with that? I know that Debian is also packaging Rust this way[1]. [0]: https://wiki.gentoo.org/wiki/Project:Rust/rust-dev [1]: https://wiki.debian.org/Rust Hoël signature.asc Description: PGP signature
Re: [gentoo-dev] mirror storage growth rate
On Fri, 2024-03-15 at 10:06 +0200, Jaco Kroon wrote: > Hi All, > > I was messing with some storage related caching on some of our hosts > this morning when I wondered about how much storage the gentoo mirrors > were consuming. I'm not too worried about the current storage, but I am > noticing that the storage requirements are creeping quite a bit (as per > attached), and if that growth rate continues it may become a problem > *eventually*. > > Can this growth be explained? > I guess the simplest explanation is that software is growing larger, and in the end we should be expecting to adding new packages faster than removing dead ones. Add to that the grotesque inefficiency of modern programming languages such as Go and Rust. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] mirror storage growth rate
Hi All, I was messing with some storage related caching on some of our hosts this morning when I wondered about how much storage the gentoo mirrors were consuming. I'm not too worried about the current storage, but I am noticing that the storage requirements are creeping quite a bit (as per attached), and if that growth rate continues it may become a problem *eventually*. Can this growth be explained? Is it expected to continue at this rate? Kind regards, Jaco
