> On Oct 18, 2019, at 9:42 AM, Michał Górny <mgo...@gentoo.org> wrote:
>
> Hi, everybody.
>
> It is my pleasure to announce that yesterday (EU) evening we've switched
> to a new distfile mirror layout. Users will be switching to the new
> layout either as they upgrade Portage to 2.3.77 or -- if they upgraded
> already -- as their caches expire (24hrs).
>
> The new layout is mostly a bow towards mirror admins, for some of whom
> having a 60000+ files in a single directory have been a problem.
> However, I suppose some of you also found e.g. the directory index
> hardly usable due to its size.
This sounds like a filesystem issue. Do we know which filesystems are suffering?
ZFS should be fine. I believe ext2/ext3 have problems with this many files.
ext4 is probably okay, but don’t quote me on that.
>
> Throughout a transitional period (whose exact length hasn't been decided
> yet), both layouts will be available. Afterwards, the old layout will
> be removed from mirrors. This has a few implications:
>
> 1. Users who don't upgrade their package managers in time will lose
> the ability of fetching from Gentoo mirrors. This shouldn't be that
> much of a problem given that the core software needed to upgrade Portage
> should all have reliable upstream SRC_URIs.
>
> 2. mirror://gentoo/file URIs will stop working. While technically you
> could use mirror://gentoo/XX/file, I'd rather recommend finally
> discarding its usage and moving distfiles to devspace.
>
> 3. Directly fetching files from distfiles.gentoo.org will become
> a little harder. To fetch a distfile named 'foo-1.tar.gz', you'd have
> to use something like:
>
> $ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2
> 1b
> $ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz
> ...
>
>
> Alternatively, you can:
>
> $ wget http://distfiles.gentoo.org/distfiles/INDEX
>
> and grep for the right path there. This INDEX is also a more
> lightweight alternative to HTML indexes generated by the servers.
>
>
> If you're interested in more background details and some plots, see [1].
>
> [1]
> https://dev.gentoo.org/~mgorny/articles/improving-distfile-mirror-structure.html
>
> --
> Best regards,
> Michał Górny
>
