Let’s switch to release tarballs On Wed, Mar 30, 2022 at 3:17 AM Richard Purdie < [email protected]> wrote:
> We've been having bandwidth trouble with downloads.yoctoproject.org so we > did > some quick analysis to see what the issue is. Basically in speeding up the > server which was the rate limit, we hit the limits of the hosting pipe. > I'd note > a few things: > > a) it isn't the sstate mirroring, it is nearly all being used by downloads. > > b) 25% of all our bandwidth is going on "git2_sourceware.org.git.binutils- > gdb.git.tar.gz" - i.e. downloading the source mirror binutils tarball > > c) 15% is on git2_sourceware.org.git.glibc.git.tar.gz i.e. glibc > > d) OE-Core has downloads.yoctoproject.org as a MIRROR > > e) poky has it as a PREMIRROR > > What are our options? As far as I can see we could: > > a) increase the pipe from downloads.yoctoproject.org but that does come > at a > non-trivial cost to the project. > > b) Seek help with hosting some of the larger mirror tarballs from people > better > able to host them and have that as a first premirror? > > c) Switch the binutils and glibc recipes to tarballs and patches. I know > Khem > finds this less convenient and they keep moving back and forward but we > keep > running into this issue and having to switch back from git. > > d) To soften the blow of c) we could add devupstream support to the > recipes? We > could script updating the recipe to add the patches? > > e) We could drop the PREMIRRORS from poky. This would stop the SCM targets > from > hitting our mirrors first. That does transfer load to the upstream project > SCMs > though and I'm not sure that will be appreciated. I did sent that patch, > I'm not > sure about it though. > > We are going to need to do *something* though as the current situation > can't > continue. I'm open to other ideas... > > Cheers, > > Richard > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11098): https://lists.yoctoproject.org/g/linux-yocto/message/11098 Mute This Topic: https://lists.yoctoproject.org/mt/90128258/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
