On Mon, Feb 25, 2019 at 8:46 PM Timothy Froehlich <[email protected]> wrote: > > I've been spending a bit too long this past week trying to build up a > reproducable build infrastructure in AWS and I've got very little experience > with cloud infrastucture and I'm wondering if I'm going in the wrong > direction. I'm attempting to host my sstate_cache as a mirror in a private S3 > bucket, and I believe I have everything configured properly, including > exposing the bucket to http requests, since I can wget files that I've > previously synced up to the bucket. However if I add in the SSTATE_MIRRORS to > my build, bitbake slows to a crawl (it's a powerful VM) and barely seems to > get anything. The EC2 instance is in the same region as the S3 bucket, roles > have been configured properly to allow access, etc. > > I'm not looking for help debugging this, I just want to know whether I'm > right that hosting my sstate in an S3 bucket should work. I've only been able > to find one mention of it being done with no reproduction hints. >
A lot of the files end up with plus signs in the name. This causes problems with retrieving files through http access with S3. S3 translates all plus signs to spaces, even those in the file path. So if my-file_v1.0+g1241876 actually exists as named in S3 an http request for that file will trigger the server to look for "my-file_v1.0 g1241876" I ran into this problem trying to host an opkg repository in S3 for upgrading. It may mostly work for you but there will be many files that it will never be able to find in your S3 hosted sstate. Maybe this has been fixed by AWS. I noticed the problem a year or two ago. https://stackoverflow.com/questions/36734171/how-to-decide-if-the-filename-has-a-plus-sign-in-it#36758133 https://news.ycombinator.com/item?id=15398804 -- _______________________________________________ yocto mailing list [email protected] https://lists.yoctoproject.org/listinfo/yocto
