On Mon, Sep 4, 2017 at 9:33 AM, Patrick Ohly <patrick.o...@intel.com> wrote:
> On Fri, 2017-09-01 at 17:04 +0200, Andrea Galbusera wrote: > > Hi Maciej, > > > > On Fri, Sep 1, 2017 at 4:08 PM, Maciej Borzęcki <maciej.borzecki@rndi > > ty.com> wrote: > > > On Fri, Sep 1, 2017 at 3:54 PM, Andrea Galbusera <giz...@gmail.com> > > > wrote: > > > > Hi! > > > > > > > > I was trying to share sstate between different hosts, but the > > > consumer build > > > > system seems to be unable to use re-use any sstate object. My > > > scenario is > > > > setup as follows: > > > > > > > > * The cache was populated by a pristine qemux86 core-image- > > > minimal build of > > > > morty. This was done in a crops/poky container (running in docker > > > on Mac) > > > > * The cache was then served via HTTP > > > > > > Make sure that you use a decent HTTP server. Simple `python3 -m > > > http.server` will quickly choke when the mirror is being checked. > > > Also > > > running bitbake -DDD -v makes investigating this much easier. > > > > To be honest, the current server was indeed setup with python's > > SimpleHTTPServer... As you suggest, I checked the verbose debug log > > and noticed what's happening behind the apparently happy "Checking > > sstate mirror object availability" step. After a first "SState: > > Successful fetch test for" that I see correctly served with 200 on > > the server side, tests for any other sstate object suddenly and > > systematically fail with logs like this: > ... > > DEBUG: checkstatus() urlopen failed: <urlopen error [Errno 9] Bad > > file descriptor> > > More recent bitbake should not fail like that anymore. It's still > better to use an HTTP server that performs better, though. > > commit 6fa07752bbd3ac345cd8617da49a70e0b2dd565f > Author: Patrick Ohly <patrick.o...@intel.com> > Date: Mon Jul 17 15:25:10 2017 +0200 > > fetch2/wget.py: improve error handling during sstate check > > When the sstate is accessed via HTTP, the existence check can fail due > to network issues, in which case bitbake silently continues without > sstate. > > One such network issue is an HTTP server like Python's own SimpleHTTP > which closes the TCP connection despite an explicit "Keep-Alive" in > the HTTP request header. The server does that without a "close" in the > HTTP response header, so the socket remains in the connection cache, > leading to "urlopen failed: <urlopen error [Errno 9] Bad file > descriptor>" (only visible in "bitbake -D -D" output) when trying to > use the cached connection again. > > The connection might also get closed for other reasons (proxy, > timeouts, etc.), so this is something that the client should be able > to handle. > > This is achieved by checking for the error, removing the bad > connection, and letting the check_status() method try again with a new > connection. It is necessary to let the second attempt fail > permanently, because bad proxy setups have been observed to also lead > to such broken connections. In that case, we need to abort for real > after trying twice, otherwise a build would just hang forever. > > [YOCTO #11782] > > -- > Best Regards, Patrick Ohly > > The content of this message is my personal opinion only and although > I am an employee of Intel, the statements I make here in no way > represent Intel's position on the issue, nor am I authorized to speak > on behalf of Intel on this matter. > Thank you Patrick for pointing that out. While debugging the issue on morty I had some reminiscence of seeing your patch on the list, but I wasn't able to find it back in morty's history hence inferring it landed later on. Anyway doing this test on morty was a requirement... Thanks again for your clarification. Would such a patch be suitable for eventually being back-ported to morty?
-- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto