Hi Alexandre, Alexandre Belloni <alexandre.bell...@bootlin.com> escreveu no dia terça, 21/09/2021 à(s) 11:43:
> Hello Jose, > > On 19/09/2021 21:51:11+0100, Jose Quaresma wrote: > > We don't need extra python collections to count the found files > > on the sstate cache and sstate mirrors. > > The main found collections provides all the files that were found, > > then we only need to count the files on sstate mirror > > > > I believe this is the cause of: > https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/3999 > https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/4039 > https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/4019 > https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/3338 > https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/4012 > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/4004 > https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/4009 > https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/4004 > > I removed both patches and all the builds passed > > > > Signed-off-by: Jose Quaresma <quaresma.j...@gmail.com> > > --- > > meta/classes/sstate.bbclass | 11 +++++------ > > 1 file changed, 5 insertions(+), 6 deletions(-) > > > > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass > > index 29679e6a5e..fc156ac81a 100644 > > --- a/meta/classes/sstate.bbclass > > +++ b/meta/classes/sstate.bbclass > > @@ -893,8 +893,6 @@ BB_HASHCHECK_FUNCTION = "sstate_checkhashes" > > > > def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, > summary=True, **kwargs): > > found = set() > > - foundLocal = set() > > - foundNet = set() > > missed = set() > > > > def gethash(task): > > @@ -927,12 +925,12 @@ def sstate_checkhashes(sq_data, d, siginfo=False, > currentcount=0, summary=True, > > if os.path.exists(sstatefile): > > bb.debug(2, "SState: Found valid sstate file %s" % > sstatefile) > > found.add(tid) > > - foundLocal.add(tid) > > continue > > else: > > - missed.add(tid) > > bb.debug(2, "SState: Looked for but didn't find file %s" % > sstatefile) > > + missed.add(tid) > > > > + foundMirrors = 0 > > mirrors = d.getVar("SSTATE_MIRRORS") > > if mirrors: > > # Copy the data object and override DL_DIR and SRC_URI > > @@ -972,8 +970,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, > currentcount=0, summary=True, > > > connection_cache=thread_worker.connection_cache) > > fetcher.checkstatus() > > bb.debug(2, "SState: Successful fetch test for %s" % > srcuri) > > + foundMirrors += 1 > After looking again at that patch I think the problem may be here as it increments the counter inside of the thread pool without any lock. I sent a new version that uses a different approach to count the files that was found on sstate mirror. https://lists.openembedded.org/g/openembedded-core/topic/patch_sstate_bbclass_count/86052575?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,86052575,previd=1633298061762412833,nextid=1633195213330301948&previd=1633298061762412833&nextid=1633195213330301948 > > found.add(tid) > > - foundNet.add(tid) > > if tid in missed: > > missed.remove(tid) > > except: > > @@ -1034,7 +1032,8 @@ def sstate_checkhashes(sq_data, d, siginfo=False, > currentcount=0, summary=True, > > match = 0 > > if total: > > match = len(found) / total * 100 > > - bb.plain("Sstate summary: Wanted %d Local %d Network %d Missed > %d Current %d (%d%% match, %d%% complete)" % (total, len(foundLocal), > len(foundNet),len(missed), currentcount, match, complete)) > > + bb.plain("Sstate summary: Wanted %d Local %d Mirrors %d Missed > %d Current %d (%d%% match, %d%% complete)" % > > + (total, len(found)-foundMirrors, foundMirrors, len(missed), > currentcount, match, complete)) > > > > if hasattr(bb.parse.siggen, "checkhashes"): > > bb.parse.siggen.checkhashes(sq_data, missed, found, d) > > -- > > 2.33.0 > > > > > > > > > > > > -- > Alexandre Belloni, co-owner and COO, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > -- Best regards, José Quaresma
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#156564): https://lists.openembedded.org/g/openembedded-core/message/156564 Mute This Topic: https://lists.openembedded.org/mt/85726949/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
