On Wed, 2023-02-22 at 17:56 +0000, Alex Kiernan wrote: > I needed to do something about our shared sstate store and waded into > the sstate cache management problem as the existing script takes hours > to run over NFS (which for better or worse is where ours is). I've set > myself the problem of replacing the existing script with something > more extensible, understandable and performant. > > I've got something which I believed was roughly right, but I'm ending > up with questions I can't answer when comparing the two outputs... > > If I run the existing shell script against a tiny sstate-cache (on my > laptop) I get 420 duplicate files eligible for removal, if I run my > script I get 491, looking into the delta, I pick out things like > these: > > $ find sstate-cache/ -name > sstate:libcap-ng:core2-64-poky-linux:0.8.3:r0:core2-64:10:*_populate_sysroot.tar.zst* > -ls > 49067 16 -rw-r--r-- 1 alexk alexk 14435 Feb 18 > 15:29 > sstate-cache/25/59/sstate:libcap-ng:core2-64-poky-linux:0.8.3:r0:core2-64:10:2559429e2a553085bc657f3d2a21a111818061448e5fa2aa16398afb5dad8b90_populate_sysroot.tar.zst > 49129 16 -rw-r--r-- 1 alexk alexk 15205 Feb 18 > 15:29 > sstate-cache/25/59/sstate:libcap-ng:core2-64-poky-linux:0.8.3:r0:core2-64:10:2559429e2a553085bc657f3d2a21a111818061448e5fa2aa16398afb5dad8b90_populate_sysroot.tar.zst.siginfo > 2490392 16 -rw-r--r-- 1 alexk alexk 15204 Feb 20 > 13:24 > sstate-cache/bf/08/sstate:libcap-ng:core2-64-poky-linux:0.8.3:r0:core2-64:10:bf08f26e6bc5ab56ed128441fb05edeef41aa881330d04eea127a93ede51713d_populate_sysroot.tar.zst.siginfo > 339439 16 -rw-r--r-- 1 alexk alexk 14423 Feb 20 > 13:24 > sstate-cache/bf/08/sstate:libcap-ng:core2-64-poky-linux:0.8.3:r0:core2-64:10:bf08f26e6bc5ab56ed128441fb05edeef41aa881330d04eea127a93ede51713d_populate_sysroot.tar.zst > > Which look to me like I should be able to delete the older ones, or am > I missing something? Trying to follow what the existing script is > supposed to do is challenging!
I'd say delete the older one but it does depend a lot on what you're building against the cache (e.g. multiple releases). The system is meant to touch files it uses and the autobuilder just ages out things not touched for X time where X has varied depending on the pressure on our NAS. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#177584): https://lists.openembedded.org/g/openembedded-core/message/177584 Mute This Topic: https://lists.openembedded.org/mt/97165650/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
