Re: [OE-core] how to "prune" dead sstate-cache?
Hi Robert, On 11/18/21 11:43 AM, Robert P. J. Day wrote: > On Thu, 18 Nov 2021, Martin Jansa wrote: > >> openembedded-core/scripts/sstate-cache-management.sh > i just now found that script, and noticed that it is not documented > anywhere in the YP docs. perhaps we need a short section on > sstate-cache management somewhere in the docs. This sounds like a good idea. I'll try to propose something. Thanks! Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158481): https://lists.openembedded.org/g/openembedded-core/message/158481 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
On Thu, 18 Nov 2021, Alexander Kanavin wrote: > There is a caveat here. This will touch all sstate that is needed to > construct an image, but won't touch sstate needed to build > components that go into the image. So target packages are guaranteed > to be kept, but native items may be deleted - I've seen this with > e.g. java compiler that is used for few specific, rarely changing > items. i'm willing to allow the pruning to go a little overboard since it will just be recreated. i'm fine with that. rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158476): https://lists.openembedded.org/g/openembedded-core/message/158476 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
There is a caveat here. This will touch all sstate that is needed to construct an image, but won't touch sstate needed to build components that go into the image. So target packages are guaranteed to be kept, but native items may be deleted - I've seen this with e.g. java compiler that is used for few specific, rarely changing items. Alex On Thu, 18 Nov 2021 at 12:11, Ross Burton wrote: > If you can say that you've touched all the sstate you'll need then the > find trick is sufficient. I use it on our CI to prune any sstate that > hasn't been touched in a month, for example. > > The script is more powerful and can selectively save or destroy > things, but can potentially be more complicated than just nuking > anything old. > > Ross > > On Thu, 18 Nov 2021 at 10:38, Robert P. J. Day > wrote: > > > > > > assuming my (correct) understanding of sstate-cache is that, as time > > goes by, it just gets larger and larger, it will increasingly contain > > content that is of little value anymore, what is the simplest way to > > purge or prune entries that are no longer involved in any desired > > builds? > > > > i'm guessing a simple way to do that is to first delete tmp/, then > > build any current targets/images, immediately followed by something > > like: > > > > $ find sstate-cache -type f -atime +1 -delete > > > > is there anything fancier than that? > > > > rday > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158475): https://lists.openembedded.org/g/openembedded-core/message/158475 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
If you can say that you've touched all the sstate you'll need then the find trick is sufficient. I use it on our CI to prune any sstate that hasn't been touched in a month, for example. The script is more powerful and can selectively save or destroy things, but can potentially be more complicated than just nuking anything old. Ross On Thu, 18 Nov 2021 at 10:38, Robert P. J. Day wrote: > > > assuming my (correct) understanding of sstate-cache is that, as time > goes by, it just gets larger and larger, it will increasingly contain > content that is of little value anymore, what is the simplest way to > purge or prune entries that are no longer involved in any desired > builds? > > i'm guessing a simple way to do that is to first delete tmp/, then > build any current targets/images, immediately followed by something > like: > > $ find sstate-cache -type f -atime +1 -delete > > is there anything fancier than that? > > rday > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158474): https://lists.openembedded.org/g/openembedded-core/message/158474 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
On 11/18/21 11:48, Alexander Kanavin wrote: It would be also nice to confirm that the script is in use by someone, and functions properly. There's a ton of stuff under scripts/ that isn't getting a lot of attention or testing. I use it regularly. /Jacob -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158473): https://lists.openembedded.org/g/openembedded-core/message/158473 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
It would be also nice to confirm that the script is in use by someone, and functions properly. There's a ton of stuff under scripts/ that isn't getting a lot of attention or testing. Alex On Thu, 18 Nov 2021 at 11:43, Robert P. J. Day wrote: > On Thu, 18 Nov 2021, Martin Jansa wrote: > > > openembedded-core/scripts/sstate-cache-management.sh > > i just now found that script, and noticed that it is not documented > anywhere in the YP docs. perhaps we need a short section on > sstate-cache management somewhere in the docs. > > rday > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158472): https://lists.openembedded.org/g/openembedded-core/message/158472 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
On Thu, 18 Nov 2021, Martin Jansa wrote: > openembedded-core/scripts/sstate-cache-management.sh i just now found that script, and noticed that it is not documented anywhere in the YP docs. perhaps we need a short section on sstate-cache management somewhere in the docs. rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158471): https://lists.openembedded.org/g/openembedded-core/message/158471 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
Not really, no. It's also important to stop all builds while this is running - sstate artefacts vanishing during a build will cause chaos. Alex On Thu, 18 Nov 2021 at 11:38, Robert P. J. Day wrote: > > assuming my (correct) understanding of sstate-cache is that, as time > goes by, it just gets larger and larger, it will increasingly contain > content that is of little value anymore, what is the simplest way to > purge or prune entries that are no longer involved in any desired > builds? > > i'm guessing a simple way to do that is to first delete tmp/, then > build any current targets/images, immediately followed by something > like: > > $ find sstate-cache -type f -atime +1 -delete > > is there anything fancier than that? > > rday > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158470): https://lists.openembedded.org/g/openembedded-core/message/158470 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] how to "prune" dead sstate-cache?
openembedded-core/scripts/sstate-cache-management.sh On Thu, Nov 18, 2021 at 11:38 AM Robert P. J. Day wrote: > > assuming my (correct) understanding of sstate-cache is that, as time > goes by, it just gets larger and larger, it will increasingly contain > content that is of little value anymore, what is the simplest way to > purge or prune entries that are no longer involved in any desired > builds? > > i'm guessing a simple way to do that is to first delete tmp/, then > build any current targets/images, immediately followed by something > like: > > $ find sstate-cache -type f -atime +1 -delete > > is there anything fancier than that? > > rday > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158469): https://lists.openembedded.org/g/openembedded-core/message/158469 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] how to "prune" dead sstate-cache?
assuming my (correct) understanding of sstate-cache is that, as time goes by, it just gets larger and larger, it will increasingly contain content that is of little value anymore, what is the simplest way to purge or prune entries that are no longer involved in any desired builds? i'm guessing a simple way to do that is to first delete tmp/, then build any current targets/images, immediately followed by something like: $ find sstate-cache -type f -atime +1 -delete is there anything fancier than that? rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158468): https://lists.openembedded.org/g/openembedded-core/message/158468 Mute This Topic: https://lists.openembedded.org/mt/87140377/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
