Hi everyone, Thank you for the discussion in the patch review meeting. After discussing this further with Michael, I’m providing a summary of the technical details and the various perspectives shared:
* Release Timing: Given how close we are to the next release, we have decided to put this patch series on hold. Modifying these crucial lines of code carries a risk that is not feasible at this stage. We will revisit the implementation and timing immediately after the release. * Architectural Goal & Cache Management: Our primary objective is to move toward standard mtime and atime behavior. Currently, non- standard usage makes it difficult for different environments to interact with the cache as expected. Adhering to standard behavior allows for better cache management, as it for example provides the necessary telemetry to identify objects that are "old" but still actively "in use." * We will have a look a the sstate-cache-management.py script and check if the script can be extended with some prune features which we need on our infrastructure. Follow-up Discussion: We also recognize the specific performance challenges with Kubernetes CephFS (particularly the server-side propagation of atime changes). While this is a key driver for these improvements, we will address the specific CephFS optimizations and related performance impacts in a follow-up discussion and dedicated discussion to keep the current architectural changes focused. Best regards, Adrian On Fri, 2026-09-11 at 09:01 +0200, Michael Haener wrote: > Marking an object in the sstate cache as in use is done with a plain > touch, which writes the modification time along with the access time. > Every build that merely reads an object therefore makes it look newly > created. > > scripts/sstate-cache-management.py --remove-duplicated keeps the > entry > with the newest modification time. As every use rewrites it, building > an > older configuration last keeps its object and removes the newer one. > > Refresh only the access time and leave the modification time alone. > The > modification time then tells when an object was created, the access > time > when it was last used. > > Signed-off-by: Michael Haener <[email protected]> > Reviewed-by: Adrian Freihofer <[email protected]> > Reviewed-by: Peter Marko <[email protected]> > --- > v3 (after review by Zhan Xusheng): > - restored the original error handling: PermissionError and EROFS are > ignored, everything else propagates as before > - use os.utime(ns=...) to round trip the modification time exactly > - replaced the vague motivation with the concrete effect on > sstate-cache-management.py --remove-duplicated > > v2: > - dropped SSTATE_ATIME_UPDATE_AFTER, refreshing only the access time > is now > unconditional > - renamed the helper to sstate_touch_atime() > - reduced the scope to the timestamp fix > > meta/classes-global/sstate.bbclass | 39 ++++++++++++++++++---------- > -- > 1 file changed, 24 insertions(+), 15 deletions(-) > > diff --git a/meta/classes-global/sstate.bbclass b/meta/classes- > global/sstate.bbclass > index b2fa93650a..76544eb954 100644 > --- a/meta/classes-global/sstate.bbclass > +++ b/meta/classes-global/sstate.bbclass > @@ -699,15 +699,7 @@ def sstate_package(ss, d): > if not os.path.exists(siginfo): > bb.siggen.dump_this_task(siginfo, d) > else: > - try: > - os.utime(siginfo, None) > - except PermissionError: > - pass > - except OSError as e: > - # Handle read-only file systems gracefully > - import errno > - if e.errno != errno.EROFS: > - raise e > + sstate_touch_atime(siginfo) > > return > > @@ -802,7 +794,10 @@ python sstate_create_and_sign_package () { > # Best effort touch > def touch(file): > try: > - file.touch() > + if file.exists(): > + sstate_touch_atime(file) > + else: > + file.touch() > except: > pass > > @@ -936,13 +931,27 @@ sstate_unpack_package () { > # Update both any file and any symlink pointing to the file for > sigs as well as the file > for file in ${SSTATE_PKG} ${SSTATE_PKG}.sig ${SSTATE_PKG}.siginfo > do > - [ ! -e $file ] || touch $file 2>/dev/null || true > - [ ! -e $file ] || touch --no-dereference $file 2>/dev/null || true > + [ ! -e $file ] || touch -a $file 2>/dev/null || true > + [ ! -e $file ] || touch -a --no-dereference $file 2>/dev/null || > true > done > } > > BB_HASHCHECK_FUNCTION = "sstate_checkhashes" > > +def sstate_touch_atime(path): > + import errno > + import time > + try: > + stat_info = os.stat(path) > + # nanoseconds avoid shifting the modification time through > float rounding > + os.utime(path, ns=(time.time_ns(), stat_info.st_mtime_ns)) > + except PermissionError: > + pass > + except OSError as e: > + # Handle read-only file systems gracefully > + if e.errno != errno.EROFS: > + raise e > + > def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, > summary=True, **kwargs): > import itertools > > @@ -978,10 +987,10 @@ def sstate_checkhashes(sq_data, d, > siginfo=False, currentcount=0, summary=True, > sstatefile = d.expand("${SSTATE_DIR}/" + getsstatefile(tid, > siginfo, d)) > > if os.path.exists(sstatefile): > - oe.utils.touch(sstatefile) > + sstate_touch_atime(sstatefile) > for ext in ['.sig', '.siginfo']: > if os.path.exists(sstatefile + ext): > - oe.utils.touch(sstatefile + ext) > + sstate_touch_atime(sstatefile + ext) > found.add(tid) > bb.debug(2, "SState: Found valid sstate file %s" % > sstatefile) > else: > @@ -1223,7 +1232,7 @@ python sstate_eventhandler() { > if not os.path.exists(siginfo): > bb.siggen.dump_this_task(siginfo, d) > else: > - oe.utils.touch(siginfo) > + sstate_touch_atime(siginfo) > } > > SSTATE_PRUNE_OBSOLETEWORKDIR ?= "1"
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#245934): https://lists.openembedded.org/g/openembedded-core/message/245934 Mute This Topic: https://lists.openembedded.org/mt/121194864/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
