On 10/10/2025 11.50, Daniel P. Berrangé wrote:
On Fri, Oct 10, 2025 at 11:32:43AM +0200, Thomas Huth wrote:
From: Thomas Huth <[email protected]>
The download cache of the functional tests is currently only growing.
But sometimes tests get removed or changed to use different assets,
thus we should clean up the stale old assets after a while when they
are not in use anymore. So add a script that looks at the time stamps
of the assets and removes them if they haven't been touched for more
than half of a year. Since there might also be some assets around that
have been added to the cache before we added the time stamp files,
assume a default time stamp that is close to the creation date of this
patch, so that we don't delete these files too early.
Signed-off-by: Thomas Huth <[email protected]>
---
MAINTAINERS | 1 +
scripts/clean_functional_cache.py | 47 +++++++++++++++++++++++++++++++
tests/Makefile.include | 1 +
3 files changed, 49 insertions(+)
create mode 100755 scripts/clean_functional_cache.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 84cfd85e1fa..4c468d45337 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4398,6 +4398,7 @@ M: Thomas Huth <[email protected]>
R: Philippe Mathieu-Daudé <[email protected]>
R: Daniel P. Berrange <[email protected]>
F: docs/devel/testing/functional.rst
+F: scripts/clean_functional_cache.py
F: tests/functional/qemu_test/
Windows Hosted Continuous Integration
diff --git a/scripts/clean_functional_cache.py
b/scripts/clean_functional_cache.py
new file mode 100755
index 00000000000..e5c4d1acaf3
--- /dev/null
+++ b/scripts/clean_functional_cache.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+"""Delete stale assets from the download cache of the functional tests"""
+
+import os
+import stat
+import sys
+import time
+from pathlib import Path
+
+
+cache_dir_env = os.getenv('QEMU_TEST_CACHE_DIR')
+if cache_dir_env:
+ cache_dir = Path(cache_dir_env, "download")
+else:
+ cache_dir = Path(Path("~").expanduser(), ".cache", "qemu", "download")
This creates a Path object but then doesn't take advantage of
any of its functionality, calling os. functions still....
Ok, you got me, looks like I'm still a python ignorant after one year of
hacking the functional testing framework ;-) Thanks for the hints how to do
it better!
+ try:
+ with open(filename + ".stamp", "r", encoding='utf-8') as fh:
+ timestamp = int(fh.read())
timestamp = file.read_text()
Hmm, but "file" points to the asset, not to the .stamp file, doesn't it?
+ except FileNotFoundError:
+ # Assume it's an old file that was already in the cache before we
+ # added the code for evicting stale assets. Use the release date
+ # of QEMU v10.1 as a default timestamp.
+ timestamp = time.mktime((2025, 8, 26, 0, 0, 0, 0, 0, 0))
The prev patch will make the precache task create the .stamp for all
files that are currently in use by the current branch. So the only
thing this does is to prevent us deleting cached files that might
still be needed by a different branch. There will be few of them,
so if we prematurely delete a handful that's not a big deal. If we
switch to checking mtime, this except won't even exist.
When hunting regressions that have been introduced recently, I often have to
do bisecting on revisions from the previous 1 or 2 QEMU releases, so I'd
prefer keeping the assets of the last few months, even if they have been
removed from the master branch in a very recent commit.
Thomas
