(Cc-ing Adam for DSA following the ping about disk space earlier.)
Cyril Brulebois (2023-06-13):
> Spotted while working on #1037479: we have a bunch of files around that
> are no longer useful, because relevant suites are EOL and archived, and
> have been dropped from the relevant config files, templates, etc.
>
> They could probably be cleaned up, but I don't want to rush this kind of
> things, hence this bug report.
Apparently we're going to have to do something soon, since the recent
updates in picconi have led to a noticeable and worrisome bump in
storage used:
/dev/mapper/vg0-srv 246G 234G 822M 100% /srv
Some cleanup happened in user directories, but I suppose it would make
sense to keep track of our disk space usage:
/dev/mapper/vg0-srv 246G 136G 99G58% /srv
And beware, there are peaks during runs, with e.g. tmp/ filing up.
Also, the mirror (pkgmirror-csail) should like this as well given:
/dev/vdb 148G 129G 12G92% /srv
See also graphs (with the usual guest authentication):
https://munin.debian.org/debian.org/picconi.debian.org/df.html
https://munin.debian.org/debian.org/pkgmirror-csail.debian.org/df.html
Our directory looked like this when I started:
426M./archive
200K./bin
268K./cache
8.0K./cgi-bin
29M ./conf
64K ./cron.d
8.0K./debian
116G./files
16M ./.git
133M./home
300K./lib
14M ./mail
8.4G./mirror
4.0K./po
196K./static
116K./templates
7.5G./tmp
2.6G./www
---
135Gtotal
I was thinking of getting rid of these:
$ du -sch $(find -name 'jessie*')
65M ./mirror/202306141247/www/jessie-backports
17M ./mirror/202306141247/www/source/jessie-backports
87M ./mirror/202306141247/www/source/jessie
3.5M./mirror/202306141247/www/source/jessie-updates
5.3M./mirror/202306141247/www/source/jessie-backports-sloppy
505M./mirror/202306141247/www/jessie
12M ./mirror/202306141247/www/jessie-updates
6.4M./mirror/202306141247/www/jessie-backports-sloppy
196K./archive/backports/jessie-backports
26M ./www/jessie-backports
5.8M./www/source/jessie-backports
11M ./www/source/jessie
2.3M./www/source/jessie-updates
3.4M./www/source/jessie-backports-sloppy
176M./www/jessie
3.1M./www/jessie-updates
4.0M./www/jessie-backports-sloppy
930Mtotal
and these:
$ du -sch $(find -name 'stretch*')
85M ./mirror/202306141247/www/source/stretch
5.1M./mirror/202306141247/www/source/stretch-backports-sloppy
15M ./mirror/202306141247/www/source/stretch-backports
3.4M./mirror/202306141247/www/source/stretch-updates
416M./mirror/202306141247/www/stretch
6.2M./mirror/202306141247/www/stretch-backports-sloppy
52M ./mirror/202306141247/www/stretch-backports
4.9M./mirror/202306141247/www/stretch-updates
11M ./www/source/stretch
3.4M./www/source/stretch-backports-sloppy
5.8M./www/source/stretch-backports
2.2M./www/source/stretch-updates
179M./www/stretch
3.9M./www/stretch-backports-sloppy
22M ./www/stretch-backports
2.6M./www/stretch-updates
-
813Mtotal
But those easy ones would only account for 1.7G combined, as mentioned
yesterday.
Searching for files containing but not starting with either jessie or
stretch, there are a lot of other things that looked like they could go
away:
- filelists_*
- filenames_*
- new_package_info_*
- package_names_*
- packages_all_*
- reverse_*
- source_names_*
- sources_all_*
and I expected no collateral damages.
To be on the safe side, I created two tarballs, one for jessie, one for
stretch, containing all those files, before removing them. I've compressed
them so that they take less space than actual files, moved them in a
directory that shouldn't be mirrored to pkgmirror-csail, and I've
scheduled their deletions in 6 months (via at), which should leave plenty
of time to restore them if needed, while making sure full cleanup happens
eventually.
pkg_user@picconi:/srv/packages.debian.org$ echo 'rm -rv
/srv/packages.debian.org/obsolete/' | at 'now + 6 months'
warning: commands will be executed using /bin/sh
job 5 at Thu Dec 14 18:56:00 2023
pkg_user@picconi:/srv/packages.debian.org$ atq
5 Thu Dec 14 18:56:00 2023 a pkg_user
Also removed, a long obsolete file:
-rw--- 1 pkg_user pkg_maint 7.5G May 2 2021 tmp/sort01fAge
Not sure why this file stays behind, but at least the name is stable, so
the non-freed space is getting reclaimed when the next run starts:
-rw-r--r-- 1 pkg_user pkg_maint 14794575872 Jun 14 17:37 tmp/sid.so
