Heads up that I'm trying to get https://github.com/systemd/systemd/pull/22998 
in before the next systemd release which should reduce the journal size by +- 
50% in a way that will be taken into account by journald's retention logic 
(unlike the btrfs compression).

Also, as soon as there's a kernel API to query compressed file size I'll update 
journald's retention logic to use that so we can take the actual file size into 
account when making retention decisions.

Cheers,

Daan De Meyer

________________________________________
From: Chris Murphy <li...@colorremedies.com>
Sent: 27 September 2022 17:12
To: fedora devel
Subject: limiting the (systemd) journal size

!-------------------------------------------------------------------|
  This Message Is From an External Sender

|-------------------------------------------------------------------!

Hi,

Fedora uses systemd-journald for system logging. By default it is a persistent 
log kept on /var, and uses up to 4G disk space, although in certain 
circumstances it can go a bit higher. See 'man journald.conf' for details.

Example:
>Sep 27 07:26:05 fovo.local systemd-journald[602]: System Journal 
>(/var/log/journal/$machine_id) is 385.9M, max 4.0G, 3.6G free.

In this example Fedora 37 Workstation system, logging is happening since August 
20, is about 10M/day of journal accumulation, or 1.12 years of journals before 
garbage collection begins.

Exactly what will trigger garbage collection depends on the system. There are 
quite a few knobs for adjusting various aspects of retention and how granular 
the garbage collection will be. e.g. it's common to see 64M system journal 
files that contain weeks of entries. It's also possible to limit the journal 
file size, thus improving granularity whether to retain a bit more or less than 
the ideal amount.

Some folks use services with verbose or debug logging. 4G might only be a few 
months of logs in such a case. Whereas other folks have a small root device in 
which even the smaller of 10% or 4G can be quite a lot and in certain cases is 
not a hard limit.

Also note that on Btrfs with compression enabled, the stored amount is quite a 
bit less. Like all of user space, systemd-journald sees the uncompressed file 
sizes, so its retention behavior hasn't changed as a result of btrfs 
compression. What has changed is we're only (physically) storing about 1/3 of 
whatever the max retention is on a given system.

The obvious bike-shedding questions are:
Is 4G is too much or too little? If so what amount it should be? Is size still 
the correct approach? Or should we consider a max retention time? And if so, 
what would it be and how granular should it be?

Also, what's the scope? Is a change needed Fedora-wide, in a manner that's 
upstreamable? That could prove difficult because any change will negatively 
impact other use cases, not least of which is what the upgrade behavior should 
be if it'll involve trimming journals. Are the current defaults optimal for 
most use cases most of the time? There will be a higher burden of persuasion to 
get a Fedora-wide change, rather than optimizing for just desktops.

But that isn't intended to limit the discussion to just the desktop case. Just 
to be aware that the broader and grander the change, the more consideration of 
the consequences there needs to be, i.e. less bike shedding.

More background and discussion upstream and Workstation working group issues. 
[1]



[1]
https://pagure.io/fedora-workstation/issue/213
https://github.com/systemd/systemd/issues/17382

--
Chris Murphy
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Reply via email to