Re: Selective rotation of journald logs

2024-02-26 Thread Nicolas George
Max Nikulin (12024-02-23):
> I am realizing that the following is not an answer to the asked question.
> The thread is no more than useless arguing anyway.
> 
> Some ideas that might be useful in close cases:
> - a bunch of filtering options and --output=export as a part of log rotation
> to have selective copy of the journal in an alternative directory

Hi.

You know what? I think it is exactly the right answer.

I think the principle for this thing is: journald maintains the current
logs in a best-effort manner, but when it comes to archival, the needs
and manners are too varied, it does not try to provide a ready-to-use
solution, but journalctl provides all the tools necessary to implement
what we want.

If I am right, if it is indeed the way it is meant to be used, I just
wish somebody would have bothered writing it in so many words at the
start of a documentation.

Thanks.

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-24 Thread Byunghee HWANG
Max Nikulin  writes:

> On 24/02/2024 21:09, Byunghee HWANG wrote:
>>> When I read the question I decided that syslog for auth facility is a
>>> kind of solution.
>> Really i would like to learn more. Can you show a more specific
>> example?
>> Also i'm using Debian (sid).
>
> Install rsyslog and logrotate and read their configuration
> files. /var/log/auth is configured by default. Journal contains more
> metadata than syslog text files however.

Thank you Max ^^^


Sincerely, Byunghee from South Korea

-- 
^고맙습니다 _布德天下_ 감사합니다_^))//


signature.asc
Description: PGP signature


Re: Selective rotation of journald logs

2024-02-24 Thread Max Nikulin

On 24/02/2024 21:09, Byunghee HWANG wrote:

When I read the question I decided that syslog for auth facility is a
kind of solution.


Really i would like to learn more. Can you show a more specific example?
Also i'm using Debian (sid).


Install rsyslog and logrotate and read their configuration files. 
/var/log/auth is configured by default. Journal contains more metadata 
than syslog text files however.





Re: Selective rotation of journald logs

2024-02-24 Thread Byunghee HWANG
Hellow Max!

Max Nikulin  writes:

> On 23/02/2024 17:15, Nicolas George wrote:
>> How do I tell systemd's logging system to keep authentication logs
>> for
>> one year and mail logs for one month?
>
> (...)
>
> P.S.
> When I read the question I decided that syslog for auth facility is a
> kind of solution.

Really i would like to learn more. Can you show a more specific example?
Also i'm using Debian (sid).


Thanks, Byunghee from South Korea

-- 
^고맙습니다 _布德天下_ 감사합니다_^))//


signature.asc
Description: PGP signature


Re: Journald's qualities (was: Selective rotation of journald logs)

2024-02-24 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 15:05:52
Nicholas Geovanis  napisał(a):

> On Fri, Feb 23, 2024, 2:57 PM Dan Ritter  wrote:
> 
> > Stefan Monnier wrote:  
> > > Makes one wonder why they don't use naive append-only "plain
> > > text" logs (tho with appropriate delimiters (maybe some kind of
> > > CSV) to make searches more reliable than with old-style plain
> > > text logs)?
> > >
> > > What are the advantages of journald's representation?
> > > I mean, to justify the slow search and large disk space usage,
> > > there is presumably some upside for some use cases.  I can see
> > > some weak argument against Sqlite based on the size of Sqlite,
> > > but what are the advantages of journald's representation compared
> > > to a naive one?  
> >
> >
> > systemd's design philosophy, observed from the outside, goes
> > like this:
> >  
> 
> bunch trimmed.
> 
> Exactly correct in my view. Systemd's use-case is the desktop, not the
> server in the datacenter. They will be using log-aggregation software
> in the datacenter anyway so no use for systemd logging. We don't
> install desktop software on servers either, no X Windows, no gnome,
> etc. Network connections are stable, no roaming :-)
> 
> Long-term logs are for servers, so systemd doesn't want them.

Right but it would be nice if it could at least forward them upstream
then! Your choices are

* use rsyslog, which does everything better than journald, including
  writing to many different databases directly, just to forward to
  remote host.
* setup super special listener for super special journald remote log
  sending method that nothing but journald supports

If it just supported standard, common protocols like RFC5424 flavour
(that adds structured data to entries), or hell, just "send a JSONed
log to that address", I could just have a bunch of my "small" devices
run journald-only and send it to central logging server. But no dice,
even my router needs to run extra daemon just to forward logs.

Actually, having pluggable "backends" would solve a lot of that, dumb
little IoT device could just have "a text file" or remote output
implemented as write method and "grep the text file" as query method.
Or DBI plugin that could write to SQLite and query it back. Or just
remote log plugin that returned "sorry, your logs are in another
castle" on query. But that's just building rsyslog but worse at that
point...

I wish I disliked C less than I miss those features...
-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Journald's qualities (was: Selective rotation of journald logs)

2024-02-24 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 12:34:34
Dan Ritter  napisał(a):

> Stefan Monnier wrote: 
> > Makes one wonder why they don't use naive append-only "plain text"
> > logs (tho with appropriate delimiters (maybe some kind of CSV) to
> > make searches more reliable than with old-style plain text logs)?
> > 
> > What are the advantages of journald's representation?
> > I mean, to justify the slow search and large disk space usage,
> > there is presumably some upside for some use cases.  I can see some
> > weak argument against Sqlite based on the size of Sqlite, but what
> > are the advantages of journald's representation compared to a naive
> > one?  
> 
> 
> systemd's design philosophy, observed from the outside, goes
> like this:
> 
> - assume that the machine is a laptop or mobile device that is
> always changing: moves from network to network, plugs and
> unplugs devices, goes to sleep and is woken up

I had thought similar thoughts many times when using it. But some of
the stuff that derived from that is pretty useful; for example we had a
service tht downloaded encryption keys from central server to unlock
the hard drive and sysV version was a bit of a mess (if service didn't
ran on boot there was no unlocked volume to mount), while in systemd
the dependencies allow it to automatically bring it up when mount is
requested, and the mount will also be automatically done if service
using it is started (if it has correct RequiresMountsFor entries). So
some actually useful stuff for more "constant config" machines had
flown from that

> - disk space is limited, but cpu time is free

In which way ? It doesn't care about wasting space, but it does assume
you have blazing fast storage or else journal related ops are sluggish
(and waste disk cache as it just reads hundreds of MBs to find last few
lines of logfile)

> - the network knows better than local config

Offtopic but since Debian switched to systemd for DNS management on
VPNs and suc I need to restart it sometimes multiple times to just get
"right" DNS servers, because there appears to be no notion of priority:

https://github.com/systemd/systemd/issues/27543

so now any time I connect to work (just openvpn tunnel, nothing fancy)
I need to spam

systemclt restart systemd-resolved ; sleep 1 ; cat /etc/resolv.conf

few times till the dice rolls the right order of DNS servers...

XANi



-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Journald's qualities (was: Selective rotation of journald logs)

2024-02-23 Thread Nicholas Geovanis
On Fri, Feb 23, 2024, 2:57 PM Dan Ritter  wrote:

> Stefan Monnier wrote:
> > Makes one wonder why they don't use naive append-only "plain text" logs
> > (tho with appropriate delimiters (maybe some kind of CSV) to make
> > searches more reliable than with old-style plain text logs)?
> >
> > What are the advantages of journald's representation?
> > I mean, to justify the slow search and large disk space usage, there is
> > presumably some upside for some use cases.  I can see some weak argument
> > against Sqlite based on the size of Sqlite, but what are the advantages
> > of journald's representation compared to a naive one?
>
>
> systemd's design philosophy, observed from the outside, goes
> like this:
>

bunch trimmed.

Exactly correct in my view. Systemd's use-case is the desktop, not the
server in the datacenter. They will be using log-aggregation software in
the datacenter anyway so no use for systemd logging. We don't install
desktop software on servers either, no X Windows, no gnome, etc. Network
connections are stable, no roaming :-)

Long-term logs are for servers, so systemd doesn't want them.
> systemd thinks logs are for finding out what just happened
> recently. If you wanted long-term logs, obviously you would
> configure a central repository on some other machine and ship
> them across the network.
>
>
> I have nothing but praise for the Debian maintainers of rsyslog,
> who have arranged it so that installing rsyslog immediately does
> appropriate things to pull data out of systemd.
>
> -dsr-
>
>


Re: Journald's qualities (was: Selective rotation of journald logs)

2024-02-23 Thread Dan Ritter
Stefan Monnier wrote: 
> Makes one wonder why they don't use naive append-only "plain text" logs
> (tho with appropriate delimiters (maybe some kind of CSV) to make
> searches more reliable than with old-style plain text logs)?
> 
> What are the advantages of journald's representation?
> I mean, to justify the slow search and large disk space usage, there is
> presumably some upside for some use cases.  I can see some weak argument
> against Sqlite based on the size of Sqlite, but what are the advantages
> of journald's representation compared to a naive one?


systemd's design philosophy, observed from the outside, goes
like this:

- assume that the machine is a laptop or mobile device that is
always changing: moves from network to network, plugs and
unplugs devices, goes to sleep and is woken up

- disk space is limited, but cpu time is free

- the network knows better than local config

- all services serve the local user's desktop, primarily

- the local user doesn't know anything but can call on a remote
  sysadmin

- systemd is the best at doing anything, so it should do
everything.

If all of these assumptions match up with your particular use
case, systemd is happy.


Long-term logs are for servers, so systemd doesn't want them.
systemd thinks logs are for finding out what just happened
recently. If you wanted long-term logs, obviously you would
configure a central repository on some other machine and ship
them across the network.


I have nothing but praise for the Debian maintainers of rsyslog,
who have arranged it so that installing rsyslog immediately does
appropriate things to pull data out of systemd.

-dsr-



Re: Journald's qualities (was: Selective rotation of journald logs)

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 10:26:49
Stefan Monnier  napisał(a):

> > [13:37:48]cthulhu:/var/log/journal☠ journalctl |dd of=/dev/zero
> > bs=1M 0+15442 records in
> > 0+15442 records out
> > 63643115 bytes (64 MB, 61 MiB) copied, 5,47791 s, 11,6 MB/s
> > du -h /var/log/journal/
> > 337M/var/log/journal/44cf6f547971fc33309d1e9e02e7
> > 337M/var/log/journal/
> >
> > (I've raised a bug 8 years ago about it
> >  https://github.com/systemd/systemd/issues/2460 )  
> 
> Oh, that bug report is quite interesting, thanks.
> Makes one wonder why they don't use naive append-only "plain text"
> logs (tho with appropriate delimiters (maybe some kind of CSV) to make
> searches more reliable than with old-style plain text logs)?
> 
> What are the advantages of journald's representation?
> I mean, to justify the slow search and large disk space usage, there
> is presumably some upside for some use cases.  

The advantage of binary format is that you have separate fields for
program, level etc and it is easier to filter by exactly what you want. 
So "systemctl status servicename" just asks log subsystem for exactly
what it needs.

But journald logs aren't build for querying speed, there is for example
no index (in memory or on disk) that tells you where last log entries
from apps are so you get pathological cases like in ticket.

The problem is that you kinda do 2 types of querying on logs most often:

* get me all/most of what happened in last x lines/hours
* get me everything recent (or just everything period) that this
  particular service returned.

Which kinda exclude the "simple" tricks" like "just hash over service
name so at worst you have to search only 1/64th of files" as they would
make the "get me everything" case slower. But maybe not by all that
much.

The size issue... it's appendable binary format which means that any
metadata systemd adds can't be deduplicated by just having join on
table with service info, it needs to be repeated every single line and
I'm guessing that's where the bloat comes from.

I rambled about SQLite as it is kinda closest comparison for embedded
binary format that can be searched and I think that being able to run
SQL queries (sqlite now even have full text search) against your local
logs with no database daemons running would be killer feature on some of
the smaller devices.


> argument against Sqlite based on the size of Sqlite, 

I'd argue that's not really technically relevant as in most cases you'd
have other app in system using sqlite for *something* and will have
installed the library anyway. And space savings gets wasted just on how
verbose journald format is.

> but what are the
> advantages of journald's representation compared to a naive one?

in short: querability without text parsing. That's about it.

Which *is* a great feature, do not get me wrong, I use it all the time,
but the slowdown caused by its unfortunate implementation detail is
also showing up when I use it pretty often as the most common use case
is "something broke, log in on server that has been running on
automation for last 3 months and see what is happening"

It does have some features around FSS (signing logs) so for folks that
need it there isn't many alternatives as it would be hard to make
similar feature on database.

But on flipside inability to separate log into log rotation groups is
*nasty*. You basically can't have verbose service in the system as that
will rotate away any old logs of other services away. And looking for
those logs is also the worst case:

[17:57:56]cthulhu:~☠ echo 3 > /proc/sys/vm/drop_caches
[17:58:14]cthulhu:~☠ time systemctl status nginx |grep Notice
Notice: journal has been rotated since unit was started, output may be 
incomplete.

real0m1,645s
user0m0,000s
sys 0m0,094s
[17:58:19]cthulhu:~☠ echo 3 > /proc/sys/vm/drop_caches
[17:58:28]cthulhu:~☠ time sqlite /tmp/log_indexed.sqlite 'select * from log 
where source = "nginx";'  >/dev/null

real0m0,374s
user0m0,000s
sys 0m0,009s


I guess another one being appendable log is that in case of crash it's easier 
to recover than embedded database...

...or so would I say if I didn't saw journald throwing away binary logs with 
some errors after crash.


> 
> 
> Stefan
> 
> 
> PS: FWIW, the only time I thought about the design of a log format,
> I was thinking along the lines of reducing redundancy, i.e. keeping
> a kind of "suffix tree" of past messages so as to replace the many
> repeated chunks of text with references to past log entries.
> Then I realized that I could get the same result much more simply by
> gzipping the files, which would naturally be done as part of
> log rotation 
> 

One-service-per-file approach is honestly good enough for most stuff.
PITA to get chronological order thought, every approach really have
some drawbacks and benefits.

But honestly ? Send it to central logger and worry there if you have a
chance, especially if you do some parsing there can 

Re: Selective rotation of journald logs

2024-02-23 Thread Max Nikulin

On 23/02/2024 17:15, Nicolas George wrote:


How do I tell systemd's logging system to keep authentication logs for
one year and mail logs for one month?


I am realizing that the following is not an answer to the asked 
question. The thread is no more than useless arguing anyway.


Some ideas that might be useful in close cases:
- a bunch of filtering options and --output=export as a part of log 
rotation to have selective copy of the journal in an alternative directory

- systemd-journald.service(8) § JOURNAL NAMESPACES for some services.


P.S.
When I read the question I decided that syslog for auth facility is a 
kind of solution.




Journald's qualities (was: Selective rotation of journald logs)

2024-02-23 Thread Stefan Monnier
> [13:37:48]cthulhu:/var/log/journal☠ journalctl |dd of=/dev/zero bs=1M
> 0+15442 records in
> 0+15442 records out
> 63643115 bytes (64 MB, 61 MiB) copied, 5,47791 s, 11,6 MB/s
> du -h /var/log/journal/
> 337M  /var/log/journal/44cf6f547971fc33309d1e9e02e7
> 337M  /var/log/journal/
>
> (I've raised a bug 8 years ago about it
>  https://github.com/systemd/systemd/issues/2460 )

Oh, that bug report is quite interesting, thanks.
Makes one wonder why they don't use naive append-only "plain text" logs
(tho with appropriate delimiters (maybe some kind of CSV) to make
searches more reliable than with old-style plain text logs)?

What are the advantages of journald's representation?
I mean, to justify the slow search and large disk space usage, there is
presumably some upside for some use cases.  I can see some weak argument
against Sqlite based on the size of Sqlite, but what are the advantages
of journald's representation compared to a naive one?


Stefan


PS: FWIW, the only time I thought about the design of a log format,
I was thinking along the lines of reducing redundancy, i.e. keeping
a kind of "suffix tree" of past messages so as to replace the many
repeated chunks of text with references to past log entries.
Then I realized that I could get the same result much more simply by
gzipping the files, which would naturally be done as part of
log rotation 



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 14:23:07
Nicolas George  napisał(a):

> Mariusz Gronczewski (12024-02-23):
> > Like, really what kind of person gets angry when they get too much
> > details in instruction?  
> 
> What kind of person writes pages of angry mail when the details are
> not liked?
> 

That would be you, the thing like "conversation" and "having common
courtesy to answer questions instead of ignoring ones you don't like"
seems like foreign concept to you too. Now please just be quiet as this
is just spam at that point and the question has already been answered.

-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Mariusz Gronczewski (12024-02-23):
> Like, really what kind of person gets angry when they get too much
> details in instruction?

What kind of person writes pages of angry mail when the details are not
liked?

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 14:09:45
Nicolas George  napisał(a):

> Greg Wooledge (12024-02-23):
> > Have you even *read* this mailing list?  Most of the people who ask
> > for help here lack experience that you might consider "baby
> > sysadmin" level, and would greatly appreciate the explanations.  
> 
> It is usually quite easy to tell the difference by the phrasing and
> accuracy of the question.
> 

It does. It looked like person that can't comprehend the manual they
have been given nor use Google



-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 14:03:50
Nicolas George  napisał(a):

> Greg Wooledge (12024-02-23):
> > What was "blind" about his anaylsis?  It looked pretty well thought
> > out to me.  He showed actual examples of how space-inefficient it
> > is, and provided a theoretical example of how one misbehaved
> > service could flush out the important logs of well-behaved
> > services.  
> 
> The selective blindness here is to look only at the bad things. The
> drawbacks mentioned exist, but they are in part there for reasons, in
> order to fix the issues of the previous solutions.

I wrote about the stuff you explicitly asked, saving the logs on disk,
because you apparently can't stand a conversation about alternatives.

First you bitch I wrote about "the old stuff you know" now you bitch
about not going thru entire manpage of journalctl features. Decide

> 
> An analysis that mentions only the bad things and conclude to
> recommend using the previous solutions without even discussing their
> own drawbacks is not worth our time.
> 

The selective blindness is your problem, not "us". You decided "old
stuff" didn't work for you and got into shouting match when I described
you in excessive detail why what you want to do is impossible under
systemd.

And you got angry that someone described you why you can't do that.

> > If anything, the person who's blindly following a path is *you*.
> > You're looking to do something that multiple people have said is
> > not possible,  
> 
> Multiple?
> 
> > and when they offer you an alternative, your claws come out.  
> 
> When somebody spends one line answering the question and then pages
> “offering an alternative” by explaining things a baby sysadmin would
> already know, I deduce they are not much above the level of baby
> sysadmin themselves, and it cancels any trust I could have put in the
> one-line answer.
> 

I assumed that you are a beginner because if you read a manpage of
journald/ctl you would know what you are asking is not possible. And
the fact you haven't mentioned any other "traditional" way of doing it
made it look like that too, a newbie linux user that didn't knew it
existed.

So I wrote it with detail needed for beginner to do the job.

I would say I am sorry for mistaking your "skill" level but you already
proven abilty to read the documentation with understand is above you,
so I am not.

Like, really what kind of person gets angry when they get too much
details in instruction?


-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Greg Wooledge (12024-02-23):
> Have you even *read* this mailing list?  Most of the people who ask
> for help here lack experience that you might consider "baby sysadmin"
> level, and would greatly appreciate the explanations.

It is usually quite easy to tell the difference by the phrasing and
accuracy of the question.

Regards,

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 13:48:35
Nicolas George  napisał(a):

> Mariusz Gronczewski (12024-02-23):
> > So to say it short: It is horrid.  
> 
> Generic bashing of systemd in favor of a blind cult of the good old
> ways are not what I am looking for either, and the unbalanced tone of
> your reply makes it look like precisely that.
> 

What was "blind" about it? I had now years of experience on few
hundreds systems dealing with it from basically since it was included
in Debian.

I assume you're one of those ignorants that assume any critique of 
product means that I somehow "hate" it  and everything around it without 
reason and that is frankly disgusting.

Systemd saved us thousands of lines of code thanks to its many useful
features and cut down whole swathes of bugs commonly made in "simple"
SysV systems and I'd recommend it on any alternative in a flash.

But journalctl is just not one of them and the way it
stores the data looks like an amateur really wanted towanted to write a
file format/database/querying system and failed at all of that, instead
of slapping SQLite (or any other existing established embedded
database) on it and calling it a day. Hell "a bunch of texfiles
prefixed with program name would've been an improvement. That's even
before we got into whole log duration management that I already
mentioned.

I *want* it to be good, system like that that is also efficient and
easy to query would be a dream for embedded system but it. is. just.
not. that. At the moment.

-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Selective rotation of journald logs

2024-02-23 Thread Greg Wooledge
On Fri, Feb 23, 2024 at 02:03:50PM +0100, Nicolas George wrote:
> When somebody spends one line answering the question and then pages
> “offering an alternative” by explaining things a baby sysadmin would
> already know, I deduce they are not much above the level of baby
> sysadmin themselves, and it cancels any trust I could have put in the
> one-line answer.

Have you even *read* this mailing list?  Most of the people who ask
for help here lack experience that you might consider "baby sysadmin"
level, and would greatly appreciate the explanations.



Re: Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Greg Wooledge (12024-02-23):
> What was "blind" about his anaylsis?  It looked pretty well thought out
> to me.  He showed actual examples of how space-inefficient it is, and
> provided a theoretical example of how one misbehaved service could
> flush out the important logs of well-behaved services.

The selective blindness here is to look only at the bad things. The
drawbacks mentioned exist, but they are in part there for reasons, in
order to fix the issues of the previous solutions.

An analysis that mentions only the bad things and conclude to recommend
using the previous solutions without even discussing their own drawbacks
is not worth our time.

> If anything, the person who's blindly following a path is *you*.  You're
> looking to do something that multiple people have said is not possible,

Multiple?

> and when they offer you an alternative, your claws come out.

When somebody spends one line answering the question and then pages
“offering an alternative” by explaining things a baby sysadmin would
already know, I deduce they are not much above the level of baby
sysadmin themselves, and it cancels any trust I could have put in the
one-line answer.

Regards,

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-23 Thread Greg Wooledge
On Fri, Feb 23, 2024 at 01:48:35PM +0100, Nicolas George wrote:
> Mariusz Gronczewski (12024-02-23):
> > So to say it short: It is horrid.
> 
> Generic bashing of systemd in favor of a blind cult of the good old ways
> are not what I am looking for either, and the unbalanced tone of your
> reply makes it look like precisely that.

What was "blind" about his anaylsis?  It looked pretty well thought out
to me.  He showed actual examples of how space-inefficient it is, and
provided a theoretical example of how one misbehaved service could
flush out the important logs of well-behaved services.

If anything, the person who's blindly following a path is *you*.  You're
looking to do something that multiple people have said is not possible,
and when they offer you an alternative, your claws come out.



Re: Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Mariusz Gronczewski (12024-02-23):
> So to say it short: It is horrid.

Generic bashing of systemd in favor of a blind cult of the good old ways
are not what I am looking for either, and the unbalanced tone of your
reply makes it look like precisely that.

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 13:02:00
Nicolas George  napisał(a):

> Mariusz Gronczewski (12024-02-23):
> > That is not a feature systemd's logging have.  
> 
> That is what it seems, but I would like second opinions.
> 
> > You'd have to make a
> > rsyslogd rule to put it in one directory  
> 
> Thanks, but my question was about systemd's infrastructure. Answers
> about the old systlog/logrotate infrastructure are a waste of time
> since I already know how they work and they are amply documented
> elsewhere.
> 

So to say it short: It is horrid. There is no any sensible indexing or
split between services or anything. Single spamming service can easily
make it rotate out more important but much more rare log entries and
there is nothing you can do about it.

It is also woefully inefficient, with each "line" of log taking few
times more than it does in text or even if thrown into SQLite database.

For example:

[13:37:01]cthulhu:/var/log/journal☠ ls -s -h /tmp/log.sqlite
79M /tmp/log.sqlite
[13:37:12]cthulhu:/var/log/journal☠ sqlite /tmp/log.sqlite 'select count(*) 
from log'
386351
journalctl |wc -l
381770
[13:37:48]cthulhu:/var/log/journal☠ journalctl |dd of=/dev/zero bs=1M
0+15442 records in
0+15442 records out
63643115 bytes (64 MB, 61 MiB) copied, 5,47791 s, 11,6 MB/s
du -h /var/log/journal/
337M/var/log/journal/44cf6f547971fc33309d1e9e02e7
337M/var/log/journal/

(I've raised a bug 8 years ago about it 
https://github.com/systemd/systemd/issues/2460 )

To summarize: the ~61MB of resulting text of logs takes around 337MB 
on disk when saved by journald, ~80MB in sqlite (~103MB when I added 
index for time and process). Use the old ways you know, it's a waste 
of time to try to wrangle journald to do what you want.


-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Re: Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Mariusz Gronczewski (12024-02-23):
> That is not a feature systemd's logging have.

That is what it seems, but I would like second opinions.

>   You'd have to make a
> rsyslogd rule to put it in one directory

Thanks, but my question was about systemd's infrastructure. Answers
about the old systlog/logrotate infrastructure are a waste of time since
I already know how they work and they are amply documented elsewhere.

-- 
  Nicolas George



Re: Selective rotation of journald logs

2024-02-23 Thread Mariusz Gronczewski
Dnia 2024-02-23, o godz. 11:15:29
Nicolas George  napisał(a):

> Hi.
> 
> It might be an obvious question, but I do not manage to find the
> obvious answer:
> 
> How do I tell systemd's logging system to keep authentication logs for
> one year and mail logs for one month?
> 
> Regards,
> 

That is not a feature systemd's logging have. You'd have to make a
rsyslogd rule to put it in one directory, another one for the other use
then tweak logrotate rules to rotate and keep each of them for
different length.

Most packages already log to separate files and have separate logrotate
rules so often it is just changing a single line, for example auth.log
is rotated with rest of the default logs:

/var/log/syslog
/var/log/mail.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/cron.log
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}

so if you wanted to change only auth.log you'd copy the section, and remove it 
from current one, like this:


/var/log/auth.log
{
rotate 53 # a year in weeks
weekly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}


-- 
Mariusz Gronczewski (XANi) 
GnuPG: 0xEA8ACE64
https://devrandom.eu



Selective rotation of journald logs

2024-02-23 Thread Nicolas George
Hi.

It might be an obvious question, but I do not manage to find the obvious
answer:

How do I tell systemd's logging system to keep authentication logs for
one year and mail logs for one month?

Regards,

-- 
  Nicolas George