Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-13 Thread Lennart Poettering
On Thu, 07.08.14 14:06, Frantisek Hanzlik (fra...@hanzlici.cz) wrote:

 Hi, I just install Fedora 20 (with systemd 208) and want log,
 if possible, without journald, only to rsyslog.

Why if I may ask?

Trying to do that is usually more a sign of misconceptions about systemd
or the journal than really technically valid.

journald is not an optional component of systemd, among other things it
is responsible for collecting logs during early boot/initrd and from
stdout/stderr from all running services, regardless when they
run. It then forwards those logs to disk (optional, but on by default),
to another syslog daemon (optional, but on by default), kmsg, the
console (both off by default) or wall (on for high-importance messages).

syslog OTOH only runs in late boot, and thus traditionally got a much
more limited view on what is happening on the systemd. By running syslog
and journald in tandem you will actually get substantially more data
into syslog than you got before: all the stuff from stdout/stderr plus
all the early-boot stuff. The journal is hence a good thing, even if you
do not intend to ever query it directly.

What you can do is turn off journald's local storage. Use the Storage=
setting in journald.conf for that.

 I have this configuration:
 
 'systemd.log_target=syslog-or-kmsg' at kernel command line

This doesn't do what you think it does. Regardless if you use
syslog-or-kmsg or journal-or-kmsg, in both cases PID 1 sends it data
to journald, just the transport is different, and if you use syslog you
simply lose a lot of context such as line numebrs and sources files log
messages are generated from. 

I have now removed syslog-or-kmsg from the documentation, to make this
less confusing. Also, internally PID 1 will now change syslog-or-kmsg
to journal-or-kmsg, thus making them entirely identical.

Also note that this setting only influences PID 1, but systemd has a lot
more processes.

 
 '/etc/systemd/system.conf':
 [Manager]
 LogTarget=syslog-or-kmsg

This is exactly the same setting as the kernel cmdline option above, and
hence redundant.

 DefaultStandardOutput=syslog

This has not the effect you might assume. It just tells journald to
forward data it receives from the daemons to syslog, but it does that
anyway by default. It's hence fully equivalent to the default of
journal, unless you globally turned off forwarding to syslog in
journald.

I have extended the man page now to explain this in more detail

 '/etc/systemd/user.conf':
 [Manager]
 LogTarget=syslog-or-kmsg

This makes little sense, as unpriviliged processes cannot lot to kmsg.

 '/etc/systemd/journald.conf':
 [Journal]
 Storage=none

This is the only relevant setting really.

 It's good that I have no /var/log/journal/* files, but
 - journald is still runnig (this isn't too imporatant, but when it
is possible work without it, it will be better)

it think you have weird definitions of goodand better

 - fundamental problem seems be, that some daemons logs not appear
in rsyslog files, for unknown reason.

Note that nowadays rsyslog doesn't even care about systemd forwarding
things to syslog, but instead pulls everything out of the journal on its
own. 

If you turn off local storage of the journal entirely, then rsyslog
cannot pull anything out of the journal anymore, since that's empty.

You can set Storage=volatile in which case journald will use a limited
ring buffer in /run, which rsyslog then can pull the data out of.

 systemd/journald man pages are not clear how solve this, please can
 someone touch me to right direction?

Yeah, we usually document how to use software, not to how to not use
it...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-13 Thread Lennart Poettering
On Thu, 07.08.14 14:34, Jon Stanley (jonstan...@gmail.com) wrote:

 
 On Thu, Aug 7, 2014 at 2:06 PM, Frantisek Hanzlik fra...@hanzlici.cz wrote:
  It's good that I have no /var/log/journal/* files, but
  - journald is still runnig (this isn't too imporatant, but when it
 is possible work without it, it will be better)
 
 Why do you want this? As Johann very tersely replied, it's not
 possible for a systemd-based system not to use the  journal. There is
 nothing that says that the journal needs to be persistent as you found
 (however you'll be eating up RAM with the journal if it's not backed
 by disk - I'm not 100% sure what happens if there is a backing store,
 i.e. does it still store in memory?)

tmpfs is swappable memory, hence gets swapped out when not used. Note
that the journal enforces size limits based on the nominal size of the
tmpfs though. See man page for details.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-13 Thread Lennart Poettering
On Thu, 07.08.14 18:11, Jóhann B. Guðmundsson (johan...@gmail.com) wrote:

 Arguably one of journals major/only shortcoming compared to what's
 out there is it's lack the ability to send syslog messages over the
 syslog network protocol but I think it's just a matter of time until
 it does, since it's arguably unavoidable ( think for example
 containers here and I would be amazed if submitted patches would be
 rejected that would add that )

Networking is a worthy goal and we (especially Zbigniew) are working
towards it, but I am not sure this implies using the BSD syslog
protocol.

BSD syslog is lossy and very weakly defined. The problems of
normalization are problems I have no intention to ever deal with. If
people want to forward the journal over BSD syslog/UDP, then that's
totally OK, but there's rsyslog for that, that can do that just fine, so
I am not sure why the journal would need that.

Note that containers already have pretty nice journal integration. For
example journalctl -M foobar gives you the logs of container foobar,
and so on. No networking involved with that, just direct disk access.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-13 Thread Lennart Poettering
On Thu, 07.08.14 15:44, Leonid Isaev (lis...@umail.iu.edu) wrote:

 Hi,
 
 On Thu, Aug 07, 2014 at 06:11:39PM +, Jóhann B. Guðmundsson wrote:
  
  On 08/07/2014 04:12 PM, Leonid Isaev wrote:
  Perhaps understanding why you're allergic to the journal would help in
  figuring out solutions to the actual underlying problem.
  There is nothing wrong with the journald per se, but it's not a 
  replacement for
  the classic syslog
  
  Yes it is.
 
 Hmm, reading my message above, I can see that it wasn't clear enough -- sorry.
 Perhaps an example can clarify things.
 
 Take dnsmasq which under normal operation logs _lots_ of DHCP-related 
 messages,
 even on a tiny network of ~20 (crappy Android) devices. These messages fall
 into 2 categories: routine (log_level info -- DHCPREQUEST, DHCPACK, etc.) and
 security-related (log_level warn -- DNS rebind attacks e.g.). I want the 
 former
 to be volatile (stored in /run/log), while the latter on-disk (in /var/log).
 
 While there are many ways to accomplish this with rsyslog/syslog-ng filters,
 I'd very much like to know how to do this with journald.

Splitting things up based on the log level sounds like a good idea, and
is in fact already on the TODO list. Happy to take patches.

However, note that I really don't want a generic regexp-or-something
based engine in journald. For that kind of stuff, please use rsyslog.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Frantisek Hanzlik
Hi, I just install Fedora 20 (with systemd 208) and want log,
if possible, without journald, only to rsyslog.

I have this configuration:

'systemd.log_target=syslog-or-kmsg' at kernel command line

'/etc/systemd/system.conf':
[Manager]
LogTarget=syslog-or-kmsg
DefaultStandardOutput=syslog


'/etc/systemd/user.conf':
[Manager]
LogTarget=syslog-or-kmsg


'/etc/systemd/journald.conf':
[Journal]
Storage=none

It's good that I have no /var/log/journal/* files, but
- journald is still runnig (this isn't too imporatant, but when it
   is possible work without it, it will be better)
- fundamental problem seems be, that some daemons logs not appear
   in rsyslog files, for unknown reason.

systemd/journald man pages are not clear how solve this, please can
someone touch me to right direction?

Thanks in advance, Franta Hanzlik
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 12:06 PM, Frantisek Hanzlik wrote:

Hi, I just install Fedora 20 (with systemd 208) and want log,
if possible, without journald, only to rsyslog.


It's not possible.

JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jon Stanley
On Thu, Aug 7, 2014 at 2:06 PM, Frantisek Hanzlik fra...@hanzlici.cz wrote:
 It's good that I have no /var/log/journal/* files, but
 - journald is still runnig (this isn't too imporatant, but when it
is possible work without it, it will be better)

Why do you want this? As Johann very tersely replied, it's not
possible for a systemd-based system not to use the  journal. There is
nothing that says that the journal needs to be persistent as you found
(however you'll be eating up RAM with the journal if it's not backed
by disk - I'm not 100% sure what happens if there is a backing store,
i.e. does it still store in memory?)

Perhaps understanding why you're allergic to the journal would help in
figuring out solutions to the actual underlying problem.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Leonid Isaev
Hi,

On Thu, Aug 07, 2014 at 02:06:07PM +0200, Frantisek Hanzlik wrote:
 
 '/etc/systemd/system.conf':
 [Manager]
 LogTarget=syslog-or-kmsg
 DefaultStandardOutput=syslog

Leave 'LogTarget=' to its default value (journal-or-kmsg).

 It's good that I have no /var/log/journal/* files, but
 - journald is still runnig (this isn't too imporatant, but when it
is possible work without it, it will be better)
 - fundamental problem seems be, that some daemons logs not appear
in rsyslog files, for unknown reason.

Could you be more specific about which daemons are affected?

Many daemons that are started by systemd service run in foreground (don't ask
me about the logic behind this). This means that all output is sent to stdout,
not syslog. Journald collects this output and forwards it to syslog -- hence my
recommendation above.

For example, sshd(8) now runs with '-D'. Of course, some daemons might send
their messages syslog even if run in foreground, but not all.

A hard way to fix this is to modify service files for daemons you use to run in
background and set 'Type=forking'.

HTH,
-- 
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6  20DF 9291 EE8A 043C B8C4
  C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D


pgpW64ANKNKmh.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Leonid Isaev
On Thu, Aug 07, 2014 at 02:34:20PM +0200, Jon Stanley wrote:
 I'm not 100% sure what happens if there is a backing store,
 i.e. does it still store in memory?

No.

 Perhaps understanding why you're allergic to the journal would help in
 figuring out solutions to the actual underlying problem.

There is nothing wrong with the journald per se, but it's not a replacement for
the classic syslog: journald offers only log storage, while syslog is a log
processing tool. This distinction is dim on a desktop, but is very apparent
even on a simple server machine.

Cheers,
-- 
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6  20DF 9291 EE8A 043C B8C4
  C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D


pgpj26XVDgpUh.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 04:12 PM, Leonid Isaev wrote:

Perhaps understanding why you're allergic to the journal would help in
figuring out solutions to the actual underlying problem.

There is nothing wrong with the journald per se, but it's not a replacement for
the classic syslog


Yes it is.

And there is a very much difference in using one of the traditional 
message printing APIs like syslog for logging as in doing this


#include syslog.h
int main(int argc, char *argv[])
   { syslog(LOG_NOTICE, Hello World);
return 0;
}


Vs using the journal's native APIs as in this.

#include systemd/sd-journal.h
int main(int argc, char *argv[])
   { sd_journal_print(LOG_NOTICE, Hello World);
return 0;
}

Arguably one of journals major/only shortcoming compared to what's out 
there is it's lack the ability to send syslog messages over the syslog 
network protocol but I think it's just a matter of time until it does, 
since it's arguably unavoidable ( think for example containers here and 
I would be amazed if submitted patches would be rejected that would add 
that )


But I guess you can hack yourself around that shortcoming by turning off 
persistent storage ( that is if you dont want to store logs as well on 
the host ) and run something like


journalctl  -o short -f  | nc ip -u 514 -w 1

that avoids the problem having two loggers running on the same host  ( 
like using syslog-ng or rsyslog alongside journal ) to solve that 
particular problem.


JBG

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Mantas Mikulėnas
On Aug 7, 2014 9:11 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote:
 Arguably one of journals major/only shortcoming compared to what's out
there is it's lack the ability to send syslog messages over the syslog
network protocol but I think it's just a matter of time until it does,
since it's arguably unavoidable ( think for example containers here and I
would be amazed if submitted patches would be rejected that would add that )

Yes, it has been mentioned a couple of times that dealing with the various
syslog protocols is the job of a syslogd, not the journal.

(That said, there already are some tools to push raw journal messages over
the network...)


 But I guess you can hack yourself around that shortcoming by turning off
persistent storage ( that is if you dont want to store logs as well on the
host ) and run something like

 journalctl  -o short -f  | nc ip -u 514 -w 1

 that avoids the problem having two loggers running on the same host  (
like using syslog-ng or rsyslog alongside journal ) to solve that
particular problem.

I don't understand why running two programs that provide distinct functions
is called a problem.

I also don't understand why running *three* programs (journald, journalctl,
netcat) that only do a halfassed job compared to rsyslog *isn't* a problem
anymore...

-- 
Mantas Mikulėnas graw...@gmail.com
// sent from phone
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Leonid Isaev
Hi,

On Thu, Aug 07, 2014 at 06:11:39PM +, Jóhann B. Guðmundsson wrote:
 
 On 08/07/2014 04:12 PM, Leonid Isaev wrote:
 Perhaps understanding why you're allergic to the journal would help in
 figuring out solutions to the actual underlying problem.
 There is nothing wrong with the journald per se, but it's not a replacement 
 for
 the classic syslog
 
 Yes it is.

Hmm, reading my message above, I can see that it wasn't clear enough -- sorry.
Perhaps an example can clarify things.

Take dnsmasq which under normal operation logs _lots_ of DHCP-related messages,
even on a tiny network of ~20 (crappy Android) devices. These messages fall
into 2 categories: routine (log_level info -- DHCPREQUEST, DHCPACK, etc.) and
security-related (log_level warn -- DNS rebind attacks e.g.). I want the former
to be volatile (stored in /run/log), while the latter on-disk (in /var/log).

While there are many ways to accomplish this with rsyslog/syslog-ng filters,
I'd very much like to know how to do this with journald.

Thanks,
-- 
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6  20DF 9291 EE8A 043C B8C4
  C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D


pgpYQhD4Lmcgr.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 07:44 PM, Leonid Isaev wrote:

Hi,

On Thu, Aug 07, 2014 at 06:11:39PM +, Jóhann B. Guðmundsson wrote:

On 08/07/2014 04:12 PM, Leonid Isaev wrote:

Perhaps understanding why you're allergic to the journal would help in
figuring out solutions to the actual underlying problem.

There is nothing wrong with the journald per se, but it's not a replacement for
the classic syslog

Yes it is.

Hmm, reading my message above, I can see that it wasn't clear enough -- sorry.
Perhaps an example can clarify things.

Take dnsmasq which under normal operation logs _lots_ of DHCP-related messages,
even on a tiny network of ~20 (crappy Android) devices. These messages fall
into 2 categories: routine (log_level info -- DHCPREQUEST, DHCPACK, etc.) and
security-related (log_level warn -- DNS rebind attacks e.g.). I want the former
to be volatile (stored in /run/log), while the latter on-disk (in /var/log).

While there are many ways to accomplish this with rsyslog/syslog-ng filters,


Give me an actual working example how this is solved using 
rsyslog/syslog-ng filters


JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 07:32 PM, Mantas Mikulėnas wrote:


On Aug 7, 2014 9:11 PM, Jóhann B. Guðmundsson johan...@gmail.com 
mailto:johan...@gmail.com wrote:
 Arguably one of journals major/only shortcoming compared to what's 
out there is it's lack the ability to send syslog messages over the 
syslog network protocol but I think it's just a matter of time until 
it does, since it's arguably unavoidable ( think for example 
containers here and I would be amazed if submitted patches would be 
rejected that would add that )


Yes, it has been mentioned a couple of times that dealing with the 
various syslog protocols is the job of a syslogd, not the journal.


(That said, there already are some tools to push raw journal messages 
over the network...)




Raw journals or journal only solution is not acceptable in large 
environment using mixed OS and or even just mixed Linux distributions 
and their releases ( think debian stable and centos7 for example )  so 
it's necessary for journal to be able to forward the logs over the 
syslog network protocol




 But I guess you can hack yourself around that shortcoming by turning 
off persistent storage ( that is if you dont want to store logs as 
well on the host ) and run something like


 journalctl  -o short -f  | nc ip -u 514 -w 1

 that avoids the problem having two loggers running on the same 
host  ( like using syslog-ng or rsyslog alongside journal ) to solve 
that particular problem.


I don't understand why running two programs that provide distinct 
functions is called a problem.




Host resources

I also don't understand why running *three* programs (journald, 
journalctl, netcat) that only do a halfassed job compared to rsyslog 
*isn't* a problem anymore...




You do realize what I proposed was a workaround right?

JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Leonid Isaev
On Thu, Aug 07, 2014 at 08:01:31PM +, Jóhann B. Guðmundsson wrote:
 Give me an actual working example how this is solved using rsyslog/syslog-ng
 filters

A quick (and probably dirty) way with syslog-ng:
--
% grep dnsmasq /etc/syslog-ng/syslog-ng.conf 
destination d_dnsmasq { file(/run/log/dnsmasq.log); };
filter f_daemon { facility(daemon) and not level(debug) and not program(hostap) 
and not program(dnsmasq-dhcp); };
filter f_dnsmasq { program(dnsmasq-dhcp); };
log { source(src); filter(f_dnsmasq); destination(d_dnsmasq); };
--

So, dnsmasq-dhcp is the prefix (used by dnsmasq by default) for normal
messages, while dnsmasq -- for everything else. Here is some statistics:
--
% uptime
 16:38:29 up 22 days, 22:05,  1 user,  load average: 0.00, 0.01, 0.05

% wc -l /run/log/dnsmasq.log*
1212 /run/log/dnsmasq.log
972 /run/log/dnsmasq.log.1
2077 /run/log/dnsmasq.log.2
958 /run/log/dnsmasq.log.3
5219 total

% head -n 2 /run/log/dnsmasq.log
2014-08-03T00:05:42.00-04:00 metal-0 dnsmasq-dhcp[460]: DHCPREQUEST(br0) 
10.0.0.4 30:39:26:e3:ec:4e
2014-08-03T00:05:42.00-04:00 metal-0 dnsmasq-dhcp[460]: DHCPACK(br0) 
10.0.0.4 30:39:26:e3:ec:4e android-2f74c9ab3fa43caa

% for i in /var/log/daemon.log*; do echo $i; grep dnsmasq $i | wc -l;  done
/var/log/daemon.log
0
/var/log/daemon.log.1
1
/var/log/daemon.log.2
2
/var/log/daemon.log.3
11
/var/log/daemon.log.4
11
/var/log/daemon.log.5
0
/var/log/daemon.log.6
0
/var/log/daemon.log.7
23
/var/log/daemon.log.8
30

% grep dnsmasq /var/log/daemon.log.1 
2014-08-02T15:46:05.00-04:00 metal-0 dnsmasq[460]: possible DNS-rebind 
attack detected: direct.stroyka.ru
--

Sorry for a long reply...
-- 
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6  20DF 9291 EE8A 043C B8C4
  C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D


pgpAhca1Wxzhm.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 08:41 PM, Leonid Isaev wrote:

Sorry for a long reply...


No problem I needed to see how you were thinking/doing this.

So basically you want to log everything to /run ( volatile ) and filter 
out everything above a certain log-level and store that persistent in 
it's own journal

( basically store the output from this journalctl -p err persistently )

Or you want to log everything to /run ( volatile ) and filter out 
everything above a certain log-level for a specific user,unit,command 
whatever and store that persistent in it's own journal.
( using your example store the output from this journalctl -p err 
_SYSTEMD_UNIT=dnsmasq.service persistently )


One of the Samsung guys proposed something similar to the former a while 
back ( and I think he signed himself up to it ) but as far as I can tell 
his work has not landed yet.
( afaikt requires changes to journald-server.c|||introduce something 
like SplitMode=priority-err |etc ).


I would not expect anything like this soon since Andy NAK their 
SCM_PROCINFO stuff and they are probably to busy 
re-writing/re-implementing it as SCM_IDENTY together with him but one of 
the Samsung guys can comment if they had started working on or had 
otherwise looked into this but as things stand now this cannot be done 
afaikt.


JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Leonid Isaev
Hi,

On Thu, Aug 07, 2014 at 09:44:47PM +, Jóhann B. Guðmundsson wrote:
 
 So basically you want to log everything to /run ( volatile ) and filter out
 everything above a certain log-level and store that persistent in it's own
 journal
 ( basically store the output from this journalctl -p err persistently )
 
 Or you want to log everything to /run ( volatile ) and filter out everything
 above a certain log-level for a specific user,unit,command whatever and
 store that persistent in it's own journal.
 ( using your example store the output from this journalctl -p err
 _SYSTEMD_UNIT=dnsmasq.service persistently )

My original motivation was to reduce HDD spin-ups (academic, I know). So I had
to identify sources of frequent logging activity and figure out which log
messages are actually valuable and which can be discarded on reboot. The same
rationality applies to remote logging, e.g. only auth-level events and critical
hardware telemetry should be sent to a log-server.

 One of the Samsung guys proposed something similar to the former a while
 back ( and I think he signed himself up to it ) but as far as I can tell his
 work has not landed yet.
 ( afaikt requires changes to journald-server.c|||introduce something like
 SplitMode=priority-err |etc ).

Thanks for letting me know aboout this work, but from the above description it
seems rather limited. I brought up the log-levels only as an example. In
practice one needs to be able to filter using _any_ message attribute.

For instance, message body (iptables traffic, output of frequently-run systemd
timers -- drop the useless Start/Stop-type messages, HostAp logs) and facility
(kernel/daemon/...).

 I would not expect anything like this soon since Andy NAK their SCM_PROCINFO
 stuff and they are probably to busy re-writing/re-implementing it as
 SCM_IDENTY together with him but one of the Samsung guys can comment if they
 had started working on or had otherwise looked into this but as things stand
 now this cannot be done afaikt.

IMHO, the central technical problem (I am not going to argue about design
principles) of journald is that it is an all or nothing solution.
Unfortunately, this inflexibility makes it only useful as a supplimentary
logger...

Cheers,
-- 
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6  20DF 9291 EE8A 043C B8C4
  C0DF 20D0 C075 C3F1 E1BE  775A A7AE F6CB 164B 5A6D


pgpGkfWoCl5FC.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] right way to log to rsyslog/syslog only?

2014-08-07 Thread Jóhann B. Guðmundsson


On 08/07/2014 10:42 PM, Leonid Isaev wrote:

Thanks for letting me know aboout this work, but from the above description it
seems rather limited. I brought up the log-levels only as an example. In
practice one needs to be able to filter using_any_  message attribute.


I just used the example to reply to your own so they are equally limited 
in that manner see systemd.journal-fields(7)


For instance, message body (iptables traffic, output of frequently-run systemd
timers -- drop the useless Start/Stop-type messages, HostAp logs) and facility
(kernel/daemon/...).


And you have configured syslog-ng and rsyslog to do that for you and how 
much time did it take?


I can understand the need for very powerful filter capabilities which 
can be used when needed and the journalctl already possesses those.


In the sample you showed me how you are doing things you did so in three 
steps 1 configure syslog-ng 2 parse through files with log level lower 
then error, parse through files with error


But I myself am a lazy old fat admin that has been administrating server 
for what 10 years now and prefer to use this journalctl -p err 
_SYSTEMD_UNIT=dnsmasq.service which yields the same result in one step 
( for each log level ) and I dont have to worry about installing or 
setting up anything basically I prefer I simply asking the journal to 
give me the information I need when I need it.


But why do you need to log all of this into their own persistent journal 
files, what practical problem are you hoping to solve,achieve or gain by 
that?


JBG

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel