I am reaching out again for some help to use imjournal to fetch journald log
entries while imuxsock is also used. I am having a systemd system,
rsyslog.service has "Requires=syslog.socket" commented out. rsyslog_v8.1903 is
used.
The /etc/rsyslog.conf I have:
module(load="imuxsock" SysSock.Use="on"
SysSock.Name="/run/systemd/journal/syslog")
module(load="builtin:omfile" dirCreateMode="0755"
fileCreateMode="0640")
module(load="mmjsonparse")action(type="mmjsonparse")
$FileOwner syslog$FileGroup syslog$FileCreateMode 0640$DirCreateMode 0755$Umask
0022$PrivDropToUser syslog$PrivDropToGroup syslog
include(file="/etc/rsyslog.d/custom-syslog.conf mode="required")
Where, /etc/rsyslog.d/custom-syslog.conf defines the
following:module(load="imjournal" PersistStateInterval="100"
StateFile="/path/to/file")if ( $inputname == "imuxsock" ) then {
action(type="omfile" file="/var/log/imuxsock_log")else if ( $inputname ==
"imjournal" ) then { action(type="omfile" file="/var/log/imjournal_log")}
I understand that the filtering of "$inputname" may or may not work as David's
response. I want to know the proper configuration to enable the use of
imjournal.
Thanks,Patrick
On Wednesday, February 5, 2020, 04:19:24 p.m. EST, David Lang
<[email protected]> wrote:
On Wed, 5 Feb 2020, Patrick Leung wrote:
> Thank you again for the information. Again, if I load both imuxsock and
> imjournal, how do I associate filters to each input module?
as you note, imjournal doesn't support binding to a ruleset, so logs from it
just go into the default ruleset.
if you log with the template RSYSLOG_DebugFormat you may find something you can
filter on (I suspect that $input will work, but I haven't tested it)
so something like
if $input == 'imjournal' and <other filter> then { action() }
David Lang
>e.g. rsyslog.confif ( filter1_condition ) then { action(xxx) }
> if ( filter2_condition ) then { action(yyy) }Associate "filter1" to process
> messages received by imuxsock and "filter2" to process messages received by
> imjournalimuxsock supports ruleset binding; but not imjournal.
> On Monday, February 3, 2020, 04:12:04 p.m. EST, David Lang <[email protected]>
>wrote:
>
> On Mon, 3 Feb 2020, Patrick Leung wrote:
>
>> Thank you for your clarification once again. If I understand it right, the
>> use of RSYSLOG_DebugFormat template gives me the generated syslog file that
>> contains structured journald log content, filtering is handled prior to
>> output log file generation.
>
> not quite, filtering happens how you define it in the config file.
>
> the debug format shows you what the contents of all the variables are at the
> point in the config file that you write the log with that format.
>
> This lets you see the contents to understand what your filters have to work
> with.
>
> filters can be the traditional facility.severity (i.e. mail.info) format, or
> they can be much more powerful if..then formats
>
>> What I would like to achieve:forward journald log to rsyslog, and use
>> rsyslog's filtering capabilities to filter journald master log file to
>> individual log file base on matched ruleset in rsyslog. I may want to filter
>> journald log base on systemd units and container services to their individual
>> log files.
>
> that's trivail
>
>> 1) Is it possible to load both imuxsock and imjournal modules when starting
>> rsyslog,
>
> yes, but if you have journald deliver the messages to imuxsock and have
> rsyslog
> fetch the messages via imjournal you will get two copies of every message
>
>
>> and have these modules process journald log and generate individual
>> log file base on the matching ruleset?
>
> yes, rsyslog runs every log message through the ruleset(s) in the config file
>
>> 2) If imjournal is loaded, I am not
>> crystal clear on rsyslog config syntax to set up the ruleset for structured
>> journald log entry filtering. Essentially I am looking for the equivalent
>> of, for example, "journalctl CONTAINER_ID=<id> > /path/to/container_id_log".
>
> you would do something like
>
> if $!CONTAINER_ID == "id" then /path/to/file
>
> note that you can also use dynamic file names so that you can use variables in
> the path, so you could end up writing to /path/to/ID/file (look for dynafile)
>
>> 3) $programname property (which is part of TAG stemmed from MSG syslog fornat
>> AFAIK) is available for ruleset matching when imuxsock is used?
>
> programname is available with both imuxsock and imjournal
>
> David Lang
>
>> Thanks again for help.
>> Thanks,Patrick
>> On Sunday, February 2, 2020, 04:59:50 a.m. EST, David Lang
>> <[email protected]> wrote:
>>
>> log the message with the template RSYSLOG_DebugFormat and it will show you
>> every
>> value it has parsed. You may need to use mmnormalize or mmjsonparse (the
>> first
>> is a batter long-term option as it is far more flexible) to extract the
>> values
>> from json if they aren't already parsed. look for a json string in the $!
>> section, every value there can be individually addressed.
>>
>> David Lang
>>
>>
>> On Sun, 2 Feb 2020,
>> Patrick Leung wrote:
>>
>>> Date: Sun, 2 Feb 2020 09:30:55 +0000 (UTC)
>>> From: Patrick Leung <[email protected]>
>>> To: Patrick Leung via rsyslog <[email protected]>,
>>> David Lang <[email protected]>
>>> Subject: Re: [rsyslog] rsyslog journald filtering
>>>
>>> Hello David,Indeed the information you have provided is useful. Another
>>> follow up question.
>>> In case of using imjournal to filter systemd-journald log that is being
>>> forwarded to rsyslog-v8, what property I can use in rsyslog config for such
>>> log filtering? What if the journald log entry contain custom field that I
>>> want to use as the filtering condition?
>>>
>>> Thanks,Patrick
>>>
>>> On Friday, January 3, 2020, 5:36:16 p.m. PST, David Lang
>>> <[email protected]> wrote:
>>>
>>> rsyslog does not change it's config during a run (with the exception of the
>>> table_lookup() function, which may work for you)
>>>
>>> syslog can filter on anything it knows about, so if you fetch from journald
>>> with
>>> imjournal so that you can see all the metadata that journald takes the time
>>> to
>>> lookup, you can filter on any of it. If you have journald write to a socket
>>> for
>>> rsyslog to read, journald doesn't send that metadata (and refuses to
>>> consider
>>> doing so, I've asked)
>>>
>>> does this answer your questions?
>>>
>>> David Lang
>>>
>>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
