Re: [rsyslog] json properties

On 9/17/19 7:48 AM, Brian Candler via rsyslog wrote: On 17/09/2019 00:42, David Lang wrote: Just to confirm what I *think* is going on: $! is only the content which was parsed by mmjsonparse (otherwise empty or does not exist) $!all-json and $!all-json-plain are $! re-serialized as a JSON

Re: [rsyslog] building rsyslog from cloned https://github.com/rsyslog/rsyslog.git issue on AmazonLinux

On 6/13/19 1:43 PM, Li, Mike via rsyslog wrote: Hi all, cat /etc/os-release NAME="Amazon Linux AMI" VERSION="2018.03" ID_LIKE="rhel fedora" VERSION_ID="2018.03" PRETTY_NAME="Amazon Linux AMI 2018.03" CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"

Re: [rsyslog] Can we move various count of fields to the different level?

On 5/29/19 1:27 PM, Noriko Hosoi via rsyslog wrote: Hello rsyslog list, [1] is an example of pre-processed log.  We'd like to keep fields defined in the environment variable [3] in the top level, and move the rest to one level down as in [2].  In this example, 2 fields "test0" and "test1" are

Re: [rsyslog] Rsyslog and elasticsearch

On 4/12/19 3:24 AM, David Lang via rsyslog wrote: to clarify, what elasticsearch and Splunk call an 'index' is not the same thing that you know of as an index under any other systems. An "Index" in Splunk or Elasticsearch is what Postgres/MYSql/Oracle/etc call a databass or table (it blurs

Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and forwarding

: rsyslog [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rich Megginson via rsyslog Sent: Wednesday, April 03, 2019 12:25 PM To: rsyslog@lists.adiscon.com Cc: Rich Megginson Subject: Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and forwarding On 4/3/19 11:19 AM, Gorman, Kevin vi

Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and forwarding

On 4/3/19 11:19 AM, Gorman, Kevin via rsyslog wrote: The *.* call linux_forward was the key. Much cleaner. Thanks! I also see the audit logs in the debug log and at the other end in tcpdump. I don't see the messages in the remote log system data, but that's another issue and product receiving

Re: [rsyslog] Rsyslog vs syslog-ng

On 2/5/19 2:40 AM, Peter Viskup via rsyslog wrote: To be honest, the main reason Debian chosen rsyslog as primary syslog daemon was that it does work with "standard syslog" configuration (more information can be read on https://wiki.debian.org/Rsyslog ). Nevertheless in newest versions of

Re: [rsyslog] Parse & use date

Use a lookup table - https://www.rsyslog.com/doc/v8-stable/rainerscript/lookup_tables.html here are some examples of using a lookup table to map priority numeric values to log level string values, and log level values to their canonical representation:

Re: [rsyslog] Parse unstructured text using mmnormalize

Indeed there is http://www.liblognorm.com/files/manual/lognormalizer.html and it is incredibly useful On 12/11/18 4:18 PM, Joe Blow via rsyslog wrote: Isn't there a lognormalize command you can use to test this stuff with? I seem to remember being able to specify a rules file, and a log file,

Re: [rsyslog] mmnormalize - unable to load module

On 12/5/18 10:11 AM, Lavanya Kanchanapalli via rsyslog wrote: Hi, I am trying to use rsyslog to parse and structure unstructured text application logs (tomcat logs). Based on the documentation and example rsyslog recipes

Re: [rsyslog] Missing messages in our databases

Jacob's answer is better, but I'll answer your questions below. On 11/20/18 8:49 AM, Mike Schleif wrote: What am I missing? On Tue, Nov 20, 2018 at 9:15 AM Rich Megginson via rsyslog < rsyslog@lists.adiscon.com> wrote: You could try this: # debug output module(load="builtin:o

Re: [rsyslog] Missing messages in our databases

On 11/20/18 8:06 AM, Mike Schleif wrote: David, On Mon, Nov 19, 2018 at 2:52 PM David Lang wrote: On Mon, 19 Nov 2018, Mike Schleif wrote: How can I do that? in your config for writing to mysql you specify a template to use, use that same template to write to a file old syntax

Re: [rsyslog] Forwarding date folder named logs on an rsyslog server

On 10/2/18 4:39 AM, John Cooper via rsyslog wrote: Hi, our incoming syslogs are stored in date folders on the rsyslog server. We also want some of those logs forwarding to another server. Using imfile, the "File" name will change for the next day meaning we would have to update the config each

[rsyslog] Ansible role for rsyslog

As part of the linux system roles project, there has recently been activity in the area of a role for a system log collector.  Notably, there has been a recent contribution of an rsyslog role: https://github.com/linux-system-roles/logging/tree/master/roles/rsyslog - the work is based on the

Re: [rsyslog] Question on multiline log messages

https://github.com/rsyslog/rsyslog/pull/3014 On 09/06/2018 05:07 PM, Rich Megginson via rsyslog wrote: On 09/06/2018 12:49 PM, Rich Megginson wrote: On 09/06/2018 11:30 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote

Re: [rsyslog] Question on multiline log messages

On 09/06/2018 12:49 PM, Rich Megginson wrote: On 09/06/2018 11:30 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for

Re: [rsyslog] Question on multiline log messages

On 09/06/2018 03:45 PM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: Just to clarify - rsyslog does not currently support "multiline" docker json-file nor crio logs because there is no endmsg.regex - we would need to add support for that first in order to use mmnormalize repeat

Re: [rsyslog] Question on multiline log messages

On 09/06/2018 11:57 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: That is, if multiline specifically means records like this: first line of recordA    recordA second line    recordA third line first line of recordB Perhaps Noriko and I mean something slightly different

Re: [rsyslog] Question on multiline log messages

On 09/06/2018 11:30 AM, David Lang wrote: On Thu, 6 Sep 2018, Rich Megginson wrote: On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages

Re: [rsyslog] Question on multiline log messages

On 09/06/2018 12:00 PM, Noriko Hosoi via rsyslog wrote: Thank you for your response, David. On 09/06/2018 06:18 AM, David Lang wrote: There is an option for imfile to split a file into multiline messages based on a regex, that is probably the right starting point. Yes, we also expected

Re: [rsyslog] mmkubernetes: how to handle 429 Busy response?

On 08/28/2018 08:20 PM, David Lang wrote: On Tue, 28 Aug 2018, Rich Megginson wrote: It would have to be done inside the module, rsyslog has back-off capabilities, but only when the entire pipeline is suspended. Another option is to handle 429 in mmkubernetes similar to how it is handled in

Re: [rsyslog] mmkubernetes: how to handle 429 Busy response?

On 08/28/2018 06:56 PM, David Lang wrote: On Tue, 28 Aug 2018, Rich Megginson wrote: On 08/28/2018 06:20 PM, David Lang wrote: On Tue, 28 Aug 2018, Rich Megginson via rsyslog wrote: As part of the fix for https://github.com/rsyslog/rsyslog/pull/2962 I've added handling for error 429 Busy

Re: [rsyslog] mmkubernetes: how to handle 429 Busy response?

On 08/28/2018 06:20 PM, David Lang wrote: On Tue, 28 Aug 2018, Rich Megginson via rsyslog wrote: As part of the fix for https://github.com/rsyslog/rsyslog/pull/2962 I've added handling for error 429 Busy.  I've seen this most commonly with large Kubernetes clusters where if you have hundreds

Re: [rsyslog] kubernetes connection to masters

On 08/28/2018 06:23 PM, David Lang wrote: As we are looking at managing and securing our k8s install, we are finding that getting a token out of the pod/container runing the management processes is non-trivial, especially to get it onto a separate log server. would it be possible to add

[rsyslog] mmkubernetes: how to handle 429 Busy response?

As part of the fix for https://github.com/rsyslog/rsyslog/pull/2962 I've added handling for error 429 Busy.  I've seen this most commonly with large Kubernetes clusters where if you have hundreds of nodes, all with open connections to the Kubernetes API service, the service will eventually hit

Re: [rsyslog] It's not necessary to separate JSON field with space?

If you are unable to submit a PR, then please submit an issue tracker. On 08/07/2018 08:14 AM, eSX wrote: I very much agree with this. There must be a lot of non-standard implementations for log parser. A flag/swith would be a good choice. I will submit PR if I have time. ;) Rich Megginson

Re: [rsyslog] It's not necessary to separate JSON field with space?

On 08/07/2018 03:26 AM, eSX via rsyslog wrote: Hi all When the log message part is JSON, a message processing like: parse json with mmjsonparser, process variables in $!. unset variables, e.g. unset $!foo!bar; Alright, send a processed message to next action. But there were some variables

Re: [rsyslog] omes in omelasticsearch should include more information

ne) and save the request body as smsg_t. _index and _type copied from response of Elasticsearch. So the request parameters we had configured("pipeline"), were lost in request header. I think, to keep the paramenters which generated by initial request is enough for now. A more easy wa

Re: [rsyslog] omes in omelasticsearch should include more information

to keep the paramenters which generated by initial request is enough for now. A more easy way to configure omelasticsearch is all in one action, but it's hard to code and inflexible? In any case, I think it's necessary to keep the variables of initial request. Rich Megginson via rsyslog

Re: [rsyslog] omes in omelasticsearch should include more information

On 08/02/2018 01:18 AM, eSX via rsyslog wrote: Hello rsyslog omes is the retry information when omelasticsearch is in failure and retry. The contents of omes are response message from Elasticsearch mainly. But the response didn't give a full information for retring. What information is

Re: [rsyslog] mmnormalize/mmkubernetes matching problem

On 07/02/2018 11:31 AM, David Lang wrote: On Mon, 2 Jul 2018, Rich Megginson wrote: On 06/29/2018 08:16 PM, David Lang wrote: On Fri, 29 Jun 2018, Rich Megginson via rsyslog wrote: On 06/29/2018 06:49 PM, David Lang wrote: On Fri, 29 Jun 2018, Rich Megginson via rsyslog wrote: On 06/29

Re: [rsyslog] mmnormalize/mmkubernetes matching problem

On 06/29/2018 09:02 PM, David Lang wrote: One other thing that I would like to see is an example of "I have the namespace and pod name for this log, now go fetch the data from the API for me". When does that happen?  That is, what is the use case when you are not immediately trying to

Re: [rsyslog] mmnormalize/mmkubernetes matching problem

On 06/29/2018 08:16 PM, David Lang wrote: On Fri, 29 Jun 2018, Rich Megginson via rsyslog wrote: On 06/29/2018 06:49 PM, David Lang wrote: On Fri, 29 Jun 2018, Rich Megginson via rsyslog wrote: On 06/29/2018 03:08 PM, David Lang wrote: the filename I have is: $!:{ "metadata": {

Re: [rsyslog] mmnormalize/mmkubernetes matching problem

On 06/29/2018 06:49 PM, David Lang wrote: On Fri, 29 Jun 2018, Rich Megginson via rsyslog wrote: On 06/29/2018 03:08 PM, David Lang wrote: the filename I have is: $!:{ "metadata": { "filename": "\/var\/log\/containers\/calico-node-jlhg

Re: [rsyslog] mmnormalize/mmkubernetes matching problem

On 06/29/2018 03:08 PM, David Lang wrote: the filename I have is: $!:{ "metadata": { "filename": "\/var\/log\/containers\/calico-node-jlhg5_kube-system_calico-node-e9abcaf4367e0264243c955699cd0987ff610df33fc9299facb97d1cb97aca2b.log", "fileoffset": "553373" } } the rule that should match it

[rsyslog] rsyslog support for output to S3?

http://rsyslog-users.1305293.n2.nabble.com/Writing-logs-to-S3-td7581369.html This was from 2013.  Is there any S3 support in rsyslog? ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog

Re: [rsyslog] omelasticsearch - failed operation handling

maybe the actual code will explain what I intend: https://github.com/rsyslog/rsyslog/pull/2733 On 05/18/2018 10:52 AM, Rainer Gerhards wrote: Just quicky chiming in, will need to catch a plane early tomorrow morning. It's complicated. At this point, the original message is no longer

Re: [rsyslog] omelasticsearch - failed operation handling

On 05/17/2018 05:52 AM, Brian Knox wrote: To my knowledge, Rich is correct. This also would explain a case we hit maybe every couple of months, where rsyslog very quickly duplicates some messages it is sending to elasticsearch. I would assume this would be a case where a batch is submitted,

Re: [rsyslog] omelasticsearch - failed operation handling

On 05/16/2018 10:08 PM, David Lang wrote: On Wed, 16 May 2018, Rich Megginson wrote: On 05/16/2018 05:58 PM, David Lang wrote: there's no need to add this extra complexity (multiple rulesets and queues) What should be happening (on any output module) is: submit a batch.    If rejected with

Re: [rsyslog] omelasticsearch - failed operation handling

On 05/16/2018 05:58 PM, David Lang wrote: there's no need to add this extra complexity (multiple rulesets and queues) What should be happening (on any output module) is: submit a batch.    If rejected with a soft error, retry/suspend the output retry of the entire batch?  see below if

[rsyslog] omelasticsearch - failed operation handling

In many cases, when adding a record to Elasticsearch, an http status not 200 or 201 does not necessarily indicate that the record cannot be added.  One case is bulk index rejection - in this case, the http status for the record in the response is 429, and it may be that a short pause is

Re: [rsyslog] segfault with mmkubernetes

I think this PR will address most of your concerns: https://github.com/rsyslog/rsyslog/pull/2635 > Is there a way to change the log-driver for just one pod/container as opposed to the entire cluster? Not sure. On 04/05/2018 08:12 PM, David Lang wrote: On Thu, 5 Apr 2018, Rich Megginson

Re: [rsyslog] segfault with mmkubernetes

On 04/06/2018 10:36 PM, David Lang wrote: On Fri, 6 Apr 2018, Rich Megginson wrote: On 04/06/2018 09:08 PM, David Lang wrote: In addition to the authentication, it looks like there is some other issue. where some info isn't there and it logs "filename: ''" and it looks like nothing is

Re: [rsyslog] segfault with mmkubernetes

On 04/06/2018 09:08 PM, David Lang wrote: In addition to the authentication, it looks like there is some other issue. where some info isn't there and it logs "filename: ''" and it looks like nothing is attempted 9992.862530379:main Q:Reg/w0  : ruleset.c: processBATCH: next msg 43: E0329

Re: [rsyslog] segfault with mmkubernetes

On 04/06/2018 04:07 PM, David Lang wrote: On Fri, 6 Apr 2018, Rich Megginson via rsyslog wrote: It is establishing a connection with the kubernetes server and passing the SSL handshake, I can't see what's happening over the connection, so I don't know what messages are being passed

Re: [rsyslog] segfault with mmkubernetes

On 04/06/2018 02:48 PM, David Lang wrote: On Thu, 5 Apr 2018, David Lang wrote: now that I have it working, what do we want to do in this release to make it better :-) (thanks for your help by the way) It still doesn't look like it is working.  There should be fields for kubernetes

Re: [rsyslog] segfault with mmkubernetes

On 04/05/2018 07:33 PM, David Lang wrote: On Thu, 5 Apr 2018, David Lang wrote: On Thu, 5 Apr 2018, Rich Megginson via rsyslog wrote: On 04/05/2018 01:59 PM, David Lang wrote: I had the wrong URL, but even fixing the URL doesn't seem to be working, I see no attempts to establish

Re: [rsyslog] segfault with mmkubernetes

On 04/05/2018 01:59 PM, David Lang wrote: I had the wrong URL, but even fixing the URL doesn't seem to be working, I see no attempts to establish a connection to the kubernetes server module(load="imuxsock") # provides support for local system logging (e.g. via logger command)

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 08:05 PM, Rich Megginson via rsyslog wrote: On 04/04/2018 07:51 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 07:18 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 06:42 PM, David Lang wrote: On Wed, 4 Apr 2018

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 07:51 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 07:18 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 06:42 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: Stacktrace? a bit hard to get

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 07:18 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 06:42 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: Stacktrace? a bit hard to get (things are being run on a production machine after umpteen layers of automation)

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 06:42 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: Stacktrace? a bit hard to get (things are being run on a production machine after umpteen layers of automation) here's what I can get so far ok - so definitely something wrong with the filenamerules

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 06:28 PM, David Lang wrote: On Wed, 4 Apr 2018, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson wrote: On 04/04/2018 05:57 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson via rsyslog wrote: On 04/04/2018 05:22 PM, David Lang wrote: I'm trying to use

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 05:57 PM, David Lang wrote: On Wed, 4 Apr 2018, Rich Megginson via rsyslog wrote: On 04/04/2018 05:22 PM, David Lang wrote: I'm trying to use mmkubernetes where the docker containers are logging via journald. my config is: module(load="imfile" PollingInterval="

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 05:43 PM, Rich Megginson via rsyslog wrote: On 04/04/2018 05:22 PM, David Lang wrote: I'm trying to use mmkubernetes where the docker containers are logging via journald. my config is: module(load="imfile" PollingInterval="10") # provided support for moni

Re: [rsyslog] segfault with mmkubernetes

On 04/04/2018 05:22 PM, David Lang wrote: I'm trying to use mmkubernetes where the docker containers are logging via journald. my config is: module(load="imfile" PollingInterval="10") # provided support for monitoring alternate log files module(load="imuxsock") # provides support for local

[rsyslog] how to parse json into top level fields with rsyslog 8.24?

I have a record which looks like this:

[rsyslog] convert high-res microsecond timestamp to rfc3339?

I have a timestamp from journal that looks like this: {unix time_t} {microseconds} e.g. 1520548089262116 how do I convert this to rfc3339 e.g. 2018-03-08T15:28:09.262116-07:00 ? ___ rsyslog mailing list

[rsyslog] Kubernetes Metadata plugin - mmkubernetes

https://github.com/rsyslog/rsyslog/pull/2466 If you are interested in adding Kubernetes pod and namespace metadata to log records, please take a look. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog

Re: [rsyslog] How to convert journald _SOURCE_REALTIME_TIMESTAMP to rfc3339 format

imjournal.c calls sd_journal_get_realtime_usec() to get the realtime timestamp of the message, which looks like it is the same as _SOURCE_REALTIME_TIMESTAMP imjournal.c then calls enqMsg() with this time value, which sets the TIMESTAMP property via msgConstructWithTime() which then can be used

[rsyslog] How to convert journald _SOURCE_REALTIME_TIMESTAMP to rfc3339 format

When using imjournal, I get the timestamp like this: _SOURCE_REALTIME_TIMESTAMP=1509546445053511 where the time value is essentially date +%s%6N I would like to convert this value to a string rfc3339 .6 format e.g. something like date --rfc-3339=ns --date=@1509546445.053511