[rsyslog] Rsyslog with queue files won't start

Just experienced issue with rsyslog with DA queue files. The process just died without any error. We do run rsyslog version 8.15.0 These are the last lines from debug output 8488.225603049:STRMEP2 queue[DA]:Reg/w0: omfwd: beginTransaction 8488.225606660:STRMEP2 queue[DA]:Reg/w0: 10.1.25.181

Re: [rsyslog] Rsyslog with queue files won't start

, David Lang <da...@lang.hm> wrote: > odds are that the queue files have been corrupted. you need to rebuild the > .qi file and then it should be able to startup. > > David Lang > > On Thu, 16 Feb 2017, Peter Viskup via rsyslog wrote: > >> Date: Thu, 16 Feb 2017 13:50:3

Re: [rsyslog] Mix of GTLS and PTCP listeners running same instance

Seems it is not possible to mix PTCP and TCP listeners. We run 8.15.0 version of rsyslog. With following configuration we are getting error messages: Jan 9 13:13:44 127.0.0.1 syslog.err rsyslogd-2081:error: driver mode 1 not supported by ptcp netstream driver [v8.15.0 try

Re: [rsyslog] Rsyslog with queue files won't start

dds are that the queue files have been corrupted. you need to rebuild the >> .qi file and then it should be able to startup. >> >> David Lang >> >> On Thu, 16 Feb 2017, Peter Viskup via rsyslog wrote: >> >>> Date: Thu, 16 Feb 2017 13:50:37 +0100

Re: [rsyslog] Rsyslog with queue files won't start

rha...@hq.adiscon.com> wrote: > You should move up to 8.25, chances are very good the issue is fixed > there. If not, we should see a better diagnostic (8.26 will have even > better diagnostics). > > Rainer > > 2017-03-23 8:40 GMT+01:00 Peter Viskup via rsyslog > <rsyslog@lists.

Re: [rsyslog] Rsyslog with queue files won't start

t;> Or any tool available out there? >>> >>> On Thu, Mar 23, 2017 at 8:43 AM, Rainer Gerhards >>> <rgerha...@hq.adiscon.com> wrote: >>>> >>>> You should move up to 8.25, chances are very good the issue is fixed >>>> there. If

Re: [rsyslog] Memory sizing issue

Just did some analysis of rsyslog stats counters and found the following. The maxrss counter is increasing accordingly to size of queue. Seems there is much higher overhead than expected. I tried the message sizes of 1840/940/640/340 characters. These are the outcomes: size

[rsyslog] Memory sizing issue

Hello all, we need to know how to do the proper sizing of memory for rsyslog server. We did the sizing of ActionQueueHighWaterMark according to the memory available on the server and median message size. Our assumption is the memory consumption should not grow "much higher" than

Re: [rsyslog] Memory sizing issue

ot;? Or in any other way? In case the whole debug line is equal to the message size in memory, the message of 162B consume 602B in memory. Is that right? -- Peter On Wed, Apr 5, 2017 at 11:11 PM, David Lang <da...@lang.hm> wrote: > On Wed, 5 Apr 2017, Peter Viskup via rsyslog wrote: >

Re: [rsyslog] Memory sizing issue

On Fri, Apr 7, 2017 at 6:32 PM, David Lang wrote: > On Fri, 7 Apr 2017, Peter Viskup wrote: > >> Just did some analysis of rsyslog stats counters and found the following. >> The maxrss counter is increasing accordingly to size of queue. >> >> Seems there is much higher overhead

Re: [rsyslog] rsyslog using gigabytes of RAM after recent updates (8.26?)

Hello Tomasz, this seems to be related to change in 8.26 about the error reporting. https://github.com/rsyslog/rsyslog/blob/master/ChangeLog Read comments for 8.26 version and "- enable internal error messages at all times". This is the reason why you do see messages which were not seen with

[rsyslog] rsyslog 8.4.2 imfile problem after logrotate

We are experiencing issue with rsyslog imfile module and logrotate. We do process 3 files with imfile in inotify mode. After logrotate 2 files are processed ok, but the third one is "stuck". This is the rsyslog imfile configuration: module(load="imfile" mode="inotify") # squid access log

Re: [rsyslog] rsyslog 8.4.2 imfile problem after logrotate

rha...@hq.adiscon.com> wrote: > I think the way to go forward is to install 8.28.0. This will probably > solve all issues. If not, we should discuss this further. > > Rainer > > 2017-07-18 11:10 GMT+02:00 Peter Viskup via rsyslog > <rsyslog@lists.adiscon.com>: >> We are experi

Re: [rsyslog] rsyslog 8.4.2 imfile problem after logrotate

Confirm with rsyslog update to backported 8.23 version the issue doesn't occur anymore with configuration intact. It is important to set delaycompress in logrotate configuration for all imfile-processed files. Peter On Tue, Jul 18, 2017 at 11:10 AM, Peter Viskup wrote: >

Re: [rsyslog] Memory sizing issue

The in-memory queue isn't dropped, only the counter is reset to 0 after a while. Just opened issue in rsyslog regarding the queue stats - seems to be there is a bug: https://github.com/rsyslog/rsyslog/issues/1585 Thus not able to do proper sizing based on the counters from impstats outputs at

Re: [rsyslog] RELP does not resume when target goes back after a few hours down

Check the rsyslog error messages on "action 'NAME' suspended, next retry is" the next message should be "action 'NAME' resumed". The options $ActionResumeInterval and $ActionResumeRetryCount needs to be configured according your expectations. More information in Documentation:

[rsyslog] List of threads changed between versions 8.15 and 8.23

Just discovered there is difference in list of threads for rsyslog 8.15 (our custom build) and 8.23 (Debian backported). Both systems running Debian8. This is the list of threads for version 8.15: ~# pstree -p 957 rsyslogd-net(957)─┬─{in:immark}(1028) ├─{in:impstats}(1029)

Re: [rsyslog] Failover Config not Working

Read queue documentation [1]. Search for discard, watermark and size parameters to limit the FS storage. Anyway sizing of queue is not as easy. At first you have to count approx. +350-500B of metadata per one message in queue. Good luck. [1]

Re: [rsyslog] Failover Config not Working

And other link to documentation with section about filled queues. https://www.rsyslog.com/doc/v8-stable/concepts/queues.html On Fri, May 25, 2018 at 2:22 PM, Peter Viskup wrote: > Read queue documentation [1]. Search for discard, watermark and size > parameters to limit the

Re: [rsyslog] Load balancing UDP logs

Hi Philippe, On Thu, Jun 14, 2018 at 1:47 PM, Maupertuis Philippe < philippe.maupert...@equensworldline.com> wrote: > Hi, > We have a load balancer (lvs+Keepalived) which is used to receive logs on four real server. Going to implement the same in next months. > Now we are requested to add udp

[rsyslog] Message bursts and TCP connection ratelimits

How to face situation when client(s) sending burst of messages to TCP input? The receiver is forwarding those messages for further processing where we want to "limit the peaks". Forwarding and processing servers run rsyslog, not all clients run rsyslogs. Standard imtcp module has ratelimit

[rsyslog] Experiences with Rsyslog with TLS

Am interested in experiences with running rsyslog as TLS sender/receiver. What rsyslog version (GnuTLS version) do you run? How many clients? What type of devices the clients are? What message and data rate? What auth method? Any issues do/did you face? Forwarding via Internet (to external IP) or

[rsyslog] Rsyslog 8.24 with omkafka and broker connection retries

Facing issue with omkafka and unavailable one of Kafka brokers. Causing approx. 420 connection retries every minute. What rsyslog omkafka or librdkafka arguments to setup to limit these connection retries? Seems that omkafka's argument ConfParam might be used to set some of the librdkafka

Re: [rsyslog] Rsyslog 8.24 with omkafka and broker connection retries

>> this is handled by librdkafka, so I would suggest to ask the question >> there - and let us know the URL. >> >> Rainer >> >> 2018-07-16 13:17 GMT+02:00 Peter Viskup via rsyslog < >> rsyslog@lists.adiscon.com>: >> > Facing issue with omkaf

[rsyslog] Binding template issue for omfwd

After rewrote of omfwd action from old-style to rainer-script with binding custom template according to information from FAQ article [1], the error messages pointing to misconfiguration: Jul 6 10:29:38 127.0.0.1 syslog.err rsyslogd-2207:error during parsing file

[rsyslog] Configuration syntax error and strange behavior

After configuration syntax error has been made, the rsyslog continued to work, but not as expected. Discovered issues with impstats and no TCP forward was active (4 are configured). Running rsyslog version 8.15. Config error: === # missing 'or' in if condition expression if not (

[rsyslog] Rsyslog TCP session reopen every ~15 seconds

Seeing errors Netstream session 0x7f2375fddeb0 closed by remote peer on rsyslog server caused by rsyslog client sending TCP FIN every ~15 seconds. Rsyslog client is of 8.15 version. Forwarding via omfwd ptcp driver with configuration: $ActionResumeInterval 30 $ActionResumeRetryCount -1

[rsyslog] TCP Keepalive on client and server side

>From my latest observation it seems the TCP Keepalive is not working as expected in our environment. We do run rsyslog 8.15, which I know is old, but cannot update. Want to make sure how the TCP Keepalive is developed in rsyslog and whether there were some changes since 8.15 release. At the

Re: [rsyslog] TCP Keepalive on client and server side

Thank you Rainer, the Changelog answered why client is not answering keepalive packets (bug fixed in 8.18). What about the TCP session open on client side? This happen every 16 seconds in parallel with other TCP session opened and used for data transfer. Following is session export from pcap:

Re: [rsyslog] TCP Keepalive on client and server side

Ack, will check after upgrade. As an workaround the tcp_retries2 kernel option was lowered according https://pracucci.com/linux-tcp-rto-min-max-and-tcp-retries2.html This make us sure the TCP forward session will be recognized as broken sooner than default 924 seconds. We are loosing messages

[rsyslog] Monitoring message delay

Interested in monitoring delay of message retrieval in syslog infrastructure. We have syslog infrastructure with more rsyslog relays in chain and would like to monitor the diff in times between timegenerated and timereported. Requirement is to be alerted when the messages will be delayed reaching

Re: [rsyslog] nanoseconds

It might be possible to extend the rfc3339 time format to rfc3339nano, but that will break rfc5424 which allow up to microseconds precision only. Similar already in use when rfc3164 syslog messages used with rfc3339 timestamps.

Re: [rsyslog] Combining two working rsyslog.conf files

Show the final config you are trying to run. It could be related to $DefaultNetstreamDriver* options which should be mentioned only once. https://www.rsyslog.com/doc/v8-stable/rainerscript/global.html?highlight=defaultnetstreamdriver In case it is needed, you can copy systemd rsyslog.service

Re: [rsyslog] how to get the original IP address in a relay chain

syslog-ng has special chain-hostname option for that. You can simulate it with exec_template with use of standard syslog format: http://rsyslog-users.1305293.n2.nabble.com/template/NamlServlet.jtp?macro=print_post=7594015 HTH -- Peter On Wed, Oct 17, 2018 at 1:38 AM wuhe wrote: > > > > Thanks

Re: [rsyslog] nanoseconds

he timestamp. > >> > >> I'm not sure that digits beyond microseconds really represent valid time, > >> but I > >> don't think it's a big deal to support it. > >> > >> David Lang > >> > >> On Mon, 29 Oct 2018, Peter Viskup via rsy

Re: [rsyslog] Forward template name based on variable

e="getFromhostip" type="string" string="_%fromhost-ip%") > > if ( $hostname == $fromhost-ip or $fromhost-ip == "127.0.0.1" ) then { > set $.ip=""; > } > else { > set $.ip=exec_template("getFromhostip"); >

Re: [rsyslog] Forward template name based on variable

e="omfile" file="/var/log/lin/lin-dyna.log" template="FileFormatDyn") On Wed, Sep 26, 2018 at 2:56 AM David Lang wrote: > > On Tue, 25 Sep 2018, Peter Viskup via rsyslog wrote: > > > Is it possible to configure omfwd action with template name chosen by

[rsyslog] Forward template name based on variable

Is it possible to configure omfwd action with template name chosen by variable? Want to use different template according the hostname value (simplified example): $template fwdrelay1,"<%PRI%>%TIMESTAMP:::date-rfc3339% %fromhost-ip%-%hostname% %syslogtag%%msg:::drop-last-lf%\n" $template

[rsyslog] rsyslog impstats disk-assisted queue size/enqueued counters

Just discovered not expected behavior. The DA queue size counter was changed, without change in enqueued counter. ~$ grep "Jan 15 12:23" /var/log/remotelogs/lin/rsyslog-lin.stats|grep main Jan 15 12:23:07 127.0.0.1 syslog.debug rsyslogd-pstats:main Q[DA]: origin=core.queue size=0 enqueued=3244357

Re: [rsyslog] Problem with substring function

Hello Oliver, try change line set $!user_name = substring(exec_template("username"),2,4); to lines: set $!user_name_tmp = exec_template("username"); set $!user_name= substring($!user_name_tmp,2,4); -- Peter On Thu, Nov 22, 2018 at 3:49 PM Neumann, Oliver wrote: > > Hi there, > > I’m in trouble

Re: [rsyslog] lognorm1 rules with optional message part

On Mon, Nov 19, 2018 at 9:29 PM David Lang wrote: > > On Mon, 19 Nov 2018, Peter Viskup via rsyslog wrote: > > > Special SD-ELEMENT [syslogTimes@123456 relay-ip="timestamp-rfc3339" > > ...] added to the end of structured-data. Every relay add it's own > > re

[rsyslog] lognorm1 rules with optional message part

It is for the first time I am working with liblognorm. Read the documentation for lognorm1, but still not sure how to write mmnormalize rules for optional parts of syslog message. The base is RFC5424 message with modified structured-data. Special SD-ELEMENT [syslogTimes@123456

[rsyslog] Keep original timestamp

Working on design of rsyslog relay servers (more than one in the path). Came to templates which are chaining fromhost-ip properties into hostname with _ delimiter. That is working fine and we can see the path the message passed. Now I would like to preserve the timestamp from the originator. By

Re: [rsyslog] strip of FDQN

Hello Adam, property replacer with use of regular expressions might help. https://www.rsyslog.com/doc/v8-stable/configuration/property_replacer.html Peter On Thu, Sep 13, 2018 at 12:30 PM Adam Barnett via rsyslog wrote: > > Hi, > > We are using rsyslog 8.24.0 > I am using templates of redirect

Re: [rsyslog] How to filter remote log on specific directory with rsyslog centralized server

Hello Jean-Marie, you can try to use exec_template [1] which was developed for such purposes. This can be a base for your configuration template(name="getFromhostip" type="string" string="%fromhost-ip:R,ERE,0,DFLT:([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})--end%") # do not forget the ';' character on

Re: [rsyslog] DA queue not dequeuing

3292.108043010:main thread: file stream N/A params: flush interval 0, async write 0 3292.108052306:main thread: file stream N/A params: flush interval 0, async write 0 Peter On Tue, Mar 5, 2019 at 3:05 PM Rainer Gerhards wrote: > > El mar., 5 mar. 2019 a las 15:00, Peter Visk

[rsyslog] DA queue mode without message variables

Within the debugging of the issues with DA queues not dequeuing, caused by already fixed bug [1], realized the DA queue consists of - standard syslog and input properties and also of localvars json array. [1] https://github.com/rsyslog/rsyslog/issues/1404 At first it is causing old versions of

Re: [rsyslog] DA queue not dequeuing

>> not. > >> > >> Rainer > >> > >> El mié., 6 mar. 2019 a las 10:26, Peter Viskup () > escribió: > >> > > >> > Following is complete log entry with 3 lines up and down: > >> > > >> > 3292.107997776:main

Re: [rsyslog] DA queue not dequeuing

main thread: file stream N/A params: flush interval > > 0, async write 0 > > > > Peter > > > > On Tue, Mar 5, 2019 at 3:05 PM Rainer Gerhards > wrote: > > > > > > El mar., 5 mar. 2019 a las 15:00, Peter Viskup via rsyslog > > > () es

Re: [rsyslog] DA queue not dequeuing

ver, state 0 > >> > > >> > Seems strange. Any thoughts? > >> > > >> > Peter > >> > > >> > > >> > On Wed, Mar 6, 2019 at 12:10 PM Rainer Gerhards < > rgerha...@hq.adiscon.com> wrote: > >> >> > >> >>

[rsyslog] DA queue not dequeuing

After rsyslog crash and recover.qi.pl run the DA queue is not dequeued. Rsyslog debug prints the message from queue.c file [1]. What is could be the reason for this? Only some servers are affected by this issue. Others dequeue just fine. [1]

[rsyslog] DA queue not dequeuing

After rsyslog crash and recover.qi.pl run the DA queue is not dequeued. Rsyslog debug prints the message from queue.c file [1]. What is could be the reason for this? Only some servers are affected by this issue. Others dequeue just fine. [1]

Re: [rsyslog] Syslog Output File Generation Frequency (HOURLY) at Syslog Server

Hello Sarjit, give it a try to have a look on time-related properties documented [1]. [1] https://www.rsyslog.com/doc/v8-stable/configuration/properties.html Peter On Tue, Mar 5, 2019 at 2:16 PM sarjit yadav via rsyslog wrote: > > Hi Experts, > > Any suggestion below query. > > On Thu, Feb 21,

Re: [rsyslog] Having issues with discard rule

You can also use RSYSLOG_DebugFormat template [1] to log into a file. You will be able to see what is the value of all properties. [1] https://www.rsyslog.com/doc/v8-stable/configuration/templates.html On Mon, Mar 11, 2019 at 10:00 PM Adam Chalkley wrote: > I'll defer to others more

Re: [rsyslog] Syslog Output File Generation Frequency (HOURLY) at Syslog Server

Copying logrotate to /etc/cron.daily is correct. Then logrotate will check the configuration files and rotate only those logs which should be rotated according the configuration. Value of maxage is in days and you should probably change the value to 3 to correspond with rotate value. Read the

Re: [rsyslog] rsyslog 5.8 and ssh issue

We have been facing the same issue. It is related to "full buffer" for /dev/log device, which is used by sudo, PAM, SSH and other services to log authentication messages. The "unavailability" is caused by SSH not able to write to /dev/log. The same issue might appear with use of any other syslog

Re: [rsyslog] rsyslog impstats disk-assisted queue size/enqueued counters

one of the reasons why you should really use the new syntax. It makes > it > much clearer what you are doing. > > David Lang > > On Tue, 5 Feb 2019, Peter Viskup via rsyslog wrote: > > > The load and configuration is done like this: > > > > $ModLoad impstats &

Re: [rsyslog] Rsyslog vs syslog-ng

To be honest, the main reason Debian chosen rsyslog as primary syslog daemon was that it does work with "standard syslog" configuration (more information can be read on https://wiki.debian.org/Rsyslog ). Nevertheless in newest versions of rsyslog you are always recommended to move to

Re: [rsyslog] rsyslog impstats disk-assisted queue size/enqueued counters

The load and configuration is done like this: $ModLoad impstats $PStatInterval 15 $PStatSeverity 7 Peter On Sun, Jan 20, 2019 at 5:09 PM Emmanuel Seyman wrote: > > * Alberto [20/01/2019 14:27] : > > > > How do you load the module? > > I use: > > module(load="impstats" >interval="86400"

[rsyslog] Status of imgssapi

Just looked for secured syslog transport in rsyslog other than TLS. Found the imgssapi module [1]. Does the module support 'advanced' format configuration? It is not mentioned in documentation. What is the experience from using this module? Does it perform well? [1]

Re: [rsyslog] imfile state file changes

The information with good explanation is available in the documentation [1]. [1] https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html#persiststateinterval Peter On Fri, Apr 12, 2019 at 2:29 PM John Chivian via rsyslog < rsyslog@lists.adiscon.com> wrote: > Hello Maintainers:

[rsyslog] call ruleset

>From reading the call documentation [1] I understand the call ruleset can be used to independent parallel message processing bypassing the standard queue-lanes behavior [2]. Is this my assumption correct? Want to come with configuration that will prevent unavailability of one destination to block

[rsyslog] Relaying queue design

Want to come with final design of two level relays for syslog flow: client -> relay11 -> -> dest1 client -> relay12 -> relay20 -> dest2 client -> relay13 -> -> dest3 Thought about the possibility to use mainQ in DA mode and omfwdQs (3 omfwd over TCP) as small in-memory or direct

Re: [rsyslog] imuxsock needs UseSpecialParser='off" to parse /var/run/log correctly on FreeBSD

Hello David, On Wed, Jun 5, 2019 at 7:08 PM David Lang via rsyslog < rsyslog@lists.adiscon.com> wrote: > I think I've seen this before and the problem is that the timestamp being > provided has too many digits after the . > > can you try to rig up a test where you send 3 digits after the .

Re: [rsyslog] Lookup table does not set variable

it entered failed state. Jun 14 13:27:26 HOST-LOCO systemd[1]: rsyslog-chroot@local.service: Failed with result 'timeout'. -- Peter On Fri, Jun 14, 2019 at 1:09 PM Rainer Gerhards wrote: > does this also happen with current 8.1905.0? > Rainer > > El vie., 14 jun. 2019 a las 12:2

[rsyslog] rsyslog 8.1904 not built with systemd in OBS repos

Tried to start rsyslog 8.1904 in chrooted environment, but got the systemd service timeout error. The sd_notify in rsyslog 8.1901 version from Debian repositories is working fine with just bind mounting host /run/systemd/notify into the chroot under the same path. The root cause seems to be the

[rsyslog] OBS repositories for Debian 10

When it is planned to make Debian 10 repositories on openSUSE build service? Debian 10 release is planned on 6.7.2019 and would be good to have some time to test it in advance. -- Peter ___ rsyslog mailing list

Re: [rsyslog] rsyslog 8.1904 not built with systemd in OBS repos

soon. Would be great if > you could check. > > Rainer > > El mar., 18 jun. 2019 a las 9:01, Peter Viskup via rsyslog > () escribió: > > > > Tried to start rsyslog 8.1904 in chrooted environment, but got the > systemd > > service timeout error. >

[rsyslog] rsyslog with TLS on Debian

What is the actual status of building rsyslog with TLS on Debian. Just remember there were some issues with ossl driver caused the Debian package cannot be built with it. Is this still the case? Should ossl driver be preferred? What is the quality of both ossl and gtls drivers in latest versions?

Re: [rsyslog] Lookup table does not set variable

> Jun 14 13:27:26 HOST-LOCO systemd[1]: rsyslog-chroot@local.service: Unit >> entered failed state. >> Jun 14 13:27:26 HOST-LOCO systemd[1]: rsyslog-chroot@local.service: >> Failed with result 'timeout'. >> >> -- >> Peter >> >> On Fri, Jun 14, 2019 at 1

Re: [rsyslog] Debian packages and what we can do better

On Thu, Jul 4, 2019 at 11:51 AM Rainer Gerhards wrote: > Hijacking the thread just slightly... > > El jue., 4 jul. 2019 a las 9:51, Peter Viskup via rsyslog > () escribió: > > > > > The use of package from backports is not always the best option as those > >

[rsyslog] UDP syslog load balancer healthcheck workaround

Want to share the ldirector_port_check script based on check_port.pl script [1] which can be used to perform the remote healthcheck for listen ports. The remote monitoring UDP listen ports is not possible. Ldirector use simple ping of remote host for UDP services, which is not sufficient. To let

Re: [rsyslog] UDP syslog load balancer healthcheck workaround

Small remark for ldirectord config. The UDP syslog service work much better with scheduler=sh (source hash) and quiescent=yes. That will let the LVS balance across real servers with source-ip going to the same destination if available. For UDP service the LVS does not route packets for unavailable

Re: [rsyslog] Debian packages and what we can do better

Hello Michael, at first, thank you for your work done. Propose rsyslog-ossl (OpenSSL driver for TLS encryption) being built and put into non-free if possible. Just to let people test or use it if they want. The libssl-dev is listed in BuildDepends list. Are there other parts of rsyslog which are

Re: [rsyslog] Debian packages and what we can do better

On Thu, Jul 4, 2019 at 1:35 PM Michael Biebl wrote: > Am Do., 4. Juli 2019 um 13:30 Uhr schrieb Peter Viskup via rsyslog > : > > The syslog infra is something which most of admins do not want to update > on > > daily basis. > > I think this is not something w

Re: [rsyslog] Rsyslog HA-style redirect

Yes it is. https://www.rsyslog.com/doc/master/tutorials/failover_syslog_server.html Peter On Fri, Aug 30, 2019 at 12:24 PM rsyslog--- via rsyslog < rsyslog@lists.adiscon.com> wrote: > Hello, > > When using TCP redirects (@@), is it possible to configure multiple > servers but only send to one

[rsyslog] Processing SQL audit messages

There are some application which write audit logs to SQL database only. Might be interesting to process them with rsyslog for the distribution to SIEM and/or archiving. Does anybody work on similar use case? Do you think input alternative of omlibdbi will make sense? -- Peter

Re: [rsyslog] funding specific features (was Re: Making sure I understand execOnlyWhenPreviousIsSuspended correctly, )

The list of open improvements waiting for funding might help. Can ask in our company about funding rsyslog project if some feature will be interesting for our deployment. Peter On Thu, Sep 5, 2019 at 9:39 PM David Lang via rsyslog < rsyslog@lists.adiscon.com> wrote: > On Thu, 5 Sep 2019, Rainer

[rsyslog] imtcp performance

Would like to know your experience with imtcp and/or imptcp. With +1100 established TCP connection we get ~100% CPU usage on imtcp thread causing the TCP stack/connections being stalled/not possible to establish. TOP screen: Threads: 295 total, 3 running, 292 sleeping, 0 stopped, 0 zombie

[rsyslog] Rsyslog regex test page not working

The page https://www.rsyslog.com/regex/ does not show Regexp results. Please check. -- Peter ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow

Re: [rsyslog] local variables not printed in debug format

FYI Found the problem. The listener used local ruleset, while the variables resided in RSYSLOG_DefaultRuleset. Moving the omfile action out of ruleset definition resolved the issue. Reading sentence "As such, any modifications made to the message object (e.g. message or local variables that are

[rsyslog] local variables not printed in debug format

Running rsyslog 8.1901 on fresh Debian10 the $.localvars are not printed in debug format. Starting rsyslog by /usr/sbin/rsyslogd -d -n -f /etc/rsyslog-2/rsyslog-lin.conf Config files are processed without any error. The message looks like this: Debug line with all properties: FROMHOST:

[rsyslog] set myhostname property value into local variable

Configuration with lines: # set local variables set $.localip = "1.1.1.1"; set $.host = $myhostname; seems not be working. Getting these errors. Jul 23 14:25:41 HOST-LOCO rsyslogd[6024]: rsyslogd: error during parsing file /etc/rsyslog.d/global/09-variables.conf, on or before line 3: invalid

[rsyslog] rainerscript control structures

Want to be sure the following configurations are the same if $hostname contains "text" then { action(type="omfwd" .) } and without curly brackets if $hostname contains "text" then action(type="omfwd" ..) The first option with brackets has to be used in case of more actions

Re: [rsyslog] rainerscript control structures

; queue.LowWaterMark="40" template="relay2ForwardTemplate" ) Creating 110-fwd-filter.conf file with simple 'if property' check make the filter and action work as expected. On Thu, Sep 19, 2019 at 4:41 PM Илья Рассадин via rsyslog < rsyslog@lists.adiscon.com> wrote: &g

[rsyslog] replace carriage return

What should be the best way to handle carriage return character on the end of message? Without setting the $EscapeControlCharactersOnReceive to off, the messages end with #015 and are also forwarded that way. With setting $EscapeControlCharactersOnReceive to off the messages are forwarded with \r

[rsyslog] Lookup table does not set variable

Running rsyslog 8.24 on Debian9. The lookup table ~# cat /etc/rsyslog.d/local/programnames.lookup { "version" : 1, "nomatch" : "local-all", "type" : "string", "table" : [ {"index" : "apache_site_access", "value" : "apache-site-access" }, {"index" : "apache_site_error", "value" :

[rsyslog] string match filter 'contains' vs. '=='

Running rsyslog 8.1901.0-1 and it seems there is some difference in processing these two filters. On the input there is message which is parsed with hostname property set to the IP address exactly. The match with use of 'contains' is not effective, while '==' is. Is this expected result? Message

Re: [rsyslog] Hostname resolution updates (remote logging) not picked up

We had a little discussion about TCP reopening (which might include name resolution) in following bug report (Reopen TCP sockets on HUP signal). https://github.com/rsyslog/rsyslog/issues/3683 The outcome is to use rebindinterval omfwd config option which makes the same, but cannot be enforced by

[rsyslog] MainQ workerthreads not effective

Experiencing high load on some rsyslog instances. Status of threads showed the mainQ thread consumed 50-100% CPU. Change of queue.workerthreads to 2 enabled the second workerthread, but this does not consume any CPU. How are the workerthreads for main queue loaded? Running on Debian 10 with

Re: [rsyslog] MainQ workerthreads not effective

to know what's going on without seeing your config. > > David Lang > > On Thu, 28 Nov 2019, Peter Viskup via rsyslog wrote: > > > Date: Thu, 28 Nov 2019 09:22:43 +0100 > > From: Peter Viskup via rsyslog > > To: rsyslog-users > > Cc: Peter Visk

Re: [rsyslog] Issue with Disk Assisted queues

Hi Malhar, try to enable impstats [1] which will provide you the evidence of the rsyslog runtime statistics and queue sizes. Also read about the rsyslog queues [2][3] a little. That might help you to understand the queuing in rsyslog. [1] https://www.rsyslog.com/how-to-use-impstats/ [2]

[rsyslog] imptcp maximum TCP sessions

What is the limit of TCP sessions the imptcp can handle? There is no option like MaxSessions of imtcp. Was not able to find the information in documentation. Discovered code which might point to that limit, but do not understand it.

Re: [rsyslog] rsyslog's programname

[Replying with mailing list address in recipients.] Thank you, Rainer, for quick answer. On Wed, Feb 12, 2020 at 3:31 PM Rainer Gerhards wrote: > El mié., 12 feb. 2020 a las 15:26, Peter Viskup via rsyslog > () escribió: > > > > In other case it seems those interna

[rsyslog] rsyslog's programname

Is there way to configure rsyslog instance to use its own programname? For example rsyslog-net or rsyslog-lin for appropriate instances which have different listen ports open. As those usually run on the same host, the error messages are logged under "rsyslog" and it is hard to decide what message

[rsyslog] Debugging rsyslog segfault

Experience regular segfaults on one rsyslog 8.15 instance. I know it is old version, but still would like to trace it as am not able to upgrade ATM. Seems it is caused by writing some message to DA cache (or by reading it from). Would it be possible to find it in debug log (already got it)? What

Re: [rsyslog] Hostname field in 5424 header Parsing

Hi Harish, good for reading and understanding https://en.wikipedia.org/wiki/Hostname https://tools.ietf.org/html/rfc5424#section-6.2.4 https://tools.ietf.org/html/rfc3164#section-4.1.2 On Tue, Jan 28, 2020 at 9:01 AM Harish Patil via rsyslog < rsyslog@lists.adiscon.com> wrote: > Ok, thanks for

[rsyslog] recovery.qi.pl update

Let me share the patch for recovery.qi.pl script with you. It does automatically create $basename.qi file (no STDOUT redirection required) and initiate $digits and $spool with defaults (they are optional). One of other improvement is the queue files are reordered when broken queue is detected. In

[rsyslog] omfile thread terminated too quick

For some weeks there are a lot of closing logfile notification via inotify seen on one syslog relay running rsyslog 8.1901 version. The messages like these May 4 15:10:04 fwd01 iWatch[31831]: * /chroot/local/var/log/h1/local-all.log is closed May 4 15:10:04 fwd01 iWatch[31831]: *

Re: [rsyslog] omfile thread terminated too quick

Reported bug for Debian package https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=959774 Following is the evidence of the rotated thread PIDs: root@fwd01:~# date; pstree -t -sap 9276 Tue 05 May 2020 08:05:27 AM UTC systemd,1 └─rsyslogd-local,9276 -n -f /etc/rsyslog.d/rsyslog-local.conf

  1   2   >