Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
On Sun, 11 Feb 2018, Nicholas von Waltsleben wrote: The only question I then have is why does it log that the action has been suspended, if enqueuing the message was the action? enqueuing the message is what happens just before the "previous is suspended" check is made. At a later time, another worker thread dequeues the message from the action queue and tries to deliver it and that action can be suspended for normal reasons. But the fact that it was suspended cannot propogate back in time to become a deciding factor on anything that takes place at the time the message is enqueued. When a message is added to a queue, it's duplicated and one duplicate goes into the queue. From that point on, nothing that happens to that message can send any signals (other than global variables) out to the prior context. David Lang k ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
On 12 February 2018 at 14:56:26, Rainer Gerhards (rgerha...@hq.adiscon.com) wrote: So you want to go it to the remote end *and* the file if the remote is unavailable? Rainer Sent from phone, thus brief. Correct, the final action is just a script that sends an SNMP trap to a monitoring system and doesn’t actually do anything with the log message, hence the “empty” template used for that action. - Nic ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
So you want to go it to the remote end *and* the file if the remote is unavailable? Rainer Sent from phone, thus brief. Am 12.02.2018 12:55 schrieb "Nicholas von Waltsleben via rsyslog" < rsyslog@lists.adiscon.com>: > On 09 February 2018 at 13:58:43, Rainer Gerhards (rgerha...@hq.adiscon.com > ) > wrote: > > 2018-02-09 5:58 GMT+01:00 Nicholas von Waltsleben via rsyslog > : > > Upon further investigation it appears this only happens when a queue is > > configured for the action. It sounds like a bug that was previously fixed > > after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there > > perhaps been a regression in this area? > > Thanks, I need to investigate, but somehow this does not look wrong. > For a quick (and better!) fix, I suggest to put the queue on the rule > set instead of the individual actions. That will also be faster. > > Rainer > > > Hi Rainer > > Thanks for the suggestion. If this queue was only there for performance > that would have been the perfect solution. For my use case I actually need > to store messages intended for a remote syslog server and in the event the > server is offline trigger the next action which is a script that sends an > alert message to a monitoring system. So when I create the queue on the > ruleset, the improg action gets executed as expected when the omfwd action > is suspended, but the messages are (for obvious reasons) no longer kept in > the queue. > > Is what I am trying to do an achievable use-case for rsyslog, or should I > consider some kind of external script for monitoring the health of the > upstream syslog server? > > - Nic > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
On 09 February 2018 at 13:58:43, Rainer Gerhards (rgerha...@hq.adiscon.com) wrote: 2018-02-09 5:58 GMT+01:00 Nicholas von Waltsleben via rsyslog : > Upon further investigation it appears this only happens when a queue is > configured for the action. It sounds like a bug that was previously fixed > after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there > perhaps been a regression in this area? Thanks, I need to investigate, but somehow this does not look wrong. For a quick (and better!) fix, I suggest to put the queue on the rule set instead of the individual actions. That will also be faster. Rainer Hi Rainer Thanks for the suggestion. If this queue was only there for performance that would have been the perfect solution. For my use case I actually need to store messages intended for a remote syslog server and in the event the server is offline trigger the next action which is a script that sends an alert message to a monitoring system. So when I create the queue on the ruleset, the improg action gets executed as expected when the omfwd action is suspended, but the messages are (for obvious reasons) no longer kept in the queue. Is what I am trying to do an achievable use-case for rsyslog, or should I consider some kind of external script for monitoring the health of the upstream syslog server? - Nic ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
The only question I then have is why does it log that the action has been suspended, if enqueuing the message was the action? - Nic On 11 February 2018 at 23:29:29, David Lang (da...@lang.hm) wrote: On Thu, 8 Feb 2018, Nicholas von Waltsleben via rsyslog wrote: > Upon further investigation it appears this only happens when a queue is > configured for the action. It sounds like a bug that was previously fixed > after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there > perhaps been a regression in this area? the 'action' is to put the log into the queue for the action. unless the queue is full, putting the log into the queue succeeds, even if the action that will later pull from the queue is blocked. adding a queue to and action means that you have no idea what the status of those actions are. we should probably add something to the documentation for this option to indicate that it doesn't work if the prior action has a queue. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
On Thu, 8 Feb 2018, Nicholas von Waltsleben via rsyslog wrote: Upon further investigation it appears this only happens when a queue is configured for the action. It sounds like a bug that was previously fixed after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there perhaps been a regression in this area? the 'action' is to put the log into the queue for the action. unless the queue is full, putting the log into the queue succeeds, even if the action that will later pull from the queue is blocked. adding a queue to and action means that you have no idea what the status of those actions are. we should probably add something to the documentation for this option to indicate that it doesn't work if the prior action has a queue. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
2018-02-09 5:58 GMT+01:00 Nicholas von Waltsleben via rsyslog
:
> Upon further investigation it appears this only happens when a queue is
> configured for the action. It sounds like a bug that was previously fixed
> after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there
> perhaps been a regression in this area?
Thanks, I need to investigate, but somehow this does not look wrong.
For a quick (and better!) fix, I suggest to put the queue on the rule
set instead of the individual actions. That will also be faster.
Rainer
>
> Regards,
> Nic
>
>
> On 08 February 2018 at 10:20:27, Nicholas von Waltsleben (
> nicholas.vonwaltsle...@voss-solutions.com) wrote:
>
>
> Hi
>
> I seem to be running into an issue with rsyslog 8.32.0 running on Ubuntu
> 14.04. I have actions configured to fire when the previous action is
> suspended, and was working fine with the version included with Ubuntu
> (7.4.4). After upgrading to 8.32.0 via the PPA this no longer works, the
> action.execOnlyWhenPreviousIsSuspended parameter no longer appears to be
> triggering. With the configuration snippet I included below, I would
> expect the actions named “penny” and “dreadful” to become suspended (due to
> the remote servers not existing) but only the first action is triggered
> despite it becoming suspended. Am I doing something wrong?
>
> Config extract:
>
> module(load="omprog")
>
> $template event_json,"%msg:2:$%\n"
> $template empty,""
>
> ruleset(name="sendEventLogs") {
> action(
> name="penny"
> type="omfwd"
> target=“172.29.21.42"
> port="10514"
> StreamDriver="gtls"
> StreamDriverMode="1"
> StreamDriverAuthMode="x509/name"
> StreamDriverPermittedPeers=“172.29.21.42”
> protocol="tcp"
> queue.type="LinkedList"
> queue.filename="event_172.29.21.42"
> queue.saveonshutdown="on"
> queue.maxdiskspace="1G"
> queue.timeoutenqueue="0"
> # action.resumeRetryCount="-1"
> action.reportSuspension="on"
> action.reportSuspensionContinuation="on"
> template="event_json"
> )
> action(
> name="dreadful"
> type="omfwd"
> target="host2.example.com"
> port="10514"
> StreamDriver="gtls"
> StreamDriverMode="1"
> StreamDriverAuthMode="x509/name"
> StreamDriverPermittedPeers="host2.example.com"
> protocol="tcp"
> queue.type="LinkedList"
> queue.filename=“event_host2.example.com"
> queue.saveonshutdown="on"
> queue.maxdiskspace="1G"
> queue.timeoutenqueue="0"
> # action.resumeRetryCount="-1"
> action.reportSuspension="on"
> action.reportSuspensionContinuation="on"
> action.execOnlyWhenPreviousIsSuspended="on"
> template="event_json"
> )
> action(
> name="jessica"
> type="omprog"
> binary="/opt/platform/apps/services/audit_failure.sh"
> template="empty"
> action.reportSuspension="on"
> action.execOnlyWhenPreviousIsSuspended="on"
> action.execOnlyOnceEveryInterval="600"
> )
> }
>
> if $programname contains 'event' then {
> call sendEventLogs
> stop
> }
>
> Log messages
>
> 2018-02-07T12:46:17.448360+00:00 NICVW-STANDALONE-172 rsyslogd: [origin
> software="rsyslogd" swVersion="8.32.0" x-pid="236951" x-info="
> http://www.rsyslog.com";] start
> 2018-02-07T12:48:24.015625+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
> connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
> http://www.rsyslog.com/e/2027 ]
> 2018-02-07T12:48:24.015633+00:00 NICVW-STANDALONE-172 rsyslogd: action
> 'penny' suspended (module 'builtin:omfwd'), retry 0. There should be
> messages before this one giving the reason for suspension. [v8.32.0 try
> http://www.rsyslog.com/e/2007 ]
> 2018-02-07T12:50:31.246040+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
> connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
> http://www.rsyslog.com/e/2027 ]
> 2018-02-07T12:50:31.246262+00:00 NICVW-STANDALONE-172 rsyslogd: action
> 'penny' suspended (module 'builtin:omfwd'), next retry is Wed Feb 7
> 12:51:01 2018, retry nbr 0. There should be messages before this one giving
> the reason for suspension. [v8.32.0 try http://www.rsyslog.com/e/2007 ]
> 2018-02-07T12:52:38.478031+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
> connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
> http://www.rsyslog.com/e/2027 ]
> 2018-02-07T12:52:38.478365+00:00 NICVW-STANDALONE-172 rsyslogd: action
> 'penny' suspended (module 'builtin:omfwd'), retry 1. There should be
> messages before this one giving the reason for suspension. [v8.32.0 try
> http://www.rsyslog.com/e/2007 ]
> 2018-02-07T12:54:45.710029+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
> connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
> http://www.rsyslog.com/e/2027 ]
> 2018-02-07T12:54:45.710363+00:00 NICVW-STANDALONE-172
Re: [rsyslog] action.execOnlyWhenPreviousIsSuspended not working as expected
Upon further investigation it appears this only happens when a queue is
configured for the action. It sounds like a bug that was previously fixed
after 8.10 (https://github.com/rsyslog/rsyslog/issues/375), has there
perhaps been a regression in this area?
Regards,
Nic
On 08 February 2018 at 10:20:27, Nicholas von Waltsleben (
nicholas.vonwaltsle...@voss-solutions.com) wrote:
Hi
I seem to be running into an issue with rsyslog 8.32.0 running on Ubuntu
14.04. I have actions configured to fire when the previous action is
suspended, and was working fine with the version included with Ubuntu
(7.4.4). After upgrading to 8.32.0 via the PPA this no longer works, the
action.execOnlyWhenPreviousIsSuspended parameter no longer appears to be
triggering. With the configuration snippet I included below, I would
expect the actions named “penny” and “dreadful” to become suspended (due to
the remote servers not existing) but only the first action is triggered
despite it becoming suspended. Am I doing something wrong?
Config extract:
module(load="omprog")
$template event_json,"%msg:2:$%\n"
$template empty,""
ruleset(name="sendEventLogs") {
action(
name="penny"
type="omfwd"
target=“172.29.21.42"
port="10514"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers=“172.29.21.42”
protocol="tcp"
queue.type="LinkedList"
queue.filename="event_172.29.21.42"
queue.saveonshutdown="on"
queue.maxdiskspace="1G"
queue.timeoutenqueue="0"
# action.resumeRetryCount="-1"
action.reportSuspension="on"
action.reportSuspensionContinuation="on"
template="event_json"
)
action(
name="dreadful"
type="omfwd"
target="host2.example.com"
port="10514"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="host2.example.com"
protocol="tcp"
queue.type="LinkedList"
queue.filename=“event_host2.example.com"
queue.saveonshutdown="on"
queue.maxdiskspace="1G"
queue.timeoutenqueue="0"
# action.resumeRetryCount="-1"
action.reportSuspension="on"
action.reportSuspensionContinuation="on"
action.execOnlyWhenPreviousIsSuspended="on"
template="event_json"
)
action(
name="jessica"
type="omprog"
binary="/opt/platform/apps/services/audit_failure.sh"
template="empty"
action.reportSuspension="on"
action.execOnlyWhenPreviousIsSuspended="on"
action.execOnlyOnceEveryInterval="600"
)
}
if $programname contains 'event' then {
call sendEventLogs
stop
}
Log messages
2018-02-07T12:46:17.448360+00:00 NICVW-STANDALONE-172 rsyslogd: [origin
software="rsyslogd" swVersion="8.32.0" x-pid="236951" x-info="
http://www.rsyslog.com";] start
2018-02-07T12:48:24.015625+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
http://www.rsyslog.com/e/2027 ]
2018-02-07T12:48:24.015633+00:00 NICVW-STANDALONE-172 rsyslogd: action
'penny' suspended (module 'builtin:omfwd'), retry 0. There should be
messages before this one giving the reason for suspension. [v8.32.0 try
http://www.rsyslog.com/e/2007 ]
2018-02-07T12:50:31.246040+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
http://www.rsyslog.com/e/2027 ]
2018-02-07T12:50:31.246262+00:00 NICVW-STANDALONE-172 rsyslogd: action
'penny' suspended (module 'builtin:omfwd'), next retry is Wed Feb 7
12:51:01 2018, retry nbr 0. There should be messages before this one giving
the reason for suspension. [v8.32.0 try http://www.rsyslog.com/e/2007 ]
2018-02-07T12:52:38.478031+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
http://www.rsyslog.com/e/2027 ]
2018-02-07T12:52:38.478365+00:00 NICVW-STANDALONE-172 rsyslogd: action
'penny' suspended (module 'builtin:omfwd'), retry 1. There should be
messages before this one giving the reason for suspension. [v8.32.0 try
http://www.rsyslog.com/e/2007 ]
2018-02-07T12:54:45.710029+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
http://www.rsyslog.com/e/2027 ]
2018-02-07T12:54:45.710363+00:00 NICVW-STANDALONE-172 rsyslogd: action
'penny' suspended (module 'builtin:omfwd'), next retry is Wed Feb 7
12:55:15 2018, retry nbr 1. There should be messages before this one giving
the reason for suspension. [v8.32.0 try http://www.rsyslog.com/e/2007 ]
2018-02-07T12:57:23.278087+00:00 NICVW-STANDALONE-172 rsyslogd: cannot
connect to 172.29.21.42:10514: Connection timed out [v8.32.0 try
http://www.rsyslog.com/e/2027 ]
2018-02-07T12:57:23.278397+00:00 NICVW-STANDALONE-172 rsyslogd: action
'penny' suspended (module 'buil
