Re: [rsyslog] rainerscript control structures
Is there any sense to not use brackets always?
On 19/09/2019 17:36, Peter Viskup via rsyslog wrote:
Want to be sure the following configurations are the same
if $hostname contains "text" then {
action(type="omfwd" .)
}
and without curly brackets
if $hostname contains "text" then
action(type="omfwd" ..)
The first option with brackets has to be used in case of more actions
following the filter.
There is only one action following the filter. Is this my
assumption correct?
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Hardening an (GnuTLS/anon) rsyslog server
Probably you can use something like fail2ban. It requires from rsyslog write not only error messages but client ip address too. I'm not sure that it's possible now with openssl or gnutls modules/ On 17/04/2019 16:59, Alan Martinovic via rsyslog wrote: It seems WAF[1] (Web Application Firewall) was the concept I was looking for. Seems like the safest option would be to set up a reverse proxy in front of rsyslog with something like ModSecurity[2]. In the conceptual scenario, the job of the WAF would be to: 1. unencrypt incoming connection (that would have to be taken away from rsyslog) 2. drop connections that don't follow certain rules. Perhaps a regex of the log content 3. pass the (now unencrypted connections) connections to rsyslog This is just a conceptual scenario, not clear on the feasibility. It seems to save rsyslog from connections that aren't legit logs. I might be missing something essential complexity which this reverse proxy might introduce, like would queuing be affected by the middle man... etc. Comments are welcome :) On Wed, Apr 17, 2019 at 3:15 PM Alan Martinovic wrote: Hey Rainer, thanks for the feedback. The IP layer filtering isn't applicable in my case. Don't know what IPs the clients might end up having. On Wed, Apr 17, 2019 at 2:50 PM Rainer Gerhards wrote: If you expose the host to the Internet, you should at least install iptables or similar solution. There is some access control directly in rsyslog, but using ip layer firewall is much more robust (by design). Rainer El mié., 17 abr. 2019 a las 14:14, Alan Martinovic via rsyslog () escribió: Hey, I have a rsyslog server which will accept everything that want's to log TLS encrypted data to it. (Server - anon, Client - x509/name) It turned out the Internet is much more interested in spamming my logging server then I thought when doing the implementation. So now I'm getting a lot of: ``` gnutls returned error on handshake: An unexpected TLS packet was received. unexpected GnuTLS error -110 in nsdsel_gtls.c:178: The TLS connection was non-properly terminated. unexpected GnuTLS error -15 in nsdsel_gtls.c:178: An unexpected TLS packet was received. gnutls returned error on handshake: Error in the pull function. ``` At some point I couldn't send any more logs before restarting rsyslog. The service was still running and there were no exceptional logs to relate to that, besides the upper ones which occur in working conditions also. Even if I introduce client authentication on the server side, that wouldn't help much against bad TLS packets from unexpected clients. Anyways, would like to hear your thoughts on how to harden an anon server. Is it possible to drop connections by log content? Or perhaps install some kind of an application layer firewall to protect rsyslog? Be Well, Alan ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] How to change field after mmnormalize parsing
For the record, my final working config to parse php-fpm slowlog into
gelf with rsyslog.
input(
type="imfile"
file="/var/www/*/logs/*log.slow"
startmsg.regex="^$"
tag="php-fpm"
readTimeout="10"
ruleset="php-fpm-slow"
addMetadata="off"
)
template(name="gelf-ext" type="list") {
constant(value="{\"version\":\"1.1\",")
constant(value="\"host\":\"") property(name="hostname")
constant(value="\",\"short_message\":\"") property(name="msg"
format="json")
constant(value="\",\"timestamp\":") property(name="timegenerated"
dateformat="unixtimestamp")
constant(value=",\"_application_name \":\"") property(name="app-name")
constant(value="\",\"level\":\"") property(name="syslogseverity")
constant(value="\",") property(name="$!all-json" position.from="2")
}
ruleset(name="php-fpm-slow") {
action(type="mmnormalize"
rulebase="/etc/rsyslog.d/rules/php_fpm_slow.rb")
set $!full_message = replace($!full_message, '\\n', "\n");
action(
type="omfwd"
Target="graylog"
Port="12201"
Protocol="tcp"
KeepAlive="on"
template="gelf-ext"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="graylog"
TCP_FrameDelimiter="0"
)
}
rulebase file
version=2
rule=:\\n[%_date:char-to:]%] [pool %_pool_name:char-to:]%] pid
%-:number%\\nscript_filename =
%_script_filename:string-to{"extradata":"\\n"}%\\n%full_message:rest%
It can be improved further:
1. clean up short message in template
2. set up timestamp from php-fpm slowlog
Many Thanks to David for helping.
On 04/01/2019 03:12, David Lang wrote:
No, mmnormalize doesn't support multi-line rules
David Lang
On Fri, 4 Jan 2019, Илья Рассадин via rsyslog wrote:
Just curious, is there a way to use mmnormalize and multiline imfile
with switched off escapeLF option directly?
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] How to change field after mmnormalize parsing
Thanks, it helped. Just curious, is there a way to use mmnormalize and multiline imfile with switched off escapeLF option directly? On 02/01/2019 15:21, David Lang wrote: look at the set and unset functions, they let you change any variable. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] How to change field after mmnormalize parsing
Hi!
I'm using rsyslog to create gelf messages and send them to graylog server.
PHP-FPM Slow Logs has special multiline format.
[02-Jan-2019 14:04:28] [pool kosmetika_proff_ru] pid 18139
script_filename = /path/to/slow/script.php
[0x7fe5ae417430] function1() /path/to/file1.php:301
[0x7fe5ae417380] function2() /path/to/file2.php:160
[0x7fe5ae4171f0] function3() /path/to/file3.php:164
So new line, two lines of diagnostic information and stacktrace.
I'm using this rulebase to parse it
version=2
rule=php-fpm,info,gelf:\\n[%date:char-to:]%] [pool
%pool_name:char-to:]%] pid %-:number%\\nscript_filename =
%script_filename:string-to{"extradata":"\\n"}%\\n%full_message:rest%
And this config to forward messages to graylog
template(name="gelf-ext" type="list") {
constant(value="{\"version\":\"1.1\",")
constant(value="\"host\":\"") property(name="hostname")
constant(value="\",\"short_message\":\"") property(name="msg"
format="json")
constant(value="\",\"timestamp\":\"") property(name="timegenerated"
dateformat="unixtimestamp")
constant(value="\",\"_application_name \":\"") property(name="app-name")
constant(value="\",\"level\":\"") property(name="syslogseverity")
constant(value="\",") property(name="$!all-json" position.from="2")
}
ruleset(name="graylog-ext") {
action(
type="omfwd"
Target="graylog"
Port="12201"
Protocol="tcp"
KeepAlive="on"
template="gelf-ext"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="graylog"
TCP_FrameDelimiter="0"
)
}
input(
type="imfile"
file="/var/www/*/logs/*log.slow"
startmsg.regex="^$"
tag="php-fpm"
readTimeout="10"
ruleset="php-fpm-slow"
)
ruleset(name="php-fpm-slow") {
action(type="mmnormalize" rulebase="/path/to/php_fpm_slow.rb")
call graylog-ext
}
Everything works fine, messages are forwarded to graylog, indexed to
elasticsearch etc, except one thing.
Stacktrace is one huge line with \\n symbol between lines.
Is there a way to alter full_message field in $!all-json variable before
send gelf message to graylog?
--
Best Regards, Ilya Rassadin.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
