More progress on this: When i move all rsyslog configuration into a single file all problems (mentioned in this thread) are gone! It seems that splitting functionality in seperate files is not working 100% as expected.
Is this a known limitation/issue? Regards, Jörgen On Wed, Nov 11, 2015 at 2:00 PM, Jörgen Maas <[email protected]> wrote: > Hi all, > > I've verified the situation with the latest 8.14 release and the official > packages, and i can confirm that the issue is still present. Including the > segmentationfault when using rsyslogd -N3. I'll file a bug report on GitHub. > > Cheers, > Jörgen > > > > On Fri, Nov 6, 2015 at 3:51 PM, Radu Gheorghe <[email protected]> > wrote: > >> Hi Jörgen, >> >> Yes, this is really weird. Can you come up with a complete >> reproduction that you'd paste in a GitHub issue? >> >> Best regards, >> Radu >> -- >> Performance Monitoring * Log Analytics * Search Analytics >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> On Fri, Nov 6, 2015 at 1:43 PM, Jörgen Maas <[email protected]> >> wrote: >> > Hi Radu, >> > >> > First with only the new syntax: >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# cat forward_syslog_tls.conf >> > # >> > # forward - syslog / tcp+tls >> > # >> > >> > >> > # testing shows that we need both the legacy and new style options >> > # when deleting options errors messages and even rsyslogd crashes occur >> > >> > #$ActionSendStreamDriver gtls >> > #$ActionSendStreamDriverMode 1 >> > #$ActionSendStreamDriverAuthMode x509/name >> > #$ActionSendStreamDriverPermittedPeers "logmanagement.xxx.yy" >> > >> > action( >> > type="omfwd" >> > target="192.168.124.100" >> > port="6514" >> > protocol="tcp" >> > template="RSYSLOG_SyslogProtocol23Format" >> > StreamDriver="gtls" >> > StreamDriverMode="1" >> > StreamDriverAuthMode="x509/name" >> > StreamDriverPermittedPeers="logmanagement.xx.yy" >> > ) >> > >> > # EOF >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# rsyslogd -N3 >> > rsyslogd: version 7.4.7, config validation run (level 3), master config >> > /etc/rsyslog.conf >> > Segmentation fault >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# systemctl restart rsyslog >> > Job for rsyslog.service failed. See 'systemctl status rsyslog.service' >> and >> > 'journalctl -xn' for details. >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# tail /var/log/syslog |grep >> > rsyslogd >> > Nov 6 12:30:52 logmanagement-client kernel: traps: rsyslogd[2192] >> general >> > protection ip:7fdab462c4bd sp:7ffd52d30a50 error:0 in libc-2.17.so >> > [7fdab45b0000+1b6000] >> > Nov 6 12:31:11 logmanagement-client rsyslogd: [origin >> software="rsyslogd" >> > swVersion="7.4.7" x-pid="577" x-info="http://www.rsyslog.com";] exiting >> on >> > signal 15. >> > >> > >> > >> > Now with only the old syntax: >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# cat forward_syslog_tls.conf >> > # >> > # forward - syslog / tcp+tls >> > # >> > >> > >> > # testing shows that we need both the legacy and new style options >> > # when deleting options errors messages and even rsyslogd crashes occur >> > >> > $ActionSendStreamDriver gtls >> > $ActionSendStreamDriverMode 1 >> > $ActionSendStreamDriverAuthMode x509/name >> > $ActionSendStreamDriverPermittedPeers "logmanagement.xxx.yy" >> > >> > action( >> > type="omfwd" >> > target="192.168.124.100" >> > port="6514" >> > protocol="tcp" >> > template="RSYSLOG_SyslogProtocol23Format" >> > #StreamDriver="gtls" >> > #StreamDriverMode="1" >> > #StreamDriverAuthMode="x509/name" >> > #StreamDriverPermittedPeers="logmanagement.xxx.yy" >> > ) >> > >> > # EOF >> > [ >> > root@logmanagement-client:/etc/rsyslog.d]# rsyslogd -N3 >> > rsyslogd: version 7.4.7, config validation run (level 3), master config >> > /etc/rsyslog.conf >> > rsyslogd: invalid or yet-unknown config file command >> > 'ActionSendStreamDriverPermittedPeers' - have you forgotten to load a >> > module? [try http://www.rsyslog.com/e/3003 ] >> > rsyslogd: End of config validation run. Bye. >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# systemctl restart rsyslog >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# tail /var/log/syslog |grep >> > rsyslogd >> > Nov 6 12:36:30 logmanagement-client rsyslogd: [origin >> software="rsyslogd" >> > swVersion="7.4.7" x-pid="2306" x-info="http://www.rsyslog.com";] start >> > Nov 6 12:36:30 logmanagement-client rsyslogd-3003: invalid or >> yet-unknown >> > config file command 'ActionSendStreamDriverPermittedPeers' - have you >> > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] >> > Nov 6 12:36:30 logmanagement-client rsyslogd-2088: error: peer name not >> > authorized - not permitted to talk to it. Names: CN: >> logmanagement.xxx.yy >> > [try http://www.rsyslog.com/e/2088 ] >> > Nov 6 12:36:30 logmanagement-client rsyslogd-2088: error: peer name not >> > authorized - not permitted to talk to it. Names: CN: >> > logmanagement.xxx.yy; [try http://www.rsyslog.com/e/2088 ] >> > >> > >> > With both old and new: >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# cat forward_syslog_tls.conf >> > # >> > # forward - syslog / tcp+tls >> > # >> > >> > >> > # testing shows that we need both the legacy and new style options >> > # when deleting options errors messages and even rsyslogd crashes occur >> > >> > $ActionSendStreamDriver gtls >> > $ActionSendStreamDriverMode 1 >> > $ActionSendStreamDriverAuthMode x509/name >> > $ActionSendStreamDriverPermittedPeers "logmanagement.xxx.yy" >> > >> > action( >> > type="omfwd" >> > target="192.168.124.100" >> > port="6514" >> > protocol="tcp" >> > template="RSYSLOG_SyslogProtocol23Format" >> > StreamDriver="gtls" >> > StreamDriverMode="1" >> > StreamDriverAuthMode="x509/name" >> > StreamDriverPermittedPeers="logmanagement.xxx.yy" >> > ) >> > >> > # EOF >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# rsyslogd -N3 >> > rsyslogd: version 7.4.7, config validation run (level 3), master config >> > /etc/rsyslog.conf >> > rsyslogd: invalid or yet-unknown config file command >> > 'ActionSendStreamDriverPermittedPeers' - have you forgotten to load a >> > module? [try http://www.rsyslog.com/e/3003 ] >> > rsyslogd: End of config validation run. Bye. >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# systemctl restart rsyslog >> > >> > [root@logmanagement-client:/etc/rsyslog.d]# tail /var/log/syslog >> > Nov 6 12:39:04 logmanagement-client rsyslogd: [origin >> software="rsyslogd" >> > swVersion="7.4.7" x-pid="2328" x-info="http://www.rsyslog.com";] start >> > Nov 6 12:39:04 logmanagement-client rsyslogd-3003: invalid or >> yet-unknown >> > config file command 'ActionSendStreamDriverPermittedPeers' - have you >> > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] >> > >> > And this configuration actually does work. >> > >> > For sure i'm hitting some bugs here ;) >> > >> > Cheers, >> > Jörgen >> > >> > >> > On Fri, Nov 6, 2015 at 9:37 AM, Radu Gheorghe < >> [email protected]> >> > wrote: >> > >> >> Hello Jörgen, >> >> >> >> So if you "translate" the $Action... directives into RainerScript it >> >> doesn't work at all? And you also don't get any configuration errors? >> >> Then it would be a bug. >> >> >> >> Best regards, >> >> Radu >> >> -- >> >> Performance Monitoring * Log Analytics * Search Analytics >> >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> >> >> >> On Thu, Nov 5, 2015 at 2:22 PM, Jörgen Maas <[email protected]> >> wrote: >> >> > Hi, thanks Radu for your feedback! >> >> > >> >> > On the client and the server all rsyslog and gnutls versions are the >> >> same. >> >> > I did recreate the certs with openssl, instead of the certutil as >> >> described >> >> > in the docs. >> >> > I now have this working, it's just like yesterdays issue a case of >> using >> >> > new and old configuration *together* to make it work... >> >> > >> >> > But doing this seems to also cause some strange issues... i can >> imagine >> >> > that this isn't really a well tested configuration (mixing old and >> new). >> >> > Current config on the sender: >> >> > >> >> > $ActionSendStreamDriver gtls >> >> > $ActionSendStreamDriverMode 1 >> >> > $ActionSendStreamDriverAuthMode x509/name >> >> > $ActionSendStreamDriverPermittedPeers "logmanagement.xxx.yy" >> >> > >> >> > action( >> >> > type="omfwd" >> >> > target="192.168.124.100" >> >> > port="6514" >> >> > protocol="tcp" >> >> > template="RSYSLOG_SyslogProtocol23Format" >> >> > StreamDriver="gtls" >> >> > StreamDriverMode="1" >> >> > StreamDriverAuthMode="x509/name" >> >> > StreamDriverPermittedPeers="logmanagement.xxx.yy" >> >> > ) >> >> > >> >> > Without the legacy options (including >> >> > $ActionSendStreamDriverPermittedPeers) rsyslogd wont even start. >> >> > And with these options my log looks like this: >> >> > >> >> > Nov 5 10:59:49 logmanagement-client rsyslogd-3003: invalid or >> >> yet-unknown >> >> > config file command 'ActionSendStreamDriverPermittedPeers' - have you >> >> > forgotten to load a module? [try http://www.rsyslog.com/e/3003 ] >> >> > Nov 5 10:59:26 logmanagement-client systemd: Stopping System Logging >> >> > Service... >> >> > Nov 5 10:59:26 logmanagement-client systemd: rsyslog.service: main >> >> process >> >> > exited, code=killed, status=6/ABRT >> >> > Nov 5 10:59:26 logmanagement-client systemd: Unit rsyslog.service >> >> entered >> >> > failed state. >> >> > Nov 5 10:59:26 logmanagement-client systemd: Starting System Logging >> >> > Service... >> >> > yikes -> Nov 5 10:59:26 logmanagement-client kernel: traps: >> >> rsyslogd[4698] >> >> > general protection ip:7fd55a8584bd sp:7ffe547d5f90 error:0 in >> >> libc-2.17.so >> >> > [7fd55a7dc000+1b6000] >> >> > >> >> > When i comment out the StreamDriverPermittedPeers, i get: >> >> > >> >> > Nov 5 11:02:58 logmanagement-client rsyslogd-2088: error: peer name >> not >> >> > authorized - not permitted to talk to it. Names: CN: >> >> > logmanagement.xxx.yy; [try http://www.rsyslog.com/e/2088 ] >> >> > >> >> > So with this "hybrid" config i can transfer logs over the tls >> channel. >> >> But >> >> > unfortunately the system is not really stable, i have seen some >> segfaults >> >> > and the general protection errors in the above log make me a bit wary >> >> using >> >> > this in a production setting. Any suggestions/hints on this specific >> >> error >> >> > and/or the segfaults? >> >> > >> >> > Thanks again! >> >> > >> >> > Regards, >> >> > Jörgen >> >> > >> >> > On Thu, Nov 5, 2015 at 8:07 AM, Radu Gheorghe < >> >> [email protected]> >> >> > wrote: >> >> > >> >> >> Hello, >> >> >> >> >> >> We had this problem at one point when having different versions of >> >> >> rsyslog (and/or gnutls) acting as client and server. Another time >> when >> >> >> I encountered this was when I didn't set up certificates properly. >> >> >> >> >> >> I hope this helps. >> >> >> >> >> >> Best regards, >> >> >> Radu >> >> >> -- >> >> >> Performance Monitoring * Log Analytics * Search Analytics >> >> >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> >> >> >> >> >> >> On Thu, Nov 5, 2015 at 7:27 AM, Jörgen Maas <[email protected]> >> >> wrote: >> >> >> > Hi all, >> >> >> > >> >> >> > With yesterdays help i've succeeded in setting up a TLS listener. >> I >> >> also >> >> >> > setup a forwarder as desribed in: >> >> >> > >> >> >> >> >> >> http://blog.sematext.com/2014/03/25/encrypting-logs-on-their-way-to-elasticsearch-part-2-tls-syslog/ >> >> >> > >> >> >> > On the server side i see this in my logs: >> >> >> > Nov 5 06:10:50 logmanagement rsyslogd-2083: gnutls returned >> error on >> >> >> > handshake: An unexpected TLS packet was received. >> >> >> > >> >> >> > I captured the network sessions and the messages are sent with >> plain >> >> tcp >> >> >> > (readable), so that explains the server side log entry. >> >> >> > >> >> >> > This is my client side config: >> >> >> > >> >> >> > action( >> >> >> > type="omfwd" >> >> >> > target="192.168.124.100" >> >> >> > port="6514" >> >> >> > protocol="tcp" >> >> >> > template="RSYSLOG_SyslogProtocol23Format" >> >> >> > StreamDriver="gtls" >> >> >> > StreamDriverMode="1" >> >> >> > StreamDriverAuthMode="x509/name" >> >> >> > StreamDriverPermittedPeers="logmanagement.xxx.yyy" >> >> >> > ) >> >> >> > >> >> >> > The "gtls" default settings are set in the global() section, as >> >> discussed >> >> >> > yesterday. >> >> >> > >> >> >> > Software version: >> >> >> > rsyslog-7.4.7-7.el7_1.1.x86_64 >> >> >> > >> >> >> > >> >> >> > What am I missing here? >> >> >> > >> >> >> > Thanks! >> >> >> > >> >> >> > >> >> >> > Regards, >> >> >> > Jörgen >> >> >> > _______________________________________________ >> >> >> > rsyslog mailing list >> >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> > http://www.rsyslog.com/professional-services/ >> >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> myriad >> >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> >> >> DON'T LIKE THAT. >> >> >> _______________________________________________ >> >> >> rsyslog mailing list >> >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> >> http://www.rsyslog.com/professional-services/ >> >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> >> >> DON'T LIKE THAT. >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> >> > >> > >> > >> > -- >> > Grtz, >> > Jörgen Maas >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > > -- Grtz, Jörgen Maas _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
