Re: [rsyslog] RHEL7.1 / rsyslog 8.x random message loss

2015-10-08 7:07 GMT+02:00 Rainer Gerhards : > Sent from phone, thus brief. > Am 07.10.2015 23:15 schrieb "David Lang" : >> >> I would have expected rsyslog to show errors in it's logs and/or problems >> in impstats when maxopenfiles is hit and it can't open

Re: [rsyslog] RFC: dynamic-stats support

-- Regards, Janmejay PS: Please blame the typos in this mail on my phone's uncivilized soft keyboard sporting it's not-so-smart-assist technology. On Oct 7, 2015 11:25 PM, "David Lang" wrote: > > On Wed, 7 Oct 2015, singh.janmejay wrote: > >> -- >> Regards, >> Janmejay >> >> PS:

Re: [rsyslog] RFC: dynamic-stats support

2015-10-08 8:30 GMT+02:00 singh.janmejay : >> Similarly, when one thread goes to output the stats, you need to lock > them so that there isn't a lost increment between the time that you read > the stat and the time you zero it. > > No, this involves the same shared

Re: [rsyslog] RHEL7.1 / rsyslog 8.x random message loss

On Thu, 8 Oct 2015, Rainer Gerhards wrote: 2015-10-08 7:07 GMT+02:00 Rainer Gerhards : Sent from phone, thus brief. Am 07.10.2015 23:15 schrieb "David Lang" : I would have expected rsyslog to show errors in it's logs and/or problems in impstats when

Re: [rsyslog] RHEL7.1 / rsyslog 8.x random message loss

2015-10-08 11:54 GMT+02:00 David Lang : > On Thu, 8 Oct 2015, Rainer Gerhards wrote: > >> 2015-10-08 7:07 GMT+02:00 Rainer Gerhards : >>> >>> Sent from phone, thus brief. >>> Am 07.10.2015 23:15 schrieb "David Lang" : I would

Re: [rsyslog] RFC: dynamic-stats support

Yep, makes sense. I second your opinion, absolute consistency between metrics is not that valuable. On Thu, Oct 8, 2015 at 8:57 PM, Rainer Gerhards wrote: > 2015-10-08 17:19 GMT+02:00 singh.janmejay : >> Did you mean it's not atomic across

Re: [rsyslog] Complex forwarding and spoofing question

Regarding the config: We do something very similar; using your example it would look like: ruleset(name="ruleset_eth0_514" queue.type="LinkedList") { if $fromhost-ip == ["1.2.3.4"] then { call action.fwd.remote1 } if $fromhost-ip == ["2.3.4.5"] then { call

Re: [rsyslog] RFC: dynamic-stats support

Atomic ops are actually rather expensive (almost as expsnsive as full locks). If you want a lockless metrics capability, you should do a separate set of variables per thread, gathering them a reporting time. And document that there is going to be inconsistancy between the different metrics

Re: [rsyslog] segfault problems

Can you create an issue here? https://github.com/rsyslog/rsyslog/issues If you have some easier way to reproduce, it would help also. Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Thu, Oct 8, 2015 at 8:07 PM, Micah

Re: [rsyslog] segfault problems

I have some gdb output following the instructions at the bottom of http://www.rsyslog.com/doc/v8-stable/troubleshooting/troubleshoot.html Is there someone I can send this to? Thanks! On 10/8/15 9:22 AM, Micah Yoder wrote: > well. I updated our spare staging server, which gets virtually no

Re: [rsyslog] segfault problems

what does your config look like? David Lang On Thu, 8 Oct 2015, Micah Yoder wrote: Date: Thu, 8 Oct 2015 09:22:23 -0500 From: Micah Yoder Reply-To: rsyslog-users To: rsyslog@lists.adiscon.com Subject: Re: [rsyslog] segfault problems

Re: [rsyslog] segfault problems

Just did. https://github.com/rsyslog/rsyslog/issues/550 Pastebin with gdb info linked therein. Unfortunately I'm not sure I can get it to a small reproducible config. David, re: config: It is a hierarchy of files generated by Puppet to put in several different elasticsearch rules and application

Re: [rsyslog] Trouble parsing CSV inputs

On Thu, 8 Oct 2015, Dave Burkholder wrote: Hello everyone. First-time experimenting in rsyslog, so bear with me! (Working on Centos 6, with rsyslog 8.13). I'm investigating the Sample rule base examples, and having trouble with both CSV examples. This example works as advertised, but my

Re: [rsyslog] RFC: dynamic-stats support

On Thu, Oct 8, 2015 at 11:07 PM, David Lang wrote: > Atomic ops are actually rather expensive (almost as expsnsive as full > locks). If you want a lockless metrics capability, you should do a separate > set of variables per thread, gathering them a reporting time. And document >

Re: [rsyslog] Complex forwarding and spoofing question

On Thu, 8 Oct 2015, Randy Baca wrote: That is a correct assessment of the flow. There is no impstats output. The line in the conf is: module (load="impstats" log.file="/var/spool/rsyslog/stats.log") There is no file created whether on 514 or 1. so the impstats line for that action

Re: [rsyslog] Complex forwarding and spoofing question

Regarding the tagging of messages, we can't really add anything due to the way the SEIM parses. If we change the format of the message from the default we lose manageability. I was told there may be compliance issues with that, also. Regarding impstat, I don't get any stats for either the

Re: [rsyslog] Complex forwarding and spoofing question

On Thu, 8 Oct 2015, Randy Baca wrote: Yes, looking at both ends simultaneously. Started a tcpdump on both hosts and I only see my telnet connections. Restarted rsyslog and waited a couple minutes and I see no attempts at all. Doesn't even send a SYN. and you are sure that you had some

Re: [rsyslog] Complex forwarding and spoofing question

rsyslogd 8.13.0, compiled with: PLATFORM: x86_64-redhat-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: No FEATURE_DEBUG (debug build, slow code): No

Re: [rsyslog] Complex forwarding and spoofing question

I'm not sure I follow... here's what I think you've built based on the thread so far: Your entire log forwarding path looks like: Source --> Loghost --{VPN}--> Collector --> SEIM A B C D E (This is entirely reasonable; we do something similar. So

Re: [rsyslog] Trouble parsing CSV inputs

well, scratch this out. I don't know how many times I read over the rule, and didn't realize that the rule was matching the "CSV: " literal before parsing the csv columns. :-( Now it's working. :-| On 10/08/2015 04:18 PM, Dave Burkholder wrote: Hello everyone. First-time experimenting in

Re: [rsyslog] Complex forwarding and spoofing question

On Thu, 8 Oct 2015, Dave Caplinger wrote: I'm not sure I follow... here's what I think you've built based on the thread so far: Your entire log forwarding path looks like: Source --> Loghost --{VPN}--> Collector --> SEIM A B C D E (This is entirely

[rsyslog] Trouble parsing CSV inputs

Hello everyone. First-time experimenting in rsyslog, so bear with me! (Working on Centos 6, with rsyslog 8.13). I'm investigating the Sample rule base examples, and having trouble with both CSV examples. This example works as advertised, but my logs will have blank fields, and this rule

Re: [rsyslog] Complex forwarding and spoofing question

Yes, looking at both ends simultaneously. Started a tcpdump on both hosts and I only see my telnet connections. Restarted rsyslog and waited a couple minutes and I see no attempts at all. Doesn't even send a SYN. From:

Re: [rsyslog] Trouble parsing CSV inputs

Thanks for your rapid replies, David! Now, can I ask another question? There's no example of quoted CSV and all my logs are quoted minimally. Is there an easy way to write a ruleset to handle columns that may be quoted, but may not be? On 10/08/2015 04:27 PM, David Lang wrote: adding -v to

Re: [rsyslog] Complex forwarding and spoofing question

Which version is that? We had ages ago a version that did the network byte order calculation incorrectly. Sent from phone, thus brief. Am 08.10.2015 23:14 schrieb "Randy Baca" : > When the port is set to 514 it works fine. When I edit the conf and > change only the port to

Re: [rsyslog] Complex forwarding and spoofing question

I think it would be the right time to create a debug log... Sent from phone, thus brief. Am 08.10.2015 22:29 schrieb "Randy Baca" : > Yes, looking at both ends simultaneously. Started a tcpdump on both hosts > and I only see my telnet connections. Restarted rsyslog and waited

Re: [rsyslog] Complex forwarding and spoofing question

On Thu, 8 Oct 2015, Randy Baca wrote: Regarding the tagging of messages, we can't really add anything due to the way the SEIM parses. If we change the format of the message from the default we lose manageability. I was told there may be compliance issues with that, also. the nice thing

Re: [rsyslog] Complex forwarding and spoofing question

That is a correct assessment of the flow. There is no impstats output. The line in the conf is: module (load="impstats" log.file="/var/spool/rsyslog/stats.log") There is no file created whether on 514 or 1. From: rsyslog-boun...@lists.adiscon.com

Re: [rsyslog] Trouble parsing CSV inputs

On Thu, 8 Oct 2015, Dave Burkholder wrote: well, scratch this out. I don't know how many times I read over the rule, and didn't realize that the rule was matching the "CSV: " literal before parsing the csv columns. :-( Now it's working. :-| adding -v to lognormalizer will help you find

Re: [rsyslog] segfault problems

On Thu, 8 Oct 2015, Micah Yoder wrote: On 10/8/15 1:25 PM, David Lang wrote: even using puppet, I think you can separate out the template generation Oh sure i can, it would just be a bit of a re-organization. worst case, create a separate include directory that only includes the templates

Re: [rsyslog] Complex forwarding and spoofing question

On Thu, 8 Oct 2015, Randy Baca wrote: That rule works better, but I still cannot get rsyslog to forward on port 1. I turned off iptables, I can make a telnet connection to the remote host on 1, but rsyslog will not even attempt to connect to the remote host on 1. It works just

Re: [rsyslog] Trouble parsing CSV inputs

On Thu, 8 Oct 2015, Dave Burkholder wrote: Thanks for your rapid replies, David! Now, can I ask another question? There's no example of quoted CSV and all my logs are quoted minimally. Is there an easy way to write a ruleset to handle columns that may be quoted, but may not be?

Re: [rsyslog] Complex forwarding and spoofing question

When the port is set to 514 it works fine. When I edit the conf and change only the port to 1 it doesn't work. When I do the testing I also set the remote host to receive on 514 or 1 as needed. From: rsyslog-boun...@lists.adiscon.com

Re: [rsyslog] segfault problems

On 10/8/15 1:25 PM, David Lang wrote: > even using puppet, I think you can separate out the template generation Oh sure i can, it would just be a bit of a re-organization. > worst case, create a separate include directory that only includes the > templates and the code that evaluates the

Re: [rsyslog] segfault problems

Well, with that exact same configuration (with the templates all moved to the front of the include chain), with it crashing in a few minutes under 8.13, I downgraded to 8.10, and it hasn't crashed yet. It's been a couple hours. ___ rsyslog mailing

Re: [rsyslog] Complex forwarding and spoofing question

On Fri, 9 Oct 2015, Randy Baca wrote: Dang! Yup, it was SELinux. Got it working end to end now, just need to test failed connections and spooling. Many thanks for everything. I think we are golden now. great to hear. And I agree with your arguments. Maybe I will use them on management

Re: [rsyslog] Complex forwarding and spoofing question

Dang! Yup, it was SELinux. Got it working end to end now, just need to test failed connections and spooling. Many thanks for everything. I think we are golden now. And I agree with your arguments. Maybe I will use them on management some time real soon. ;-)

Re: [rsyslog] segfault problems

On Thu, 8 Oct 2015, Micah Yoder wrote: Just did. https://github.com/rsyslog/rsyslog/issues/550 Pastebin with gdb info linked therein. Unfortunately I'm not sure I can get it to a small reproducible config. David, re: config: It is a hierarchy of files generated by Puppet to put in several

Re: [rsyslog] Complex forwarding and spoofing question

That rule works better, but I still cannot get rsyslog to forward on port 1. I turned off iptables, I can make a telnet connection to the remote host on 1, but rsyslog will not even attempt to connect to the remote host on 1. It works just fine if the omfwd port="514" and