Thanks. So nice of you. On Mon, Sep 14, 2015 at 11:41 AM, David Lang <[email protected]> wrote:
> On Mon, 14 Sep 2015, Muhammad Asif wrote: > > Tanks for reply, >> >> Please elaborate two things: >> what is winlog here and secondly why did we use <%pri%> to send remote. >> > > winlog: is a arbitrary tag. Since there is nothing useful being provided, > we need to make something up to fill the spot in the protocol. > > <%pri%> passes the facility/severity information to the remote system. > It's a required part of the syslog protocol > > The syslog protocol (which your sending software isn't complying with) is > > <###>Mon DD HH:MM:SS hostname programname[optionalpid]: message > > where ### is a number representing the combination of facility and > severity, together called Pri (priority). > > what you are getting is missing the programname and the PRI info, so when > we send to a remote system, we need to put at least a placeholder there or > the remote system is going to continue to mis-parse the message. > > David Lang > > > On Fri, Sep 11, 2015 at 11:21 PM, David Lang <[email protected]> wrote: >> >> On Fri, 11 Sep 2015, Muhammad Asif wrote: >>> >>> Please have a look on raw message. >>> >>>> >>>> Sep 11 11:52:15 172.20.16.54 >>>> >>>> >>>> AgentDevice=WindowsLog#011AgentLogFile=Application#011PluginVersion=7.2.2.959003#011Source=MSSQL$MICROSOFT##WID#011Computer=rdsadc.ciit.local#011OriginatingComputer=#011User=NETWORK >>>> SERVICE#011Domain=NT >>>> >>>> >>>> AUTHORITY#011EventID=3221243928#011EventIDCode=18456#011EventType=16#011EventCategory=4#011RecordNumber=282236#011TimeGenerated=1441997534#011TimeWritten=1441997534#011Level=0#011Keywords=0#011Task=0#011Opcode=0#011Message=Login >>>> failed for user 'NT AUTHORITY\NETWORK SERVICE'. Reason: Could not find a >>>> login matching the name provided. [CLIENT: <named pipe>] >>>> >>>> >>> Ok, this message is missing the syslogtag, so you would need to do >>> something along the lines of >>> >>> $template fixwinlog,"%timestamp% %hostname% winlog: %syslogtag%%msg%" >>> >>> if $fromhost-ip == 172.20.16.54 then /var/log/dcc.log;fixwinlog >>> >>> and if you need to send this elsewhere, you'd need a template like: >>> >>> $template fixwinlogremote,"<%pri%>%timestamp% %hostname% winlog: >>> %syslogtag%%msg%" >>> >>> David Lang >>> >>> >>> >>> On Thu, Sep 10, 2015 at 3:47 PM, David Lang <[email protected]> wrote: >>>> >>>> On Thu, 10 Sep 2015, Muhammad Asif wrote: >>>> >>>>> >>>>> We are using IBM wincollect. Is nxlog opensource. >>>>> >>>>> >>>>>> >>>>>> there is a community versin (opensource) and a paid version, I use the >>>>> open one >>>>> >>>>> Here is a raw message I >>>>> >>>>> write in a file using rsyslog. >>>>>> >>>>>> >>>>>> unfortunantly this is after it's gone through the rsyslog >>>>> parsing/heuristics, can you write a message using rawmsg? so we can see >>>>> exactly what is on the wire. >>>>> >>>>> David Lang >>>>> >>>>> >>>>> Sep 10 11:26:41 192.168.6.197 AgentDevice=WindowsLog# >>>>> >>>>> 011AgentLogFile=Security#011PluginVersion=7.2.2.959003# >>>>>> >>>>>> >>>>>> 011Source=Microsoft-Windows-Security-Auditing#011Computer=WIN-C9JDK1MDI5G# >>>>>> 011OriginatingComputer=#011User=#011Domain=#011EventID=5156# >>>>>> 011EventIDCode=5156#011EventType=8#011EventCategory=12810# >>>>>> 011RecordNumber=622562#011TimeGenerated=1441866149# >>>>>> >>>>>> >>>>>> >>>>>> 011TimeWritten=1441866149#011Level=0#011Keywords=0#011Task=0#011Opcode=0#011Message=The >>>>>> Windows Filtering Platform has permitted a connection. Application >>>>>> Information: Process ID: 912 Application Name: >>>>>> \device\harddiskvolume2\windows\system32\svchost.exe >>>>>> Network Information: Direction: Inbound Source Address: >>>>>> 224.0.0.252 >>>>>> Source Port: 5355 Destination Address: 192.168.6.109 Destination >>>>>> Port: >>>>>> 50215 Protocol: 17 Filter Information: Filter Run-Time ID: 66094 >>>>>> Layer Name: Receive/Accept Layer Run-Time ID: 44 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Sep 10, 2015 at 3:17 PM, David Lang <[email protected]> wrote: >>>>>> >>>>>> On Thu, 10 Sep 2015, Muhammad Asif wrote: >>>>>> >>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> >>>>>>> Thanks for being so prompt helper and guider. We can not enforce >>>>>>>> windows >>>>>>>> server 2012 to send logs in specific format. >>>>>>>> But we solve the issue by using the following template. >>>>>>>> >>>>>>>> $template msgonly,"%rawmsg%\n" >>>>>>>> >>>>>>>> >>>>>>>> What else solution can be? >>>>>>>> >>>>>>>> >>>>>>>> well, I don't know what your rawmsg looks like, so I can't say >>>>>>>> what's >>>>>>>> >>>>>>> wrong with it, but you probably should do something more than just >>>>>>> send >>>>>>> the >>>>>>> rawmsg on. >>>>>>> >>>>>>> what software are you using on the windows 2012 servers to send the >>>>>>> logs? >>>>>>> Microsoft doesn't include any software to do so that I am aware of. >>>>>>> >>>>>>> David Lang >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>>> On Thu, Sep 10, 2015 at 12:08 PM, David Lang <[email protected]> wrote: >>>>>>>> >>>>>>>> On Thu, 10 Sep 2015, Muhammad Asif wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi Geeks, >>>>>>>>> >>>>>>>>> >>>>>>>>> We are using tcp protocol for sending logs from rsyslog to fluentd >>>>>>>>> on >>>>>>>>> >>>>>>>>>> same >>>>>>>>>> system. We also test tcp to send logs to other system on tcp. >>>>>>>>>> But problem is that rsyslog dont' send log complete. >>>>>>>>>> >>>>>>>>>> On Sending server >>>>>>>>>> >>>>>>>>>> *.* action(type="omfwd" target="192.168.6.193" port="514" >>>>>>>>>> protocol="tcp" >>>>>>>>>> queue.filename="forwarding" queue.size="1000000 >>>>>>>>>> queue.maxdiskspace="5g" >>>>>>>>>> queue.highwatermark="900000" queue.lowwatermark= "200000" >>>>>>>>>> queue.type="LinkedList" ) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On receiving server >>>>>>>>>> >>>>>>>>>> if ($fromhost-ip == '192.168.6.192') then /var/log/dccc.log >>>>>>>>>> & ~ >>>>>>>>>> >>>>>>>>>> But incomplete message is received. >>>>>>>>>> >>>>>>>>>> Sent Message >>>>>>>>>> >>>>>>>>>> Sep 10 11:26:41 192.168.6.197 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> AgentDevice=WindowsLog#011AgentLogFile=Security#011PluginVersion=7.2.2.959003#011Source=Microsoft-Windows-Security-Auditing#011Computer=WIN-C9JDK1MDI5G#011OriginatingComputer=#011User=#011Domain=#011EventID=5156#011EventIDCode=5156#011EventType=8#011EventCategory=12810#011RecordNumber=622562#011TimeGenerated=1441866149#011TimeWritten=1441866149#011Level=0#011Keywords=0#011Task=0#011Opcode=0#011Message=The >>>>>>>>>> Windows Filtering Platform has permitted a connection. >>>>>>>>>> Application >>>>>>>>>> Information: Process ID: 912 Application Name: >>>>>>>>>> \device\harddiskvolume2\windows\system32\svchost.exe Network >>>>>>>>>> Information: >>>>>>>>>> Direction: Inbound Source Address: 224.0.0.252 Source Port: >>>>>>>>>> 5355 >>>>>>>>>> Destination Address: 192.168.6.109 Destination Port: 50215 >>>>>>>>>> Protocol: >>>>>>>>>> 17 Filter Information: Filter Run-Time ID: 66094 Layer Name: >>>>>>>>>> Receive/Accept Layer Run-Time ID: 44 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Received Message >>>>>>>>>> >>>>>>>>>> Sep 10 11:26:41 192.168.6.197 AgentDevice=WindowsLog#011AgentL >>>>>>>>>> Windows >>>>>>>>>> Filtering Platform has permitted a connection. Application >>>>>>>>>> Information: >>>>>>>>>> Process ID: 912 Application Name: >>>>>>>>>> \device\harddiskvolume2\windows\system32\svchost.exe Network >>>>>>>>>> Information: >>>>>>>>>> Direction: Inbound Source Address: 224.0.0.252 Source Port: >>>>>>>>>> 5355 >>>>>>>>>> Destination Address: 192.168.6.109 Destination Port: 50215 >>>>>>>>>> Protocol: >>>>>>>>>> 17 Filter Information: Filter Run-Time ID: 66094 Layer Name: >>>>>>>>>> Receive/Accept Layer Run-Time ID: 44 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The problem is that your sent mesasge is malformed. After the >>>>>>>>>> >>>>>>>>>> hostname/IP >>>>>>>>> address, you are supposed to have the programname[with optional >>>>>>>>> pid]: >>>>>>>>> that >>>>>>>>> is limtied by teh spec to 32 characters. >>>>>>>>> >>>>>>>>> instead you have >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> AgentDevice=WindowsLog#011AgentLogFile=Security#011PluginVersion=7.2.2.959003#011Source=Microsoft-Windows-Security-Auditing#011Computer=WIN-C9JDK1MDI5G#011OriginatingComputer=#011User=#011Domain=#011EventID=5156#011EventIDCode=5156#011EventType=8#011EventCategory=12810#011RecordNumber=622562#011TimeGenerated=1441866149#011TimeWritten=1441866149#011Level=0#011Keywords=0#011Task=0#011Opcode=0#011Message=The >>>>>>>>> >>>>>>>>> which is a LOT longer than 32 characters. >>>>>>>>> >>>>>>>>> whatever it is that you have slurping up the logs from windows and >>>>>>>>> sending >>>>>>>>> them to rsyslog in the first place needs to be fixed. >>>>>>>>> >>>>>>>>> David Lang >>>>>>>>> _______________________________________________ >>>>>>>>> rsyslog mailing list >>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> myriad >>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>> you >>>>>>>>> DON'T LIKE THAT. >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>> you >>>>>>>> DON'T LIKE THAT. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>>> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>> you >>>>>>> DON'T LIKE THAT. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>>> _______________________________________________ >>>>>> >>>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> _______________________________________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> _______________________________________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
