Thank you David. I am going to try and go this way. Philippe
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of David Lang Sent: Friday, June 24, 2016 2:28 PM To: rsyslog-users Subject: Re: [rsyslog] Dealing with malformed messages On Fri, 24 Jun 2016, Maupertuis Philippe wrote: > My main goal is to store messages according to sending clients even for > malformed messages. > Finding which clients are sending malformed messages would be a plus. > Broadly speaking our naming convention says that a hostname is 8 characters > long ending with a specific letter according to the location. > > In my understanding, whenever a syslog relay is involved, cares should be > taken on the first relay because after only the message itself is supposed to > contain information from the sender. > So maybe a no straightforward way is to add json field on the first relay to > keep fromhost and/or fromhost-ip and to reuse these additional fields on the > ultimate central log server. > Would that be a significant overhead ? > Any thoughts on this ? If you are using a current version, it's not a huge overhead (significant depends on your traffic :-) and there are things available to reduce it if you do find it significant. Haivng your first-tier relay boxes take the incoming message and send it out as JSON and include the fromhost-ip is the only way to be sure that you are splitting the messages per source no matter what they contain. David Lang > Philippe > >> -----Original Message----- >> From: [email protected] [mailto:rsyslog- >> [email protected]] On Behalf Of David Lang >> Sent: Thursday, June 23, 2016 7:08 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] Dealing with malformed messages >> >> On Thu, 23 Jun 2016, Maupertuis Philippe wrote: >> >>> Hi list, >>> We have a central log server in place with logs going to files >>> according to the >> sending host. >>> The template is : >>> $template DYNfile,"/dailylog/HOSTS/%HOSTNAME%/%$YEAR%- >> %$MONTH%-%$DAY%/%syslogfacility-text%-%$HOUR%" >>> This works as expected for mosts clients. >>> However some clients are probably sending malformed messages and we >> end up with funny hostnames like "unicast_addr_effective_port" or >> "ventID". >>> So I would like to retain the hostname only if it abide by our >>> naming >> convention and otherwise retain the sender's IP address. >> >> This is a really hard thing to do, what is your naming convention? >> >>> I would also like to build a list of daily culprit to try to correct >>> the messages on the client if possible or to set them apart. What >>> would be the best way to do that. >> >> are you asking the best way to build the list of culprits? or the >> best way to deal with them once you have the list? >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: >> This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > > !!!******************************************************************* > ****************** "Ce message et les pièces jointes sont > confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut > également être protégé par le secret professionnel. Si vous recevez ce > message par erreur, merci d'en avertir immédiatement l'expéditeur et de le > détruire. L'intégrité du message ne pouvant être assurée sur Internet, la > responsabilité de Worldline ne pourra être recherchée quant au contenu de ce > message. Bien que les meilleurs efforts soient faits pour maintenir cette > transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à > cet égard et sa responsabilité ne saurait être recherchée pour tout dommage > résultant d'un virus transmis. > > This e-mail and the documents attached are confidential and intended solely > for the addressee; it may also be privileged. If you receive this e-mail in > error, please notify the sender immediately and destroy it. As its integrity > cannot be secured on the Internet, the Worldline liability cannot be > triggered for the message content. Although the sender endeavours to maintain > a computer virus-free network, the sender does not warrant that this > transmission is virus-free and will not be liable for any damages resulting > from any virus transmitted.!!!" > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. !!!************************************************************************************* "Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.!!!" _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
