i have seen things like this when the client machine's /etc/hosts is missing an entry for itself (meaning the IP that is being logged), (or reverse DNS fails to answer).
Alan ________________________________________ From: [email protected] [[email protected]] On Behalf Of Dan Finn [[email protected]] Sent: 09 December 2013 23:38 To: rsyslog-users Subject: Re: [rsyslog] strange change in filtering since upgrading to v7 To answer my own question I believe I have been able to do this with tcpdump. It would appear that the app is submitting the log entries to the client rsyslog on port 514 with the IP. For some reason it used to use the hostname and now it no longer is. Sorry for the wasted 1’s and 0’s. Thanks, Dan On 12/9/13, 3:52 PM, "Dan Finn" <[email protected]> wrote: >Maybe a dumb question but how would you go about following that chain? > >On 12/9/13, 3:22 PM, "David Lang" <[email protected]> wrote: > >>The hostname field is supposed to be filled out by the system that's >>generating >>the log, if it sends a log without a hostname field, the first machine >>that >>receives the log tries to fill it in, with a hostname from reverse DNS if >>configured, or with the sending IP address if the DNS lookup fails (or is >>disabled) >> >>follow the chain from the originating system on and see where it's >>picking up >>the IP address. >> >>David Lang >> >> >>On Mon, 9 Dec 2013, Dan Finn wrote: >> >>> Date: Mon, 9 Dec 2013 18:43:14 +0000 >>> From: Dan Finn <[email protected]> >>> Reply-To: rsyslog-users <[email protected]> >>> To: rsyslog-users <[email protected]> >>> Subject: [rsyslog] strange change in filtering since upgrading to v7 >>> >>> Remote servers: >>> CentOS 5.10 >>> rsyslog-7.4.6-1.el5.centos >>> >>> Client servers: >>> RHEL 5.4 >>> rsyslog-5.8.6-1.ep >>> >>> Remote config : http://pastebin.com/xp5wy02d >>> Client config: http://pastebin.com/17qYD6WX >>> >>> As far as I know nothing has changed on the client side and the only >>>change that we have made recently to our logging environment is >>>upgrading our remote servers from v4 to v7 (huge performance >>>improvement!). It was noticed recently that the filtering on one of our >>>apps is no longer working as expected. Instead of the logs getting >>>written to /var/log/apps/year/hostname/day/hour/jboss.log they are >>>ending up in /var/log/apps/year/IP address/day/hour/jboss.log. I >>>personally don¹t think this is related to the v7 upgrade but this is >>>causing some pain for us with our log scrapers and I¹ve been asked to >>>verify with the mailing list. >>> >>> So far it looks like this is the only application that we have that >>>this is happening with. This is Jboss and it¹s logging via log4j. All >>>other apps and OS logs seems to be working just fine. >>> >>> Here is a current log entry: >>> >>> 2013-12-09T11:00:00-07:00 10.42.30.10 local4: 11:00:00,094 >>>atgprod1-prod_public_8180 atg-log WARN [BrandCategoryLookupD >>> roplet] 1414419202 dogfunk >>>7CE0625028E65F3843D63FE249CD0125.atgprod1-prod_public_8180 >>>/Store/catalog/brandLanding.jsp?b >>> >>>randId=100000630&categoryId=dfCat100434&p=discountPercentUS%3A%5B40+TO+* >>>% >>>5D%7Csize%3Asmall (http-0.0.0.0-8180-187) Brand >>> category not found for brand: 100000630 category: dfCat100239 >>> >>> And here¹s a log entry from the same server from last week sometime: >>> >>> 2013-12-01T23:00:00-07:00 atgprod1 local4: 23:00:00,103 >>>atgprod1-prod_public_8080 atg-log INFO [ProfileFormHandler] 1387361000 >>>b678482057 bcs >>>4C185612E4845821163BF0AEDAE745CE.atgprod1-prod_public_8080 >>>/Store/account/login.jsp?locale=en_US&_DARGS=/Store/authModal/includes/m >>>o >>>dalLoginForm.jsp.login-form&_=1385963950864 (http-0.0.0.0-8080-130) User >>>Œ[email protected]' attempted to log in, but failed. >>> >>> We run rsyslog with the x option on the remote servers. This is how >>>we were also doing it with v4. I tested removing that flag to enable >>>dns lookups and it didn¹t seem to make a difference for this issue. >>> >>> As I understand it, that info is coming from the header of the log >>>message and would be getting set on the client side? Is that also going >>>to be something that is set by the application when it submits the log >>>or would the rsyslog client be inserting that? My app guys are saying >>>that nothing has changed on their side so we really aren¹t sure what >>>could be causing this change. >>> >>> Any insight into what might be causing this would be really >>>appreciated. >>> >>> Thanks, >>> Dan >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
