Thanks so much David, I changed the filter and it seems to be working properly with DNS lookups, I have a few thousand items sending logs so renaming them might be a little more than I can chew, but currently its working well on 7.5.8. Shame about the table lookups, they would have come in handy, but this will do!
Thanks again! Robert Robert On Sat, Jan 18, 2014 at 2:44 AM, David Lang <[email protected]> wrote: > On Fri, 17 Jan 2014, robert s wrote: > >> David, >> >> thanks for the response, >> >> I see, so currently I have a custom /etc/host file with 100+ host >> names they are formated like this: >> >> ipaddres host1.firewall (literraly host1, and so on) >> ipaddress host2.firewall >> >> ipaddress host3.shib >> ipaddress host4.shib >> >> and I was filtering with ":hostname, contains "firewall" or "shib", >> and I thought it was working that way on our tests? >> >> Is there a way to filter the way I mentioned or do I need to explore >> another way? > > > to use the /etc/hosts file you would need to enable DNS lookups, and filter > on fromhost instead of hostname (and not go through a relay between the > sender and filtering system) > > hostname is only populated from /etc/hosts if the sending machine leaves it > out in a way that rsyslog can detect > > in your prior testing, you probably had the hostname on the sending machines > set to this sort of pattern and just didn't realize it. > > There is not currently a very good way to do what you are trying to do > nicely. > > you can do array matches on their hostnames/fromhost-ip fields > > you can filter on fromhost and do DNS lookups (much less costly in 7.x than > in 5.x and before, but still not cheap) > > you can make sure all your sending machines are named in a way that makes > filtering easy > > What you are trying to do is exactly what table lookups were designed for, > but the sponsorship for them fell through :-( > > David Lang > > >> Thanks again for your help >> >> Robert >> >> >> >> Robert >> >> >> On Fri, Jan 17, 2014 at 2:08 PM, David Lang <[email protected]> wrote: >>> >>> I think you are confusing yourself. >>> >>> the hostname field is not supposed to be set by DNS or /etc/hosts, the >>> fromhost field is. But if the message arrives without a valid hostname >>> field, rsyslog will use fromhost instead. >>> >>> The hostname field is supposed to be set by the machine generating the >>> log >>> in the first place, and it should not change as the message is passed >>> through log relays. >>> >>> The machine generating the log should know it's own name, and depending >>> on >>> how it's configured, it may use a short name or it may use a long name. >>> The >>> older RFC specified that short names should be used (and relays should >>> strip >>> out 'extra' domain info), in the new RFC (RFC5424) the format is >>> different >>> and full names should be used and kept. Almost nothing actually uses the >>> new >>> RFC format >>> >>> log some messages with RSYSLOG_DebugFormat and with the -x you should see >>> that %fromhost% contains the same IP address as %fromhost-ip% instead of >>> a >>> name that you would see without the -x >>> >>> %hostname% should match what is in the raw message that rsyslog receives. >>> >>> David Lang >>> >>> On Fri, 17 Jan 2014, robert s wrote: >>> >>>> Date: Fri, 17 Jan 2014 10:39:16 -0500 >>>> >>>> From: robert s <[email protected]> >>>> Reply-To: rsyslog-users <[email protected]> >>>> To: rsyslog-users <[email protected]> >>>> Subject: Re: [rsyslog] dns lookup disabled >>>> >>>> >>>> ok I may be looking at this incorrectly, from other posts on the >>>> mailing lists, and responses, rsyslog does not really look at >>>> /etc/hosts? >>>> >>>> from some of the responses that I am reading I see that it may depend >>>> on the distro >>>> >>>> the gethostname() I guess determines the localhost >>>> >>>> so the question is this: >>>> >>>> I am filtering with hostname contains "myownmadeupname" then omfile >>>> to /path/to/log >>>> >>>> what I am running into is that rsyslog is resolving hostnames from >>>> what it seems to be my main dns server which would not work for my >>>> current scenario of custom "contains" parameters. >>>> >>>> if I disable dns lookups with the -x option I dont seem to getting any >>>> logs at all. >>>> >>>> So I think I may be confusing myself..? >>>> >>>> any input will be appreciated >>>> >>>> Robert >>>> >>>> >>>> On Fri, Jan 17, 2014 at 9:06 AM, robert s <[email protected]> >>>> wrote: >>>>> >>>>> >>>>> it shows that the flag is started >>>>> >>>>> ps -fe | grep rsyslog >>>>> root 8064 1 1 09:05 ? 00:00:00 /sbin/rsyslogd -x -i >>>>> /var/run/syslogd.pid >>>>> >>>>> >>>>> Robert >>>>> >>>>> >>>>> On Fri, Jan 17, 2014 at 9:02 AM, robert s <[email protected]> >>>>> wrote: >>>>>> >>>>>> >>>>>> I tried changing nsswitch to not do dns, and configured nscd to not do >>>>>> lookups, and have even disabled nscd, and removing resolve.conf >>>>>> settings >>>>>> >>>>>> Robert >>>>>> >>>>>> >>>>>> On Thu, Jan 16, 2014 at 3:14 PM, robert s <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Thanks for the response Mike, I rsysced the configuration from the >>>>>>> test servers to the production servers. >>>>>>> >>>>>>> Robert >>>>>>> >>>>>>> >>>>>>> On Thu, Jan 16, 2014 at 3:08 PM, Mike Hoskins (michoski) >>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Not sure why it would just start on your prod servers, but I got bit >>>>>>>> one >>>>>>>> time by copy/paste and using %FROMHOST% vs %HOSTNAME% in templates. >>>>>>>> Are >>>>>>>> you using the same config on your test and prod hosts? >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: robert s <[email protected]> >>>>>>>> Reply-To: rsyslog-users <[email protected]> >>>>>>>> Date: Thursday, January 16, 2014 3:02 PM >>>>>>>> To: rsyslog-users <[email protected]> >>>>>>>> Subject: [rsyslog] dns lookup disabled >>>>>>>> >>>>>>>>> Hey guys long time >>>>>>>>> >>>>>>>>> So far I have tested Rsyslog to the max and was able to log 400,000 >>>>>>>>> messages per second on a 1gb line, so Im loving rsyslog! >>>>>>>>> >>>>>>>>> I have build production servers, rhel 6.4 with rsyslog 7.5.6, but >>>>>>>>> currently I am having an issue with dns lookup, I have >>>>>>>>> /init.d/rsyslog configured with -x -i. to disable dns lookup, but >>>>>>>>> for >>>>>>>>> some reason Rsyslog still resolving names. >>>>>>>>> >>>>>>>>> I have flushed dns, changed nsswitch disabling dns. >>>>>>>>> >>>>>>>>> Any ideas why that may be? >>>>>>>>> >>>>>>>>> Thanks in advanced >>>>>>>>> >>>>>>>>> Robert >>>>>>>>> _______________________________________________ >>>>>>>>> rsyslog mailing list >>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> myriad >>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>> you >>>>>>>>> DON'T LIKE THAT. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>> POST if >>>>>>>> you DON'T LIKE THAT. >>>> >>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T >>>> LIKE THAT. >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
