On Tue, 16 Nov 2010, Joe Williams wrote:
List,
I am trying to standardize the hostnames that we see in our logs. It seems that
the services (haproxy, etc) that log directly to the rsyslog server do this
differently. Here's an example:
./ec2-<snip>.us-west-1.compute.amazonaws.com/haproxy
./domu-<snip>/haproxy
./domu-<snip>/haproxy
./domu-<snip>/haproxy
./ip-<snip>.ec2.internal/haproxy
All the standard logs (user.log, syslog, messages, etc) all use a standard
format like:
./domu-<snip>/syslog
./ip-<snip>/syslog
As you see like in the case of ip-<snip>, ".ec2.internal" gets appended on with haproxy.
In the case of ec2-<snip>.us-west-1.compute.amazonaws.com is actually coming from a host with a
hostname like ip-<snip>. Hopefully this makes sense.
Across the board I am using %hostname:::lowercase% to create the directories.
As a test to see what the application sees vs rsyslog we added code to log the
hostname in one of our applications:
ec2-<snip>.us-west-1.compute.amazonaws.com/ellison:Nov 16 20:34:22
ec2-<snip>.us-west-1.compute.amazonaws.com local3: 2010-11-16 20:34:22,123 INFO [main]
ejje.Ejje - Address ip-<snip>
As you can see rsyslog is logging this with the "ec2-" style hostname but what the
application is seeing for the hostname is the "ip-" style.
Interestingly the hostname style that rsyslog sees seems to depend on what EC2 availability zone the node is in.
"ec2-" hostnames are only in the us-west-1 and ap-southeast-1 zones, "ip-*.ec2.internal" and
"domu-*" hostnames in us-east-1. So I think this would suggest something with DNS configurations in
different zones. Additionally the "ec2-" style hostnames are actually public hostnames that aren't assigned
to the machines but to a MIP or VIP, which again suggests some sort of DNS lookup.
I have tried using %fromhost% with the same results. Any thoughts on what might
be going on and how to fix it?
fromhost is the DNS lookup of the IP address of the machine that last
touched the logs.
if the sending host set hostname in it's logs, then hostname is that
value.
If the sending host did not put something that looks like a hostname in
the log messages, the first instance of rsyslog that receives the message
fills the hostname field with fromhost.
it sounds as if your sending systems are not setting the hostname in the
logs, so rsyslog is filling in the fromhost.
If you setup /etc/hosts entries for the IP addresses of these machines
with a short name first, I believe that rsyslog will use that as the
result of the name lookup.
The better option is to go to the sending machines and figure out why they
aren't putting hostname in their outbound logs.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
