I'm making progress on my nxlog deployment design, but have a worrying
hurdle with regard to log rotation.
Here are the relevant extracts from my current config:
<Input in_syslog_json_ietf_tls>
Module im_ssl
Port 10514
Host 0.0.0.0
CertKeyFile /etc/pki/tls/private/...
CertFile /etc/pki/tls/certs/...
CAFile /etc/pki/tls/certs/...
RequireCert FALSE
Exec parse_syslog_ietf(); parse_json($Message);
Exec if $UoO_application_stack == undef \
{ \
$UoO_application_stack = "common"; \
} \
if $UoO_log_type == undef \
{ \
$UoO_log_type = "unknown"; \
}
</Input>
<Output out_json_syslog_ietf>
Module om_file
Exec dir_make("/logs/current/" + $our_application_stack + "/" +
$our_environment + "/" + $MessageSourceAddress);
File "/logs/current/" + $our_application_stack + "/" +
$our_environment + "/" + $MessageSourceAddress + "/" + $our_log_type +
".json"
Exec to_json();
Exec if out_json_syslog_ietf->file_size() > 1M { \
$tmpfile = "/logs/current/" + $our_application_stack +
"/" + $our_environment + "/" + $MessageSourceAddress + "/" + $our_log_type
+ "-" + strftime(now(), "%FT%H:%M%:%S%z") + ".json"; \
$archivefile = replace($tmpfile, "/logs/current/",
"/logs/archive/", 1); \
dir_make("/logs/archive/" + $our_application_stack +
"/" + $our_environment + "/" + $MessageSourceAddress); \
out_json_syslog_ietf->rotate_to($tmpfile); \
}
</Output>
<Route 3>
Path in_syslog_json_ietf_tls => out_json_syslog_ietf
</Route>
Currently I only have a few machines pointed to this, but one of those
(Active Directory) does log at a high volume.
I see the following in my server's nxlog.log, which indicates (and I have
verified this to be the case) that one system's logs are being rotated to
another systems path. (Ignore the faulty time formatting for now, and the
apparently empty UoO_log_type variable.)
2014-07-25 09:19:19 INFO om_file successfully rotated file
'/logs/current/active_directory/prod/ACTIVE_DIRECTORY_IP/unknown.json' to
'/logs/current/sst_workstation//MY_WORKSTATIONS_IP
/windows_event_log-2014-07-25T09:19%:19+1200.json'
2014-07-25 09:19:20 INFO om_file successfully rotated file
'/logs/current/sst_workstation//MY_WORKSTATIONS_IP/windows_event_log.json'
to
'/logs/current/sst_workstation//MY_WORKSTATIONS_IP/windows_event_log-2014-07-25T09:19%:20+1200.json'
2014-07-25 09:19:20 INFO om_file successfully rotated file
'/logs/current/active_directory/prod/ACTIVE_DIRECTORY_IP/unknown.json' to
'/logs/current/active_directory/prod/ACTIVE_DIRECTORY_IP/unknown-2014-07-25T09:19%:20+1200.json'
2014-07-25 09:19:21 INFO last message repeated 23 times
As you can tell from the last line of that log sample, the Active Directory
logs is very high-volume, so I'm thinking this is likely some sort
race-condition; or is my intuition of the semantics at fault?
If I change out_json_syslog_ietf->file_size() to just be file_size() -- and
similarly with the call to rotate_to($tmpfile) -- then the problems doesn't
appear to occur.
Can someone enlighten me on the difference?
--
Cameron Kerr <cameron.kerr...@gmail.com>
See my blog at http://distracted-it.blogspot.co.nz/ (previously
http://humbledown.org/)
Skype me on cameron.kerr.nz
------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
