On Wed, 23 Nov 2016, [email protected] wrote:
there are probably ways to simplify the configs, 5K lines of configs seems
excessive :-) how much of this is rulebase config vs rsyslog config?
Each app generates app-access.log, app-tomcat.log, app-application.log files.
imfile allow me to add filename as metadata, but nothing more.
As each application belongs to a workgroup, part of an organizational unit
and is running on some (multiple) hosts, at the end I have approximately this
for each app:
template(name="json_appX" type="list") {
property(name="hostname")
constant(value=" ")
property(name="syslogtag")
constant(value=" {")
constant(value="\"group\":\"group\","\"unit\":\"unit\",\"app\":\"appX\",")
constant(value="\",\"file\":\"")
property(name="$!metadata!filename")
constant(value="\",\"msg\":\"")
property(name="msg" format="jsonr")
constant(value="\"}")
}
ruleset(name="json_appX") {
action(
template="json_appX"
type="omrelp"
target="server"
port="20514"
action.resumeRetryCount="-1"
action.reportSuspension="on"
queue.maxdiskspace="5M"
queue.type="LinkedList"
queue.filename="appX.qi"
queue.SaveOnShutdown="on"
)
}
input(type="imfile" file="/logs/appX/access.log" tag="group/appX"
addMetadata="on" ruleset="json_appX" PersistStateInterval="1")
input(type="imfile" file="/logs/appX/tomcat.log" tag="group/appX"
addMetadata="on" ruleset="json_appX" PersistStateInterval="1")
input(type="imfile" file="/logs/appX/application.log" tag="group/appX"
addMetadata="on" ruleset="json_appX" PersistStateInterval="1")
you should be able to collapse all the different templates into one. Instead of
hard-coding the group/unit/app in each template, you should have that be a
variable that you set.
The ugly way to do this would be a series of
if $programname = "group/appX" then set $.owner =
"\"group\":\"group\","\"unit\":\"unit\",\"app\":\"appX\",";
statements.
a far more elegant way to do this would be to do a table lookup on the
programname and have it return the string.
you can also simplify the template a bit. Instead of crafting the json in the
template, create a variable that has what you want in/under it and output that
variable. but compared to collapsing all the templates together, that's a minor
change :-)
I question the value of having a separate sending queue for each app. I think
it's better to send them in one combined firehose and split them on the
receiving side. It makes it less disruptive when you find you want to change the
groupings of things and all those queues on the sender can eat up a lot of ram.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
