* Stefan G. Weichinger:
> Maybe I look into mongodb as well, for example I found this small
> howto: https://www.fluentd.org/guides/recipes/maillog-mongodb
That looks unnecessarily complicated to me. While you can of course move
data from an existing log file into MongoDB, I find configuring syslog
to use a MongoDB destination (in addition to your files or as a full
replacement) much easier.
See [1] section "Storing messages in a MongoDB database". I have also
done it with rsyslog, but that took a bit more work.
Here's a syslog-ng destination I use. Note that using uri() allows
passing parameters to modern MongoDB drivers which the older servers()
statement cannot cope with.
destination d_mongo {
mongodb(
uri("mongodb://user:pw@hostname:27017/syslog?authSource=admin&ssl=true")
collection("messages")
value-pairs(
scope("selected-macros" "nv-pairs")
pair("DATE", datetime("$UNIXTIME"))
pair("PID", int64("$PID"))
pair("SEQNUM", int64("$SEQNUM"))
exclude("HOST*")
exclude("LEGACY*")
exclude("SOURCE*")
exclude("TAGS")
)
);
};
Values are strings to begin with. This example excludes some values I am
not interested in, and performs type conversion on others, for example
mapping DATE to MongoDB's date/time data type (see ISODate) and PID to a
numeric value. Conversion can of course happen during analysis, but
since syslog-ng is smart enough to do it when writing data, I prefer
that.
[1]
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.16/administration-guide/37#TOPIC-956524
-Ralph
