On 20/04/2023 13:36, Nikolaos Milas <nmi...@noa.gr> wrote:
I was wondering whether nfcapd 1.7 (from nfdump project) coupled with
nfinflux (https://github.com/phaag/nfinflux/blob/master/README.md) would
be a workable scenario.
Splunk / Elastic would then be able to read data directly from an InfluxDB?
As far as I can see, nfinflux only exports metrics (counters), not the
raw flows.
In any case, I'm not sure that putting another database in between
nfdump and elastic would be a great idea, as you'd need another tool to
slurp out of influxdb into elastic, and you'd need to manage the
influxdb instance (e.g. clean out old records).
If you want the raw flows, you could invoke nfdump -o json directly, and
write a small program to forward the records to elastic. You could run
it from cron, or even write it as an nfsen plugin
<https://nfsen.sourceforge.net/PluginGuide/plugin-guide.html>.
You could probably even to use jq to transform nfdump -o json output
into something suitable for going into elastic's Bulk API
<https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html>,
something like this:
/usr/local/bin/nfdump -M '/var/nfsen/profiles-data/live/gw1:gw2' -R
'2020/06/23/nfcapd.202006230000:2020/06/23/nfcapd.202006230000' -o json |
jq 'map({"doc":., "doc_as_upsert": true})'
(completely untested, but you get the idea. Add curl to write the
records to elastic). Or you could do processing in an ingest pipeline
<https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html>.
You'll end up writing your own dashboards though, unless someone has
already made dashboards that use the data in nfsump -o json format.
HTH,
Brian.
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
