Hi Paolo,
With things in that configuration I do actually get time-based bins,
however...
nfacctd seems to start by spawning a single mongodb process:
pmacct-1.5.0rc1 # ps aux|grep Mongo
root 15527 2.2 0.1 117040 47720 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
After a little bit this grows to two, then three:
pmacct-1.5.0rc1 # ps aux|grep Mongo
root 15527 2.8 0.3 120868 78504 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
root 15545 0.6 0.1 123384 44896 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
root 15567 0.0 0.0 8952 664 pts/0 R+ 23:39 0:00 grep
--colour=auto Mongo
pmacct-1.5.0rc1 # ps aux|grep Mongo
root 15527 3.0 0.3 121528 82244 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
root 15545 0.6 0.1 123384 44896 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
root 15568 0.0 0.1 128148 49136 pts/8 S+ 23:39 0:00
nfacctd: MongoDB Plugin [all]
After another minute, this grows very, very quickly until the box will OOM:
fresh-netflow pmacct-1.5.0rc1 # ps aux|grep -c Mongo
72
Is this possibly just because MongoDB can't keep up with the insert rate
I'm asking it for here?
-Adam
On 12/1/13, 10:24 PM, Paolo Lucente wrote:
Hi Adam,
You should set mongo_history to something, ie.:
mongo_history[all]: 5m
mongo_history_roundoff[all]: m
This enables historical accounting and time-bins and you should then
see collection names being populated with the correct date (and time).
Let me know if this works for you.
When you say you want 1:1 packets, you mean flow records, correct?
Cheers,
Paolo
On Sun, Dec 01, 2013 at 02:01:12AM -0500, Adam Jacob Muller wrote:
Hi,
I have a relatively simple configuration I'm testing out:
nfacctd_ip:a.b.c.d
nfacctd_port:2104
plugins: mongodb[all]
aggregate[all]: dst_host
mongo_table[all]: nfacctd_whk.hist_%Y_%m_%d
mongo_host[all]: e.f.g.h
That's a very highly simplified version of my final goal (I really
want nearly 1:1 packets into mongodb records eventually)
This mostly works, with the exception that the variables in
mongo_table are not properly substituted. Records get put into the
collection "hist_1969_12_31". This is obviously something getting
set to unixtime 0. nfacctd_time_new appears to exist for the purpose
where my netflow sender is not setting the proper date/time headers
(which, actually, I don't think is the case here) but even setting
nfacctd_time_new: true does not fix the issue. The collection is
still created with the wrong date.
If I include timestamp_start, timestamp_end in the aggregate
definition, the timestamps do appear correct in the resulting
mongodb entries.
{
"_id" : ObjectId("529ade3f70fd695100000013"),
"ip_dst" : "i.j.k.l",
"timestamp_start" : Timestamp(1385881117, 160000000),
"timestamp_end" : Timestamp(1385881117, 160000000),
"packets" : NumberLong(1),
"bytes" : NumberLong(52)
}
I am running:
NetFlow Accounting Daemon, nfacctd 1.5.0rc1 (20130829-00)
--enable-mongodb
Thanks,
-Adam
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
