Hi, I want to create statistics for every 5-minute interval based on a filter for use with something similar to RRDTool and I'd like to use nfdump to get it.
The simple solution is to just use the appropriately named 5-minute file nfcapd.201311281820, and run a filter on that. But I'm curious as to whether I'm missing a way to do this with the -t option or some other option. Naively, I tried -t 2013/11/28.18:10-2013/11/28.18:15 to get statistics about all the data in the 18:10 - 18:15 interval. But that returned zero flows. It looks like nfdump only includes flows where the entire period is within the to-from times in -t. See "details" below. (Would be nice if man nfdump went into a little more detail about -t) But, if some flows are short lived (e.g. webserver) and some are long-lived (e.g. ssh-connection) I don't see how I can use the -t option to get an idea of how much traffic occurred between 18:10 and 18:15. I guess I was hoping for a -t option that looked exclusively at e.g. flow end, or some way to use flow-end in a filter. Then I could get an idea about traffic in a 5minute period. Where long-running flows would be calculated as-if they occurred entirely at the flow-end time, but this is exactly what you get by looking at one nfcapd.* file at a time, isn't it? The Rolls Royce would be that if a flow ran from 18:09-18:13, 75% of the traffic from that flow would be added to the 18:10-18:15, because 75% of the time is within that period, but hey, I can see that's a little wild. I also tried to experiment with +/-10, but could not get this to work at all. What we *can* do is use only a single time for -t as in "-t 2013/11/28.18:05" and then look at every flow and discard any that end outside the 18:10-18:15 interval. But that is very time consuming especially because one needs to comparisons on time strings such as e.g. "2013-11-28 18:27:46.362". (It would be nice if there was something similar to -N that printed times as unixtime, so the heavy conversion (also in nfdump?) isn't necessary.) Can -t be used to get data from 18:10-18:15? Am I missing something (else) obvious? ( Appart from "just use nfsen" - I have other reasons not to, and I'm trying to understand -t in nfdump ) Sincerely, Peter ========== Details ========== I have a test nfcapd file[1], with NetFlow records from nothing but a single long-running ssh connection. Looking at all the data in the file, I get this: > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' Duration Date flow start Date flow end Src IP Addr Src Pt Dst IP Addr Dst Pt <snip> 300.716 2013-11-28 17:56:31.752 2013-11-28 18:01:32.468 1.2.3.4: 2222 172.22.216.119: 43654 300.710 2013-11-28 18:01:42.485 2013-11-28 18:06:43.195 1.2.3.4: 2222 172.22.216.119: 43654 300.796 2013-11-28 18:06:53.207 2013-11-28 18:11:54.003 1.2.3.4: 2222 172.22.216.119: 43654 310.813 2013-11-28 18:12:04.019 2013-11-28 18:17:14.832 1.2.3.4: 2222 172.22.216.119: 43654 <snip> Summary: total flows: 21, total bytes: 12.1 M, total packets: 33103, avg bps: 14730, avg pps: 5, avg bpp: 364 Time window: 2013-11-28 17:20:09 - 2013-11-28 19:09:22 Total flows processed: 21, Blocks skipped: 0, Bytes read: 1168 Sys: 0.008s flows/second: 2625.0 Wall: 0.000s flows/second: 30882.4 And experimenting with the -t option, I see that > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' -t 2013/11/28.18:12:04-2013/11/28.18:17:14 is the tightest I can go with -t and still get any data around that time period. Assuming I know that the NetFlow collector transmits every 5 minutes, I guess I could do -t 18:10-18:20 and then know that I'll likely only get one flow record from the long-running ssh connection, and then in the next period use 18:15-18:25. That would work most of the time (tm) for the ssh connection. But any short lived flows e.g. entirely inside 18:18 will be counted in both intervals. :-( About +/-10: How am I supposed to use this? > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' -t +10 Time Window error: No time slot information available > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' -t -10 Time Window error: No time slot information available > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' -t 2013/11/28.18:12:04+10 Time format error at '04+10': unexpected character: '+'. > nfdump -r singleSSH.nfcapd -o 'fmt: %td %ts %te %sa:%sp %da:%dp' -t 2013/11/28.18:12:04-10 Time format error: '10' unexpected. 1: attached and at http://ge.tt/5Rthyl41/v/0?c -- Peter Valdemar Mørch http://www.morch.com
singleSSH.nfcapd
Description: Binary data
------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
_______________________________________________ Nfdump-discuss mailing list Nfdump-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
