It would definitely take some work and be quite a bit of overhead. Perhaps it would be better suited as a plugin?
The tool I'm thinking of is like the ARM (Application Response Measurement) with Sniffer Pro. I don't know exactly how they do it, but it's very handy to monitor and alert you when a specific tcp/udp conversation is "slow" - based on whatever thresholds you define. Sniffer is just so much $$$ it's difficult to justify a dozen copies. If you were trying to seperate / account for network (on wire) delay as well as processing time, such as SQL lookups, etc. - it would involve a couple extra processes and of course more work. If however you were only concerned with the "gross" latency - that is the time of the request to the time of the response - it would be much easier. I THINK this is how ARM works, but I'm not certain. If you think this type of advanced latency / ARM monitor would be a good plugin, let me know and I'll add that to the list of items I'll try to get sponsored. Thanks! Gary >>> [EMAIL PROTECTED] 4/1/2006 11:02:31 AM >>> Gary - WRT to latency other than at startup - once the application starts up, the best you can do would involve matching up two packets (in and out) and assuming (1) that they are related and (2) zero internal processing time. Neither of those is a good assumption, nor a good fit w/ ntop... To compute 5m rolling ping-pong times you would need to store the interarrival times for every packet (the standard formulas for computing mean, stddev etc w/o storage don't work) - that's going to be a huge increase in the per-host traffic structure size. As for the processing, say for sake of argument you are seeing a mysql host ... The select() statement is going to run for several 10s of ms. Then the various row retrievals will all have tiny processing overheads until the next select... Etc. A web server serving up a mix of static and dynamic pages would have the same problematic pattern. So is the 'right' answer the smallest? Maybe ... But that's basically the 3way handshake - which is already how we do it. A per session average/min/max/stddev might work ... But still, the minimum is going to almost certainly be the startup. What you are trying to pull out - application level response time - is almost impossible w/o a detailed understanding of the application - and a general purpose tools such as ntop isn't the right thing for this purpose. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Friday, March 31, 2006 3:25 PM To: [EMAIL PROTECTED]; [email protected] Subject: RE: [Ntop] -o disables latency stats? -m seems to be working OK. I have several large nets defined as local. In fact, mostly I'm concerned with local-local latency which are all part of my 10.x.x.x network defined with -m If latency is only during the handshake it's not as interesting to me as say... a rolling 5 min average of the session. I'd still like to see it work again, but if it represented the more of the session than simply the handshake that would be ideal. I'll add that to my wish list I sent you offline :) Thanks Burton. Have a great weekend! Gary >>> [EMAIL PROTECTED] 3/31/2006 3:12:32 PM >>> Latency is solely based on the handshake - that's the only time we can be reasonably sure that there's no processing occurring on the packet and hence we are actually seeing solely network (+ tcp/ip stack) latency... First thought - check your local subnet settings (-m) ... We only track sessions for 'nonFullyRemoteSession' (i.e. at least one side is local). -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Friday, March 31, 2006 1:39 PM To: [EMAIL PROTECTED]; [email protected] Subject: RE: [Ntop] -o disables latency stats? Sorry, I should've clarified what interface I was referring to. I'm running netflow and standard pcap. In theory they're both seeing the same .. traffic, so I 'm kinda comparing pros and cons of each method. The latency is interesting to me. I noticed on the netflow interface latency was always blank - I posted another question about that issue. This issue is concerning the "real" interface using pcap. Until I set the -t 5 -K and -o flags, I had latency stats. When I set these flags, no more latency. I guess it could be the K or t 5, but I was assuming it had more to do with the -o - maybe it was looking at interframe time between MACS. But, since you said it's at the TCP layer - then I don't know. What would you need from me to look into this further? Another question you made me think of: is the latency stat fixed and based solely on the handshake times - or is it a dynamic average over the life of the session (or some length, like last 5 mins) of the session? Gary >>> [EMAIL PROTECTED] 3/31/2006 12:24:44 PM >>> Latency is different for netFlow / non-netFlow For non-netFlow, it's based on the timestamps inside the packets, i.e. the SYN and SYN|ACK of the tcp 3-way handshake, from handleTCPSession() in session.c: if(tp->th_flags == TH_SYN) { theSession->nwLatency.tv_sec = h->ts.tv_sec; theSession->nwLatency.tv_usec = h->ts.tv_usec; theSession->sessionState = FLAG_STATE_SYN; } And then the block beginning: /* Latency measurement */ if((tp->th_flags == (TH_SYN|TH_ACK)) && (theSession->sessionState == FLAG_STATE_SYN)) { theSession->sessionState = FLAG_FLAG_STATE_SYN_ACK; } else if((tp->th_flags == TH_ACK) && (theSession->sessionState == FLAG_FLAG_STATE_SYN_ACK)) { if(h->ts.tv_sec >= theSession->nwLatency.tv_sec) { ... } For netFlow, it's just stored - IF it is part of the flow to begin with. There's nothing specific in the netFlow plugin WRT -o (myGlobals.runningPref.dontTrustMACaddr). You could turn on the debug line in netFlowPlugin.c: /* traceEvent(CONST_TRACE_INFO, "DEBUG: Nw Latency=%d.%d [%s:%d -> %s:%d]", record->nw_latency_sec, record->nw_latency_usec, srcHost->hostNumIpAddress, sport, dstHost->hostNumIpAddress, dport); */ (remove the /* and */ so it's no longer a comment) - that would show if it's even seeing the data (vs associating incorrectly. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Thursday, March 30, 2006 4:08 PM To: [email protected] Subject: [Ntop] -o disables latency stats? FreeBSD 6.0, nTop 3.2.1, compiled from CVS - I think.... I'm collecting data through a cisco SPAN port. This port is a mirror of the our primary Frame-Relay WAN router interface. The idea is to see global WAN stats. Without -o I have latency stats, however, "all" the traffic gets associated with the router. Not good. Restarted with -o and I have all the individual hosts I wanted, but now there's no latency stats. Am I missing something, or is this just the way it is? Thanks! Gary _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
