Ok, but given your definition it's impossible to measure. For two reasons...
(1) The timestamps on the packets (the request and the ack) are generated on different hosts and there's no guarantee that they are even CLOSE in time-sync, let alone the same so you can subtract the timestamps. That leads to (2) We don't see when the packet arrives - the only piece of info we have is the timestamp from the sending host of when it was sent. I think you are right about tools measuring RTT. It might be possible, but it gets ugly to do anything more than an inception to date # because we can't store all the packets. As I said somewhere else, however, min/avg/max stddev is certainly feasible - maybe. We need to keep storage down, so we probably can't handle multi-threaded servers with long running processes... Conceptually we would store the last packet info, check the port and ack and - if it matches - compute the RTT for that packet, which would be added into the min/avg/max stddev #s. This would have more limits and so might be harder to explain than it would be worth. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Monday, April 03, 2006 10:49 AM To: [EMAIL PROTECTED]; [email protected] Subject: RE: [Ntop] -o disables latency stats? Thanks Chris. Glad to see I'm not crazy - at least about this issue anyway. I think there might be a symantics issue with "latency" and what I was thinking this statistic was showing. I would define "latency" as time on wire - purely end-to-end transport time, no application processing time. This will of course vary with load hence what will make it difficult to determine load/congestion induced delay from application processing time. I'm looking for a Sniffer Pro like ARM tool, which I think simply measures end-to-end RTT including application processing time. See other post on this. Thanks again for the help. Gary >>> [EMAIL PROTECTED] 4/3/2006 9:24:05 AM >>> I don't normally look at the Ntop latency stats - I use Smokeping where I need them - but just to validate (perhaps) what Gary is saying I tested it and I see the same thing. With -o NO latency figures show up. Without -o they do. At least some of them - not every session has a latency figure attached. Your point is well taken, Burton - it's not easy to determine latency from sniffing packets. But it does seem that -o does make a difference. Chris -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Burton Strauss Sent: Saturday, April 01, 2006 10:03 AM To: [email protected] Subject: RE: [Ntop] -o disables latency stats? Gary - WRT to latency other than at startup - once the application starts up, the best you can do would involve matching up two packets (in and out) and assuming (1) that they are related and (2) zero internal processing time. Neither of those is a good assumption, nor a good fit w/ ntop... To compute 5m rolling ping-pong times you would need to store the interarrival times for every packet (the standard formulas for computing mean, stddev etc w/o storage don't work) - that's going to be a huge increase in the per-host traffic structure size. As for the processing, say for sake of argument you are seeing a mysql host ... The select() statement is going to run for several 10s of ms. Then the various row retrievals will all have tiny processing overheads until the next select... Etc. A web server serving up a mix of static and dynamic pages would have the same problematic pattern. So is the 'right' answer the smallest? Maybe ... But that's basically the 3way handshake - which is already how we do it. A per session average/min/max/stddev might work ... But still, the minimum is going to almost certainly be the startup. What you are trying to pull out - application level response time - is almost impossible w/o a detailed understanding of the application - and a general purpose tools such as ntop isn't the right thing for this purpose. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Friday, March 31, 2006 3:25 PM To: [EMAIL PROTECTED]; [email protected] Subject: RE: [Ntop] -o disables latency stats? -m seems to be working OK. I have several large nets defined as local. In fact, mostly I'm concerned with local-local latency which are all part of my 10.x.x.x network defined with -m If latency is only during the handshake it's not as interesting to me as say... a rolling 5 min average of the session. I'd still like to see it work again, but if it represented the more of the session than simply the handshake that would be ideal. I'll add that to my wish list I sent you offline :) Thanks Burton. Have a great weekend! Gary >>> [EMAIL PROTECTED] 3/31/2006 3:12:32 PM >>> Latency is solely based on the handshake - that's the only time we can be reasonably sure that there's no processing occurring on the packet and hence we are actually seeing solely network (+ tcp/ip stack) latency... First thought - check your local subnet settings (-m) ... We only track sessions for 'nonFullyRemoteSession' (i.e. at least one side is local). -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Friday, March 31, 2006 1:39 PM To: [EMAIL PROTECTED]; [email protected] Subject: RE: [Ntop] -o disables latency stats? Sorry, I should've clarified what interface I was referring to. I'm running netflow and standard pcap. In theory they're both seeing the same .. traffic, so I 'm kinda comparing pros and cons of each method. The latency is interesting to me. I noticed on the netflow interface latency was always blank - I posted another question about that issue. This issue is concerning the "real" interface using pcap. Until I set the -t 5 -K and -o flags, I had latency stats. When I set these flags, no more latency. I guess it could be the K or t 5, but I was assuming it had more to do with the -o - maybe it was looking at interframe time between MACS. But, since you said it's at the TCP layer - then I don't know. What would you need from me to look into this further? Another question you made me think of: is the latency stat fixed and based solely on the handshake times - or is it a dynamic average over the life of the session (or some length, like last 5 mins) of the session? Gary >>> [EMAIL PROTECTED] 3/31/2006 12:24:44 PM >>> Latency is different for netFlow / non-netFlow For non-netFlow, it's based on the timestamps inside the packets, i.e. the SYN and SYN|ACK of the tcp 3-way handshake, from handleTCPSession() in session.c: if(tp->th_flags == TH_SYN) { theSession->nwLatency.tv_sec = h->ts.tv_sec; theSession->nwLatency.tv_usec = h->ts.tv_usec; theSession->sessionState = FLAG_STATE_SYN; } And then the block beginning: /* Latency measurement */ if((tp->th_flags == (TH_SYN|TH_ACK)) && (theSession->sessionState == FLAG_STATE_SYN)) { theSession->sessionState = FLAG_FLAG_STATE_SYN_ACK; } else if((tp->th_flags == TH_ACK) && (theSession->sessionState == FLAG_FLAG_STATE_SYN_ACK)) { if(h->ts.tv_sec >= theSession->nwLatency.tv_sec) { ... } For netFlow, it's just stored - IF it is part of the flow to begin with. There's nothing specific in the netFlow plugin WRT -o (myGlobals.runningPref.dontTrustMACaddr). You could turn on the debug line in netFlowPlugin.c: /* traceEvent(CONST_TRACE_INFO, "DEBUG: Nw Latency=%d.%d [%s:%d -> %s:%d]", record->nw_latency_sec, record->nw_latency_usec, srcHost->hostNumIpAddress, sport, dstHost->hostNumIpAddress, dport); */ (remove the /* and */ so it's no longer a comment) - that would show if it's even seeing the data (vs associating incorrectly. -----Burton -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary Gatten Sent: Thursday, March 30, 2006 4:08 PM To: [email protected] Subject: [Ntop] -o disables latency stats? FreeBSD 6.0, nTop 3.2.1, compiled from CVS - I think.... I'm collecting data through a cisco SPAN port. This port is a mirror of the our primary Frame-Relay WAN router interface. The idea is to see global WAN stats. Without -o I have latency stats, however, "all" the traffic gets associated with the router. Not good. Restarted with -o and I have all the individual hosts I wanted, but now there's no latency stats. Am I missing something, or is this just the way it is? Thanks! Gary _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop ********************************************************************** Confidential/Proprietary Note The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. Thank you. Guardian Mtg Documents, Inc. 225 Union Boulevard, Suite 200 Lakewood, CO 80228. ********************************************************************** _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
