Gents, This is not actually true - you can measure one-way delays without completely accurately synchronised clocks (they have to be reasonably precise, not accurate) - see CERN thesis at http://goo.gl/ss6EBq
It is possible, with appropriate measurements, to construct arguments that make marketeers salivate (or the appropriate metaphor) - you can compare the relative effects of technology, location and instantaneous congestion. See slideshare at http://goo.gl/6vytmD Neil On 14 Sep 2014, at 00:32, Jonathan Morton <[email protected]> wrote: >>>> When reading it, it strikes me, that you don't directly tell them what to >>>> do; e.g. add a latency test during upload and download. ... >>> >>> Does round trip latency have enough info, or do you need to know how much >>> is >>> contributed by each direction? >> >> RTT is fine, uni-directional transfer time would be too good to be true ;). > > To expand on this: to measure one-way delay, you would need finely > synchronised clocks (to within a couple of ms) at both ends. The average > consumer doesn't have that sort of setup - not even if they happen to use > NTP. So it's not a goal worth pursuing at this level - save it for > scientific investigations, where the necessary effort and equipment can be > made available. > >>> If I gave you a large collection of latency data from a test run, how do >>> you >>> reduce it to something simple that a marketer could compare with the >>> results >>> from another test run? >> >> I believe the added latency under load would be a marketable number, >> but we had a discussion in the past where it was argued that marketing wants >> a number which increases with goodness, so larger = better, something the >> raw difference is not going to deliver… > > The obvious solution is to report the inverse value in Hz, a figure of merit > that gamers are familiar with (if not exactly in this context). > > For example, I occasionally play World Of Tanks, which has a latency meter > (in ms) in the top corner, and I can roughly invert that in my head - if I > set my shaper too optimistically, I get something like 500ms if something is > updating in the background, but this goes down immediately to 100ms once I > adjust the setting to a more conservative value - it's a 3G connection, so > it's not going to get much better than that. The corresponding inverted > readings would be 2Hz (where I miss most of my shots) and 10Hz (where I can > carry the team). It's probably worth reporting to one decimal place. > > WoT isn't exactly the "twitchiest" of online games, either - have you any > idea how long it takes to aim a tank gun? Even so, when some tanks can move > at over 30km/h, a half-second difference in position is a whole tank length, > so with the slower response I no longer know *where* or *when* to fire, > unless the target is a sitting duck. Even though my framerate is at 30Hz or > more and appears smooth, my performance as a player is dependent on the > Internet's response frequency, because that is lower. > > > So here's the outline of a step-by-step methodology: > > - Prepare space for a list of latency measurements. Each measurement needs > to be tagged with information about what else was going on at the same time, > ie. idle/upload/download/both. Some latency probes may be lost, and this > fact should also be recorded on a per-tag basis. > > - Start taking latency measurements, tagging as idle to begin with. Keep on > doing so continuously, changing the tag as required, until several seconds > after the bandwidth measurements are complete. > > - Latency measurements should be taken sufficiently frequently (several times > a second is okay) that there will be at least a hundred samples with each > tag, and the frequency of sampling should not change during the test. Each > probe must be tagged with a unique ID, so that losses or re-ordering of > packets can be detected and don't confuse the measurement. > > - The latency probes should use UDP, not ICMP. They should also use the same > Diffserv/TOS tag as the bandwidth measurement traffic; the default > "best-effort" tag is fine. > > - To satisfy the above requirements, the latency tester must *not* wait for a > previous reply to return before sending the next one. It should send at > regular intervals based on wall-clock time. But don't send so many probes > that they themselves clog the link. > > - Once several seconds of "idle" samples are recorded, start the download > test. Change the tag to "download" at this point. > > - The download test is complete when all the data sent by the server has > reached the client. Change the tag back to "idle" at this moment. > > - Repeat the previous two steps for the upload measurement, using the > "upload" tag. > > - Repeat again, but perform upload and download tests at the same time > (predicting, if necessary, that the bandwidth in each direction should be > similar to that previously measured), and use the "both" tag. Uploads and > downloads tend to interfere with each other when the loaded response > frequency is poor, so don't simply assume that the results will be the same > as in the individual tests - *measure* it. > > - Once a few seconds of "idle" samples have been taken, stop measuring and > start analysis. > > - Separate the list of latency samples by tag, and sort the four resulting > lists in ascending order. > > - In each list, find the sample nearest 98% of the way through the list. > This is the "98th percentile", a good way of finding the "highest" value > while discarding irrelevant outliers. The highest latency is the one that > users will notice. Typically poor results: idle 50ms, download 250ms, upload > 500ms, both 800ms. > > - Correct the 98-percentile latencies for packet loss, by multiplying it by > the number of probes *sent* with the appropriate tag, and then dividing it by > the number of probes *received* with that tag. It is not necessary to report > packet loss in any other way, *except* for the "idle" tag. > > - Convert the corrected 98-percentile latencies into "response frequencies" > by dividing one second by them. The typical figures above would become: idle > 20.0 Hz, download 4.0 Hz, upload 2.0 Hz, both 1.25 Hz - assuming there was no > packet loss. These figures are comparable in meaning and importance to > "frames per second" figures in games. > > - Report these response frequencies, to a precision of at least one decimal > place, alongside and with equal visual importance to, the bandwidth figures. > For example: > > IDLE: Response 20.0 Hz Packet loss 0.00 % > DOWNLOAD: Response 4.0 Hz Bandwidth 20.00 Mbps > UPLOAD: Response 2.0 Hz Bandwidth 2.56 Mbps > BIDIRECT: Response 1.3 Hz Bandwidth 15.23 / 2.35 Mbps > > - Improving the response figures in the loaded condition will probably also > improve the *bidirectional* bandwidth figures as a side-effect, while having > a minimal effect on the *unidirectional* figures. A connection with such > qualities can legitimately be described as supporting multiple activities at > the same time. A connection with the example figures shown above can *not*. > > > The next trick is getting across the importance of acknowledging that more > than one person in the average household wants to use the Internet at the > same time these days, and they often want to do different things from each > other. In this case, the simplest valid argument probably has a lot going > for it. > > An illustration might help to get the concept across. A household with four > users in different rooms: father in the study downloading a video, mother in > the kitchen on the (VoIP) phone, son in his bedroom playing a game, daughter > in her bedroom uploading photos. All via a single black-box modem and > connection. Captions should emphasise that mother and son both need low > latency (high response frequency), while father and daughter need high > bandwidth (in opposite directions!), and that they're doing all these things > at the same time. > > - Jonathan Morton > > _______________________________________________ > Bloat mailing list > [email protected] > https://lists.bufferbloat.net/listinfo/bloat _______________________________________________ Bloat mailing list [email protected] https://lists.bufferbloat.net/listinfo/bloat
