Hi Neil,
On Jul 25, 2014, at 19:14 , Neil Davies <[email protected]> wrote: > Try this thesis - Lucian used this for work at CERN, the description is in > there.... (see one of the appendicies) Analysis and predictive modeling of > the performance of the ATLAS TDAQ network Sweet, thanks a lot. best regards sebastian > > > On 25 Jul 2014, at 18:12, Sebastian Moeller <[email protected]> wrote: > >> Hello Martin, >> >> thanks a lot. >> >> On Jul 25, 2014, at 18:32 , Martin Geddes <[email protected]> wrote: >> >>> So what is ΔQ and how do you "compute" it (to the extent it is a "computed" >>> thing)? >>> >>> Starting point: the only observable effect of a network is to lose and >>> delay data -- i.e. to "attenuate quality" by adding the toxic effects of >>> time to distributed computations. ΔQ is a morphism that relates the >>> "quality attenuation" that the network imposes to the application >>> performance, and describes the trading spaces at all intermediate layers of >>> abstraction. It is shown in the attached graphic. >>> >>> Critically, it frames quality as something that can only be lost >>> ("attenuated"), both by the network and the application. Additionally, it >>> is stochastic, and works with random variables and distributions. >>> >>> At its most concrete level, it is the individual impairment encountered by >>> every packet when the network in operation. But we don't want to have to >>> track every packet - 1:1 scale maps are pretty useless. So we need to >>> abstract that in order to create a model that has value. >>> >>> Next abstraction: an improper random variable. This unifies loss and delay >>> into a single stochastic object. >>> Next abstraction: received transport, which is a CDF where we are >>> interested in the properties of the "tail". >>> >>> Next abstraction, that joins network performance and application QoE (as >>> relates to performance): relate the CDF to the application through a >>> Quality Transport Agreement. This "stochastic contract" is both necessary >>> and sufficient to deliver the application outcome. >>> >>> Next concretisation towards QoE: offered load of demand, as a CDF. >>> Next concretisation towards QoE: breach hazard metric, which abstracts the >>> application performance. Indicates the likelihood of the QTA contract being >>> broken, and how badly. >>> Final concretisation: the individual application performance encountered by >>> every user. Again, a 1:1 map that isn't very helpful. >>> >>> So as you can see, it's about as far away from a single point average >>> metric as you can possibly get. A far richer model is required in order to >>> achieve robust performance engineering. >>> >>> It is "computed" using multi-point measurements to capture the >>> distribution. The G/S/V charts you see are based on processing that data to >>> account for various issues, including clock skew. >>> >>> I hope that helps. We need to document more of this in public, which is an >>> ongoing process. >> >> You lost me, I think what I should have asked for is a real example >> with numbers and the formulas ;) I guess that is deep in “secret sauce” >> territory. Alas, if that should be true it also means that deltaQ is not >> going to help me understand my network any better … >> >> Best Regards >> Sebastian >> >> >>> >>> Martin >>> >>> On 25 July 2014 16:58, Sebastian Moeller <[email protected]> wrote: >>> Hi Martin, >>> >>> thanks for the pointers, >>> >>> >>> On Jul 25, 2014, at 16:25 , Martin Geddes <[email protected]> wrote: >>> >>> > You may find the following useful background reading on the state of the >>> > art in network measurement, and a primer on ΔQ (which is the property we >>> > wish to measure). >>> > >>> > First, start with this presentation: Network performance optimisation >>> > using high-fidelity measures >>> > Then read this one to decompose ΔQ into G, S and V: Fundamentals of >>> > network performance engineering >>> > Then read this one to get a bit more sense on what ΔQ is about: >>> > Introduction to ΔQ and Network Performance Science (extracts) >>> > >>> > Then read these essays: >>> > >>> > Foundation of Network Science >>> > How to do network performance chemistry >>> > How to X-ray a telecoms network >>> > There is no quality in averages: IPX case study >>> >>> All of this makes intuitively sense, but it is a bit light on how >>> deltaQ is to be computed ;). >>> As far as I understand it also has not much bearing on my home >>> network; the only one under my control. Now, following the buffer bloat >>> discussion for some years, I have internalized the idea that bandwidth >>> alone does not suffice to describe the quality of my network connection. I >>> think that the latency increase under load (for unrelated flows) is the >>> best of all the bad single number measures of network dynamics/quality. I >>> should be related to what I understood deltaQ to depend on (as packet loss >>> for non real time flows will cause an increase in latency). I think that >>> continuous measurements make a to n of sense for ISPs, backbone-operators, >>> mobile carriers … but at home, basically, I operate as my own network >>> quality monitor ;) (that is I try to pin point and debug (transient) >>> anomalies). >>> >>> > >>> > Martin >>> > >>> > For fresh thinking about telecoms sign up for my free newsletter or visit >>> > the Geddes Think Tank. >>> > LinkedIn Twitter Mobile: +44 7957 499219 Skype: mgeddes >>> > Martin Geddes Consulting Ltd, Incorporated in Scotland, number SC275827 >>> > VAT Number: 859 5634 72 Registered office: 17-19 East London Street, >>> > Edinburgh, EH7 4BN >>> > >>> > >>> > >>> > On 25 July 2014 15:17, Sebastian Moeller <[email protected]> wrote: >>> > Hi Neil, >>> > >>> > >>> > On Jul 25, 2014, at 14:24 , Neil Davies <[email protected]> wrote: >>> > >>> > > Rich >>> > > >>> > > I have a deep worry over this style of single point measurement - and >>> > > hence speed - as an appropriate measure. >>> > >>> > But how do you propose to measure the (bottleneck) link capacity >>> > then? It turns out for current CPE and CMTS/DSLAM equipment one typically >>> > can not relay on good QoE out of the box, since typically these devices >>> > do not use their (largish) buffers wisely. Instead the current remedy is >>> > to take back control over the bottleneck link by shaping the actually >>> > sent traffic to stay below the hardware link capacity thereby avoiding >>> > feeling the consequences of the over-buffering. But to do this is is >>> > quite helpful to get an educated guess what the bottleneck links capacity >>> > actually is. And for that purpose a speediest seems useful. >>> > >>> > >>> > > We know, and have evidence, that throughput/utilisation is not a good >>> > > proxy for the network delivering suitable quality of experience. We >>> > > work with organisation (Telco’s, large system integrators etc) where we >>> > > spend a lot of time having to “undo” the consequences of “maximising >>> > > speed”. Just like there is more to life than work, there is more to QoE >>> > > than speed. >>> > > >>> > > For more specific comments see inline >>> > > >>> > > On 25 Jul 2014, at 13:09, Rich Brown <[email protected]> wrote: >>> > > >>> > >> Neil, >>> > >> >>> > >> Thanks for the note and the observations. My thoughts: >>> > >> >>> > >> 1) I note that speedof.me does seem to overstate the speed results. At >>> > >> my home, it reports 5.98mbps down, and 638kbps up, while >>> > >> betterspeedtest.sh shows 5.49/0.61 mbps. (speedtest.net gives numbers >>> > >> similar to the betterspeedtest.net script.) >>> > >> >>> > >> 2) I think we're in agreement about the peak upload rate that you >>> > >> point out is too high. Their measurement code runs in the browser. It >>> > >> seems likely that the browser pumps out a few big packets before >>> > >> getting flow control information, thus giving the impression that they >>> > >> can send at a higher rate. This comports with the obvious decay that >>> > >> ramps toward the long-term rate. >>> > > >>> > > I think that its simpler than that, it is measuring the rate at which >>> > > it can push packets out the interface - its real time rate is precisely >>> > > that - it can not be the rate being reported by the far end, it can >>> > > never exceed the limiting link. The long term average (if it is like >>> > > other speed testers we’ve had to look into) is being measured at the >>> > > TCP/IP SDU level by measuring the difference in time between the first >>> > > and last timestamps of data stream and dividing that into the total >>> > > data sent. Their “over-estimate” is because there are packets buffered >>> > > in the CPE that have left the machine but not arrived at the far end. >>> > >>> > Testing from an openwrt router located at a >>> > high-symmetric-bandwidth location shows that speedof.me does not scale >>> > higher than ~ 130 Mbps server to client and ~15Mbps client to server (on >>> > the same connection I can get 130Mbps S2C and ~80Mbps C2S, so the >>> > asymmetry in the speedof.me results is not caused by my local >>> > environment). >>> > @Rich and Dave, this probably means that for the upper end of >>> > fiber and cable and VDSL connections speed of.me is not going to be a >>> > reliable speed measure… Side note www.sppedtest.net shows ~100Mbps S2C >>> > and ~100Mbps C2S, so might be better suited to high-upload links... >>> > >>> > > >>> > >> >>> > >> 3) But that long-term speed should be at or below the theoretical >>> > >> long-term rate, not above it. >>> > > >>> > > Agreed, but in this case knowing the sync rate already defines that >>> > > maximum. >>> > >>> > I fully agree, but for ADSL the sync rate also contains a lot of >>> > encapsulation, so the maximum achievable TCP rate is at best ~90% of link >>> > rate. Note for cerowrt’s SQM system the link rate is exactly the right >>> > number to start out with at that system can take the encapsulation into >>> > account. But even then it is somewhat unintuitive to deduce the expected >>> > good-put from the link rate. >>> > >>> > > >>> > >> >>> > >> Two experiments for you to try: >>> > >> >>> > >> a) What does betterspeedtest.sh show? (It's in the latest CeroWrt, in >>> > >> /usr/lib/CeroWrtScripts, or get it from github: >>> > >> https://github.com/richb-hanover/CeroWrtScripts ) >>> > >> >>> > >> b) What does www.speedtest.net show? >>> > >> >>> > >> I will add your question (about the inaccuracy) to the note that I >>> > >> want to send out to speedof.me this weekend. I will also ask that they >>> > >> include min/max latency measurements to their test, and an option to >>> > >> send for > 10 seconds to minimize any effect of PowerBoost… >>> > >>> > I think they do already, at least for the download bandwidth; >>> > they start with 128Kb and keep doubling the file size until a file takes >>> > longer than 8 seconds to transfer, they only claim to report the numbers >>> > from that last transferred file, so worst case with a stable link and a >>> > bandwidth > 16kbps ;), it has taken at least 12 seconds (4 plus 8) of >>> > measuring before the end of the plot, so the bandwidth of at least the >>> > last half of the download plot should be representative even assuming >>> > power boost. Caveat, I assume that power boost will not be reset by the >>> > transient lack of data transfer between the differently sized files (but >>> > since it should involve the same IPs and port# why should power boost >>> > reset itself?). >>> > >>> > Best Regards >>> > Sebastian >>> > >>> > >>> > >>> > >> >>> > >> Best regards, >>> > >> >>> > >> Rich >>> > >> >>> > >> >>> > >> >>> > >> On Jul 25, 2014, at 5:10 AM, Neil Davies <[email protected]> wrote: >>> > >> >>> > >>> Rich >>> > >>> >>> > >>> You may want to check how accurate they are to start. >>> > >>> >>> > >>> I just ran a “speed test” on my line (which I have complete control >>> > >>> and visibility over the various network elements) and it reports an >>> > >>> average “speed” (in the up direction) that is in excess of the >>> > >>> capacity of the line, it reports the maximum rate at nearly twice the >>> > >>> best possible rate of the ADSL connection. >>> > >>> >>> > >>> Doesn’t matter how pretty it is, if its not accurate it is of no use. >>> > >>> This is rather ironic as the web site claims it is the “smartest and >>> > >>> most accurate”! >>> > >>> >>> > >>> Neil >>> > >>> >>> > >>> <speedof_me_14-07-25.png> >>> > >>> >>> > >>> PS pretty clear to me what mistake they’ve made in the measurement >>> > >>> process - its to do with incorrect inference and hence missing the >>> > >>> buffering effects. >>> > >>> >>> > >>> On 20 Jul 2014, at 14:19, Rich Brown <[email protected]> wrote: >>> > >>> >>> > >>>> Doc Searls >>> > >>>> (http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need-to-climb/) >>> > >>>> mentioned in passing that he uses a new speed test website. I >>> > >>>> checked it out, and it was very cool… >>> > >>>> >>> > >>>> www.speedof.me is an all-HTML5 website that seems to make accurate >>> > >>>> measurements of the up and download speeds of your internet >>> > >>>> connection. It’s also very attractive, and the real-time plots of >>> > >>>> the speed show interesting info. (screen shot at: >>> > >>>> http://richb-hanover.com/speedof-me/) >>> > >>>> >>> > >>>> Now if we could get them to a) allow longer/bigger tests to >>> > >>>> circumvent PowerBoost, and b) include a latency measurement so >>> > >>>> people could point out their bufferbloated equipment. >>> > >>>> >>> > >>>> I'm going to send them a note. Anything else I should add? >>> > >>>> >>> > >>>> Rich >>> > >>>> _______________________________________________ >>> > >>>> Bloat mailing list >>> > >>>> [email protected] >>> > >>>> https://lists.bufferbloat.net/listinfo/bloat >>> > >>> >>> > >> >>> > > >>> > > _______________________________________________ >>> > > Bloat mailing list >>> > > [email protected] >>> > > https://lists.bufferbloat.net/listinfo/bloat >>> > >>> > _______________________________________________ >>> > Bloat mailing list >>> > [email protected] >>> > https://lists.bufferbloat.net/listinfo/bloat >>> > >>> >>> >>> <ΔQ morphism.png> >> > _______________________________________________ Bloat mailing list [email protected] https://lists.bufferbloat.net/listinfo/bloat
