Hi Martin,
thanks for the pointers,
On Jul 25, 2014, at 16:25 , Martin Geddes <m...@martingeddes.com> wrote:
> You may find the following useful background reading on the state of the art
> in network measurement, and a primer on ΔQ (which is the property we wish to
> measure).
>
> First, start with this presentation: Network performance optimisation using
> high-fidelity measures
> Then read this one to decompose ΔQ into G, S and V: Fundamentals of network
> performance engineering
> Then read this one to get a bit more sense on what ΔQ is about: Introduction
> to ΔQ and Network Performance Science (extracts)
>
> Then read these essays:
>
> Foundation of Network Science
> How to do network performance chemistry
> How to X-ray a telecoms network
> There is no quality in averages: IPX case study
All of this makes intuitively sense, but it is a bit light on how
deltaQ is to be computed ;).
As far as I understand it also has not much bearing on my home network;
the only one under my control. Now, following the buffer bloat discussion for
some years, I have internalized the idea that bandwidth alone does not suffice
to describe the quality of my network connection. I think that the latency
increase under load (for unrelated flows) is the best of all the bad single
number measures of network dynamics/quality. I should be related to what I
understood deltaQ to depend on (as packet loss for non real time flows will
cause an increase in latency). I think that continuous measurements make a to
n of sense for ISPs, backbone-operators, mobile carriers … but at home,
basically, I operate as my own network quality monitor ;) (that is I try to pin
point and debug (transient) anomalies).
>
> Martin
>
> For fresh thinking about telecoms sign up for my free newsletter or visit the
> Geddes Think Tank.
> LinkedIn Twitter Mobile: +44 7957 499219 Skype: mgeddes
> Martin Geddes Consulting Ltd, Incorporated in Scotland, number SC275827 VAT
> Number: 859 5634 72 Registered office: 17-19 East London Street, Edinburgh,
> EH7 4BN
>
>
>
> On 25 July 2014 15:17, Sebastian Moeller <moell...@gmx.de> wrote:
> Hi Neil,
>
>
> On Jul 25, 2014, at 14:24 , Neil Davies <neil.dav...@pnsol.com> wrote:
>
> > Rich
> >
> > I have a deep worry over this style of single point measurement - and hence
> > speed - as an appropriate measure.
>
> But how do you propose to measure the (bottleneck) link capacity
> then? It turns out for current CPE and CMTS/DSLAM equipment one typically can
> not relay on good QoE out of the box, since typically these devices do not
> use their (largish) buffers wisely. Instead the current remedy is to take
> back control over the bottleneck link by shaping the actually sent traffic to
> stay below the hardware link capacity thereby avoiding feeling the
> consequences of the over-buffering. But to do this is is quite helpful to get
> an educated guess what the bottleneck links capacity actually is. And for
> that purpose a speediest seems useful.
>
>
> > We know, and have evidence, that throughput/utilisation is not a good proxy
> > for the network delivering suitable quality of experience. We work with
> > organisation (Telco’s, large system integrators etc) where we spend a lot
> > of time having to “undo” the consequences of “maximising speed”. Just like
> > there is more to life than work, there is more to QoE than speed.
> >
> > For more specific comments see inline
> >
> > On 25 Jul 2014, at 13:09, Rich Brown <richb.hano...@gmail.com> wrote:
> >
> >> Neil,
> >>
> >> Thanks for the note and the observations. My thoughts:
> >>
> >> 1) I note that speedof.me does seem to overstate the speed results. At my
> >> home, it reports 5.98mbps down, and 638kbps up, while betterspeedtest.sh
> >> shows 5.49/0.61 mbps. (speedtest.net gives numbers similar to the
> >> betterspeedtest.net script.)
> >>
> >> 2) I think we're in agreement about the peak upload rate that you point
> >> out is too high. Their measurement code runs in the browser. It seems
> >> likely that the browser pumps out a few big packets before getting flow
> >> control information, thus giving the impression that they can send at a
> >> higher rate. This comports with the obvious decay that ramps toward the
> >> long-term rate.
> >
> > I think that its simpler than that, it is measuring the rate at which it
> > can push packets out the interface - its real time rate is precisely that -
> > it can not be the rate being reported by the far end, it can never exceed
> > the limiting link. The long term average (if it is like other speed testers
> > we’ve had to look into) is being measured at the TCP/IP SDU level by
> > measuring the difference in time between the first and last timestamps of
> > data stream and dividing that into the total data sent. Their
> > “over-estimate” is because there are packets buffered in the CPE that have
> > left the machine but not arrived at the far end.
>
> Testing from an openwrt router located at a high-symmetric-bandwidth
> location shows that speedof.me does not scale higher than ~ 130 Mbps server
> to client and ~15Mbps client to server (on the same connection I can get
> 130Mbps S2C and ~80Mbps C2S, so the asymmetry in the speedof.me results is
> not caused by my local environment).
> @Rich and Dave, this probably means that for the upper end of fiber
> and cable and VDSL connections speed of.me is not going to be a reliable
> speed measure… Side note www.sppedtest.net shows ~100Mbps S2C and ~100Mbps
> C2S, so might be better suited to high-upload links...
>
> >
> >>
> >> 3) But that long-term speed should be at or below the theoretical
> >> long-term rate, not above it.
> >
> > Agreed, but in this case knowing the sync rate already defines that maximum.
>
> I fully agree, but for ADSL the sync rate also contains a lot of
> encapsulation, so the maximum achievable TCP rate is at best ~90% of link
> rate. Note for cerowrt’s SQM system the link rate is exactly the right number
> to start out with at that system can take the encapsulation into account. But
> even then it is somewhat unintuitive to deduce the expected good-put from the
> link rate.
>
> >
> >>
> >> Two experiments for you to try:
> >>
> >> a) What does betterspeedtest.sh show? (It's in the latest CeroWrt, in
> >> /usr/lib/CeroWrtScripts, or get it from github:
> >> https://github.com/richb-hanover/CeroWrtScripts )
> >>
> >> b) What does www.speedtest.net show?
> >>
> >> I will add your question (about the inaccuracy) to the note that I want to
> >> send out to speedof.me this weekend. I will also ask that they include
> >> min/max latency measurements to their test, and an option to send for > 10
> >> seconds to minimize any effect of PowerBoost…
>
> I think they do already, at least for the download bandwidth; they
> start with 128Kb and keep doubling the file size until a file takes longer
> than 8 seconds to transfer, they only claim to report the numbers from that
> last transferred file, so worst case with a stable link and a bandwidth >
> 16kbps ;), it has taken at least 12 seconds (4 plus 8) of measuring before
> the end of the plot, so the bandwidth of at least the last half of the
> download plot should be representative even assuming power boost. Caveat, I
> assume that power boost will not be reset by the transient lack of data
> transfer between the differently sized files (but since it should involve the
> same IPs and port# why should power boost reset itself?).
>
> Best Regards
> Sebastian
>
>
>
> >>
> >> Best regards,
> >>
> >> Rich
> >>
> >>
> >>
> >> On Jul 25, 2014, at 5:10 AM, Neil Davies <neil.dav...@pnsol.com> wrote:
> >>
> >>> Rich
> >>>
> >>> You may want to check how accurate they are to start.
> >>>
> >>> I just ran a “speed test” on my line (which I have complete control and
> >>> visibility over the various network elements) and it reports an average
> >>> “speed” (in the up direction) that is in excess of the capacity of the
> >>> line, it reports the maximum rate at nearly twice the best possible rate
> >>> of the ADSL connection.
> >>>
> >>> Doesn’t matter how pretty it is, if its not accurate it is of no use.
> >>> This is rather ironic as the web site claims it is the “smartest and most
> >>> accurate”!
> >>>
> >>> Neil
> >>>
> >>> <speedof_me_14-07-25.png>
> >>>
> >>> PS pretty clear to me what mistake they’ve made in the measurement
> >>> process - its to do with incorrect inference and hence missing the
> >>> buffering effects.
> >>>
> >>> On 20 Jul 2014, at 14:19, Rich Brown <richb.hano...@gmail.com> wrote:
> >>>
> >>>> Doc Searls
> >>>> (http://blogs.law.harvard.edu/doc/2014/07/20/the-cliff-peronal-clouds-need-to-climb/)
> >>>> mentioned in passing that he uses a new speed test website. I checked
> >>>> it out, and it was very cool…
> >>>>
> >>>> www.speedof.me is an all-HTML5 website that seems to make accurate
> >>>> measurements of the up and download speeds of your internet connection.
> >>>> It’s also very attractive, and the real-time plots of the speed show
> >>>> interesting info. (screen shot at: http://richb-hanover.com/speedof-me/)
> >>>>
> >>>> Now if we could get them to a) allow longer/bigger tests to circumvent
> >>>> PowerBoost, and b) include a latency measurement so people could point
> >>>> out their bufferbloated equipment.
> >>>>
> >>>> I'm going to send them a note. Anything else I should add?
> >>>>
> >>>> Rich
> >>>> _______________________________________________
> >>>> Bloat mailing list
> >>>> bl...@lists.bufferbloat.net
> >>>> https://lists.bufferbloat.net/listinfo/bloat
> >>>
> >>
> >
> > _______________________________________________
> > Bloat mailing list
> > bl...@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
> _______________________________________________
> Bloat mailing list
> bl...@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel
