[Bloat] RFC: Realtime Response Under Load (rrul) test specification
I have been working on developing a specification for testing networks more effectively for various side effects of bufferbloat, notably gaming and voip performance, and especially web performance as well as a few other things that concerned me, such as IPv6 behavior, and the effects of packet classification. A key goal is to be able to measure the quality of the user experience while a network is otherwise busy, with complex stuff going on in the background, but with a simple presentation of the results in the end, in under 60 seconds. While it's not done yet, it escaped into the wild today, and I might as well solicit wider opinions on it, sooo... get the spec at: https://github.com/dtaht/deBloat/blob/master/spec/rrule.doc?raw=true Portions of the test are being prototyped in the netperf-wrappers repo on github. The initial results of the rrul test on several hotel networks I've tried it on are interesting. Example: http://www.teklibre.com/~d/rrul2_conference.pdf A major sticking point at the moment is to come up with an equivalent of the chrome-benchmarks for measuring relative web page performance with and without a network load, or to merely incorporate some automated form of that benchmark into the overall test load. The end goal is to have a complex, comprehensive benchmark of some core networking issues, that produces simple results, whether they be via a java tool like icsi's, or via flash on the web, or the command line, via something like netperf. Related resources: netperf 2.6 or later running on a fairly nearby server https://github.com/tohojo/netperf-wrapper python-matplotlib I look forward to your comments. -- Dave Täht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] RFC: Realtime Response Under Load (rrul) test specification
On Tue, 06 Nov 2012, Dave Taht wrote: I have been working on developing a specification for testing networks more effectively for various side effects of bufferbloat, notably gaming and voip performance, and especially web performance as well as a few other things that concerned me, such as IPv6 behavior, and the effects of packet classification. When it is reasonably complete, it would be nice to have it as an informational or better yet, standards-track IETF RFC. IETF RFC non-experimental status allows us to require RRUL testing prior to service acceptance, and even add it as one of the SLA metrics on public tenders, which goes a long way into pushing anything into more widespread usage. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] RFC: Realtime Response Under Load (rrul) test specification
On Tue, Nov 6, 2012 at 2:42 PM, Henrique de Moraes Holschuh h...@hmh.eng.br wrote: On Tue, 06 Nov 2012, Dave Taht wrote: I have been working on developing a specification for testing networks more effectively for various side effects of bufferbloat, notably gaming and voip performance, and especially web performance as well as a few other things that concerned me, such as IPv6 behavior, and the effects of packet classification. When it is reasonably complete, it would be nice to have it as an informational or better yet, standards-track IETF RFC. IETF RFC non-experimental status allows us to require RRUL testing prior to service acceptance, and even add it as one of the SLA metrics on public tenders, which goes a long way into pushing anything into more widespread usage. It was my intent to write this as a real, standards track rfc, and also submit it as a prospective test to the ITU and other testing bodies such as nist, undewriter labratories, consumer reports, and so on. However I: A) got intimidated by the prospect of dealing with the rfc editor B) Have some sticky problems with two aspects of the test methodology (and that's just what I know about) which I am prototyping around. Running the prototype tests on various real networks has had very interesting results... (I do hope others try the prototype tests, too, on their networks) C) thought it would be clearer to write the shortest document possible on this go-round. D) Am not particularly fond of the rrule name. (suggestions?) I now plan (after feedback) to produce and submit this as a standards track RFC in the march timeframe. It would give me great joy to have this test series included in various SLA metrics, in the long run. -- Dave Täht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Codel] RFC: Realtime Response Under Load (rrul) test specification
On 11/6/2012 8:56 AM, Dave Taht wrote: On Tue, Nov 6, 2012 at 2:42 PM, Henrique de Moraes Holschuh h...@hmh.eng.br wrote: On Tue, 06 Nov 2012, Dave Taht wrote: I have been working on developing a specification for testing networks more effectively for various side effects of bufferbloat, notably gaming and voip performance, and especially web performance as well as a few other things that concerned me, such as IPv6 behavior, and the effects of packet classification. When it is reasonably complete, it would be nice to have it as an informational or better yet, standards-track IETF RFC. IETF RFC non-experimental status allows us to require RRUL testing prior to service acceptance, and even add it as one of the SLA metrics on public tenders, which goes a long way into pushing anything into more widespread usage. It was my intent to write this as a real, standards track rfc, and also submit it as a prospective test to the ITU and other testing bodies such as nist, undewriter labratories, consumer reports, and so on. However I: A) got intimidated by the prospect of dealing with the rfc editor B) Have some sticky problems with two aspects of the test methodology (and that's just what I know about) which I am prototyping around. Running the prototype tests on various real networks has had very interesting results... (I do hope others try the prototype tests, too, on their networks) C) thought it would be clearer to write the shortest document possible on this go-round. D) Am not particularly fond of the rrule name. (suggestions?) I now plan (after feedback) to produce and submit this as a standards track RFC in the march timeframe. It would give me great joy to have this test series included in various SLA metrics, in the long run. Hi Dave, in my role as IETF TSV AD, I would be happy to help you get this into the IETF. Please note that you can't get a Standards Track RFC published without a working group adopting it or an AD sponsoring it. This topic would be of interest for the IPPM and BMWG working groups, and I know it is of interest to me as a TSV AD, so we should be able to find a way to bring it in. In fact, the timing is good, as FCC folks are at the IETF this week talking about their vision for broadband test and measurement architecture, and these tests may relate nicely to that proposed work. -- Wes Eddy MTI Systems ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Codel] RFC: Realtime Response Under Load (rrul) test specification
On 11/06/2012 04:42 AM, Dave Taht wrote: I have been working on developing a specification for testing networks more effectively for various side effects of bufferbloat, notably gaming and voip performance, and especially web performance as well as a few other things that concerned me, such as IPv6 behavior, and the effects of packet classification. A key goal is to be able to measure the quality of the user experience while a network is otherwise busy, with complex stuff going on in the background, but with a simple presentation of the results in the end, in under 60 seconds. Would you like fries with that? Snark aside, I think that being able to capture the state of the user experience in only 60 seconds is daunting at best. Especially if this testing is going to run over the Big Bad Internet (tm) rather than in a controlled test lab. While it's not done yet, it escaped into the wild today, and I might as well solicit wider opinions on it, sooo... get the spec at: https://github.com/dtaht/deBloat/blob/master/spec/rrule.doc?raw=true Github is serving that up as a plain text file, which then has Firefox looking to use gedit to look at the file, and gedit does not seem at all happy with it. It was necessary to download the file and open it manually in LibreOffice. MUST run long enough to defeat bursty bandwidth optimizations such as PowerBoost and discard data from that interval. I'll willingly display my ignorance, but for how long does PowerBoost and its cousins boost bandwidth? I wasn't looking for PowerBoost, and given the thing being examined I wasn't seeing that, but recently when I was evaluating the network performance of something out there in the cloud (not my home cloud as it were though) I noticed performance spikes repeating at intervals which would require 60 seconds to defeat MUST track and sum bi-directional throughput, using estimates for ACK sizes of ipv4, ipv6, and encapsulated ipv6 packets, udp and tcp_rr packets, etc. Estimating the bandwidth consumed by ACKs and/or protocol headers, using code operating at user-space, is going to be guessing. Particularly portable user-space. While those things may indeed affect the user's experience, the user doesn't particularly care about ACKs or header sizes. She cares how well the page loads or the call sounds. MUST have the test server(s) within 80ms of the testing client Why? Perhaps there is something stating that some number of nines worth of things being accessed are within 80ms of the user. If there is, that should be given in support of the requirement. This portion of the test will take your favorite website as a target and show you how much it will slow down, under load. Under load on the website itself, or under load on one's link. I ass-u-me the latter, but that should be made clear. And while the chances of the additional load on a web site via this testing is likely epsilon, there is still the matter of its optics if you will - how it looks. Particularly if there is going to be something distributed with a default website coded into it. Further, websites are not going to remain static, so there will be the matter of being able to compare results over time. Perhaps that can be finessed with the unloaded (again I assume relative to the link of interest/test) measurement. rick jones ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Cerowrt-devel] RFC: Realtime Response Under Load (rrul) test specification
Dave Taht dave.t...@gmail.com wrote: DT However I: DT A) got intimidated by the prospect of dealing with the rfc DT editor been there, done that, I could volunteer to operate the xml2rfc and get your document posted. I think that this might be uptaked by the bmwg. DT B) Have some sticky problems with two aspects of the test DT methodology (and that's just what I know about) which I am DT prototyping around. Running the prototype tests on various real DT networks has had very interesting results... (I do hope others DT try the prototype tests, too, on their networks) Others might be able to help with this. DT I now plan (after feedback) to produce and submit this as a DT standards track RFC in the march timeframe. DT It would give me great joy to have this test series included in DT various SLA metrics, in the long run. good. Can I help? I could create the -00 document from your .doc file? I'll fork your dtaht repo I think that I can also debianize your debloat.sh script, which I think is also in that repo. -- Michael Richardson -on the road- pgpscr2b36rJq.pgp Description: PGP signature ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
