Hi Fred,
since you have a very good test with iPlayer, you can simply repeat your
experiments to figure out where shaping becomes to unstable for iPlayer. It
would be interesting to see whether RRUL (or other netsurf-wrapper tests) show
qualitative differences around the same numerical shaping values.
On Aug 25, 2013, at 21:31 , Fred Stratton <fredstrat...@imap.cc> wrote:
> Re-reading your comment, I have reset the upload rate higher to 900 kbits/s.
>
>
> On 25 Aug 2013, at 20:08, Fred Stratton <fredstrat...@imap.cc> wrote:
>
>> That is very helpful.
>>
>> With a sync rate of about 12000 kbits/s, and a download rate of about 10900
>> kbits/s. I have set the download rate to 5000 kbits/s. For upload similarly
>> 1200/970/500, all kbits/s.
>>
>> I can now mostly watch video in iPlayer and download at circa 300 - 400
>> kbits/s simultaneously, using htb, with tc-stab disabled.
>>
>> QED
So slowly increase both shaped rates until iPlayer becomes "unhappy" to
better define the threshold?
Best
Sebastian
>>
>>
>> On 25 Aug 2013, at 19:41, Dave Taht <dave.t...@gmail.com> wrote:
>>
>>> So it sounds like you need a lower setting for the download than what you
>>> are using? It's not the upload that is your problem.
>>>
>>> Netanalyzer sends one packet stream and thus measures 1 queue only.
>>> fq_codel will happily give it one big queue for a while, while still
>>> interleaving other flows's packets into the stream at every opportunity.
>>>
>>> as for parsing rrul I generally draw a line with my hand and multiply by 4,
>>> then fudge in the numbers for the reverse ack and measurement streams.
>>
>> You are saying that you judge the result solely by eye. presumably.
>>>
>>> As written it was targetted at 4Mbit and up which is why the samples are
>>> discontinuous in your much lower bandwidth situation.
>>
>> Aha. Problem solved.
>>>
>>> I do agree that rrul could use a simpler implementation, perhaps one that
>>> tested two download streams only, and provided an estimate as to the actual
>>> bandwidth usage, and scale below 4Mbit better.
>>>
>>>
>>> On Sun, Aug 25, 2013 at 11:30 AM, Fred Stratton <fredstrat...@imap.cc>
>>> wrote:
>>>
>>> On 25 Aug 2013, at 18:53, Sebastian Moeller <moell...@gmx.de> wrote:
>>>
>>> > Hi Fred,
>>> >
>>> >
>>> > On Aug 25, 2013, at 16:26 , Fred Stratton <fredstrat...@imap.cc> wrote:
>>> >
>>> >> Thank you.
>>> >>
>>> >> This is an initial response.
>>> >>
>>> >> Am using 3.10.2-1 currently, with the standard AQM interface. This does
>>> >> not have the pull down menu of your interface, which is why I ask if
>>> >> both are active.
>>> >
>>> > I have seen your follow-up mail that you actually used 3.10.9-2. I
>>> > think that has the first cut of the script modifications that still allow
>>> > to select both. Since I have not tested it any other way I would
>>> > recommend to enable just one of them at the same time. Since the
>>> > implementation of both is somewhat orthogonal and htb_private actually
>>> > works in 3.10.9, best case you might actually get the link layer
>>> > adjustments (LLA) and the overhead applied twice, wasting bandwidth. So
>>> > please either use the last set of modified files I send around or wait
>>> > for Dave to include them in ceropackages…
>>>
>>> I have retained the unmodified script. I shall return to that.
>>>
>>>
>>> >
>>> >> On 25 Aug 2013, at 14:59, Sebastian Moeller <moell...@gmx.de> wrote:
>>> >>
>>> >>> Hi Fred,
>>> >>>
>>> >>>
>>> >>> On Aug 25, 2013, at 12:17 , Fred Stratton <fredstrat...@imap.cc> wrote:
>>> >>>
>>> >>>>
>>> >>>> On 25 Aug 2013, at 10:21, Fred Stratton <fredstrat...@imap.cc> wrote:
>>> >>>>
>>> >>>>> As the person with the most flaky ADSL link, I point out that None of
>>> >>>>> these recent, welcome, changes, are having any effect here, with an
>>> >>>>> uplink sped of circa 950 kbits/s.
>>> >>>
>>> >>> Okay, how flaky is you link? What rate of Errors do you have while
>>> >>> testing? I am especially interested in CRC errors and ES SES and HEC,
>>> >>> just to get an idea how flaky the line is...
>>> >>>
>>> >>>>>
>>> >>>>> The reason I mention this is that it is still impossible to watch
>>> >>>>> iPlayer Flash streaming video and download at the same time, The
>>> >>>>> iPlayer stream fails. The point of the exercise was to achieve this.
>>> >>>>>
>>> >>>>> The uplink delay is consistently around 650ms, which appears to be
>>> >>>>> too high for effective streaming. In addition, the uplink stream has
>>> >>>>> multiple breaks, presumably outages, if the uplink rate is capped at,
>>> >>>>> say, 700 kbits/s.
>>> >>>
>>> >>> Well, watching video is going to stress your downlink so the uplink
>>> >>> should not saturate by the ACKs and the concurrent downloads also do
>>> >>> not stress your uplink except for the ACKs, so this points to downlink
>>> >>> errors as far as I can tell from the data you have given. If the up
>>> >>> link has repeated outages however, your problems might be unfixable
>>> >>> because these, if long enough, will cause lost ACKs and will probably
>>> >>> trigger retransmission, independent of whether the link layer
>>> >>> adjustments work or not. (You could test this by shaping you up and
>>> >>> downlink to <= 50% of the link rates and disable all link layer
>>> >>> adjustments, 50% is larger than the ATM worst case so should have you
>>> >>> covered. Well unless you del link has an excessive number of tones
>>> >>> reserved for forward error correction (FEC)).
>>> >>
>>> >> Uptime 100655
>>> >> downstream 12162 kbits/s
>>> >> CRC errors 10154
>>> >> FEC Errors 464
>>> >> hEC Errors 758
>>> >>
>>> >> upstream 1122 kbits/s
>>> >> no errors in period.
>>> >
>>> > Ah, I think you told me in the past that "Target snr upped to 12
>>> > deciBel. Line can sustain 10 megabits/s with repeated loss of sync.at
>>> > lower snr. " so sync at 12162 might be too aggressive, no? But the point
>>> > is that as I understand iPlayer works fine without competing download
>>> > traffic? To my eye the error numbers look small enough to not be
>>> > concerned about. Do you know how long the error correction period is?
>>>
>>> The correction period is probably circa 28 hours. Have moved to using the
>>> HG612. This is uses the Broadcom 6368 SoC. Like most of the devices I use,
>>> it fell out of a BT van and on to ebay. It is the standard device used for
>>> connecting FTTC installations in the UK. With a simple modification, it
>>> will work stably with ADSL2+.
>>>
>>> Ihe sync rate has gone up considerably, not because I have changed the
>>> Target SNR from 12 Decibel, but because I am now using a Broadcom chipset
>>> and software blob with a DSLAM which returns BDCM when interrogated.
>>> >
>>> >
>>> >>
>>> >>> Could you perform the following test by any chance: state iPlayer
>>> >>> and yor typical downloads and then have a look at
>>> >>> http://gw.home.lan:81und the following tab chain Status -> Realtime
>>> >>> Graphs -> Traffic -> Realtime Traffic. If during your test the Outbound
>>> >>> rate stays well below you shaped limit and you still encounter the
>>> >>> stream failure I would say it is save to ignore the link layer
>>> >>> adjustments as cause of your issues.
>>> >>
>>> >> Am happy reducing rate to fifty per cent, but the uplink appears to have
>>> >> difficulty operating below circa 500 kbits/s. This should not be so. I
>>> >> shall try a fourth time.
>>> >
>>> > That sounds weird, if you shape to below 500 upload stops working
>>> > or just gets choppier? Looking at your sync data 561 would fit the ~50%
>>> > and above 500 requirements.
>>>
>>> I was basing the judgment on Netalyzr data. DT and you now say this is
>>> suspect. However, netsurf-wrapper traces are discontinuous. The actual real
>>> time trace looks perfectly normal.
>>>
>>> iPlayer is a Flash based player which is web page embedded. The ipv4 user
>>> address is parsed to see if it is in the UK. It plays BBC TV programs. It
>>> most likely is badly designed and written. It is the way I watch TV. Like
>>> all UK residents, I pay the bloated bureaucracy of the BBC a yearly fee of
>>> about 200 euro. If I do not pay, I will be fined. You will be surprised
>>> that I am not a fan of the BBC. iPlayer starts and runs fine, but if a
>>> download is commenced whilst it is running, so I can watch the propaganda
>>> put out as national news, the video will stall and the continue, but most
>>> commonly will stop.
>>> >
>>> >
>>> >>>
>>> >>>
>>> >>>>>
>>> >>>>> YouTube has no problems.
>>> >>>>>
>>> >>>>> I remain unclear whether the use of tc-stab and htb are mutually
>>> >>>>> exclusive options, using the present stock interface.
>>> >>>
>>> >>> Well, depending on the version of the cerowrt you use, <3.10.9-1 I
>>> >>> believe lacks a functional HTB link layer adjustment mechanism, so you
>>> >>> should select tc_stab. My most recent modifications to Toke and Dave's
>>> >>> AQM package does only allow you to select one or the other. In any case
>>> >>> selecting BOTH is not a reasonable thing to do, because best case it
>>> >>> will only apply overhead twice, worst case it would also do the (link
>>> >>> layer adjustments) LLA twice
>>> >>
>>> >>
>>> >>> See initial comments.
>>> >>>
>>> >>>>>
>>> >>>>> The current ISP connection is IPoA LLC.
>>> >>>>
>>> >>>> Correction - Bridged LLC.
>>> >>>
>>> >>> Well, I think you should try to figure out your overhead
>>> >>> empirically and check the encapsulation. I would recommend you run the
>>> >>> following script on you r link over night and send me the log file it
>>> >>> produces:
>>> >>>
>>> >>> #! /bin/bash
>>> >>> # TODO use seq or bash to generate a list of the requested sizes (to
>>> >>> alow for non-equdistantly spaced sizes)
>>> >>>
>>> >>> # Telekom Tuebingen Moltkestrasse 6
>>> >>> TECH=ADSL2
>>> >>> # finding a proper target IP is somewhat of an art, just traceroute a
>>> >>> remote site
>>> >>> # and find the nearest host reliably responding to pings showing the
>>> >>> smallet variation of pingtimes
>>> >>> TARGET=87.186.197.70 # T
>>> >>> DATESTR=`date +%Y%m%d_%H%M%S` # to allow multiple sequential
>>> >>> records
>>> >>> LOG=ping_sweep_${TECH}_${DATESTR}.txt
>>> >>>
>>> >>>
>>> >>> # by default non-root ping will only end one packet per second, so work
>>> >>> around that by calling ping independently for each package
>>> >>> # empirically figure out the shortest period still giving the standard
>>> >>> ping time (to avoid being slow-pathed by our host)
>>> >>> PINGPERIOD=0.01 # in seconds
>>> >>> PINGSPERSIZE=10000
>>> >>>
>>> >>> # Start, needed to find the per packet overhead dependent on the ATM
>>> >>> encapsulation
>>> >>> # to reliably show ATM quantization one would like to see at least two
>>> >>> steps, so cover a range > 2 ATM cells (so > 96 bytes)
>>> >>> SWEEPMINSIZE=16 # 64bit systems seem to require 16 bytes of
>>> >>> payload to include a timestamp...
>>> >>> SWEEPMAXSIZE=116
>>> >>>
>>> >>>
>>> >>> n_SWEEPS=`expr ${SWEEPMAXSIZE} - ${SWEEPMINSIZE}`
>>> >>>
>>> >>>
>>> >>> i_sweep=0
>>> >>> i_size=0
>>> >>>
>>> >>> while [ ${i_sweep} -lt ${PINGSPERSIZE} ]
>>> >>> do
>>> >>> (( i_sweep++ ))
>>> >>> echo "Current iteration: ${i_sweep}"
>>> >>> # now loop from sweepmin to sweepmax
>>> >>> i_size=${SWEEPMINSIZE}
>>> >>> while [ ${i_size} -le ${SWEEPMAXSIZE} ]
>>> >>> do
>>> >>> echo "${i_sweep}. repetition of ping size ${i_size}"
>>> >>> ping -c 1 -s ${i_size} ${TARGET} >> ${LOG} &
>>> >>> (( i_size++ ))
>>> >>> # we need a sleep binary that allows non integer times (GNU sleep
>>> >>> is fine as is sleep of macosx 10.8.4)
>>> >>> sleep ${PINGPERIOD}
>>> >>> done
>>> >>> done
>>> >>>
>>> >>> #tail -f ${LOG}
>>> >>>
>>> >>> echo "Done... ($0)"
>>> >>>
>>> >>>
>>> >>> Please set TARGET to the closest IP host on the ISP side of your link
>>> >>> that gives reliable ping RTTs (using "ping -c 100 -s 16
>>> >>> your.best.host.ip"). Also test whether the RTTs are in the same
>>> >>> ballpark when you reduce the ping period to 0.01 (you might have to
>>> >>> increase the period until the RTTs are close to the standard 1 ping per
>>> >>> second case). I can then run this through my matlab code to detect the
>>> >>> actual overhead. (I am happy to share the code as well, if you have
>>> >>> matlab available; it might even run under octave but I have not tested
>>> >>> that since the last major changes).
>>> >>
>>> >> To follow at some point.
>>> >
>>> > Oh, I failed to mention at the given parameters the script takes
>>> > almost 3 hours, during which the link should be otherwise idle...
>>> >
>>> >>>
>>> >>>
>>> >>>>
>>> >>>>> Whatever byte value is used for tc-stab makes no change.
>>> >>>
>>> >>> I assume you talk about the overhead? Missing link layer adjustment
>>> >>> will eat between 50% and 10% of your link bandwidth, while missing
>>> >>> overhead values will be more benign. The only advise I can give is to
>>> >>> pick the overhead that actually describes your link. I am willing to
>>> >>> help you figure this out.
>>> >>
>>> >> The link is bridged LLC. Have been using 18 and 32 for test purposes. I
>>> >> shall move to PPPoA VC-MUX in 4 months.
>>> >
>>> > I guess figuring out you exact overhead empirically is going to be
>>> > fun.
>>> >
>>> >>>
>>> >>>>>
>>> >>>>> I have applied the ingress modification to simple.qos, keeping the
>>> >>>>> original version., and tested both.
>>> >>>
>>> >>> For which cerowrt version? It is only expected to do something for
>>> >>> 3.10.9-1 and upwards, before that the HTB lionklayer adjustment did NOT
>>> >>> work.
>>> >>
>>> >> Using 3.10.9-2
>>> >
>>> > Yeah as stated above, I would recommend to use either or, not both.
>>> > If you took RRUL data you might be able to compare the three conditions.
>>> > I would estimate the most interesting part would be in the sustained
>>> > ravager up and download rates here.
>>>
>>> How do you obtain an average i.e. mean rate from the RRUL graph?
>>> >
>>> >
>>> >>
>>> >>>
>>> >>>>>
>>> >>>>> I have changed the Powerline adaptors I use to ones with known
>>> >>>>> smaller buffers, though this is unlikely to be a ate-limiting step.
>>> >>>>>
>>> >>>>> I have changed the 2Wire gateway, known to be heavily buffered, with
>>> >>>>> a bridged Huawei HG612, with a Broadcom 6368 SoC.
>>> >>>>>
>>> >>>>> This device has a permanently on telnet interface, with a simple
>>> >>>>> password, which cannot be changed other than by firmware
>>> >>>>> recompilation…
>>> >>>>>
>>> >>>>> Telnet, however, allows txqueuelen to be reduced from 1000 to 0.
>>> >>>>>
>>> >>>>> None of these changes affect the problematic uplink delay.
>>> >>>
>>> >>> So how did you measure the uplink delay? The RRUL plots you sent me
>>> >>> show an increase in ping RTT from around 50ms to 80ms with tc_stab and
>>> >>> fq_codel on simplest.qos, how does that reconcile with 650ms uplink
>>> >>> delay, netalyzr?
>>> >>
>>> >> Max Planck and Netalyzr produce the same figure. I use both, but Max
>>> >> Planck gives you circa 3 tries per IP address per 24 hours.
>>> >
>>> > Well, both use the same method which is not to meaningful if you
>>> > use fq_codel on a shaped link (unless you want to optimize your system
>>> > for UDP floods :) )
>>> >
>>> > [snipp]
>>> >
>>> >
>>> > Best Regards
>>> > Sebastian
>>>
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>>
>>>
>>>
>>> --
>>> Dave Täht
>>>
>>> Fixing bufferbloat with cerowrt:
>>> http://www.teklibre.com/cerowrt/subscribe.html
>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel
