I think the argument you site is a difficult one, it's not without merits but
it's also likely the losing position ultimately. There's no clear cut "you took
something away from the consumer" bit in there that would trigger most consumer
protection triggers. ANY legislation that doesn't have a clear case for public
benefit can be endlessly attacked and special exemptions to it made on every win
against it. Makes it something most lawmakers cringe at the thought of dealing
with for an issue that isn't in the headlines ie doesn't help them get (or lose)
votes.
I think the middle ground is actually really simple. Opt-In or Opt-Out. Hook
up new service and in the installation checklist there's a "we offer connection
enhancements free of charge, can I enable that for you?". All consumer
protections are out when the customer agrees. Even Cambium/Bequant is perfectly
legal in California if the customer opts-in to it (I don't believe you can do
opt-out in California's language). Then in libreQOS or whatever just have an
option to use a fifo instead of fq_codel or cake. Most people will take that
without question, some might ask what it is and it's pretty easy to explain that
it makes sure all their stuff gets a fair share of the connection instead of the
Xbox downloads causing Netflix to buffer. fifo buffers hurt the consumer, not
the operator, and we can always just turn it on when they call in "would you
like to try the enhanced service to see if that makes things better?"
We've actually considered offering service plans based around it. I was looking
at both PQ and Cambium pretty hard and in that would offer 'NN' packages and
then 'Enhanced' with a number of bulletpoints for customers. ie, NN is a 100x20
bare connection (fifo...) while the same price could get a 200x100 enhanced
(AQM) that would cap streaming lower and elevate gaming etc. All in the open
and opt-in via the purchase of the 'enhancement' on the service.
Ultimately, the lack of top level shapers as table stakes is what got me into
libreqos in the first place.
On Sat, Nov 12, 2022 at 12:31 PM Dave Taht <[email protected]
<mailto:[email protected]>> wrote:
Changing the subject line. This is a very nuanced topic and I would
rather enjoy the day AFK! To stir the pot, tho:
I have had a cable industry insider claim that fq_codel violated the
california code you cite. Wrote about it here:
https://blog.cerowrt.org/post/net_neutrality_customers/
<https://blog.cerowrt.org/post/net_neutrality_customers/>
And taking on the other side:
https://blog.cerowrt.org/post/net_neutrality_isps/
<https://blog.cerowrt.org/post/net_neutrality_isps/>
I have asked multiple lawyers about it since, and the rules need to be
clarified, here. The level of doublethink in the cable/telco industry
about NN has to be experienced to be believed. Just spent 6 years
losing a fight about it on l4s and related, so now they can hide
behind the IETFs skirts now.
I didn't make friends with any side that way, and of my relationship
from that period, with the eff and the ACLU, I don't know... but I did
write a nice piece of rhetoric:
"fq codel (now IETF standard RFC8290) is a uniquely “American”
algorithm. It gives the “little guy” - the little packet, the first
packets in a new connection to anywhere, a little boost until the flow
achieves parity with other flows from other sources, with minimal
buffering. This means that all network traffic gets treated equally -
faster. Isn’t that what you want in a network neutral framework? DNS,
gaming traffic, voip, videoconferencing, and the first packets of any
new flow, to anywhere, get a small boost. That’s it. Big flows - from
anybody - from netflix to google to comcast - all achieve parity, with
minimal delay and buffering, at a wide variety of real round trip
times."
Just to throw another bit of controversial thought before I gaily
depart for a nice lunch... I tended to support WISPA's non-defense of
NN
in multiple respects. Title II, and additional regulation seemed
worse, then, than having some mildly chastized ISPs remain free to do
whatever they wanted. With my brutal experiences with politics
starting with the clipper chip, the CDA, SOPA, network neutrality, and
of late the NTIA broadband funds act....
My basic conclusion was this:
There must be a middle path, and it starts with less lawyers being in
charge at the FCC and extends to, in general, less politicians on any
side of the debate, making policy and slogans that don't line up with
reality.
"For a successful technology .... " - richard feynman.
On Sat, Nov 12, 2022 at 9:39 AM dan via LibreQoS
<[email protected] <mailto:[email protected]>>
wrote:
>
> I have to argue with you on 'equally limited' streaming being ok. That
allows for 'internet-like' service but no/poor streaming, a service that
clamps down really hard on streaming and then offers you an upgrade to make
it work. Neutrality doesn't mean 'kind of' neutral. If it's called
'Internet' then no specific service or even type of service should be
limited for the provider's direct benefit or whims. I can see no positive
case for a provider limiting any service/type below the customers quoted
service. The provider gets to save money, but since they offer an inferior
product that is essentially false advertised and customers have no way to
know this then capitalist tools like consumer choice can't work. Consumers
don't know they can get a better suited service, they don't show demand in a
market, that prevents competition from having an opportunity, etc. Most
people have just 1 fast internet option.
>
> Does a customer that can't figure out why their new Roku 4K on their new
4K TV refuses to stream 4K Netflix on their 100M internet service say
"Netflix sucks!". or Roku sucks or Sony sucks or whatever? So this spills
over into making other services take the blame for bad connections.
>
>
>
>
> On Sat, Nov 12, 2022 at 10:19 AM Herbert Wolverson via LibreQoS
<[email protected] <mailto:[email protected]>>
wrote:
>>
>> Net neutrality is complex (I'm generally in favor of the concept).
There's nothing stopping an ISP from having rules that video streaming be
limited - but ALL video streaming has to be equally limited. So ComCast
could use Sandvine, but only without any vendor-specific limitations. Same
for Cambium. Limit streams that look like video, and you're fine. With that
said, it runs into something I say a lot: don't penalize your customers for
using your product. If you're buying a 100mbit/s pipe, hooking it to your
fancy 4k streaming system - it's entirely reasonable to expect sharp 4k
video. In fact, your shaping emphasis should be on keeping their video nice
and watchable even though someone just decided to download the newest Call
of Duty, update their xbox, and update the firmware on their smart fridge.
Likewise, if someone really wants the smallest 5 mbit connection you once
offered - because they only ever use it to check email (we actually had a
customer complain when we offered them a free upgrade from a 5mbit/s plan,
years ago!) - you want to do your best to make it a usable tiny plan for
them. Fair queuing (along with a bit of user education; we eventually
convinced the customer that "no change in fees" actually meant not charging
them more money) helps with all of that.
>>
>> It can win customers, too. We have a couple of business customers who
switched us from their backup to their primary because "our service felt
snappier". We had a business go with us because their 24x7 lobby streaming
video looked better on our (smaller) connection, and a couple of
fraternities who likewise decided to use us because their overkill 4k video
setup looked better (even with the 20+ Xboxes we could see on their
network).
>>
>> QUIC makes me chuckle, because it's exactly what we were doing in
gamedev land in the late 90s to make deathmatches run smoothly. (Seriously,
QuakeWorld and the original Unreal Tournament had the most amazing network
code; they basically implemented what is now known as "reliable UDP" to
incredible results).
>>
>> With that said, I think we've got a remarkable amount of wiggle-room to
make things better (and it's not at all shoddy right now!). XDP is amazing
as-is, and is improving fast. (I'm currently reading
https://github.com/ilejn/xdpbridge <https://github.com/ilejn/xdpbridge> - an
older project that doesn't seem to have gained much traction, but I'm
wondering if it couldn't provide some seriously fast bridge offloading; I'll
have to setup a better test environment to find out)
>>
>> On Sat, Nov 12, 2022 at 10:09 AM Robert Chacón via LibreQoS
<[email protected] <mailto:[email protected]>>
wrote:
>>>
>>> Thank you, Dan.
>>> I completely agree with you 100% there. It's a risky way to save a tiny
amount of money on bandwidth costs.
>>> Comcast paid out $16 million in a settlement over using Sandvine's DPI
in this way.
>>> Since then, Comcast has switched from DPI to fair queueing with
DOCSIS-pie.
>>>
>>> California's SB-822 will be used as a template by many other states
going forward on Net Neutrality.
>>> SB-822 even clarified that "Reasonable Network Management" needs to be
"as application-agnostic as possible".
>>> These DPI approaches like Cambium's will not hold up in court the way
Fair Queuing (Preseem, LibreQoS) based solutions will.
>>> We can point to years of public research and RFCs when defending a Fair
Queuing QoE approach.
>>> They can't so easily defend using a product that advertised its big
"scale down 4k to save money" knob or "accelerate their speeds only when
they run speed tests" button.
>>> It's concerning to me that these QoE vendors are not disclosing "hey,
don't use this in NN states like California please".
>>> The WISP industry will be hurt long-term by these products with
consumers viewing us as an inferior technology that limits their usage in
intrusive ways.
>>> Like you said, let's let them use their plan however they want. It
doesn't hurt us. But things like capping end-users at HD will hurt us
long-term in lost sales.
>>>
>>> Regarding TCP acceleration, I'm leaning toward the argument made by Dan
from Preseem that QUIC adoption is growing pretty quickly and we may not
want to mess with TCP at the expense of UDP given the rise of QUIC.
>>> Cloudlfare says HTTP/3 and QUIC are already at 28% and growing.
>>> One point he brought up is that if we use TCP acceleration, and a
middlebox has to be rebooted or something - all client TCP sessions will
break when OSPF switches paths around the box.
>>>
>>> On Sat, Nov 12, 2022 at 8:36 AM dan via LibreQoS
<[email protected] <mailto:[email protected]>>
wrote:
>>>>
>>>> I think there's a thin grey line between what other vendors (paraqum,
bequant/cambium) and libreQoS is trying to do. The first two are REALLY
pushing this selecting narrowing of traffic and they'll go straight after
streaming for their demos. Most people that drop in the cambium QoE
appliance turn the knobs on Netflix down and then praise the 40% savings on
their network. Same with PQ. I think this is fundamentally wrong. This
is the entire reason Net Neutrality keeps popping up. Why does the ISP get
to say "You only need 25Mbps for your netflix". That's double wrong if they
offer their own video services. It's not about improving the customer's
experience, it's about spending less on upstream bandwidth at the customer's
expense. As soon as they say 'pays for itself in bandwidth savings' you
know what the product is primarily for.
>>>>
>>>> This is why I like the preseem and libreqos model of just using a
'stock' cake and just finessing the data coming in to keep the pipe from
clogging. Who cares if someone pulls 90M bursts of apple TV+ on their 100M
pipe. Only dial that back so other requests they make get through cleanly.
>>>>
>>>> preseem is going more towards wireless monitoring etc and kinda
letting shaping be a secondary.... if they were doing more libreqos like
stuff I'd just stick with preseem.
>>>>
>>>> libreQoS may end up being the best and most net neutral product you
can get. The only current disadvantage (ignoring DPI) is lack of TCP
accelleration that PQ and Bequant/cambium have.
>>>>
>>>> I should add that 3 states already have laws on the books that
straight up make PQ and Cambium's solution illegal because they target
certain services for throttling. My state (Montana) has the same rule
except only when working with a government contract. Libreqos is actually
compliant (as is preseem).
>>>>
>>>> On Sat, Nov 12, 2022 at 8:11 AM Dave Taht via LibreQoS
<[email protected] <mailto:[email protected]>>
wrote:
>>>>>
>>>>> this report predates the libreqos list...
>>>>>
>>>>> ---------- Forwarded message ---------
>>>>> From: Dave Taht <[email protected] <mailto:[email protected]>>
>>>>> Date: Mon, Oct 17, 2022 at 8:15 PM
>>>>> Subject: A quick report from the WISPA conference
>>>>> To: Sina Khanifar <[email protected] <mailto:[email protected]>>
>>>>> Cc: Cake List <[email protected]
<mailto:[email protected]>>, Make-Wifi-fast
>>>>> <[email protected]
<mailto:[email protected]>>, Rpm
>>>>> <[email protected] <mailto:[email protected]>>,
Stuart Cheshire <[email protected] <mailto:[email protected]>>,
>>>>> bloat <[email protected]
<mailto:[email protected]>>
>>>>>
>>>>>
>>>>> On Mon, Oct 17, 2022 at 7:51 PM Sina Khanifar <[email protected]
<mailto:[email protected]>> wrote:
>>>>> >
>>>>> > Positive or negative, I can claim a bit of credit for this video
:). We've been working with LTT on a few projects and we pitched them on
doing something around bufferbloat. We've seen more traffic to our Waveforn
test than ever before, which has been fun!
>>>>>
>>>>> Thank you. Great job with that video! And waveform has become the
goto
>>>>> site for many now.
>>>>>
>>>>> I can't help but wonder tho... are you collecting any statistics,
over
>>>>> time, as to how much better the problem is getting?
>>>>>
>>>>> And any chance they could do something similar explaining wifi?
>>>>>
>>>>> ...
>>>>>
>>>>> I was just at WISPA conference week before last. Preseem's booth
>>>>> (fq_codel) was always packed. Vilo living had put cake in their wifi
6
>>>>> product. A
>>>>> keynote speaker had deployed it and talked about it with waveform
>>>>> results on the big screen (2k people there). A large wireless vendor
>>>>> demo'd privately to me their flent results before/after cake on their
>>>>> next-gen radios... and people dissed tarana without me prompting for
>>>>> their bad bufferbloat... and the best thing of all that happened to
me
>>>>> was... besides getting a hug from a young lady (megan) who'd salvaged
>>>>> her schooling in alaska using sqm - I walked up to the paraqum booth
>>>>> (another large QoE middlebox maker centered more in india) and asked.
>>>>>
>>>>> "So... do y'all have fq_codel yet?"
>>>>>
>>>>> And they smiled and said: "No, we have something better... we've got
cake."
>>>>>
>>>>> "Cake? What's that?" - I said, innocently.
>>>>>
>>>>> They then stepped me through their 200Gbps (!!) product, which uses a
>>>>> bunch of offloads, and can track rtt down to a ms with the intel
>>>>> ethernet card they were using. They'd modifed cake to provide 16 (?)
>>>>> levels of service, and were running under dpdk (I am not sure if cake
>>>>> was). It was a great, convincing pitch...
>>>>>
>>>>> ... then I told 'em who I was. There's a video of the in-both concert
after.
>>>>>
>>>>> ...
>>>>>
>>>>> The downside to me (and the subject of my talk) was that in nearly
>>>>> every person I talked to, fq_codel was viewed as a means to better
>>>>> subscriber bandwidth plan enforcement (which is admittedly the market
>>>>> that preseem pioneered) and it was not understood that I'd got
>>>>> involved in this whole thing because I'd wanted an algorithm to deal
>>>>> with "rain fade", running directly on the radios. People wanted to
use
>>>>> the statistics on the radios to drive the plan enforcement better
>>>>> (which is an ok approach, I guess), and for 10+ I'd been whinging
>>>>> about the... physics.
>>>>>
>>>>> So I ranted about rfc7567 a lot and begged people now putting
routerOS
>>>>> 7.2 and later out there (mikrotik is huge in this market), to kill
>>>>> their fifos and sfqs at the native rates of the interfaces... and
>>>>> watch their network improve that way also.
>>>>>
>>>>> I think one more wispa conference will be a clean sweep of everyone
in
>>>>> the fixed wireless market to not only adopt these algorithms for plan
>>>>> enforcement, but even more directly on the radios and more CPE.
>>>>>
>>>>> I also picked up enough consulting business to keep me busy the rest
>>>>> of this year, and possibly more than I can handle (anybody looking?)
>>>>>
>>>>> I wonder what will happen at a fiber conference?
>>>>>
>>>>> > On Mon, Oct 17, 2022 at 7:45 PM Dave Taht via Bloat
<[email protected] <mailto:[email protected]>> wrote:
>>>>> >>
>>>>> >> On Mon, Oct 17, 2022 at 5:02 PM Stuart Cheshire
<[email protected] <mailto:[email protected]>> wrote:
>>>>> >> >
>>>>> >> > On 9 Oct 2022, at 06:14, Dave Taht via Make-wifi-fast
<[email protected]
<mailto:[email protected]>> wrote:
>>>>> >> >
>>>>> >> > > This was so massively well done, I cried. Does anyone know how
to get in touch with the ifxit folk?
>>>>> >> > >
>>>>> >> > > https://www.youtube.com/watch?v=UICh3ScfNWI
<https://www.youtube.com/watch?v=UICh3ScfNWI>
>>>>> >> >
>>>>> >> > I’m surprised that you liked this video. It seems to me that it
repeats all the standard misinformation. The analogy they use is the
standard terrible example of waiting in a long line at a grocery store, and
the “solution” is letting certain traffic “jump the line, angering everyone
behind them”.
>>>>> >>
>>>>> >> Accuracy be damned. The analogy to common experience resonates
more.
>>>>> >>
>>>>> >> >
>>>>> >> > Some quotes from the video:
>>>>> >> >
>>>>> >> > > it would be so much more efficient for them to let you skip
the line and just check out, especially since you’re in a hurry, but they’re
rudely refusing
>>>>> >>
>>>>> >> I think the person with the cheetos pulling out a gun and shooting
>>>>> >> everyone in front of him (AQM) would not go down well.
>>>>> >>
>>>>> >> > > to go back to our grocery store analogy this would be like if
a worker saw you standing at the back ... and either let you skip to the
front of the line or opens up an express lane just for you
>>>>> >>
>>>>> >> Actually that analogy is fairly close to fair queuing. The
multiple
>>>>> >> checker analogy is one of the most common analogies in queue
theory
>>>>> >> itself.
>>>>> >>
>>>>> >> >
>>>>> >> > The video describes the problem of bufferbloat, and then
describes the same failed solution that hasn’t worked for the last three
decades.
>>>>> >>
>>>>> >> Hmm? It establishes the scenario, explains the problem *quickly*,
>>>>> >> disses gamer routers for not getting it right.. *points to an
>>>>> >> accurate test*, and then to the ideas and products that *actually
>>>>> >> work* with "smart queueing", with a screenshot of the most common
>>>>> >> (eero's optimize for gaming and videoconferencing), and fq_codel
and
>>>>> >> cake *by name*, and points folk at the best known solution
available,
>>>>> >> openwrt.
>>>>> >>
>>>>> >> Bing, baddabang, boom. Also the comments were revealing. A goodly
>>>>> >> percentage already knew the problem, more than a few were
inspired to
>>>>> >> take the test,
>>>>> >> there was a whole bunch of "Aha!" success stories and 360k views,
>>>>> >> which is more people than we've ever been able to reach in for
>>>>> >> example, a nanog conference.
>>>>> >>
>>>>> >> I loved that folk taking the test actually had quite a few A
results,
>>>>> >> without having had to do anything. At least some ISPs are getting
it
>>>>> >> more right now!
>>>>> >>
>>>>> >> At this point I think gamers in particular know what "brands"
we've
>>>>> >> tried to establish - "Smart queues", "SQM", "OpenWrt", fq_codel
and
>>>>> >> now "cake" are "good" things to have, and are stimulating demand
by
>>>>> >> asking for them, It's certainly working out better and better
for
>>>>> >> evenroute, firewalla, ubnt and others, and I saw an uptick in
>>>>> >> questions about this on various user forums.
>>>>> >>
>>>>> >> I even like that there's a backlash now of people saying "fixing
>>>>> >> bufferbloat doesn't solve everything" -
>>>>> >>
>>>>> >> > Describing the obvious simple-minded (wrong) solution that any
normal person would think of based on their personal human experience
waiting in grocery stores and airports, is not describing the solution to
bufferbloat. The solution to bufferbloat is not that if you are privileged
then you get to “skip to the front of the line”. The solution to bufferbloat
is that there is no line!
>>>>> >>
>>>>> >> I like the idea of a guru floating above a grocery cart with a
better
>>>>> >> string of explanations, explaining
>>>>> >>
>>>>> >> - "no, grasshopper, the solution to bufferbloat is no line...
at all".
>>>>> >>
>>>>> >> >
>>>>> >> > With grocery stores and airports people’s arrivals are
independent and not controlled. There is no way for a grocery store or
airport to generate backpressure to tell people to wait at home when a queue
begins to form. The key to solving bufferbloat is generating timely
backpressure to prevent the queue forming in the first place, not accepting
a huge queue and then deciding who deserves special treatment to get better
service than all the other peons who still have to wait in a long queue,
just like before.
>>>>> >>
>>>>> >> I am not huge on the word "backpressure" here. Needs to signal the
>>>>> >> other side to slow down, is more accurate. So might say timely
>>>>> >> signalling rather than timely backpressure?
>>>>> >>
>>>>> >> Other feedback I got was that the video was too smarmy (I agree),
>>>>> >> different audiences than gamers need different forms of
outreach...
>>>>> >>
>>>>> >> but to me, winning the gamers has always been one of the most
>>>>> >> important things, as they make a lot of buying decisions, and they
>>>>> >> benefit the most for
>>>>> >> fq and packet prioritization as we do today in gamer routers and
in
>>>>> >> cake + qosify.
>>>>> >>
>>>>> >> maybe that gets in the way of more serious markets. Certainly I
would
>>>>> >> like another video explaining what goes wrong with
videoconferencing.
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> >
>>>>> >> > Stuart Cheshire
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> This song goes out to all the folk that thought Stadia would work:
>>>>> >>
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
<https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz>
>>>>> >> Dave Täht CEO, TekLibre, LLC
>>>>> >> _______________________________________________
>>>>> >> Bloat mailing list
>>>>> >> [email protected] <mailto:[email protected]>
>>>>> >> https://lists.bufferbloat.net/listinfo/bloat
<https://lists.bufferbloat.net/listinfo/bloat>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> This song goes out to all the folk that thought Stadia would work:
>>>>>
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
<https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz>
>>>>> Dave Täht CEO, TekLibre, LLC
>>>>>
>>>>>
>>>>> --
>>>>> This song goes out to all the folk that thought Stadia would work:
>>>>>
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
<https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz>
>>>>> Dave Täht CEO, TekLibre, LLC
>>>>> _______________________________________________
>>>>> LibreQoS mailing list
>>>>> [email protected]
<mailto:[email protected]>
>>>>> https://lists.bufferbloat.net/listinfo/libreqos
<https://lists.bufferbloat.net/listinfo/libreqos>
>>>>
>>>> _______________________________________________
>>>> LibreQoS mailing list
>>>> [email protected] <mailto:[email protected]>
>>>> https://lists.bufferbloat.net/listinfo/libreqos
<https://lists.bufferbloat.net/listinfo/libreqos>
>>>
>>>
>>>
>>> --
>>> Robert Chacón
>>> CEO | JackRabbit Wireless LLC
>>> Dev | LibreQoS.io
>>>
>>> _______________________________________________
>>> LibreQoS mailing list
>>> [email protected] <mailto:[email protected]>
>>> https://lists.bufferbloat.net/listinfo/libreqos
<https://lists.bufferbloat.net/listinfo/libreqos>
>>
>> _______________________________________________
>> LibreQoS mailing list
>> [email protected] <mailto:[email protected]>
>> https://lists.bufferbloat.net/listinfo/libreqos
<https://lists.bufferbloat.net/listinfo/libreqos>
>
> _______________________________________________
> LibreQoS mailing list
> [email protected] <mailto:[email protected]>
> https://lists.bufferbloat.net/listinfo/libreqos
<https://lists.bufferbloat.net/listinfo/libreqos>
--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
<https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz>
Dave Täht CEO, TekLibre, LLC
