Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[Jonathan Morton]

> On 10 Jul, 2021, at 2:01 am, Leonard Kleinrock  wrote:
> 
> No question that non-stationarity and instability are what we often see in 
> networks.  And, non-stationarity and instability are both topics that lead to 
> very complex analytical problems in queueing theory.  You can find some 
> results on the transient analysis in the queueing theory literature 
> (including the second volume of my Queueing Systems book), but they are 
> limited and hard. Nevertheless, the literature does contain some works on 
> transient analysis of queueing systems as applied to network congestion 
> control - again limited. On the other hand, as you said, control theory 
> addresses stability head on and does offer some tools as well, but again, it 
> is hairy. 

I was just about to mention control theory.

One basic characteristic of Poisson traffic is that it is inelastic, and 
assumes there is no control feedback whatsoever.  This means it can only be a 
valid model when the following are both true:

1: The offered load is *below* the link capacity, for all links, averaged over 
time.

2: A high degree of statistical multiplexing exists.

If 1: is not true and the traffic is truly inelastic, then the queues will 
inevitably fill up and congestion collapse will result, as shown from ARPANET 
experience in the 1980s; the solution was to introduce control feedback to the 
traffic, initially in the form of TCP Reno.  If 2: is not true then the traffic 
cannot be approximated as Poisson arrivals, regardless of load relative to 
capacity, because the degree of correlation is too high.

Taking the iPhone introduction anecdote as an illustrative example, measuring 
utilisation as very close to 100% is a clear warning sign that the Poisson 
model was inappropriate, and a control-theory approach was needed instead, to 
capture the feedback effects of congestion control.  The high degree of 
statistical multiplexing inherent to a major ISP backhaul is irrelevant to that 
determination.

Such a model would have found that the primary source of control feedback was 
human users giving up in disgust.  However, different humans have different 
levels of tolerance and persistence, so this feedback was not sufficient to 
reduce the load sufficiently to give the majority of users a good service; 
instead, *all* users received a poor service and many users received no usable 
service.  Introducing a technological control feedback, in the form of packet 
loss upon overflow of correctly-sized queues, improved service for everyone.

(BTW, DNS becomes significantly unreliable around 1-2 seconds RTT, due to 
protocol timeouts, which is inherited by all applications that rely on DNS 
lookups.  Merely reducing the delays consistently below that threshold would 
have improved perceived reliability markedly.)

Conversely, when talking about the traffic on a single ISP subscriber's 
last-mile link, the Poisson model has to be discarded due to criterion 2 being 
false.  The number of flows going to even a family household is probably in the 
low dozens at best.  A control-theory approach can also work here.

 - Jonathan Morton
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[Toke Høiland-Jørgensen via Bloat]

"Holland, Jake via Bloat"  writes:

> Hi David,
>
> That’s an interesting point, and I think you’re right that packet
> arrival is poorly modeled as a Poisson process, because in practice
> packet transmissions are very rarely unrelated to other packet
> transmissions.
>
> But now you’ve got me wondering what the right approach is. Do you
> have any advice for how to improve this kind of modeling?

I actually tried my hand at finding something better for my master's
thesis and came across something called a Markov-Modulated Poisson
Process (MMPP/D/1 queue)[0]. It looked promising, but unfortunately I
failed to make it produce any useful predictions. Most likely this was
as much a result of my own failings as a queueing theorist as it was the
fault of the model (I was in way over my head by the time I got to that
model); so I figured I'd mention it here in case anyone more qualified
would have any opinion on it.

I did manage to get the Linux kernel to produce queueing behaviour that
resembled that of a standard M/M/1 queue (if you squint a bit); all you
have to do is to use a traffic generator that emits packets with the
distribution the model assumes... :)

The full thesis is still available[1] for the perusal of morbidly curious.

-Toke

[0] https://www.sciencedirect.com/science/article/abs/pii/016653169390035S
[1] https://rucforsk.ruc.dk/ws/files/57613884/thesis-final.pdf
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[Leonard Kleinrock]

David,

No question that non-stationarity and instability are what we often see in 
networks.  And, non-stationarity and instability are both topics that lead to 
very complex analytical problems in queueing theory.  You can find some results 
on the transient analysis in the queueing theory literature (including the 
second volume of my Queueing Systems book), but they are limited and hard. 
Nevertheless, the literature does contain some works on transient analysis of 
queueing systems as applied to network congestion control - again limited. On 
the other hand, as you said, control theory addresses stability head on and 
does offer some tools as well, but again, it is hairy. 

Averages are only averages, but they can provide valuable information. For 
sure, latency can and does confound behavior.  But, as you point out, it is the 
proliferation of control protocols that are, in some cases, deployed 
willy-nilly in networks without proper evaluation of their behavior that can 
lead to the nasty cycle of large transient latency, frantic repeating of web 
requests, protocols sending multiple copies, lack of awareness of true capacity 
or queue size or throughput, etc, all of which you articulate so well, create 
the chaos and frustration in the network.  Analyzing that is really difficult, 
and if we don’t measure and sense, we have no hope of understanding, 
controlling, or ameliorating such situations.  

Len

> On Jul 9, 2021, at 12:31 PM, David P. Reed  wrote:
> 
> Len - I admit I made a mistake in challenging Little's Law as being based on 
> Poisson processes. It is more general. But it tells you an "average" in its 
> base form, and latency averages are not useful for end user applications.
>  
> However, Little's Law does assume something that is not actually valid about 
> the kind of distributions seen in the network, and in fact, it is NOT true 
> that networks converge on Poisson arrival times.
>  
> The key issue is well-described in the sandard analysis of the M/M/1 queue 
> (e.g. https://en.wikipedia.org/wiki/M/M/1_queue 
> ) , which is done only for Poisson 
> processes, and is also limited to "stable" systems. But networks are never 
> stable when fully loaded. They get unstable and those instabilities persist 
> for a long time in the network. Instability is at core the underlying 
> *requirement* of the Internet's usage.
>  
> So specifically: real networks, even large ones, and certainly the Internet 
> today, are not asymptotic limits of sums of stationary stochastic arrival 
> processes. Each esternal terminal of any real network has a real user there, 
> running a real application, and the network is a complex graph. This makes it 
> completely unlike a single queue. Even the links within a network carry a 
> relatively small number of application flows. There's no ability to apply the 
> Law of Large Numbers to the distributions, because any particular path 
> contains only a small number of serialized flows with hightly variable rates.
>  
> Here's an example of what really happens in a real network (I've observed 
> this in 5 different cities on ATT's cellular network, back when it was 
> running Alcatel Lucent HSPA+ gear in those cities).
> But you can see this on any network where transient overload occurs, creating 
> instability.
>  
>  
> At 7 AM, the data transmission of the network is roughty stable. That's 
> because no links are overloaded within the network. Little's Law can tell you 
> by observing the delay and throughput on any path that the average delay in 
> the network is X.
>  
> Continue sampling delay in the network as the day wears on. At about 10 AM, 
> ping delay starts to soar into the multiple second range. No packers are 
> lost. The peak ping time is about 4000 milliseconds - 4 seconds in most of 
> the networks. This is in downtown, no radio errors are reported, no link 
> errors.
> So it is all queueing delay. 
>  
> Now what Little's law doesn't tell you much about average delay, because 
> clearly *some* subpiece of the network is fully saturated. But what is 
> interesting here is what is happening and where. You can't tell what is 
> saturated, and in fact the entire network is quite unstable, because the peak 
> is constantly varying and you don't know where the throughput is. All the 
> packets are now arriving 4 seconds or so later.
>  
> Why is the situaton not worse than 4 seconds? Well, there are multiple things 
> going on:
>  
> 1) TCP may be doing a lot of retransmissions (non-Poisson at all, not random 
> either. The arrival process is entirely deterministic in each source, based 
> on the retransmission timeout) or it may not be.
>  
> 2) Users are pissed off, because they clicked on a web page, and got nothing 
> back. They retry on their screen, or they try another site. Meanwhile, the 
> underlying TCP connection remains there, pumping the network full of more 
> packets on that old path, whic

Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[Holland, Jake via Bloat]

Hi David,

That’s an interesting point, and I think you’re right that packet arrival is 
poorly modeled as a Poisson process, because in practice packet transmissions 
are very rarely unrelated to other packet transmissions.

But now you’ve got me wondering what the right approach is.  Do you have any 
advice for how to improve this kind of modeling?

I’m thinking maybe a useful adjustment is to use Poisson start times on packet 
bursts, with a distribution on some burst characteristics? (Maybe like 
duration, rate, choppiness?)  Part of the point being that burst parameters 
then have a chance to become significant, as well as the load from aggregate 
user behavior.

And although I think user behavior is probably often ok to model as independent 
(outside of a background average that changes by time of day), in some contexts 
maybe it needs a 2nd overlay for bursts in user activity to address 
user-synchronizing events...  But for some problems I expect this kind of 
approach might still miss important feedback loop effects, and maybe for some 
problems it needs a more generalized suite of patterns besides a “burst”.  But 
maybe it would still be a step in the right direction for examining network 
loading problems in the abstract?

Or maybe it’s better to ask a different question:
Are there any good exemplars to follow here?  Any network traffic analysis (or 
related) work you’d recommend as having useful results that apply more broadly 
than a specific set of simulation/testing parameters, and that you wish more 
people would follow their example?

Also related: any particular papers come to mind that you wish someone would 
re-do with a better model?


Anyway, coming back to where that can of worms opened, I gotta say I like the 
“language of math” idea as a goal to aim for, and it would be surprising to me 
if no such useful information could be extracted from iperf runs.

A Little’s Law-based average queue estimate sounds possibly useful to me 
(especially compared across different runs or against external stats on 
background cross-traffic activity), and some kind of tail analysis on latency 
samples also sounds relevant to user experience. Maybe there’s some other 
things that would be better to include?

Best regards,
Jake


From: "David P. Reed" 
Date: Fri,2021-07-09 at 12:31 PM
To: Luca Muscariello 
Cc: Cake List , Make-Wifi-fast 
, Leonard Kleinrock , 
Bob McMahon , "starl...@lists.bufferbloat.net" 
, "co...@lists.bufferbloat.net" 
, cerowrt-devel 
, bloat , Ben 
Greear 
Subject: Re: [Bloat] Little's Law mea culpa, but not invalidating my main point


Len - I admit I made a mistake in challenging Little's Law as being based on 
Poisson processes. It is more general. But it tells you an "average" in its 
base form, and latency averages are not useful for end user applications.



However, Little's Law does assume something that is not actually valid about 
the kind of distributions seen in the network, and in fact, it is NOT true that 
networks converge on Poisson arrival times.



The key issue is well-described in the sandard analysis of the M/M/1 queue 
(e.g. 
https://en.wikipedia.org/wiki/M/M/1_queue)
 , which is done only for Poisson processes, and is also limited to "stable" 
systems. But networks are never stable when fully loaded. They get unstable and 
those instabilities persist for a long time in the network. Instability is at 
core the underlying *requirement* of the Internet's usage.



So specifically: real networks, even large ones, and certainly the Internet 
today, are not asymptotic limits of sums of stationary stochastic arrival 
processes. Each esternal terminal of any real network has a real user there, 
running a real application, and the network is a complex graph. This makes it 
completely unlike a single queue. Even the links within a network carry a 
relatively small number of application flows. There's no ability to apply the 
Law of Large Numbers to the distributions, because any particular path contains 
only a small number of serialized flows with hightly variable rates.



Here's an example of what really happens in a real network (I've observed this 
in 5 different cities on ATT's cellular network, back when it was running 
Alcatel Lucent HSPA+ gear in those cities).

But you can see this on any network where transient overload occurs, creating 
instability.





At 7 AM, the data transmission of the network is roughty stable. That's because 
no links are overloaded within the network. Little's Law can tell you by 
observing the delay and throughput on any path that the average delay in the 
network is X.



Continue sampling delay in the network as the day wears on. At about 10 AM, 
ping delay starts to soar into the multiple second range. No packers are lost. 
The peak ping time is about 4000 milliseconds - 4 seconds

Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[Bob McMahon via Bloat]

A bit off topic per the control and queueing theory discussion; a four
second latency is going to fail our regression automation rigs. Way too
many WiFi users, particularly for games, require sub few hundreds of
milliseconds and sometimes even much lower. A TCP connect() getting behind
a 4 second buffer bloat queue is also a big fail for these test rigs.
Completely agree that average latency isn't what "users" complain about -
it typically requires a tail analysis.

Bob


On Fri, Jul 9, 2021 at 12:31 PM David P. Reed  wrote:

> Len - I admit I made a mistake in challenging Little's Law as being based
> on Poisson processes. It is more general. But it tells you an "average" in
> its base form, and latency averages are not useful for end user
> applications.
>
>
>
> However, Little's Law does assume something that is not actually valid
> about the kind of distributions seen in the network, and in fact, it is NOT
> true that networks converge on Poisson arrival times.
>
>
>
> The key issue is well-described in the sandard analysis of the M/M/1 queue
> (e.g. https://en.wikipedia.org/wiki/M/M/1_queue) , which is done only for
> Poisson processes, and is also limited to "stable" systems. But networks
> are never stable when fully loaded. They get unstable and those
> instabilities persist for a long time in the network. Instability is at
> core the underlying *requirement* of the Internet's usage.
>
>
>
> So specifically: real networks, even large ones, and certainly the
> Internet today, are not asymptotic limits of sums of stationary stochastic
> arrival processes. Each esternal terminal of any real network has a real
> user there, running a real application, and the network is a complex graph.
> This makes it completely unlike a single queue. Even the links within a
> network carry a relatively small number of application flows. There's no
> ability to apply the Law of Large Numbers to the distributions, because any
> particular path contains only a small number of serialized flows with
> hightly variable rates.
>
>
>
> Here's an example of what really happens in a real network (I've observed
> this in 5 different cities on ATT's cellular network, back when it was
> running Alcatel Lucent HSPA+ gear in those cities).
>
> But you can see this on any network where transient overload occurs,
> creating instability.
>
>
>
>
>
> At 7 AM, the data transmission of the network is roughty stable. That's
> because no links are overloaded within the network. Little's Law can tell
> you by observing the delay and throughput on any path that the average
> delay in the network is X.
>
>
>
> Continue sampling delay in the network as the day wears on. At about 10
> AM, ping delay starts to soar into the multiple second range. No packers
> are lost. The peak ping time is about 4000 milliseconds - 4 seconds in most
> of the networks. This is in downtown, no radio errors are reported, no link
> errors.
>
> So it is all queueing delay.
>
>
>
> Now what Little's law doesn't tell you much about average delay, because
> clearly *some* subpiece of the network is fully saturated. But what is
> interesting here is what is happening and where. You can't tell what is
> saturated, and in fact the entire network is quite unstable, because the
> peak is constantly varying and you don't know where the throughput is. All
> the packets are now arriving 4 seconds or so later.
>
>
>
> Why is the situaton not worse than 4 seconds? Well, there are multiple
> things going on:
>
>
>
> 1) TCP may be doing a lot of retransmissions (non-Poisson at all, not
> random either. The arrival process is entirely deterministic in each
> source, based on the retransmission timeout) or it may not be.
>
>
>
> 2) Users are pissed off, because they clicked on a web page, and got
> nothing back. They retry on their screen, or they try another site.
> Meanwhile, the underlying TCP connection remains there, pumping the network
> full of more packets on that old path, which is still backed up with
> packets that haven't been delivered that are sitting in queues. The real
> arrival process is not Poisson at all, its a deterministic, repeated
> retrsnsmission plus a new attempt to connect to a new site.
>
>
>
> 3) When the users get a web page back eventually, it is filled with names
> of other pieces needed to display that web page, which causes some number
> (often as many as 100) new pages to be fetched, ALL at the same time.
> Certainly not a stochastic process that will just obey the law of large
> numbers.
>
>
>
> All of these things are the result of initial instability, causing queues
> to build up.
>
>
>
> So what is the state of the system? is it stable? is it stochastic? Is it
> the sum of enough stochastic stable flows to average out to Poisson?
>
>
>
> The answer is clearly NO. Control theory (not queuing theory) suggests
> that this system is completely uncontrolled and unstable.
>
>
>
> So if the system is in this state, what does Little's Lemma tell us? W

Re: [Bloat] Little's Law mea culpa, but not invalidating my main point

[David P. Reed]

Len - I admit I made a mistake in challenging Little's Law as being based on 
Poisson processes. It is more general. But it tells you an "average" in its 
base form, and latency averages are not useful for end user applications.
 
However, Little's Law does assume something that is not actually valid about 
the kind of distributions seen in the network, and in fact, it is NOT true that 
networks converge on Poisson arrival times.
 
The key issue is well-described in the sandard analysis of the M/M/1 queue 
(e.g. [ https://en.wikipedia.org/wiki/M/M/1_queue ]( 
https://en.wikipedia.org/wiki/M/M/1_queue )) , which is done only for Poisson 
processes, and is also limited to "stable" systems. But networks are never 
stable when fully loaded. They get unstable and those instabilities persist for 
a long time in the network. Instability is at core the underlying *requirement* 
of the Internet's usage.
 
So specifically: real networks, even large ones, and certainly the Internet 
today, are not asymptotic limits of sums of stationary stochastic arrival 
processes. Each esternal terminal of any real network has a real user there, 
running a real application, and the network is a complex graph. This makes it 
completely unlike a single queue. Even the links within a network carry a 
relatively small number of application flows. There's no ability to apply the 
Law of Large Numbers to the distributions, because any particular path contains 
only a small number of serialized flows with hightly variable rates.
 
Here's an example of what really happens in a real network (I've observed this 
in 5 different cities on ATT's cellular network, back when it was running 
Alcatel Lucent HSPA+ gear in those cities).
But you can see this on any network where transient overload occurs, creating 
instability.
 
 
At 7 AM, the data transmission of the network is roughty stable. That's because 
no links are overloaded within the network. Little's Law can tell you by 
observing the delay and throughput on any path that the average delay in the 
network is X.
 
Continue sampling delay in the network as the day wears on. At about 10 AM, 
ping delay starts to soar into the multiple second range. No packers are lost. 
The peak ping time is about 4000 milliseconds - 4 seconds in most of the 
networks. This is in downtown, no radio errors are reported, no link errors.
So it is all queueing delay. 
 
Now what Little's law doesn't tell you much about average delay, because 
clearly *some* subpiece of the network is fully saturated. But what is 
interesting here is what is happening and where. You can't tell what is 
saturated, and in fact the entire network is quite unstable, because the peak 
is constantly varying and you don't know where the throughput is. All the 
packets are now arriving 4 seconds or so later.
 
Why is the situaton not worse than 4 seconds? Well, there are multiple things 
going on:
 
1) TCP may be doing a lot of retransmissions (non-Poisson at all, not random 
either. The arrival process is entirely deterministic in each source, based on 
the retransmission timeout) or it may not be.
 
2) Users are pissed off, because they clicked on a web page, and got nothing 
back. They retry on their screen, or they try another site. Meanwhile, the 
underlying TCP connection remains there, pumping the network full of more 
packets on that old path, which is still backed up with packets that haven't 
been delivered that are sitting in queues. The real arrival process is not 
Poisson at all, its a deterministic, repeated retrsnsmission plus a new attempt 
to connect to a new site.
 
3) When the users get a web page back eventually, it is filled with names of 
other pieces needed to display that web page, which causes some number (often 
as many as 100) new pages to be fetched, ALL at the same time. Certainly not a 
stochastic process that will just obey the law of large numbers.
 
All of these things are the result of initial instability, causing queues to 
build up.
 
So what is the state of the system? is it stable? is it stochastic? Is it the 
sum of enough stochastic stable flows to average out to Poisson?
 
The answer is clearly NO. Control theory (not queuing theory) suggests that 
this system is completely uncontrolled and unstable.
 
So if the system is in this state, what does Little's Lemma tell us? What is 
the meaning of that hightly variable 4 second delay on ping packets, in terms 
of average utilizaton of the network?
 
We don't even know what all the users really might need, if the system hadn't 
become unstable, because some users have given up, and others are trying even 
harder, and new users are arriving.
 
What we do know, because ATT (at my suggestion) reconfigured their system after 
blaming Apple Computer company for "bugs" in the original iPhone in public, is 
that simply *dropping* packets sitting in queues more than a couple 
milliseconds MADE THE USERS HAPPY. Apparently the required capacity was there 
all

Re: [Bloat] [Make-wifi-fast] [Starlink] [Cerowrt-devel] Due Aug 2: Internet Quality workshop CFP for the internet architecture board

[Luca Muscariello]

For those who might be interested in Little's law
there is a nice paper by John Little on the occasion
of the 50th anniversary  of the result.

https://www.informs.org/Blogs/Operations-Research-Forum/Little-s-Law-as-Viewed-on-its-50th-Anniversary

https://www.informs.org/content/download/255808/2414681/file/little_paper.pdf

Nice read.
Luca

P.S.
Who has not a copy of L. Kleinrock's books? I do have and am not ready to
lend them!

On Fri, Jul 9, 2021 at 11:01 AM Leonard Kleinrock  wrote:

> David,
>
> I totally appreciate  your attention to when and when not analytical
> modeling works. Let me clarify a few things from your note.
>
> First, Little's law (also known as Little’s lemma or, as I use in my book,
> Little’s result) does not assume Poisson arrivals -  it is good for *any*
> arrival process and any service process and is an equality between time
> averages.  It states that the time average of the number in a system (for a
> sample path *w)* is equal to the average arrival rate to the system
> multiplied by the time-averaged time in the system for that sample path.
> This is often written as   NTimeAvg =λ·TTimeAvg .  Moreover, if the
> system is also ergodic, then the time average equals the ensemble average
> and we often write it as N ̄ = λ T ̄ .  In any case, this requires
> neither Poisson arrivals nor exponential service times.
>
> Queueing theorists often do study the case of Poisson arrivals.  True, it
> makes the analysis easier, yet there is a better reason it is often used,
> and that is because the sum of a large number of independent stationary
> renewal processes approaches a Poisson process.  So nature often gives us
> Poisson arrivals.
>
> Best,
> Len
>
>
>
> On Jul 8, 2021, at 12:38 PM, David P. Reed  wrote:
>
> I will tell you flat out that the arrival time distribution assumption
> made by Little's Lemma that allows "estimation of queue depth" is totally
> unreasonable on ANY Internet in practice.
>
>
> The assumption is a Poisson Arrival Process. In reality, traffic arrivals
> in real internet applications are extremely far from Poisson, and, of
> course, using TCP windowing, become highly intercorrelated with crossing
> traffic that shares the same queue.
>
>
> So, as I've tried to tell many, many net-heads (people who ignore
> applications layer behavior, like the people that think latency doesn't
> matter to end users, only throughput), end-to-end packet arrival times on a
> practical network are incredibly far from Poisson - and they are more like
> fractal probability distributions, very irregular at all scales of time.
>
>
> So, the idea that iperf can estimate queue depth by Little's Lemma by just
> measuring saturation of capacity of a path is bogus.The less Poisson, the
> worse the estimate gets, by a huge factor.
>
>
>
>
> Where does the Poisson assumption come from?  Well, like many theorems, it
> is the simplest tractable closed form solution - it creates a simplified
> view, by being a "single-parameter" distribution (the parameter is called
> lambda for a Poisson distribution).  And the analysis of a simple queue
> with poisson arrival distribution and a static, fixed service time is the
> first interesting Queueing Theory example in most textbooks. It is
> suggestive of an interesting phenomenon, but it does NOT characterize any
> real system.
>
>
> It's the queueing theory equivalent of "First, we assume a spherical
> cow...". in doing an example in a freshman physics class.
>
>
> Unfortunately, most networking engineers understand neither queuing theory
> nor application networking usage in interactive applications. Which makes
> them arrogant. They assume all distributions are poisson!
>
>
>
>
> On Tuesday, July 6, 2021 9:46am, "Ben Greear" 
> said:
>
> > Hello,
> >
> > I am interested to hear wish lists for network testing features. We make
> test
> > equipment, supporting lots
> > of wifi stations and a distributed architecture, with built-in udp, tcp,
> ipv6,
> > http, ... protocols,
> > and open to creating/improving some of our automated tests.
> >
> > I know Dave has some test scripts already, so I'm not necessarily
> looking to
> > reimplement that,
> > but more fishing for other/new ideas.
> >
> > Thanks,
> > Ben
> >
> > On 7/2/21 4:28 PM, Bob McMahon wrote:
> > > I think we need the language of math here. It seems like the network
> > power metric, introduced by Kleinrock and Jaffe in the late 70s, is
> something
> > useful.
> > > Effective end/end queue depths per Little's law also seems useful.
> Both are
> > available in iperf 2 from a test perspective. Repurposing test
> techniques to
> > actual
> > > traffic could be useful. Hence the question around what exact telemetry
> > is useful to apps making socket write() and read() calls.
> > >
> > > Bob
> > >
> > > On Fri, Jul 2, 2021 at 10:07 AM Dave Taht  > > wrote:
> > >
> > > In terms of trying to find "Quality" I have tried to encourage folk to
> > > both read "zen and t

Re: [Bloat] [Starlink] [Make-wifi-fast] [Cerowrt-devel] Due Aug 2: Internet Quality workshop CFP for the internet architecture board

[Leonard Kleinrock]

David,

I totally appreciate  your attention to when and when not analytical modeling 
works. Let me clarify a few things from your note.

First, Little's law (also known as Little’s lemma or, as I use in my book, 
Little’s result) does not assume Poisson arrivals -  it is good for any arrival 
process and any service process and is an equality between time averages.  It 
states that the time average of the number in a system (for a sample path w) is 
equal to the average arrival rate to the system multiplied by the time-averaged 
time in the system for that sample path.  This is often written as   NTimeAvg 
=λ·TTimeAvg .  Moreover, if the system is also ergodic, then the time average 
equals the ensemble average and we often write it as N ̄ = λ T ̄ .  In any 
case, this requires neither Poisson arrivals nor exponential service times.  

Queueing theorists often do study the case of Poisson arrivals.  True, it makes 
the analysis easier, yet there is a better reason it is often used, and that is 
because the sum of a large number of independent stationary renewal processes 
approaches a Poisson process.  So nature often gives us Poisson arrivals.  

Best,
Len



> On Jul 8, 2021, at 12:38 PM, David P. Reed  wrote:
> 
> I will tell you flat out that the arrival time distribution assumption made 
> by Little's Lemma that allows "estimation of queue depth" is totally 
> unreasonable on ANY Internet in practice.
>  
> The assumption is a Poisson Arrival Process. In reality, traffic arrivals in 
> real internet applications are extremely far from Poisson, and, of course, 
> using TCP windowing, become highly intercorrelated with crossing traffic that 
> shares the same queue.
>  
> So, as I've tried to tell many, many net-heads (people who ignore 
> applications layer behavior, like the people that think latency doesn't 
> matter to end users, only throughput), end-to-end packet arrival times on a 
> practical network are incredibly far from Poisson - and they are more like 
> fractal probability distributions, very irregular at all scales of time.
>  
> So, the idea that iperf can estimate queue depth by Little's Lemma by just 
> measuring saturation of capacity of a path is bogus.The less Poisson, the 
> worse the estimate gets, by a huge factor.
>  
>  
> Where does the Poisson assumption come from?  Well, like many theorems, it is 
> the simplest tractable closed form solution - it creates a simplified view, 
> by being a "single-parameter" distribution (the parameter is called lambda 
> for a Poisson distribution).  And the analysis of a simple queue with poisson 
> arrival distribution and a static, fixed service time is the first 
> interesting Queueing Theory example in most textbooks. It is suggestive of an 
> interesting phenomenon, but it does NOT characterize any real system.
>  
> It's the queueing theory equivalent of "First, we assume a spherical cow...". 
> in doing an example in a freshman physics class.
>  
> Unfortunately, most networking engineers understand neither queuing theory 
> nor application networking usage in interactive applications. Which makes 
> them arrogant. They assume all distributions are poisson!
>  
>  
> On Tuesday, July 6, 2021 9:46am, "Ben Greear"  said:
> 
> > Hello,
> > 
> > I am interested to hear wish lists for network testing features. We make 
> > test
> > equipment, supporting lots
> > of wifi stations and a distributed architecture, with built-in udp, tcp, 
> > ipv6,
> > http, ... protocols,
> > and open to creating/improving some of our automated tests.
> > 
> > I know Dave has some test scripts already, so I'm not necessarily looking to
> > reimplement that,
> > but more fishing for other/new ideas.
> > 
> > Thanks,
> > Ben
> > 
> > On 7/2/21 4:28 PM, Bob McMahon wrote:
> > > I think we need the language of math here. It seems like the network
> > power metric, introduced by Kleinrock and Jaffe in the late 70s, is 
> > something
> > useful.
> > > Effective end/end queue depths per Little's law also seems useful. Both 
> > > are
> > available in iperf 2 from a test perspective. Repurposing test techniques to
> > actual
> > > traffic could be useful. Hence the question around what exact telemetry
> > is useful to apps making socket write() and read() calls.
> > >
> > > Bob
> > >
> > > On Fri, Jul 2, 2021 at 10:07 AM Dave Taht  > > wrote:
> > >
> > > In terms of trying to find "Quality" I have tried to encourage folk to
> > > both read "zen and the art of motorcycle maintenance"[0], and Deming's
> > > work on "total quality management".
> > >
> > > My own slice at this network, computer and lifestyle "issue" is aiming
> > > for "imperceptible latency" in all things. [1]. There's a lot of
> > > fallout from that in terms of not just addressing queuing delay, but
> > > caching, prefetching, and learning more about what a user really needs
> > > (as opposed to wants) to know via intelligent agents.
> > >
> > > [0] If you want to get dep

Re: [Bloat] [Cerowrt-devel] Abandoning Window-based CC Considered Harmful (was Re: Bechtolschiem)

[Erik Auerswald]

Thank you, David!


On Thu, Jul 08, 2021 at 04:14:00PM -0400, David P. Reed wrote:
> 
> Keep It Simple, Stupid.
>  
> That's a classic architectural principle that still applies. Unfortunately 
> folks who only think hardware want to add features to hardware, but don't 
> study the actual real world version of the problem.
>  
> IMO, and it's based on 50 years of experience in network and operating 
> systems performance, latency (response time) is almost always the primary 
> measure users care about. They never care about maximizing "utilization" of 
> resources. After all, in a city, you get maximum utilization of roads when 
> you create a traffic jam. That's not the normal state. In communications, the 
> network should always be at about 10% utilization, because you never want a 
> traffic jam across the whole system to accumulate. Even the old Bell System 
> was engineered to not saturate the links on the worst minute of the worst 
> hour of the worst day of the year (which was often Mother's Day, but could be 
> when a power blackout occurs).
>  
> Yet, academics become obsessed with achieving constant very high utilization. 
> And sometimes low leve communications folks adopt that value system, until 
> their customers start complaining.
>  
> Why doesn't this penetrate the Net-Shaped Heads of switch designers and 
> others?
>  
> What's excellent about what we used to call "best efforts" packet delivery 
> (drop early and often to signal congestion) is that it is robust and puts the 
> onus on the senders of traffic to sort out congestion as quickly as possible. 
> The senders ALL observe congested links quite early if their receivers are 
> paying attention, and they can collaborate *without even knowing who the 
> others congesting the link are*. And by picking the heaviest congestors with 
> higher probability to drop, fq_codel pushes back in a "fair" way when 
> congestion actually crops up. (probabilistically).
>  
> It isn't the responsibility of routers to get packets through at any cost. 
> It's their responsibility to signal congestion early enough that it doesn't 
> persist very long at all due to source based rate adaptation.
> In other words, a router's job is to route packets and do useful telemetry 
> for the end points using it at the instant.
>  
> Please stop focusing on what is an irrelevant metric (maximum throughput with 
> maximum utilization in a special situation only).
>  
> Focus on what routers can do well because they actually observe it 
> (instantaneous congestion events) and keep them simple.
> .
> On Thursday, July 8, 2021 10:40am, "Jonathan Morton"  
> said:
> 
> 
> 
> > > On 8 Jul, 2021, at 4:29 pm, Matt Mathis via Bloat
> >  wrote:
> > >
> > > That said, it is also true that multi-stream BBR behavior is quite
> > complicated and needs more queue space than single stream. This complicates 
> > the
> > story around the traditional workaround of using multiple streams to 
> > compensate
> > for Reno & CUBIC lameness at larger scales (ordinary scales today). 
> > Multi-stream does not help BBR throughput and raises the queue occupancy, 
> > to the
> > detriment of other users.
> > 
> > I happen to think that using multiple streams for the sake of maximising
> > throughput is the wrong approach - it is a workaround employed 
> > pragmatically by
> > some applications, nothing more. If BBR can do just as well using a single 
> > flow,
> > so much the better.
> > 
> > Another approach to improving the throughput of a single flow is 
> > high-fidelity
> > congestion control. The L4S approach to this, derived rather directly from 
> > DCTCP,
> > is fundamentally flawed in that, not being fully backwards compatible with 
> > ECN, it
> > cannot safely be deployed on the existing Internet.
> > 
> > An alternative HFCC design using non-ambiguous signalling would be 
> > incrementally
> > deployable (thus applicable to Internet scale) and naturally overlaid on 
> > existing
> > window-based congestion control. It's possible to imagine such a flow 
> > reaching
> > optimal cwnd by way of slow-start alone, then "cruising" there in a true
> > equilibrium with congestion signals applied by the network. In fact, we've
> > already shown this occurring under lab conditions; in other cases it still 
> > takes
> > one CUBIC cycle to get there. BBR's periodic probing phases would not be 
> > required
> > here.
> > 
> > > IMHO, two approaches seem to be useful:
> > > a) congestion-window-based operation with paced sending
> > > b) rate-based/paced sending with limiting the amount of inflight data
> > 
> > So this corresponds to approach a) in Roland's taxonomy.
> > 
> > - Jonathan Morton
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat