Hi Jack,

We do have challenges with network metrics and network telemetry. While better than the days of SNMP, there is a long way to go. Any agency, e.g. the FCC, trying to make quality statements about broadband networks really need to be steeped in these topics. Particularly before govt allocates financial resources to help out.

The goal with iperf 2 is to behave as a application using TCP end to end as that's typically the performance folks primarily care about. Microsecond timestamps are supported at the app level. It does require clock sync for one way delays and the GPS atomic clock is pretty good. But Richard Roy will remind us that there really is no such thing as clock sync per general relativity. The kernel does provide TCP stack stats in realtime too.

I don't think we've instrumented most networks well enough even today. The focus has mostly been on speeds & feeds and then now it's becoming low latency (though latency became important for high frequency traders about a decode ago for their latency arbitrage in trading - so those networks are close to the limits of nature and spacetime) and less on test & measurement. The few of us in T&M do our best.

I added little's law to iperf 2's output probably 5 years ago or more. It's still not utilized well across the industry. One can tell because the displacement units of bloat are bytes or packets but instead we use time or round trips. The bandwidth delay of a link isn't hard to figure out so determining the amount of excess queue memory in proper units has been doable for awhile now.

We're way behind on multivariate analysis too. Even basic statistical process controls (SPC) aren't heavily used. Machines could be carrying more of the load than they do.

There is a lot to do and only so much time. So-called AI and ChatGPT has most everyone's attention it seems.

Bob
During the 90s, I was part of a small team responsible for operating
a corporate small-i internet.  We used cisco equipment, in offices in
100 or so countries, all interconnected with circuits from various
"phone companies".  No fiber or wifi around yet.  Our internet was not
connected to The Internet, but used all the same equipment and
software.

To figure out how to respond to user complaints (e.g., "The network is
really slow today!"), we discovered that we needed good
instrumentation not only in the various routers but also in the
computers in the user environments.   TCP, implemented in the user
computers, did a good job at hiding problems.  For example, TCPs could
get into a semi-stable state where every datagram was getting
transmitted twice with the second copy being discarded at the
destination.  That caused "network throughput" (datagrams/second) to
go way up, but user performance to go way down.

To get the data, we used SNMP and simple scripts to probe the various
devices from our NOC, stuff all the data into a database, and use the
available data analysis tools to figure out what was happening.  At
the time, SNMP "MIB"s were defined for both routers and computers.
So we could collect data from the TCPs involved in a problem, as well
as data from router operations.  When routers reported N
datagrams/time, the associated TCPs might report N/2 retransmissions
from the sender, and N/2 discards from the receiver.

With today's ability to synchronize clocks across the network, it
should now also be possible to report data about latency.
Interactive operations - (A/V conferencing, gaming, telepresence,
etc.) probably have lots of metrics that could be useful much as we
found TCP's to be.

I'm way out of touch with network operations now.  Do current network
managers look at metrics from the enduser computers' perspective?
Can latency be measured between two user devices involved in a problem
report?

Jack

On 11/14/23 13:45, rjmcmahon via Nnagain wrote:

I think the missing metrics & test vectors are around latency more
than bandwidth.

I've attached a WiFi low latency table. Feel free to comment.

Good metrics could allow for a comprehensive analysis, at least from
a WiFi perspective.

Latency under load is a good start, but likely not enough.

Bob

PS. Agreed, the digital transition with storage has many engineers
who have contributed over decades. I'm very grateful to all of them.


I am glad you are reaching out, but it may be difficult for us to do
a
joint filing.

In particular I question the seeming assumption that more wifi
devices
will drive demand for more bandwidth,
and extrapolating from 18 devices forward may also well be a trend
that will reverse completely in favor of more bluetooth and thread
implementations from phone to device.

Of those 20 wifi devices today are probably

1 or more laptops
1 or more tablets
1 or more phones
1 or more tvs

and of those usually only one will be active per person, while they
are in the home, and even then....as one semi-hard number, even at
peak hours (with the libreqos data I have), only 1/6th of households

are watching video, and very, very few, more than one stream at the
same time.

The steady upload bandwidth pumpers are primarily video surveillance

devices (which as a personal preference I would prefer remain in the

home unless otherwise activated). I do not presently know much about

the frame rates or real bandwidth requirements of popular devices
like
ring, etc.  Similarly I am biased towards "Babycams" sending video
from up to downstairs only and not into the cloud. I know I am
bucking
the trend on this, but it will make me skeptical of much "data" that

exists today on it.

Then you have loads of extremely low bandwidth devices - alexa and
other automation is measured in bits/ms, light switches, a couple
bits
a day, audio streaming 128kbit/s (when you use it). Automatic
updates
to phones and tablets, etc, take place entirely asynchronously
nowadays and do not need much bandwidth. A small business just needs

to
*reliably* clear credit card transactions every few minutes.

Perhaps the biggest steady-state bandwidth suck is home gaming
updates, but while a big market, if you haven't noticed birthrates
are
down, and immigration being canceled.

Thus I feel that the opposite number of 70-80% two people or less
per
household  that you are not optimizing for, dominates the curves.

Looking at the actual useage disparity (delta) between fiber'd
cities
and rural, uptake of passive video streaming services vs spotify,
would give me a more pessimistic projection than most. Regrettably I

lack the time and as few fund accurate scenarios, I would merely be
willing to write down my estimate and find some sort of online
"futures" market to place puts on.

Lastly, a goodly percentage of the people I know just need food,
shelter, a job and a phone, and with broadband costs skyrocketing,
aside from the gaming market and business, that is all they can
afford, even with ACP.  And all that they need. Nobody has a
landline
anymore, and if it weren't for "Tv", few would want broadband at
even
25/10.

On Tue, Nov 14, 2023 at 2:40 PM rjmcmahon via Nnagain
<[email protected]> wrote:

Thanks for sharing this. I agree this works for researchers.

I think we're at a different state and economic returns matter too.

I sent the following to our engineers in hopes we can all better
understand what we're all trying to accomplish.

Hi All,

The attached Notice of Inquiry by the FCC shows how much our work
matters to most everyone in our country (and, by inference,
worldwide.)
Broadband networks are no longer entertainment or social networks
but
they are critical to all regardless of gender, age, race, ethnic
group,
etc. People's health, learning, and ability to earn for their
families
all depend on us providing world class engineering to our customers
who
in turn provide these networks for each and all of us, our friends &

families, our neighbors, and most everyone else.

Early in my career, I worked at Cisco and had the privilege to work
on
some of the first BGP routers that enabled the commercial build out
of
the internet, and I'm very thankful we did that way ahead of the
2019
pandemic. There was no "pandemic use case" that drove us - we just
wanted to build the best products that engineers could build. A
worldwide pandemic w/o the internet could have been disastrous - so
that
work by many in the mid 1990s seems to have paid off well.

I hope you each realize, today, what you've accomplished since then
and
continue to be a part of. It's truly significant. It's been a high
honor
to work with so many of you over the last 14+ years.

This is beautiful, btw. I feel much the same way about linux being
now
so used heavily in the space program,
and all our code, and hardware, that will propagate across the solar

system, and of the millions of people, that contributed to it.

To the FCC report:

We begin this annual inquiry in the wake of the COVID-19 pandemic
during
which time Americans increasingly turned to their broadband
connections
to conduct their lives online by using telemedicine to access
healthcare, working from home, attending classes remotely,
connecting by
video with out-of-town family and friends, and streaming
entertainment.
Our experiences with the pandemic made it clear that broadband is no

longer a luxury but a necessity that will only become more important

with time. Never before has the critical importance of ensuring that
all
Americans have access to high-speed, affordable broadband been more
evident.

Also note, we have more work to do. We need to increase resiliency
as an
example. Also, the thing I'm most passionate about is low latency.
The
FCC is now recognizing the importance of that. People are slowly
learning why latency is becoming equally important to capacity when
it
comes to quality of service.

Bob

PS. The rest is TLDR but I thought I post some snippets for those
interested

We believe that in examining household use cases, a simple summation
of
required speeds for individual activities may provide a misleading
picture of actual broadband needs for at least three reasons. First,
we
believe it is appropriate to take into account at least occasional
downloads of very large files which can be bandwidth-intensive.
Second,
it is important to account for larger households; in 2022,
approximately
21% of all U.S. households had four or more people, and the number
of
families seeking out multigenerational homes to live with additional

relatives rose.57 Households of all sizes must have sufficient
bandwidth
to satisfy their needs. In addition, the number of connected devices
per
household continues to grow, from 18.6 in the average household in
2021
to 20.2 in the first half of 2022.58 Taking these factors into
account
suggests that fixed broadband download/upload needs could easily
exceed
100/20 Mbps.

...

Service Quality. We recognize that other factors, besides the speed
of a
broadband connection, can affect consumers’ ability to use the
services
effectively. Chief among these factors is latency, which is the
measure
of the time it takes a packet of data to travel from one point in
the
network to another, and which is typically measured by round-trip
time
in milliseconds (ms). As a measurement of advanced
telecommunications
capability, latency can be critical because it affects a
consumer’s
ability to use real-time applications, including voice over Internet

Protocol, video calling, distance learningapplications, and online
gaming. Actual (as opposed to advertised) speed received,
consistency of
speed, and data allowances are also important. Such factors are not
simply a matter of service interruptions and consumer
satisfaction—they
have a real and significant effect on Americans’ ability to use
critical
web-based applications, including those that facilitate telehealth,
telework, and virtual learning.

In the beginning days of the Arpanet, circa early 1970s, ARPA made
a
policy decision about use of the Arpanet.  First, Arpa Program
Managers, located on the East Coast of the US, were assigned
computer
accounts on USC-ISIA, located on the West Coast in LA.  Thus to do

their work, exchanging email, editting documents, and such, they
had
to *use* the Arpanet to connect their terminals in Washington to
the
PDP-10 in California - 3000 miles away.

Second, ARPA began requiring all of their contractors (researchers
at
Universities etc.) to interact with Arpa using email and FTP.   If

your site was "on the Arpanet", you had to use the Arpanet.  If
you
wanted your proposal for next year's research to be funded, you
had to
submit your proposal using the net.

This policy caused a profound attention, by everyone involved, to
making the Arpanet work and be useful as a collaboration tool.

JCR Licklider (aka Lick) was my advisor at MIT, and then my boss
when
I joined the Research Staff.   Lick had been at ARPA for a while,
promoting his vision of a "Galactic Network" that resulted in the
Arpanet as a first step.  At MIT, Lick still had need for lots of
interactions with others.   My assignment was to build and operate
the
email system for Lick's group at MIT on our own PDP-10.  Lick had
a
terminal in his office and was online a lot.   If email didn't
work, I
heard about it.   If the Arpanet didn't work, BBN heard about it.

This pressure was part of Arpa policy.   Sometimes it's referred
to as
"eating your own dog food" -- i.e., making sure your "dog" will
get
the same kind of nutrition you enjoy.   IMHO, that pressure policy
was
important, perhaps crucial, to the success of the Arpanet.

In the 70s, meetings still occurred, but a lot of progress was
made
through the use of the Arpanet.   You can only do so much with
email
and file interactions.  Today, the possibilities for far richer
interactions are much more prevalent.   But IMHO they are held
back,
possibly because no one is feeling the pressure to "make it work".

Gigabit throughputs are common, but why does my video and audio
still
break up...?

It's important to have face-to-face meetings, but perhaps if the
IETF
scheduled a future meeting to be online only, whatever needs to
happen
to make it work would happen?  Perhaps...

Even a "game" might drive progress.  At Interop '92, we
resurrected
the old "MazeWars" game using computers scattered across the show
exhibit halls.  The engineers in the control room above the floor
felt
the pressure to make sure the Game continued to run.  At the time,
the
Internet itself was too slow for enjoyable gameplay at any
distance.
Will the Internet 30 years later work?

Or perhaps the IETF, or ISOC, or someone could take on a highly
visible demo involving non-techie end users.   An online meeting
of
the UN General Assembly?   Or some government bodies - US
Congress,
British Parliament, etc.

Such an event would surface the issues, both technical and policy,
to
the engineers, corporations, policy-makers, and others who might
have
the ability and interest to "make it work".

Jack

On 11/14/23 10:10, Sebastian Moeller wrote:

Hi Jack,

On Nov 14, 2023, at 13:02, Jack Haverty via Nnagain
<[email protected]> wrote:

If video conferencing worked well enough, they would not have to

all get together in one place and would instead hold IETF
meetings
online ...?

[SM] Turns out that humans are social creatures, and some things
work better face-to-face and in the hallway (and if that is only
building trust and sympathy) than over any remote technology.

Did anyone measure latency?   Does anyone measure throughput of
"useful" traffic - e.g., excluding video/audio data that didn't
arrive in time to be actually used on the screen or speaker?

[SM] Utility is in the eye of the beholder, no?

Jack Haverty

On 11/14/23 09:25, Vint Cerf via Nnagain wrote:

if they had not been all together they would have been consuming
tons of video capacity doing video conference calls....

:-))
v

On Tue, Nov 14, 2023 at 10:46 AM Livingood, Jason via Nnagain
<[email protected]> wrote:
On the subject of how much bandwidth does one household need,
here's
a fun stat for you.

At the IETF’s 118th meeting last week (Nov 4 – 10, 2023),
there
were over 1,000 engineers in attendance. At peak there were 870
devices connected to the WiFi network. Peak bandwidth usage:

• Downstream peak ~750 Mbps
• Upstream ~250 Mbps

From my pre-meeting Twitter poll
(https://twitter.com/jlivingood/status/1720060429311901873):

<image001.png>

_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain

--
Please send any postal/overnight deliveries to:
Vint Cerf
Google, LLC
1900 Reston Metro Plaza, 16th Floor
Reston, VA 20190
+1 (571) 213 1346

until further notice

_______________________________________________
Nnagain mailing list

[email protected]
https://lists.bufferbloat.net/listinfo/nnagain

_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain
_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain
_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain

_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain
_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain
_______________________________________________
Nnagain mailing list
[email protected]
https://lists.bufferbloat.net/listinfo/nnagain

Reply via email to