Here I discuss the forms the end-to-end principle, considered as a
computer systems architectural principle, has taken over the years,
how it is reshaping society, and how it is merely an extension of the
fundamental human right to freedom of speech --- and the consequences.
I'm sorry to be sending out such a rough draft. It probably should be
completely rewritten.
Contents
--------
The End-To-End Principle in Mainframes
The End-To-End Principle in the Internet
Scholarly Writing and the End-To-End Principle
The OSI Layer Cake
Content-Centric Networking
Who Are The Ends?
Manifesto
Footnotes
The End-To-End Principle in Mainframes
--------------------------------------
The end-to-end principle was first stated, as far as I know, in Butler
Lampson's "Hints For Systems Design" paper back in the 1970s.
Basically, as I remember it, Butler's argument is that any kind of
error-checking or error correction has to be done by the original
creator of the information and the final user in order to be
trustworthy. In Butler's example (again, as I remember it), in
getting a data file from the program that originally created it to the
program that eventually uses it, there are a number of places the file
can be corrupted:
- the operating system routines that accept data from the program may
have defects that corrupt the data;
- the memory in which those routines store the data before writing it
to disk may be flaky and corrupt the data;
- the bus that connects the memory to the disk controller may be
flaky;
- the disk controller may be flaky;
- the wires from the disk controller to the disk may be noisy;
- the magnetic domains on the spinning disk may demagnetize over time;
- and when reading the data back, again we must contend with wires,
disk controllers, buses, memory, and operating system.
Today we could add Solid Oak Software Cyber Patrol corrupting the data
in transit over a network as another potential defect.
In this example, if the original program adds a CRC to the data file,
the program that uses the file will be able to detect all of these
errors with high probability; while if, say, the disk controller adds
a CRC to each data block on disk, it will cost just as much, but will
detect only a fraction of the errors. While a chain of overlapping
error-checks, each covering only two or three links of the
communications path, could in principle detect an error at any point
along the path, such a chain is much more complex than a single
end-to-end check, and also much more likely to fail.
So this kind of "hop-by-hop" error detection and correction is useful
"as an optimization", because it can often *recover* from errors more
cheaply --- but you shouldn't depend on it.
Butler wrote his paper in the 1970s, when many expensive computers
offered a lot of "hop-by-hop" error detection --- in their CPUs, in
their memory interfaces, in the buses to their peripherals, and so on.
A few years later, Tandem stole the high-reliability computer market
with a design which, I believe, hewed much more closely to the
end-to-end principle: rather than adding trimodular redundancy and
CRCs and the like to each step in the path, they built their computers
as what we would now call "clusters" of individual computers, which (I
believe) had relatively little redundancy inside each one.
The End-To-End Principle in the Internet
----------------------------------------
Telephone networks are designed for high reliability, hop-by-hop: each
node in the path of your call is specified to keep running 99.999% of
the time, at least in the US. Telco packet data protocols like X.25
used a similar approach --- each router in an X.25 path acknowledges
each packet it receives from the previous router, so that a lost or
corrupted packet can be retransmitted from the previous router,
instead of from the original sender. Telco protocols for digital
"isochronous" communications, such as voice data, reserve periodic
timeslots for data transmission --- to make sure that no data is ever
delayed or lost in transit, unless something goes wrong.
The internet protocols have a completely different design. Routers
are free to drop packets at any time without telling anybody, and they
do, even if nothing is going wrong --- simply due to network
congestion. Typically they only drop about 0.1% to 5% of the packets,
at least according to mtr, but they do drop packets, all the time.
They drop even more packets when a router goes down.
This allows internet protocols to work correctly over lossy radio
networks, noisy modem lines, and long network paths operated by a
dozen or more independent companies.
The sending and receiving computers are responsible for making sure
nothing gets lost or corrupted, usually using a protocol called TCP.
Each packet of data has a checksum to detect (with some probability)
whether the packet has been corrupted in transit, and a sequence
number so the receiver knows what order the packets go in, whether
it's receiving duplicate packets, and whether it's missing any.
Radio networks typically do retransmit packets at a hop-by-hop
level, because they often have packet loss rates above 20% per
hop, and rarely below 5%. Amateur radio operators use a variant
of X.25 called AX.25 for this. If you're transmitting over a
string of five noisy radio hops, each of which drops 20% of your
packets, that's 67% packet loss, and TCP doesn't work very well
any more. So while TCP is more *reliable* at recovering from
packet losses (since it handles routers crashing and being buggy),
AX.25 is more *efficient* at it. A side effect of radio-network
packet retransmission is that sometimes it's the acknowledgement,
not the data packet, that got lost, so sometimes you get multiple
copies of a packet if you have an 802.11 link in your path ---
something you'll almost never see with wireline networks like
Ethernet.
One nice effect of this is that internet communications costs a lot
less than phone communications, because you can do it with shoddier
equipment and more casual agreements among service providers.
The more important effect of this end-to-end principle is that, since
almost all the smarts of an application like voice over IP or file
transfer or hypertext lives in your computer and the computer you're
talking to, not in the dozen routers in between you, you can make a
new networked application and start running it just by convincing one
person to install it on their computer. This is why telephone
networks have been promising videophones for 50 years, investing lots
of money in services, and never delivering, but it took one creative
college student a few days to revolutionize the music industry with
Napster.
Scholarly Writing and the End-To-End Principle
----------------------------------------------
In mainstream journalism, for some reason, it seems to be considered
bad form to cite sources, so that, for example, your readers can judge
whether a quote was taken out of context; perhaps you're supposed to
trust the journalist to have investigated adequately and represented
the facts accurately, not argue with them. As you might expect, this
kind of situation seems to attract the kind of writers who are least
likely to investigate adequately.
By contrast, in academic publishing, there seems to be a norm that no
assertion, however widely accepted, should make it into your paper
without a footnote to tell your readers why you believe it, unless
it's a result of your personal direct observation, in which case
you're expected to say that. Sometimes this makes for arduous reading
and even more arduous writing, but it has the benefit that the reader
can follow the references and, perhaps, find that the citing author
has made a mistake; this makes it less likely that a simple error will
be propagated from generation to generation.
I have seen it asserted somewhere that this norm is what defines
"scholarly writing".
There's a clear analogy here to end-to-end error-correction in
computer systems. The reader is expected to read the words written by
the guy who originally asserted something and make up his own mind
what they mean and whether they're well-supported --- not rely on
error-free transmission of that information by a pyramid of
intermediate authors.
A weaker form of this norm has arisen in the blogosphere: if you refer
to some published document, whether a blog post or a government
report, whether to cite it with approval or dispute it, you are quite
strongly expected to provide a link to that document, so that your
readers can make up their own minds. (You are also expected to link
to the source from which you heard about it.)
The OSI Layer Cake
------------------
Often, computer networks are described in terms of a somewhat bogus
seven-layer cake model developed by OSI, the failed phone-company
attempt to reinvent the internet as something stupid. The lowest
layers are things like the physical medium, the internetworking packet
protocol that packages stuff up so it can move from physical medium to
physical medium, and the stuff that puts checksums and sequence
numbers on it to make sure it can be retransmitted and corruption
detected, and the highest layers are things like data representation
formats and the "application layer", which is the actual "application"
you're using the network for --- HTTP to implement a distributed
hypertext system, say, or IRC for online chatting, or NTP for setting
your computer clock.
The idea is that each communication starts at the top of the stack,
tunnels its way down through the layers to the bottom of the cake,
getting wrapped in successively more goop as it goes, bumps along the
bottom couple of layers to get to its destination, and then erupts up
through the layers at the destination, losing the goop as it goes.
Different network systems have slightly different actual sets of
layers, which map with more or less fudging onto the seven-layer OSI
model.
To be painfully concrete, and use the real layers that actually exist
on the internet, an HTTP request starts as my browser, say, deciding
it wants to load an image from some URL. So it packages that request
into an HTTP GET request, which is a blob of text a few dozen words
long, and asks the operating system to make it a connection to the
server. The operating system [0] sends some DNS packets to random
places around the network, which hopefully tell it the IP address of
the server, and so it sends a TCP SYN packet to that IP address asking
to open a new connection. To actually send the TCP packet, it puts it
inside an IP packet, and then tries to figure out what to do with the
IP packet, which involves consulting its routing tables and probably
deciding it should send it to the same router it uses for everything
else, which is connected to its Ethernet port, at which point it sends
an ARP request out on the Ethernet port asking for the Ethernet
address of the router. The router sends back an ARP reply with the
router's Ethernet "MAC" address, and then the computer sends the
router an Ethernet packet or "frame" (by wiggling voltages on the wire
between the computer and the router), within which is an IP packet,
within which is the TCP packet or "segment". Later on, my computer
will send the router more Ethernet frames containing IP packets
containing TCP segments containing the actual text of the HTTP
request.
At the other end, the router connected to the web server sends a
different Ethernet frame to the web server, containing the same IP
packet containing TCP containing HTTP, and the web server goes through
more or less the same process in reverse: examine the IP packet to see
if it's to the web server or to someone else (and who it's from),
examine the TCP packet to see which connection it's for, hand the HTTP
data to the particular server program handling that connection, and
then figure out what the HTTP data means and what to do about it.
So in this case there are actually only five layers --- HTTP, TCP, IP,
Ethernet MAC, and voltages on a wire, not seven. Other connections
have more or fewer layers, and you can increase the layer count almost
arbitrarily if you start doing stuff like tunneling IP over HTTP,
which you can do. So those are some of the reasons why the
seven-layer model is pretty bogus, but that doesn't make it useless
for discussion.
Anyway, the "application layer" protocol like HTTP is often called
"layer 7", since it corresponds to the seventh (or maybe sixth and
seventh) layers in the OSI model. But humorous discussions of network
architecture often mention layers 8 and 9: layer 8 is the guy sitting
at the keyboard clicking the link, and layer 9 is the organization he
works for.
Content-Centric Networking
--------------------------
Van Jacobson is a guy who's famous for developing a lot of the stuff
that keeps the current internet running. He's recently been working
on some stuff he calls "content-centric networking", on the thesis
that in, say, HTTP, the "ends" being connected are not two computer
application programs, like a web browser and a web server --- they are
a web browser and a named piece of information like an image. The web
server is often merely a passive conduit through which that
information passes.
He points out that a big part of the difference between the internet
and the telephone network is that the telephone network is basically
providing the service of connecting wires together, so that if you
wiggle the voltage level on one wire, the voltage level on the other
wire wiggles too; while all the internet and computer networking work
is layers on top of that. And it turned out that it's a lot easier to
route data around at that higher level instead of hooking up a shared
wire with whoever you wanted to talk to.
So the internet was designed, basically, to provide remote-login and
voice transmission services between a guy and a computer, or between
two guys: making a connection. And all of our security and
performance stuff has been about how to make those connections more
secure and faster and more reliable.
But the web server, which is the ultimate endpoint of the connection
from the HTTP/TCP/IP point of view, is really just another kind of
conduit, and a heck of a lot of the current usage of TCP/IP is for
this kind of thing. So maybe we should be focusing on talking to the
information instead, which is what Freenet and BitTorrent do, not to
the server.
Who Are The Ends?
-----------------
I talk to people over AIM and Yahoo Messenger all the time. They both
run on top of a "reliable" protocol, TCP, which is supposed to keep
data from getting lost or corrupted. But I frequently have trouble
with data getting lost or corrupted with these protocols, for several
reasons:
- Their laptop runs out of battery.
- I lose my wireless connection for too long to keep the TCP
connection alive.
- I'm using a locutorio computer with Cyber Patrol installed, and it
cuts the connection because the other person said, "fuck".
- My IM client crashes.
- I send a message that gets eaten by the buggy IM server that relays
data between us.
- I don't speak Spanish very well, and they're speaking Spanish.
So I use end-to-end error-detection --- I try to remain alert to signs
of miscommunication and communication loss in the conversation, and
"retransmit" by repeating myself or explaining things in a different
way.
Likewise, even the image on the web server is not really the endpoint
of my communication --- it's really the person who put it there, maybe
the owner of the server or maybe not. The server, in this case, is
merely a medium for communication among people. [1]
Generally, in all of these examples, computer programs and computer
data are merely proxies for people who want to communicate.
People are the ends.
PEOPLE ARE THE ENDS!
Manifesto
---------
We have begun to transform our society with the power of
communications networks based on the end-to-end principle, as it
changes us in return. Fundamentally, the power of these
communications networks is the power of free speech or free
communication, which enables the independent investigation of truth.
Free communication is the ability to communicate with anyone who wants
to communicate with you, about anything, as much as you want, in
private or public, as you choose. End-to-end computer networks make
this ability not only legal, but practical and widespread.
In a world of free communication, abusive dictatorships cannot stand;
human suffering cannot be concealed; information arbitrage in markets
becomes marginal; and political propaganda becomes obvious for what it
is, and only those who seek it out will be deceived by it.
Furthermore, the world of free communication decimates the barriers to
knowledge that keep so much of the human race in poverty.
Finally, this new practical ability to exercise our fundamental right
of free speech has given rise to new ways of developing knowledge and
organizing work: ranging from Wikipedia and the free-software movement
(producing fruits like the Firefox web browser and the Linux kernel)
to eBay, Google, and the blogosphere.
Our world already abounds with the fruits of these changes.
Given these blessings, only in the most extreme cases, such as
preventing imprisoned convicts from plotting further crimes, can
interference with this right of free communication be tolerated.
In particular, this means that the following are intolerable:
- restrictions on the use, ownership, importation, or manufacture of
particular equipment for communications, particularly restrictions
intended to prevent people from having access to certain
information, such as legal enforcement of Digital Restrictions
Management schemes;
- restrictions on the permissible content of public communications;
- restrictions on the permissible identities of public speakers, or
requirements that these speakers not be anonymous;
- schemes to prevent anonymous speech and publication, such as
requiring communications technology to attach traceable serial
numbers to each communication;
- schemes to prevent private communication, such as prohibitions or
restrictions on access to encryption hardware or software;
- "takedown notice" mechanisms that enable purported copyright holders
to exercise prior restraint over the speech of others, particularly
without prior judicial review.
Furthermore, these are intolerable not just when they are imposed by
the state, but also when they are imposed by private actors with
sufficient power to make them difficult to escape --- for example, by
a telecommunications company.
Footnotes
---------
[0] Actually, this is usually a library which talks to a server at my
ISP, which does the actual work of spewing DNS packets all over the
internet for me.
[1] There are exceptions, like network monitoring reports and computer
simulations, but nearly all the communication over the web is from
people to people, not between people and computer programs.
