On Sun, Nov 16, 2008 at 09:24:19PM +0000, Eris Discordia wrote:
That isn't happening. All we have is one TCP connection and one small
program exporting file service.
I see. But then, is it the "small program exporting file service" that
does the multiplexing? I mean, if two machines import a gateway's /net
and both run HTTP servers binding to and listening on *:80 what takes
care of which packet belongs to which HTTP server?
I don't think you've quite got it yet.... also I swore I wouldn't post in
this thread. Oh well, here goes.
First, let me draw a picture, just to introduce the characters:
+----------+ +---------+
| Internal |<--Ethernet-->| Gateway |<--Ethernet-->(Internet)
| Computer | ^ +---------+
+----------+ |
|
+----------+ |
| Other |<--+
| Internal |
| Computer |
+----------+
OK. Here, we have two internal computers (IC and OIC) and a gateway G.
There are two Ethernet networks in flight, one connecting IC, OIC, and G,
and the other connecting G to the internet at large, somehow (e.g. ADSL).
IC and OIC both initialize somehow, say using DHCP from G, and bring their
network stacks up using this information. Their kernels now provide the
services that will be mounted, by convention, at /net, again using this
private information. G initializes statically, bringing its IP stack up
using two interfaces, with two routes: one for internal traffic, and one
default route.
So far, it's all very similar between Plan 9 and Linux. Here's where
our story diverges.
In Linux, IC and OIC are both taught that they have default routes to the
Internet At Large by sending IP datagrams to G's internal ethernet
interface. How this works in more detail, they don't care. G will also
send them corresponding IP datagrams from its interface; that's all they
care about. G works furiously to decode the network traffic bound for the
internet and NA(P)T it out to its gateway, maintaining very detailed
tables about TCP connections, UDP datagrams, etc, etc. How could it be
but any other way?
Let's redraw the picture, as things stand now, under Plan 9, just so
things are clear:
(OIC similar to IC)
+----Internal Computer--------------------+ |
| bind '#I' /net | |
| # I : Kernel IP stack (192.168.1.2) | |
| # l : Kernel ethernet driver |<---Ethernet---+
+-----------------------------------------+ |
|
+----Gateway------------------------------+ |
| bind '#I' /net | |
| # I : Kernel IP stack (192.168.1.1) | |
| (4.2.2.2) | |
| # l : Kernel ethernet driver (ether0) |<---Ethernet---+
| (ether1) |<---Ethernet----->(Internet)
+-----------------------------------------+
In Plan 9, G behaves like any other machine, building a /net out of pieces
exported by its kernel, including the bits that know how to reach the
internet at large through the appropriate interface. Good so far?
Let's have G run an exportfs, exposing its /net on the internal IP
address. This /net knows how to talk to the internal addresses and the
external ones.
Meanwhile, IC can reach out and import G's /net, binding it at /net.alt,
let's say. Now, programs can talk to the Internet by opening files in
/net.alt. These open requests will be carried by IC's mount driver, and
then IC's network stack, to G, whereupon the exportfs (in G's userland)
will forward them to its idea of /net (by open()ing, read()ing,
write()ing, etc.), which is the one built on G's kernel, which knows how
to reach the Internet. Tada! Picture time:
(OIC)
+----Internal Computer-------------------------+ |
| abaco: open /net.alt/tcp/clone | |
| | |
| import tcp!192.168.1.1!9fs /net.alt (devmnt) | |
| bind '#I' /net | |
| # I : Kernel IP stack (192.168.1.2) | |
| # l : Kernel ethernet driver |<---------+
+----------------------------------------------+ |
|
+----Gateway------------------------------+ |
| exportfs -a -r /net | |
| | |
| bind '#I' /net | |
| # I : Kernel IP stack (192.168.1.1) | |
| (4.2.2.2) | |
| # l : Kernel ethernet driver (ether0) |<---Ethernet---+
| (ether1) |<---Ethernet----->(Internet)
+-----------------------------------------+
This works perfectly for making connections: IC's IP stack is aware only
of devmnt requests, and G's IP stack is aware of some trafic to&from a
normal process called exportfs, and that that process happens to be
making network requests via #I bound at /net.
The beauty of this design is just how well it works, everywhere, for
everything you'd ever want.
Now, suppose IC goes to listen on TCP:80, by opening /net.alt/tcp/clone.
The same flow of events happen, and to a certain extent, G's network stack
thinks that the exportfs program (running on G) is listening on TCP:80.
exportfs dutifully copies the /net data back to its client.
Naturally, if another program on G were already listening on TCP:80, or
the same program (possibly exportfs) attempted to listen twice (if, say,
OIC played the same game and also tried to listen on G's TCP:80), it
would be told that the port was busy. This error would be carried back
along the exportfs path just as any other.
So as you see, there is no need to take care of "which packet belongs to
which server" since there can, of course, be only one server listening on
TCP:80. That server is running on G, and behaves like any other program.
That it just so happens to be an exportfs program, relaying data to and
from another computer, is immaterial.
This also works for FTP, and UDP, and ESP (which are notorious problems in
the NAT world), and for IL, and for IPv6, and for ethernet frames (!), and
... you get the idea. It does this with no special tools, no complex
code, and a very minimal per-connection overhead (just the IC and G
kernels and G's exportfs tracking a file descriptor).
There are no connection tracking tables anywhere in this design. There
are just normal IP requests over normal ethernet frames, and a few more
TCP (or IL) connections transporting 9P data.
On a UNIX clone, or on Windows, because there is exactly one TCP/IP
protocol stack in usual setups no two programs can bind to the same port
at the same time. I thought Plan 9's approach eliminated that by keeping
a distinct instance of the stack for each imported /net.
There can, in fact, be multiple IP stacks on a Plan 9 box, bound to
multiple Ethernet cards, as you claim. (In fact, one can import another
computer's ethernet cards and run snoopy, or build a network stack, using
those instead.) I don't think that's relevant to the example at hand,
though, as should be clear from the above.
--nwf;
