On 2016-05-04 01:15, Hittner, David T (IS) wrote:
Many of the older OS systems create rather small buffers for Ethernet packets,
and do not handle the fast real-time receive rates of 100Mbps and 1000Mb
Ethernet in the simulators very well.
I would prefer if you said "few" instead of "small". There is a
difference, and the issues you describe are more related to the numbers
than the size.
Older machines, like the PDP-11 many times also create rather small
buffers, since protocols like DECnet could be tuned to use smaller than
1500 byte packets. This did not prevent the hardware from receiving 1500
byte packets, but such packets would then use several buffers, that are
chained.
Also, the capacity limitations are often tied to hardware, and not the
OS. The bandwidth of a Unibus is only a couple of Mbit/s, so a 10 Mbit/s
ethernet can produce data faster than the hardware can ever deal with
it. There is some local ram on the ethernet controller, which can deal
with a few packets, but after that, the controller will drop packets.
That said, even if you keep the rates at what the hardware can deal
with, there isn't enough processing power around to work through the
packets fast enough to give memory back to the controller at that rate
either.
This was due to:
1) the OS expectation of being able to service the (maximum, never
seen) Ethernet 10Mbps/sec buffered packets within the time frame allowed
That was not an explicit expectation. The OS will process the packets as
fast as it can.
2) the OS expectation that packets not buffered (dropped) would
eventually be retransmitted at some point by delivery protocols, which have an
implied ACK/NACK delay
This is true of all protocols, even today.
3) Limited physical memory was available for Ethernet packet buffering.
True. But that is, as I said, not the only limitation.
Simulator speeds (for PDPs and VAXen) and modern network speeds are so fast compared to the
original hardware that frequently the OS Ethernet buffers are just too small for efficient network
packet processing. OS-side inefficiency can be addressed by patching the OS with larger Ethernet
buffers. Emulator-side inefficiency can be fixed by "limiting" the Ethernet emulation
speed by throttling it down to a speed more appropriate to the OS buffer processing rates. But this
may cause extra dropped packets, which will likely be "fixed" by protocol retransmission.
Well, I disagree with several claims here.
As the simlator speed increases, the OS can process packets faster, and
thus return the memory back to the controller faster, thereby enabling
higher data rates. This is not a problem, but in fact just allows higher
transmission rates without any problems. And once more, the size of the
buffers are not the issue. The number of buffers are. But on RSX, this
do not appear to be an issue. RSX also do not need any patching to
utilize the network as much as the OS possibly can.
So, in short, RSX is an example that contradicts several of your statements.
There is one issue that I have seen with RSX, and that is if you have a
very fast machine (such as a simulator) talk to a very slow machine
(such as a real machine) using DECnet, and that is that DECnet behaves
very bad when you have high packet loss. Yes, DECnet will retransmit
data, but the timeouts keep increasing, as every time you start
retransmitting, DECnet sends a burst of packets, and the hardware just
can't receive the whole burst, and drop some, causing the timeout to
increase again. Transmission speeds essentially drops to abmyssal values
using DECnet for this reason. The only solution to that is to pace the
transmission from the fast machine, so that the slow one do not drop
packets. Drops not only happen because you run out of buffers, but also
because switches drops packets since we're talking about converting from
100 Mbit/s or GbE down to 10 Mbit/s ethernet. If you just keep blasting
at full speed, the switches will have to drop packets. There is nothing
to avoid this. So it's not even something you can blame, or deal with,
at the slow machine.
I've written a TCP/IP for RSX. It uses the DECnet ethernet drivers, and
TCP/IP just deals better with packet loss, so with TCP/IP such pacing is
not needed.
The goal of early networking was to send and receive packets, not network
efficiency. It wasn't until near-line-rate 10Mbps Ethernet and 100Mbps Fast
Ethernet that network drivers started to think about efficiency by creating
larger buffers, and network efficiency wasn't really fully implemented until 1G
Ethernet became supported by (some) OSs, because it forces much larger buffers
and better processing algorithms.
I would not agree with that. Network efficiency is not something new. It
was very much something they though about back then as well. In some
ways more than now, since the hardware resources were so much more
limited, which forced them to try and be efficient, or else the
performance became really bad.
Later model network cards started implementing Early Receive and Early Transmit
Interrupts, so that the card could "help" the OS understand that the buffers
were almost exhausted and needed to be processed immediately .
I can't comment on that idea much, as I don't know how it works. But I
can say that such a solution would not help one bit in RSX. The PDP-11
ethernet cards give an interrupt as soon as there is at least one
received packet to process. And the software can process all received
packets at that one interrupt, and RSX does that. Additional interrupts
cannot make this any better. If you are running out of packets, it is
because the controller is then producing packets faster than RSX is
consuming them, but RSX is consuming them as fast as it can at that point.
As far as transmit interrupts go, you only get an interrupt when
transmits are finished. If you add packets while transmit is ongoing,
your new packets will also be transmitted without any more work on the
host side. And you can add packets while transmit is in progress. There
is no point in additional transmit interrupts.
C-Kermit was able to solve a lot of the synchronous packet transfer inefficiency by
creating asynchronous "sliding window" transmission acknowledgements for both
serial and network communications, which replaced the synchronous send-ACK/NACK cycle
with asynchronous send1-send2-send-3/ACK-1/send-4/NACK-2/send-2(retransmit)/ACK-3/send-5,
etc. sliding window scheme. It was astounding how much faster the sliding window
algorithm was than the original Kermit. Note that some network protocols also have a
similar asynchronous ACK/NACK processes.
The is not a C-Kermit thing. Sliding window protocols are old, and it's
been in Kermit for a long time.
DECnet also implement a sliding window protocol, as do TCP. I would
expect that almost any "modern" networking protocol have sliding windows.
The problem that Paul Koning discovered with RSTS/E are very specific to
RSTS/E. No reason to apply that to all software and hardware from that era.
Johnny
Dave
-----Original Message-----
From: Simh [mailto:[email protected]] On Behalf Of Paul Koning
Sent: Monday, May 02, 2016 1:38 PM
To: SIMH
Subject: EXT :Re: [Simh] RSTS and slow DECnet operation in SIMH
On Apr 19, 2016, at 2:46 PM, Paul Koning <[email protected]> wrote:
With help from Mark Pizzolato, I've been looking at why RSTS (DECnet/E)
operates so slowly when it's dealing with one way transfers. This is
independent of protocol and datalink type; it shows up very clearly in NFT (any
kind of file transfer or directory listing) and also in NET (Set Host). The
symptom is that data comes across in fairly short bursts, separated by about a
second of pause.
This turns out to be an interaction between the DECnet/E queueing rules and the very fast
operation of SIMH on modern hosts. DECnet/E will queue up to some small number of NSP
segments for any given connection, set by the executor parameter "data transmit
queue max". The default value is 4 or 5, but it can be set higher, and that helps
some.
The trouble is this: if you have a one way data flow, for example NFT or FAL doing a
copy, the sending program simply fires off a sequence of send-packet operations until it
gets a "queue full" reject from the kernel. At that point it delays, but the
delay is one second since sleep operations have one second granularity. The other end
acks all that data quite promptly, but since the emulation runs so fast, the whole
transmit queue can fill up before the ack from the other end arrives, so the queue full
condition occurs, then a one second delay, then the process starts over.
This sort of thing doesn't happen on request/response exchanges; for example
the NCP command LOOP NODE runs at top speed because traffic is going both ways.
I tried fiddling with the data queue limit to see if increasing it would help. It seems
to, but it's not sufficient. What does work is a larger queue limit (32 looks good)
combined with CPU throttling to slow things down a bit. I used "set throttle
2000/1" (which produces a 1 ms delay every 2000 instructions, i.e., roughly 2 MIPS
processing speed which is at the high end of what real PDP-11s can do). Those two
changes combined make file transfer run smoothly and fast.
Ideally DECnet/E should cancel the program sleep when the queue transitions from full to
not-full, but that's not part of the existing logic (at least not unless the program
explicitly asks for "link status notifications"). I could probably add that;
the question is how large a change it is -- does it exceed what's feasible for a patch.
I may still do that, but at least for now the above should be helpful.
Followup: I created a patch that implements the "wake up when the queue goes
not-full". Or more precisely, it wakes up the process whenever an ack is received;
that covers the probem case and probably doesn't create many other wakeups since the
program is unlikely to be sleeping otherwise.
The attached patch script does the job. This is for RSTS V10.1. I will take a
look at RSTS 9.6; the patch is unlikely to apply there (offsets probably don't
match) but the concept will apply there too. I don't have other DECnet/E
versions, let alone source listings which is what's needed to create the patch.
With this patch, you can run at full emulation speed, with the default queue limit (5).
In fact, I would recommend setting that limit; if you make the queue limit significantly
larger, the patch doesn't help and things are still slow. I suspect that comes from
overrunning the queue limits at the receiving end. (Note that DECnet/E leaves the flow
control choice to the application, and most use "no" flow control, i.e., on/off
only which isn't effective if the sender can overrun the buffer pool of the receiver.)
To apply the patch, give it to ONLPAT and select the monitor SIL (just <CR>
will give you the installed one). Or you can do it with the PATCH option at boot
time, in that case enter the information manually. The manual will spell this out
some more, I expect.
I have no idea if this issue can appear on real PDP-11 systems. Possibly, if
you have a fast CPU, a fast network (Ethernet) and enough latency to make the
issue visible (more than a few milliseconds but way under a second). In any
case, it's unlikely to hurt, and it clearly helps a great deal in emulated
systems.
paul
_______________________________________________
Simh mailing list
[email protected]
http://mailman.trailing-edge.com/mailman/listinfo/simh
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: [email protected] || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
_______________________________________________
Simh mailing list
[email protected]
http://mailman.trailing-edge.com/mailman/listinfo/simh
