>bc: From: Bill Cattey <[EMAIL PROTECTED]>
>bc:
>bc: > > Actually, NFS scales reasonably well.
>bc: > > With 10,000 accounts, we put 1000 accounts on each NFS server,
>bc: > > probalistically, only 10% of those are reading or writing a file at a
>bc: > time, so the system does just fine.
>jr: [EMAIL PROTECTED] (492)
>jr:
>jr: Where did you put the volumes, like /usr, that every client uses all the time?
>bc: We put them on remote virtual disk, a protocol optimized for giving MANY
>bc: > clients read-only access to disk blocks.
Sounds like some sort of cache manager...
It also sounds like "10%" is an awfully optimistic view of the load
on the server. Presumably, what people were accessing was their personal
filespace? The "Compuserve" factor for traditional timesharing
usage is that at any particular time, 3% of the accounts will be
logged in, and presumably, at that, much of the time is spent
by the user reading the screen, and not waiting for disk requests.
Hmm. Using your 'notable quote'. 6000+ logins per day, across 10 servers,
making, say, 600 users accessing their home directory on an NFS file server
per day - 100 of those 600 users read/writing files "at the same time"
suggests some very interesting synchronicities. Odd applications, or
user habits, or something. Do users only compute at lunch time or something?
Anyways, depending on which factor you use, that means between 30 & 100
"active" users (whatever that means) per NFS disk server. Old time wisdom
had it that the right NFS workstation/server ratio was about 30, which
is (more or less) consistent with this. This is not very impressive
compared to AFS.
>bc:
>bc: > We migrated off NFS over the Summer of 1992, not because NFS didn't
>bc: > scale, but because managing those 1000 filesystems on each server was a
>bc: > nightmare.
>
>jr: > In other words, NFS doesn't scale.
>bc:
>bc: In a different sense of scale, yes. In my original sense, the
>bc: technology would not pump the bits across for useful performance. In
>bc: your sense, the human effort required to migrate filesystems around as
>bc: different users deviated from the probabalistic 'normal curve' was
>bc: prohibitive.
>bc:
Now, if *that* was the only obstacle, that's not so bad. It
really suggests some sort of automated nightly process run
that would look for heavily used servers and move volumes around
to relatively lazy servers, "sort of like the one for AFS".
It't be a bit harder for NFS, primarily to deal with issues
like what if the user is using it at the same time, but
certainly not impossible. I suppose you could just do it
sunday morning or something, perhaps in conjunction with backups.
>bc:
>bc: What does NOT scale, is the intermediate server technology, like the
>bc: NFS-to-AFS translator. No good technology for load balancing is
>bc: incorporated there. The semantic differences of AFS and NFS filesystems
>bc: cause havoc. And, as you pointed out, the cache managers could be
>bc: browsed.
Actually, I said nothing about browsing the cache. Intermediate
servers need some attention to security - keeping them in locked
machine rooms & not letting people log into them is only sensible.
But you probably already do this with your routers and your file servers.
>bc:
>bc: For intermediate servers to scale, they must balance load, be robust
>bc: across crashes so that required state information can be recovered, and
>bc: they must be secure. I am reasonably sure that the AFS-to-NFS server
>bc: meets none of these criteria at the present time. Now I am coming to
>bc: understand that PC Venus does not either. Does anyone know of
>bc: intermediate servers with AFS on one side, and PC's and Macs on the
>bc: other that DO?
If they don't crash often, keeping state is not so important.
NFS seems to have satisfied your needs for years, and NFS is
notoriously only "almost" state free. Ditto for security -- if
NFS met your security needs, then no need to knock intermediate
servers for any lack thereof. The major limitations on intermediate
server security is due to limitations in the basic protocols (be they
AFP, NFS, or SNAP.)
Load balancing is a bit harder -- again, that's due to limitations in
the client software. With PC Venus, actually, you *do* have a fighting
chance to put load balancing in; if you could modify the login process
to select the "best" intermediary. Actually, once most DNS name servers
reach 4.9 levels, this becomes a bit easier -- just list multiple `A'
records for the intermediate servers, and the name server will return
a random address to clients. Voila' - instant "load balancing"--albeit
of a most decidedly primitive sort. Since NFS isn't connection
oriented, I don't think the same trick will work. With AFP,
the obstacle becomes NBP, although, actually, there are some
interesting theoretical tricks there that one could play
that might accomplish the trick.
In one sense, you are absolutely right, intermediate servers "don't scale".
That's inherent in the protocol. NFS generates enough packet flow
that about 30 workstations per server is not unreasonable - whether
that server has to talk to AFS or can fetch it off the disk is incidental
to the basic protocol problem. If you've got a thousand PC's,
that might mean as many as 30 intermediate translators. (Or,
if your 100 user limit is right, only 10. Or, if PC users are less
network disk intensive (and they probably are!), perhaps only 3-4!)
But keeping those intermediate translators running is probably
not going to be nearly the pain that keeping 10 NFS file servers
was, and keeping the AFS file servers behind that was clearly
already a win for you guys. So, basically, what's the problem?
As I said, the real obstacle is the client software, and behind
that, the client hardware. Give me source to AppleShare, guarantee
me all Mac's will have 8+ Meg's of RAM, a 68040 or better, and
300+ Meg's of disk, and I'll see you have an AFS cache manager that
will knock your socks off. As long as clients don't have caches,
and talk various proprietary protocols, intermediate servers have
a definite use.
>mw: > It seems to me that NFS has a more basic security flaw; in most (all?)
>mw: > implementations, authentication only happens at login time. That means
>mw: > it should be possible for the bad guy, especially one that can sniff on
>mw: > the network, to high-jack an existing connection. That's difficult to
>mw: > fix without source to both the client & server sides, but one might at
>mw: > least ask that the initial authentication exchange not send passwords in
>mw: > the clear.
>
>bc:
>bc: The MIT uid mapping technology allows for expiration of mappings, and
>bc: explicit destruction of the mappings. Alas, I'm not knowledgeable
>bc: enough about security issues to speak intelligently about the
>bc: highjacking issue. For all I know, you may be completely right.
I seem to remember hearing that Peter Honeyman once gave his
students the assignment of writing an NFS file browser. As
I recall, nobody failed that class. It's not hard to do research
on this, NFS is well documented in the internet RFC's, & there are
plenty of implementations, many with source, to look at.
-Marcus Watts
UM ITD RS Umich Systems Group
