> We've actually experimented a bit with this, since most of the nodes in > our system are disk-less. In fact, they're -less just about everything > except for processors, memory, and a connection to our internal fabric. > We've tried loading "canned" MDT/OST images into memory on some nodes > and serving from there, and it does seem to work. There are two > downsides, though. One is that the Linux loopback driver is a real > performance bottleneck, ever since some bright person had the idea to > make it less multi-threaded than it had been. Another is that booting > tends to involve metadata-heavy access patterns which are not exactly > Lustre's strength - a situation made worse when you have nearly a > thousand clients doing it at the same time and your MDS is a relatively > small node like the others. So far we've found that NBD serves us > better in the boot/root filesystem role, though that means a read-only > root which involves its own complexity. Your mileage will almost > certainly vary.
Does the metadata updates problem go away in a read-only root environment? That makes life a lot easier. I don't want to have some random node that's either hacked or out to lunch be able to make changes to the filesystem. When I used NFS root, we had one node that was allowed to write, and all the rest were read-only mounts. I find AFS much more convenient since I can log in as root, and authenticate to the filesystem from any node to make changes. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
