Erich,
and suddenly server3 literally explodes, like, there's no way I can get anything on the disk back, is my entire file system screwed? Or just some files?
You will lose, - Any file whose stripe is stored on that node is screwed. - You won't lose the entire file system unless the node that crashed was the MD server and you configured the system with exactly 1 MD server. - Access to any files whose metadata was stored on the node that crashed. You can still access parts of the file provided the server that went down did not store the metadata required to access the file.
For example, if I have a 4MB file, is that single file spread out over the 4 nodes or is it striped in a linear fashion all on nodes, i.e. does node 1 fill up first then go to node 2, etc.?
It is striped in a linear fashion across all the nodes in a round-robin fashion (by default). There are other distributions for a given file. Spose file stripe is 4 MB, you will have 0-4MB on node 0, 4-8 MB on node1, and so on. The starting node number is randomized on file creation so that we don't fill up node 0's disk space unfairly and cause needless hotspots.
I'm assuming it's not exactly set up as a RAID5 type redundant system, such that if I lose a node and then replace it, it will "rebuild" the file system data on that node?
Nope; there is no RAID-5. It is a classic RAID-0 style distribution.
I need to be clear on this before putting a lot of time into it, but it sounds like this might be a good solution for our firm, as we have a 200 node cluster each with one 500GB disk, 400GB of which can be leveraged to a massive parallel file system (400GB x 200 nodes = one big ~80TB distributed file system). But that assumes that there is no redundancy, other wise that 80TB would be more like 50-60TB max or something because there would be some redundancy in there... ?
There is no in-built redundancy yet. Unless you setup something like drbd. Hope you find this information useful. Do keep us posted. thanks, Murali
Thanks for any clarification! -Erich _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
