Hi Chuck, > I am in the process of ordering 5 3T 1U machines. I was originally > going to try to use gfs in this cluster to create a large contiguous > filesystem for my users. The gfs irc gents told me that their product > was not designed for this, at least not at my scale. I need it to scale > out to 120T possibly. I'm thinking of using pvfs2 as my filesystem.
Remember that pvfs2 does not have a native on-disk format and relies on other on-disk file systems. Therefore your scaling will be limited by the choice of on-disk FS that you choose on each of the 3T nodes. ext3's max volume size ranges from 2TB to 32 TB (block sizes 1KB to 8KB) Remember that block sizes are directly related to the max. architectural limit of PAGE size and the higher block sizes don't work on IA32. XFS scales upto 8 EB and hence won't have any problems. ext4 will also probably scale to this capacity *if* it stabilizes. > looked at gfs, pvfs2, and lustre, but I like pvfs2 the most because I > can actually compile and run it pretty simply. Unfortunately, there > isn't really system level fallbacks, so if one of my nodes goes, so too > does my fs, and of course I cant store a db on it, which might be nice. > What do people out there suggest? Is there something else better for my > purposes? I'm not storing homes on this fs, though that might be nice, > mostly I'll be storing images and text files. Thanks for any comments! It seems like PVFS2 should do just fine for storing images and text files. Keep in mind though that there are no redundancy options at the PVFS layers and if any of the disks go bad/get corrupted there is a good chance some/all of the FS will be left inaccessible. Node failures is not that big a problem. It is disk failures that will leave you in a bad state. Cannot comment on gfs or lustre since I havent' used them. Hope this helps, thanks, Murali > -Chuck > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
