Hi James, Glad you're giving PVFS2 a try. See answers below.
On Fri, May 21, 2010 at 02:02:34PM -0700, James Gao wrote: > Hi everyone, > > I'm planning to set up a PVFS2 system for my lab. I have a few questions > about its scalability. > We currently have a 10 tb RAID box, completely full. We just acquired an > additional 24 tb RAID box, and enough hard drives to upgrade the old RAID. > My plan was to set up pvfs2 on the 24tb RAID, copy all the data off the > current RAID, upgrade that box, and add it to the pvfs2 cluster. The > original plan was to use Lustre, but since I can't get it to compile on our > systems, I'm trying out pvfs2. > > So, the questions: > 1) As I understand it, pvfs2 will stripe the data across all available > object stores. Will this improve overall bandwidth? It will improve overall disk bandwidth two the sum of the two RAID boxes. However, the interconnect between client nodes can also be a bottleneck so the throughput may become network bound. It just depends on your setup. > 2) What happens when we add the new RAID? Will we only be able to use 24-10 > = 14 * 2 tb of space? You should be able to use all 24TB. I would recommend, when starting out with the single 24TB system, to only use 50%-75% of the handle ranges for the first system. Then, when you add the second system (which will be an offline activity) use the other handle range(s) that are not currently assigned to the 24TB system. I don't have an exact formula. Maybe others with experience in this type of activity could give some guidance on how they carved up the handle space. A guess would be since the 24TB system will have more storage available I would allocate more (2 times as many) data handles. Metadata handles maybe just 50/50. > 3) If we fill up our 48 tb RAID system and we would like to add another, how > would that work? In Lustre, if an OST fills up, you can deactivate writes to > it, so the rest of the space is still usable. Will pvfs2 only throw out a > "no free space" error if we add additional storage in the future? Adding a new system (or several) currently requires pre-planning in my understanding. You will want to keep handle space un-allocated that new systems can be given as they are added. I'm not familar with the OST deactivation process. If no space is available on the PVFS backing filesystems it will throw an error. The recourse then is to add an additional system using un-allocated handles or grow the backing filesystem. Others may have more experience with planning or migration between systems. Thanks, Michael > > Thanks in advance! > > -James Gao > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
