Thanks for the answer! I didn't realize you could create the filesystem that way -- I was thinking of it too much like a normal block device. Will the future distribution have a smooth migration route? Our network is just a standard gigabit ethernet setup. There are four GbE connections to the two RAIDs. Ideally, we'd like to have four separate computers read data from the RAIDs at full speed.
One final question, is there any good documentation about how the handle allocation works? I've read briefly about them, but I don't understand how they work. If we do want to reserve space for additional servers, I'd like to learn how the handle allocation works before I start setting it up. Thanks again! -James On Sat, May 22, 2010 at 12:27 PM, Walter Ligon <[email protected]> wrote: > > James, funny you should ask. We are working on migration tools for PVFS > right now. > > PVFS isn't particularly good at adding a new server - that's what we're > working on. There is an experimental distribution in the source code that > automatically adjusts the amount of data written to each server in order to > balance the load. I've never used it. > > For your situation I would suggest the following: > > 1) Copy all your data to the new RAID using the standard FS. > 2) Upgrade the old RAID. > 3) Create the PVFS system on both RAIDS - you can do that even with the > data on the drives, all that happens is you create a dir to hold the PVFS > data. > 4) Move your files from the old FS to the PVFS system, file by file will be > copied into PVFS and removed from the old system. > > Then your data will be balanced. Hopefully you won't want to add a 3rd > system until we get the new migration tools done. They would help you > rebalance after adding more storage. > > What Michael said about the network being a bottleneck an certainly be an > issue, so you should consider that carefully. > > Walt > > Becky Ligon wrote: > >> >> ------------------------------------------------------------------------ >> >> Hi everyone, >> >> I'm planning to set up a PVFS2 system for my lab. I have a few questions >> about its scalability. >> We currently have a 10 tb RAID box, completely full. We just acquired an >> additional 24 tb RAID box, and enough hard drives to upgrade the old RAID. >> My plan was to set up pvfs2 on the 24tb RAID, copy all the data off the >> current RAID, upgrade that box, and add it to the pvfs2 cluster. The >> original plan was to use Lustre, but since I can't get it to compile on our >> systems, I'm trying out pvfs2. >> >> So, the questions: >> 1) As I understand it, pvfs2 will stripe the data across all available >> object stores. Will this improve overall bandwidth? >> 2) What happens when we add the new RAID? Will we only be able to use >> 24-10 = 14 * 2 tb of space? >> 3) If we fill up our 48 tb RAID system and we would like to add another, >> how would that work? In Lustre, if an OST fills up, you can deactivate >> writes to it, so the rest of the space is still usable. Will pvfs2 only >> throw out a "no free space" error if we add additional storage in the >> future? >> >> Thanks in advance! >> >> -James Gao >> >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
