Here's how it works. You do the following to get a list of all the files to rebalance:
du -a PVFS2_FS | sort -n | sed 's/^[0-9][0-9]*\t\t*//' > SOME_TEMP_FILEWhere PVFS2_FS is the mounted PVFS2 volume and SOME_TEMP_FILE is just that.
The theory is that for fairly evenly mix of small and large files on the system, if you rebalance the smaller files first it will free up space that can be used to rebalance the large files. This only matters if you're not adding space that is equal in size (or actually slightly larger) than your largest file. This is a kind of an inverse Brazil nut effect. What this should do is as all these files are rebalanced across all the nodes, available space should start increasing on the original 4 nodes and therefor on the volume as a whole. I've attached the script for comment.
In actuality, my volume filled up while rebalancing and curiously the new nodes now have as much space taken up on the their PVFS2 disks as was available on each of the original nodes prior to the rebalancing. That is, each new node's disk has about 27-28G used. I'm not sure how this is possible. I've checked that I didn't actually create or duplicate each rebalanced file by comparing the filenames before rebalancing and after the attempt and they're the same. So there are no new files on the volume. I've verified that the rebalanced files were distributed across only the original 4 nodes before rebalancing and over all 8 nodes after using pvfs2-viewdist. So I'm completely baffled how rebalancing actually used up more space seeing as no new files were actually added. Anyone have any ideas? Anyone see anything the matter with my rebalancing script? Thanks!
pvfs2-rebalance
Description: Binary data
_______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
