Hi Abhinav,
The PVFS file system will pretty much be full if one of the 60 GB drives
fills up. You can continue to read from it, of course, but you won't be
able to write more data.
Technically it is possible to specify an explicit layout for each file
(ie, dictate exactly which servers to stripe across). In theory that
would let you continue to use the servers with more capacity, but it
would be really awkward and the interface isn't available to most
applications.
-Phil
Abhinav Chawade wrote:
Thank you Phil and Rob.
I could see the file size properly with pvfs2-statfs and karma gui. Phil
you mentioned in detail how pvfs2 calculates free space. Now if one of
the drives becomes full(say one of the 60 GB drives) and there is space
on other servers, will pvfs2 distribute data among them evenly or the
pvfs2 volume will beunusable?
On Fri, Feb 6, 2009 at 11:16 AM, Phil Carns <[email protected]
<mailto:[email protected]>> wrote:
Hi Abhinav,
To summarize, in total it sounds like you have PVFS servers running
on 14 nodes. Most of those nodes only have 60 GB hard drives, but
one of them has 640 GB of storage.
PVFS doesn't have any mechanism to account for unbalanced storage on
each of the servers. If any one of your servers runs out of space
then the PVFS volume is pretty much out of space as well. So when
it calculates the total space on the file system (for df) it uses
this algorithm:
<capacity of smallest server> * <number of servers>
... which gets you to the 850 GB value that df is reporting.
As far as the 52% being allocated already, this probably means that
on at least some of your servers, the server's storage directory is
on the same partition as your root file system (or some other data).
That means that things external to PVFS are counting against the
capacity on some servers. There is nothing wrong with that
necessarily. Most likely one or more of your machines has used up
half of its 60 GB already and PVFS therefore only has maybe 30 GB *
14 available out of the 60 GB * 14 total.
If you want more detail about how much space is available on each
individual server, you can use the pvfs2-statfs command.
There isn't any way to set a limit on each server from the PVFS
perspective. You could do it at the node level by using a separate
partition for the PVFS data, though.
-Phil
Abhinav Chawade wrote:
I have a 14 node cluster with 1 storage server 1 head node and
12 computational node running centos 5.2
I deployed pvfs2 on all of them and head and storage node are
metadata servers. Each computational node has 60GB hard drive
and the storage server has 640 GB storage.
When i mount the file system and ping it, all servers are
running. I try to see how much free space is left using df
command. df output shows all in all 850 GB storage out of which
52% is allocated already. There isn't a single file in pvfs2
mounted partition. How can i check size of pvfs2 space i
created? Is there a way to set limit on storage on each server?
Abhinav
------------------------------------------------------------------------
_______________________________________________
Pvfs2-users mailing list
[email protected]
<mailto:[email protected]>
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
