Hi David,
We have discussed with the group at PNNL the idea of integrating some of
their active storage work into PVFS in the future, but that work hasn't
started quite yet.
The idea of using FUSE as a way to hook into the I/O path is pretty
interesting and might be a good way to prototype.
pvfs2-viewdist can be used to look at how files are distributed. Data
for files is stored in a single UNIX file on each server in a
subdirectory under the storage space directory. Generally you can choose
how many servers a file is spread across and what strip size is used
through configuration parameters and extended attributes.
Regards,
Rob
David Brown wrote:
I know there's been lots of research into investigating the
possibilities of putting some form of processing on archives with
massive amounts of disks (active-storage). I was wondering if anyone
has approached the pvfs developers with ideas of integrating an
active-storage like system to the servers to allow them to perform
preprocessing on the data before it gets sent over the network?
One could implement something very quick using fuse and have the
servers work on top of the fuse file system. However, that would
depend on how the raw data of the files are stored on each of the
storage servers. Could someone give me a place to start looking as to
where the raw data of the files are located? algorithms used for
distributing files among the storage servers? block sizes available?
Any information would be helpful.
Thanks,
- David Brown
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
