Doug Cutting wrote:
Ian C. Blenke wrote:
When NDFS is exposed to userspace for scripts to use, admins types
will embrace it for managing the cluster.
Our intent is to add some servlets which run on each datanode
providing access to the filesystem for non-Java programs.
Most operations would be quite simple, e.g.:
- to write a file, post its content to a url like:
http://datanode:XXXX/write?name=my.file
- to read a file, get file content from urls like:
http://datanode:XXXX/read?name=my.file
http://datanode:XXXX/read?name=my.file&start=2048&length=1024
- to remove a file:
http://datanode:XXX/remove?name=my.file
Similarly for rename, copy, etc.
Not very RESTful, but simple.
The only somewhat complicated thing would be directory listings.
These would be handled with a simple REST interface, where some simple
XML is returned. Ideally a stylesheet could be specified so that one
can use the directory listing url to view the filesystem from a brower.
From a bash scripting standpoint, this would be complicated to access
without a userspace command to wrap it.
A RESTish interface works well for perl/python/ruby, though I think they
would much rather have a native object wrapper (SWIG something together).
These servlets could easily be implemented in terms of the
NutchFileSystem API, and deployed with Jetty. To my knowledge, no one
is currently working on this. A volunteer would be welcome.
If portability is a key goal, FUSE or FiST probably aren't the ideal (no
Windows or OS/X ports, for example).
A simple WebDAV interface seems like the closest thing to a standard
that you are attempting to approximate with the RESTful interface. The
added benefit would be support from DavFS2, Finder, Microsoft
Webfolders, etc.
Perhaps something that plugs into Jakarta Slide? A NDFS backend to Slide
would potentially benefit a distributed CMS as well (without a
versioning history, as that appears to be beyond the scope of NDFS).
I would be interested in implementing something like this if there is
indeed interest.
- Ian C. Blenke <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> http://ian.blenke.com/
