On Wed, Mar 24, 2010 at 3:28 AM, Steve Loughran <ste...@apache.org> wrote:
> > I'm looking at what it currently takes to implement new back end > filestores, getting lost in the details. > > This is my current understanding -am I wrong? > > 1. There is an AbstractFileSystem, came in with HADOOP-6223, and is in > SVN_HEAD only > https://issues.apache.org/jira/browse/HADOOP-6223 > > And also FileContext. AbstractFileSystem and FileContext are a refactoring of FileSystem to separate the interfaces for file system developers (AFS) and users of file systems (FC). 2. There is FileSystem, which exists and includes methods like append() > > 3. There is not currently any generic stress test for a filesystem other > than terasort, but that doesn't really push the limit on FS operations, > concurrency issues, appends, etc. > Not that I'm aware of. > > 4. There is not currently much in the way of subclasses of > AbstractFileSystem other than a bridge to the classic FileSystem > > The biggest is Hdfs (the replacement for DistributedFileSystem). > 5. Because the AbstractFileSystem hasn't yet shipped, it offers the > opportunity to add things like progress callbacks (HADOOP-6324). > > That's my understanding. Adding new APIs to AFS is not out of the question. 6. Because the AbstractFileSystem hasn't yet shipped, if you want people on > 0.20.x or 0.21.x to use your filesystem, you need to implement FileSystem. > Correct, at least for 20. It's not clear if 21 will be released as is, or skipped, or rebased on a future version of trunk (which will have AFS and FC). Thanks, Eli I'm curious as I need to know where to point some people. Short term, > FileSystem looks like the one to target, and because AFS wraps it, you could > stay with it for a while before worrying about AFS support altogether... > > -steve >
