Steve, For testing, have a look at FileSystemContractBaseTest and the FileContext*BaseTest classes. You can subclass these to get a lot of basic tests for free. (These don't cover the kind of stress tests you mentioned though.)
Cheers Tom On Wed, Mar 24, 2010 at 3:28 AM, Steve Loughran <ste...@apache.org> wrote: > > I'm looking at what it currently takes to implement new back end filestores, > getting lost in the details. > > This is my current understanding -am I wrong? > > 1. There is an AbstractFileSystem, came in with HADOOP-6223, and is in > SVN_HEAD only > https://issues.apache.org/jira/browse/HADOOP-6223 > > 2. There is FileSystem, which exists and includes methods like append() > > 3. There is not currently any generic stress test for a filesystem other > than terasort, but that doesn't really push the limit on FS operations, > concurrency issues, appends, etc. > > 4. There is not currently much in the way of subclasses of > AbstractFileSystem other than a bridge to the classic FileSystem > > 5. Because the AbstractFileSystem hasn't yet shipped, it offers the > opportunity to add things like progress callbacks (HADOOP-6324). > > 6. Because the AbstractFileSystem hasn't yet shipped, if you want people on > 0.20.x or 0.21.x to use your filesystem, you need to implement FileSystem. > > I'm curious as I need to know where to point some people. Short term, > FileSystem looks like the one to target, and because AFS wraps it, you could > stay with it for a while before worrying about AFS support altogether... > > -steve >
