We at Quantcast have released QFS 1.0 (Quantcast File System) to open source. This is based on the KFS 0.5 (Kosmos Distributed File System), a C++ distributed filesystem implementation. KFS plugs into Apache Hadoop via the 'kfs' shim that is part of Hadoop codebase.
QFS has added support for permissions, and also, provides fault tolerance through Reed-Solomon encoding as well as replication. There are also a number of performance and stability improvements, including a rewrite of the client library to allow parallel concurrent I/Os. Going forward, new releases of KFS will come from QFS. The open source release of QFS is at http://quantcast.github.com/qfs QFS plugs into Apache Hadoop the same way KFS does. Currently, one would apply the patches or JARs from the QFS source tree onto Apache Hadoop to make Hadoop use QFS. The patch for Apache Hadoop 1.0.X can be found at https://github.com/quantcast/qfs/blob/master/hadoop/hadoop-1.0.X.patch In order to make the integration seamless, we would like to add a 'qfs' shim to Apache Hadoop so that the current active branches (1.0.X, 2.X.X, 0.23.X) of Apache Hadoop can use QFS. Towards this, I've submited an ASF JIRA feature ticket (HADOOP-8885) under hadoop-common project, and send a pull request with the QFS shim changes to https://github.com/apache/hadoop-common/tree/branch-1.0.2 I will subsequently submit pull requests to the other active Hadoop branches. If you have any question, I will be happy yo answer or provide more details on QFS. - Thilee
