On 01/02/2014 04:16 AM, David Timothy Strauss wrote: > We maintain a FOSS file system [1] for users of our platform. One of > the challenges, as we move move functionality into the client, is > dealing with our copy-on-write volume-cloning. To get implicit > copy-on-write, we reference SHA512 hashes of file content. Same > content, same copy. > > For a next-generation design, the DHT support in GNUnet would fit our > use-case well.
I'm wondering why you're looking at the DHT, and not at FS for this. Have you seen gnunet-fuse? (We used to have write-support in it, and it would be interesting to see someone revive it.) > But, I'm concerned about guaranteeing that a file is > available within the sort of F2F networks we'd be setting up. Is there > any way to guarantee that, aside from republishing constantly? If your nodes are trustworthy and are not experiencing churn, only the expiration time limits how long data stays around. So your use-case (and specifically, environment) largely determines how often you have to republish to ensure availability. > Is > there interest in support for secondary backends, like S3 (with or > without encryption of the content), in order to provide arbitrarily > large storage? The DHT uses a subsystem "DATACACHE" for storage, and FS uses "DATASTORE". Both have a pluggable backend (sqlite, mysql, postgres) and it should be simple to add support for other databases. You can find the API(s) that you'd have to implement here: https://gnunet.org/svn/gnunet/src/include/gnunet_datastore_plugin.h https://gnunet.org/svn/gnunet/src/include/gnunet_datacache_plugin.h So the easiest way to do this might be to (1) add an S3 plugin to datastore/datacache, and then (2) extend gnunet-fuse with CoW support. A hard design question for us was always how to nicely return the URI (or file-system mount point) to the user that resulted from the copy. If you have a good answer for how this should be done (essentially, how do you _nicely_ tell the user the top-level identifier after umount), that'd be interesting. Happy hacking! Christian _______________________________________________ GNUnet-developers mailing list [email protected] https://lists.gnu.org/mailman/listinfo/gnunet-developers
