On Wed, Mar 10, 2010 at 11:14 AM, Jody Harris <[email protected]> wrote: > Has the Tahoe Dev ever discussed the possibility of encrypting and storing > blocks instead of files? > I don't know: > 1) if this has already been discussed, or
Yes. Tahoe-LAFS's predecessor -- a proprietary product code-named "Mountain View" which was written by allmydata.com -- and Mountain View's predecessors -- Mnet and Mojo Nation -- did this. > If the encrypt/store algorithms encrypt in (for example) 128 KB blocks, > then > - you would never hit the large memory overhead problems seen with 100 MB > and larger files. > - processing and storage would just take the file one chunk at a time > - retrieval and decryption (seems) that it would stream nicely since > allocation of the next block(s) could take place while the current block is > streaming out. This is all currently the case for immutable files. Our plan is to make it be the case for mutable files as well. (This is one of the reasons why we recommend immutable files instead of mutable files for most uses. Another is that immutability is a nice property that can make it easier to keep your stuff organized.) Note that the question of processing and transferring segments of the file is somewhat independent of the question of storing segments of the file on different servers. We want to continue storing shares of a file all on one server in order to correlate failures (there is an interesting technical reason why it is safer to do it this way than to spread out blocks from one share to multiple servers) and in order to reduce the number of different servers that the client has to talk to and the number of different objects that the clients and servers have to keep track of. Regards, Zooko _______________________________________________ tahoe-dev mailing list [email protected] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
