David-Sarah Hopwood wrote: > Kevan Carstensen wrote: >> At 2010-03-10 23:27 (+0000), David-Sarah Hopwood wrote: >> >>> That it isn't true for mutable files is, as far as I know, just a >>> temporary limitation of the current mutable download/upload code >>> (which does work in terms of segments, but hasn't been optimized to >>> reduce its memory usage by doing so). >> This is true in the sense that the entire mutable file is treated as >> one segment, AFAIK. Is that what you meant? > > In the sense that the storage format and the 'mutable retrieve' code > still works in terms of arbitrary-length segments (as far as I can > tell), but the 'mutable publish' code happens to choose the segment > size so that there is only one segment.
I tried to make that true when I wrote the mutable code, but I'm not sure I pulled it off. There's a hash tree that's prepared to handle multiple blocks (just like CHK), which would let you use normal-sized 128KiB segments (just like CHK), but since the download-size Retrieve class is hard-wired to expect a single segment, we couldn't change that without breaking existing clients. So even though the protocol nominally has room for this improvement, in practice we'll probably have to have some sort of version bump. Also, if we want to be able to update just a portion of the file (without rewriting the whole thing), then the protocol will need to be extended to have a unique salt per segment. (this would of course reveal which segments you're modifying to the server and everybody else). That would certainly require a big version bump. So it may be the case that SDMF is O(filesize) RAM, and we need to define+build+deploy MDMF to fix that. cheers, -Brian _______________________________________________ tahoe-dev mailing list [email protected] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
