Jason Dagit wrote:
> Now, it's also possible to make the lazy and eager versions use less
memory by storing the hunks in files and just reloading them whenever
we need to look at them or show them to the user. In fact, I think I
know what we should/could do to help a lot of the memory pressure. I
think we should make it so that patch sequences are a zipper-ish data
structure that is stored on the disk. For example, each patch would
be stored as a separate file on the disk in some special directory.
You could then request the next or previous patch in that sequence and
in the background there would be a read of that file. The contract
for the caller would be a promise to not hold on to the patches longer
than necessary. We would be trading lots of "on demand" disk reads
for in process memory usage. One could imagine optimizing for small
patches by grouping them into files of about X mb. It's entirely
possible that we could implement this file backed sequence as a single
file.
Thoughts? Anyone know if Petr's work can be applied here? Does camp
have an optimization for this?
It sounds a lot like Petr's thoughts on chunky hunks to me... The idea
that once a chunk gets above a certain threshold, the chunk's data can
easily be dumped into hashed-storage as a separate unit and the patch
itself then merely contains a hash reference to that chunk data.
In this particular case it simply becomes a matter of chunking that data
into hashed-storage as soon as possible during record to avoid huge
memory usage (keeping only the hash references in memory). (The only
real complication just being that there are good garbage collection
policies in place should the record be canceled.)
--
--Max Battcher--
http://worldmaker.net
_______________________________________________
darcs-users mailing list
[email protected]
http://lists.osuosl.org/mailman/listinfo/darcs-users
