On Jan 14, 2006, at 12:22 PM, Jason Dagit wrote:
On Jan 14, 2006, at 6:13 AM, Ian Lynagh wrote:
On Sat, Jan 14, 2006 at 04:24:16AM -0800, Jason Dagit wrote:
I was looking a issue80 and I discovered that Repository.writePatch
reads the freshly written patch. This is a very expensive operation
given the current patch format. So I started to wonder why it does
that.
To avoid the data all being kept in memory.
As an experiment I removed the reading the patch and just let
Repository.writePatch return the same patch it was given. On a test
case of recording a 37meg file I went from 39 seconds for the record
to just 3.9 seconds. Total memory allocated during the run when
from
3.5 gigs to 444 megs. This is a very significant savings.
The peak memory usage is far more important.
"darcs record -a" ought to be a constant space operation.
I'll go double check this but I've seen no evidence of this with
the current implementation. For me it would appear that darcs uses
a ton of memory on large files, and issue80 speaks directly to this
as a problem.
My double check just completed. The setup was to record a 500mb text
file in both current darcs-unstable versus the same darcs with my
patch applied, called orig and no-reread respectively.
Total time (wall clock)
orig: 5 hours +
no-reread: 6 minutes
Peak RES (as measured by top)
orig: 940MB
no-reread: 950MB
Peak VIRT (as measured by top)
orig: 1756MB
no-reread: 1289MB
When I realized that orig had consumed more time and space than no-
reread I killed it so I'm not sure where it would have peaked or how
much time it would have taken to reach that peak.
I think the numbers speak for themselves.
Thanks,
Jason
_______________________________________________
darcs-devel mailing list
[email protected]
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel
