George Banfill wrote:
However in some places in our repository we have people committing large files
into source safe. This has caused me some problems when converting them into
the subversion dumpfile. After a bit of debugging through the Perl code I
traced the problem to Dumpfile.pm line 748 where it tries to write out the
contents in one go. This is obviously going to cause a problem if you have a
350Mb binary file to be written in one go.
So I’ve produced a patch that works for me. However it has a problem that I’d
appreciate some help sorting out:
* I didn’t know what would be a good size for the block to write out in one
go. I chose 100000 characters. What’s the underlying buffers size? Setting that
as the size of the text to output seems sensible to me.
Thanks George, this issue is one of those long-standing tickets I was
just referring to!
<http://www.pumacode.org/projects/vss2svn/ticket/25>
My proposed solution was to use File::Copy[1] instead to do a buffered
write of the file contents directly to the dumpfile when needed. So your
patch solves half the problem, although the script is still reading the
entire file contents into memory at once. However solving the problem
fully would require some more re-architecting, because each dumpfile
"node" is passed around quite a bit before finally being written.
A possible solution would be to delay the get_export_contents() call
(which currently reads in the VSS file of a particular path and version)
and instead create some specially-blessed node which is just a pointer
to the file and version. Then, output_content() (the method you
modified) would notice that pointer and call get_export_contents() at
that point, using File::Copy to do a buffered read directly to the
dumpfile's filehandle.
For the record, File::Copy defaults to a 2MB buffer, but I don't know if
that's because of some knowledge of how Perl works, or just an arbitrary
number.
I'll attach your patch to the ticket, in case I don't get around to
implementing the full solution as I mentioned (of course if you are so
inclined you are welcome to do that as well!)
toby
[1]<http://search.cpan.org/~nwclark/perl-5.8.8/lib/File/Copy.pm>
_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user
