Toby Johnson wrote:
> Flavio Stanchina wrote:
>> The first patch adds option --incremental to vss2svn.pl that makes it
>> create one dump file per revision. [...]
> Hello Flavio, thanks for the patches; submitting them via the mailing
First, I apologize for sending the patch a bit too early: it would forget
about existing paths because it destroyed the $dumpfile object at every
iteration, so later revisions would do adds rather than changes. I've
attached a fixed patch that works as expected: unfortunately, it is a bit
more invasive as I needed a new Dumpfile->switch_fh() method to change the
output file on the run.
> list is fine. I'm not sure what the goal is of creating one dumpfile per
> revision? Are you manipulating those files somehow afterwards? Or is it
> just an issue that the files were too big to transfer?
I had to manipulate the files after the conversion. Apart from filtering
away old stuff, I had to fix a few orphans and other issues caused mostly
by less-than-optimal management of the VSS database in the past. Having one
file per revision was just easier to handle, especially with well over 1GB
of stuff.
(for the record, vim could handle the original 1.6GB dumpfile just fine on
a Linux box with 768MB of RAM, but it was taking a few minutes to load and
save it and many seconds to search through it)
> The patch to write the filename seems reasonable, but I'm not sure what
> you think should be done to handle upper/lowercase names on Linux. We
> certainly can't try 2^8 filename combinations for each file. I've never
> tried the conversion on Linux myself; doesn't it always look for an
> all-uppercase name? In that case we should probably instruct people to
> always make sure their filenames are all-uppercase.
I had to make them all lowercase. To be exact, vss2svn couldn't find the
"names.dat" file: I noticed it was named in all-uppercase on disk so I
changed everything to lowercase and it worked.
The VSS DB was more than 10 years old and I think it was originally created
on a NetWare volume, so file names might not have been in the same case as
a fresh DB created on Windows.
I suppose we should suggest to run a
find -print0 | xargs -0 rename "y/A-Z/a-z/"
on the copy of the VSS database before running the conversion.
--
Flavio Stanchina
Informatica e Servizi
Trento - Italy
Index: script/Vss2Svn/Dumpfile.pm
===================================================================
--- script/Vss2Svn/Dumpfile.pm (revision 301)
+++ script/Vss2Svn/Dumpfile.pm (working copy)
@@ -74,6 +74,26 @@
} # End finish
###############################################################################
+# switch_fh
+###############################################################################
+sub switch_fh {
+ my($self, $fh) = @_;
+
+ # switch
+ $self->{fh} = $fh;
+
+ # prevent perl from doing line-ending conversions
+ binmode($fh);
+
+ my $old = select($fh);
+ $| = 1;
+ select($old);
+
+ print $fh "SVN-fs-dump-format-version: 2\n\n";
+
+} # End switch_fh
+
+###############################################################################
# begin_revision
###############################################################################
sub begin_revision {
Index: script/vss2svn.pl
===================================================================
--- script/vss2svn.pl (revision 301)
+++ script/vss2svn.pl (working copy)
@@ -813,9 +813,12 @@
my $fh;
my $file = $gCfg{dumpfile};
- open $fh, ">$file"
- or &ThrowError("Could not create dumpfile '$file'");
+ if (!$gCfg{incremental}) {
+ open $fh, ">$file"
+ or &ThrowError("Could not create dumpfile '$file'");
+ }
+
my($sql, $sth, $action_sth, $row, $revision, $actions, $action, $physname,
$itemtype);
my %exported = ();
@@ -836,14 +839,31 @@
$action_sth = $gCfg{dbh}->prepare($sql);
my $autoprops = Vss2Svn::Dumpfile::AutoProps->new($gCfg{auto_props}) if
$gCfg{auto_props};
- my $dumpfile = Vss2Svn::Dumpfile->new($fh, $autoprops);
+ my $dumpfile;
+ if (!$gCfg{incremental}) {
+ $dumpfile = Vss2Svn::Dumpfile->new($fh, $autoprops);
+ }
+
REVISION:
while(defined($row = $sth->fetchrow_hashref() )) {
my $t0 = new Benchmark;
$revision = $row->{revision_id};
+ if ($gCfg{incremental}) {
+ my $paddedRev = sprintf("%06d", $revision);
+ $file = "$gCfg{dumpfile}.$paddedRev";
+ open $fh, ">$file"
+ or &ThrowError("Could not create dumpfile '$file'");
+
+ if ($dumpfile) {
+ $dumpfile->switch_fh($fh);
+ } else {
+ $dumpfile = Vss2Svn::Dumpfile->new($fh, $autoprops);
+ }
+ }
+
$dumpfile->begin_revision($row);
# next REVISION if $revision == 0;
@@ -869,6 +889,12 @@
$dumpfile->do_action($action, $exported{$physname});
}
+
+ if ($gCfg{incremental}) {
+ $dumpfile->finish();
+ close $fh;
+ }
+
print "revision $revision: ", timestr(timediff(new Benchmark,
$t0)),"\n"
if $gCfg{timing};
}
@@ -879,8 +905,10 @@
map { &ThrowWarning($_) } @err;
}
- $dumpfile->finish();
- close $fh;
+ if (!$gCfg{incremental}) {
+ $dumpfile->finish();
+ close $fh;
+ }
} # End CreateSvnDumpfile
@@ -1583,6 +1611,7 @@
GetOptions(\%gCfg,'vssdir=s','tempdir=s','dumpfile=s','resume','verbose',
'debug','timing+','task=s','revtimerange=i','ssphys=s',
+ 'incremental',
'encoding=s','trunkdir=s','auto_props=s');
&GiveHelp("Must specify --vssdir") if !defined($gCfg{vssdir});
@@ -1716,6 +1745,9 @@
default is ./_vss2svn
--dumpfile <file> : specify the subversion dumpfile to be created;
default is ./vss2svn-dumpfile.txt
+ --incremental : one dump file per revision;
+ the files will be named <file>.NNNNNN
+
--revtimerange <sec> : specify the difference between two ss actions
that are treated as one subversion revision;
default is 3600 seconds (== 1hour)
_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user
