On 4/2/07, Philipp Marek <[EMAIL PROTECTED]> wrote:
...
> However, I saw a problem last night that performance is degraded quite
> a bit when committing a large number of files. More specifically, I
> see performance degrade when the total size of all the files are
> larger than about 130M.
Is it the file data, or the file amount?
Sorry, I should have specified that. This is file size, i.e., the total
amount of disk space used by a combination of all files.
Depending on your configuration (bdb vs. fsfs repository) and your
filesystem
(reiserfs, ext2, ext3 with/without dir-index) you might see a file lookup
problem in the transaction directory.
There's a note in doc/PERFORMANCE.
AFAIK, it's bdb. The LaCie was formatted as ext3 with dir_index (using
tune2fs).
Where I see this problem occur is in the size of the transaction/.../rev file
for the given svn transaction. The rev file becomes large. Once it gets
beyong 130M or so, disk access becomes much slower. Once it reached 2GB,
there was an access problem, and fsvs said as much, and the transaction was
rolled back (I lost the trace information, and I'm not sure I want to go
through that again--it was dreadfully slow).
Again, apologies that I didn't provide as much info in the initial email.
...
> but
> thought I should ask whether there might be a workaround that I can
> use. The workaround I'm using now is to run fsvs commit in batches,
> where I commit directories smaller than about 75M.
Is it the filesize, or the number of files?
Again, total file size, as in the transaction rev file in the repo.
> I would like to request a feature to be able to have fsvs commit an
> svn transaction after a given total file size is processed. (Provided
> I'm not out to lunch on the above, and doing something egregiously
> wrong.)
>
> In all, I have about 36GB of data I'd like to backup (moving to a new
> laptop soon--thanks to my employer), but doing this piecemeal is not
> quite what I had in mind.
I'd recommend updating your packages - mostly subversion and apr.
I'm using ( subversion-1.4.3 and apr-1.2.8, which afaik, are the latest
versions (though not dev versions). bdb is at 4.5.20_p2 (again, the latest).
> Any ideas would be appreciated.
Hmmm, that's not that easy. I'd say, the easiest way would be to do
something
like this, going the 2nd level of directories one-by-one:
cd $WC
find . -depth -maxdepth 2 -type d -exec fsvs ci -m {} {} \;
Yeah, I thought about that. The successful commits (that I did last night)
was doing limited commits like that.
If you'return versioning your *full* system, it might be better to use 3
levels
for /usr.
For now, I'm just versioning $HOME.
Thanks,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
