I have been experimenting with putting images (20000 of them) into a darcs repository. This is mainly to help me keep track of when individual images have been updated, and to ease deployment.
I was amazed to see how rapidly the speed of 'darcs record'
diminished.
I wrote a quick test example to show the rate of records. I ran the
test with 3000 patches, but the rate continues to decrease. To record
all 20000 of my images took about 24 hours.
mkdir image_test
cd image_test
darcs init
for number in `seq 1 3000`
do
echo "generating image $number"
convert -background white -fill black -size 150x100 -gravity center
label:$number $number.jpg
echo `date +"%s.%N"` >> create.log
done
for number in `seq 1 3000`
do
darcs add $number.jpg
darcs record --all --author image_test -m "initial import of image $number"
$number.jpg
echo `date +"%s.%N"` >> record.log
done
cat create.log | perl -e 'my $a; while (my $b = <STDIN>) {
chomp($b);
print ++$c . "\t" . (60 / ($b - $a)) . "\n" if $a; $a = $b
}' > create_rate.txt
echo 'set term png; set output "create_rate.png"; plot "create_rate.txt"' |
gnuplot -
cat record.log | perl -e 'my $a; while (my $b = <STDIN>) {
chomp($b);
print ++$c . "\t" . (60 / ($b - $a)) . "\n" if $a; $a = $b
}' > record_rate.txt
echo 'set term png; set output "record_rate.png"; plot "record_rate.txt"' |
gnuplot -
display record_rate.png create_rate.png
cd ..
It appears that darcs repositories with non-binary files do the same
thing, which may mean that the slow-down has to do with the number of
files, rather than the number of commits.
mkdir text_test; cd text_test; darcs init; for number in `seq 1 3000`; do echo
"generating file $number"; echo "this is file $number" > $number.txt; echo
`date +"%s.%N"` >> create.log; done; for number in `seq 1 3000`; do darcs add
$number.txt; darcs record --all --author text_test -m "initial import of file
$number" $number.txt; echo `date +"%s.%N"` >> record.log; done; cat record.log
| perl -e 'my $a; while (my $b = <STDIN>) { chomp($b); print ++$c . "\t" . (60
/ ($b - $a)) . "\n" if $a; $a = $b }' > record_rate.txt; echo 'set term png;
set output "record_rate.png"; plot "record_rate.txt"' | gnuplot -; display
record_rate.png; cd ..
My attempts at changing a single file 3000 times results in a repo
with less than 100 patches, because most of the time the change is not
detected. I am not sure how to overcome this problem and get useful
benchmarking.
mkdir text_update_test; cd text_update_test; darcs init; touch 1.txt; darcs add
1.txt; for number in `seq 1 3000`; do echo "this is change $number" > 1.txt;
cat 1.txt; darcs record --all --author text_update_test -m "updated to number
$number" 1.txt; echo `date +"%s.%N"` >> record.log; done; cat record.log | perl
-e 'my $a; while (my $b = <STDIN>) { chomp($b); print ++$c . "\t" . (60 / ($b -
$a)) . "\n" if $a; $a = $b }' > record_rate.txt; echo 'set term png; set output
"record_rate.png"; plot "record_rate.txt"' | gnuplot -; display
record_rate.png; cd ..
I am attaching the rate graphs for all three tests.
-kolibrie
<<attachment: record_rate_image.png>>
<<attachment: record_rate_text.png>>
<<attachment: record_rate_update_text.png>>
signature.asc
Description: Digital signature
_______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
