> > - CVS wants to make sure that the second changes before it exits (which
> > means only one operation per second, which is frustrating with a hundred
> > thousand CVS changesets). To avoid this, I rsync'd a new copy of the
> > repository (so that there was no possibility of corruption) and then
> > modified CVS to not at the end.
>
> I'm not sure what this mean: I see you modified CVS source, but it's not
> clear what the impact on tailor is. Could you please write down a note
> for the README or the wiki?
Sure...
The patch fixes problems with a slow CVS import. The problem is with CVS, not
Tailor, but is of interest to Tailor users as it can speed up imports from CVS
tremendously. By default (with the version of CVS that I use), CVS will
ensure that any update command takes at least one second, by sleeping if
necessary until the second counter has ticked over before releasing its locks.
I believe that it's due to possible races on filesystems that only have one
second precision on their timestamp fields, and CVS tries to be safe in this
regard. In interactive use of CVS, it's not a problem. When using an
automated tool like Tailor, however, it can add significantly to the import
time. For example, in our CVS repository that includes 100,000 different CVS
changesets, 100,000 seconds (about 30 hours) are added unnecessarily to the
import time. Without the patch, it takes about 30 hours to import, whereas it
takes 2 hours without.
You can see the difference in the following logs:
before (unpatched CVS):
09:48:44 [I] Bootstrap completed
09:48:44 [I] Updating "jeremy-papers" in "/export/indexes/import/jeremy-papers"
09:48:44 [I] Applying pending upstream changesets
09:48:44 [I] Changeset "2001-02-01 06:23:28 by jeremy"
09:48:44 [I] Log message: Initial revision
09:48:44 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 al-benefits/Makefile
09:48:45 [I] [Ok]
09:48:45 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/al-benefits.tex
09:48:46 [I] [Ok]
09:48:46 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/cum-freq-senses.m
09:48:47 [I] [Ok]
09:48:47 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/rationalist-empirical.fig
09:48:48 [I] [Ok]
09:48:48 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/sense_stats.cc
09:48:49 [I] [Ok]
09:48:49 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 bibliography/papers.bib
09:48:50 [I] [Ok]
09:48:50 [I] /export/indexes/import/jeremy-papers $ cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 build/papers-include.mk
09:48:51 [I] [Ok]
Notice that exactly one CVS command per second occurs (look at the
timestamp at the start of the line).
after (patched CVS):
09:52:11 [I] Bootstrap completed
09:52:11 [I] Updating "jeremy-papers" in "/export/indexes/import/jeremy-papers"
09:52:11 [I] Applying pending upstream changesets
09:52:11 [I] Changeset "2001-02-01 06:23:28 by jeremy"
09:52:11 [I] Log message: Initial revision
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 al-benefits/Makefile
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/al-benefits.tex
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/cum-freq-senses.m
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/rationalist-empirical.fig
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1
al-benefits/sense_stats.cc
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 bibliography/papers.bib
09:52:11 [I] [Ok]
09:52:11 [I] /export/indexes/import/jeremy-papers $
/home/jeremy/packages/rpm/BUILD/cvs-1.11.19/src/cvs -d
/home/jeremy/server/mycvsroot/ -q update -kk -d -r 1.1 build/papers-include.mk
09:52:11 [I] [Ok]
Notice that the commands all occur within the same second. This leads to a
**much** faster import, as I mentioned above.
Jeremy
--- BUILD/cvs-1.11.19/src/update.c 2005-01-31 17:18:01.000000000 -0500
+++ BUILD/cvs-1.11.19-nowait/src/update.c 2006-05-27 21:52:27.621272500
-0400
@@ -525,11 +525,11 @@
#endif
/* see if we need to sleep before returning to avoid time-stamp races */
if (last_register_time)
{
- sleep_past (last_register_time);
+ /* sleep_past (last_register_time); */
}
return err;
}
_______________________________________________
Tailor mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/tailor
