In my earlier series introducing "git cat-file --batch-check=<format>",
I spent a little time optimizing revindex generation, and measured by
requesting information on a single object from a large repository. This
series takes the next logical step: requesting a large number of objects
from a large repository.
There are two major optimizations here:
1. Avoiding extra ref lookups due to the warning in 798c35f (get_sha1:
warn about full or short object names that look like refs,
2. Avoiding extra work for delta type resolution when the user has not
asked for %(objecttype).
I prepared the series on top of jk/in-pack-size-measurement, and
certainly optimization 2 is pointless without it (before that topic,
--batch-check always printed the type).
However, the first optimization affects regular --batch-check, and
represents a much more serious performance regression. It looks like
798c35f is in master, but hasn't been released yet, so assuming these
topics graduate before the next release, it should be OK. But if not, we
should consider pulling the first patch out and applying it (or
something like it) separately.
The results for running (in linux.git):
$ git rev-list --objects --all >objects
$ git cat-file --batch-check='%(objectsize:disk)' <objects >/dev/null
real 1m17.143s 0m7.205s
user 0m27.684s 0m6.580s
sys 0m49.320s 0m0.608s
Now, _most_ of that speedup is coming from the first patch, and it's
quite trivial. The rest of the patches involve a lot of refactoring, and
only manage to eke out one more second of performance, so it may not be
worth it (though I think the result actually cleans up the
sha1_object_info_extended interface a bit, and is worth it). Individual
timings are in the commit messages.
The patches are:
[1/7]: cat-file: disable object/refname ambiguity check for batch mode
[2/7]: sha1_object_info_extended: rename "status" to "type"
[3/7]: sha1_loose_object_info: make type lookup optional
[4/7]: packed_object_info: hoist delta type resolution to helper
[5/7]: packed_object_info: make type lookup optional
[6/7]: sha1_object_info_extended: make type calculation optional
[7/7]: sha1_object_info_extended: pass object_info to helpers
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html