On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vap...@gentoo.org> wrote:
> i think our current work flows also significantly impede the smooth running of
> this. if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> their behalf.
>
In theory, yes. In practice, git is too slow to handle 30,000 files.
Even simple operations like git add become painful even if you put the
whole of portage on tmpfs since git does a stat() on every single file
in the repository with every operation.
Simple test: do a git init followed by git add && git commit -m
"Initial commit" in your portage dir (.gitignore packages/ and
distfiles/)
Once this is done, you can test how it'll feel like to use a DSCM on
portage (without history). Unless you have a really fast SSD and
processor, you'll want to go back to the good old days of CVS with its
network-bound latencies on just 5-6 files in the current dir.
Besides this, there is the problem of accommodating people who use a
subtree of gentoo-x86, and those who don't want the entire CVS history
on their hard drives. In summation, robbat2 needs *our* help in the
following:
a) Push functionality in shallow clones (patches exist upstream)
b) Partial-tree checkouts (patches exist upstream)
c) Optimize git so it can handle 30,000 files
- Maybe maintain a cache of directory timestamps and only stat()
directories?
- Implement recursive timestamps on directories in various
filesystems and then in git (via xattrs perhaps)? People want to do
this for things like Tracker too. Prelim patches might exist.
d) Implement scripts/infra for people to fetch repository (shallow and
deep) bundles to initialize their local git clones (similar to portage
snapshots)
- git clone from scratch taxes the server too much, just like
rsync from scratch
e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS
- We want this for overlays right now too.
f) (Optional) Fix http cloning in git to make it "smarter" to help
people behind firewalls get anonymous clones (patches exist upstream)
Did I miss something Robin?
--
~Nirbheek Chauhan
Gentoo GNOME+Mozilla Team
