[cross-posting to dev-gaia]

TL;DR a shallow clone of the main B2G repo downloads 1/10th of the stuff
needed for a full clone and takes half the space on disk (including
built objects). Unless you have a very important reason for keeping the
whole project history around go ahead and use it, it's achieved this way
(for the flame in my example):

git clone https://github.com/mozilla-b2g/B2G.git
cd B2G
env REPO_INIT_FLAGS="--depth=1" ./config.sh flame-kk

Longer version:

I've been experimenting with shallow clones of the full B2G repo and
found that they provide an efficient way to have the complete sources
needed for a build while significantly cutting download times and
storage requirements so I wanted to share my experience with everybody.

Shallow cloning involves pulling all the repositories that are required
for a Firefox OS device/emulator build without their histories. This
means that the resulting clone will be usable as a regular one
(including sync'ing it) but it will not contain *any history* past the
very last commit for each repository.

For most developers this shouldn't be a problem. I've personally always
used separate clones of the repos I was working on (gaia, gecko,
gonk-misc, etc...) and never modified the sources of a full B2G clone
directly.

Another issue that it's worth keeping in mind is that switching branches
of the whole repo is going to be slower than with a full clone (but it's
already quite slow so I doubt you'll notice the difference).

To create a shallow clone you need to start from scratch, it won't work
on an existing clone that has full history.

First of all clone the B2G repo as usual:

git clone https://github.com/mozilla-b2g/B2G.git
cd B2G

Then configure it by passing the additional '--depth=1' parameter to the
'repo init' command. This will force git to fetch only the latest
version of the sources and no history:

env REPO_INIT_FLAGS="--depth=1" ./config.sh flame-kk

To speed up things I suggest also running more jobs, to saturate my 20Mb
ADSL connection I need at least ten which is more than the default 4 we
use in the config.sh script:

env REPO_INIT_FLAGS="--depth=1" REPO_SYNC_FLAGS="-j10" ./config.sh flame-kk

From this point on './repo sync' will work just as it used to with a
regular clone.

For this example the shallow clone downloads ~1/10th of the data
required for a full clone so that it now takes minutes whereas it used
to take hours. The resulting space usage on disk looks like this:

       default shallow
repos      20G    2.3G
total      27G    9.6G
build      35G     17G

'repos' represents the space required to store the repositories
themselves (.repo, .git), 'total' the above plus the checked out sources
and 'build' the above plus the object files. Even with a full build this
is half of a full clone, a pretty nice improvement especially for SSD
users with multiple clones as well as people building within VMs.

I'd like to add a note to
https://developer.mozilla.org/en-US/Firefox_OS/Preparing_for_your_first_B2G_build
because this would be especially useful for newcomers that don't want to
waste an entire day downloading sources. However I'm not sure where to
put it, it should be rather prominent IMHO as a common complaint from
new contributors is often the time spent during the first clone.

 Gabriele

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g

Reply via email to