Gabriele - This looks really great, thanks for the tip! It seems like needing the history of things pulled in by config.sh would be the exception, and not the norm. How do you feel about making this the default?
Best, Kevin On Tue, May 12, 2015 at 4:01 PM, Gabriele Svelto <[email protected]> wrote: > [cross-posting to dev-gaia] > > TL;DR a shallow clone of the main B2G repo downloads 1/10th of the stuff > needed for a full clone and takes half the space on disk (including > built objects). Unless you have a very important reason for keeping the > whole project history around go ahead and use it, it's achieved this way > (for the flame in my example): > > git clone https://github.com/mozilla-b2g/B2G.git > cd B2G > env REPO_INIT_FLAGS="--depth=1" ./config.sh flame-kk > > Longer version: > > I've been experimenting with shallow clones of the full B2G repo and > found that they provide an efficient way to have the complete sources > needed for a build while significantly cutting download times and > storage requirements so I wanted to share my experience with everybody. > > Shallow cloning involves pulling all the repositories that are required > for a Firefox OS device/emulator build without their histories. This > means that the resulting clone will be usable as a regular one > (including sync'ing it) but it will not contain *any history* past the > very last commit for each repository. > > For most developers this shouldn't be a problem. I've personally always > used separate clones of the repos I was working on (gaia, gecko, > gonk-misc, etc...) and never modified the sources of a full B2G clone > directly. > > Another issue that it's worth keeping in mind is that switching branches > of the whole repo is going to be slower than with a full clone (but it's > already quite slow so I doubt you'll notice the difference). > > To create a shallow clone you need to start from scratch, it won't work > on an existing clone that has full history. > > First of all clone the B2G repo as usual: > > git clone https://github.com/mozilla-b2g/B2G.git > cd B2G > > Then configure it by passing the additional '--depth=1' parameter to the > 'repo init' command. This will force git to fetch only the latest > version of the sources and no history: > > env REPO_INIT_FLAGS="--depth=1" ./config.sh flame-kk > > To speed up things I suggest also running more jobs, to saturate my 20Mb > ADSL connection I need at least ten which is more than the default 4 we > use in the config.sh script: > > env REPO_INIT_FLAGS="--depth=1" REPO_SYNC_FLAGS="-j10" ./config.sh flame-kk > > From this point on './repo sync' will work just as it used to with a > regular clone. > > For this example the shallow clone downloads ~1/10th of the data > required for a full clone so that it now takes minutes whereas it used > to take hours. The resulting space usage on disk looks like this: > > default shallow > repos 20G 2.3G > total 27G 9.6G > build 35G 17G > > 'repos' represents the space required to store the repositories > themselves (.repo, .git), 'total' the above plus the checked out sources > and 'build' the above plus the object files. Even with a full build this > is half of a full clone, a pretty nice improvement especially for SSD > users with multiple clones as well as people building within VMs. > > I'd like to add a note to > > https://developer.mozilla.org/en-US/Firefox_OS/Preparing_for_your_first_B2G_build > because this would be especially useful for newcomers that don't want to > waste an entire day downloading sources. However I'm not sure where to > put it, it should be rather prominent IMHO as a common complaint from > new contributors is often the time spent during the first clone. > > Gabriele > > > _______________________________________________ > dev-b2g mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-b2g > >
_______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
