-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15.08.2013 16:02, Merlijn van Deen wrote: > On 15 August 2013 12:27, Amir Ladsgroup <[email protected] > <mailto:[email protected]>> wrote: > > I asked about a solution of the 200M everywhere. The best option > is cloning shallowly > > > > No, it's not. After running git gc, I can re-clone the existing > repository by only transferring 8MB: > > valhallasw@lisilwen:~/src/pywikibot-compat$ git daemon --reuseaddr > --base-path=. --export-all --verbose & > valhallasw@lisilwen:~/src/pywikibot-compat$ cd .. > valhallasw@lisilwen:~/src$ git clone git://localhost/ pwb2 Cloning > into 'pwb2'... [6373] Connection from 127.0.0.1:36357 > <http://127.0.0.1:36357> [6373] Extended attributes (16 bytes) > exist <host=localhost> [6373] Request upload-pack for '/' remote: > Counting objects: 37384, done. remote: Compressing objects: 100% > (10563/10563), done. [6370] [6373] Disconnected2525/37384) remote: > Total 37384 (delta 26576), reused 37384 (delta 26576) Receiving > objects: 100% (37384/37384), 7.96 MiB, done. Resolving deltas: 100% > (26576/26576), done. > > So it should be possible to solve this issue on the Wikimedia end: > downloading 7MB sounds very reasonable to me.
+1 I had a deeper look of this, consider e.g. compat. First cloning everything (including externals) and then garbage collecting: $ git clone --recursive https://gerrit.wikimedia.org/r/pywikibot/compat.git [target-dir] $ cd [target-dir] $ git gc --aggressive --prune $ cd i18n/ $ git gc --aggressive --prune $ cd ../externals/spelling/ $ git gc --aggressive --prune results in ~24MB size. But after cloning it is ~210MB so you need that much of free disk space and it is slow. An alternative, not needing that much free space and beeing a lot faster, is shallow cloning: $ git clone --depth 1 --recursive https://gerrit.wikimedia.org/r/pywikibot/compat.git [target-dir] and results in a repo size of ~36MB. The problem here is in order to shallow clone the submodules also you need either git 1.8.4 [1] or to follow a quite complex sequence of steps. [1] http://stackoverflow.com/questions/6238590/set-git-submodule-to-shallow-clone-sparse-checkout Now forgetting about externals and looking at compat only results in: $ git clone https://gerrit.wikimedia.org/r/pywikibot/compat.git [target-dir] $ cd [target-dir] $ git gc --aggressive --prune gives a size of ~15MB as mentioned by Merljin already, and: $ git clone --depth 1 https://gerrit.wikimedia.org/r/pywikibot/compat.git [target-dir] one of only ~6MB so it is possible to reduce it even more, but only if not considering externals. Recognize also that garbage collecting works better when working with the externals, otherwise shallow cloning gives smaller sizes. Sizes around 20MB for everything - including externals - are ok for me, but I would prefer not having to do garbage collection (gc). This would solve issues with small quotas (disk sizes) and reduce the possible sources of other problems (e.g. reducing the number of command to enter ;). Therefore, solving this on the server side would be a good thing! Greetings DrTrigon -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIQqsQACgkQAXWvBxzBrDDU3wCeOFo5OfezBT8K12g22uLiPLOB e/0AoK2aHP2q4hYH856MsVDHptNYgiTr =EZU7 -----END PGP SIGNATURE----- _______________________________________________ Pywikipedia-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
