TL;DR: dumb HTTP clone from a certain badly-packed repo goes from ~2 hours to ~30 min memory usage drops from 2G to 360M
I hadn't packed the public repo at https://public-inbox.org/git for a few weeks. As an admin of a small server limited memory and CPU resources but fairly good bandwidth, I prefer clients use dumb HTTP for initial clones. Unfortunately, I noticed my dinky netbook runs out-of-memory when using GIT_SMART_HTTP=0 to clone this giant repo; and a machine with more memory still takes over two hours depending on network conditions (and uses around 2GB RSS!). Anyways, https://public-inbox.org/git is better packed, now; but I've kept https://80x24.org/git-i-forgot-to-pack available with over 7K loose objects to illustrate the problem: (this is dumb HTTP-only) git clone --mirror https://80x24.org/git-i-forgot-to-pack The primary problem is fixed by PATCH 3/3 in this series, and I can now clone the above in around 30 minutes and "only" seems to use around 360M memory. I'll leave git-i-forgot-to-pack up for a few months/year so others can test and hammer away at it. The following changes since commit 5c589a73de4394ad125a4effac227b3aec856fa1: Third batch of topics for 2.10 (2016-07-06 13:42:58 -0700) are available in the git repository at: git://bogomips.org/git-svn.git dumb-speedups for you to fetch changes up to b9d5aca4b8e6c9f7fb5ee4e0ce33bb42c4ea2992: http-walker: reduce O(n) ops with doubly-linked list (2016-07-11 20:25:51 +0000) ---------------------------------------------------------------- Eric Wong (3): http-walker: remove unused parameter from fetch_object http: avoid disconnecting on 404s for loose objects http-walker: reduce O(n) ops with doubly-linked list http-walker.c | 55 ++++++++++---------- http.c | 16 +++++- list.h | 164 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 204 insertions(+), 31 deletions(-) create mode 100644 list.h -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
