joerg.sonnenberger created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches.
REVISION SUMMARY The new REQUIREDRAM option allows a client to skip bundles it isn't expected to handle well, e.g. without swapping. This allows a fallback path to be provided e.g. using zstd level 10 instead of 22. REPOSITORY rHG Mercurial BRANCH stable REVISION DETAIL https://phab.mercurial-scm.org/D8645 AFFECTED FILES mercurial/exchange.py relnotes/next tests/test-clonebundles.t CHANGE DETAILS diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t --- a/tests/test-clonebundles.t +++ b/tests/test-clonebundles.t @@ -567,3 +567,88 @@ searching for changes no changes found 2 local changesets published + $ killdaemons.py + +A manifest with a gzip bundle requiring too much memory for a 16MB system and working +on a 32MB system. + + $ "$PYTHON" $TESTDIR/dumbhttp.py -p $HGPORT1 --pid http.pid + $ cat http.pid >> $DAEMON_PIDS + $ hg -R server serve -d -p $HGPORT --pid-file hg.pid --accesslog access.log + $ cat hg.pid >> $DAEMON_PIDS + + $ cat > server/.hg/clonebundles.manifest << EOF + > http://localhost:$HGPORT1/gz-a.hg BUNDLESPEC=gzip-v2 REQUIREDRAM=12MB + > EOF + + $ hg clone -U --debug --config ui.available-memory=16MB http://localhost:$HGPORT gzip-too-large + using http://localhost:$HGPORT/ + sending capabilities command + sending clonebundles command + filtering http://localhost:$HGPORT1/gz-a.hg as it needs more than 2/3 of system memory + no compatible clone bundles available on server; falling back to regular clone + (you may want to report this to the server operator) + query 1; heads + sending batch command + requesting all changes + sending getbundle command + bundle2-input-bundle: with-transaction + bundle2-input-part: "changegroup" (params: 1 mandatory 1 advisory) supported + adding changesets + add changeset 53245c60e682 + add changeset aaff8d2ffbbf + adding manifests + adding file changes + adding bar revisions + adding foo revisions + bundle2-input-part: total payload size 920 + bundle2-input-part: "listkeys" (params: 1 mandatory) supported + bundle2-input-part: "phase-heads" supported + bundle2-input-part: total payload size 24 + bundle2-input-part: "cache:rev-branch-cache" (advisory) supported + bundle2-input-part: total payload size 59 + bundle2-input-bundle: 4 parts total + checking for updated bookmarks + updating the branch cache + added 2 changesets with 2 changes to 2 files + new changesets 53245c60e682:aaff8d2ffbbf + calling hook changegroup.lfiles: hgext.largefiles.reposetup.checkrequireslfiles + (sent 4 HTTP requests and 1576 bytes; received 1124 bytes in responses) + + $ hg clone -U --debug --config ui.available-memory=32MB http://localhost:$HGPORT gzip-too-large2 + using http://localhost:$HGPORT/ + sending capabilities command + sending clonebundles command + applying clone bundle from http://localhost:$HGPORT1/gz-a.hg + bundle2-input-bundle: 1 params with-transaction + bundle2-input-part: "changegroup" (params: 1 mandatory 1 advisory) supported + adding changesets + add changeset 53245c60e682 + add changeset aaff8d2ffbbf + adding manifests + adding file changes + adding bar revisions + adding foo revisions + bundle2-input-part: total payload size 920 + bundle2-input-part: "cache:rev-branch-cache" (advisory) supported + bundle2-input-part: total payload size 59 + bundle2-input-bundle: 2 parts total + updating the branch cache + added 2 changesets with 2 changes to 2 files + finished applying clone bundle + query 1; heads + sending batch command + searching for changes + all remote heads known locally + no changes found + sending getbundle command + bundle2-input-bundle: with-transaction + bundle2-input-part: "listkeys" (params: 1 mandatory) supported + bundle2-input-part: "phase-heads" supported + bundle2-input-part: total payload size 24 + bundle2-input-bundle: 2 parts total + checking for updated bookmarks + 2 local changesets published + calling hook changegroup.lfiles: hgext.largefiles.reposetup.checkrequireslfiles + (sent 4 HTTP requests and 1602 bytes; received 741 bytes in responses) + $ killdaemons.py diff --git a/relnotes/next b/relnotes/next --- a/relnotes/next +++ b/relnotes/next @@ -29,6 +29,10 @@ * `hg debugmergestate` output is now templated, which may be useful e.g. for IDEs that want to help the user resolve merge conflicts. + * clonebundles can be annotated with the expected memory requirements + using the `REQUIREDRAM` option. This allows clients to skip + bundles created with large zstd windows and fallback to larger, but + less demanding bundles. == New Experimental Features == diff --git a/mercurial/exchange.py b/mercurial/exchange.py --- a/mercurial/exchange.py +++ b/mercurial/exchange.py @@ -3025,6 +3025,23 @@ ) continue + if b'REQUIREDRAM' in entry: + try: + requiredram = util.sizetoint(entry[b'REQUIREDRAM']) + except error.ParseError: + repo.ui.debug( + b'filtering %s due to a bad REQUIREDRAM attribute\n' + % entry[b'URL'] + ) + continue + actualram = util.estimatememory(repo.ui) + if actualram is not None and actualram * 0.66 < requiredram: + repo.ui.debug( + b'filtering %s as it needs more than 2/3 of system memory\n' + % entry[b'URL'] + ) + continue + newentries.append(entry) return newentries To: joerg.sonnenberger, #hg-reviewers Cc: mercurial-patches, mercurial-devel _______________________________________________ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
