joerg.sonnenberger updated this revision to Diff 4932.
joerg.sonnenberger retitled this revision from "wireproto: support for 
pullbundles" to "* wireproto: support for pullbundles".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1856?vs=4853&id=4932

REVISION DETAIL
  https://phab.mercurial-scm.org/D1856

AFFECTED FILES
  hgext/clonebundles.py
  mercurial/bundle2.py
  mercurial/commands.py
  mercurial/configitems.py
  mercurial/exchange.py
  mercurial/help/config.txt
  mercurial/wireproto.py
  tests/common-pattern.py
  tests/test-acl.t
  tests/test-bookmarks-pushpull.t
  tests/test-debugcommands.t
  tests/test-help.t
  tests/test-http-bad-server.t
  tests/test-pull-bundle.t
  tests/test-ssh-bundle1.t
  tests/test-ssh.t

CHANGE DETAILS

diff --git a/tests/test-ssh.t b/tests/test-ssh.t
--- a/tests/test-ssh.t
+++ b/tests/test-ssh.t
@@ -486,7 +486,7 @@
   devel-peer-request: between
   devel-peer-request:   pairs: 81 bytes
   sending between command
-  remote: 384
+  remote: 399
   remote: capabilities: lookup changegroupsubset branchmap pushkey known 
getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN
   remote: 1
   query 1; heads
diff --git a/tests/test-ssh-bundle1.t b/tests/test-ssh-bundle1.t
--- a/tests/test-ssh-bundle1.t
+++ b/tests/test-ssh-bundle1.t
@@ -467,7 +467,7 @@
   running .* ".*/dummyssh" ['"]user@dummy['"] ('|")hg -R remote serve 
--stdio('|") (re)
   sending hello command
   sending between command
-  remote: 384
+  remote: 399
   remote: capabilities: lookup changegroupsubset branchmap pushkey known 
getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN
   remote: 1
   preparing listkeys for "bookmarks"
diff --git a/tests/test-pull-bundle.t b/tests/test-pull-bundle.t
new file mode 100644
--- /dev/null
+++ b/tests/test-pull-bundle.t
@@ -0,0 +1,155 @@
+  $ hg init repo
+  $ cd repo
+  $ echo foo > foo
+  $ hg ci -qAm 'add foo'
+  $ echo >> foo
+  $ hg ci -m 'change foo'
+  $ hg up -qC 0
+  $ echo bar > bar
+  $ hg ci -qAm 'add bar'
+
+  $ hg log
+  changeset:   2:effea6de0384
+  tag:         tip
+  parent:      0:bbd179dfa0a7
+  user:        test
+  date:        Thu Jan 01 00:00:00 1970 +0000
+  summary:     add bar
+  
+  changeset:   1:ed1b79f46b9a
+  user:        test
+  date:        Thu Jan 01 00:00:00 1970 +0000
+  summary:     change foo
+  
+  changeset:   0:bbd179dfa0a7
+  user:        test
+  date:        Thu Jan 01 00:00:00 1970 +0000
+  summary:     add foo
+  
+  $ cd ..
+
+Test pullbundle functionality
+
+  $ cd repo
+  $ cat <<EOF > .hg/hgrc
+  > [server]
+  > pullbundle = True
+  > [extensions]
+  > blackbox =
+  > EOF
+  $ hg bundle --base null -r 0 .hg/0.hg
+  1 changesets found
+  $ hg bundle --base 0 -r 1 .hg/1.hg
+  1 changesets found
+  $ hg bundle --base 1 -r 2 .hg/2.hg
+  1 changesets found
+  $ cat <<EOF > .hg/pullbundles.manifest
+  > 2.hg heads=effea6de0384e684f44435651cb7bd70b8735bd4 
bases=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
+  > 1.hg heads=ed1b79f46b9a29f5a6efa59cf12fcfca43bead5a 
bases=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
+  > 0.hg heads=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
+  > EOF
+  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid
+  listening at http://localhost:$HGPORT2/ (bound to $LOCALIP:$HGPORT2)
+  $ cat ../repo.pid >> $DAEMON_PIDS
+  $ cd ..
+  $ hg clone -r 0 http://localhost:$HGPORT2/ repo.pullbundle
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets bbd179dfa0a7
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cd repo.pullbundle
+  $ hg pull -r 1
+  pulling from http://localhost:$HGPORT2/
+  searching for changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets ed1b79f46b9a
+  (run 'hg update' to get a working copy)
+  $ hg pull -r 2
+  pulling from http://localhost:$HGPORT2/
+  searching for changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files (+1 heads)
+  new changesets effea6de0384
+  (run 'hg heads' to see heads, 'hg merge' to merge)
+  $ cd ..
+  $ killdaemons.py
+  $ grep 'sending pullbundle ' repo/.hg/blackbox.log
+  * sending pullbundle "0.hg" (glob)
+  * sending pullbundle "1.hg" (glob)
+  * sending pullbundle "2.hg" (glob)
+  $ rm repo/.hg/blackbox.log
+
+Test pullbundle functionality for incremental pulls
+
+  $ cd repo
+  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid
+  listening at http://localhost:$HGPORT2/ (bound to $LOCALIP:$HGPORT2)
+  $ cat ../repo.pid >> $DAEMON_PIDS
+  $ cd ..
+  $ hg clone http://localhost:$HGPORT2/ repo.pullbundle2
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files (+1 heads)
+  new changesets bbd179dfa0a7:ed1b79f46b9a
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ killdaemons.py
+  $ grep 'sending pullbundle ' repo/.hg/blackbox.log
+  * sending pullbundle "0.hg" (glob)
+  * sending pullbundle "2.hg" (glob)
+  * sending pullbundle "1.hg" (glob)
+  $ rm repo/.hg/blackbox.log
+
+Test recovery from misconfigured server sending no new data
+
+  $ cd repo
+  $ cat <<EOF > .hg/pullbundles.manifest
+  > 0.hg heads=ed1b79f46b9a29f5a6efa59cf12fcfca43bead5a 
bases=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
+  > 0.hg heads=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
+  > EOF
+  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid
+  listening at http://localhost:$HGPORT2/ (bound to $LOCALIP:$HGPORT2)
+  $ cat ../repo.pid >> $DAEMON_PIDS
+  $ cd ..
+  $ hg clone -r 0 http://localhost:$HGPORT2/ repo.pullbundle3
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets bbd179dfa0a7
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cd repo.pullbundle3
+  $ hg pull -r 1
+  pulling from http://localhost:$HGPORT2/
+  searching for changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 0 changesets with 0 changes to 1 files
+  abort: 00changelog.i@ed1b79f46b9a: no node!
+  [255]
+  $ cd ..
+  $ killdaemons.py
+  $ grep 'sending pullbundle ' repo/.hg/blackbox.log
+  * sending pullbundle "0.hg" (glob)
+  * sending pullbundle "0.hg" (glob)
+  $ rm repo/.hg/blackbox.log
diff --git a/tests/test-http-bad-server.t b/tests/test-http-bad-server.t
--- a/tests/test-http-bad-server.t
+++ b/tests/test-http-bad-server.t
@@ -118,9 +118,9 @@
   write(23) -> Server: badhttpserver\r\n
   write(37) -> Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
   write(41) -> Content-Type: application/mercurial-0.1\r\n
-  write(21) -> Content-Length: 417\r\n
+  write(21) -> Content-Length: 432\r\n
   write(2) -> \r\n
-  write(417) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(432) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(4? from 65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n (glob)
   readline(1? from -1) -> (1?) Accept-Encoding* (glob)
   read limit reached; closing socket
@@ -159,9 +159,9 @@
   write(23) -> Server: badhttpserver\r\n
   write(37) -> Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
   write(41) -> Content-Type: application/mercurial-0.1\r\n
-  write(21) -> Content-Length: 417\r\n
+  write(21) -> Content-Length: 432\r\n
   write(2) -> \r\n
-  write(417) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(432) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(13? from 65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n (glob)
   readline(1?? from -1) -> (27) Accept-Encoding: identity\r\n (glob)
   readline(8? from -1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n (glob)
@@ -216,9 +216,9 @@
   write(23) -> Server: badhttpserver\r\n
   write(37) -> Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
   write(41) -> Content-Type: application/mercurial-0.1\r\n
-  write(21) -> Content-Length: 430\r\n
+  write(21) -> Content-Length: 445\r\n
   write(2) -> \r\n
-  write(430) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httppostargs 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
+  write(445) -> lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httppostargs httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline\(14[67] from 65537\) -> \(2[67]\) POST /\?cmd=batch HTTP/1.1\\r\\n 
(re)
   readline\(1(19|20) from -1\) -> \(27\) Accept-Encoding: identity\\r\\n (re)
   readline(9? from -1) -> (41) content-type: application/mercurial-0.1\r\n 
(glob)
@@ -275,7 +275,7 @@
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
-  abort: HTTP request error (incomplete response; expected 397 bytes got 20)
+  abort: HTTP request error (incomplete response; expected 412 bytes got 20)
   (this may be an intermittent network failure; if the error persists, 
consider contacting the network or server operator)
   [255]
 
@@ -292,16 +292,16 @@
   write(23 from 23) -> (121) Server: badhttpserver\r\n
   write(37 from 37) -> (84) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
   write(41 from 41) -> (43) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (22) Content-Length: 417\r\n
+  write(21 from 21) -> (22) Content-Length: 432\r\n
   write(2 from 2) -> (20) \r\n
-  write(20 from 417) -> (0) lookup changegroupsu
+  write(20 from 432) -> (0) lookup changegroupsu
   write limit reached; closing socket
 
   $ rm -f error.log
 
 Server sends incomplete headers for batch request
 
-  $ hg serve --config badserver.closeaftersendbytes=695 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=710 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 TODO this output is horrible
@@ -323,13 +323,13 @@
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
   readline(-1) -> (49) user-agent: mercurial/proto-1.0 (Mercurial 4.2)\r\n
   readline(-1) -> (2) \r\n
-  write(36 from 36) -> (659) HTTP/1.1 200 Script output follows\r\n
-  write(23 from 23) -> (636) Server: badhttpserver\r\n
-  write(37 from 37) -> (599) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
-  write(41 from 41) -> (558) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (537) Content-Length: 417\r\n
-  write(2 from 2) -> (535) \r\n
-  write(417 from 417) -> (118) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(36 from 36) -> (674) HTTP/1.1 200 Script output follows\r\n
+  write(23 from 23) -> (651) Server: badhttpserver\r\n
+  write(37 from 37) -> (614) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
+  write(41 from 41) -> (573) Content-Type: application/mercurial-0.1\r\n
+  write(21 from 21) -> (552) Content-Length: 432\r\n
+  write(2 from 2) -> (550) \r\n
+  write(432 from 432) -> (118) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
@@ -350,7 +350,7 @@
 
 Server sends an incomplete HTTP response body to batch request
 
-  $ hg serve --config badserver.closeaftersendbytes=760 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=775 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 TODO client spews a stack due to uncaught ValueError in batch.results()
@@ -371,13 +371,13 @@
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
   readline(-1) -> (49) user-agent: mercurial/proto-1.0 (Mercurial 4.2)\r\n
   readline(-1) -> (2) \r\n
-  write(36 from 36) -> (724) HTTP/1.1 200 Script output follows\r\n
-  write(23 from 23) -> (701) Server: badhttpserver\r\n
-  write(37 from 37) -> (664) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
-  write(41 from 41) -> (623) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (602) Content-Length: 417\r\n
-  write(2 from 2) -> (600) \r\n
-  write(417 from 417) -> (183) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(36 from 36) -> (739) HTTP/1.1 200 Script output follows\r\n
+  write(23 from 23) -> (716) Server: badhttpserver\r\n
+  write(37 from 37) -> (679) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
+  write(41 from 41) -> (638) Content-Type: application/mercurial-0.1\r\n
+  write(21 from 21) -> (617) Content-Length: 432\r\n
+  write(2 from 2) -> (615) \r\n
+  write(432 from 432) -> (183) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
@@ -400,7 +400,7 @@
 
 Server sends incomplete headers for getbundle response
 
-  $ hg serve --config badserver.closeaftersendbytes=907 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=922 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
 TODO this output is terrible
@@ -423,13 +423,13 @@
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
   readline(-1) -> (49) user-agent: mercurial/proto-1.0 (Mercurial 4.2)\r\n
   readline(-1) -> (2) \r\n
-  write(36 from 36) -> (871) HTTP/1.1 200 Script output follows\r\n
-  write(23 from 23) -> (848) Server: badhttpserver\r\n
-  write(37 from 37) -> (811) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
-  write(41 from 41) -> (770) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (749) Content-Length: 417\r\n
-  write(2 from 2) -> (747) \r\n
-  write(417 from 417) -> (330) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(36 from 36) -> (886) HTTP/1.1 200 Script output follows\r\n
+  write(23 from 23) -> (863) Server: badhttpserver\r\n
+  write(37 from 37) -> (826) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
+  write(41 from 41) -> (785) Content-Type: application/mercurial-0.1\r\n
+  write(21 from 21) -> (764) Content-Length: 432\r\n
+  write(2 from 2) -> (762) \r\n
+  write(432 from 432) -> (330) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
@@ -449,7 +449,7 @@
   readline(65537) -> (30) GET /?cmd=getbundle HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
-  readline(-1) -> (422) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
+  readline(-1) -> (439) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apartial-pull%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
   readline(-1) -> (48) x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$\r\n
   readline(-1) -> (35) accept: application/mercurial-0.1\r\n
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
@@ -466,7 +466,7 @@
 
 Server sends empty HTTP body for getbundle
 
-  $ hg serve --config badserver.closeaftersendbytes=945 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=960 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -484,13 +484,13 @@
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
   readline(-1) -> (49) user-agent: mercurial/proto-1.0 (Mercurial 4.2)\r\n
   readline(-1) -> (2) \r\n
-  write(36 from 36) -> (909) HTTP/1.1 200 Script output follows\r\n
-  write(23 from 23) -> (886) Server: badhttpserver\r\n
-  write(37 from 37) -> (849) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
-  write(41 from 41) -> (808) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (787) Content-Length: 417\r\n
-  write(2 from 2) -> (785) \r\n
-  write(417 from 417) -> (368) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(36 from 36) -> (924) HTTP/1.1 200 Script output follows\r\n
+  write(23 from 23) -> (901) Server: badhttpserver\r\n
+  write(37 from 37) -> (864) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
+  write(41 from 41) -> (823) Content-Type: application/mercurial-0.1\r\n
+  write(21 from 21) -> (802) Content-Length: 432\r\n
+  write(2 from 2) -> (800) \r\n
+  write(432 from 432) -> (368) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
@@ -510,7 +510,7 @@
   readline(65537) -> (30) GET /?cmd=getbundle HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
-  readline(-1) -> (422) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
+  readline(-1) -> (439) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apartial-pull%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
   readline(-1) -> (48) x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$\r\n
   readline(-1) -> (35) accept: application/mercurial-0.1\r\n
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
@@ -529,7 +529,7 @@
 
 Server sends partial compression string
 
-  $ hg serve --config badserver.closeaftersendbytes=969 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=984 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -547,13 +547,13 @@
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
   readline(-1) -> (49) user-agent: mercurial/proto-1.0 (Mercurial 4.2)\r\n
   readline(-1) -> (2) \r\n
-  write(36 from 36) -> (933) HTTP/1.1 200 Script output follows\r\n
-  write(23 from 23) -> (910) Server: badhttpserver\r\n
-  write(37 from 37) -> (873) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
-  write(41 from 41) -> (832) Content-Type: application/mercurial-0.1\r\n
-  write(21 from 21) -> (811) Content-Length: 417\r\n
-  write(2 from 2) -> (809) \r\n
-  write(417 from 417) -> (392) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
bundle2=HG20%0Abookmarks%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
 unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=none
+  write(36 from 36) -> (948) HTTP/1.1 200 Script output follows\r\n
+  write(23 from 23) -> (925) Server: badhttpserver\r\n
+  write(37 from 37) -> (888) Date: Fri, 14 Apr 2017 00:00:00 GMT\r\n
+  write(41 from 41) -> (847) Content-Type: application/mercurial-0.1\r\n
+  write(21 from 21) -> (826) Content-Length: 432\r\n
+  write(2 from 2) -> (824) \r\n
+  write(432 from 432) -> (392) lookup changegroupsubset branchmap pushkey 
known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_NO_PHASES_CAPS$ unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx compression=none
   readline(65537) -> (26) GET /?cmd=batch HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
@@ -573,7 +573,7 @@
   readline(65537) -> (30) GET /?cmd=getbundle HTTP/1.1\r\n
   readline(-1) -> (27) Accept-Encoding: identity\r\n
   readline(-1) -> (29) vary: X-HgArg-1,X-HgProto-1\r\n
-  readline(-1) -> (422) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
+  readline(-1) -> (439) x-hgarg-1: 
bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apartial-pull%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=96ee1d7354c4ad7372047672c36a1f561e3a6a4c&listkeys=phases%2Cbookmarks\r\n
   readline(-1) -> (48) x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$\r\n
   readline(-1) -> (35) accept: application/mercurial-0.1\r\n
   readline(-1) -> (2?) host: localhost:$HGPORT\r\n (glob)
@@ -595,7 +595,7 @@
 
 Server sends partial bundle2 header magic
 
-  $ hg serve --config badserver.closeaftersendbytes=966 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=981 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -619,7 +619,7 @@
 
 Server sends incomplete bundle2 stream params length
 
-  $ hg serve --config badserver.closeaftersendbytes=975 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=990 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -644,7 +644,7 @@
 
 Servers stops after bundle2 stream params header
 
-  $ hg serve --config badserver.closeaftersendbytes=978 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=993 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -669,7 +669,7 @@
 
 Server stops sending after bundle2 part header length
 
-  $ hg serve --config badserver.closeaftersendbytes=987 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1002 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -695,7 +695,7 @@
 
 Server stops sending after bundle2 part header
 
-  $ hg serve --config badserver.closeaftersendbytes=1034 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1049 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -725,7 +725,7 @@
 
 Server stops after bundle2 part payload chunk size
 
-  $ hg serve --config badserver.closeaftersendbytes=1055 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1070 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -756,7 +756,7 @@
 
 Server stops sending in middle of bundle2 payload chunk
 
-  $ hg serve --config badserver.closeaftersendbytes=1516 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1531 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -788,7 +788,7 @@
 
 Server stops sending after 0 length payload chunk size
 
-  $ hg serve --config badserver.closeaftersendbytes=1547 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1562 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -825,7 +825,7 @@
 Server stops sending after 0 part bundle part header (indicating end of 
bundle2 payload)
 This is before the 0 size chunked transfer part that signals end of HTTP 
response.
 
-  $ hg serve --config badserver.closeaftersendbytes=1722 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1737 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
@@ -869,7 +869,7 @@
 
 Server sends a size 0 chunked-transfer size without terminating \r\n
 
-  $ hg serve --config badserver.closeaftersendbytes=1725 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
+  $ hg serve --config badserver.closeaftersendbytes=1740 -p $HGPORT -d 
--pid-file=hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
   $ hg clone http://localhost:$HGPORT/ clone
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -1487,6 +1487,7 @@
    bookmarks create a new bookmark or list existing bookmarks
    clone     make a copy of an existing repository
    paths     show aliases for remote repositories
+   pull      pull changes from the specified source
    update    update working directory (or switch revisions)
   
   Extensions:
diff --git a/tests/test-debugcommands.t b/tests/test-debugcommands.t
--- a/tests/test-debugcommands.t
+++ b/tests/test-debugcommands.t
@@ -373,6 +373,7 @@
       pushkey
     hgtagsfnodes
     listkeys
+    partial-pull
     phases
       heads
     pushkey
diff --git a/tests/test-bookmarks-pushpull.t b/tests/test-bookmarks-pushpull.t
--- a/tests/test-bookmarks-pushpull.t
+++ b/tests/test-bookmarks-pushpull.t
@@ -129,10 +129,10 @@
   bundle2-output: bundle parameter: 
   bundle2-output: start of parts
   bundle2-output: bundle part: "replycaps"
-  bundle2-output-part: "replycaps" 195 bytes payload
+  bundle2-output-part: "replycaps" 208 bytes payload
   bundle2-output: part 0: "REPLYCAPS"
   bundle2-output: header chunk size: 16
-  bundle2-output: payload chunk size: 195
+  bundle2-output: payload chunk size: 208
   bundle2-output: closing payload chunk
   bundle2-output: bundle part: "check:bookmarks"
   bundle2-output-part: "check:bookmarks" 23 bytes payload
@@ -162,9 +162,9 @@
   bundle2-input: part parameters: 0
   bundle2-input: found a handler for part replycaps
   bundle2-input-part: "replycaps" supported
-  bundle2-input: payload chunk size: 195
+  bundle2-input: payload chunk size: 208
   bundle2-input: payload chunk size: 0
-  bundle2-input-part: total payload size 195
+  bundle2-input-part: total payload size 208
   bundle2-input: part header size: 22
   bundle2-input: part type: "CHECK:BOOKMARKS"
   bundle2-input: part id: "1"
@@ -241,10 +241,10 @@
   bundle2-output: bundle parameter: 
   bundle2-output: start of parts
   bundle2-output: bundle part: "replycaps"
-  bundle2-output-part: "replycaps" 195 bytes payload
+  bundle2-output-part: "replycaps" 208 bytes payload
   bundle2-output: part 0: "REPLYCAPS"
   bundle2-output: header chunk size: 16
-  bundle2-output: payload chunk size: 195
+  bundle2-output: payload chunk size: 208
   bundle2-output: closing payload chunk
   bundle2-output: bundle part: "check:bookmarks"
   bundle2-output-part: "check:bookmarks" 23 bytes payload
@@ -275,9 +275,9 @@
   bundle2-input: part parameters: 0
   bundle2-input: found a handler for part replycaps
   bundle2-input-part: "replycaps" supported
-  bundle2-input: payload chunk size: 195
+  bundle2-input: payload chunk size: 208
   bundle2-input: payload chunk size: 0
-  bundle2-input-part: total payload size 195
+  bundle2-input-part: total payload size 208
   bundle2-input: part header size: 22
   bundle2-input: part type: "CHECK:BOOKMARKS"
   bundle2-input: part id: "1"
diff --git a/tests/test-acl.t b/tests/test-acl.t
--- a/tests/test-acl.t
+++ b/tests/test-acl.t
@@ -93,14 +93,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -156,14 +156,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -222,14 +222,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -298,14 +298,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -366,14 +366,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -439,14 +439,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -509,14 +509,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -584,14 +584,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -656,14 +656,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -730,14 +730,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -813,14 +813,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -894,14 +894,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -970,14 +970,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1057,14 +1057,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1143,14 +1143,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1225,14 +1225,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1304,14 +1304,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1387,14 +1387,14 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 24 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 24 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 24
   bundle2-input-part: "check:heads" supported
@@ -1507,14 +1507,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1591,14 +1591,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1668,14 +1668,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1741,14 +1741,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1808,14 +1808,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1897,14 +1897,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -1985,14 +1985,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -2057,14 +2057,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
@@ -2139,14 +2139,14 @@
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 178 bytes payload
+  bundle2-output-part: "replycaps" 191 bytes payload
   bundle2-output-part: "check:phases" 48 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "phase-heads" 48 bytes payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 178
+  bundle2-input-part: total payload size 191
   bundle2-input-part: "check:phases" supported
   bundle2-input-part: total payload size 48
   bundle2-input-part: "check:heads" supported
diff --git a/tests/common-pattern.py b/tests/common-pattern.py
--- a/tests/common-pattern.py
+++ b/tests/common-pattern.py
@@ -16,6 +16,7 @@
      br'error%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250A'
      br'hgtagsfnodes%250A'
      br'listkeys%250A'
+     br'partial-pull%250A'
      br'phases%253Dheads%250A'
      br'pushkey%250A'
      br'remote-changegroup%253Dhttp%252Chttps',
@@ -30,12 +31,26 @@
      br'error%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0A'
      br'hgtagsfnodes%0A'
      br'listkeys%0A'
+     br'partial-pull%0A'
      br'phases%3Dheads%0A'
      br'pushkey%0A'
      br'remote-changegroup%3Dhttp%2Chttps',
      # (replacement patterns)
      br'$USUAL_BUNDLE2_CAPS$'
     ),
+    (br'bundle2=HG20%0A'
+     br'bookmarks%0A'
+     br'changegroup%3D01%2C02%0A'
+     br'digests%3Dmd5%2Csha1%2Csha512%0A'
+     br'error%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0A'
+     br'hgtagsfnodes%0A'
+     br'listkeys%0A'
+     br'partial-pull%0A'
+     br'pushkey%0A'
+     br'remote-changegroup%3Dhttp%2Chttps',
+     # (replacement patterns)
+     br'$USUAL_BUNDLE2_NO_PHASES_CAPS$'
+    ),
     # HTTP log dates
     (br' - - \[\d\d/.../2\d\d\d \d\d:\d\d:\d\d] "GET',
      br' - - [$LOGDATE$] "GET'
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -831,6 +831,65 @@
     opts = options('debugwireargs', ['three', 'four'], others)
     return repo.debugwireargs(one, two, **pycompat.strkwargs(opts))
 
+def find_pullbundle(repo, opts, clheads, heads, common):
+    """Return a file object for the first matching pullbundle.
+
+    Pullbundles are specified in .hg/pullbundles.manifest similar to
+    clonebundles.
+    For each entry, the bundle specification is checked for compatibility:
+    - Client features vs the BUNDLESPEC.
+    - Revisions shared with the clients vs base revisions of the bundle.
+      A bundle can be applied only if all its base revisions are known by
+      the client.
+    - At least one leaf of the bundle's DAG is missing on the client.
+    - Every leaf of the bundle's DAG is part of node set the client wants.
+      E.g. do not send a bundle of all changes if the client wants only
+      one specific branch of many.
+    """
+    def decodehexstring(s):
+        return set([h.decode('hex') for h in s.split(';')])
+
+    manifest = repo.vfs.tryread('pullbundles.manifest')
+    if not manifest:
+        return None
+    res = exchange.parseclonebundlesmanifest(repo, manifest)
+    res = exchange.filterclonebundleentries(repo, res)
+    if not res:
+        return None
+    cl = repo.changelog
+    heads_anc = cl.ancestors([cl.rev(rev) for rev in heads], inclusive=True)
+    common_anc = cl.ancestors([cl.rev(rev) for rev in common], inclusive=True)
+    for entry in res:
+        if 'heads' in entry:
+            try:
+                bundle_heads = decodehexstring(entry['heads'])
+            except TypeError:
+                # Bad heads entry
+                continue
+            if bundle_heads.issubset(common):
+                continue # Nothing new
+            if all(cl.rev(rev) in common_anc for rev in bundle_heads):
+                continue # Still nothing new
+            if any(cl.rev(rev) not in heads_anc and
+                   cl.rev(rev) not in common_anc for rev in bundle_heads):
+                continue
+        if 'bases' in entry:
+            try:
+                bundle_bases = decodehexstring(entry['bases'])
+            except TypeError:
+                # Bad bases entry
+                continue
+            if not all(cl.rev(rev) in common_anc for rev in bundle_bases):
+                continue
+        path = entry['URL']
+        repo.ui.debug('sending pullbundle "%s"\n' % path)
+        try:
+            return repo.vfs.open(path)
+        except IOError:
+            repo.ui.debug('pullbundle "%s" not accessible\n' % path)
+            continue
+    return None
+
 @wireprotocommand('getbundle', '*')
 def getbundle(repo, proto, others):
     opts = options('getbundle', gboptsmap.keys(), others)
@@ -861,12 +920,26 @@
                               hint=bundle2requiredhint)
 
     try:
+        clheads = set(repo.changelog.heads())
+        heads = set(opts.get('heads', set()))
+        common = set(opts.get('common', set()))
+        common.discard(nullid)
+        b2caps = {}
+        for bcaps in opts.get('bundlecaps', []):
+            if bcaps.startswith('bundle2='):
+                blob = urlreq.unquote(bcaps[len('bundle2='):])
+                b2caps.update(bundle2.decodecaps(blob))
+
+        if (repo.ui.configbool('server', 'pullbundle') and
+            'partial-pull' in b2caps):
+            # Check if a pre-built bundle covers this request.
+            bundle = find_pullbundle(repo, opts, clheads, heads, common)
+            if bundle:
+                return streamres(gen=util.filechunkiter(bundle),
+                                 prefer_uncompressed=True)
+
         if repo.ui.configbool('server', 'disablefullbundle'):
             # Check to see if this is a full clone.
-            clheads = set(repo.changelog.heads())
-            heads = set(opts.get('heads', set()))
-            common = set(opts.get('common', set()))
-            common.discard(nullid)
             if not common and clheads == heads:
                 raise error.Abort(
                     _('server has pull-based clones disabled'),
diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -1772,6 +1772,14 @@
     are highly recommended. Partial clones will still be allowed.
     (default: False)
 
+``pullbundle``
+    When set, the server will check pullbundle.manifest for bundles
+    covering the requested heads and common nodes. The first matching
+    entry will be streamed to the client.
+
+    For HTTP transport, the stream will still use zlib compression
+    for older clients.
+
 ``concurrent-push-mode``
     Level of allowed race condition between two pushing clients.
 
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1353,7 +1353,36 @@
         streamclone.maybeperformlegacystreamclone(pullop)
         _pulldiscovery(pullop)
         if pullop.canusebundle2:
-            _pullbundle2(pullop)
+            # The server may send a partial reply, i.e. when inlining
+            # pre-computed bundles. In that case, update the common
+            # set based on the results and pull another bundle.
+            #
+            # There are two indicators that the process is finished:
+            # - no changeset has been added, or
+            # - all remote heads are known locally.
+            # The head check must use the unfiltered view as obsoletion
+            # markers can hide heads.
+            unfi = repo.unfiltered()
+            unficl = unfi.changelog
+            def headsofdiff(h1, h2):
+                """Returns heads(h1 % h2)"""
+                res = unfi.set('heads(%ln %% %ln)', h1, h2)
+                return set(ctx.node() for ctx in res)
+            def headsofunion(h1, h2):
+                """Returns heads((h1 + h2) - null)"""
+                res = unfi.set('heads((%ln + %ln - null))', h1, h2)
+                return set(ctx.node() for ctx in res)
+            while True:
+                old_heads = unficl.heads()
+                clstart = len(unficl)
+                _pullbundle2(pullop)
+                if clstart == len(unficl):
+                    break
+                if all(unficl.hasnode(n) for n in pullop.rheads):
+                    break
+                new_heads = headsofdiff(unficl.heads(), old_heads)
+                pullop.common = headsofunion(new_heads, pullop.common)
+                pullop.rheads = set(pullop.rheads) - pullop.common
         _pullchangeset(pullop)
         _pullphase(pullop)
         _pullbookmarks(pullop)
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -886,6 +886,9 @@
 coreconfigitem('server', 'disablefullbundle',
     default=False,
 )
+coreconfigitem('server', 'pullbundle',
+    default=False,
+)
 coreconfigitem('server', 'maxhttpheaderlen',
     default=1024,
 )
diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -1389,12 +1389,13 @@
       i) tip
 
       When cloning from servers that support it, Mercurial may fetch
-      pre-generated data from a server-advertised URL. When this is done,
-      hooks operating on incoming changesets and changegroups may fire twice,
-      once for the bundle fetched from the URL and another for any additional
-      data not fetched from this URL. In addition, if an error occurs, the
-      repository may be rolled back to a partial clone. This behavior may
-      change in future releases. See :hg:`help -e clonebundles` for more.
+      pre-generated data from a server-advertised URL or inline from the
+      same stream. When this is done, hooks operating on incoming changesets
+      and changegroups may fire more than once, once for each pre-generated
+      bundle and as well as for any additional remaining data. In addition,
+      if an error occurs, the repository may be rolled back to a partial
+      clone. This behavior may change in future releases.
+      See :hg:`help -e clonebundles` for more.
 
       Examples:
 
@@ -3953,6 +3954,12 @@
     -R is specified). By default, this does not update the copy of the
     project in the working directory.
 
+    When cloning from servers that support it, Mercurial may fetch
+    pre-generated data. When this is done, hooks operating on incoming
+    changesets and changegroups may fire more than once, once for each
+    pre-generated bundle and as well as for any additional remaining
+    data. See :hg:`help -e clonebundles` for more.
+
     Use :hg:`incoming` if you want to see what would have been added
     by a pull at the time you issued this command. If you then decide
     to add those changes to the repository, you should use :hg:`pull
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1486,6 +1486,7 @@
                 'remote-changegroup': ('http', 'https'),
                 'hgtagsfnodes': (),
                 'phases': ('heads',),
+                'partial-pull': (),
                }
 
 def getrepocaps(repo, allowpushback=False):
diff --git a/hgext/clonebundles.py b/hgext/clonebundles.py
--- a/hgext/clonebundles.py
+++ b/hgext/clonebundles.py
@@ -6,7 +6,8 @@
 "clonebundles" is a server-side extension used to advertise the existence
 of pre-generated, externally hosted bundle files to clients that are
 cloning so that cloning can be faster, more reliable, and require less
-resources on the server.
+resources on the server. "pullbundles" is a related feature for sending
+pre-generated bundle files to clients as part of pull operations.
 
 Cloning can be a CPU and I/O intensive operation on servers. Traditionally,
 the server, in response to a client's request to clone, dynamically generates
@@ -16,8 +17,12 @@
 servers with large repositories or with high clone volume, the load from
 clones can make scaling the server challenging and costly.
 
-This extension provides server operators the ability to offload potentially
-expensive clone load to an external service. Here's how it works.
+This extension provides server operators the ability to offload
+potentially expensive clone load to an external service. Pre-generated
+bundles also allow using more CPU intensive compression, reducing the
+effective bandwidth requirements.
+
+Here's how clone bundles work:
 
 1. A server operator establishes a mechanism for making bundle files available
    on a hosting service where Mercurial clients can fetch them.
@@ -33,22 +38,36 @@
 7. The client reconnects to the original server and performs the equivalent
    of :hg:`pull` to retrieve all repository data not in the bundle. (The
    repository could have been updated between when the bundle was created
-   and when the client started the clone.)
+   and when the client started the clone.) This may use "pullbundles".
 
 Instead of the server generating full repository bundles for every clone
 request, it generates full bundles once and they are subsequently reused to
 bootstrap new clones. The server may still transfer data at clone time.
 However, this is only data that has been added/changed since the bundle was
 created. For large, established repositories, this can reduce server load for
 clones to less than 1% of original.
 
+Here's how pullbundles work:
+
+1. A manifest file listing available bundles and describing the revisions
+   is added to the Mercurial repository on the server.
+2. A new-enough client informs the server that it supports partial pulls
+   and initiates a pull.
+3. If the server has pull bundles enabled and sees the client advertising
+   partial pulls, it checks for a matching pull bundle in the manifest.
+   A bundle matches if the format is supported by the client, the client
+   has the required revisions already and needs something from the bundle.
+4. If there is at least one matching bundle, the server sends it to the client.
+5. The client applies the bundle and notices that the server reply was
+   incomplete. It initiates another pull.
+
 To work, this extension requires the following of server operators:
 
 * Generating bundle files of repository content (typically periodically,
   such as once per day).
-* A file server that clients have network access to and that Python knows
-  how to talk to through its normal URL handling facility (typically an
-  HTTP server).
+* Clone bundles: A file server that clients have network access to and that
+  Python knows how to talk to through its normal URL handling facility
+  (typically an HTTP/HTTPS server).
 * A process for keeping the bundles manifest in sync with available bundle
   files.
 
@@ -61,7 +80,7 @@
 :hg:`bundle --all` is used to produce a bundle of the entire repository.
 
 :hg:`debugcreatestreamclonebundle` can be used to produce a special
-*streaming clone bundle*. These are bundle files that are extremely efficient
+*streaming clonebundle*. These are bundle files that are extremely efficient
 to produce and consume (read: fast). However, they are larger than
 traditional bundle formats and require that clients support the exact set
 of repository data store formats in use by the repository that created them.
@@ -73,7 +92,8 @@
 A server operator is responsible for creating a ``.hg/clonebundles.manifest``
 file containing the list of available bundle files suitable for seeding
 clones. If this file does not exist, the repository will not advertise the
-existence of clone bundles when clients connect.
+existence of clone bundles when clients connect. For pull bundles,
+``.hg/pullbundles.manifest`` is used.
 
 The manifest file contains a newline (\\n) delimited list of entries.
 
@@ -85,6 +105,9 @@
 pairs describing additional properties of this bundle. Both keys and values
 are URI encoded.
 
+For pull bundles, the URL is a path under the ``.hg`` directory of the
+repository.
+
 Keys in UPPERCASE are reserved for use by Mercurial and are defined below.
 All non-uppercase keys can be used by site installations. An example use
 for custom properties is to use the *datacenter* attribute to define which
@@ -133,6 +156,15 @@
 
    Value should be "true".
 
+heads
+   Used for pull bundles. This contains the ``;`` separated changeset
+   hashes of the heads of the bundle content.
+
+bases
+   Used for pull bundles. This contains the ``;`` separated changeset
+   hashes of the roots of the bundle content. This can be skipped if
+   the bundle was created without ``--base``.
+
 Manifests can contain multiple entries. Assuming metadata is defined, clients
 will filter entries from the manifest that they don't support. The remaining
 entries are optionally sorted by client preferences



To: joerg.sonnenberger, #hg-reviewers, indygreg
Cc: indygreg, durin42, mercurial-devel
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Reply via email to