D5130: lfs: autoload the extension when cloning from repo with lfs enabled
mharbison72 created this revision.
mharbison72 added a reviewer: indygreg.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This is based on a patch by Gregory Szorc. I made small adjustments to
clean up the messaging when the server has the extension enabled, but the
client has it disabled (to prevent autoloading). Additionally, I added
a second server capability to distinguish between the server having the
extension enabled, and the server having LFS commits. This helps prevent
unnecessary requirement propagation- the client shouldn't add a requirement
that the server doesn't have, just because the server had the extension
loaded. The TODO I had about advertising a capability when the server can
natively serve up blobs isn't relevant anymore (we've had 2 releases that
support this), so I dropped it.
Currently, we lazily add the "lfs" requirement to a repo when we first
encounter LFS data. Due to a pretxnchangegroup hook that looks for LFS
data, this can happen at the end of clone.
Now that we have more control over how repositories are created, we can
do better.
This commit adds a repo creation option to add the "lfs" requirement.
hg.clone() sets this creation option if the remote peer is advertising
lfs usage (as opposed to just support needed to push).
So, what this change effectively does is have cloned repos
automatically inherit the "lfs" requirement.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5130
AFFECTED FILES
hgext/lfs/wrapper.py
mercurial/hg.py
mercurial/localrepo.py
tests/test-lfs-serve-access.t
tests/test-lfs-serve.t
CHANGE DETAILS
diff --git a/tests/test-lfs-serve.t b/tests/test-lfs-serve.t
--- a/tests/test-lfs-serve.t
+++ b/tests/test-lfs-serve.t
@@ -304,10 +304,10 @@
$ grep 'lfs' .hg/requires $SERVER_REQUIRES
$TESTTMP/server/.hg/requires:lfs
-TODO: fail more gracefully.
-
- $ hg clone -q http://localhost:$HGPORT $TESTTMP/client4_clone
- abort: HTTP Error 500: Internal Server Error
+ $ hg clone http://localhost:$HGPORT $TESTTMP/client4_clone
+ (remote is using large file support (lfs), but it is explicitly disabled in
the local configuration)
+ abort: repository requires features unknown to this Mercurial: lfs!
+ (see https://mercurial-scm.org/wiki/MissingRequirement for more information)
[255]
$ grep 'lfs' $TESTTMP/client4_clone/.hg/requires $SERVER_REQUIRES
grep: $TESTTMP/client4_clone/.hg/requires: $ENOENT$
@@ -661,8 +661,6 @@
$ cat $TESTTMP/errors.log | grep '^[A-Z]'
Traceback (most recent call last):
ValueError: no common changegroup version
- Traceback (most recent call last):
- ValueError: no common changegroup version
#else
$ cat $TESTTMP/errors.log
#endif
diff --git a/tests/test-lfs-serve-access.t b/tests/test-lfs-serve-access.t
--- a/tests/test-lfs-serve-access.t
+++ b/tests/test-lfs-serve-access.t
@@ -42,6 +42,7 @@
Downloads fail...
$ hg clone http://localhost:$HGPORT httpclone
+ (remote is using large file support (lfs); lfs will be enabled for this
repository)
requesting all changes
adding changesets
adding manifests
@@ -76,6 +77,7 @@
$ hg clone --debug http://localhost:$HGPORT/subdir/mount/point cloned2
using http://localhost:$HGPORT/subdir/mount/point
sending capabilities command
+ (remote is using large file support (lfs); lfs will be enabled for this
repository)
query 1; heads
sending batch command
requesting all changes
@@ -88,7 +90,6 @@
adding file changes
adding lfs.bin revisions
added 1 changesets with 1 changes to 1 files
- calling hook pretxnchangegroup.lfs: hgext.lfs.checkrequireslfs
bundle2-input-part: total payload size 648
bundle2-input-part: "listkeys" (params: 1 mandatory) supported
bundle2-input-part: "phase-heads" supported
@@ -239,6 +240,7 @@
Test an I/O error in localstore.verify() (Batch API) with GET
$ hg clone http://localhost:$HGPORT1 httpclone2
+ (remote is using large file support (lfs); lfs will be enabled for this
repository)
requesting all changes
adding changesets
adding manifests
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -2895,6 +2895,9 @@
if createopts.get('narrowfiles'):
requirements.add(repository.NARROW_REQUIREMENT)
+if createopts.get('lfs'):
+requirements.add('lfs')
+
return requirements
def filterknowncreateopts(ui, createopts):
@@ -2912,6 +2915,7 @@
they know how to handle.
"""
known = {
+'lfs',
'backend',
'narrowfiles',
'sharedrepo',
@@ -2931,6 +2935,9 @@
backend
The storage backend to use.
+lfs
+ Repository will be created with ``lfs`` requirement. The lfs extension
+ will automatically be loaded when the repository is accessed.
narrowfiles
Set up repository to
D5110: tests: test-parseindex.t requires no-pure
adgar added a comment. I had been seeing test failures, but I suspect it is due to a misconfigured build and not an actual bug. Withdrawing this change. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5110 To: adgar, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5121: procutil: port over windows encoding fixes from logtoprocess
mharbison72 accepted this revision. mharbison72 added a comment. LGTM, thanks. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5121 To: durin42, #hg-reviewers, mharbison72 Cc: mharbison72, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5125: WIP: rename requirements and wireproto methods
durin42 abandoned this revision. durin42 added a comment. Replaced by a more principled collection of refactors that has the benefit of working. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5125 To: durin42, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5128: remotefilelog: consolidate and rename on-disk store requirement
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
The value of this constant appeared in too many places. While we're
here, rename it to be more consistent with our naming conventions for
experimental functionality.
https://phab.mercurial-scm.org/D5125
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5128
AFFECTED FILES
hgext/remotefilelog/__init__.py
hgext/remotefilelog/constants.py
hgext/remotefilelog/debugcommands.py
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/remotefilelogserver.py
hgext/remotefilelog/shallowbundle.py
hgext/remotefilelog/shallowrepo.py
tests/test-remotefilelog-clone-tree.t
tests/test-remotefilelog-clone.t
tests/test-remotefilelog-log.t
CHANGE DETAILS
diff --git a/tests/test-remotefilelog-log.t b/tests/test-remotefilelog-log.t
--- a/tests/test-remotefilelog-log.t
+++ b/tests/test-remotefilelog-log.t
@@ -28,9 +28,9 @@
$ cd shallow
$ cat .hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
diff --git a/tests/test-remotefilelog-clone.t b/tests/test-remotefilelog-clone.t
--- a/tests/test-remotefilelog-clone.t
+++ b/tests/test-remotefilelog-clone.t
@@ -25,9 +25,9 @@
$ cd shallow
$ cat .hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
@@ -58,9 +58,9 @@
$ cd shallow2
$ cat .hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
$ ls .hg/store/data
@@ -106,8 +106,8 @@
$ ls shallow3/.hg/store/data
$ cat shallow3/.hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
diff --git a/tests/test-remotefilelog-clone-tree.t
b/tests/test-remotefilelog-clone-tree.t
--- a/tests/test-remotefilelog-clone-tree.t
+++ b/tests/test-remotefilelog-clone-tree.t
@@ -28,9 +28,9 @@
$ cd shallow
$ cat .hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
treemanifest
@@ -68,9 +68,9 @@
$ cd shallow2
$ cat .hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
treemanifest
@@ -109,9 +109,9 @@
$ ls shallow3/.hg/store/data
$ cat shallow3/.hg/requires
dotencode
+ exp-remotefilelog-repo-req-1
fncache
generaldelta
- remotefilelog
revlogv1
store
treemanifest
diff --git a/hgext/remotefilelog/shallowrepo.py
b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -40,7 +40,6 @@
from mercurial.utils import procutil
_hgexecutable = procutil.hgexecutable
-requirement = "remotefilelog"
_prefetching = _('prefetching')
# These make*stores functions are global so that other extensions can replace
diff --git a/hgext/remotefilelog/shallowbundle.py
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -26,8 +26,6 @@
LocalFiles = 1
AllFiles = 2
-requirement = "remotefilelog"
-
def shallowgroup(cls, self, nodelist, rlog, lookup, units=None, reorder=None):
if not isinstance(rlog, remotefilelog.remotefilelog):
for c in super(cls, self).group(nodelist, rlog, lookup,
@@ -56,7 +54,7 @@
class shallowcg1packer(changegroup.cgpacker):
def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
-if "remotefilelog" in self._repo.requirements:
+if constants.SHALLOWREPO_REQUIREMENT in self._repo.requirements:
fastpathlinkrev = False
return super(shallowcg1packer, self).generate(commonrevs, clnodes,
@@ -71,7 +69,7 @@
linknodes, commonrevs, source = args
except ValueError:
commonrevs, source, mfdicts, fastpathlinkrev, fnodes, clrevs = args
-if requirement in self._repo.requirements:
+if constants.SHALLOWREPO_REQUIREMENT in self._repo.requirements:
repo = self._repo
if isinstance(repo, bundlerepo.bundlerepository):
# If the bundle contains filelogs, we can't pull from it, since
@@ -93,7 +91,7 @@
def shouldaddfilegroups(self, source):
repo = self._repo
-if not requirement in repo.requirements:
+if not constants.SHALLOWREPO_REQUIREMENT in repo.requirements:
return AllFiles
if source == "push" or source == "bundle":
@@ -141,7 +139,7 @@
yield delta
def makechangegroup(orig, repo, outgoing, version, source, *args, **kwargs):
-if not requirement in repo.requirements:
+if not constants.SHALLOWREPO_REQUIREMENT in repo.requirements:
return orig(repo, outgoing, version, source, *args, **kwargs)
original =
D5129: remotefilelog: rename wireproto methods and remaining capabilities
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This is a mess, in part because there should be more constants
throughout. I know we typically do exp- instead of the x_ business in
this change, but I also had to use this in some function names, so I
figured until I can break that coupling I'd go with this. If it's too
unpleasant during review, let me know and I can probably clean it up
some more.
1. no-check-commit due to new foo_bar naming - too hard to avoid right now :(
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5129
AFFECTED FILES
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/remotefilelogserver.py
tests/remotefilelog-getflogheads.py
tests/test-remotefilelog-http.t
tests/test-remotefilelog-pull-noshallow.t
tests/test-remotefilelog-push-pull.t
CHANGE DETAILS
diff --git a/tests/test-remotefilelog-push-pull.t
b/tests/test-remotefilelog-push-pull.t
--- a/tests/test-remotefilelog-push-pull.t
+++ b/tests/test-remotefilelog-push-pull.t
@@ -24,12 +24,12 @@
$ cd master
$ echo 'hello' | hg -R . serve --stdio | grep capa | identifyrflcaps
exp-remotefilelog-ssh-getfiles-1
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ echo 'capabilities' | hg -R . serve --stdio | identifyrflcaps ; echo
exp-remotefilelog-ssh-getfiles-1
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
# pull to shallow from full
diff --git a/tests/test-remotefilelog-pull-noshallow.t
b/tests/test-remotefilelog-pull-noshallow.t
--- a/tests/test-remotefilelog-pull-noshallow.t
+++ b/tests/test-remotefilelog-pull-noshallow.t
@@ -37,12 +37,12 @@
$ cd master
$ echo 'hello' | hg -R . serve --stdio | grep capa | identifyrflcaps
exp-remotefilelog-ssh-getfiles-1
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ echo 'capabilities' | hg -R . serve --stdio | identifyrflcaps ; echo
exp-remotefilelog-ssh-getfiles-1
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
Pull to the child repository. Use our custom setupremotefilelog extension
diff --git a/tests/test-remotefilelog-http.t b/tests/test-remotefilelog-http.t
--- a/tests/test-remotefilelog-http.t
+++ b/tests/test-remotefilelog-http.t
@@ -16,16 +16,16 @@
Build a query string for later use:
$ GET=`hg debugdata -m 0 | $PYTHON -c \
- > 'import sys ; print [("?cmd=getfile=%s=%s" %
tuple(s.split("\0"))) for s in sys.stdin.read().splitlines()][0]'`
+ > 'import sys ; print [("?cmd=x_rfl_getfile=%s=%s" %
tuple(s.split("\0"))) for s in sys.stdin.read().splitlines()][0]'`
$ cd ..
$ cat hg1.pid >> $DAEMON_PIDS
$ hgcloneshallow http://localhost:$HGPORT/ shallow -q
2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over *s (glob)
$ grep getfile access.log
- * "GET /?cmd=batch HTTP/1.1" 200 -
x-hgarg-1:cmds=getfile+*node%3D1406e74118627694268417491f018a4a883152f0* (glob)
+ * "GET /?cmd=batch HTTP/1.1" 200 -
x-hgarg-1:cmds=x_rfl_getfile+*node%3D1406e74118627694268417491f018a4a883152f0*
(glob)
Clear filenode cache so we can test fetching with a modified batch size
$ rm -r $TESTTMP/hgcache
@@ -37,17 +37,17 @@
The 'remotefilelog' capability should *not* be exported over http(s),
as the getfile method it offers doesn't work with http.
$ get-with-headers.py localhost:$HGPORT '?cmd=capabilities' | grep lookup |
identifyrflcaps
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ get-with-headers.py localhost:$HGPORT '?cmd=hello' | grep lookup |
identifyrflcaps
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ get-with-headers.py localhost:$HGPORT '?cmd=this-command-does-not-exist' |
head -n 1
400 no such method: this-command-does-not-exist
- $ get-with-headers.py localhost:$HGPORT '?cmd=getfiles' | head -n 1
- 400 no such method: getfiles
+ $ get-with-headers.py localhost:$HGPORT '?cmd=x_rfl_getfiles' | head -n 1
+ 400 no such method: x_rfl_getfiles
Verify serving from a shallow clone doesn't allow for remotefile
fetches. This also serves to test the error handling for our batchable
diff --git a/tests/remotefilelog-getflogheads.py
b/tests/remotefilelog-getflogheads.py
--- a/tests/remotefilelog-getflogheads.py
+++ b/tests/remotefilelog-getflogheads.py
@@ -22,7 +22,7 @@
dest = repo.ui.expandpath('default')
peer = hg.peer(repo, {}, dest)
-flogheads = peer.getflogheads(path)
+flogheads = peer.x_rfl_getflogheads(path)
if flogheads:
for head in flogheads:
diff --git a/hgext/remotefilelog/remotefilelogserver.py
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -77,11 +77,11 @@
# support file content requests
wireprotov1server.wireprotocommand(
-'getflogheads', 'path',
D5126: remotefilelog: rename capability for legacy ssh file fetching method
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
I think I want to delete this, but for now we'll just rename this so
it's easy to tell apart from other strings that have the same value.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5126
AFFECTED FILES
hgext/remotefilelog/__init__.py
hgext/remotefilelog/constants.py
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/remotefilelogserver.py
tests/test-remotefilelog-pull-noshallow.t
tests/test-remotefilelog-push-pull.t
CHANGE DETAILS
diff --git a/tests/test-remotefilelog-push-pull.t
b/tests/test-remotefilelog-push-pull.t
--- a/tests/test-remotefilelog-push-pull.t
+++ b/tests/test-remotefilelog-push-pull.t
@@ -23,13 +23,13 @@
$ cd master
$ echo 'hello' | hg -R . serve --stdio | grep capa | identifyrflcaps
+ exp-remotefilelog-ssh-getfiles-1
getfile
getflogheads
- remotefilelog
$ echo 'capabilities' | hg -R . serve --stdio | identifyrflcaps ; echo
+ exp-remotefilelog-ssh-getfiles-1
getfile
getflogheads
- remotefilelog
# pull to shallow from full
diff --git a/tests/test-remotefilelog-pull-noshallow.t
b/tests/test-remotefilelog-pull-noshallow.t
--- a/tests/test-remotefilelog-pull-noshallow.t
+++ b/tests/test-remotefilelog-pull-noshallow.t
@@ -36,13 +36,13 @@
$ cd master
$ echo 'hello' | hg -R . serve --stdio | grep capa | identifyrflcaps
+ exp-remotefilelog-ssh-getfiles-1
getfile
getflogheads
- remotefilelog
$ echo 'capabilities' | hg -R . serve --stdio | identifyrflcaps ; echo
+ exp-remotefilelog-ssh-getfiles-1
getfile
getflogheads
- remotefilelog
Pull to the child repository. Use our custom setupremotefilelog extension
diff --git a/hgext/remotefilelog/remotefilelogserver.py
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -29,6 +29,7 @@
wireprotov1server,
)
from . import (
+constants,
shallowrepo,
shallowutil,
)
@@ -197,7 +198,7 @@
ui.configbool('remotefilelog', 'server'))):
if isinstance(proto, _sshv1server):
# legacy getfiles method which only works over ssh
-caps.append(shallowrepo.requirement)
+caps.append(constants.NETWORK_CAP_LEGACY_SSH_GETFILES)
caps.append('getflogheads')
caps.append('getfile')
return caps
diff --git a/hgext/remotefilelog/fileserverclient.py
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -78,7 +78,8 @@
def _updatecallstreamopts(self, command, opts):
if command != 'getbundle':
return
-if 'remotefilelog' not in self.capabilities():
+if (constants.NETWORK_CAP_LEGACY_SSH_GETFILES
+not in self.capabilities()):
return
if not util.safehasattr(self, '_localrepo'):
return
@@ -377,9 +378,8 @@
try:
with self._connect() as conn:
remote = conn.peer
-# TODO: deduplicate this with the constant in
-# shallowrepo
-if remote.capable("remotefilelog"):
+if remote.capable(
+constants.NETWORK_CAP_LEGACY_SSH_GETFILES):
if not isinstance(remote, _sshv1peer):
raise error.Abort('remotefilelog requires ssh '
'servers')
diff --git a/hgext/remotefilelog/constants.py b/hgext/remotefilelog/constants.py
--- a/hgext/remotefilelog/constants.py
+++ b/hgext/remotefilelog/constants.py
@@ -6,6 +6,8 @@
REQUIREMENT = "remotefilelog"
+NETWORK_CAP_LEGACY_SSH_GETFILES = 'exp-remotefilelog-ssh-getfiles-1'
+
FILENAMESTRUCT = '!H'
FILENAMESIZE = struct.calcsize(FILENAMESTRUCT)
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -114,6 +114,7 @@
util,
)
from . import (
+constants,
debugcommands,
fileserverclient,
remotefilectx,
@@ -282,7 +283,7 @@
# patterns.
def stream_out_shallow(orig):
caps = remote.capabilities()
-if shallowrepo.requirement in caps:
+if constants.NETWORK_CAP_LEGACY_SSH_GETFILES in caps:
opts = {}
if repo.includepattern:
opts['includepattern'] = '\0'.join(repo.includepattern)
To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
D5127: remotefilelog: consolidate and rename bundle2 capability
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
Bonus: we catch a spot where the shallowrepo requirement was being
misused as the bundle2 capability.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5127
AFFECTED FILES
hgext/remotefilelog/__init__.py
hgext/remotefilelog/constants.py
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/remotefilelogserver.py
hgext/remotefilelog/shallowbundle.py
CHANGE DETAILS
diff --git a/hgext/remotefilelog/shallowbundle.py
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -17,6 +17,7 @@
pycompat,
)
from . import (
+constants,
remotefilelog,
shallowutil,
)
@@ -100,7 +101,7 @@
caps = self._bundlecaps or []
if source == "serve" or source == "pull":
-if 'remotefilelog' in caps:
+if constants.BUNDLE2_CAPABLITY in caps:
return LocalFiles
else:
# Serving to a full repo requires us to serve everything
diff --git a/hgext/remotefilelog/remotefilelogserver.py
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -45,7 +45,7 @@
def generatefiles(orig, self, changedfiles, linknodes, commonrevs, source,
*args, **kwargs):
caps = self._bundlecaps or []
-if shallowrepo.requirement in caps:
+if constants.BUNDLE2_CAPABLITY in caps:
# only send files that don't match the specified patterns
includepattern = None
excludepattern = None
@@ -184,7 +184,7 @@
def getbundleshallow(repo, proto, others):
bundlecaps = others.get('bundlecaps', '')
bundlecaps = set(bundlecaps.split(','))
-bundlecaps.add('remotefilelog')
+bundlecaps.add(constants.BUNDLE2_CAPABLITY)
others['bundlecaps'] = ','.join(bundlecaps)
return wireprotov1server.commands["getbundle"][0](repo, proto, others)
diff --git a/hgext/remotefilelog/fileserverclient.py
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -99,7 +99,7 @@
# getbundle args before it goes across the wire. Once we get rid
# of bundle1, we can use bundle2's _pullbundle2extraprepare to
# do this more cleanly.
-bundlecaps.append('remotefilelog')
+bundlecaps.append(constants.BUNDLE2_CAPABLITY)
if self._localrepo.includepattern:
patterns = '\0'.join(self._localrepo.includepattern)
includecap = "includepattern=" + patterns
diff --git a/hgext/remotefilelog/constants.py b/hgext/remotefilelog/constants.py
--- a/hgext/remotefilelog/constants.py
+++ b/hgext/remotefilelog/constants.py
@@ -8,6 +8,8 @@
NETWORK_CAP_LEGACY_SSH_GETFILES = 'exp-remotefilelog-ssh-getfiles-1'
+BUNDLE2_CAPABLITY = "exp-remotefilelog-b2cap-1"
+
FILENAMESTRUCT = '!H'
FILENAMESIZE = struct.calcsize(FILENAMESTRUCT)
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -962,7 +962,7 @@
**kwargs):
if not bundlecaps:
bundlecaps = set()
-bundlecaps.add('remotefilelog')
+bundlecaps.add(constants.BUNDLE2_CAPABLITY)
return orig(source, heads=heads, common=common, bundlecaps=bundlecaps,
**kwargs)
To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4895: narrow: don't include manifests the client already has
martinvonz updated this revision to Diff 12184.
martinvonz marked an inline comment as done.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4895?vs=12175=12184
REVISION DETAIL
https://phab.mercurial-scm.org/D4895
AFFECTED FILES
hgext/narrow/narrowbundle2.py
hgext/narrow/narrowwirepeer.py
mercurial/bundle2.py
mercurial/changegroup.py
mercurial/exchange.py
CHANGE DETAILS
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -2145,14 +2145,12 @@
if kwargs.get(r'narrow', False):
include = sorted(filter(bool, kwargs.get(r'includepats', [])))
exclude = sorted(filter(bool, kwargs.get(r'excludepats', [])))
-filematcher = narrowspec.match(repo.root, include=include,
- exclude=exclude)
+matcher = narrowspec.match(repo.root, include=include, exclude=exclude)
else:
-filematcher = None
+matcher = None
cgstream = changegroup.makestream(repo, outgoing, version, source,
- bundlecaps=bundlecaps,
- filematcher=filematcher)
+ bundlecaps=bundlecaps, matcher=matcher)
part = bundler.newpart('changegroup', data=cgstream)
if cgversions:
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -727,14 +727,17 @@
progress.complete()
class cgpacker(object):
-def __init__(self, repo, filematcher, version,
+def __init__(self, repo, oldmatcher, matcher, version,
builddeltaheader, manifestsend,
forcedeltaparentprev=False,
bundlecaps=None, ellipses=False,
shallow=False, ellipsisroots=None, fullnodes=None):
"""Given a source repo, construct a bundler.
-filematcher is a matcher that matches on files to include in the
+oldmatcher is a matcher that matches on files the client already has.
+These will not be included in the changegroup.
+
+matcher is a matcher that matches on files to include in the
changegroup. Used to facilitate sparse changegroups.
forcedeltaparentprev indicates whether delta parents must be against
@@ -761,8 +764,10 @@
ellipsis because for very large histories we expect this to be
significantly smaller.
"""
-assert filematcher
-self._filematcher = filematcher
+assert oldmatcher
+assert matcher
+self._oldmatcher = oldmatcher
+self._matcher = matcher
self.version = version
self._forcedeltaparentprev = forcedeltaparentprev
@@ -1027,7 +1032,7 @@
tree, nodes = tmfnodes.popitem()
store = mfl.getstorage(tree)
-if not self._filematcher.visitdir(store.tree[:-1] or '.'):
+if not self._matcher.visitdir(store.tree[:-1] or '.'):
# No nodes to send because this directory is out of
# the client's view of the repository (probably
# because of narrow clones).
@@ -1051,7 +1056,16 @@
fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis)
-yield tree, deltas
+if not self._oldmatcher.visitdir(store.tree[:-1] or '.'):
+yield tree, deltas
+else:
+# 'deltas' is a generator and we need to consume it even if
+# we are not going to send it because a side-effect is that
+# it updates tmdnodes (via lookupfn)
+for d in deltas:
+pass
+if not tree:
+yield tree, []
def _prunemanifests(self, store, nodes, commonrevs):
# This is split out as a separate method to allow filtering
@@ -1066,7 +1080,8 @@
# The 'source' parameter is useful for extensions
def generatefiles(self, changedfiles, commonrevs, source,
mfdicts, fastpathlinkrev, fnodes, clrevs):
-changedfiles = list(filter(self._filematcher, changedfiles))
+changedfiles = [f for f in changedfiles
+if self._matcher(f) and not self._oldmatcher(f)]
if not fastpathlinkrev:
def normallinknodes(unused, fname):
@@ -1151,12 +1166,13 @@
progress.complete()
-def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False,
- shallow=False, ellipsisroots=None, fullnodes=None):
+def _makecg1packer(repo, oldmatcher, matcher, bundlecaps,
+ ellipses=False, shallow=False, ellipsisroots=None,
+ fullnodes=None):
builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
d.node, d.p1node, d.p2node, d.linknode)
-return cgpacker(repo,
D4895: narrow: don't include manifests the client already has
martinvonz planned changes to this revision. martinvonz marked an inline comment as done. martinvonz added inline comments. INLINE COMMENTS > pulkit wrote in changegroup.py:1056 > Also, I think this is not the right place to case this because not yielding > these deltas will prevent the lookupfn being called and will result in > tmfnodes being not updated correctly. Fixed. I had not realized that `deltas` is a generator. Thanks for point that out. > changegroup.py:1056 > > -yield tree, deltas > +if not self._oldfilematcher.visitdir(store.tree[:-1] or '.'): > +yield tree, deltas The name "_oldfilematcher" is a little ironic since it's not used for file (only directories). The reason that we don't consult it for files is that we get passed a differenecematcher when widening and the subtraction of files the client already has is done by that. If we're making changegroup.generate() aware of widening like this patch does, perhaps we should make the caller not pass a differencematcher and we should instead check _oldfilematcher for files too. That would be more consistent and easier to understand. I guess there's not much reason not to do that, so I'll work on that (I'll update this patch). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4895 To: martinvonz, durin42, #hg-reviewers Cc: pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4931: narrow: only send the narrowspecs back if ACL in play
martinvonz added a comment. In https://phab.mercurial-scm.org/D4931#76571, @indygreg wrote: > In https://phab.mercurial-scm.org/D4931#76568, @martinvonz wrote: > > > In https://phab.mercurial-scm.org/D4931#76476, @pulkit wrote: > > > > > @martinvonz Looks like this got missed for discussing in sprint, can you have a look at this please? This is important because once we have narrowspecs greater than 255 length and and don't use ellipses, we hit https://bz.mercurial-scm.org/show_bug.cgi?id=5952 > > > > > > Maybe we can instead require `experimental.httppostargs` to be on when using narrow? > > > The problem is the narrow specs are being transported as a bundle2 part parameter and those have a limit of 255 bytes. The narrow spec needs to be transported as a bundle2 part payload, which can be infinite length. (Or not transferred at all.) Oh, I had completely forgotten that we sometimes send that back. This patch looks fine then, I just have one little suggestion. INLINE COMMENTS > narrowbundle2.py:273 > kwargs = exchange.applynarrowacl(repo, kwargs) > +kwargs['narrow_acl'] = True > maybe this line belongs on exchange.py:1903? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4931 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: indygreg, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4944: remotefilelog: rip out lz4 support
durin42 updated this revision to Diff 12182. durin42 edited the summary of this revision. durin42 retitled this revision from "remotefilelog: rip out lz4 support (WIP)" to "remotefilelog: rip out lz4 support". REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4944?vs=11810=12182 REVISION DETAIL https://phab.mercurial-scm.org/D4944 AFFECTED FILES hgext/remotefilelog/datapack.py hgext/remotefilelog/debugcommands.py hgext/remotefilelog/fileserverclient.py hgext/remotefilelog/lz4wrapper.py hgext/remotefilelog/remotefilelogserver.py tests/test-remotefilelog-bgprefetch.t tests/test-remotefilelog-gc.t tests/test-remotefilelog-gcrepack.t tests/test-remotefilelog-http.t tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t CHANGE DETAILS diff --git a/tests/test-remotefilelog-repack.t b/tests/test-remotefilelog-repack.t --- a/tests/test-remotefilelog-repack.t +++ b/tests/test-remotefilelog-repack.t @@ -47,17 +47,17 @@ $TESTTMP/hgcache/master/packs $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos # Test that the packs are readonly $ ls_l $CACHEDIR/master/packs -r--r--r--1145 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx -r--r--r-- 172 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack - -r--r--r--1074 add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx - -r--r--r-- 69 add67cb28ae0a2962111588ce49467ca9ebb9195.datapack + -r--r--r--1074 b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx + -r--r--r-- 72 b1e0cfc7f345e408a7825e3081501959488d59ce.datapack -rw-r--r-- 0 repacklock # Test that the data in the new packs is accessible @@ -80,8 +80,8 @@ $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216 $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos @@ -93,16 +93,16 @@ $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216 $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx - $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx + $TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos $ hg repack --traceback $ find $CACHEDIR -type f | sort - $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.dataidx - $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.datapack + $TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.dataidx + $TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack $TESTTMP/hgcache/master/packs/repacklock @@ -121,8 +121,8 @@ # and did not change the pack names $ hg repack $ find $CACHEDIR -type f | sort - $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.dataidx - $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.datapack + $TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.dataidx + $TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack $TESTTMP/hgcache/master/packs/repacklock @@ -145,8 +145,8 @@ 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob) $ find $CACHEDIR -type f |
D4943: remotefilelog: bump pack file formats to version 2
durin42 updated this revision to Diff 12181. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4943?vs=11809=12181 REVISION DETAIL https://phab.mercurial-scm.org/D4943 AFFECTED FILES hgext/remotefilelog/basepack.py hgext/remotefilelog/datapack.py hgext/remotefilelog/historypack.py hgext/remotefilelog/repack.py tests/test-remotefilelog-bgprefetch.t tests/test-remotefilelog-datapack.py tests/test-remotefilelog-gc.t tests/test-remotefilelog-gcrepack.t tests/test-remotefilelog-histpack.py tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t CHANGE DETAILS diff --git a/tests/test-remotefilelog-repack.t b/tests/test-remotefilelog-repack.t --- a/tests/test-remotefilelog-repack.t +++ b/tests/test-remotefilelog-repack.t @@ -45,19 +45,19 @@ $TESTTMP/hgcache $TESTTMP/hgcache/master $TESTTMP/hgcache/master/packs - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histidx - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histpack - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.dataidx - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.datapack + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos # Test that the packs are readonly $ ls_l $CACHEDIR/master/packs - -r--r--r--1145 276d308429d0303762befa376788300f0310f90e.histidx - -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack - -r--r--r--1074 8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.dataidx - -r--r--r-- 69 8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.datapack + -r--r--r--1145 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx + -r--r--r-- 172 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack + -r--r--r--1074 add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx + -r--r--r-- 69 add67cb28ae0a2962111588ce49467ca9ebb9195.datapack -rw-r--r-- 0 repacklock # Test that the data in the new packs is accessible @@ -78,10 +78,10 @@ $ find $CACHEDIR -type f | sort $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216 - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histidx - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histpack - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.dataidx - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.datapack + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos @@ -91,20 +91,20 @@ $ find $CACHEDIR -type f | sort $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216 - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histidx - $TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histpack - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.dataidx - $TESTTMP/hgcache/master/packs/8e25dec685d5e0bb1f1b39df3acebda0e0d75c6e.datapack + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histidx + $TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.dataidx + $TESTTMP/hgcache/master/packs/add67cb28ae0a2962111588ce49467ca9ebb9195.datapack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos $ hg repack --traceback $ find $CACHEDIR -type f | sort - $TESTTMP/hgcache/master/packs/077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histidx - $TESTTMP/hgcache/master/packs/077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack - $TESTTMP/hgcache/master/packs/935861cae0be6ce41a0d47a529e4d097e9e68a69.dataidx - $TESTTMP/hgcache/master/packs/935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack + $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.dataidx + $TESTTMP/hgcache/master/packs/1bd27e610ee06450e5f3bb0cd3afb6870e4cf375.datapack + $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx + $TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack $TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/repos
D5125: WIP: rename requirements and wireproto methods
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This is a mess, in part because there should be more constants
throughout.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5125
AFFECTED FILES
hgext/remotefilelog/__init__.py
hgext/remotefilelog/fileserverclient.py
hgext/remotefilelog/remotefilelogserver.py
hgext/remotefilelog/shallowbundle.py
hgext/remotefilelog/shallowrepo.py
tests/test-remotefilelog-http.t
CHANGE DETAILS
diff --git a/tests/test-remotefilelog-http.t b/tests/test-remotefilelog-http.t
--- a/tests/test-remotefilelog-http.t
+++ b/tests/test-remotefilelog-http.t
@@ -16,16 +16,16 @@
Build a query string for later use:
$ GET=`hg debugdata -m 0 | $PYTHON -c \
- > 'import sys ; print [("?cmd=getfile=%s=%s" %
tuple(s.split("\0"))) for s in sys.stdin.read().splitlines()][0]'`
+ > 'import sys ; print [("?cmd=x_rfl_getfile=%s=%s" %
tuple(s.split("\0"))) for s in sys.stdin.read().splitlines()][0]'`
$ cd ..
$ cat hg1.pid >> $DAEMON_PIDS
$ hgcloneshallow http://localhost:$HGPORT/ shallow -q
2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over *s (glob)
$ grep getfile access.log
- * "GET /?cmd=batch HTTP/1.1" 200 -
x-hgarg-1:cmds=getfile+*node%3D1406e74118627694268417491f018a4a883152f0* (glob)
+ * "GET /?cmd=batch HTTP/1.1" 200 -
x-hgarg-1:cmds=x_rfl_getfile+*node%3D1406e74118627694268417491f018a4a883152f0*
(glob)
Clear filenode cache so we can test fetching with a modified batch size
$ rm -r $TESTTMP/hgcache
@@ -37,17 +37,17 @@
The 'remotefilelog' capability should *not* be exported over http(s),
as the getfile method it offers doesn't work with http.
$ get-with-headers.py localhost:$HGPORT '?cmd=capabilities' | grep lookup |
identifyrflcaps
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ get-with-headers.py localhost:$HGPORT '?cmd=hello' | grep lookup |
identifyrflcaps
- getfile
- getflogheads
+ x_rfl_getfile
+ x_rfl_getflogheads
$ get-with-headers.py localhost:$HGPORT '?cmd=this-command-does-not-exist' |
head -n 1
400 no such method: this-command-does-not-exist
- $ get-with-headers.py localhost:$HGPORT '?cmd=getfiles' | head -n 1
- 400 no such method: getfiles
+ $ get-with-headers.py localhost:$HGPORT '?cmd=x_rfl_getfiles' | head -n 1
+ 400 no such method: x_rfl_getfiles
Verify serving from a shallow clone doesn't allow for remotefile
fetches. This also serves to test the error handling for our batchable
diff --git a/hgext/remotefilelog/shallowrepo.py
b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -40,7 +40,7 @@
from mercurial.utils import procutil
_hgexecutable = procutil.hgexecutable
-requirement = "remotefilelog"
+requirement = "x_rfl_remotefilelog"
_prefetching = _('prefetching')
# These make*stores functions are global so that other extensions can replace
diff --git a/hgext/remotefilelog/shallowbundle.py
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -25,7 +25,7 @@
LocalFiles = 1
AllFiles = 2
-requirement = "remotefilelog"
+requirement = "x_rfl_remotefilelog"
def shallowgroup(cls, self, nodelist, rlog, lookup, units=None, reorder=None):
if not isinstance(rlog, remotefilelog.remotefilelog):
diff --git a/hgext/remotefilelog/remotefilelogserver.py
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -77,11 +77,11 @@
# support file content requests
wireprotov1server.wireprotocommand(
-'getflogheads', 'path', permission='pull')(getflogheads)
+'x_rfl_getflogheads', 'path', permission='pull')(getflogheads)
wireprotov1server.wireprotocommand(
-'getfiles', '', permission='pull')(getfiles)
+'x_rfl_getfiles', '', permission='pull')(getfiles)
wireprotov1server.wireprotocommand(
-'getfile', 'file node', permission='pull')(getfile)
+'x_rfl_getfile', 'file node', permission='pull')(getfile)
class streamstate(object):
match = None
@@ -198,8 +198,8 @@
if isinstance(proto, _sshv1server):
# legacy getfiles method which only works over ssh
caps.append(shallowrepo.requirement)
-caps.append('getflogheads')
-caps.append('getfile')
+caps.append('x_rfl_getflogheads')
+caps.append('x_rfl_getfile')
return caps
extensions.wrapfunction(wireprotov1server, '_capabilities', _capabilities)
@@ -215,7 +215,7 @@
context.basefilectx, '_adjustlinkrev', _adjustlinkrev)
def _iscmd(orig, cmd):
-if cmd == 'getfiles':
+if cmd == 'x_rfl_getfiles':
return False
return orig(cmd)
diff
D5122: remotefilelog: remove support for fetching packs
durin42 created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY We've never used this at Google, and it's strictly an optimization that indygreg and I are confident we can handle more elegantly later. Let's rip it out to keep the surface area of the imported remotefilelog small. Some tests have changes because the tests for regular packs and repacking were mixed with tests of fetching packs. I'm pretty sure I've done the right thing here. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5122 AFFECTED FILES hgext/remotefilelog/__init__.py hgext/remotefilelog/fileserverclient.py hgext/remotefilelog/shallowrepo.py tests/test-remotefilelog-prefetch.t tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t CHANGE DETAILS diff --git a/tests/test-remotefilelog-repack.t b/tests/test-remotefilelog-repack.t --- a/tests/test-remotefilelog-repack.t +++ b/tests/test-remotefilelog-repack.t @@ -293,38 +293,32 @@ > [remotefilelog] > data.generations=60 > 150 - > fetchpacks=True > EOF Single pack - repack does nothing $ hg prefetch -r 0 - 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob) + 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob) $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack - -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack + [1] $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack - -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack + [1] $ hg repack --incremental $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack - -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack + -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack 3 gen1 packs, 1 gen0 pack - packs 3 gen1 into 1 $ hg prefetch -r 1 - 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob) + 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob) $ hg prefetch -r 2 - 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob) - $ hg prefetch -r 3 - 1 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over * (glob) + 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob) + $ hg prefetch -r 38 + abort: unknown revision '38'! + [255] $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack - -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack - -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack - -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack - -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack + -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack - -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack - -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack - -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack For the data packs, setting the limit for the repackmaxpacksize to be 64 such @@ -335,58 +329,46 @@ $ hg repack --incremental --config remotefilelog.data.repackmaxpacksize=64 \ > --config remotefilelog.history.repackmaxpacksize=0 $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack - -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack - -r--r--r-- 65 6c499d21350d79f92fd556b4b7a902569d88e3c9.datapack - -r--r--r-- 61 817d294043bd21a3de01f807721971abe45219ce.datapack - -r--r--r-- 63 ff45add45ab3f59c4f75efc6a087d86c821219d6.datapack + -r--r--r-- 147 935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack + -r--r--r-- 67 b5a62f3496ccbd2479497cdbc7345f3304735f33.datapack $ ls_l $TESTTMP/hgcache/master/packs/ | grep histpack -r--r--r-- 254 077e7ce5dfe862dc40cc8f3c9742d96a056865f2.histpack - -r--r--r-- 336 094b530486dad4427a0faf6bcbc031571b99ca24.histpack - -r--r--r-- 172 276d308429d0303762befa376788300f0310f90e.histpack -r--r--r-- 90 c3399b56e035f73c3295276ed098235a08a0ed8c.histpack Setting limit for the repackmaxpacksize to be the size of the biggest pack file which ensures that it is effectively ignored in the incremental repacking. $ hg repack --incremental --config remotefilelog.data.repackmaxpacksize=65 \ > --config remotefilelog.history.repackmaxpacksize=336 $ ls_l $TESTTMP/hgcache/master/packs/ | grep datapack - -r--r--r-- 59 5b7dec902026f0cddb0ef8acb62f27b5698494d4.datapack - -r--r--r-- 225 8fe685c56f6f7edf550bfcec74eeecc5f3c2ba15.datapack + -r--r--r-- 147 935861cae0be6ce41a0d47a529e4d097e9e68a69.datapack + -r--r--r-- 67
D5123: remotefilelogserver: remove pack-serving functionality
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5123
AFFECTED FILES
hgext/remotefilelog/remotefilelogserver.py
CHANGE DETAILS
diff --git a/hgext/remotefilelog/remotefilelogserver.py
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -12,29 +12,25 @@
import time
from mercurial.i18n import _
-from mercurial.node import bin, hex, nullid, nullrev
+from mercurial.node import bin, hex, nullid
from mercurial import (
-ancestor,
changegroup,
changelog,
context,
error,
extensions,
match,
-pycompat,
store,
streamclone,
util,
wireprotoserver,
wireprototypes,
wireprotov1server,
)
from . import (
-constants,
lz4wrapper,
shallowrepo,
shallowutil,
-wirepack,
)
_sshv1server = wireprotoserver.sshv1protocolhandler
@@ -86,8 +82,6 @@
'getfiles', '', permission='pull')(getfiles)
wireprotov1server.wireprotocommand(
'getfile', 'file node', permission='pull')(getfile)
-wireprotov1server.wireprotocommand(
-'getpackv1', '*', permission='pull')(getpack)
class streamstate(object):
match = None
@@ -421,134 +415,3 @@
os.remove(filepath)
ui.progress(_removing, None)
-
-def getpack(repo, proto, args):
-"""A server api for requesting a pack of file information.
-"""
-if shallowrepo.requirement in repo.requirements:
-raise error.Abort(_('cannot fetch remote files from shallow repo'))
-if not isinstance(proto, _sshv1server):
-raise error.Abort(_('cannot fetch remote files over non-ssh protocol'))
-
-def streamer():
-"""Request format:
-
-[,...]\0\0
-filerequest =
- [,...]
-
-Response format:
-[,...]<10 null bytes>
-fileresponse =
-history = [,...]
-historyentry =
-
-deltas = [,...]
-deltaentry =
-
-"""
-fin = proto._fin
-files = _receivepackrequest(fin)
-
-# Sort the files by name, so we provide deterministic results
-for filename, nodes in sorted(files.iteritems()):
-fl = repo.file(filename)
-
-# Compute history
-history = []
-for rev in ancestor.lazyancestors(fl.parentrevs,
- [fl.rev(n) for n in nodes],
- inclusive=True):
-linkrev = fl.linkrev(rev)
-node = fl.node(rev)
-p1node, p2node = fl.parents(node)
-copyfrom = ''
-linknode = repo.changelog.node(linkrev)
-if p1node == nullid:
-copydata = fl.renamed(node)
-if copydata:
-copyfrom, copynode = copydata
-p1node = copynode
-
-history.append((node, p1node, p2node, linknode, copyfrom))
-
-# Scan and send deltas
-chain = _getdeltachain(fl, nodes, -1)
-
-for chunk in wirepack.sendpackpart(filename, history, chain):
-yield chunk
-
-yield wirepack.closepart()
-proto._fout.flush()
-
-return wireprototypes.streamres(streamer())
-
-def _receivepackrequest(stream):
-files = {}
-while True:
-filenamelen = shallowutil.readunpack(stream,
- constants.FILENAMESTRUCT)[0]
-if filenamelen == 0:
-break
-
-filename = shallowutil.readexactly(stream, filenamelen)
-
-nodecount = shallowutil.readunpack(stream,
- constants.PACKREQUESTCOUNTSTRUCT)[0]
-
-# Read N nodes
-nodes = shallowutil.readexactly(stream, constants.NODESIZE * nodecount)
-nodes = set(nodes[i:i + constants.NODESIZE] for i in
-pycompat.xrange(0, len(nodes), constants.NODESIZE))
-
-files[filename] = nodes
-
-return files
-
-def _getdeltachain(fl, nodes, stophint):
-"""Produces a chain of deltas that includes each of the given nodes.
-
-`stophint` - The changeset rev number to stop at. If it's set to >= 0, we
-will return not only the deltas for the requested nodes, but also all
-necessary deltas in their delta chains, as long as the deltas have link
revs
->= the stophint. This allows us to return an approximately minimal delta
-chain when the user performs a pull. If `stophint` is set to -1, all nodes
-will return full texts. """
-chain = []
-
-seen = set()
-for node in nodes:
-startrev = fl.rev(node)
-cur = startrev
-while True:
-if cur in
D5124: remotefilelog: remove now-unused wirepack code
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5124
AFFECTED FILES
hgext/remotefilelog/wirepack.py
CHANGE DETAILS
diff --git a/hgext/remotefilelog/wirepack.py b/hgext/remotefilelog/wirepack.py
deleted file mode 100644
--- a/hgext/remotefilelog/wirepack.py
+++ /dev/null
@@ -1,235 +0,0 @@
-# wirepack.py - wireprotocol for exchanging packs
-#
-# Copyright 2017 Facebook, Inc.
-#
-# This software may be used and distributed according to the terms of the
-# GNU General Public License version 2 or any later version.
-from __future__ import absolute_import
-
-import StringIO
-import collections
-import struct
-
-from mercurial.i18n import _
-from mercurial.node import nullid
-from mercurial import (
-pycompat,
-)
-from . import (
-constants,
-datapack,
-historypack,
-shallowutil,
-)
-
-def sendpackpart(filename, history, data):
-"""A wirepack is formatted as follows:
-
-wirepack =
- [,...]
- [,...]
-
-hist rev =
-
-
-
-
-
-
-data rev =
-
-
-
-"""
-rawfilenamelen = struct.pack(constants.FILENAMESTRUCT,
- len(filename))
-yield '%s%s' % (rawfilenamelen, filename)
-
-# Serialize and send history
-historylen = struct.pack('!I', len(history))
-rawhistory = ''
-for entry in history:
-copyfrom = entry[4] or ''
-copyfromlen = len(copyfrom)
-tup = entry[:-1] + (copyfromlen,)
-rawhistory += struct.pack('!20s20s20s20sH', *tup)
-if copyfrom:
-rawhistory += copyfrom
-
-yield '%s%s' % (historylen, rawhistory)
-
-# Serialize and send data
-yield struct.pack('!I', len(data))
-
-# TODO: support datapack metadata
-for node, deltabase, delta in data:
-deltalen = struct.pack('!Q', len(delta))
-yield '%s%s%s%s' % (node, deltabase, deltalen, delta)
-
-def closepart():
-return '\0' * 10
-
-def receivepack(ui, fh, packpath):
-receiveddata = []
-receivedhistory = []
-shallowutil.mkstickygroupdir(ui, packpath)
-totalcount = 0
-ui.progress(_("receiving pack"), totalcount)
-with datapack.mutabledatapack(ui, packpath) as dpack:
-with historypack.mutablehistorypack(ui, packpath) as hpack:
-pendinghistory = collections.defaultdict(dict)
-while True:
-filename = shallowutil.readpath(fh)
-count = 0
-
-# Store the history for later sorting
-for value in readhistory(fh):
-node = value[0]
-pendinghistory[filename][node] = value
-receivedhistory.append((filename, node))
-count += 1
-
-for node, deltabase, delta in readdeltas(fh):
-dpack.add(filename, node, deltabase, delta)
-receiveddata.append((filename, node))
-count += 1
-
-if count == 0 and filename == '':
-break
-totalcount += 1
-ui.progress(_("receiving pack"), totalcount)
-
-# Add history to pack in toposorted order
-for filename, nodevalues in sorted(pendinghistory.iteritems()):
-def _parentfunc(node):
-p1, p2 = nodevalues[node][1:3]
-parents = []
-if p1 != nullid:
-parents.append(p1)
-if p2 != nullid:
-parents.append(p2)
-return parents
-sortednodes = reversed(shallowutil.sortnodes(
-nodevalues.iterkeys(),
-_parentfunc))
-for node in sortednodes:
-node, p1, p2, linknode, copyfrom = nodevalues[node]
-hpack.add(filename, node, p1, p2, linknode, copyfrom)
-ui.progress(_("receiving pack"), None)
-
-return receiveddata, receivedhistory
-
-def readhistory(fh):
-count = shallowutil.readunpack(fh, '!I')[0]
-for i in pycompat.xrange(count):
-entry = shallowutil.readunpack(fh,'!20s20s20s20sH')
-if entry[4] != 0:
-copyfrom = shallowutil.readexactly(fh, entry[4])
-else:
-copyfrom = ''
-entry = entry[:4] + (copyfrom,)
-yield entry
-
-def readdeltas(fh):
-count = shallowutil.readunpack(fh, '!I')[0]
-for i in pycompat.xrange(count):
-node, deltabase, deltalen = shallowutil.readunpack(fh, '!20s20sQ')
-delta = shallowutil.readexactly(fh, deltalen)
-yield (node, deltabase, delta)
-
-class wirepackstore(object):
-def __init__(self, wirepack):
-
D5121: procutil: port over windows encoding fixes from logtoprocess
durin42 created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5121 AFFECTED FILES mercurial/utils/procutil.py CHANGE DETAILS diff --git a/mercurial/utils/procutil.py b/mercurial/utils/procutil.py --- a/mercurial/utils/procutil.py +++ b/mercurial/utils/procutil.py @@ -481,8 +481,10 @@ # we can't use close_fds *and* redirect stdin. I'm not sure that we # need to because the detached process has no console connection. subprocess.Popen( -script, shell=shell, env=env, close_fds=True, -creationflags=_creationflags, stdout=stdout, stderr=stderr) +tonativestr(script), +shell=shell, env=tonativeenv(env), close_fds=True, +creationflags=_creationflags, stdout=stdout, +stderr=stderr) else: def runbgcommand(cmd, env, shell=False, stdout=None, stderr=None): '''Spawn a command without waiting for it to finish.''' To: durin42, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4782: remotefilelog: import pruned-down remotefilelog extension from hg-experimental
durin42 updated this revision to Diff 12176. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4782?vs=11802=12176 REVISION DETAIL https://phab.mercurial-scm.org/D4782 AFFECTED FILES hgext/remotefilelog/README.md hgext/remotefilelog/__init__.py hgext/remotefilelog/basepack.py hgext/remotefilelog/basestore.py hgext/remotefilelog/cacheclient.py hgext/remotefilelog/connectionpool.py hgext/remotefilelog/constants.py hgext/remotefilelog/contentstore.py hgext/remotefilelog/datapack.py hgext/remotefilelog/debugcommands.py hgext/remotefilelog/extutil.py hgext/remotefilelog/fileserverclient.py hgext/remotefilelog/historypack.py hgext/remotefilelog/lz4wrapper.py hgext/remotefilelog/metadatastore.py hgext/remotefilelog/remotefilectx.py hgext/remotefilelog/remotefilelog.py hgext/remotefilelog/remotefilelogserver.py hgext/remotefilelog/repack.py hgext/remotefilelog/shallowbundle.py hgext/remotefilelog/shallowrepo.py hgext/remotefilelog/shallowstore.py hgext/remotefilelog/shallowutil.py hgext/remotefilelog/shallowverifier.py hgext/remotefilelog/wirepack.py setup.py tests/ls-l.py tests/remotefilelog-getflogheads.py tests/remotefilelog-library.sh tests/test-remotefilelog-bad-configs.t tests/test-remotefilelog-bgprefetch.t tests/test-remotefilelog-blame.t tests/test-remotefilelog-bundle2-legacy.t tests/test-remotefilelog-bundle2.t tests/test-remotefilelog-bundles.t tests/test-remotefilelog-cacheprocess.t tests/test-remotefilelog-clone-tree.t tests/test-remotefilelog-clone.t tests/test-remotefilelog-corrupt-cache.t tests/test-remotefilelog-datapack.py tests/test-remotefilelog-gc.t tests/test-remotefilelog-gcrepack.t tests/test-remotefilelog-histpack.py tests/test-remotefilelog-http.t tests/test-remotefilelog-keepset.t tests/test-remotefilelog-linknodes.t tests/test-remotefilelog-local.t tests/test-remotefilelog-log.t tests/test-remotefilelog-partial-shallow.t tests/test-remotefilelog-permissions.t tests/test-remotefilelog-prefetch.t tests/test-remotefilelog-pull-noshallow.t tests/test-remotefilelog-push-pull.t tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t tests/test-remotefilelog-sparse.t tests/test-remotefilelog-tags.t tests/test-remotefilelog-wireproto.t To: durin42, #hg-reviewers Cc: indygreg, spectral, mjpieters, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4931: narrow: only send the narrowspecs back if ACL in play
indygreg added a comment. In https://phab.mercurial-scm.org/D4931#76568, @martinvonz wrote: > In https://phab.mercurial-scm.org/D4931#76476, @pulkit wrote: > > > @martinvonz Looks like this got missed for discussing in sprint, can you have a look at this please? This is important because once we have narrowspecs greater than 255 length and and don't use ellipses, we hit https://bz.mercurial-scm.org/show_bug.cgi?id=5952 > > > Maybe we can instead require `experimental.httppostargs` to be on when using narrow? The problem is the narrow specs are being transported as a bundle2 part parameter and those have a limit of 255 bytes. The narrow spec needs to be transported as a bundle2 part payload, which can be infinite length. (Or not transferred at all.) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4931 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: indygreg, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4895: narrow: don't include manifests the client already has
martinvonz updated this revision to Diff 12175. martinvonz retitled this revision from "RFC: narrow: don't include manifests the client already has" to "narrow: don't include manifests the client already has". REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4895?vs=12129=12175 REVISION DETAIL https://phab.mercurial-scm.org/D4895 AFFECTED FILES hgext/narrow/narrowwirepeer.py mercurial/bundle2.py mercurial/changegroup.py CHANGE DETAILS diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -727,7 +727,7 @@ progress.complete() class cgpacker(object): -def __init__(self, repo, filematcher, version, +def __init__(self, repo, filematcher, oldfilematcher, version, builddeltaheader, manifestsend, forcedeltaparentprev=False, bundlecaps=None, ellipses=False, @@ -762,7 +762,9 @@ significantly smaller. """ assert filematcher +assert oldfilematcher self._filematcher = filematcher +self._oldfilematcher = oldfilematcher self.version = version self._forcedeltaparentprev = forcedeltaparentprev @@ -1051,7 +1053,16 @@ fullclnodes=self._fullclnodes, precomputedellipsis=self._precomputedellipsis) -yield tree, deltas +if not self._oldfilematcher.visitdir(store.tree[:-1] or '.'): +yield tree, deltas +else: +# 'deltas' is a generator and we need to consume it even if +# we are not going to send it because a side-effect is that +# it updates tmdnodes (via lookupfn) +for d in deltas: +pass +if not tree: +yield tree, [] def _prunemanifests(self, store, nodes, commonrevs): # This is split out as a separate method to allow filtering @@ -1151,12 +1162,13 @@ progress.complete() -def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False, - shallow=False, ellipsisroots=None, fullnodes=None): +def _makecg1packer(repo, filematcher, oldfilematcher, bundlecaps, + ellipses=False, shallow=False, ellipsisroots=None, + fullnodes=None): builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack( d.node, d.p1node, d.p2node, d.linknode) -return cgpacker(repo, filematcher, b'01', +return cgpacker(repo, filematcher, oldfilematcher, b'01', builddeltaheader=builddeltaheader, manifestsend=b'', forcedeltaparentprev=True, @@ -1166,26 +1178,28 @@ ellipsisroots=ellipsisroots, fullnodes=fullnodes) -def _makecg2packer(repo, filematcher, bundlecaps, ellipses=False, - shallow=False, ellipsisroots=None, fullnodes=None): +def _makecg2packer(repo, filematcher, oldfilematcher, bundlecaps, + ellipses=False, shallow=False, ellipsisroots=None, + fullnodes=None): builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack( d.node, d.p1node, d.p2node, d.basenode, d.linknode) -return cgpacker(repo, filematcher, b'02', +return cgpacker(repo, filematcher, oldfilematcher, b'02', builddeltaheader=builddeltaheader, manifestsend=b'', bundlecaps=bundlecaps, ellipses=ellipses, shallow=shallow, ellipsisroots=ellipsisroots, fullnodes=fullnodes) -def _makecg3packer(repo, filematcher, bundlecaps, ellipses=False, - shallow=False, ellipsisroots=None, fullnodes=None): +def _makecg3packer(repo, filematcher, oldfilematcher, bundlecaps, + ellipses=False, shallow=False, ellipsisroots=None, + fullnodes=None): builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack( d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags) -return cgpacker(repo, filematcher, b'03', +return cgpacker(repo, filematcher, oldfilematcher, b'03', builddeltaheader=builddeltaheader, manifestsend=closechunk(), bundlecaps=bundlecaps, @@ -1253,12 +1267,14 @@ return min(versions) def getbundler(version, repo, bundlecaps=None, filematcher=None, - ellipses=False, shallow=False, ellipsisroots=None, - fullnodes=None): + oldfilematcher=None, ellipses=False, shallow=False, + ellipsisroots=None, fullnodes=None): assert version in supportedoutgoingversions(repo) if filematcher is None: filematcher = matchmod.alwaysmatcher(repo.root, '') +if oldfilematcher is None: +
D4931: narrow: only send the narrowspecs back if ACL in play
martinvonz added a comment. In https://phab.mercurial-scm.org/D4931#76476, @pulkit wrote: > @martinvonz Looks like this got missed for discussing in sprint, can you have a look at this please? This is important because once we have narrowspecs greater than 255 length and and don't use ellipses, we hit https://bz.mercurial-scm.org/show_bug.cgi?id=5952 Maybe we can instead require `experimental.httppostargs` to be on when using narrow? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4931 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[Bug 6002] New: allsuccessors(-1) complains about nullid being pruned
https://bz.mercurial-scm.org/show_bug.cgi?id=6002 Bug ID: 6002 Summary: allsuccessors(-1) complains about nullid being pruned Product: Mercurial Version: unspecified Hardware: PC OS: Mac OS Status: UNCONFIRMED Severity: feature Priority: wish Component: evolution Assignee: bugzi...@mercurial-scm.org Reporter: h...@pewpew.net CC: mercurial-devel@mercurial-scm.org, pierre-yves.da...@ens-lyon.org ≻ cat test-nullrev.t $ cat >> $HGRCPATH < [extensions] > evolve = > EOF $ hg init r $ cd r $ echo hi > a $ hg ci -qAm a This works, as long as there's no obsolescence in the repo $ hg log -r 'allsuccessors(-1)' $ echo hi2 > a $ hg ci -qm a2 Add some obsolescence (strangely, if I use `hg debugobsolete` this does NOT reproduce) $ hg prune -r tip 1 files updated, 0 files merged, 0 files removed, 0 files unresolved working directory now at 54894f3cc22c 1 changesets pruned Now it doesn't work $ hg log -r 'allsuccessors(-1)' abort: hidden revision '-1' is pruned! (use --hidden to access hidden revisions) [255] nullid always seems to work $ hg log -r 'allsuccessors(0)' -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5120: match: fix up a repr to not crash on Python 3
durin42 created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5120 AFFECTED FILES mercurial/match.py CHANGE DETAILS diff --git a/mercurial/match.py b/mercurial/match.py --- a/mercurial/match.py +++ b/mercurial/match.py @@ -1173,7 +1173,7 @@ else: dir = '.' return dir in dirs -regex = b'rootfilesin: %s' % sorted(dirs) +regex = b'rootfilesin: %s' % stringutil.pprint(list(sorted(dirs))) matchfuncs.append(mf) else: regex, mf = _buildregexmatch(kindpats, globsuffix) To: durin42, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5107: localrepo: ensure we properly %-format int in exception throw
durin42 updated this revision to Diff 12173.
durin42 retitled this revision from "localrepo: ensure we properly %-format
ints vs strings in exception throw" to "localrepo: ensure we properly %-format
int in exception throw".
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5107?vs=12139=12173
REVISION DETAIL
https://phab.mercurial-scm.org/D5107
AFFECTED FILES
mercurial/localrepo.py
CHANGE DETAILS
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1282,7 +1282,8 @@
raise error.FilteredRepoLookupError(_("filtered revision '%s'")
% pycompat.bytestr(changeid))
except (IndexError, LookupError):
-raise error.RepoLookupError(_("unknown revision '%s'") % changeid)
+raise error.RepoLookupError(
+_("unknown revision '%s'") % pycompat.bytestr(changeid))
except error.WdirUnsupported:
return context.workingctx(self)
To: durin42, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5116: testing: switch to inserting deltas
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2c0aa02ecd5a: testing: switch to inserting deltas (authored
by indygreg, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5116?vs=12164=12172
REVISION DETAIL
https://phab.mercurial-scm.org/D5116
AFFECTED FILES
mercurial/testing/storage.py
tests/test-storage.py
CHANGE DETAILS
diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -49,7 +49,8 @@
if rawtext is not None:
fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags)
elif delta is not None:
-raise error.Abort('support for storing raw deltas not yet supported')
+fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags,
+ cachedelta=delta)
else:
raise error.Abort('must supply rawtext or delta arguments')
diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py
--- a/mercurial/testing/storage.py
+++ b/mercurial/testing/storage.py
@@ -952,13 +952,12 @@
with self.assertRaises(error.StorageError):
f.read(node1)
-diff = mdiff.textdiff(fulltext1, fulltext2)
node2 = storageutil.hashrevisionsha1(fulltext2, node1, nullid)
-deltas = [(node2, node1, nullid, b'\x01' * 20, node1, diff, 0)]
-# This /might/ fail on some backends.
with self._maketransactionfn() as tr:
-f.addgroup(deltas, lambda x: 0, tr)
+delta = mdiff.textdiff(fulltext1, fulltext2)
+self._addrawrevisionfn(f, tr, node2, node1, nullid,
+ 2, delta=(1, delta))
self.assertEqual(len(f), 3)
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5115: testing: remove expectation of error on bad node insert
This revision was automatically updated to reflect the committed changes. Closed by commit rHGddeb510d6815: testing: remove expectation of error on bad node insert (authored by indygreg, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5115?vs=12163=12171 REVISION DETAIL https://phab.mercurial-scm.org/D5115 AFFECTED FILES mercurial/testing/storage.py CHANGE DETAILS diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py --- a/mercurial/testing/storage.py +++ b/mercurial/testing/storage.py @@ -1073,14 +1073,7 @@ fulltext0 = b'foo' delta0 = mdiff.trivialdiffheader(len(fulltext0)) + fulltext0 -deltas = [ -(b'\x01' * 20, nullid, nullid, nullid, nullid, delta0, 0), -] - with self._maketransactionfn() as tr: -with self.assertRaises(error.StorageError): -f.addgroup(deltas, linkmapper, tr, addrevisioncb=cb) - node0 = f.add(fulltext0, None, tr, 0, nullid, nullid) f = self._makefilefn() To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Sprint 5.0 (Asia, Spring 2019)
On Sun, Oct 14, 2018 at 12:30 PM Kyle Lippincott wrote: > Per the recent discussion at the 4.8 Sprint, let's discuss the next sprint > now. The next one, for what is currently assumed to be the 5.0 release in > Spring 2019, is currently scheduled to happen in Asia and I'm proposing > Japan specifically. > > The past two sprints have been planned by Googlers and hosted by Google. > Ideally a separate entity offers hosting space and handle logistics for > this one. If no other entity steps up, I will attempt to organize this > through Google again at Google's Tokyo office, but it is unclear if this is > going to be acceptable. For Google's Tokyo office in particular, > scheduling of a conference room is difficult, so getting agreement on the > host, location, and timing earlier is *extremely* beneficial; let's be > proactive about this. :) > > Please let me know if there are any questions or concerns. > >From a visa and transportation standpoint, Japan is likely our best option in Asia. I think Hong Kong and Taiwan would also be up there. But I believe some have more difficulty with visas for those locations compared to Japan. I would volunteer Mozilla to host. But I'm pretty sure we don't have a suitable space in Japan. We /may/ have a suitable space in Taiwan. I'd have to look into it... ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5119: exchange: pass includepats and excludepats as arguments to getbundle()
indygreg requested changes to this revision. indygreg added inline comments. This revision now requires changes to proceed. INLINE COMMENTS > exchange.py:1631 > +kwargs['includepats'] = pullop.includepats or None > +kwargs['excludepats'] = pullop.excludepats or None > + I'd prefer if we didn't send empty arguments across the wire if it can be avoided. I'm pretty sure it can be avoided in this case. More importantly, the sending of these arguments needs to be conditional on a server capabilitiy because servers barf when receiving an unknown argument (the type of the argument isn't registered and servers barf when trying to decode the argument value). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5119 To: pulkit, #hg-reviewers, indygreg Cc: indygreg, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 4] storage: also use `deltamode argument` for ifiledata
On Tue, Oct 16, 2018 at 10:38 AM Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1539120395 -7200
> # Tue Oct 09 23:26:35 2018 +0200
> # Node ID c835d3020e536e8ef368223d628b9e0c6d0c8251
> # Parent a075ac3487d6d266ec5f2dbd6901da21c38d4ed9
> # EXP-Topic slim-bundle
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> c835d3020e53
> storage: also use `deltamode argument` for ifiledata
>
> Now that lower level uses such argument, we can propagate the change to
> higher
> layers. At some of the higher level, we use `None` as the default value to
> simplify imports and argument forwarding (instead of
> `storageutil.DELTAMODE_STD`). Strictly speaking, we could import
> `storageutil`
> everywhere, however, it did not seem to help readability.
>
> diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
> --- a/mercurial/changegroup.py
> +++ b/mercurial/changegroup.py
> @@ -29,6 +29,10 @@ from . import (
> util,
> )
>
> +from .utils import (
> +storageutil,
> +)
> +
> _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
> _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
> _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
> @@ -697,12 +701,16 @@ def deltagroup(repo, store, nodes, ischa
> progress = repo.ui.makeprogress(topic, unit=_('chunks'),
> total=len(nodes))
>
> +deltamode = storageutil.DELTAMODE_STD
> +if forcedeltaparentprev:
> +deltamode = storageutil.DELTAMODE_PREV
> +
> revisions = store.emitrevisions(
> nodes,
> nodesorder=nodesorder,
> revisiondata=True,
> assumehaveparentrevisions=not ellipses,
> -deltaprevious=forcedeltaparentprev)
> +deltamode=deltamode)
>
> for i, revision in enumerate(revisions):
> if progress:
> diff --git a/mercurial/filelog.py b/mercurial/filelog.py
> --- a/mercurial/filelog.py
> +++ b/mercurial/filelog.py
> @@ -77,11 +77,11 @@ class filelog(object):
>
> def emitrevisions(self, nodes, nodesorder=None,
>revisiondata=False, assumehaveparentrevisions=False,
> - deltaprevious=False):
> + deltamode=None):
> return self._revlog.emitrevisions(
> nodes, nodesorder=nodesorder, revisiondata=revisiondata,
> assumehaveparentrevisions=assumehaveparentrevisions,
> -deltaprevious=deltaprevious)
> +deltamode=deltamode)
>
> def addrevision(self, revisiondata, transaction, linkrev, p1, p2,
> node=None, flags=revlog.REVIDX_DEFAULT_FLAGS,
> diff --git a/mercurial/manifest.py b/mercurial/manifest.py
> --- a/mercurial/manifest.py
> +++ b/mercurial/manifest.py
> @@ -1575,11 +1575,11 @@ class manifestrevlog(object):
>
> def emitrevisions(self, nodes, nodesorder=None,
>revisiondata=False, assumehaveparentrevisions=False,
> - deltaprevious=False):
> + deltamode=None):
> return self._revlog.emitrevisions(
> nodes, nodesorder=nodesorder, revisiondata=revisiondata,
> assumehaveparentrevisions=assumehaveparentrevisions,
> -deltaprevious=deltaprevious)
> +deltamode=deltamode)
>
> def addgroup(self, deltas, linkmapper, transaction,
> addrevisioncb=None):
> return self._revlog.addgroup(deltas, linkmapper, transaction,
> diff --git a/mercurial/repository.py b/mercurial/repository.py
> --- a/mercurial/repository.py
> +++ b/mercurial/repository.py
> @@ -602,7 +602,7 @@ class ifiledata(interfaceutil.Interface)
>nodesorder=None,
>revisiondata=False,
>assumehaveparentrevisions=False,
> - deltaprevious=False):
> + deltamode=None):
> """Produce ``irevisiondelta`` for revisions.
>
> Given an iterable of nodes, emits objects conforming to the
> @@ -645,10 +645,10 @@ class ifiledata(interfaceutil.Interface)
> The ``linknode`` attribute on the returned ``irevisiondelta`` may
> not
> be set and it is the caller's responsibility to resolve it, if
> needed.
>
> -If ``deltaprevious`` is True and revision data is requested, all
> -revision data should be emitted as deltas against the revision
> -emitted just prior. The initial revision should be a delta against
> -its 1st parent.
> +If ``deltamode`` is storageutil.DELTAMODE_PREV and revision data
> is
> +requested, all revision data should be emitted as deltas against
> the
> +revision emitted just prior. The initial revision should be a
> delta
> +against its 1st parent.
> """
>
I'm not thrilled about referencing a constant not in repository.py here: a
Re: [PATCH 2 of 4] changegroup: refactor emitrevision to use a `deltamode` argument
On Tue, Oct 16, 2018 at 10:36 AM Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1539115321 -7200 > # Tue Oct 09 22:02:01 2018 +0200 > # Node ID a075ac3487d6d266ec5f2dbd6901da21c38d4ed9 > # Parent a69790aaac1946e80e35b28c8f41b14a7f6c0f64 > # EXP-Topic slim-bundle > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > a075ac3487d6 > changegroup: refactor emitrevision to use a `deltamode` argument > > This new argument gathers the semantic of `sendfulltext` and > `deltaprevious` > in a single value. We are about to introduce a new type of constraints. > Avoiding yet another argument sounds like a plus. > I have mixed feelings about this patch. I agree the API is cleaner. It's just that I feel with shallow clone, we'll need more flexibility in terms of delta generation - too much flexibility to shoehorn into a single "deltamode" argument. I think we'll need multiple arguments to influence delta generation behavior. But that is the future and this patch is now and is in support of a feature to facilitate performance testing. We shouldn't let the future dictate today. So I think I'll take this once the rest of the patches in this series clear. > > diff --git a/mercurial/revlog.py b/mercurial/revlog.py > --- a/mercurial/revlog.py > +++ b/mercurial/revlog.py > @@ -2229,6 +2229,12 @@ class revlog(object): > if nodesorder is None and not self._generaldelta: > nodesorder = 'storage' > > +deltamode = storageutil.DELTAMODE_STD > +if deltaprevious: > +deltamode = storageutil.DELTAMODE_PREV > +elif not self._storedeltachains: > +deltamode = storageutil.DELTAMODE_FULL > + > return storageutil.emitrevisions( > self, nodes, nodesorder, revlogrevisiondelta, > deltaparentfn=self.deltaparent, > @@ -2236,10 +2242,9 @@ class revlog(object): > rawsizefn=self.rawsize, > revdifffn=self.revdiff, > flagsfn=self.flags, > -sendfulltext=not self._storedeltachains, > +deltamode=deltamode, > revisiondata=revisiondata, > -assumehaveparentrevisions=assumehaveparentrevisions, > -deltaprevious=deltaprevious) > +assumehaveparentrevisions=assumehaveparentrevisions) > > DELTAREUSEALWAYS = 'always' > DELTAREUSESAMEREVS = 'samerevs' > diff --git a/mercurial/utils/storageutil.py > b/mercurial/utils/storageutil.py > --- a/mercurial/utils/storageutil.py > +++ b/mercurial/utils/storageutil.py > @@ -265,11 +265,14 @@ def resolvestripinfo(minlinkrev, tiprev, > > return strippoint, brokenrevs > > +DELTAMODE_STD = 'default' > +DELTAMODE_PREV = 'previous' > +DELTAMODE_FULL = 'fulltext' > + > def emitrevisions(store, nodes, nodesorder, resultcls, deltaparentfn=None, >candeltafn=None, rawsizefn=None, revdifffn=None, > flagsfn=None, > - sendfulltext=False, > - revisiondata=False, assumehaveparentrevisions=False, > - deltaprevious=False): > + deltamode=DELTAMODE_STD, > + revisiondata=False, assumehaveparentrevisions=False): > """Generic implementation of ifiledata.emitrevisions(). > > Emitting revision data is subtly complex. This function attempts to > @@ -320,14 +323,17 @@ def emitrevisions(store, nodes, nodesord > Callable receiving a revision number and returns the integer flags > value for it. If not defined, flags value will be 0. > > -``sendfulltext`` > +``deltamode`` > + constaint on delta to be sent: > + * DELTAMODE_STD - normal mode, try to reuse storage deltas, > + * DELTAMODE_PREV - only delta against "prev", > + * DELTAMODE_FULL - only issue full snapshot. > + > Whether to send fulltext revisions instead of deltas, if allowed. > > ``nodesorder`` > ``revisiondata`` > ``assumehaveparentrevisions`` > -``deltaprevious`` > - See ``ifiledata.emitrevisions()`` interface documentation. > """ > > fnode = store.node > @@ -343,7 +349,7 @@ def emitrevisions(store, nodes, nodesord > > prevrev = None > > -if deltaprevious or assumehaveparentrevisions: > +if deltamode == DELTAMODE_PREV or assumehaveparentrevisions: > prevrev = store.parentrevs(revs[0])[0] > > # Set of revs available to delta against. > @@ -362,11 +368,11 @@ def emitrevisions(store, nodes, nodesord > deltaparentrev = nullrev > > # Forced delta against previous mode. > -if deltaprevious: > +if deltamode == DELTAMODE_PREV: > baserev = prevrev > > # We're instructed to send fulltext. Honor that. > -elif sendfulltext: > +elif deltamode == DELTAMODE_FULL: > baserev = nullrev > > # There is a delta in storage. We try to use
Re: [PATCH 1 of 4] changegroup: if possible fallback to delta against parents instead of prev
On Tue, Oct 16, 2018 at 10:35 AM Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1539114828 -7200 > # Tue Oct 09 21:53:48 2018 +0200 > # Node ID a69790aaac1946e80e35b28c8f41b14a7f6c0f64 > # Parent 43f0a37bd9ed20b45d73f667acbb0c3035492e1a > # EXP-Topic slim-bundle > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > a69790aaac19 > changegroup: if possible fallback to delta against parents instead of prev > > Deltas against parent are likely to be smaller. This is especially useful > since we stop reordering changeset in the bundle. > > diff --git a/mercurial/utils/storageutil.py > b/mercurial/utils/storageutil.py > --- a/mercurial/utils/storageutil.py > +++ b/mercurial/utils/storageutil.py > @@ -394,9 +394,11 @@ def emitrevisions(store, nodes, nodesord > baserev = nullrev > > # Storage has a fulltext revision. > - > -# Let's use the previous revision, which is as good a guess as > any. > -# There is definitely room to improve this logic. > +elif assumehaveparentrevisions and p1rev != nullrev: > +baserev = p1rev > +elif assumehaveparentrevisions and p2rev != nullrev: > +baserev = p2rev > +# revision is a root, try our luck against the previous revision > I'm worried this will kill performance on non-generaldelta revlogs. Yes, this change is for the case where storage has a fulltext. So we need to resolve the baserev's fulltext regardless. But since revlogs cache the last fulltext and the last fulltext has a good chance of being "prev," this change might take away some cache hits. Could you please measure the impact of bundling performance on a non-generaldelta repo? Preferably one with a lot of merges, like Netbeans. > elif prevrev is not None: > baserev = prevrev > else: > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel > ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4928: sqlitestore: file storage backend using SQLite
indygreg added a subscriber: lothiraldan. indygreg added a comment. This patch will be invalidated by @lothiraldan's changegroup series, which I'm reviewing right now. The rebase should be trivial. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4928 To: indygreg, #hg-reviewers Cc: lothiraldan, mjpieters, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5119: exchange: pass includepats and excludepats as arguments to getbundle()
pulkit created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This will help in implementing narrow stream clone. The narrow extension does add these arguments, but we want to move narrow bits to core and this will help in it. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5119 AFFECTED FILES mercurial/exchange.py mercurial/wireprototypes.py tests/test-clonebundles.t tests/test-http-bad-server.t tests/test-http-proxy.t tests/test-http.t tests/test-lfs-serve-access.t tests/test-ssh.t tests/test-subrepo-recursion.t CHANGE DETAILS diff --git a/tests/test-subrepo-recursion.t b/tests/test-subrepo-recursion.t --- a/tests/test-subrepo-recursion.t +++ b/tests/test-subrepo-recursion.t @@ -343,23 +343,29 @@ $LOCALIP - - [$LOGDATE$] "GET /?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=0 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) - $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) + $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - +x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1===1326fa26d0c00d2146c63b56bb6a45149d7325ac==bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D1326fa26d0c00d2146c63b56bb6a45149d7325ac x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) - $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=1326fa26d0c00d2146c63b56bb6a45149d7325ac=1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) + $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - +x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=1326fa26d0c00d2146c63b56bb6a45149d7325ac==1326fa26d0c00d2146c63b56bb6a45149d7325ac==bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=0 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) - $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) + $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - +x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1===65903cebad86f1a84bd4f1134f62fa7dcb7a1c98==bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D65903cebad86f1a84bd4f1134f62fa7dcb7a1c98 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) - $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) + $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - +x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=65903cebad86f1a84bd4f1134f62fa7dcb7a1c98==65903cebad86f1a84bd4f1134f62fa7dcb7a1c98==bookmarks=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=0 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=capabilities HTTP/1.1" 200 - (glob) $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob) - $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=getbundle HTTP/1.1" 200 -
D5117: storageutil: convert fileid to bytes to avoid cast to %s
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6994a8be3663: storageutil: convert fileid to bytes to avoid
cast to %s (authored by indygreg, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5117?vs=12165=12169
REVISION DETAIL
https://phab.mercurial-scm.org/D5117
AFFECTED FILES
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -182,7 +182,8 @@
try:
return store.node(fileid)
except IndexError:
-raise error.LookupError(fileid, identifier, _('no match found'))
+raise error.LookupError('%d' % fileid, identifier,
+_('no match found'))
if len(fileid) == 20:
try:
To: indygreg, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5114: tests: use byte literals in test-storage.py
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9b2e1b00ee94: tests: use byte literals in test-storage.py
(authored by indygreg, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5114?vs=12162=12168
REVISION DETAIL
https://phab.mercurial-scm.org/D5114
AFFECTED FILES
tests/test-storage.py
CHANGE DETAILS
diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -30,7 +30,7 @@
return fl
def maketransaction(self):
-vfsmap = {'plain': STATE['vfs'], 'store': STATE['vfs']}
+vfsmap = {b'plain': STATE['vfs'], b'store': STATE['vfs']}
return transaction.transaction(STATE['ui'].warn, STATE['vfs'], vfsmap,
b'journal', b'undo')
To: indygreg, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[Bug 6001] New: Performance regression on bundle in 4.7
https://bz.mercurial-scm.org/show_bug.cgi?id=6001 Bug ID: 6001 Summary: Performance regression on bundle in 4.7 Product: Mercurial Version: default branch Hardware: PC OS: Linux Status: UNCONFIRMED Severity: bug Priority: wish Component: Mercurial Assignee: bugzi...@mercurial-scm.org Reporter: lothiral...@gmail.com CC: mercurial-devel@mercurial-scm.org Our automatic tooling has detected a regression that impact both bundle http://perf.octobus.net/#basic_commands.BundleTimeTestSuite.time_bundle?branch=default=4971c972-98995b68=bce1c1af-41fcdfe3='netbeans-2018-08-01'=1000 and incoming with http or ssh http://perf.octobus.net/#basic_commands.ExchangeTimeSuite.time_incoming?branch=default=bce1c1af-41fcdfe3='netbeans-2018-08-01'_type='ssh'='last-thousand'=None and push http://perf.octobus.net/#basic_commands.PushPullTimeSuite.time_push?branch=default=bce1c1af-41fcdfe3='netbeans-2018-08-01'_type='local'='last-thousand'=None for the netbeans repository. Bisection have identified https://www.mercurial-scm.org/repo/hg/rev/db5501d9 as the main responsible for the regression. Other repositories doesn't seems impacted. The regression has been reproduced locally. -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4928: sqlitestore: file storage backend using SQLite
indygreg updated this revision to Diff 12167.
indygreg edited the summary of this revision.
indygreg retitled this revision from "sqlitestore: file storage backend using
SQLite (RFC)" to "sqlitestore: file storage backend using SQLite".
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4928?vs=11770=12167
REVISION DETAIL
https://phab.mercurial-scm.org/D4928
AFFECTED FILES
hgext/sqlitestore.py
tests/hghave.py
tests/test-storage.py
CHANGE DETAILS
diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -17,6 +17,16 @@
storage as storagetesting,
)
+from hgext import (
+sqlitestore,
+)
+
+try:
+from mercurial import zstd
+zstd.__version__
+except ImportError:
+zstd = None
+
STATE = {
'lastindex': 0,
'ui': uimod.ui(),
@@ -70,5 +80,42 @@
maketransaction,
addrawrevision)
+def makesqlitefile(self):
+path = STATE['vfs'].join(b'db-%d.db' % STATE['lastindex'])
+STATE['lastindex'] += 1
+
+db = sqlitestore.makedb(path)
+
+compression = b'zstd' if zstd else b'zlib'
+
+return sqlitestore.sqlitefilestore(db, b'dummy-path', compression)
+
+def addrawrevisionsqlite(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
+ delta=None, censored=False, ellipsis=False,
+ extstored=False):
+flags = 0
+
+if censored:
+flags |= sqlitestore.FLAG_CENSORED
+
+if ellipsis | extstored:
+raise error.Abort(b'support for ellipsis and extstored flags not '
+ b'supported')
+
+if rawtext is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2, flags=flags)
+elif delta is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2,
+ storedelta=delta, flags=flags)
+else:
+raise error.Abort(b'must supply rawtext or delta arguments')
+
+sqlitefileindextests = storagetesting.makeifileindextests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefiledatatests = storagetesting.makeifiledatatests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefilemutationtests = storagetesting.makeifilemutationtests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+
if __name__ == '__main__':
silenttestrunner.main(__name__)
diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -787,6 +787,16 @@
def has_repofncache():
return 'fncache' in getrepofeatures()
+@check('sqlite', 'sqlite3 module is available')
+def has_sqlite():
+try:
+import sqlite3
+sqlite3.sqlite_version
+except ImportError:
+return False
+
+return matchoutput('sqlite3 -version', b'^3\.\d+')
+
@check('vcr', 'vcr http mocking library')
def has_vcr():
try:
diff --git a/hgext/sqlitestore.py b/hgext/sqlitestore.py
new file mode 100644
--- /dev/null
+++ b/hgext/sqlitestore.py
@@ -0,0 +1,1113 @@
+# sqlitestore.py - Storage backend that uses SQLite
+#
+# Copyright 2018 Gregory Szorc
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+"""store repository data in SQLite (EXPERIMENTAL)
+
+The sqlitestore extension enables the storage of repository data in SQLite.
+
+This extension is HIGHLY EXPERIMENTAL. There are NO BACKWARDS COMPATIBILITY
+GUARANTEES. This means that repositories created with this extension may
+only be usable with the exact version of this extension/Mercurial that was
+used. The extension attempts to enforce this in order to prevent repository
+corruption.
+
+In addition, several features are not yet supported or have known bugs:
+
+* Only some data is stored in SQLite. Changeset, manifest, and other repository
+ data is not yet stored in SQLite.
+* Transactions are not robust. If the process is aborted at the right time
+ during transaction close/rollback, the repository could be in an inconsistent
+ state. This problem will diminish once all repository data is tracked by
+ SQLite.
+* Bundle repositories do not work (the ability to use e.g.
+ `hg -R log` to automatically overlay a bundle on top of the
+ existing repository).
+* Various other features don't work.
+
+This extension should work for basic clone/pull, update, and commit workflows.
+Some history rewriting operations may fail due to lack of support for bundle
+repositories.
+
+To use, activate the extension and set the ``storage.new-repo-backend`` config
+option to ``sqlite`` to enable new repositories to use SQLite for storage.
+"""
+
+# To run the test suite with repos using SQLite by default, execute the
+# following:
+#
+# HGREPOFEATURES="sqlitestore" run-tests.py \
+# --extra-config-opt extensions.sqlitestore= \
+#
D5118: storageutil: extract most of peek_censored from revlog
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This function is super hacky and isn't correct 100% of the time. I'm going
to need this functionality on a future non-revlog store.
Let's copy things to storageutil so this code only exists once.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5118
AFFECTED FILES
mercurial/revlog.py
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -9,6 +9,7 @@
import hashlib
import re
+import struct
from ..i18n import _
from ..node import (
@@ -449,3 +450,31 @@
delta=delta)
prevrev = rev
+
+def deltaiscensored(delta, baserev, baselenfn):
+"""Determine if a delta represents censored revision data.
+
+``baserev`` is the base revision this delta is encoded against.
+``baselenfn`` is a callable receiving a revision number that resolves the
+length of the revision fulltext.
+
+Returns a bool indicating if the result of the delta represents a censored
+revision.
+"""
+# Fragile heuristic: unless new file meta keys are added alphabetically
+# preceding "censored", all censored revisions are prefixed by
+# "\1\ncensored:". A delta producing such a censored revision must be a
+# full-replacement delta, so we inspect the first and only patch in the
+# delta for this prefix.
+hlen = struct.calcsize(">lll")
+if len(delta) <= hlen:
+return False
+
+oldlen = baselenfn(baserev)
+newlen = len(delta) - hlen
+if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
+return False
+
+add = "\1\ncensored:"
+addlen = len(add)
+return newlen >= addlen and delta[hlen:hlen + addlen] == add
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2109,23 +2109,7 @@
if not self._censorable:
return False
-# Fragile heuristic: unless new file meta keys are added alphabetically
-# preceding "censored", all censored revisions are prefixed by
-# "\1\ncensored:". A delta producing such a censored revision must be a
-# full-replacement delta, so we inspect the first and only patch in the
-# delta for this prefix.
-hlen = struct.calcsize(">lll")
-if len(delta) <= hlen:
-return False
-
-oldlen = self.rawsize(baserev)
-newlen = len(delta) - hlen
-if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
-return False
-
-add = "\1\ncensored:"
-addlen = len(add)
-return newlen >= addlen and delta[hlen:hlen + addlen] == add
+return storageutil.deltaiscensored(delta, baserev, self.rawsize)
def getstrippoint(self, minlink):
"""find the minimum rev that must be stripped to strip the linkrev
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5116: testing: switch to inserting deltas
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
As the comment in the test specifies, this was relying on storage backend
implementation details. We switch to inserting a raw delta, skipping the
regular insert path to ensure we have the desired outcome. This required
implementing support for handling deltas in the revlog testing code.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5116
AFFECTED FILES
mercurial/testing/storage.py
tests/test-storage.py
CHANGE DETAILS
diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -49,7 +49,8 @@
if rawtext is not None:
fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags)
elif delta is not None:
-raise error.Abort('support for storing raw deltas not yet supported')
+fl._revlog.addrawrevision(rawtext, tr, linkrev, p1, p2, node, flags,
+ cachedelta=delta)
else:
raise error.Abort('must supply rawtext or delta arguments')
diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py
--- a/mercurial/testing/storage.py
+++ b/mercurial/testing/storage.py
@@ -952,13 +952,12 @@
with self.assertRaises(error.StorageError):
f.read(node1)
-diff = mdiff.textdiff(fulltext1, fulltext2)
node2 = storageutil.hashrevisionsha1(fulltext2, node1, nullid)
-deltas = [(node2, node1, nullid, b'\x01' * 20, node1, diff, 0)]
-# This /might/ fail on some backends.
with self._maketransactionfn() as tr:
-f.addgroup(deltas, lambda x: 0, tr)
+delta = mdiff.textdiff(fulltext1, fulltext2)
+self._addrawrevisionfn(f, tr, node2, node1, nullid,
+ 2, delta=(1, delta))
self.assertEqual(len(f), 3)
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5117: storageutil: convert fileid to bytes to avoid cast to %s
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
test-storage.py manages to trigger this on Python 3.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5117
AFFECTED FILES
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -182,7 +182,8 @@
try:
return store.node(fileid)
except IndexError:
-raise error.LookupError(fileid, identifier, _('no match found'))
+raise error.LookupError('%d' % fileid, identifier,
+_('no match found'))
if len(fileid) == 20:
try:
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5115: testing: remove expectation of error on bad node insert
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY addgroup() doesn't necessarily validate the hashes of each incoming revision. This is an optimization that allows delta group application to complete faster. The fact that revlog raises in this particular test is an implementation detail due to the way revlogs are testing multiple deltas. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5115 AFFECTED FILES mercurial/testing/storage.py CHANGE DETAILS diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py --- a/mercurial/testing/storage.py +++ b/mercurial/testing/storage.py @@ -1073,14 +1073,7 @@ fulltext0 = b'foo' delta0 = mdiff.trivialdiffheader(len(fulltext0)) + fulltext0 -deltas = [ -(b'\x01' * 20, nullid, nullid, nullid, nullid, delta0, 0), -] - with self._maketransactionfn() as tr: -with self.assertRaises(error.StorageError): -f.addgroup(deltas, linkmapper, tr, addrevisioncb=cb) - node0 = f.add(fulltext0, None, tr, 0, nullid, nullid) f = self._makefilefn() To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5114: tests: use byte literals in test-storage.py
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This fixes a Python 3 breakage due to unknown key due to str/bytes type
mismatch.
1. skip-blame just b'' literals
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5114
AFFECTED FILES
tests/test-storage.py
CHANGE DETAILS
diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -30,7 +30,7 @@
return fl
def maketransaction(self):
-vfsmap = {'plain': STATE['vfs'], 'store': STATE['vfs']}
+vfsmap = {b'plain': STATE['vfs'], b'store': STATE['vfs']}
return transaction.transaction(STATE['ui'].warn, STATE['vfs'], vfsmap,
b'journal', b'undo')
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4820: logtoprocess: sends the canonical command name to the subprocess
lothiraldan updated this revision to Diff 12161.
lothiraldan edited the summary of this revision.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4820?vs=11521=12161
REVISION DETAIL
https://phab.mercurial-scm.org/D4820
AFFECTED FILES
hgext/logtoprocess.py
mercurial/dispatch.py
CHANGE DETAILS
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -66,6 +66,9 @@
# low-level repo state (for example, changelog) before extensions.
self.prereposetups = prereposetups or []
+# store the parsed and canonical command
+self.canonical_command = None
+
def _runexithandlers(self):
exc = None
handlers = self.ui._exithandlers
@@ -243,7 +246,8 @@
req.ui.log('uiblocked', 'ui blocked ms',
**pycompat.strkwargs(req.ui._blockedtimes))
req.ui.log("commandfinish", "%s exited %d after %0.2f seconds\n",
- msg, ret & 255, duration)
+ msg, ret & 255, duration,
+ canonical_command=req.canonical_command)
try:
req._runexithandlers()
except: # exiting, so no re-raises
@@ -853,6 +857,9 @@
fullargs = args
cmd, func, args, options, cmdoptions = _parse(lui, args)
+# store the canonical command name in request object for later access
+req.canonical_command = cmd
+
if options["config"] != req.earlyoptions["config"]:
raise error.Abort(_("option --config may not be abbreviated!"))
if options["cwd"] != req.earlyoptions["cwd"]:
diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -40,6 +40,9 @@
import sys
from mercurial import (
+dispatch,
+encoding,
+extensions,
pycompat,
)
@@ -110,6 +113,12 @@
script = self.config('logtoprocess', event)
if script:
if msg:
+
+# extract the canonical command from the passed opts
+canonical_command = opts.pop("canonical_command", None)
+if canonical_command is None:
+canonical_command = ""
+
# try to format the log message given the remaining
# arguments
try:
@@ -134,7 +143,8 @@
for key, value in opts.iteritems())
env = dict(itertools.chain(procutil.shellenviron().items(),
msgpairs, optpairs),
- EVENT=event, HGPID=str(os.getpid()))
+ EVENT=event, HGPID=str(os.getpid()),
+ LTP_COMMAND=canonical_command)
runshellcommand(script, env)
return super(logtoprocessui, self).log(event, *msg, **opts)
To: lothiraldan, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 6] py3: convert string literals to bytes in contrib/hgclient.py
On Tue, Oct 16, 2018 at 10:28 AM Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1539666041 -7200 > # Tue Oct 16 07:00:41 2018 +0200 > # Node ID 0001bda2e00c73f9f82591d3ce7b44f552d0b936 > # Parent da1629c7dda17853c8cbaad0ef14a899af6e9361 > py3: convert string literals to bytes in contrib/hgclient.py > > # skip-blame just many b'' prefixes > Queued this series, many thanks! ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5113: revisions: when using prefixhexnode, ensure we prefix "0"
spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
Previously, if using `experimental.revisions.disambiguatewithin` (and it
didn't
include rev0), and '0' was the shortest identifier in that disambiguation set,
we printed it as the shortest *without* a prefix. This was because we had
logic
to determine "if the prefix is a pure integer, but starts with 0, we don't
need
to prefix with 'x': 01 is not a synonym for revision #1", but didn't handle
the
case where prefix == 0 (which is a pure integer, and starts with 0... but it
*is* "rev0").
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5113
AFFECTED FILES
mercurial/scmutil.py
tests/test-template-functions.t
CHANGE DETAILS
diff --git a/tests/test-template-functions.t b/tests/test-template-functions.t
--- a/tests/test-template-functions.t
+++ b/tests/test-template-functions.t
@@ -909,6 +909,55 @@
$ cd ..
+Test prefixhexnode when the first character of the hash is 0.
+ $ hg init hashcollision2
+ $ cd hashcollision2
+ $ cat <> .hg/hgrc
+ > [experimental]
+ > evolution.createmarkers=True
+ > EOF
+ $ echo 0 > a
+ $ hg ci -qAm 0
+ $ echo 21 > a
+ $ hg ci -qm 21
+ $ hg up -q null
+ $ hg log -r0: -T '{rev}:{node}\n'
+ 0:b4e73ffab476aa0ee32ed81ca51e07169844bc6a
+ 1:0cf177ba2b1dc3862a00fb81715fec90950201be
+
+ we need the 'x' prefix to ensure we aren't colliding with rev0. We *do*
+ identify the collision if we aren't using disambiguatewithin, so we need to
set
+ that as well.
+ $ hg --config experimental.revisions.disambiguatewithin='descendants(0)' \
+ >--config experimental.revisions.prefixhexnode=yes \
+ >log -r 1 -T '{rev}:{shortest(node, 0)}\n'
+ 1:x0
+
+ $ hg debugobsolete 0cf177ba2b1dc3862a00fb81715fec90950201be
+ obsoleted 1 changesets
+ $ hg up -q 0
+ $ echo 61 > a
+ $ hg ci -m 61
+ $ hg log -r0: -T '{rev}:{node}\n'
+ 0:b4e73ffab476aa0ee32ed81ca51e07169844bc6a
+ 2:01384dde84b3a511ae0835f35ac40bd806c99bb8
+
+ we still have the 'x' prefix because '0' is still the shortest prefix, since
+ rev1's '0c' is hidden.
+ $ hg --config experimental.revisions.disambiguatewithin=0:-1-0 \
+ >--config experimental.revisions.prefixhexnode=yes \
+ >log -r 0:-1-0 -T '{rev}:{shortest(node, 0)}\n'
+ 2:x0
+
+ we don't have the 'x' prefix on 2 because '01' is not a synonym for rev1.
+ $ hg --config experimental.revisions.disambiguatewithin=0:-1-0 \
+ >--config experimental.revisions.prefixhexnode=yes \
+ >log -r 0:-1-0 -T '{rev}:{shortest(node, 0)}\n' --hidden
+ 1:0c
+ 2:01
+
+ $ cd ..
+
Test pad function
$ cd r
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -476,8 +476,9 @@
i = int(prefix)
# if we are a pure int, then starting with zero will not be
# confused as a rev; or, obviously, if the int is larger
-# than the value of the tip rev
-if prefix[0:1] == b'0' or i >= len(repo):
+# than the value of the tip rev. We still need to disambiguate if
+# prefix == '0', since that *is* a valid revnum.
+if (prefix != b'0' and prefix[0:1] == b'0') or i >= len(repo):
return False
return True
except ValueError:
To: spectral, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4931: narrow: only send the narrowspecs back if ACL in play
pulkit added a comment. @martinvonz Looks like this got missed for discussing in sprint, can you have a look at this please? This is important because once we have narrowspecs greater than 255 length and and don't use ellipses, we hit https://bz.mercurial-scm.org/show_bug.cgi?id=5952 REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4931 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5102: beautifygraph: add configs for customizing the characters
spectral added a comment. hooper and I talked at the sprint, and they wrote up the following for why they feel we should keep this out of graphmod/templatekw for now: > I don't think configurable graph log characters in core is a great idea right now, since it makes the empty promise that you can make logs look correct by using common fonts. E.g. the problem that pipe characters are used in several situations that just can't be handled well by one character. As a specific example, there's no unicode line drawing characters that are fully-extended lines at 0 degrees (straight up) and 135 degrees (down to the right to the corner of the character box), so there will always be cases that can't really be represented "perfectly". I don't generally agree with this reasoning - limitations of unicode should not be a reason to require that we go through a separate extension like this. The reasoning for the extension instead of just making the actual graph characters customizable was two-fold, if I remember correctly: 1. there is currently some awkward code to handle extending one of the characters used to draw parent/grandparent lines for a few lines and then switch to the other. (https://www.mercurial-scm.org/repo/hg/file/tip/tests/test-glog.t#l3041 shows this in action) - the code to implement this is not unicode safe (at least in python2) 2. the current graphlog code does not currently have config knobs to control every character, I think. I think we should fix the code issue in #1 (I just sent https://phab.mercurial-scm.org/D5112 for this), and then we can extended graphlog to have whatever character(s) we need, instead of attempting to do the replacements in an extension like we're currently doing? The extension could just become a set of "recommended" values for these config knobs. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5102 To: hooper, #hg-reviewers Cc: spectral, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5112: graphmod: remove support for graph lines mixing parent/grandparent styles (BC)
spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
Currently, if the configuration for a graph edge draw style has multiple bytes
(at least on python2), it is interpreted as "this is a request to draw the
line
partially in the style of the parent, partially in the style of the
grandparent". This precludes the configuration handling unicode characters
(which trigger the `len > 1` check, at least on python2), and I believe was
part
of the reason that beautifygraph was written the way it was.
Talking with the person who implemented this, it appears to have been to
achieve
feature parity with the rendering of the smartlog extension. I suspect that
this
isn't actually used outside of that situation, so I think that we can remove
it
without much issue.
This will make it so that multi-character edges are possible, and render any
existing configuration that uses this feature with these multiple characters.
This is *not* going to adjust the width of everything to make it line up
correctly, please see the test that's being modified in this changeset for an
example of how the previous configuration now renders.
Note also that the previous configuration seems to have been broken, or at
least
it was behaving in a really non-obvious way - it was avoiding the grandparent
character(s) when it should have been displaying them! This is why so many "!"
characters changed to "3."; I don't know if this was intentional.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5112
AFFECTED FILES
mercurial/graphmod.py
tests/test-glog.t
CHANGE DETAILS
diff --git a/tests/test-glog.t b/tests/test-glog.t
--- a/tests/test-glog.t
+++ b/tests/test-glog.t
@@ -3028,12 +3028,14 @@
date:Thu Jan 01 00:00:04 1970 +
summary: (4) merge two known; one immediate left, one immediate
right
-Draw only part of a grandparent line differently with ""; only the
-last N lines (for positive N) or everything but the first N lines (for
-negative N) along the current node use the style, the rest of the edge uses
-the parent edge styling.
+Previously, one could specify graphstyle.grandparent = to draw
+on only the last N lines (for positive N) or everything but the first N lines
+(for negative N), with the rest of the edge using the parent edge styling.
-Last 3 lines:
+This was removed, and this test now shows that muliple characters being
+specified in graphstyle.grandparent aren't treated specially (including in
width
+calculations; there's no specific reason to *avoid* handling the width
+calculations, but it's difficult to do correctly and efficiently).
$ cat << EOF >> $HGRCPATH
> [experimental]
@@ -3043,77 +3045,77 @@
> EOF
$ hg log -G -r '36:18 & file("a")' -m
@ changeset: 36:08a19a744424
- ! branch: branch
- ! tag: tip
- ! parent: 35:9159c3644c5e
- ! parent: 35:9159c3644c5e
- ! user:test
- . date:Thu Jan 01 00:00:36 1970 +
- . summary: (36) buggy merge: identical parents
- .
+ 3. branch: branch
+ 3. tag: tip
+ 3. parent: 35:9159c3644c5e
+ 3. parent: 35:9159c3644c5e
+ 3. user:test
+ 3. date:Thu Jan 01 00:00:36 1970 +
+ 3. summary: (36) buggy merge: identical parents
+ 3.
ochangeset: 32:d06dffa21a31
!\ parent: 27:886ed638191b
- ! ! parent: 31:621d83e11f67
- ! ! user:test
- ! . date:Thu Jan 01 00:00:32 1970 +
- ! . summary: (32) expand
- ! .
- o ! changeset: 31:621d83e11f67
- !\! parent: 21:d42a756af44d
- ! ! parent: 30:6e11cd4b648f
- ! ! user:test
- ! ! date:Thu Jan 01 00:00:31 1970 +
- ! ! summary: (31) expand
- ! !
- o !changeset: 30:6e11cd4b648f
+ ! 3. parent: 31:621d83e11f67
+ ! 3. user:test
+ ! 3. date:Thu Jan 01 00:00:32 1970 +
+ ! 3. summary: (32) expand
+ ! 3.
+ o 3. changeset: 31:621d83e11f67
+ !\3. parent: 21:d42a756af44d
+ ! 3. parent: 30:6e11cd4b648f
+ ! 3. user:test
+ ! 3. date:Thu Jan 01 00:00:31 1970 +
+ ! 3. summary: (31) expand
+ ! 3.
+ o 3. changeset: 30:6e11cd4b648f
!\ \ parent: 28:44ecd0b9ae99
- ! ~ ! parent: 29:cd9bb2be7593
- ! ! user:test
- ! ! date:Thu Jan 01 00:00:30 1970 +
- ! ! summary: (30) expand
+ ! ~ 3. parent: 29:cd9bb2be7593
+ ! 3. user:test
+ ! 3. date:Thu Jan 01 00:00:30 1970 +
+ ! 3. summary: (30) expand
! /
- o !changeset: 28:44ecd0b9ae99
+ o 3. changeset: 28:44ecd0b9ae99
!\ \ parent: 1:6db2ef61d156
- ! ~ ! parent: 26:7f25b6c2f0b9
- ! ! user:test
- ! ! date:Thu Jan 01 00:00:28 1970 +
- ! ! summary: (28) merge
[Bug 6000] New: ui.tweakdefaults needs to be better documented
https://bz.mercurial-scm.org/show_bug.cgi?id=6000 Bug ID: 6000 Summary: ui.tweakdefaults needs to be better documented Product: Mercurial Version: default branch Hardware: PC OS: Mac OS Status: UNCONFIRMED Severity: feature Priority: wish Component: Mercurial Assignee: bugzi...@mercurial-scm.org Reporter: duri...@gmail.com CC: mercurial-devel@mercurial-scm.org We need to make it easier to figure out what ui.tweakdefaults is doing, and how to reverse any parts of it the user finds objectionable. -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@40312: 4 new changesets
4 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/1bd3e922de18 changeset: 40309:1bd3e922de18 user:Yuya Nishihara date:Tue Oct 16 12:39:21 2018 +0200 summary: check-commit: update test expectation per removal of "double empty line" rule https://www.mercurial-scm.org/repo/hg/rev/d894d2372ffe changeset: 40310:d894d2372ffe user:Boris Feld date:Thu Oct 11 03:13:53 2018 +0200 summary: revset: document the `revset(...)` syntax https://www.mercurial-scm.org/repo/hg/rev/4aa04d009167 changeset: 40311:4aa04d009167 user:Boris Feld date:Thu Oct 11 03:15:04 2018 +0200 summary: revset: drop special case of 'revset(...)' function in analyze https://www.mercurial-scm.org/repo/hg/rev/5644f7c8982e changeset: 40312:5644f7c8982e bookmark:@ tag: tip user:Martijn Pieters date:Fri Aug 31 19:58:41 2018 +0100 summary: branchmap: remove redundant sort -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5111: branchmap: remove redundant sort
This revision was automatically updated to reflect the committed changes. Closed by commit rHG5644f7c8982e: branchmap: remove redundant sort (authored by mjpieters, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5111?vs=12157=12158 REVISION DETAIL https://phab.mercurial-scm.org/D5111 AFFECTED FILES mercurial/branchmap.py CHANGE DETAILS diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py --- a/mercurial/branchmap.py +++ b/mercurial/branchmap.py @@ -296,7 +296,6 @@ # This have been tested True on all internal usage of this function. # run it again in case of doubt # assert not (set(bheadrevs) & set(newheadrevs)) -newheadrevs.sort() bheadset.update(newheadrevs) # This prunes out two kinds of heads - heads that are superseded by To: mjpieters, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2] revset: document the `revset(...)` syntax
On Tue, 16 Oct 2018 12:09:01 +0200, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1539220433 -7200
> # Thu Oct 11 03:13:53 2018 +0200
> # Node ID 16ae7c19c2a2bb89ad4d201630cd87f6c635b7e0
> # Parent a66594c5fad422499bfaf669d182e02fe3c0be32
> # EXP-Topic revset-doc
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 16ae7c19c2a2
> revset: document the `revset(...)` syntax
Queued, thanks.
> +@predicate('revset(set)', safe=True, takeorder=True)
> +def revsetpredicate(repo, subset, x, order):
> +"""Strictly interpret the content as a revset.
> +
> +The content of this special predicate will be strictly interpreted as a
> +revset. For example ``revset(id(0))`` will interpret "id(0)" without
> +possible ambiguity with a possible "id(0)" bookmark or tag.
I've removed one of these "possible"s, and done minor tweaks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5111: branchmap: remove redundant sort
mjpieters created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY There is absoluty no benefit in sorting a list that's being merged into a set on the next line. The changelog.ancestors() call later on also doesn't benefit from a sorted sequence of revs. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5111 AFFECTED FILES mercurial/branchmap.py CHANGE DETAILS diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py --- a/mercurial/branchmap.py +++ b/mercurial/branchmap.py @@ -296,7 +296,6 @@ # This have been tested True on all internal usage of this function. # run it again in case of doubt # assert not (set(bheadrevs) & set(newheadrevs)) -newheadrevs.sort() bheadset.update(newheadrevs) # This prunes out two kinds of heads - heads that are superseded by To: mjpieters, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@40308: new changeset
New changeset in mercurial: https://www.mercurial-scm.org/repo/hg/rev/47084b5ffd80 changeset: 40308:47084b5ffd80 bookmark:@ tag: tip user:Martijn Pieters date:Sun Oct 14 15:40:16 2018 +0200 summary: style: drop requirement to only use single lines between top-level objects -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] revset: drop special case of 'revset(...)' function in analyze
# HG changeset patch
# User Boris Feld
# Date 1539220504 -7200
# Thu Oct 11 03:15:04 2018 +0200
# Node ID c1241e62986a91ebaf5a421c6d78560113be7111
# Parent 16ae7c19c2a2bb89ad4d201630cd87f6c635b7e0
# EXP-Topic revset-doc
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
c1241e62986a
revset: drop special case of 'revset(...)' function in analyze
We now have a valid no-op function. We no longer need the special case.
diff --git a/mercurial/revsetlang.py b/mercurial/revsetlang.py
--- a/mercurial/revsetlang.py
+++ b/mercurial/revsetlang.py
@@ -355,9 +355,6 @@ def _analyze(x):
elif op == 'keyvalue':
return (op, x[1], _analyze(x[2]))
elif op == 'func':
-f = getsymbol(x[1])
-if f == 'revset':
-return _analyze(x[2])
return (op, x[1], _analyze(x[2]))
raise ValueError('invalid operator %r' % op)
diff --git a/tests/test-revset.t b/tests/test-revset.t
--- a/tests/test-revset.t
+++ b/tests/test-revset.t
@@ -2926,16 +2926,20 @@ test multiline revset with errors
(symbol '0'
* analyzed:
(func
-(symbol 'first')
+(symbol 'revset')
(func
- (symbol 'rev')
- (symbol '0')))
+ (symbol 'first')
+ (func
+(symbol 'rev')
+(symbol '0'
* optimized:
(func
-(symbol 'first')
+(symbol 'revset')
(func
- (symbol 'rev')
- (symbol '0')))
+ (symbol 'first')
+ (func
+(symbol 'rev')
+(symbol '0'
* set:
0
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2] revset: document the `revset(...)` syntax
# HG changeset patch
# User Boris Feld
# Date 1539220433 -7200
# Thu Oct 11 03:13:53 2018 +0200
# Node ID 16ae7c19c2a2bb89ad4d201630cd87f6c635b7e0
# Parent a66594c5fad422499bfaf669d182e02fe3c0be32
# EXP-Topic revset-doc
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
16ae7c19c2a2
revset: document the `revset(...)` syntax
We introduce a new "no-op" function to bear the documentation. In practice, the
parsing step is skipping it so it is not even called. This will get fixed in
the next changeset.
diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -1751,6 +1751,16 @@ def rev(repo, subset, x):
return baseset()
return subset & baseset([l])
+@predicate('revset(set)', safe=True, takeorder=True)
+def revsetpredicate(repo, subset, x, order):
+"""Strictly interpret the content as a revset.
+
+The content of this special predicate will be strictly interpreted as a
+revset. For example ``revset(id(0))`` will interpret "id(0)" without
+possible ambiguity with a possible "id(0)" bookmark or tag.
+"""
+return getset(repo, subset, x, order)
+
@predicate('matching(revision [, field])', safe=True)
def matching(repo, subset, x):
"""Changesets in which a given set of fields match the set of fields in the
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5094: merge-tools: when calling external merge tool, describe the resolve inputs
yuja added a comment.
> +def _describemerge(ui, repo, env, toolpath, args):
> +template = ui.config('ui', 'pre-merge-tool-output-template')
> +if not template:
> +return
> +
> +# Remove HG_ prefix from entries in `env` and lowercase them
> +def sanitizeenv(k):
> +if k.startswith('HG_'):
> +return k[3:].lower()
> +return k
Nit: `HG_FILE` should be called as `path` per
https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary
We might want to structure these variables as `base.node|short` instead of
`base_node` for example, but that would require more work.
> +data = {sanitizeenv(k): v for k, v in env.items()}
> +data['toolpath'] = toolpath
> +data['toolargs'] = args
> +
> +# TODO: make all of this something that can be specified on a
per-tool basis
> +template = templater.unquotestring(template)
> +
> +fm = ui.formatter("extmerge", pycompat.byteskwargs({'template':
template}))
Unnecessary. It's bytes dict.
> +fm.data(repo=repo, **data)
^^
Here pycompat.strkwargs() would be needed.
And use `fm.context(repo=repo)` or `fm.context(fctx=fcd)` since `repo` is
unprintable.
> +coreconfigitem('ui', 'pre-merge-tool-output-template',
> +default=None,
> +)
Can you update the help/config.txt as well?
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5094
To: spectral, #hg-reviewers
Cc: yuja, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5094: merge-tools: when calling external merge tool, describe the resolve inputs
> +def _describemerge(ui, repo, env, toolpath, args):
> +template = ui.config('ui', 'pre-merge-tool-output-template')
> +if not template:
> +return
> +
> +# Remove HG_ prefix from entries in `env` and lowercase them
> +def sanitizeenv(k):
> +if k.startswith('HG_'):
> +return k[3:].lower()
> +return k
Nit: `HG_FILE` should be called as `path` per
https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary
We might want to structure these variables as `base.node|short` instead of
`base_node` for example, but that would require more work.
> +data = {sanitizeenv(k): v for k, v in env.items()}
> +data['toolpath'] = toolpath
> +data['toolargs'] = args
> +
> +# TODO: make all of this something that can be specified on a per-tool
> basis
> +template = templater.unquotestring(template)
> +
> +fm = ui.formatter("extmerge", pycompat.byteskwargs({'template':
> template}))
Unnecessary. It's bytes dict.
> +fm.data(repo=repo, **data)
^^
Here pycompat.strkwargs() would be needed.
And use `fm.context(repo=repo)` or `fm.context(fctx=fcd)` since `repo` is
unprintable.
> +coreconfigitem('ui', 'pre-merge-tool-output-template',
> +default=None,
> +)
Can you update the help/config.txt as well?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 4] changegroup: allow to force delta to be against p1
# HG changeset patch
# User Boris Feld
# Date 1539123662 -7200
# Wed Oct 10 00:21:02 2018 +0200
# Node ID 153b15991f547ab7aa05e06999a8aa6ee56b7fc3
# Parent c835d3020e536e8ef368223d628b9e0c6d0c8251
# EXP-Topic slim-bundle
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
153b15991f54
changegroup: allow to force delta to be against p1
This new developer option is useful to general more "generic" bundle. Without
this option, a bundle generated from the repository use deltas similar to the
one stored in the specific repository it was generated from. This makes
performance testing a bit tricky.
Using deltas similar to the final result means all delta stored in the bundle
can be applied to the target repository without any further processing (except
for the rare case of a full snapshot). The application of such bundles
(almost) never exercises the (slower) path of searching for a new valid delta.
This result in unrealistic and too favorable timing and profile.
Instead, we introduce an option to make sure all revisions are stored as a
delta against p1. It might not be the best generation option, but it
guarantees that the content will be "generic", not favoring a specific target.
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -704,6 +704,8 @@ def deltagroup(repo, store, nodes, ischa
deltamode = storageutil.DELTAMODE_STD
if forcedeltaparentprev:
deltamode = storageutil.DELTAMODE_PREV
+elif repo.ui.configbool('devel', 'bundle.delta-p1'):
+deltamode = storageutil.DELTAMODE_PREV
revisions = store.emitrevisions(
nodes,
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -329,6 +329,9 @@ coreconfigitem('devel', 'all-warnings',
coreconfigitem('devel', 'bundle2.debug',
default=False,
)
+coreconfigitem('devel', 'bundle.delta-p1',
+default=False,
+)
coreconfigitem('devel', 'cache-vfs',
default=None,
)
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -268,6 +268,7 @@ def resolvestripinfo(minlinkrev, tiprev,
DELTAMODE_STD = 'default'
DELTAMODE_PREV = 'previous'
DELTAMODE_FULL = 'fulltext'
+DELTAMODE_P1 = 'p1'
def emitrevisions(store, nodes, nodesorder, resultcls, deltaparentfn=None,
candeltafn=None, rawsizefn=None, revdifffn=None,
flagsfn=None,
@@ -374,6 +375,9 @@ def emitrevisions(store, nodes, nodesord
# We're instructed to send fulltext. Honor that.
elif deltamode == DELTAMODE_FULL:
baserev = nullrev
+# We're instructed to use p1. Honor that
+elif deltamode == DELTAMODE_P1:
+baserev = p1rev
# There is a delta in storage. We try to use that because it
# amounts to effectively copying data from storage and is
diff --git a/tests/test-bundle.t b/tests/test-bundle.t
--- a/tests/test-bundle.t
+++ b/tests/test-bundle.t
@@ -900,3 +900,8 @@ the warning shouldn't be emitted
$ hg update -R ../update2bundled.hg -r 0
0 files updated, 0 files merged, 2 files removed, 0 files unresolved
#endif
+
+Test the option that create slim bundle
+
+ $ hg bundle -a --config devel.bundle.delta-p1=yes ./slim.hg
+ 3 changesets found
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 4] storage: also use `deltamode argument` for ifiledata
# HG changeset patch
# User Boris Feld
# Date 1539120395 -7200
# Tue Oct 09 23:26:35 2018 +0200
# Node ID c835d3020e536e8ef368223d628b9e0c6d0c8251
# Parent a075ac3487d6d266ec5f2dbd6901da21c38d4ed9
# EXP-Topic slim-bundle
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
c835d3020e53
storage: also use `deltamode argument` for ifiledata
Now that lower level uses such argument, we can propagate the change to higher
layers. At some of the higher level, we use `None` as the default value to
simplify imports and argument forwarding (instead of
`storageutil.DELTAMODE_STD`). Strictly speaking, we could import `storageutil`
everywhere, however, it did not seem to help readability.
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -29,6 +29,10 @@ from . import (
util,
)
+from .utils import (
+storageutil,
+)
+
_CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
_CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
_CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
@@ -697,12 +701,16 @@ def deltagroup(repo, store, nodes, ischa
progress = repo.ui.makeprogress(topic, unit=_('chunks'),
total=len(nodes))
+deltamode = storageutil.DELTAMODE_STD
+if forcedeltaparentprev:
+deltamode = storageutil.DELTAMODE_PREV
+
revisions = store.emitrevisions(
nodes,
nodesorder=nodesorder,
revisiondata=True,
assumehaveparentrevisions=not ellipses,
-deltaprevious=forcedeltaparentprev)
+deltamode=deltamode)
for i, revision in enumerate(revisions):
if progress:
diff --git a/mercurial/filelog.py b/mercurial/filelog.py
--- a/mercurial/filelog.py
+++ b/mercurial/filelog.py
@@ -77,11 +77,11 @@ class filelog(object):
def emitrevisions(self, nodes, nodesorder=None,
revisiondata=False, assumehaveparentrevisions=False,
- deltaprevious=False):
+ deltamode=None):
return self._revlog.emitrevisions(
nodes, nodesorder=nodesorder, revisiondata=revisiondata,
assumehaveparentrevisions=assumehaveparentrevisions,
-deltaprevious=deltaprevious)
+deltamode=deltamode)
def addrevision(self, revisiondata, transaction, linkrev, p1, p2,
node=None, flags=revlog.REVIDX_DEFAULT_FLAGS,
diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -1575,11 +1575,11 @@ class manifestrevlog(object):
def emitrevisions(self, nodes, nodesorder=None,
revisiondata=False, assumehaveparentrevisions=False,
- deltaprevious=False):
+ deltamode=None):
return self._revlog.emitrevisions(
nodes, nodesorder=nodesorder, revisiondata=revisiondata,
assumehaveparentrevisions=assumehaveparentrevisions,
-deltaprevious=deltaprevious)
+deltamode=deltamode)
def addgroup(self, deltas, linkmapper, transaction, addrevisioncb=None):
return self._revlog.addgroup(deltas, linkmapper, transaction,
diff --git a/mercurial/repository.py b/mercurial/repository.py
--- a/mercurial/repository.py
+++ b/mercurial/repository.py
@@ -602,7 +602,7 @@ class ifiledata(interfaceutil.Interface)
nodesorder=None,
revisiondata=False,
assumehaveparentrevisions=False,
- deltaprevious=False):
+ deltamode=None):
"""Produce ``irevisiondelta`` for revisions.
Given an iterable of nodes, emits objects conforming to the
@@ -645,10 +645,10 @@ class ifiledata(interfaceutil.Interface)
The ``linknode`` attribute on the returned ``irevisiondelta`` may not
be set and it is the caller's responsibility to resolve it, if needed.
-If ``deltaprevious`` is True and revision data is requested, all
-revision data should be emitted as deltas against the revision
-emitted just prior. The initial revision should be a delta against
-its 1st parent.
+If ``deltamode`` is storageutil.DELTAMODE_PREV and revision data is
+requested, all revision data should be emitted as deltas against the
+revision emitted just prior. The initial revision should be a delta
+against its 1st parent.
"""
class ifilemutation(interfaceutil.Interface):
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2221,7 +2221,8 @@ class revlog(object):
return res
def emitrevisions(self, nodes, nodesorder=None, revisiondata=False,
-
[PATCH 2 of 4] changegroup: refactor emitrevision to use a `deltamode` argument
# HG changeset patch # User Boris Feld # Date 1539115321 -7200 # Tue Oct 09 22:02:01 2018 +0200 # Node ID a075ac3487d6d266ec5f2dbd6901da21c38d4ed9 # Parent a69790aaac1946e80e35b28c8f41b14a7f6c0f64 # EXP-Topic slim-bundle # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r a075ac3487d6 changegroup: refactor emitrevision to use a `deltamode` argument This new argument gathers the semantic of `sendfulltext` and `deltaprevious` in a single value. We are about to introduce a new type of constraints. Avoiding yet another argument sounds like a plus. diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -2229,6 +2229,12 @@ class revlog(object): if nodesorder is None and not self._generaldelta: nodesorder = 'storage' +deltamode = storageutil.DELTAMODE_STD +if deltaprevious: +deltamode = storageutil.DELTAMODE_PREV +elif not self._storedeltachains: +deltamode = storageutil.DELTAMODE_FULL + return storageutil.emitrevisions( self, nodes, nodesorder, revlogrevisiondelta, deltaparentfn=self.deltaparent, @@ -2236,10 +2242,9 @@ class revlog(object): rawsizefn=self.rawsize, revdifffn=self.revdiff, flagsfn=self.flags, -sendfulltext=not self._storedeltachains, +deltamode=deltamode, revisiondata=revisiondata, -assumehaveparentrevisions=assumehaveparentrevisions, -deltaprevious=deltaprevious) +assumehaveparentrevisions=assumehaveparentrevisions) DELTAREUSEALWAYS = 'always' DELTAREUSESAMEREVS = 'samerevs' diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -265,11 +265,14 @@ def resolvestripinfo(minlinkrev, tiprev, return strippoint, brokenrevs +DELTAMODE_STD = 'default' +DELTAMODE_PREV = 'previous' +DELTAMODE_FULL = 'fulltext' + def emitrevisions(store, nodes, nodesorder, resultcls, deltaparentfn=None, candeltafn=None, rawsizefn=None, revdifffn=None, flagsfn=None, - sendfulltext=False, - revisiondata=False, assumehaveparentrevisions=False, - deltaprevious=False): + deltamode=DELTAMODE_STD, + revisiondata=False, assumehaveparentrevisions=False): """Generic implementation of ifiledata.emitrevisions(). Emitting revision data is subtly complex. This function attempts to @@ -320,14 +323,17 @@ def emitrevisions(store, nodes, nodesord Callable receiving a revision number and returns the integer flags value for it. If not defined, flags value will be 0. -``sendfulltext`` +``deltamode`` + constaint on delta to be sent: + * DELTAMODE_STD - normal mode, try to reuse storage deltas, + * DELTAMODE_PREV - only delta against "prev", + * DELTAMODE_FULL - only issue full snapshot. + Whether to send fulltext revisions instead of deltas, if allowed. ``nodesorder`` ``revisiondata`` ``assumehaveparentrevisions`` -``deltaprevious`` - See ``ifiledata.emitrevisions()`` interface documentation. """ fnode = store.node @@ -343,7 +349,7 @@ def emitrevisions(store, nodes, nodesord prevrev = None -if deltaprevious or assumehaveparentrevisions: +if deltamode == DELTAMODE_PREV or assumehaveparentrevisions: prevrev = store.parentrevs(revs[0])[0] # Set of revs available to delta against. @@ -362,11 +368,11 @@ def emitrevisions(store, nodes, nodesord deltaparentrev = nullrev # Forced delta against previous mode. -if deltaprevious: +if deltamode == DELTAMODE_PREV: baserev = prevrev # We're instructed to send fulltext. Honor that. -elif sendfulltext: +elif deltamode == DELTAMODE_FULL: baserev = nullrev # There is a delta in storage. We try to use that because it @@ -427,7 +433,7 @@ def emitrevisions(store, nodes, nodesord baserevisionsize = len(store.revision(baserev, raw=True)) -elif baserev == nullrev and not deltaprevious: +elif baserev == nullrev and not deltamode == DELTAMODE_PREV: revision = store.revision(node, raw=True) available.add(rev) else: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 4] changegroup: if possible fallback to delta against parents instead of prev
# HG changeset patch # User Boris Feld # Date 1539114828 -7200 # Tue Oct 09 21:53:48 2018 +0200 # Node ID a69790aaac1946e80e35b28c8f41b14a7f6c0f64 # Parent 43f0a37bd9ed20b45d73f667acbb0c3035492e1a # EXP-Topic slim-bundle # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r a69790aaac19 changegroup: if possible fallback to delta against parents instead of prev Deltas against parent are likely to be smaller. This is especially useful since we stop reordering changeset in the bundle. diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -394,9 +394,11 @@ def emitrevisions(store, nodes, nodesord baserev = nullrev # Storage has a fulltext revision. - -# Let's use the previous revision, which is as good a guess as any. -# There is definitely room to improve this logic. +elif assumehaveparentrevisions and p1rev != nullrev: +baserev = p1rev +elif assumehaveparentrevisions and p2rev != nullrev: +baserev = p2rev +# revision is a root, try our luck against the previous revision elif prevrev is not None: baserev = prevrev else: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@40307: 28 new changesets
28 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/170cd2a5a1da changeset: 40280:170cd2a5a1da user:Augie Fackler date:Sun Oct 14 04:11:35 2018 -0400 summary: fuzz: try *even harder* to prevent Python from looking up usernames https://www.mercurial-scm.org/repo/hg/rev/29b0e9cd02f4 changeset: 40281:29b0e9cd02f4 user:Mark Thomas date:Sun Oct 14 09:07:43 2018 + summary: py3: fix test-revlog.t https://www.mercurial-scm.org/repo/hg/rev/e4f82db071a4 changeset: 40282:e4f82db071a4 user:Mark Thomas date:Sun Oct 14 09:24:36 2018 + summary: py3: fix test-diff-color.t https://www.mercurial-scm.org/repo/hg/rev/c7a363ca04b9 changeset: 40283:c7a363ca04b9 user:Augie Fackler date:Sun Oct 14 04:33:47 2018 -0400 summary: notify: add some b prefixes https://www.mercurial-scm.org/repo/hg/rev/46b55f279571 changeset: 40284:46b55f279571 user:Augie Fackler date:Sun Oct 14 05:28:01 2018 -0400 summary: notify: adapt to new location of email module's errors https://www.mercurial-scm.org/repo/hg/rev/96164fb9b913 changeset: 40285:96164fb9b913 user:Augie Fackler date:Sun Oct 14 04:37:25 2018 -0400 summary: logcmdutil: add a helpful assertion to catch mistyped templates early https://www.mercurial-scm.org/repo/hg/rev/af52181f71ff changeset: 40286:af52181f71ff user:Yuya Nishihara date:Sun Oct 07 20:55:51 2018 +0900 summary: rust-chg: suppress panic while writing chg error to stderr https://www.mercurial-scm.org/repo/hg/rev/7623199def92 changeset: 40287:7623199def92 user:Yuya Nishihara date:Sat Oct 06 20:07:11 2018 +0900 summary: rust-chg: depend on log and tokio_timer https://www.mercurial-scm.org/repo/hg/rev/87c76e5f3427 changeset: 40288:87c76e5f3427 user:Yuya Nishihara date:Sun Oct 07 11:32:42 2018 +0900 summary: rust-chg: install logger if $CHGDEBUG is set https://www.mercurial-scm.org/repo/hg/rev/7d3285f799cc changeset: 40289:7d3285f799cc user:Yuya Nishihara date:Sat Oct 06 21:13:59 2018 +0900 summary: rust-chg: add struct holding information needed to spawn server process https://www.mercurial-scm.org/repo/hg/rev/091f9b8a1051 changeset: 40290:091f9b8a1051 user:Yuya Nishihara date:Sun Oct 14 13:35:47 2018 +0200 summary: notify: just use email.errors https://www.mercurial-scm.org/repo/hg/rev/170926caf44c changeset: 40291:170926caf44c user:rdama...@google.com date:Sat Oct 13 05:03:50 2018 -0700 summary: help: adding support for command categories https://www.mercurial-scm.org/repo/hg/rev/9c6473d2038b changeset: 40292:9c6473d2038b user:Rodrigo Damazio date:Fri Oct 12 17:57:36 2018 +0200 summary: help: splitting the topics by category https://www.mercurial-scm.org/repo/hg/rev/c303d65d2e34 changeset: 40293:c303d65d2e34 user:rdama...@google.com date:Sat Oct 13 02:17:41 2018 -0700 summary: help: assigning categories to existing commands https://www.mercurial-scm.org/repo/hg/rev/fabbf9310025 changeset: 40294:fabbf9310025 user:Rodrigo Damazio date:Fri Oct 12 18:06:32 2018 +0200 summary: help: assigning topic categories https://www.mercurial-scm.org/repo/hg/rev/fa88170c10bb changeset: 40295:fa88170c10bb user:Rodrigo Damazio date:Fri Oct 12 18:49:11 2018 +0200 summary: help: adding a proper declaration for shortlist/basic commands (API) https://www.mercurial-scm.org/repo/hg/rev/e787d97e90ad changeset: 40296:e787d97e90ad user:Mark Thomas date:Sun Oct 14 14:02:32 2018 + summary: py3: fix test-dirstate-race.t https://www.mercurial-scm.org/repo/hg/rev/d33611280add changeset: 40297:d33611280add user:Mark Thomas date:Sun Oct 14 14:10:38 2018 + summary: py3: fix test-propertycache.py https://www.mercurial-scm.org/repo/hg/rev/9cadb0f5f227 changeset: 40298:9cadb0f5f227 user:Georges Racinet date:Thu Sep 27 16:55:44 2018 +0200 summary: rust: hooking into Python code https://www.mercurial-scm.org/repo/hg/rev/3570adf20d91 changeset: 40299:3570adf20d91 user:Matt Harbison date:Sun Oct 14 01:39:22 2018 -0400 summary: help: fix a missing quote character in ui.tweakdefaults https://www.mercurial-scm.org/repo/hg/rev/72b94f946e90 changeset: 40300:72b94f946e90 user:Georges Racinet date:Mon Oct 08 19:11:41 2018 +0200 summary: rust: rustlazyancestors.__contains__ https://www.mercurial-scm.org/repo/hg/rev/b929bce413f4 changeset: 40301:b929bce413f4 user:Georges Racinet date:Mon Oct 15 11:16:12 2018 +0200 summary: rust: rustfmt config for hg-direct-ffi https://www.mercurial-scm.org/repo/hg/rev/af2306bf7d5d changeset: 40302:af2306bf7d5d user:Augie Fackler date:Sun Oct 14 11:05:41 2018 -0400
D5105: style: drop requirement to only use single lines between top-level objects
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG47084b5ffd80: style: drop requirement to only use single
lines between top-level objects (authored by mjpieters, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5105?vs=12134=12156
REVISION DETAIL
https://phab.mercurial-scm.org/D5105
AFFECTED FILES
contrib/check-commit
CHANGE DETAILS
diff --git a/contrib/check-commit b/contrib/check-commit
--- a/contrib/check-commit
+++ b/contrib/check-commit
@@ -39,8 +39,6 @@
"summary keyword should be most user-relevant one-word command or topic"),
(afterheader + r".*\.\s*\n", "don't add trailing period on summary line"),
(afterheader + r".{79,}", "summary line too long (limit is 78)"),
-(r"\n\+\n( |\+)\n", "adds double empty line"),
-(r"\n \n\+\n", "adds double empty line"),
# Forbid "_" in function name.
#
# We skip the check for cffi related functions. They use names mapping the
To: mjpieters, #hg-reviewers, durin42, indygreg
Cc: durin42, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5105: style: drop requirement to only use single lines between top-level objects
indygreg accepted this revision. indygreg added a comment. This revision is now accepted and ready to land. At the 4.8 sprint, we agreed to repo mass rewriting with style that is more PEP-8 like. The tentative plan is to use black, which is very opinionated and doesn't give us much room for style flexibility. But regardless if we do that, I distinctly remember us saying we wanted to adopt double newline between module elements, allowing `_` in names, and using CamelCase for class names. The former would be added automatically by black. The latter 2 would lead to inconsistent style in the long term, which we're OK with. Since we agreed to allow double newlines and this patch formalizes that policy decision, I'm taking it. (I would also accept patches that removes the bit that complains about new symbols using `_` and uppercase in names.) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5105 To: mjpieters, #hg-reviewers, durin42, indygreg Cc: durin42, indygreg, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4944: remotefilelog: rip out lz4 support (WIP)
indygreg accepted this revision as: indygreg.
indygreg added a comment.
Get that non-standard LZ4 format outta here!
Yes, we should probably add a compression engine API for doing one-shot
compression/decompression. I could use that in the SQLite extension. If you
don't implement it, I may do it for you in the natural course of making that
extension :)
I'm holding off formally accepting as a reviewer because the patch is marked
WIP.
Please resubmit without that.
I'll need to take a pass over the final version of this code before
committing. But I'm pretty sure the current state of this series is good enough
or pretty close to being landable.
INLINE COMMENTS
> datapack.py:303-304
>
> -# it has to be at least long enough for the lz4 header.
> -assert deltalen >= 4
> -
> -# python-lz4 stores the length of the uncompressed field as a
> -# little-endian 32-bit integer at the start of the data.
> -uncompressedlen = struct.unpack(' +# TODO(augie): we should store a header that is the
> +# uncompressed size.
> +uncompressedlen = len(zlib.decompress(
Yes we should. But this is no worse than revlogs.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4944
To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6] py3: flush std streams before/after running user code in heredoctest.py
# HG changeset patch
# User Yuya Nishihara
# Date 1539667110 -7200
# Tue Oct 16 07:18:30 2018 +0200
# Node ID 8190570be13dc584dfab6b8470546c298d6900b6
# Parent a202f1b4611205a08a6730db63dfead59b69f42b
py3: flush std streams before/after running user code in heredoctest.py
Otherwise, things written to stdout.buffer would be interleaved.
diff --git a/tests/heredoctest.py b/tests/heredoctest.py
--- a/tests/heredoctest.py
+++ b/tests/heredoctest.py
@@ -2,6 +2,10 @@ from __future__ import absolute_import,
import sys
+def flush():
+sys.stdout.flush()
+sys.stderr.flush()
+
globalvars = {}
lines = sys.stdin.readlines()
while lines:
@@ -15,6 +19,9 @@ while lines:
snippet += l[4:]
c = compile(snippet, '', 'single')
try:
+flush()
exec(c, globalvars)
+flush()
except Exception as inst:
+flush()
print(repr(inst))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6] py3: byte-stringify literals in test-keyword.t
# HG changeset patch # User Yuya Nishihara # Date 1539667178 -7200 # Tue Oct 16 07:19:38 2018 +0200 # Node ID b618c834d8a9a30b7a16927b7ab59f272dee7a43 # Parent 8190570be13dc584dfab6b8470546c298d6900b6 py3: byte-stringify literals in test-keyword.t # skip-blame just some b'' prefixes diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist --- a/contrib/python3-whitelist +++ b/contrib/python3-whitelist @@ -291,6 +291,7 @@ test-issue842.t test-journal-exists.t test-journal-share.t test-journal.t +test-keyword.t test-known.t test-largefiles-cache.t test-largefiles-misc.t diff --git a/tests/test-keyword.t b/tests/test-keyword.t --- a/tests/test-keyword.t +++ b/tests/test-keyword.t @@ -1423,9 +1423,9 @@ suppress expanding keywords at subsequen ... # hello block ... readchannel(server) ... - ... runcommand(server, ['cat', 'm']) - ... runcommand(server, ['diff', '-c', '.', 'm']) - ... runcommand(server, ['cat', 'm']) + ... runcommand(server, [b'cat', b'm']) + ... runcommand(server, [b'diff', b'-c', b'.', b'm']) + ... runcommand(server, [b'cat', b'm']) *** runcommand cat m $Id: m 800511b3a22d Thu, 01 Jan 1970 00:00:00 + test $ bar ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6] py3: rewrite StringIO fallback for Python 3
# HG changeset patch # User Yuya Nishihara # Date 1539669977 -7200 # Tue Oct 16 08:06:17 2018 +0200 # Node ID a202f1b4611205a08a6730db63dfead59b69f42b # Parent 94b452b2cfab5442a50b43daf3c6796c8ce8126b py3: rewrite StringIO fallback for Python 3 diff --git a/contrib/hgclient.py b/contrib/hgclient.py --- a/contrib/hgclient.py +++ b/contrib/hgclient.py @@ -1,6 +1,8 @@ # A minimal client for Mercurial's command server from __future__ import absolute_import, print_function + +import io import os import re import signal @@ -10,23 +12,19 @@ import subprocess import sys import time -try: -import cStringIO as io -stringio = io.StringIO -except ImportError: -import io -stringio = io.StringIO - if sys.version_info[0] >= 3: stdout = sys.stdout.buffer stderr = sys.stderr.buffer +stringio = io.BytesIO def bprint(*args): # remove b'' as well for ease of test migration pargs = [re.sub(br'''\bb(['"])''', br'\1', b'%s' % a) for a in args] stdout.write(b' '.join(pargs) + b'\n') else: +import cStringIO stdout = sys.stdout stderr = sys.stderr +stringio = cStringIO.StringIO bprint = print def connectpipe(path=None): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6] py3: reinvent print() function for contrib/hgclient.py
# HG changeset patch
# User Yuya Nishihara
# Date 1539666247 -7200
# Tue Oct 16 07:04:07 2018 +0200
# Node ID 94b452b2cfab5442a50b43daf3c6796c8ce8126b
# Parent ec5c140cee73da79dc3f48c4e75991cf6e34b7cd
py3: reinvent print() function for contrib/hgclient.py
diff --git a/contrib/hgclient.py b/contrib/hgclient.py
--- a/contrib/hgclient.py
+++ b/contrib/hgclient.py
@@ -2,6 +2,7 @@
from __future__ import absolute_import, print_function
import os
+import re
import signal
import socket
import struct
@@ -19,9 +20,14 @@ except ImportError:
if sys.version_info[0] >= 3:
stdout = sys.stdout.buffer
stderr = sys.stderr.buffer
+def bprint(*args):
+# remove b'' as well for ease of test migration
+pargs = [re.sub(br'''\bb(['"])''', br'\1', b'%s' % a) for a in args]
+stdout.write(b' '.join(pargs) + b'\n')
else:
stdout = sys.stdout
stderr = sys.stderr
+bprint = print
def connectpipe(path=None):
cmdline = [b'hg', b'serve', b'--cmdserver', b'pipe']
@@ -90,7 +96,7 @@ def sep(text):
def runcommand(server, args, output=stdout, error=stderr, input=None,
outfilter=lambda x: x):
-print(b'*** runcommand', b' '.join(args))
+bprint(b'*** runcommand', b' '.join(args))
stdout.flush()
server.stdin.write(b'runcommand\n')
writeblock(server, b'\0'.join(args))
@@ -113,10 +119,10 @@ def runcommand(server, args, output=stdo
elif ch == b'r':
ret, = struct.unpack('>i', data)
if ret != 0:
-print(b' [%d]' % ret)
+bprint(b' [%d]' % ret)
return ret
else:
-print(b"unexpected channel %c: %r" % (ch, data))
+bprint(b"unexpected channel %c: %r" % (ch, data))
if ch.isupper():
return
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 6] py3: work around unicode stdio streams in contrib/hgclient.py
# HG changeset patch # User Yuya Nishihara # Date 1539666492 -7200 # Tue Oct 16 07:08:12 2018 +0200 # Node ID ec5c140cee73da79dc3f48c4e75991cf6e34b7cd # Parent 0001bda2e00c73f9f82591d3ce7b44f552d0b936 py3: work around unicode stdio streams in contrib/hgclient.py diff --git a/contrib/hgclient.py b/contrib/hgclient.py --- a/contrib/hgclient.py +++ b/contrib/hgclient.py @@ -16,6 +16,13 @@ except ImportError: import io stringio = io.StringIO +if sys.version_info[0] >= 3: +stdout = sys.stdout.buffer +stderr = sys.stderr.buffer +else: +stdout = sys.stdout +stderr = sys.stderr + def connectpipe(path=None): cmdline = [b'hg', b'serve', b'--cmdserver', b'pipe'] if path: @@ -81,10 +88,10 @@ def readchannel(server): def sep(text): return text.replace(b'\\', b'/') -def runcommand(server, args, output=sys.stdout, error=sys.stderr, input=None, +def runcommand(server, args, output=stdout, error=stderr, input=None, outfilter=lambda x: x): print(b'*** runcommand', b' '.join(args)) -sys.stdout.flush() +stdout.flush() server.stdin.write(b'runcommand\n') writeblock(server, b'\0'.join(args)) @@ -114,7 +121,7 @@ def runcommand(server, args, output=sys. return def check(func, connect=connectpipe): -sys.stdout.flush() +stdout.flush() server = connect() try: return func(server) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6] py3: convert string literals to bytes in contrib/hgclient.py
# HG changeset patch
# User Yuya Nishihara
# Date 1539666041 -7200
# Tue Oct 16 07:00:41 2018 +0200
# Node ID 0001bda2e00c73f9f82591d3ce7b44f552d0b936
# Parent da1629c7dda17853c8cbaad0ef14a899af6e9361
py3: convert string literals to bytes in contrib/hgclient.py
# skip-blame just many b'' prefixes
diff --git a/contrib/hgclient.py b/contrib/hgclient.py
--- a/contrib/hgclient.py
+++ b/contrib/hgclient.py
@@ -17,9 +17,9 @@ except ImportError:
stringio = io.StringIO
def connectpipe(path=None):
-cmdline = ['hg', 'serve', '--cmdserver', 'pipe']
+cmdline = [b'hg', b'serve', b'--cmdserver', b'pipe']
if path:
-cmdline += ['-R', path]
+cmdline += [b'-R', path]
server = subprocess.Popen(cmdline, stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
@@ -41,9 +41,9 @@ class unixconnection(object):
class unixserver(object):
def __init__(self, sockpath, logpath=None, repopath=None):
self.sockpath = sockpath
-cmdline = ['hg', 'serve', '--cmdserver', 'unix', '-a', sockpath]
+cmdline = [b'hg', b'serve', b'--cmdserver', b'unix', b'-a', sockpath]
if repopath:
-cmdline += ['-R', repopath]
+cmdline += [b'-R', repopath]
if logpath:
stdout = open(logpath, 'a')
stderr = subprocess.STDOUT
@@ -64,7 +64,7 @@ class unixserver(object):
self.server.wait()
def writeblock(server, data):
-server.stdin.write(struct.pack('>I', len(data)))
+server.stdin.write(struct.pack(b'>I', len(data)))
server.stdin.write(data)
server.stdin.flush()
@@ -73,43 +73,43 @@ def readchannel(server):
if not data:
raise EOFError
channel, length = struct.unpack('>cI', data)
-if channel in 'IL':
+if channel in b'IL':
return channel, length
else:
return channel, server.stdout.read(length)
def sep(text):
-return text.replace('\\', '/')
+return text.replace(b'\\', b'/')
def runcommand(server, args, output=sys.stdout, error=sys.stderr, input=None,
outfilter=lambda x: x):
-print('*** runcommand', ' '.join(args))
+print(b'*** runcommand', b' '.join(args))
sys.stdout.flush()
-server.stdin.write('runcommand\n')
-writeblock(server, '\0'.join(args))
+server.stdin.write(b'runcommand\n')
+writeblock(server, b'\0'.join(args))
if not input:
input = stringio()
while True:
ch, data = readchannel(server)
-if ch == 'o':
+if ch == b'o':
output.write(outfilter(data))
output.flush()
-elif ch == 'e':
+elif ch == b'e':
error.write(data)
error.flush()
-elif ch == 'I':
+elif ch == b'I':
writeblock(server, input.read(data))
-elif ch == 'L':
+elif ch == b'L':
writeblock(server, input.readline(data))
-elif ch == 'r':
+elif ch == b'r':
ret, = struct.unpack('>i', data)
if ret != 0:
-print(' [%d]' % ret)
+print(b' [%d]' % ret)
return ret
else:
-print("unexpected channel %c: %r" % (ch, data))
+print(b"unexpected channel %c: %r" % (ch, data))
if ch.isupper():
return
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5110: tests: test-parseindex.t requires no-pure
yuja added a comment. > Custom python code in test-parseindex.t calls reachableroots2 which is only > available in the C revlog index implementation. Marking it #require no-pure > ensure it will not run for --pure test runs. It's surrounded with `#if no-pure`. Did you experience any test failure? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5110 To: adgar, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5110: tests: test-parseindex.t requires no-pure
> Custom python code in test-parseindex.t calls reachableroots2 which is only > available in the C revlog index implementation. Marking it #require no-pure > ensure it will not run for --pure test runs. It's surrounded with `#if no-pure`. Did you experience any test failure? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] py3: byteify extension in test-relink.t
On Mon, 15 Oct 2018 22:21:23 -0400, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison > # Date 1539536753 14400 > # Sun Oct 14 13:05:53 2018 -0400 > # Node ID bb67d54942a16a556cdf257d67ef8f98638728a7 > # Parent 0947a5818862b8cbd0689d9bb943baa35361a436 > py3: byteify extension in test-relink.t Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
