D4787: narrow: pass 'narrow_widen' as source while generating changegroup
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
Although this is called as a part of exchange.pull(), we don't want other
extensions to wrap this pull call because it's not a normal pull and it's
widening.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4787
AFFECTED FILES
hgext/narrow/narrowbundle2.py
CHANGE DETAILS
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -51,15 +51,14 @@
caps[NARROWCAP] = ['v0']
return caps
-def widen_changegroup(repo, diffmatcher, common, cgversion, source):
+def widen_changegroup(repo, diffmatcher, common, cgversion):
"""generates changegroup for widening a narrow clone
repo is the localrepository instance
diffmatcher is a differencemacther of '(newincludes, newexcludes) -
(oldincludes, oldexcludes)'
common is set of common revs between server and client
cgversion is the changegroup version to send
-source is the command which called this codepath
returns changegroup data of the changegroup built or return None if there
are no common revs
@@ -76,7 +75,7 @@
filematcher=diffmatcher,
fullnodes=commonnodes)
cgdata = packer.generate(set([nullid]), list(commonnodes), False,
- source, changelog=False)
+ 'narrow_widen', changelog=False)
return cgdata
@@ -110,7 +109,7 @@
common = set(common or [nullid])
if (oldinclude != include or oldexclude != exclude):
-cgdata = widen_changegroup(repo, diffmatch, common, version, source)
+cgdata = widen_changegroup(repo, diffmatch, common, version)
if cgdata is not None:
part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version)
@@ -189,7 +188,7 @@
shallow=depth is not None,
ellipsisroots=newellipsis,
fullnodes=newfull)
-cgdata = packer.generate(common, newvisit, False, source)
+cgdata = packer.generate(common, newvisit, False, 'narrow_widen')
part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version)
@@ -207,7 +206,7 @@
shallow=depth is not None,
ellipsisroots=ellipsisroots,
fullnodes=relevant_nodes)
-cgdata = packer.generate(common, visitnodes, False, source)
+cgdata = packer.generate(common, visitnodes, False, 'narrow_widen')
part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version)
To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4788: narrow: don't do the dirstate dance if ellipses is not enabled
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
I believe we set dirstate parents to nullid before widening pull because in
ellipses cases, the parent might be stripped off with a new changeset. However
the second ds.setparents() call invalidate my assumption. I am not sure why we
do this. So here is a patch.
This patch also adds tests showing we break nothing in non-ellipses cases.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4788
AFFECTED FILES
hgext/narrow/narrowcommands.py
tests/test-narrow-widen-no-ellipsis.t
CHANGE DETAILS
diff --git a/tests/test-narrow-widen-no-ellipsis.t
b/tests/test-narrow-widen-no-ellipsis.t
--- a/tests/test-narrow-widen-no-ellipsis.t
+++ b/tests/test-narrow-widen-no-ellipsis.t
@@ -88,6 +88,9 @@
added upstream revisions.
$ cd narrow
+ $ hg id -n
+ 2
+
$ hg tracked --addinclude widest/f --debug
comparing with ssh://user@dummy/master
running python "*dummyssh" *user@dummy* *hg -R master serve --stdio* (glob)
@@ -127,6 +130,9 @@
$ cat widest/f
widest
+ $ hg id -n
+ 2
+
Pull down the newly added upstream revision.
$ hg pull
diff --git a/hgext/narrow/narrowcommands.py b/hgext/narrow/narrowcommands.py
--- a/hgext/narrow/narrowcommands.py
+++ b/hgext/narrow/narrowcommands.py
@@ -254,6 +254,14 @@
def _widen(ui, repo, remote, commoninc, newincludes, newexcludes):
newmatch = narrowspec.match(repo.root, newincludes, newexcludes)
+# for now we assume that if a server has ellipses enabled, we will be
+# exchanging ellipses nodes. In future we should add ellipses as a client
+# side requirement (maybe) to distinguish a client is shallow or not and
+# then send that information to server whether we want ellipses or not.
+# Theoretically a non-ellipses repo should be able to use narrow
+# functionality from an ellipses enabled server
+ellipsesremote = narrowwirepeer.ELLIPSESCAP in remote.capabilities()
+
def pullbundle2extraprepare_widen(orig, pullop, kwargs):
orig(pullop, kwargs)
# The old{in,ex}cludepats have already been set by orig()
@@ -272,15 +280,17 @@
overrides = {('devel', 'all-warnings'): False}
with ui.uninterruptable():
-ds = repo.dirstate
-p1, p2 = ds.p1(), ds.p2()
-with ds.parentchange():
-ds.setparents(node.nullid, node.nullid)
+if ellipsesremote:
+ds = repo.dirstate
+p1, p2 = ds.p1(), ds.p2()
+with ds.parentchange():
+ds.setparents(node.nullid, node.nullid)
common = commoninc[0]
with wrappedextraprepare, repo.ui.configoverride(overrides, 'widen'):
exchange.pull(repo, remote, heads=common)
-with ds.parentchange():
-ds.setparents(p1, p2)
+if ellipsesremote:
+with ds.parentchange():
+ds.setparents(p1, p2)
repo.setnewnarrowpats()
actions = {k: [] for k in 'a am f g cd dc r dm dg m e k p pr'.split()}
To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4786: narrow: factor out logic to create cg while widening into separate fn
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This patch takes out the logic which generates a changegroup for widening a
narrow clone when ellipses are disabled. This is done because future patches
will introduce a narrow_widen() wireprotocol command which will send a
changegroup and will use this function.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4786
AFFECTED FILES
hgext/narrow/narrowbundle2.py
CHANGE DETAILS
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -51,6 +51,37 @@
caps[NARROWCAP] = ['v0']
return caps
+def widen_changegroup(repo, diffmatcher, common, cgversion, source):
+"""generates changegroup for widening a narrow clone
+
+repo is the localrepository instance
+diffmatcher is a differencemacther of '(newincludes, newexcludes) -
+(oldincludes, oldexcludes)'
+common is set of common revs between server and client
+cgversion is the changegroup version to send
+source is the command which called this codepath
+
+returns changegroup data of the changegroup built or return None if there
+are no common revs
+"""
+common = repo.revs("::%ln", common)
+commonnodes = set()
+cl = repo.changelog
+for c in common:
+commonnodes.add(cl.node(c))
+if commonnodes:
+# XXX: we should only send the filelogs (and treemanifest). user
+# already has the changelog and manifest
+packer = changegroup.getbundler(cgversion, repo,
+filematcher=diffmatcher,
+fullnodes=commonnodes)
+cgdata = packer.generate(set([nullid]), list(commonnodes), False,
+ source, changelog=False)
+
+return cgdata
+
+return None
+
def getbundlechangegrouppart_widen(bundler, repo, source, bundlecaps=None,
b2caps=None, heads=None, common=None,
**kwargs):
@@ -79,20 +110,8 @@
common = set(common or [nullid])
if (oldinclude != include or oldexclude != exclude):
-common = repo.revs("::%ln", common)
-commonnodes = set()
-cl = repo.changelog
-for c in common:
-commonnodes.add(cl.node(c))
-if commonnodes:
-# XXX: we should only send the filelogs (and treemanifest). user
-# already has the changelog and manifest
-packer = changegroup.getbundler(version, repo,
-filematcher=diffmatch,
-fullnodes=commonnodes)
-cgdata = packer.generate(set([nullid]), list(commonnodes), False,
- source, changelog=False)
-
+cgdata = widen_changegroup(repo, diffmatch, common, version, source)
+if cgdata is not None:
part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version)
if 'treemanifest' in repo.requirements:
To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4789: narrow: check for servers' narrow support before doing anything (BC)
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
Recently we introduced narrow capabilities for the server. So we can check
whether a server has narrow clone support or not before doing anything. This
is
BC because new clients won't be able to extend from old narrow-enabled
servers.
I *think* narrow is not used much (maybe just inside Google), also it's
experimental so I think we can change this. We will need to this someday
anyway.
The "doesn't" in error is changed to "does not" because I think that's we do
in
core and also thats what I typed while writing the error message.
I am fine with that being dropped too.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4789
AFFECTED FILES
hgext/narrow/narrowcommands.py
tests/test-narrow-clone-non-narrow-server.t
CHANGE DETAILS
diff --git a/tests/test-narrow-clone-non-narrow-server.t
b/tests/test-narrow-clone-non-narrow-server.t
--- a/tests/test-narrow-clone-non-narrow-server.t
+++ b/tests/test-narrow-clone-non-narrow-server.t
@@ -60,7 +60,5 @@
looking for local changes to affected paths
$ hg tracked --addinclude f1 http://localhost:$HGPORT1/
comparing with http://localhost:$HGPORT1/
- searching for changes
- no changes found
- abort: server doesn't support narrow clones
+ abort: server does not support narrow clones
[255]
diff --git a/hgext/narrow/narrowcommands.py b/hgext/narrow/narrowcommands.py
--- a/hgext/narrow/narrowcommands.py
+++ b/hgext/narrow/narrowcommands.py
@@ -407,6 +407,13 @@
url, branches = hg.parseurl(remotepath)
ui.status(_('comparing with %s\n') % util.hidepassword(url))
remote = hg.peer(repo, opts, url)
+
+# check narrow support before doing anything if widening needs to be
+# performed. In future we should also abort if client is ellipses and
+# server does not support ellipses
+if widening and narrowwirepeer.NARROWCAP not in remote.capabilities():
+raise error.Abort(_("server does not support narrow clones"))
+
commoninc = discovery.findcommonincoming(repo, remote)
oldincludes, oldexcludes = repo.narrowpats
To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4788: narrow: don't do the dirstate dance if ellipses is not enabled
pulkit added a subscriber: martinvonz. pulkit added a comment. I am not sure about this one. I was unable to think of a reason why we need to do this dirstate dance in non-ellipses cases. @martinvonz @durin42 do you know why we do this? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4788 To: pulkit, durin42, #hg-reviewers Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4790: wireprotov2: derive "required" from presence of default value
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
If we define a default value for all optional arguments, we no
longer need to explicitly declare whether the argument is required.
Instead, we can derive it from the presence of a default value
callable.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4790
AFFECTED FILES
mercurial/wireprotov2server.py
CHANGE DETAILS
diff --git a/mercurial/wireprotov2server.py b/mercurial/wireprotov2server.py
--- a/mercurial/wireprotov2server.py
+++ b/mercurial/wireprotov2server.py
@@ -603,9 +603,6 @@
A callable returning the default value for this argument. If not
specified, ``None`` will be the default value.
- ``required``
- Bool indicating whether the argument is required.
-
``example``
An example value for this argument.
@@ -671,13 +668,9 @@
raise error.ProgrammingError('%s argument for command %s does not '
'declare example field' % (arg, name))
-if 'default' in meta and meta.get('required'):
-raise error.ProgrammingError('%s argument for command %s is marked
'
- 'as required but has a default value'
%
- (arg, name))
+meta['required'] = 'default' not in meta
meta.setdefault('default', lambda: None)
-meta.setdefault('required', False)
meta.setdefault('validvalues', None)
def register(func):
@@ -792,14 +785,17 @@
args={
'noderange': {
'type': 'list',
+'default': lambda: None,
'example': [[b'0123456...'], [b'abcdef...']],
},
'nodes': {
'type': 'list',
+'default': lambda: None,
'example': [b'0123456...'],
},
'nodesdepth': {
'type': 'int',
+'default': lambda: None,
'example': 10,
},
'fields': {
@@ -968,7 +964,6 @@
},
'nodes': {
'type': 'list',
-'required': True,
'example': [b'0123456...'],
},
'fields': {
@@ -979,7 +974,6 @@
},
'path': {
'type': 'bytes',
-'required': True,
'example': b'foo.txt',
}
},
@@ -1074,7 +1068,6 @@
args={
'namespace': {
'type': 'bytes',
-'required': True,
'example': b'ns',
},
},
@@ -1091,7 +1084,6 @@
args={
'key': {
'type': 'bytes',
-'required': True,
'example': b'foo',
},
},
@@ -1109,7 +1101,6 @@
args={
'nodes': {
'type': 'list',
-'required': True,
'example': [b'0123456...'],
},
'haveparents': {
@@ -1125,7 +1116,6 @@
},
'tree': {
'type': 'bytes',
-'required': True,
'example': b'',
},
},
@@ -1183,22 +1173,18 @@
args={
'namespace': {
'type': 'bytes',
-'required': True,
'example': b'ns',
},
'key': {
'type': 'bytes',
-'required': True,
'example': b'key',
},
'old': {
'type': 'bytes',
-'required': True,
'example': b'old',
},
'new': {
'type': 'bytes',
-'required': True,
'example': 'new',
},
},
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4793: filelog: remove checkhash() (API)
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY It is unused. While a caller may want to ask the store to validate the hash of some provided text, since there are no in-core consumers of this method, let's drop it for now. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4793 AFFECTED FILES mercurial/filelog.py mercurial/repository.py mercurial/testing/storage.py CHANGE DETAILS diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py --- a/mercurial/testing/storage.py +++ b/mercurial/testing/storage.py @@ -368,12 +368,6 @@ with self.assertRaises(IndexError): f.size(i) -with self.assertRaises(error.StorageError): -f.checkhash(b'', nullid) - -with self.assertRaises(error.LookupError): -f.checkhash(b'', b'\x01' * 20) - self.assertEqual(f.revision(nullid), b'') self.assertEqual(f.revision(nullid, raw=True), b'') @@ -426,18 +420,6 @@ with self.assertRaises(IndexError): f.size(1) -f.checkhash(fulltext, node) -f.checkhash(fulltext, node, nullid, nullid) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext + b'extra', node) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext, node, b'\x01' * 20, nullid) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext, node, nullid, b'\x01' * 20) - self.assertEqual(f.revision(node), fulltext) self.assertEqual(f.revision(node, raw=True), fulltext) @@ -510,20 +492,6 @@ with self.assertRaises(IndexError): f.size(3) -f.checkhash(fulltext0, node0) -f.checkhash(fulltext1, node1) -f.checkhash(fulltext1, node1, node0, nullid) -f.checkhash(fulltext2, node2, node1, nullid) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext1, b'\x01' * 20) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext1 + b'extra', node1, node0, nullid) - -with self.assertRaises(error.StorageError): -f.checkhash(fulltext1, node1, node0, node0) - self.assertEqual(f.revision(node0), fulltext0) self.assertEqual(f.revision(node0, raw=True), fulltext0) self.assertEqual(f.revision(node1), fulltext1) diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -545,12 +545,6 @@ Any metadata is excluded from size measurements. """ -def checkhash(fulltext, node, p1=None, p2=None, rev=None): -"""Validate the stored hash of a given fulltext and node. - -Raises ``error.StorageError`` is hash validation fails. -""" - def revision(node, raw=False): Obtain fulltext data for a node. diff --git a/mercurial/filelog.py b/mercurial/filelog.py --- a/mercurial/filelog.py +++ b/mercurial/filelog.py @@ -71,10 +71,6 @@ def iscensored(self, rev): return self._revlog.iscensored(rev) -# Might be unused. -def checkhash(self, text, node, p1=None, p2=None, rev=None): -return self._revlog.checkhash(text, node, p1=p1, p2=p2, rev=rev) - def revision(self, node, _df=None, raw=False): return self._revlog.revision(node, _df=_df, raw=raw) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4792: filelog: remove revdiff() (API)
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This proxy method is no longer used. While it might be useful to query a storage backend for the delta between any 2 revisions because the store could have a delta cached and could compute it more efficiently than the caller calling revision() twice in order to compute a delta, since nothing in core is using this API now, I feel comfortable nuking it. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4792 AFFECTED FILES mercurial/filelog.py mercurial/repository.py mercurial/testing/storage.py tests/simplestorerepo.py CHANGE DETAILS diff --git a/tests/simplestorerepo.py b/tests/simplestorerepo.py --- a/tests/simplestorerepo.py +++ b/tests/simplestorerepo.py @@ -487,16 +487,6 @@ return nodes -def revdiff(self, rev1, rev2): -validaterev(rev1) -validaterev(rev2) - -node1 = self.node(rev1) -node2 = self.node(rev2) - -return mdiff.textdiff(self.revision(node1, raw=True), - self.revision(node2, raw=True)) - def heads(self, start=None, stop=None): # This is copied from revlog.py. if start is None and stop is None: diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py --- a/mercurial/testing/storage.py +++ b/mercurial/testing/storage.py @@ -396,17 +396,6 @@ with self.assertRaises(error.LookupError): f.cmp(b'\x01' * 20, b'irrelevant') -self.assertEqual(f.revdiff(nullrev, nullrev), b'') - -with self.assertRaises(IndexError): -f.revdiff(0, nullrev) - -with self.assertRaises(IndexError): -f.revdiff(nullrev, 0) - -with self.assertRaises(IndexError): -f.revdiff(0, 0) - # Emitting empty list is an empty generator. gen = f.emitrevisions([]) with self.assertRaises(StopIteration): @@ -459,14 +448,6 @@ self.assertFalse(f.cmp(node, fulltext)) self.assertTrue(f.cmp(node, fulltext + b'extra')) -self.assertEqual(f.revdiff(0, 0), b'') -self.assertEqual(f.revdiff(nullrev, 0), - b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x07%s' % - fulltext) - -self.assertEqual(f.revdiff(0, nullrev), - b'\x00\x00\x00\x00\x00\x00\x00\x07\x00\x00\x00\x00') - # Emitting a single revision works. gen = f.emitrevisions([node]) rev = next(gen) @@ -577,14 +558,6 @@ with self.assertRaises(error.LookupError): f.cmp(b'\x01' * 20, b'irrelevant') -self.assertEqual(f.revdiff(0, 1), - b'\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x04\x01' + - fulltext1) - -self.assertEqual(f.revdiff(0, 2), - b'\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x04\x02' + - fulltext2) - # Nodes should be emitted in order. gen = f.emitrevisions([node0, node1, node2], revisiondata=True) diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -586,15 +586,6 @@ TODO better document the copy metadata and censoring logic. """ -def revdiff(rev1, rev2): -"""Obtain a delta between two revision numbers. - -Operates on raw data in the store (``revision(node, raw=True)``). - -The returned data is the result of ``bdiff.bdiff`` on the raw -revision data. -""" - def emitrevisions(nodes, nodesorder=None, revisiondata=False, diff --git a/mercurial/filelog.py b/mercurial/filelog.py --- a/mercurial/filelog.py +++ b/mercurial/filelog.py @@ -78,9 +78,6 @@ def revision(self, node, _df=None, raw=False): return self._revlog.revision(node, _df=_df, raw=raw) -def revdiff(self, rev1, rev2): -return self._revlog.revdiff(rev1, rev2) - def emitrevisions(self, nodes, nodesorder=None, revisiondata=False, assumehaveparentrevisions=False, deltaprevious=False): To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4791: localrepo: define storage backend in creation options (API)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
We add an experimental config option to define the storage backend
for new repositories. By default, it uses "revlogv1," which maps to
the current and only modern supported repository format.
We add a "backend" creation option to control which backend to
use. It defaults to using the value from the config option.
newreporequirements() will now barf if it sees a "backend" value
that isn't "revlogv1." This forces extensions to monkeypatch the
function to handle requirements derivation for custom backends.
In order for this to "just work," we factored out obtaining the
default creation options into its own function and made callers
of newreporequirements() responsible for passing in valid data.
Without this, direct callers of newreporequirements() wouldn't
get the proper results.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4791
AFFECTED FILES
mercurial/configitems.py
mercurial/localrepo.py
mercurial/upgrade.py
CHANGE DETAILS
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -199,7 +199,8 @@
@staticmethod
def _newreporequirements(ui):
-return localrepo.newreporequirements(ui)
+return localrepo.newreporequirements(
+ui, localrepo.defaultcreateopts(ui))
@classmethod
def fromrepo(cls, repo):
@@ -747,7 +748,8 @@
# FUTURE there is potentially a need to control the wanted requirements via
# command arguments or via an extension hook point.
-newreqs = localrepo.newreporequirements(repo.ui)
+newreqs = localrepo.newreporequirements(
+repo.ui, localrepo.defaultcreateopts(repo.ui))
newreqs.update(preservedrequirements(repo))
noremovereqs = (repo.requirements - newreqs -
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -2747,14 +2747,26 @@
def islocal(path):
return True
-def newreporequirements(ui, createopts=None):
+def defaultcreateopts(ui, createopts=None):
+"""Populate the default creation options for a repository.
+
+A dictionary of explicitly requested creation options can be passed
+in. Missing keys will be populated.
+"""
+createopts = dict(createopts or {})
+
+if 'backend' not in createopts:
+# experimental config: storage.new-repo-backend
+createopts['backend'] = ui.config('storage', 'new-repo-backend')
+
+return createopts
+
+def newreporequirements(ui, createopts):
"""Determine the set of requirements for a new local repository.
Extensions can wrap this function to specify custom requirements for
new repositories.
"""
-createopts = createopts or {}
-
# If the repo is being created from a shared repository, we copy
# its requirements.
if 'sharedrepo' in createopts:
@@ -2766,6 +2778,14 @@
return requirements
+if 'backend' not in createopts:
+raise error.ProgrammingError('backend key not present in createopts; '
+ 'was defaultcreateopts() called?')
+
+if createopts['backend'] != 'revlogv1':
+raise error.Abort(_('unable to determine repository requirements for '
+'storage backend: %s') % createopts['backend'])
+
requirements = {'revlogv1'}
if ui.configbool('format', 'usestore'):
requirements.add('store')
@@ -2824,6 +2844,7 @@
they know how to handle.
"""
known = {
+'backend',
'narrowfiles',
'sharedrepo',
'sharedrelative',
@@ -2840,6 +2861,8 @@
The following keys for ``createopts`` are recognized:
+backend
+ The storage backend to use.
narrowfiles
Set up repository to support narrow file storage.
sharedrepo
@@ -2851,7 +2874,7 @@
shareditems
Set of items to share to the new repository (in addition to storage).
"""
-createopts = createopts or {}
+createopts = defaultcreateopts(ui, createopts=createopts)
unknownopts = filterknowncreateopts(ui, createopts)
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -941,6 +941,9 @@
coreconfigitem('push', 'pushvars.server',
default=False,
)
+coreconfigitem('storage', 'new-repo-backend',
+default='revlogv1',
+)
coreconfigitem('storage', 'revlog.optimize-delta-parent-choice',
default=True,
alias=[('format', 'aggressivemergedeltas')],
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4796: testing: add more testing for ifileindex.lookup()
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The tests demonstrate some... questionable behavior of revlog.lookup(). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4796 AFFECTED FILES mercurial/testing/storage.py CHANGE DETAILS diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py --- a/mercurial/testing/storage.py +++ b/mercurial/testing/storage.py @@ -90,6 +90,30 @@ with self.assertRaises(error.LookupError): f.lookup(b'%d' % nullrev) +with self.assertRaises(error.LookupError): +f.lookup(b'badvalue') + +self.assertEqual(f.lookup(hex(nullid)[0:12]), nullid) + +with self.assertRaises(error.LookupError): +f.lookup(b'-2') + +# TODO this is wonky. +self.assertEqual(f.lookup(b'0'), nullid) + +with self.assertRaises(error.LookupError): +f.lookup(b'1') + +with self.assertRaises(error.LookupError): +f.lookup(b'11') + +for i in range(-5, 5): +if i == nullrev: +continue + +with self.assertRaises(LookupError): +f.lookup(i) + self.assertEqual(f.linkrev(nullrev), nullrev) for i in range(-5, 5): @@ -170,8 +194,22 @@ self.assertEqual(f.lookup(node), node) self.assertEqual(f.lookup(0), node) +self.assertEqual(f.lookup(-1), nullid) self.assertEqual(f.lookup(b'0'), node) self.assertEqual(f.lookup(hex(node)), node) +self.assertEqual(f.lookup(hex(node)[0:12]), node) + +with self.assertRaises(IndexError): +f.lookup(-2) + +with self.assertRaises(error.LookupError): +f.lookup(b'-2') + +with self.assertRaises(IndexError): +f.lookup(1) + +with self.assertRaises(error.LookupError): +f.lookup(b'1') self.assertEqual(f.linkrev(0), 0) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4794: dagop: extract descendants() from revlog module
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This method needs to be implemented in other storage backends and is generic if we parameterize the functions for retrieving revision numbers and parent revision numbers. Let's extract it to dagop so backends don't need to reinvent the wheel. I'm not too worried about performance overhead of the extra function call because I don't expect anyone to be calling descendants() in a tight loop. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4794 AFFECTED FILES mercurial/dagop.py mercurial/revlog.py CHANGE DETAILS diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -748,25 +748,7 @@ inclusive=inclusive) def descendants(self, revs): -"""Generate the descendants of 'revs' in revision order. - -Yield a sequence of revision numbers starting with a child of -some rev in revs, i.e., each revision is *not* considered a -descendant of itself. Results are ordered by revision number (a -topological sort).""" -first = min(revs) -if first == nullrev: -for i in self: -yield i -return - -seen = set(revs) -for i in self.revs(start=first + 1): -for x in self.parentrevs(i): -if x != nullrev and x in seen: -seen.add(i) -yield i -break +return dagop.descendantrevs(revs, self.revs, self.parentrevs) def findcommonmissing(self, common=None, heads=None): """Return a tuple of the ancestors of common and the ancestors of heads diff --git a/mercurial/dagop.py b/mercurial/dagop.py --- a/mercurial/dagop.py +++ b/mercurial/dagop.py @@ -9,6 +9,9 @@ import heapq +from .node import ( +nullrev, +) from .thirdparty import ( attr, ) @@ -225,6 +228,37 @@ startdepth, stopdepth) return generatorset(gen, iterasc=True) +def descendantrevs(revs, revsfn, parentrevsfn): +"""Generate revision number descendants in revision order. + +Yields revision numbers starting with a child of some rev in +``revs``. Results are ordered by revision number and are +therefore topological. Each revision is not considered a descendant +of itself. + +``revsfn`` is a callable that with no argument iterates over all +revision numbers and with a ``start`` argument iterates over revision +numbers beginning with that value. + +``parentrevsfn`` is a callable that receives a revision number and +returns an iterable of parent revision numbers, whose values may include +nullrev. +""" +first = min(revs) + +if first == nullrev: +for rev in revsfn(): +yield rev +return + +seen = set(revs) +for rev in revsfn(start=first + 1): +for prev in parentrevsfn(rev): +if prev != nullrev and prev in seen: +seen.add(rev) +yield rev +break + def _reachablerootspure(repo, minroot, roots, heads, includepath): """return (heads(:: and ::)) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4795: dagop: extract DAG local heads functionality from revlog
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
As part of implementing an alternate storage backend, I found myself
having to reimplement this function.
The DAG traversal logic is generic and can be factored out into a
standalone function.
We could probably combine this with headrevs(). But I'll leave that
for another time.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4795
AFFECTED FILES
mercurial/dagop.py
mercurial/revlog.py
CHANGE DETAILS
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1069,25 +1069,16 @@
return [self.node(r) for r in self.headrevs()]
if start is None:
-start = nullid
-if stop is None:
-stop = []
-stoprevs = set([self.rev(n) for n in stop])
-startrev = self.rev(start)
-reachable = {startrev}
-heads = {startrev}
-
-parentrevs = self.parentrevs
-for r in self.revs(start=startrev + 1):
-for p in parentrevs(r):
-if p in reachable:
-if r not in stoprevs:
-reachable.add(r)
-heads.add(r)
-if p in heads and p not in stoprevs:
-heads.remove(p)
-
-return [self.node(r) for r in heads]
+start = nullrev
+else:
+start = self.rev(start)
+
+stoprevs = set(self.rev(n) for n in stop or [])
+
+revs = dagop.headrevssubset(self.revs, self.parentrevs, startrev=start,
+stoprevs=stoprevs)
+
+return [self.node(rev) for rev in revs]
def children(self, node):
"""find the children of a given node"""
diff --git a/mercurial/dagop.py b/mercurial/dagop.py
--- a/mercurial/dagop.py
+++ b/mercurial/dagop.py
@@ -773,6 +773,42 @@
return headrevs
+def headrevssubset(revsfn, parentrevsfn, startrev=None, stoprevs=None):
+"""Returns the set of all revs that have no children with control.
+
+``revsfn`` is a callable that with no arguments returns an iterator over
+all revision numbers in topological order. With a ``start`` argument, it
+returns revision numbers starting at that number.
+
+``parentrevsfn`` is a callable receiving a revision number and returns an
+iterable of parent revision numbers, where values can include nullrev.
+
+``startrev`` is a revision number at which to start the search.
+
+``stoprevs`` is an iterable of revision numbers that, when encountered,
+will stop DAG traversal beyond them. Parents of revisions in this
+collection will be heads.
+"""
+if startrev is None:
+startrev = nullrev
+
+stoprevs = set(stoprevs or [])
+
+reachable = {startrev}
+heads = {startrev}
+
+for rev in revsfn(start=startrev + 1):
+for prev in parentrevsfn(rev):
+if prev in reachable:
+if rev not in stoprevs:
+reachable.add(rev)
+heads.add(rev)
+
+if prev in heads and prev not in stoprevs:
+heads.remove(prev)
+
+return heads
+
def linearize(revs, parentsfn):
"""Linearize and topologically sort a list of revisions.
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4798: storageutil: consistently raise LookupError (API)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
The interface docs say this is supposed to raise LookupError on
failure. But for invalid revision number input, it could raise
IndexError because ifileindex.node() is documented to raise
IndexError.
lookup() for files isn't used that much (pretty much just in
basefilectx in core AFAICT). And callers should already be catching
LookupError. So I don't anticipate that much fallout from this
change.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4798
AFFECTED FILES
mercurial/testing/storage.py
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -121,7 +121,10 @@
Raises ``error.LookupError`` on failure.
"""
if isinstance(fileid, int):
-return store.node(fileid)
+try:
+return store.node(fileid)
+except IndexError:
+raise error.LookupError(fileid, identifier, _('no match found'))
if len(fileid) == 20:
try:
diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py
--- a/mercurial/testing/storage.py
+++ b/mercurial/testing/storage.py
@@ -199,13 +199,13 @@
with self.assertRaises(error.LookupError):
f.lookup(hex(node)[0:12])
-with self.assertRaises(IndexError):
+with self.assertRaises(error.LookupError):
f.lookup(-2)
with self.assertRaises(error.LookupError):
f.lookup(b'-2')
-with self.assertRaises(IndexError):
+with self.assertRaises(error.LookupError):
f.lookup(1)
with self.assertRaises(error.LookupError):
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4797: storageutil: implement file identifier resolution method (BC)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
revlog.lookup() has a number of advanced features, including partial
node matching.
These advanced features aren't needed for file id lookup because file
identifiers are almost always from internal sources. (An exception to
this is hgweb, which appears to have some code paths that attempt
to resolve a user-supplied string to a node.)
This commit implements a new function for resolving file identifiers
to nodes. It behaves like revlog.lookup() but without the
advanced features.
Tests reveal behavior changes:
- Partial hex nodes are no longer resolved to nodes.
- "-1" now returns nullid instead of raising LookupError.
- "0" on an empty store now raises LookupError instead of returning nullid.
I'm not sure why "-1" wasn't recognized before. But it seems reasonable
to accept it as a value since integer -1 resolves to nullid.
These changes all seem reasonable to me. And with the exception of
partial hex node matching, we may want to consider changing
revlog.lookup() as well.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4797
AFFECTED FILES
mercurial/filelog.py
mercurial/repository.py
mercurial/testing/storage.py
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -10,10 +10,13 @@
import hashlib
import re
+from ..i18n import _
from ..node import (
+bin,
nullid,
)
from .. import (
+error,
pycompat,
)
@@ -99,3 +102,53 @@
stop = storelen
return pycompat.xrange(start, stop, step)
+
+def fileidlookup(store, fileid, identifier):
+"""Resolve the file node for a value.
+
+``store`` is an object implementing the ``ifileindex`` interface.
+
+``fileid`` can be:
+
+* A 20 byte binary node.
+* An integer revision number
+* A 40 byte hex node.
+* A bytes that can be parsed as an integer representing a revision number.
+
+``identifier`` is used to populate ``error.LookupError`` with an identifier
+for the store.
+
+Raises ``error.LookupError`` on failure.
+"""
+if isinstance(fileid, int):
+return store.node(fileid)
+
+if len(fileid) == 20:
+try:
+store.rev(fileid)
+return fileid
+except error.LookupError:
+pass
+
+if len(fileid) == 40:
+try:
+rawnode = bin(fileid)
+store.rev(rawnode)
+return rawnode
+except TypeError:
+pass
+
+try:
+rev = int(fileid)
+
+if b'%d' % rev != fileid:
+raise ValueError
+
+try:
+return store.node(rev)
+except (IndexError, TypeError):
+pass
+except (ValueError, OverflowError):
+pass
+
+raise error.LookupError(fileid, identifier, _('no match found'))
diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py
--- a/mercurial/testing/storage.py
+++ b/mercurial/testing/storage.py
@@ -85,21 +85,19 @@
self.assertEqual(f.lookup(nullid), nullid)
self.assertEqual(f.lookup(nullrev), nullid)
self.assertEqual(f.lookup(hex(nullid)), nullid)
-
-# String converted to integer doesn't work for nullrev.
-with self.assertRaises(error.LookupError):
-f.lookup(b'%d' % nullrev)
+self.assertEqual(f.lookup(b'%d' % nullrev), nullid)
with self.assertRaises(error.LookupError):
f.lookup(b'badvalue')
-self.assertEqual(f.lookup(hex(nullid)[0:12]), nullid)
+with self.assertRaises(error.LookupError):
+f.lookup(hex(nullid)[0:12])
with self.assertRaises(error.LookupError):
f.lookup(b'-2')
-# TODO this is wonky.
-self.assertEqual(f.lookup(b'0'), nullid)
+with self.assertRaises(error.LookupError):
+f.lookup(b'0')
with self.assertRaises(error.LookupError):
f.lookup(b'1')
@@ -197,7 +195,9 @@
self.assertEqual(f.lookup(-1), nullid)
self.assertEqual(f.lookup(b'0'), node)
self.assertEqual(f.lookup(hex(node)), node)
-self.assertEqual(f.lookup(hex(node)[0:12]), node)
+
+with self.assertRaises(error.LookupError):
+f.lookup(hex(node)[0:12])
with self.assertRaises(IndexError):
f.lookup(-2)
diff --git a/mercurial/repository.py b/mercurial/repository.py
--- a/mercurial/repository.py
+++ b/mercurial/repository.py
@@ -1099,9 +1099,6 @@
that can be converted to an integer.
Raises ``error.LookupError`` if a ndoe could not be resolved.
-
-TODO this is only used by debug* commands and can probably be deleted
-easily.
"""
def
D4800: storageutil: extract copy metadata retrieval out of filelog
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY As part of implementing an alternate storage backend, I found myself reinventing this wheel. Let's create a utility function for doing the work. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4800 AFFECTED FILES mercurial/filelog.py mercurial/utils/storageutil.py CHANGE DETAILS diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -89,6 +89,25 @@ offset = text.index(b'\x01\n', 2) return text[offset + 2:] +def filerevisioncopied(store, node): +"""Resolve file revision copy metadata. + +Returns ``False`` if the file has no copy metadata. Otherwise a +2-tuple of the source filename and node. +""" +if store.parents(node)[0] != nullid: +return False + +meta = parsemeta(store.revision(node))[0] + +# copy and copyrev occur in pairs. In rare cases due to old bugs, +# one can occur without the other. So ensure both are present to flag +# as a copy. +if meta and b'copy' in meta and b'copyrev' in meta: +return meta[b'copy'], bin(meta[b'copyrev']) + +return False + def iterrevs(storelen, start=0, stop=None): """Iterate over revision numbers in a store.""" step = 1 diff --git a/mercurial/filelog.py b/mercurial/filelog.py --- a/mercurial/filelog.py +++ b/mercurial/filelog.py @@ -115,15 +115,7 @@ return self.addrevision(text, transaction, link, p1, p2) def renamed(self, node): -if self.parents(node)[0] != revlog.nullid: -return False -t = self.revision(node) -m = storageutil.parsemeta(t)[0] -# copy and copyrev occur in pairs. In rare cases due to bugs, -# one can occur without the other. -if m and "copy" in m and "copyrev" in m: -return (m["copy"], revlog.bin(m["copyrev"])) -return False +return storageutil.filerevisioncopied(self, node) def size(self, rev): """return the size of a given revision""" To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4801: storageutil: extract filelog.cmp() to a standalone function
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
As part of implementing an alternate storage backend, I found myself
reimplementing this code.
With a little massaging, we can extract filelog.cmp() to a standalone
function.
As part of this, the call to revlog.cmp() was inlined (it is just a
2-line function).
I also tweaked some variable names to improve readability. I'll
further tweak names in a subsequent commit.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4801
AFFECTED FILES
mercurial/filelog.py
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -108,6 +108,32 @@
return False
+def filerevisiondifferent(store, node, filedata):
+"""Determines whether file data is equivalent to a stored node."""
+
+if filedata.startswith(b'\x01\n'):
+revisiontext = b'\x01\n\x01\n' + filedata
+else:
+revisiontext = filedata
+
+p1, p2 = store.parents(node)
+
+computednode = hashrevisionsha1(revisiontext, p1, p2)
+
+if computednode == node:
+return False
+
+# Censored files compare against the empty file.
+if store.iscensored(store.rev(node)):
+return filedata != b''
+
+# Renaming a file produces a different hash, even if the data
+# remains unchanged. Check if that's the case.
+if store.renamed(node):
+return store.read(node) != filedata
+
+return True
+
def iterrevs(storelen, start=0, stop=None):
"""Iterate over revision numbers in a store."""
step = 1
diff --git a/mercurial/filelog.py b/mercurial/filelog.py
--- a/mercurial/filelog.py
+++ b/mercurial/filelog.py
@@ -139,26 +139,7 @@
returns True if text is different than what is stored.
"""
-
-t = text
-if text.startswith('\1\n'):
-t = '\1\n\1\n' + text
-
-samehashes = not self._revlog.cmp(node, t)
-if samehashes:
-return False
-
-# censored files compare against the empty file
-if self.iscensored(self.rev(node)):
-return text != ''
-
-# renaming a file produces a different hash, even if the data
-# remains unchanged. Check if it's the case (slow):
-if self.renamed(node):
-t2 = self.read(node)
-return t2 != text
-
-return True
+return storageutil.filerevisiondifferent(self, node, text)
def verifyintegrity(self, state):
return self._revlog.verifyintegrity(state)
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4802: storageutil: invert logic of file data comparison
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY IMO things make more sense when the function is explicitly a test for file data equivalence. Not bothering with API since the function was introduced by the previous commit. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4802 AFFECTED FILES mercurial/filelog.py mercurial/utils/storageutil.py CHANGE DETAILS diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -108,8 +108,18 @@ return False -def filerevisiondifferent(store, node, filedata): -"""Determines whether file data is equivalent to a stored node.""" +def filedataequivalent(store, node, filedata): +"""Determines whether file data is equivalent to a stored node. + +Returns True if the passed file data would hash to the same value +as a stored revision and False otherwise. + +When a stored revision is censored, filedata must be empty to have +equivalence. + +When a stored revision has copy metadata, it is ignored as part +of the compare. +""" if filedata.startswith(b'\x01\n'): revisiontext = b'\x01\n\x01\n' + filedata @@ -121,18 +131,18 @@ computednode = hashrevisionsha1(revisiontext, p1, p2) if computednode == node: -return False +return True # Censored files compare against the empty file. if store.iscensored(store.rev(node)): -return filedata != b'' +return filedata == b'' # Renaming a file produces a different hash, even if the data # remains unchanged. Check if that's the case. if store.renamed(node): -return store.read(node) != filedata +return store.read(node) == filedata -return True +return False def iterrevs(storelen, start=0, stop=None): """Iterate over revision numbers in a store.""" diff --git a/mercurial/filelog.py b/mercurial/filelog.py --- a/mercurial/filelog.py +++ b/mercurial/filelog.py @@ -139,7 +139,7 @@ returns True if text is different than what is stored. """ -return storageutil.filerevisiondifferent(self, node, text) +return not storageutil.filedataequivalent(self, node, text) def verifyintegrity(self, state): return self._revlog.verifyintegrity(state) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4799: storageutil: extract functionality for resolving strip revisions
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
I found myself having to copy this method as part of implementing a new
storage backend. With a little tweaking, we can extract it to a
standalone function so it can be reused easily.
We'll likely want to implement a better method for removing revisions
on the storage interface someday. But until then, let's use what we
have.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4799
AFFECTED FILES
mercurial/revlog.py
mercurial/utils/storageutil.py
CHANGE DETAILS
diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -14,6 +14,7 @@
from ..node import (
bin,
nullid,
+nullrev,
)
from .. import (
error,
@@ -155,3 +156,54 @@
pass
raise error.LookupError(fileid, identifier, _('no match found'))
+
+def resolvestripinfo(minlinkrev, tiprev, headrevs, linkrevfn, parentrevsfn):
+"""Resolve information needed to strip revisions.
+
+Finds the minimum revision number that must be stripped in order to
+strip ``minlinkrev``.
+
+Returns a 2-tuple of the minimum revision number to do that and a set
+of all revision numbers that have linkrevs that would be broken
+by that strip.
+
+``tiprev`` is the current tip-most revision. It is ``len(store) - 1``.
+``headrevs`` is an iterable of head revisions.
+``linkrevfn`` is a callable that receives a revision and returns a linked
+revision.
+``parentrevsfn`` is a callable that receives a revision number and returns
+an iterable of its parent revision numbers.
+"""
+brokenrevs = set()
+strippoint = tiprev + 1
+
+heads = {}
+futurelargelinkrevs = set()
+for head in headrevs:
+headlinkrev = linkrevfn(head)
+heads[head] = headlinkrev
+if headlinkrev >= minlinkrev:
+futurelargelinkrevs.add(headlinkrev)
+
+# This algorithm involves walking down the rev graph, starting at the
+# heads. Since the revs are topologically sorted according to linkrev,
+# once all head linkrevs are below the minlink, we know there are
+# no more revs that could have a linkrev greater than minlink.
+# So we can stop walking.
+while futurelargelinkrevs:
+strippoint -= 1
+linkrev = heads.pop(strippoint)
+
+if linkrev < minlinkrev:
+brokenrevs.add(strippoint)
+else:
+futurelargelinkrevs.remove(linkrev)
+
+for p in parentrevsfn(strippoint):
+if p != nullrev:
+plinkrev = linkrevfn(p)
+heads[p] = plinkrev
+if plinkrev >= minlinkrev:
+futurelargelinkrevs.add(plinkrev)
+
+return strippoint, brokenrevs
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2091,39 +2091,9 @@
Returns a tuple containing the minimum rev and a set of all revs that
have linkrevs that will be broken by this strip.
"""
-brokenrevs = set()
-strippoint = len(self)
-
-heads = {}
-futurelargelinkrevs = set()
-for head in self.headrevs():
-headlinkrev = self.linkrev(head)
-heads[head] = headlinkrev
-if headlinkrev >= minlink:
-futurelargelinkrevs.add(headlinkrev)
-
-# This algorithm involves walking down the rev graph, starting at the
-# heads. Since the revs are topologically sorted according to linkrev,
-# once all head linkrevs are below the minlink, we know there are
-# no more revs that could have a linkrev greater than minlink.
-# So we can stop walking.
-while futurelargelinkrevs:
-strippoint -= 1
-linkrev = heads.pop(strippoint)
-
-if linkrev < minlink:
-brokenrevs.add(strippoint)
-else:
-futurelargelinkrevs.remove(linkrev)
-
-for p in self.parentrevs(strippoint):
-if p != nullrev:
-plinkrev = self.linkrev(p)
-heads[p] = plinkrev
-if plinkrev >= minlink:
-futurelargelinkrevs.add(plinkrev)
-
-return strippoint, brokenrevs
+return storageutil.resolvestripinfo(minlink, len(self) - 1,
+self.headrevs(),
+self.linkrev, self.parentrevs)
def strip(self, minlink, transaction):
"""truncate the revlog on the first revision with a linkrev >= minlink
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
D4804: storageutil: make all callables optional
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Not all storage backends may implement these callables. That's part of the reason these methods aren't exposed on the storage interface. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4804 AFFECTED FILES mercurial/utils/storageutil.py CHANGE DETAILS diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -18,6 +18,7 @@ ) from .. import ( error, +mdiff, pycompat, ) @@ -263,8 +264,9 @@ return strippoint, brokenrevs -def emitrevisions(store, revs, resultcls, deltaparentfn, candeltafn, - rawsizefn, revdifffn, flagsfn, sendfulltext=False, +def emitrevisions(store, revs, resultcls, deltaparentfn=None, candeltafn=None, + rawsizefn=None, revdifffn=None, flagsfn=None, + sendfulltext=False, revisiondata=False, assumehaveparentrevisions=False, deltaprevious=False): """Generic implementation of ifiledata.emitrevisions(). @@ -282,26 +284,40 @@ A type implementing the ``irevisiondelta`` interface that will be constructed and returned. -``deltaparentfn`` +``deltaparentfn`` (optional) Callable receiving a revision number and returning the revision number of a revision that the internal delta is stored against. This delta will be preferred over computing a new arbitrary delta. -``candeltafn`` + If not defined, a delta will always be computed from raw revision + data. + +``candeltafn`` (optional) Callable receiving a pair of revision numbers that returns a bool indicating whether a delta between them can be produced. -``rawsizefn`` + If not defined, it is assumed that any two revisions can delta with + each other. + +``rawsizefn`` (optional) Callable receiving a revision number and returning the length of the ``store.revision(rev, raw=True)``. -``revdifffn`` + If not defined, ``len(store.revision(rev, raw=True))`` will be called. + +``revdifffn`` (optional) Callable receiving a pair of revision numbers that returns a delta between them. -``flagsfn`` + If not defined, a delta will be computed by invoking mdiff code + on ``store.revision()`` results. + + Defining this function allows a precomputed or stored delta to be + used without having to compute on. + +``flagsfn`` (optional) Callable receiving a revision number and returns the integer flags - value for it. + value for it. If not defined, flags value will be 0. ``sendfulltext`` Whether to send fulltext revisions instead of deltas, if allowed. @@ -327,9 +343,13 @@ continue node = fnode(rev) -deltaparentrev = deltaparentfn(rev) p1rev, p2rev = store.parentrevs(rev) +if deltaparentfn: +deltaparentrev = deltaparentfn(rev) +else: +deltaparentrev = nullrev + # Forced delta against previous mode. if deltaprevious: baserev = prevrev @@ -373,7 +393,7 @@ # But we can't actually use our chosen delta base for whatever # reason. Reset to fulltext. -if baserev != nullrev and not candeltafn(baserev, rev): +if baserev != nullrev and (candeltafn and not candeltafn(baserev, rev)): baserev = nullrev revision = None @@ -388,21 +408,30 @@ revision = e.tombstone if baserev != nullrev: -baserevisionsize = rawsizefn(baserev) +if rawsizefn: +baserevisionsize = rawsizefn(baserev) +else: +baserevisionsize = len(store.revision(baserev, + raw=True)) elif baserev == nullrev and not deltaprevious: revision = store.revision(node, raw=True) available.add(rev) else: -delta = revdifffn(baserev, rev) +if revdifffn: +delta = revdifffn(baserev, rev) +else: +delta = mdiff.textdiff(store.revision(baserev, raw=True), + store.revision(rev, raw=True)) + available.add(rev) yield resultcls( node=node, p1node=fnode(p1rev), p2node=fnode(p2rev), basenode=fnode(baserev), -flags=flagsfn(rev), +flags=flagsfn(rev) if flagsfn else 0, baserevisionsize=baserevisionsize, revision=revision, delta=delta) To:
D4803: storageutil: extract most of emitrevisions() to standalone function
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY As part of implementing a storage backend, I found myself copying most of revlog.emitrevisions(). This code is highly nuanced and it bothered me greatly to be copying such low-level code. This commit extracts the bulk of revlog.emitrevisions() into a new standalone function. In order to make the function generally usable, all "self" function calls that aren't exposed on the ifilestorage interface are passed in via callable arguments. No meaningful behavior should have changed as part of the port. Upcoming commits will tweak behavior to make the code more generically usable. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4803 AFFECTED FILES mercurial/revlog.py mercurial/utils/storageutil.py CHANGE DETAILS diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -262,3 +262,149 @@ futurelargelinkrevs.add(plinkrev) return strippoint, brokenrevs + +def emitrevisions(store, revs, resultcls, deltaparentfn, candeltafn, + rawsizefn, revdifffn, flagsfn, sendfulltext=False, + revisiondata=False, assumehaveparentrevisions=False, + deltaprevious=False): +"""Generic implementation of ifiledata.emitrevisions(). + +Emitting revision data is subtly complex. This function attempts to +encapsulate all the logic for doing so in a backend-agnostic way. + +``store`` + Object conforming to ``ifilestorage`` interface. + +``revs`` + List of integer revision numbers whose data to emit. + +``resultcls`` + A type implementing the ``irevisiondelta`` interface that will be + constructed and returned. + +``deltaparentfn`` + Callable receiving a revision number and returning the revision number + of a revision that the internal delta is stored against. This delta + will be preferred over computing a new arbitrary delta. + +``candeltafn`` + Callable receiving a pair of revision numbers that returns a bool + indicating whether a delta between them can be produced. + +``rawsizefn`` + Callable receiving a revision number and returning the length of the + ``store.revision(rev, raw=True)``. + +``revdifffn`` + Callable receiving a pair of revision numbers that returns a delta + between them. + +``flagsfn`` + Callable receiving a revision number and returns the integer flags + value for it. + +``sendfulltext`` + Whether to send fulltext revisions instead of deltas, if allowed. + +``revisiondata`` +``assumehaveparentrevisions`` +``deltaprevious`` + See ``ifiledata.emitrevisions()`` interface documentation. +""" + +fnode = store.node + +prevrev = None + +if deltaprevious or assumehaveparentrevisions: +prevrev = store.parentrevs(revs[0])[0] + +# Set of revs available to delta against. +available = set() + +for rev in revs: +if rev == nullrev: +continue + +node = fnode(rev) +deltaparentrev = deltaparentfn(rev) +p1rev, p2rev = store.parentrevs(rev) + +# Forced delta against previous mode. +if deltaprevious: +baserev = prevrev + +# We're instructed to send fulltext. Honor that. +elif sendfulltext: +baserev = nullrev + +# There is a delta in storage. We try to use that because it +# amounts to effectively copying data from storage and is +# therefore the fastest. +elif deltaparentrev != nullrev: +# Base revision was already emitted in this group. We can +# always safely use the delta. +if deltaparentrev in available: +baserev = deltaparentrev + +# Base revision is a parent that hasn't been emitted already. +# Use it if we can assume the receiver has the parent revision. +elif (assumehaveparentrevisions + and deltaparentrev in (p1rev, p2rev)): +baserev = deltaparentrev + +# No guarantee the receiver has the delta parent. Send delta +# against last revision (if possible), which in the common case +# should be similar enough to this revision that the delta is +# reasonable. +elif prevrev is not None: +baserev = prevrev +else: +baserev = nullrev + +# Storage has a fulltext revision. + +# Let's use the previous revision, which is as good a guess as any. +# There is definitely room to improve this logic. +elif prevrev is not None: +baserev = prevrev +else: +baserev = nullrev + +
D4805: storageutil: pass nodes into emitrevisions()
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The main emitrevisions() uses nodes. So it makes sense to use nodes for the helper API. Not bothering with API since this function was introduced a few commits ago. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4805 AFFECTED FILES mercurial/revlog.py mercurial/utils/storageutil.py CHANGE DETAILS diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py --- a/mercurial/utils/storageutil.py +++ b/mercurial/utils/storageutil.py @@ -17,6 +17,7 @@ nullrev, ) from .. import ( +dagop, error, mdiff, pycompat, @@ -264,8 +265,8 @@ return strippoint, brokenrevs -def emitrevisions(store, revs, resultcls, deltaparentfn=None, candeltafn=None, - rawsizefn=None, revdifffn=None, flagsfn=None, +def emitrevisions(store, nodes, nodesorder, resultcls, deltaparentfn=None, + candeltafn=None, rawsizefn=None, revdifffn=None, flagsfn=None, sendfulltext=False, revisiondata=False, assumehaveparentrevisions=False, deltaprevious=False): @@ -277,8 +278,8 @@ ``store`` Object conforming to ``ifilestorage`` interface. -``revs`` - List of integer revision numbers whose data to emit. +``nodes`` + List of revision nodes whose data to emit. ``resultcls`` A type implementing the ``irevisiondelta`` interface that will be @@ -322,13 +323,23 @@ ``sendfulltext`` Whether to send fulltext revisions instead of deltas, if allowed. +``nodesorder`` ``revisiondata`` ``assumehaveparentrevisions`` ``deltaprevious`` See ``ifiledata.emitrevisions()`` interface documentation. """ fnode = store.node +frev = store.rev + +if nodesorder == 'nodes': +revs = [frev(n) for n in nodes] +elif nodesorder == 'storage': +revs = sorted(frev(n) for n in nodes) +else: +revs = set(frev(n) for n in nodes) +revs = dagop.linearize(revs, store.parentrevs) prevrev = None diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -2187,19 +2187,8 @@ if nodesorder is None and not self._generaldelta: nodesorder = 'storage' -frev = self.rev - -if nodesorder == 'nodes': -revs = [frev(n) for n in nodes] -elif nodesorder == 'storage': -revs = sorted(frev(n) for n in nodes) -else: -assert self._generaldelta -revs = set(frev(n) for n in nodes) -revs = dagop.linearize(revs, self.parentrevs) - return storageutil.emitrevisions( -self, revs, revlogrevisiondelta, +self, nodes, nodesorder, revlogrevisiondelta, deltaparentfn=self.deltaparent, candeltafn=self.candelta, rawsizefn=self.rawsize, To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 5] pullreport: add a test to show misreporting of visible changeset
# HG changeset patch
# User Boris Feld
# Date 1538064373 -7200
# Thu Sep 27 18:06:13 2018 +0200
# Node ID b36914d9928effac212d851c9617de93d6260746
# Parent 850324b80f9c305a14ea37740dabd3abbc6e4f1f
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
b36914d9928e
pullreport: add a test to show misreporting of visible changeset
The current code ignores all obsolete changesets including the visible one. We
add a test showing this behavior before fixing the behavior.
diff --git a/tests/test-obsolete-distributed.t
b/tests/test-obsolete-distributed.t
--- a/tests/test-obsolete-distributed.t
+++ b/tests/test-obsolete-distributed.t
@@ -487,3 +487,55 @@ decision is made in that case, so receiv
ef908e42ce65ef57f970d799acaddde26f58a4cc
5ffb9e311b35f6ab6f76f667ca5d6e595645481b 0 (Thu Jan 01 00:00:00 1970 +)
{'ef1': '4', 'operation': 'rebase', 'user': 'bob'}
$ cd ..
+
+Test pull report consistency
+
+
+obsolete but visible should be reported
+---
+
+Setup
+
+ $ hg init repo-a
+ $ cat << EOF >> repo-a/.hg/hgrc
+ > [ui]
+ > username=test
+ > EOF
+ $ cd repo-a
+ $ hg debugbuilddag ..
+ $ hg debugobsolete `getid tip`
+ obsoleted 1 changesets
+ $ cd ../
+ $ hg clone --pull repo-a repo-b
+ requesting all changes
+ adding changesets
+ adding manifests
+ adding file changes
+ added 1 changesets with 0 changes to 0 files
+ new changesets 1ea73414a91b (1 drafts)
+ updating to branch default
+ 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ $ hg -R repo-a up tip --hidden
+ 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ updated to hidden changeset 66f7d451a68b
+ (hidden revision '66f7d451a68b' is pruned)
+ $ hg -R repo-a branch foo
+ marked working directory as branch foo
+ (branches are permanent and global, did you want a bookmark?)
+ $ hg -R repo-a commit -m foo
+ 1 new orphan changesets
+
+Actual test
+(BROKEN)
+
+ $ hg -R repo-b pull
+ pulling from
$TESTTMP/distributed-chain-building/distributed-chain-building/repo-a
+ searching for changes
+ adding changesets
+ adding manifests
+ adding file changes
+ added 2 changesets with 0 changes to 0 files
+ 1 new obsolescence markers
+ 1 new orphan changesets
+ new changesets 95d586532b49 (1 drafts)
+ (run 'hg update' to get a working copy)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 5] pullreport: skip filtered revs instead of obsolete ones
# HG changeset patch
# User Boris Feld
# Date 1538058910 -7200
# Thu Sep 27 16:35:10 2018 +0200
# Node ID 4bd42e72e7ba8c0ee9dc4e153127882e6961602a
# Parent b36914d9928effac212d851c9617de93d6260746
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
4bd42e72e7ba
pullreport: skip filtered revs instead of obsolete ones
Obsolescence is closely related to visibility but still a distinct concept. We
can receive changesets that are obsolete but visible (eg: when pulling
orphans). Such changeset should be reported too. In addition, the filtering
level can be anything, we should respect it.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1603,13 +1603,11 @@ def registersummarycallback(repo, otr, t
if origrepolen >= len(repo):
return
-# Compute the bounds of new revisions' range, excluding obsoletes.
-unfi = repo.unfiltered()
-revs = unfi.revs('%d: and not obsolete()', origrepolen)
+# Compute the bounds of new visible revisions' range.
+revs = list(repo.changelog.revs(start=origrepolen))
if not revs:
-# Got only obsoletes.
return
-minrev, maxrev = repo[revs.min()], repo[revs.max()]
+minrev, maxrev = repo[revs[0]], repo[revs[-1]]
if minrev == maxrev:
revrange = minrev
diff --git a/tests/test-obsolete-distributed.t
b/tests/test-obsolete-distributed.t
--- a/tests/test-obsolete-distributed.t
+++ b/tests/test-obsolete-distributed.t
@@ -537,5 +537,5 @@ Actual test
added 2 changesets with 0 changes to 0 files
1 new obsolescence markers
1 new orphan changesets
- new changesets 95d586532b49 (1 drafts)
+ new changesets 66f7d451a68b:95d586532b49 (2 drafts)
(run 'hg update' to get a working copy)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 5] pullreport: issue a message about "extinct" pulled changesets
# HG changeset patch
# User Boris Feld
# Date 1538059945 -7200
# Thu Sep 27 16:52:25 2018 +0200
# Node ID cd5d5d6586b124fc7d4476223ccce0bc8eb04c50
# Parent e91cce6bdd293a7c498bac3925b47a9f94dd22e9
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
cd5d5d6586b1
pullreport: issue a message about "extinct" pulled changesets
Changeset pulled from a remote repository while already obsolete locally can
end up hidden after the pull. Hiding obsolete changesets is a good behavior
but silently "skipping" some of the pulled content can get confusing.
We now detect this situation and emit a message about it. The message is
simple and the wording could be improved, however, we focus on the detection
here. Evolution is still an experimental feature, so the output is open to
changes.
In particular, we could point out at the latest successors of the obsolete
changesets, however, it can get tricky is there are many of them. So we delay
these improvements to another adventure.
Another easy improvement would be to merge this message with the previous line
about the new nodes and their phases.
This is a good example of cases where we can only transmit a limited amount of
data to users by default. We need some sort of "transaction journal" we could
point the user to.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1631,6 +1631,17 @@ def registersummarycallback(repo, otr, t
raise error.ProgrammingError(errormsg)
repo.ui.status(msg)
+# search new changesets directly pulled as obsolete
+obsadded = unfi.revs('%d: and obsolete()', origrepolen)
+cl = repo.changelog
+extinctadded = [r for r in obsadded if r not in cl]
+if extinctadded:
+# They are not just obsolete, but obsolete and invisible
+# we call them "extinct" internally but the terms have not been
+# exposed to users.
+msg = '(%d other changesets obsolete on arrival)\n'
+repo.ui.status(msg % len(extinctadded))
+
@reportsummary
def reportphasechanges(repo, tr):
"""Report statistics of phase changes for changesets pre-existing
diff --git a/tests/test-obsolete-bundle-strip.t
b/tests/test-obsolete-bundle-strip.t
--- a/tests/test-obsolete-bundle-strip.t
+++ b/tests/test-obsolete-bundle-strip.t
@@ -170,6 +170,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-A1")'
@@ -248,6 +249,7 @@ Actual testing
# unbundling: added 2 changesets with 2 changes to 2 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with prune children
@@ -339,6 +341,7 @@ problematic)
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files
# unbundling: 1 new obsolescence markers
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg update' to get a working copy)
$ testrevs 'desc("C-A1")'
@@ -437,6 +440,7 @@ bundling multiple revisions
# unbundling: added 3 changesets with 3 changes to 3 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (2 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with precursors also pruned
@@ -503,6 +507,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-A1")'
@@ -578,6 +583,7 @@ Actual testing
# unbundling: added 2 changesets with 2 changes to 2 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with missing prune
@@ -836,6 +842,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-B")'
@@ -864,6 +871,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
#
[PATCH 3 of 5] pullreport: skip or rework some early return
# HG changeset patch
# User Boris Feld
# Date 1538060400 -7200
# Thu Sep 27 17:00:00 2018 +0200
# Node ID e91cce6bdd293a7c498bac3925b47a9f94dd22e9
# Parent 4bd42e72e7ba8c0ee9dc4e153127882e6961602a
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
e91cce6bdd29
pullreport: skip or rework some early return
We are about to add more logic in this report. Before that, we need it to not
quit so early.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1600,35 +1600,36 @@ def registersummarycallback(repo, otr, t
def reportnewcs(repo, tr):
"""Report the range of new revisions pulled/unbundled."""
origrepolen = tr.changes.get('origrepolen', len(repo))
-if origrepolen >= len(repo):
+unfi = repo.unfiltered()
+if origrepolen >= len(unfi):
return
# Compute the bounds of new visible revisions' range.
revs = list(repo.changelog.revs(start=origrepolen))
-if not revs:
-return
-minrev, maxrev = repo[revs[0]], repo[revs[-1]]
+if revs:
+minrev, maxrev = repo[revs[0]], repo[revs[-1]]
-if minrev == maxrev:
-revrange = minrev
-else:
-revrange = '%s:%s' % (minrev, maxrev)
-draft = len(repo.revs('%ld and draft()', revs))
-secret = len(repo.revs('%ld and secret()', revs))
-if not (draft or secret):
-msg = _('new changesets %s\n') % revrange
-elif draft and secret:
-msg = _('new changesets %s (%d drafts, %d secrets)\n')
-msg %= (revrange, draft, secret)
-elif draft:
-msg = _('new changesets %s (%d drafts)\n')
-msg %= (revrange, draft)
-elif secret:
-msg = _('new changesets %s (%d secrets)\n')
-msg %= (revrange, secret)
-else:
-raise error.ProgrammingError('entered unreachable condition')
-repo.ui.status(msg)
+if minrev == maxrev:
+revrange = minrev
+else:
+revrange = '%s:%s' % (minrev, maxrev)
+draft = len(repo.revs('%ld and draft()', revs))
+secret = len(repo.revs('%ld and secret()', revs))
+if not (draft or secret):
+msg = _('new changesets %s\n') % revrange
+elif draft and secret:
+msg = _('new changesets %s (%d drafts, %d secrets)\n')
+msg %= (revrange, draft, secret)
+elif draft:
+msg = _('new changesets %s (%d drafts)\n')
+msg %= (revrange, draft)
+elif secret:
+msg = _('new changesets %s (%d secrets)\n')
+msg %= (revrange, secret)
+else:
+errormsg = 'entered unreachable condition'
+raise error.ProgrammingError(errormsg)
+repo.ui.status(msg)
@reportsummary
def reportphasechanges(repo, tr):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 5] pullreport: rev duplicated and extinct into account
# HG changeset patch
# User Boris Feld
# Date 1538060106 -7200
# Thu Sep 27 16:55:06 2018 +0200
# Node ID 779c6fdd8cf7057f93785f4f653ae2c4860af576
# Parent cd5d5d6586b124fc7d4476223ccce0bc8eb04c50
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
779c6fdd8cf7
pullreport: rev duplicated and extinct into account
If we already have some obsolete and hidden nodes locally and the server send
them again to you, it seems useful to point it out instead of being silent about
it.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1632,7 +1632,9 @@ def registersummarycallback(repo, otr, t
repo.ui.status(msg)
# search new changesets directly pulled as obsolete
-obsadded = unfi.revs('%d: and obsolete()', origrepolen)
+duplicates = tr.changes.get('revduplicates', ())
+obsadded = unfi.revs('(%d: + %ld) and obsolete()',
+ origrepolen, duplicates)
cl = repo.changelog
extinctadded = [r for r in obsadded if r not in cl]
if extinctadded:
diff --git a/tests/test-obsolete.t b/tests/test-obsolete.t
--- a/tests/test-obsolete.t
+++ b/tests/test-obsolete.t
@@ -806,6 +806,7 @@ check hgweb does not explode
adding file changes
added 62 changesets with 63 changes to 9 files (+60 heads)
new changesets 50c51b361e60:c15e9edfca13 (62 drafts)
+ (2 other changesets obsolete on arrival)
(run 'hg heads .' to see heads, 'hg merge' to merge)
$ for node in `hg log -r 'desc(babar_)' --template '{node}\n'`;
> do
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] py3: ensure printing to stdout uses str in test-hgweb-no-request-uri.t
On Fri, 28 Sep 2018 00:38:59 -0400, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison
> # Date 1538100432 14400
> # Thu Sep 27 22:07:12 2018 -0400
> # Node ID 777d233b5174c0ca9789080b52225247eb36bdec
> # Parent 591764c38fedebc1bd32d5454603888ca57c1656
> py3: ensure printing to stdout uses str in test-hgweb-no-request-uri.t
Queued, thanks.
> @@ -55,11 +56,11 @@ should be used from d74fc8dec2b4 onward
>>
>> def process(app):
>> content = app(env, startrsp)
> - > sys.stdout.write(output.getvalue())
> - > sys.stdout.write(''.join(content))
> + > sys.stdout.write(encoding.strfromlocal(output.getvalue()))
> + > sys.stdout.write(encoding.strfromlocal(b''.join(content)))
It's probably better to write directly to the underlying bytes stream (i.e.
sys.stdout.buffer), but I don't care as this is a test code.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4781: repo: don't look up context for tip node if it's not needed
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1834dabb38e0: repo: dont look up context for tip node
if its not needed (authored by martinvonz, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4781?vs=11453=11466
REVISION DETAIL
https://phab.mercurial-scm.org/D4781
AFFECTED FILES
contrib/python-hook-examples.py
hgext/mq.py
tests/test-commit.t
CHANGE DETAILS
diff --git a/tests/test-commit.t b/tests/test-commit.t
--- a/tests/test-commit.t
+++ b/tests/test-commit.t
@@ -650,7 +650,7 @@
> def filectxfn(repo, memctx, path):
> return context.memfilectx(repo, memctx, path,
> b'[hooks]\nupdate = echo owned')
- > c = context.memctx(r, [r[b'tip'].node(), node.nullid],
+ > c = context.memctx(r, [r.changelog.tip(), node.nullid],
>b'evil', [notrc], filectxfn, 0)
> r.commitctx(c)
> EOF
diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -980,10 +980,10 @@
files += mergedsubstate.keys()
match = scmutil.matchfiles(repo, files or [])
-oldtip = repo['tip']
+oldtip = repo.changelog.tip()
n = newcommit(repo, None, message, ph.user, ph.date, match=match,
force=True)
-if repo['tip'] == oldtip:
+if repo.changelog.tip() == oldtip:
raise error.Abort(_("qpush exactly duplicates child
changeset"))
if n is None:
raise error.Abort(_("repository commit failed"))
diff --git a/contrib/python-hook-examples.py b/contrib/python-hook-examples.py
--- a/contrib/python-hook-examples.py
+++ b/contrib/python-hook-examples.py
@@ -19,7 +19,7 @@
node = kwargs['node']
first = repo[node].p1().node()
if 'url' in kwargs:
-last = repo['tip'].node()
+last = repo.changelog.tip()
else:
last = node
diff = patch.diff(repo, first, last)
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4779: bundle: consistently put revnums in "base" collection
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG84a44ed337c2: bundle: consistently put revnums in
base collection (authored by martinvonz, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4779?vs=11451=11464
REVISION DETAIL
https://phab.mercurial-scm.org/D4779
AFFECTED FILES
mercurial/commands.py
CHANGE DETAILS
diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -1224,7 +1224,7 @@
"a destination"))
if opts.get('base'):
ui.warn(_("ignoring --base because --all was specified\n"))
-base = ['null']
+base = [nullrev]
else:
base = scmutil.revrange(repo, opts.get('base'))
if cgversion not in changegroup.supportedoutgoingversions(repo):
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 5] pullreport: skip filtered revs instead of obsolete ones
On 28/09/2018 13:56, Yuya Nishihara wrote:
On Fri, 28 Sep 2018 12:22:19 +0200, Boris Feld wrote:
# HG changeset patch
# User Boris Feld
# Date 1538058910 -7200
# Thu Sep 27 16:35:10 2018 +0200
# Node ID 4bd42e72e7ba8c0ee9dc4e153127882e6961602a
# Parent b36914d9928effac212d851c9617de93d6260746
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
4bd42e72e7ba
pullreport: skip filtered revs instead of obsolete ones
Obsolescence is closely related to visibility but still a distinct concept. We
can receive changesets that are obsolete but visible (eg: when pulling
orphans). Such changeset should be reported too. In addition, the filtering
level can be anything, we should respect it.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1603,13 +1603,11 @@ def registersummarycallback(repo, otr, t
if origrepolen >= len(repo):
return
-# Compute the bounds of new revisions' range, excluding obsoletes.
-unfi = repo.unfiltered()
-revs = unfi.revs('%d: and not obsolete()', origrepolen)
+# Compute the bounds of new visible revisions' range.
+revs = list(repo.changelog.revs(start=origrepolen))
Use revset? It's probably better to not construct a list of tens of thousands
of integers.
The issue here is that we don't have a way to express what we want as a
revset (to our knowledge). The revnum `origrepolen` could be filtered so
it cannot be used explicitly in "%d"
We either need:
unfi.revs("%d - _filteredrevs(%s)", origrepolen, repo.filtername)
or
repo.revs("firstrevgreaterthan(%s):", origrepolen)
Should we implement either one of the possibility above or do you have
another idea?
if not revs:
-# Got only obsoletes.
return
-minrev, maxrev = repo[revs.min()], repo[revs.max()]
+minrev, maxrev = repo[revs[0]], repo[revs[-1]]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 5 RFC] rust: iterator bindings to C code
# HG changeset patch
# User Georges Racinet
# Date 1538059896 -7200
# Thu Sep 27 16:51:36 2018 +0200
# Node ID de88c09512565ed1c12e2ff9159e06ed8d762d15
# Parent d8c9571755a64e1fc3429587dfd3949b9862eceb
# EXP-Topic rustancestors-rfc
rust: iterator bindings to C code
In this changeset, still made of Rust code only,
we expose the Rust iterator for instantiation and
consumption from C code.
The idea is that both the index and index_get_parents()
will be passed from the C extension, hence avoiding a hard
link dependency to parsers.so, so that the crate can
still be built and tested independently.
On the other hand, parsers.so will use the symbols
defined in this changeset.
diff -r d8c9571755a6 -r de88c0951256 mercurial/rust/Cargo.lock
--- a/mercurial/rust/Cargo.lock Thu Sep 27 17:03:16 2018 +0200
+++ b/mercurial/rust/Cargo.lock Thu Sep 27 16:51:36 2018 +0200
@@ -1,4 +1,14 @@
[[package]]
name = "hgancestors"
version = "0.1.0"
+dependencies = [
+ "libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
+]
+[[package]]
+name = "libc"
+version = "0.2.43"
+source = "registry+https://github.com/rust-lang/crates.io-index;
+
+[metadata]
+"checksum libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)"
= "76e3a3ef172f1a0b9a9ff0dd1491ae5e6c948b94479a3021819ba7d860c8645d"
diff -r d8c9571755a6 -r de88c0951256 mercurial/rust/Cargo.toml
--- a/mercurial/rust/Cargo.toml Thu Sep 27 17:03:16 2018 +0200
+++ b/mercurial/rust/Cargo.toml Thu Sep 27 16:51:36 2018 +0200
@@ -2,3 +2,9 @@
name = "hgancestors"
version = "0.1.0"
authors = ["Georges Racinet "]
+
+[dependencies]
+libc = "*"
+
+[lib]
+crate-type = ["dylib"]
diff -r d8c9571755a6 -r de88c0951256 mercurial/rust/src/cpython.rs
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/rust/src/cpython.rs Thu Sep 27 16:51:36 2018 +0200
@@ -0,0 +1,170 @@
+// Copyright 2018 Georges Racinet
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+//! Bindings for CPython extension code
+//!
+//! This exposes methods to build and use a `rustlazyancestors` iterator
+//! from C code, using an index and its parents function that are passed
+//! from the caller at instantiation.
+
+extern crate libc;
+
+use std::boxed::Box;
+use self::libc::{c_void, c_int, ssize_t};
+use super::ancestors::{Graph, AncestorsIterator};
+
+type IndexPtr = *mut c_void;
+type IndexParentsFn = extern "C" fn(index: IndexPtr,
+rev: ssize_t,
+ps: *mut [c_int; 2],
+max_rev: c_int)
+-> c_int;
+
+/// A Graph backed up by objects and functions from revlog.c
+///
+/// This implementation of the Graph trait, relies on (pointers to)
+/// - the C index object (`index` member)
+/// - the `index_get_parents()` function (`parents` member)
+pub struct Index {
+index: IndexPtr,
+parents: IndexParentsFn,
+}
+
+impl Index {
+pub fn new(index: IndexPtr, parents: IndexParentsFn) -> Self {
+Index {
+index: index,
+parents: parents,
+}
+}
+}
+
+impl Graph for Index {
+/// wrap a call to the C extern parents function
+fn parents(, rev: i32) -> (i32, i32) {
+let mut res: [c_int; 2] = [0; 2];
+// surprisingly the call below is not unsafe, whereas calling the
+// same extern function directly (not through a pointer) would.
+// Maybe that depends on rustc version, though.
+let code = (self.parents)(
+self.index,
+rev as ssize_t,
+ res as *mut [c_int; 2],
+rev,
+);
+if code != 0 {
+// TODO panic! and FFI don't get well together
+panic!("Corrupted index");
+}
+(res[0], res[1])
+}
+}
+
+/// Wrapping of AncestorsIterator constructor, for C callers.
+///
+/// Besides `initrevs`, `stoprev` and `inclusive`, that are converted
+/// we receive the index and the parents function as pointers
+#[no_mangle]
+pub extern "C" fn rustlazyancestors_init(
+index: IndexPtr,
+parents: IndexParentsFn,
+initrevslen: usize,
+initrevs: *mut i64,
+stoprev: i64,
+inclusive: i64,
+) -> *mut AncestorsIterator {
+let v: Vec =
+unsafe { Vec::from_raw_parts(initrevs, initrevslen, initrevslen) };
+let inclb = match inclusive {
+0 => false,
+1 => true,
+_ => panic!("Did not understand boolean FFI convention"),
+};
+
+Box::into_raw(Box::new(AncestorsIterator::new(
+Index::new(index, parents),
+,
+stoprev,
+inclb,
+)))
+}
+
+/// Deallocator to be called from C code
+#[no_mangle]
+pub extern "C" fn rustlazyancestors_drop(
+raw_iter: *mut AncestorsIterator,
+) {
+unsafe {
+Box::from_raw(raw_iter);
+}
+}
+
+/// Iteration main method to be called from C code
[PATCH 1 of 5 RFC] rust: pure Rust lazyancestors iterator
# HG changeset patch
# User Georges Racinet
# Date 1538060596 -7200
# Thu Sep 27 17:03:16 2018 +0200
# Node ID d8c9571755a64e1fc3429587dfd3949b9862eceb
# Parent d3d4b4b5f725124ef9e93cf74d779a7a05aa11b7
# EXP-Topic rustancestors-rfc
rust: pure Rust lazyancestors iterator
This is the first of a patch series aiming to provide an
alternative implementation in the Rust programming language
of the _lazyancestorsiter from the ancestor module.
This iterator has been brought to our attention by the people at
Octobus, as a potential good candidate for incremental "oxydation"
(rewriting in Rust), because it has shown performance issues lately
and it merely deals with ints (revision numbers) obtained by calling
the index, whih should be directly callable from Rust code,
being itself implemented as a C extension.
The idea behind this series is to provide a minimal example of Rust code
collaborating with existing C and Python code. To open the way to gradually
rewriting more of Mercurial's python code in Rust, without being forced to pay
a large initial cost of rewriting the existing fast core into Rust.
This patch does not introduce any bindings to other Mercurial code
yet, but someone with a rustc/cargo installation may chdir into
mercurial/rust and run the tests by issuing:
cargo test
The algorithm is a bit simplified (see details in docstrings),
and at its simplest becomes rather trivial, showcasing that Rust has
batteries included too: BinaryHeap, the Rust analog of Python's heapq
does actually all the work.
The implementation can be further optimized and be made more idiomatic
Rust.
diff -r d3d4b4b5f725 -r d8c9571755a6 mercurial/rust/Cargo.lock
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/rust/Cargo.lock Thu Sep 27 17:03:16 2018 +0200
@@ -0,0 +1,4 @@
+[[package]]
+name = "hgancestors"
+version = "0.1.0"
+
diff -r d3d4b4b5f725 -r d8c9571755a6 mercurial/rust/Cargo.toml
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/rust/Cargo.toml Thu Sep 27 17:03:16 2018 +0200
@@ -0,0 +1,4 @@
+[package]
+name = "hgancestors"
+version = "0.1.0"
+authors = ["Georges Racinet "]
diff -r d3d4b4b5f725 -r d8c9571755a6 mercurial/rust/rustfmt.toml
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/rust/rustfmt.toml Thu Sep 27 17:03:16 2018 +0200
@@ -0,0 +1,3 @@
+max_width = 79
+wrap_comments = true
+error_on_line_overflow = true
diff -r d3d4b4b5f725 -r d8c9571755a6 mercurial/rust/src/ancestors.rs
--- /dev/null Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/rust/src/ancestors.rs Thu Sep 27 17:03:16 2018 +0200
@@ -0,0 +1,203 @@
+// ancestors.rs
+//
+// Copyright 2018 Georges Racinet
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+//! Rust versions of generic DAG ancestors algorithms for Mercurial
+
+use std::iter::Iterator;
+use std::collections::{BinaryHeap, HashSet};
+
+/// The simplest expression of what we need of Mercurial DAGs.
+///
+/// As noted in revlog.c, revision numbers are actually encoded in
+/// 4 bytes, and are liberally converted to ints, whence the i32. It would
+/// probably be a good idea to make this trait generic over a suitable integer
+/// type.
+pub trait Graph {
+fn parents(, i32) -> (i32, i32);
+}
+
+/// Iterator over the ancestors of a given list of revisions
+/// This is a generic type, defined and implemented for any Graph, so that
+/// it's easy to
+///
+/// - unit test in pure Rust
+/// - bind to main Mercurial code, potentially in several ways and have these
+/// bindings evolve over time
+pub struct AncestorsIterator {
+graph: G,
+visit: BinaryHeap,
+seen: HashSet,
+stoprev: i32,
+}
+
+impl AncestorsIterator {
+/// Constructor.
+///
+/// We actually expect initrevs to be i64, and we coerce them to the
+/// i32 of our internal representations. The reason is that from C
+/// code, they actually come as such.
+/// Here it would be better to take any iterable over our internal
+/// revision numbers, and have the conversion be made from the caller.
+pub fn new(
+graph: G,
+initrevs: ,
+stoprev: i64,
+inclusive: bool,
+) -> Self {
+let stop32 = stoprev as i32;
+// because in iteration, contrary to what happens in ancestor.py, we
+// compare with stoprev before pushing, we have to prefilter in the
+// constructor too.
+let deref_filter = initrevs.iter()
+.map(|| x as i32).filter(|x| stop32 <= *x);
+
+if inclusive {
+return AncestorsIterator {
+visit: deref_filter.clone().collect(),
+seen: deref_filter.collect(),
+stoprev: stop32,
+graph: graph,
+};
+}
+let mut this = AncestorsIterator {
+visit: BinaryHeap::new(),
+seen: HashSet::new(),
+stoprev: stop32,
+
[PATCH 5 of 5 RFC] rust: making runnable without LD_LIBRARY_PATH
# HG changeset patch # User Georges Racinet # Date 1538059603 -7200 # Thu Sep 27 16:46:43 2018 +0200 # Node ID e754741646b16fee5534974da44b112a036404c1 # Parent 0fcc7c5de05aa47449cc428e826ca2e76c7517ec # EXP-Topic rustancestors-rfc rust: making runnable without LD_LIBRARY_PATH Building the Rust code as a static library makes setup.py link it within parsers.so, so that this Rust enhanced proof-of-concept can be tested, benched etc with no modification of tooling, besides the need for a working rustc/cargo (ususally also involving GitHub access). In the long run, a better runtime linking solution should probably be investigated. Notably, if we needed to enhance several extensions with Rust, then this staticlib hack would result in a copy of the Rust standard libraries in each produced extension. diff -r 0fcc7c5de05a -r e754741646b1 mercurial/rust/Cargo.toml --- a/mercurial/rust/Cargo.toml Thu Sep 27 16:55:44 2018 +0200 +++ b/mercurial/rust/Cargo.toml Thu Sep 27 16:46:43 2018 +0200 @@ -7,4 +7,4 @@ libc = "*" [lib] -crate-type = ["dylib"] +crate-type = ["staticlib"] ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 5 RFC] rust: pure Rust lazyancestors iterator
On 9/28/18 3:31 PM, Georges Racinet wrote: > # HG changeset patch > # User Georges Racinet > # Date 1538060596 -7200 > # Thu Sep 27 17:03:16 2018 +0200 > # Node ID d8c9571755a64e1fc3429587dfd3949b9862eceb > # Parent d3d4b4b5f725124ef9e93cf74d779a7a05aa11b7 > # EXP-Topic rustancestors-rfc > rust: pure Rust lazyancestors iterator Hi, since this is my first attempt to contribute to Mercurial, I feel it's appropriate to introduce myself. I've been using Mercurial professionally for more than ten years, mostly within small teams of developers, so I guess it's time to give back to the community and thank everyone that's been working hard to make it happen. In the past few years, I've grown a big fondness for the Rust programming language, and been looking for opportunities to use it besides some personal toy projects. I'm aware that there's probably much to be said about this patch series, at least with code organization, choice of names etc. I read the general guidelines, tried and respect them, but it can't be perfect. I'm looking forward to comments on this RFC, especially from the authors of the OxidationPlan wiki page, hoping the approach taken in this patch series would fit enough with that. Regards, -- Georges Racinet Anybox SAS, http://anybox.fr Téléphone: +33 6 51 32 07 27 GPG: B59E 22AB B842 CAED 77F7 7A7F C34F A519 33AB 0A35, sur serveurs publics ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 5 RFC] rust: exposing in parsers module
# HG changeset patch
# User Georges Racinet
# Date 1538060175 -7200
# Thu Sep 27 16:56:15 2018 +0200
# Node ID d834b99b2d6588e8ed42dd36685a2dfc2d78fd8e
# Parent de88c09512565ed1c12e2ff9159e06ed8d762d15
# EXP-Topic rustancestors-rfc
rust: exposing in parsers module
To build with the Rust code, set the HGWITHRUSTEXT
environment variable.
At this point, it's possible to instantiate and use
a rustlazyancestors object from a Python interpreter,
provided one points LD_LIBRARY_PATH to
mercurial/rust/target/release
The changes in setup.py are obviously a quick hack,
just good enough to test/bench without much
refactoring. We'd be happy to improve on that with
help from the community.
With respect to the plans at
https://www.mercurial-scm.org/wiki/OxidationPlan
this would probably qualify as "roll our own FFI".
Also, it doesn't quite meet the target of getting
rid of C code, since it brings actually more, yet:
- the new C code does nothing else than parsing
arguments and calling Rust functions.
In particular, there's no fancy allocation involved.
- subsequent changes could rewrite more of revlog.c, this
time resulting in an overall decrease of C code and
unsafety.
diff -r de88c0951256 -r d834b99b2d65 mercurial/cext/revlog.c
--- a/mercurial/cext/revlog.c Thu Sep 27 16:51:36 2018 +0200
+++ b/mercurial/cext/revlog.c Thu Sep 27 16:56:15 2018 +0200
@@ -2290,6 +2290,145 @@
return NULL;
}
+#ifdef WITH_RUST
+
+/* rustlazyancestors: iteration over ancestors implemented in Rust
+ *
+ * This class holds a reference to an index and to the Rust iterator.
+ */
+typedef struct rustlazyancestorsObjectStruct rustlazyancestorsObject;
+
+struct rustlazyancestorsObjectStruct {
+ PyObject_HEAD
+ /* Type-specific fields go here. */
+indexObject *index;/* Ref kept to avoid GC'ing the index */
+void *iter;/* Rust iterator */
+};
+
+/* FFI exposed from Rust code */
+rustlazyancestorsObject *rustlazyancestors_init(
+indexObject *index,
+/* to pass index_get_parents() */
+int (*)(indexObject *, Py_ssize_t, int*, int),
+/* intrevs vector */
+int initrevslen, long *initrevs,
+long stoprev,
+int inclusive);
+void rustlazyancestors_drop(rustlazyancestorsObject *self);
+int rustlazyancestors_next(rustlazyancestorsObject *self);
+
+/* CPython instance methods */
+static int rustla_init(rustlazyancestorsObject *self,
+ PyObject *args) {
+ PyObject *initrevsarg = NULL;
+ PyObject *inclusivearg = NULL;
+ long stoprev = 0;
+ long *initrevs = NULL;
+ int inclusive = 0;
+ Py_ssize_t i;
+
+ indexObject *index;
+ if (!PyArg_ParseTuple(args, "O!O!lO!",
+, ,
+_Type, ,
+,
+_Type, ))
+
+return -1;
+ Py_INCREF(index);
+ self->index = index;
+
+ if (inclusivearg == Py_True)
+inclusive = 1;
+
+ Py_ssize_t linit = PyList_GET_SIZE(initrevsarg);
+
+ initrevs = (long*)malloc(linit * sizeof(long));
+
+ if (initrevs == NULL) {
+PyErr_NoMemory();
+goto bail;
+ }
+
+ for (i=0; iiter = rustlazyancestors_init(index,
+ index_get_parents,
+ linit, initrevs,
+ stoprev, inclusive);
+ return 0;
+
+bail:
+free(initrevs);
+return -1;
+};
+
+static void rustla_dealloc(rustlazyancestorsObject *self)
+{
+ Py_XDECREF(self->index);
+ if (self->iter != NULL) { /* can happen if rustla_init failed */
+rustlazyancestors_drop(self->iter);
+ }
+ PyObject_Del(self);
+}
+
+static PyObject *rustla_next(rustlazyancestorsObject *self) {
+ int res = rustlazyancestors_next(self->iter);
+ if (res == -1) {
+/* Setting an explicit exception seems unnecessary
+ as examples from Python source code (Objects/rangeobjets.c and
+ Modules/_io/stringio.c) seem to demonstrate.
+ If that's false, then it would look like:
+ PyErr_SetNone(PyExc_StopIterationError);
+*/
+return NULL;
+ }
+ return PyInt_FromLong(res);
+}
+
+static PyTypeObject rustlazyancestorsType = {
+ PyVarObject_HEAD_INIT(NULL, 0) /* header */
+ "parsers.rustlazyancestors", /* tp_name */
+ sizeof(rustlazyancestorsObject), /* tp_basicsize */
+ 0, /* tp_itemsize */
+ (destructor)rustla_dealloc, /* tp_dealloc */
+ 0, /* tp_print */
+ 0, /* tp_getattr */
+ 0, /* tp_setattr */
+ 0, /* tp_compare */
+ 0, /* tp_repr */
+ 0, /* tp_as_number */
+ 0, /* tp_as_sequence */
+ 0, /* tp_as_mapping */
+ 0, /* tp_hash */
+ 0, /* tp_call */
+
[PATCH 4 of 5 RFC] rust: hooking into Python code
# HG changeset patch
# User Georges Racinet
# Date 1538060144 -7200
# Thu Sep 27 16:55:44 2018 +0200
# Node ID 0fcc7c5de05aa47449cc428e826ca2e76c7517ec
# Parent d834b99b2d6588e8ed42dd36685a2dfc2d78fd8e
# EXP-Topic rustancestors-rfc
rust: hooking into Python code
We introduce a new class called 'rustlazyancestors'
in the ancestors module, which is used only if
parsers.rustlazyancestors does exist.
The implementation of __contains__ stays unchanged,
but is now backed by the Rust iterator. It would
probably be a good candidate for further development,
though, as it is mostly looping, and duplicates the
'seen' set.
The Rust code could be further optimized, however it already
gives rise to performance improvements:
median timing from hg perfancestors:
- on pypy:
before: 0.113749s
after: 0.018628s -84%
- on mozilla central:
before: 0.329075s
after: 0.083889s -75%
- on a private repository (about one million revisions):
before: 1.982365s
after: 0.329907s -83%
- on another private repository (about 400 000 revisions):
before: 0.826686s
after: 0.123760s -85%
median timing for hg perfbranchmap base
- on pypy:
before: 1.808351s
after: 0.480814s -73%
- on mozilla central:
before: 18.493269s
after: 1.305514s -93%
- on a private repository (about one million revisions):
before: 9.153785s
after: 3.662155s -60%
- on another private repository (about 400 000 revisions):
before: 98.034737s
after: 18.109361s -81%
diff -r d834b99b2d65 -r 0fcc7c5de05a mercurial/ancestor.py
--- a/mercurial/ancestor.py Thu Sep 27 16:56:15 2018 +0200
+++ b/mercurial/ancestor.py Thu Sep 27 16:55:44 2018 +0200
@@ -11,9 +11,12 @@
from .node import nullrev
from . import (
+policy,
pycompat,
)
+parsers = policy.importmod(r'parsers')
+
def commonancestorsheads(pfunc, *nodes):
"""Returns a set with the heads of all common ancestors of all nodes,
heads(::nodes[0] and ::nodes[1] and ...) .
@@ -379,3 +382,25 @@
# free up memory.
self._containsiter = None
return False
+
+class rustlazyancestors(lazyancestors):
+
+def __init__(self, index, revs, stoprev=0, inclusive=False):
+self._index = index
+self._stoprev = stoprev
+self._inclusive = inclusive
+# no need to prefilter out init revs that are smaller than stoprev,
+# it's done by rustlazyancestors constructor.
+# we need to convert to a list, because our ruslazyancestors
+# constructor (from C code) doesn't understand anything else yet
+self._initrevs = initrevs = list(revs)
+
+self._containsseen = set()
+self._containsiter = parsers.rustlazyancestors(
+index, initrevs, stoprev, inclusive)
+
+def __iter__(self):
+return parsers.rustlazyancestors(self._index,
+ self._initrevs,
+ self._stoprev,
+ self._inclusive)
diff -r d834b99b2d65 -r 0fcc7c5de05a mercurial/changelog.py
--- a/mercurial/changelog.pyThu Sep 27 16:56:15 2018 +0200
+++ b/mercurial/changelog.pyThu Sep 27 16:55:44 2018 +0200
@@ -20,14 +20,18 @@
from . import (
encoding,
error,
+policy,
pycompat,
revlog,
+util,
)
from .utils import (
dateutil,
stringutil,
)
+parsers = policy.importmod(r'parsers')
+
_defaultextra = {'branch': 'default'}
def _string_escape(text):
@@ -343,6 +347,14 @@
if i not in self.filteredrevs:
yield i
+def ancestors(self, revs, *args, **kwargs):
+if util.safehasattr(parsers, 'rustlazyancestors') and
self.filteredrevs:
+missing = self.filteredrevs.difference(revs)
+if missing:
+# raise the lookup error
+self.rev(min(missing))
+return super(changelog, self).ancestors(revs, *args, **kwargs)
+
def reachableroots(self, minroot, heads, roots, includepath=False):
return self.index.reachableroots2(minroot, heads, roots, includepath)
diff -r d834b99b2d65 -r 0fcc7c5de05a mercurial/revlog.py
--- a/mercurial/revlog.py Thu Sep 27 16:56:15 2018 +0200
+++ b/mercurial/revlog.py Thu Sep 27 16:55:44 2018 +0200
@@ -802,6 +802,10 @@
See the documentation for ancestor.lazyancestors for more details."""
+if util.safehasattr(parsers, 'rustlazyancestors'):
+return ancestor.rustlazyancestors(
+self.index, revs,
+stoprev=stoprev, inclusive=inclusive)
return ancestor.lazyancestors(self.parentrevs, revs, stoprev=stoprev,
inclusive=inclusive)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D4710: lfs: add repository feature denoting the use of LFS
> +wrapfunction(localrepo, 'makefilestorage', > wrapper.localrepomakefilestorage) As I pointed out before, `makefilestorage` can't be wrapped in this way because it's captured as `REPO_INTERFACES[1][1]` earlier. The easiest workaround I can think of is to wrap the reference in `REPO_INTERFACES` with lambda, so the `makefilestorage` is looked up by name every time. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@39851: 2 new changesets
2 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/d89d5bc06eaa changeset: 39850:d89d5bc06eaa user:Gregory Szorc date:Wed Sep 19 14:36:57 2018 -0700 summary: localrepo: define "features" on repository instances (API) https://www.mercurial-scm.org/repo/hg/rev/1f7b3b980af8 changeset: 39851:1f7b3b980af8 bookmark:@ tag: tip user:Gregory Szorc date:Wed Sep 19 13:48:59 2018 -0700 summary: lfs: add repository feature denoting the use of LFS -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2 V3] py3: convert os.readlink() path to native strings on Windows
# HG changeset patch # User Matt Harbison # Date 1537924572 14400 # Tue Sep 25 21:16:12 2018 -0400 # Node ID 1b472f36e081a9a98e45f84a0cf34f5b6008dd47 # Parent 1834dabb38e08a082cde354e0f0e66310bc98cb3 py3: convert os.readlink() path to native strings on Windows Windows insisted that it needs to be str. I skipped the stuff in the posix module, and left `tests/f` and `run-tests.py` alone for now. diff --git a/hgext/convert/gnuarch.py b/hgext/convert/gnuarch.py --- a/hgext/convert/gnuarch.py +++ b/hgext/convert/gnuarch.py @@ -223,7 +223,7 @@ class gnuarch_source(common.converter_so def _getfile(self, name, rev): mode = os.lstat(os.path.join(self.tmppath, name)).st_mode if stat.S_ISLNK(mode): -data = os.readlink(os.path.join(self.tmppath, name)) +data = util.readlink(os.path.join(self.tmppath, name)) if mode: mode = 'l' else: diff --git a/mercurial/posix.py b/mercurial/posix.py --- a/mercurial/posix.py +++ b/mercurial/posix.py @@ -43,6 +43,7 @@ except AttributeError: def oslink(src, dst): raise OSError(errno.EINVAL, 'hardlinks not supported: %s to %s' % (src, dst)) +readlink = os.readlink unlink = os.unlink rename = os.rename removedirs = os.removedirs diff --git a/mercurial/util.py b/mercurial/util.py --- a/mercurial/util.py +++ b/mercurial/util.py @@ -112,6 +112,7 @@ parsepatchoutput = platform.parsepatchou pconvert = platform.pconvert poll = platform.poll posixfile = platform.posixfile +readlink = platform.readlink rename = platform.rename removedirs = platform.removedirs samedevice = platform.samedevice @@ -1841,7 +1842,7 @@ def makelock(info, pathname): def readlock(pathname): try: -return os.readlink(pathname) +return readlink(pathname) except OSError as why: if why.errno not in (errno.EINVAL, errno.ENOSYS): raise diff --git a/mercurial/vfs.py b/mercurial/vfs.py --- a/mercurial/vfs.py +++ b/mercurial/vfs.py @@ -206,7 +206,7 @@ class abstractvfs(object): return util.rename(srcpath, dstpath) def readlink(self, path): -return os.readlink(self.join(path)) +return util.readlink(self.join(path)) def removedirs(self, path=None): """Remove a leaf directory and all empty intermediate ones diff --git a/mercurial/windows.py b/mercurial/windows.py --- a/mercurial/windows.py +++ b/mercurial/windows.py @@ -519,6 +519,9 @@ def groupname(gid=None): If gid is None, return the name of the current group.""" return None +def readlink(pathname): +return pycompat.fsencode(os.readlink(pycompat.fsdecode(pathname))) + def removedirs(name): """special version of os.removedirs that does not remove symlinked directories or junction points if they actually contain files""" ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@39871: 11 new changesets
11 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/db5501d93bcf changeset: 39861:db5501d93bcf user:Gregory Szorc date:Mon Sep 24 09:41:42 2018 -0700 summary: changegroup: remove reordering control (BC) https://www.mercurial-scm.org/repo/hg/rev/5a9ab91e0a45 changeset: 39862:5a9ab91e0a45 user:Gregory Szorc date:Fri Sep 21 14:28:21 2018 -0700 summary: revlog: new API to emit revision data https://www.mercurial-scm.org/repo/hg/rev/c73f9f345ec0 changeset: 39863:c73f9f345ec0 user:Gregory Szorc date:Fri Sep 21 14:54:59 2018 -0700 summary: tests: use more complex file storage test https://www.mercurial-scm.org/repo/hg/rev/7b752bf08435 changeset: 39864:7b752bf08435 user:Gregory Szorc date:Mon Sep 24 09:48:02 2018 -0700 summary: wireprotov2server: port to emitrevisions() https://www.mercurial-scm.org/repo/hg/rev/31b7e8e7132e changeset: 39865:31b7e8e7132e user:Gregory Szorc date:Fri Sep 21 18:47:04 2018 -0700 summary: changegroup: port to emitrevisions() (issue5976) https://www.mercurial-scm.org/repo/hg/rev/e23c03dc5cf9 changeset: 39866:e23c03dc5cf9 user:Gregory Szorc date:Mon Sep 24 09:59:19 2018 -0700 summary: revlog: drop emitrevisiondeltas() and associated functionality (API) https://www.mercurial-scm.org/repo/hg/rev/4b816a83e17e changeset: 39867:4b816a83e17e user:Gregory Szorc date:Mon Sep 24 10:08:58 2018 -0700 summary: filelog: drop _generaldelta attribute (API) https://www.mercurial-scm.org/repo/hg/rev/b06303a208be changeset: 39868:b06303a208be user:Gregory Szorc date:Wed Sep 26 11:27:41 2018 -0700 summary: lfs: drop unused import https://www.mercurial-scm.org/repo/hg/rev/14e500b58263 changeset: 39869:14e500b58263 user:Gregory Szorc date:Mon Sep 24 11:56:48 2018 -0700 summary: revlog: add method for obtaining storage info (API) https://www.mercurial-scm.org/repo/hg/rev/b399ff55ee6d changeset: 39870:b399ff55ee6d user:Gregory Szorc date:Mon Sep 24 12:39:34 2018 -0700 summary: upgrade: use storageinfo() for obtaining storage metadata https://www.mercurial-scm.org/repo/hg/rev/01c0f01b562b changeset: 39871:01c0f01b562b bookmark:@ tag: tip user:Martin von Zweigbergk date:Wed Sep 26 12:06:44 2018 -0700 summary: tests: de-flake test-narrow-debugrebuilddirstate.t -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@39876: 5 new changesets
5 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/733db72f0f54 changeset: 39872:733db72f0f54 user:Gregory Szorc date:Mon Sep 24 11:27:47 2018 -0700 summary: revlog: move revision verification out of verify https://www.mercurial-scm.org/repo/hg/rev/2ac4f3e97813 changeset: 39873:2ac4f3e97813 user:Gregory Szorc date:Mon Sep 24 11:16:33 2018 -0700 summary: filelog: stop proxying flags() (API) https://www.mercurial-scm.org/repo/hg/rev/9596cf2a550d changeset: 39874:9596cf2a550d user:Gregory Szorc date:Mon Sep 24 12:42:03 2018 -0700 summary: filelog: stop proxying "opener" (API) https://www.mercurial-scm.org/repo/hg/rev/d909c44d29e1 changeset: 39875:d909c44d29e1 user:Gregory Szorc date:Mon Sep 24 12:49:17 2018 -0700 summary: filelog: stop proxying rawsize() (API) https://www.mercurial-scm.org/repo/hg/rev/a269fa55467e changeset: 39876:a269fa55467e bookmark:@ tag: tip user:Gregory Szorc date:Mon Sep 24 13:35:50 2018 -0700 summary: filelog: stop proxying deltaparent() (API) -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 5 V2] pullreport: add a test to show misreporting of visible changeset
On Fri, 28 Sep 2018 19:49:07 +0200, Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1538064373 -7200 > # Thu Sep 27 18:06:13 2018 +0200 > # Node ID 78ef4cda114aabd21835d7bc08b0a7c04040d80f > # Parent 4e3e9163c676af92f765683958dbdc68b9dc16bd > # EXP-Topic obsolete-duplicates > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > 78ef4cda114a > pullreport: add a test to show misreporting of visible changeset Queued these, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4710: lfs: add repository feature denoting the use of LFS
yuja added a comment. > +wrapfunction(localrepo, 'makefilestorage', wrapper.localrepomakefilestorage) As I pointed out before, `makefilestorage` can't be wrapped in this way because it's captured as `REPO_INTERFACES[1][1]` earlier. The easiest workaround I can think of is to wrap the reference in `REPO_INTERFACES` with lambda, so the `makefilestorage` is looked up by name every time. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4710 To: indygreg, #hg-reviewers, mharbison72 Cc: yuja, mharbison72, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4713: largefiles: automatically load largefiles extension when required (BC)
yuja added a comment.
> 1. Map of requirements to list of extensions to load automatically when
> 2. requirement is present. autoextensions = { +b'largefiles':
[b'largefiles'], b'lfs': [b'lfs'], }
Can we add some warnings here? The largefiles is IMHO one of the most buggy
extensions, and loading it has a side effect (e.g. fsmonitor is disabled.)
It shouldn't be silently loaded into the process.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4713
To: indygreg, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D4713: largefiles: automatically load largefiles extension when required (BC)
> # Map of requirements to list of extensions to load automatically when
> # requirement is present.
> autoextensions = {
> +b'largefiles': [b'largefiles'],
> b'lfs': [b'lfs'],
> }
Can we add some warnings here? The largefiles is IMHO one of the most buggy
extensions, and loading it has a side effect (e.g. fsmonitor is disabled.)
It shouldn't be silently loaded into the process.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 V3] util: use a context manager in readlock()
# HG changeset patch # User Matt Harbison # Date 1538187525 14400 # Fri Sep 28 22:18:45 2018 -0400 # Node ID cddac40b1a06c37cede9757914d930de92250612 # Parent 1b472f36e081a9a98e45f84a0cf34f5b6008dd47 util: use a context manager in readlock() diff --git a/mercurial/util.py b/mercurial/util.py --- a/mercurial/util.py +++ b/mercurial/util.py @@ -1848,10 +1848,8 @@ def readlock(pathname): raise except AttributeError: # no symlink in os pass -fp = posixfile(pathname, 'rb') -r = fp.read() -fp.close() -return r +with posixfile(pathname, 'rb') as fp: +return fp.read() def fstat(fp): '''stat file object that may not have fileno method.''' ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@39860: 9 new changesets
9 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/2c2fadbc9851 changeset: 39852:2c2fadbc9851 user:Gregory Szorc date:Thu Sep 20 15:06:43 2018 -0700 summary: localrepo: automatically load lfs extension when required (BC) https://www.mercurial-scm.org/repo/hg/rev/bcf72d7b1524 changeset: 39853:bcf72d7b1524 user:Gregory Szorc date:Thu Sep 20 15:18:13 2018 -0700 summary: lfs: don't add extension to hgrc after clone or share (BC) https://www.mercurial-scm.org/repo/hg/rev/823a580448d7 changeset: 39854:823a580448d7 user:Gregory Szorc date:Thu Sep 20 15:30:00 2018 -0700 summary: largefiles: automatically load largefiles extension when required (BC) https://www.mercurial-scm.org/repo/hg/rev/62a532045e71 changeset: 39855:62a532045e71 user:Gregory Szorc date:Thu Sep 20 17:47:34 2018 -0700 summary: lfs: access revlog directly https://www.mercurial-scm.org/repo/hg/rev/96838b620b9c changeset: 39856:96838b620b9c user:Gregory Szorc date:Thu Sep 20 18:07:42 2018 -0700 summary: filelog: store filename directly on revlog instance https://www.mercurial-scm.org/repo/hg/rev/8dab7c8a93eb changeset: 39857:8dab7c8a93eb user:Gregory Szorc date:Mon Sep 24 09:37:19 2018 -0700 summary: upgrade: report size of backing files, not internal storage size https://www.mercurial-scm.org/repo/hg/rev/9534fe1e5d28 changeset: 39858:9534fe1e5d28 user:Gregory Szorc date:Thu Sep 20 19:20:01 2018 -0700 summary: manifest: add rawsize() proxy (API) https://www.mercurial-scm.org/repo/hg/rev/32d3ed3023bb changeset: 39859:32d3ed3023bb user:Gregory Szorc date:Mon Sep 24 09:38:27 2018 -0700 summary: upgrade: use rawsize() instead of revlog index https://www.mercurial-scm.org/repo/hg/rev/d9b3cc3d5d07 changeset: 39860:d9b3cc3d5d07 bookmark:@ tag: tip user:Gregory Szorc date:Thu Sep 20 19:31:07 2018 -0700 summary: filelog: drop index attribute (API) -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4752: filelog: return correct size when content begins with metadata prefix
yuja added a comment. > The new call to read() should be fast because the revision > fulltext should be cached as part of calling renamed(). So the > overhead here should be minimal. That isn't true unfortunately. `renamed()` reads the text only if p1 is null, so `size()` was cheap in most cases. This patch makes `size()` as slow as `len(read())`. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4752 To: indygreg, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D4752: filelog: return correct size when content begins with metadata prefix
> The new call to read() should be fast because the revision > fulltext should be cached as part of calling renamed(). So the > overhead here should be minimal. That isn't true unfortunately. `renamed()` reads the text only if p1 is null, so `size()` was cheap in most cases. This patch makes `size()` as slow as `len(read())`. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2679: [PoC] obsolete: config option to enable local only obsolescence mode
lothiraldan added a comment.
In https://phab.mercurial-scm.org/D2679#70935, @durin42 wrote:
> In https://phab.mercurial-scm.org/D2679#70934, @lothiraldan wrote:
>
> > In https://phab.mercurial-scm.org/D2679#69388, @durin42 wrote:
> >
> > > I'm slowly becoming convinced that the long-unquestioned axiom that
"all markers are distributed globally" isn't correct, and this is part of why:
it's potentially of great value to be able to restore a change by re-pulling
it, even though the obsmarkers would normally cause it to be deleted.
> >
> >
> > Avoiding silent reappearance of the locally obsolete changeset that we
see on a remote repository is a core feature of obsolescence. Actually, it is
the one issue that prompted the creation of changeset evolution in the first
place.
>
>
> There's a difference (in my view) between something accidentally coming
back and intentionally coming back. The current state of affairs prevents both.
I feel like we can do better.
I feel like detecting if an obsolete changeset is re-added accidentally or
intentionally is a near impossible problem. We have submitted code recently to
detect when an obsolete changeset is re-added, but again we don't know if it is
accidentally or intentionally. The safe path here is to inform the user about
suspicious situations and give them the tool to explicitly restore changesets
they want to be restored. That why we have been working on an explicit `hg
rewind` command in evolve.
>
>
>> We and the other people using distributed evolution rely on this behavior
on a daily basis. The includes people who picked up the evolve extension on
their own and came up with their own workflow without ever speaking to us. We
can see some user interfaces adjustment in the future, but the set of
synchronized markers and the associated behavior is something we are happy
about. It has been well tested in diverse teams and situation for multiple
years now.
>
> I've brought this up repeatedly and been dismissed with this exact argument
every time. I weary of it, because it's not a new argument, it's just blind
devotion to the initial design without any attempt at reflection if we can do
better.
Since I joined Octobus, I've witnessed the exploration of all the remaining
uncharted territories and solving of most of the resulting associated
challenges. In particular, some important aspects of distributed evolution:
instabilities resolutions and efficient obsmarkers exchanges have been solved
and have working implementations used on a daily basis. We did alter the
initial design when deemed so by new information from exploration or feedback
of evolve users.
During these developments, we did not find reasons to challenge the
obsolescence core concepts. They, in fact, proved a useful help to build
solutions to the distributed evolution challenges. To name a few of these
progress and design changes: vocabulary renaming, the missing commands for
history exploration and reviving, the new discovery protocol and more recently
how to track fold in obsmarkers.
In a similar way, the numbers and types of distributed evolution users have
grown and we had more and more opportunity to exchange with them. We base our
plans on more than "some people using it without complains". They are a diverse
and large corpus of people we have been talking to and studied their workflows.
Here again, core concepts, in particular, the global state and instabilities
are things that, practically helps teams to use distributed evolution workflows
and newcomers to get a good grasp of it. Important behavior designed and tested
in the past 5 years, like the exchange behavior, have been battle tested and
real people rely on them in their day to day workflow.
So, it might seem like the same argument, but it is actually a stronger one.
The data to back it kept growing in the past couple of years.
If you need more details, check our recent blog post about evolution:
https://octobus.net/blog/2018-09-03-evolve-status-2018-08.html
> I don't dispute that evolution as currently implemented is a good thing,
but I feel like there's room for significant improvement while simultaneously
simplifying the exchange algorithms. Care to try and convince me why I'm wrong,
without anecdotal "many people use this and haven't complained" arguments?
Sure, the proposal in this PoC ("unobsolete" all re-added changesets) break
basic distributed evolution workflow very quickly. Here are small examples
among many. They display simple draft changesets roundtrips:
- Example A:
- I am working on a repository with a changeset A that is present also on
my default server
- I amend A into A'
- I pull from my default server, the server doesn't know yet that A has
been rewritten into A'. A is sent by the server.
- A is hidden before the pull. Getting A to be visible again would be
confusing
D4782: remotefilelog: import pruned-down remotefilelog extension from hg-experimental
durin42 updated this revision to Diff 11468. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4782?vs=11467=11468 REVISION DETAIL https://phab.mercurial-scm.org/D4782 AFFECTED FILES hgext/remotefilelog/README.md hgext/remotefilelog/__init__.py hgext/remotefilelog/basepack.py hgext/remotefilelog/basestore.py hgext/remotefilelog/cacheclient.py hgext/remotefilelog/connectionpool.py hgext/remotefilelog/constants.py hgext/remotefilelog/contentstore.py hgext/remotefilelog/datapack.py hgext/remotefilelog/debugcommands.py hgext/remotefilelog/extutil.py hgext/remotefilelog/fileserverclient.py hgext/remotefilelog/historypack.py hgext/remotefilelog/lz4wrapper.py hgext/remotefilelog/metadatastore.py hgext/remotefilelog/remotefilectx.py hgext/remotefilelog/remotefilelog.py hgext/remotefilelog/remotefilelogserver.py hgext/remotefilelog/repack.py hgext/remotefilelog/shallowbundle.py hgext/remotefilelog/shallowrepo.py hgext/remotefilelog/shallowstore.py hgext/remotefilelog/shallowutil.py hgext/remotefilelog/shallowverifier.py hgext/remotefilelog/wirepack.py setup.py tests/ls-l.py tests/remotefilelog-getflogheads.py tests/remotefilelog-library.sh tests/test-remotefilelog-bad-configs.t tests/test-remotefilelog-bgprefetch.t tests/test-remotefilelog-blame.t tests/test-remotefilelog-bundle2-legacy.t tests/test-remotefilelog-bundle2.t tests/test-remotefilelog-bundles.t tests/test-remotefilelog-cacheprocess.t tests/test-remotefilelog-clone-tree.t tests/test-remotefilelog-clone.t tests/test-remotefilelog-corrupt-cache.t tests/test-remotefilelog-datapack.py tests/test-remotefilelog-gc.t tests/test-remotefilelog-gcrepack.t tests/test-remotefilelog-histpack.py tests/test-remotefilelog-http.t tests/test-remotefilelog-keepset.t tests/test-remotefilelog-linknodes.t tests/test-remotefilelog-local.t tests/test-remotefilelog-log.t tests/test-remotefilelog-partial-shallow.t tests/test-remotefilelog-permissions.t tests/test-remotefilelog-prefetch.t tests/test-remotefilelog-pull-noshallow.t tests/test-remotefilelog-push-pull.t tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t tests/test-remotefilelog-sparse.t tests/test-remotefilelog-tags.t tests/test-remotefilelog-wireproto.t To: durin42, #hg-reviewers Cc: spectral, mjpieters, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4782: remotefilelog: import pruned-down remotefilelog extension from hg-experimental
durin42 updated this revision to Diff 11467. durin42 edited the summary of this revision. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D4782?vs=11454=11467 REVISION DETAIL https://phab.mercurial-scm.org/D4782 AFFECTED FILES hgext/remotefilelog/README.md hgext/remotefilelog/__init__.py hgext/remotefilelog/basepack.py hgext/remotefilelog/basestore.py hgext/remotefilelog/cacheclient.py hgext/remotefilelog/connectionpool.py hgext/remotefilelog/constants.py hgext/remotefilelog/contentstore.py hgext/remotefilelog/datapack.py hgext/remotefilelog/debugcommands.py hgext/remotefilelog/extutil.py hgext/remotefilelog/fileserverclient.py hgext/remotefilelog/historypack.py hgext/remotefilelog/lz4wrapper.py hgext/remotefilelog/metadatastore.py hgext/remotefilelog/remotefilectx.py hgext/remotefilelog/remotefilelog.py hgext/remotefilelog/remotefilelogserver.py hgext/remotefilelog/repack.py hgext/remotefilelog/shallowbundle.py hgext/remotefilelog/shallowrepo.py hgext/remotefilelog/shallowstore.py hgext/remotefilelog/shallowutil.py hgext/remotefilelog/shallowverifier.py hgext/remotefilelog/wirepack.py setup.py tests/ls-l.py tests/remotefilelog-getflogheads.py tests/remotefilelog-library.sh tests/test-remotefilelog-bad-configs.t tests/test-remotefilelog-bgprefetch.t tests/test-remotefilelog-blame.t tests/test-remotefilelog-bundle2-legacy.t tests/test-remotefilelog-bundle2.t tests/test-remotefilelog-bundles.t tests/test-remotefilelog-cacheprocess.t tests/test-remotefilelog-clone-tree.t tests/test-remotefilelog-clone.t tests/test-remotefilelog-corrupt-cache.t tests/test-remotefilelog-datapack.py tests/test-remotefilelog-gc.t tests/test-remotefilelog-gcrepack.t tests/test-remotefilelog-histpack.py tests/test-remotefilelog-http.t tests/test-remotefilelog-keepset.t tests/test-remotefilelog-linknodes.t tests/test-remotefilelog-local.t tests/test-remotefilelog-log.t tests/test-remotefilelog-partial-shallow.t tests/test-remotefilelog-permissions.t tests/test-remotefilelog-prefetch.t tests/test-remotefilelog-pull-noshallow.t tests/test-remotefilelog-push-pull.t tests/test-remotefilelog-repack-fast.t tests/test-remotefilelog-repack.t tests/test-remotefilelog-sparse.t tests/test-remotefilelog-tags.t tests/test-remotefilelog-wireproto.t To: durin42, #hg-reviewers Cc: spectral, mjpieters, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[Bug 5994] New: hg metaedit changes timestamp by default
https://bz.mercurial-scm.org/show_bug.cgi?id=5994 Bug ID: 5994 Summary: hg metaedit changes timestamp by default Product: Mercurial Version: default branch Hardware: PC OS: Linux Status: UNCONFIRMED Severity: feature Priority: wish Component: evolution Assignee: bugzi...@mercurial-scm.org Reporter: z...@zash.se CC: mercurial-devel@mercurial-scm.org, pierre-yves.da...@ens-lyon.org `hg metaedit` appears to change the timestamp by default, something one would not expect since there exists a flag (-D / --current-date) for doing this. This was done in a relatively clean environment: ~$ echo hello > hello ~$ hg --config extensions.evolve= add hello ~$ hg --config extensions.evolve= ci -m 'hello' -u test ~$ hg --config extensions.evolve= metaed -m 'hello' --config ui.username=me 0 files updated, 0 files merged, 0 files removed, 0 files unresolved ~$ hg exp obsolete feature not enabled but 1 markers found! # HG changeset patch # User test # Date 1538144129 -7200 # Fri Sep 28 16:15:29 2018 +0200 # Node ID 2afbcecc3e47f2dae2c3f4e1a3627d27ee677f7f # Parent hello diff -r -r 2afbcecc3e47 hello --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/hello Fri Sep 28 16:15:29 2018 +0200 @@ -0,0 +1,1 @@ +hello ~$ hg obslog hg: unknown command 'obslog' (did you mean log?) ~$ hg --config extensions.evolve= obslog @ 2afbcecc3e47 (1) hello | x e223779fd241 (0) hello rewritten(date) as 2afbcecc3e47 using metaedit by me (Fri Sep 28 16:15:29 2018 +0200) ~$ hg exp --hidden e223779fd241 obsolete feature not enabled but 1 markers found! # HG changeset patch # User test # Date 1538144097 -7200 # Fri Sep 28 16:14:57 2018 +0200 # Node ID e223779fd24168832fa6d81afbb85c19db74ca4f # Parent hello diff -r -r e223779fd241 hello --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/hello Fri Sep 28 16:14:57 2018 +0200 @@ -0,0 +1,1 @@ +hello Observe change in timestamp. -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4780: repo: look up nullrev context by revnum, not symbolic name
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1737ab4490e7: repo: look up nullrev context by revnum, not
symbolic name (authored by martinvonz, committed by ).
CHANGED PRIOR TO COMMIT
https://phab.mercurial-scm.org/D4780?vs=11452=11465#toc
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4780?vs=11452=11465
REVISION DETAIL
https://phab.mercurial-scm.org/D4780
AFFECTED FILES
mercurial/archival.py
mercurial/scmutil.py
CHANGE DETAILS
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -21,6 +21,7 @@
bin,
hex,
nullid,
+nullrev,
short,
wdirid,
wdirrev,
@@ -730,7 +731,7 @@
if len(parents) > 1:
return parents
if repo.ui.debugflag:
-return [parents[0], repo['null']]
+return [parents[0], repo[nullrev]]
if parents[0].rev() >= intrev(ctx) - 1:
return []
return parents
diff --git a/mercurial/archival.py b/mercurial/archival.py
--- a/mercurial/archival.py
+++ b/mercurial/archival.py
@@ -16,6 +16,9 @@
import zlib
from .i18n import _
+from .node import (
+nullrev,
+)
from . import (
error,
@@ -76,7 +79,7 @@
# repo[0] may be hidden
for rev in repo:
return repo[rev]
-return repo['null']
+return repo[nullrev]
# {tags} on ctx includes local tags and 'tip', with no current way to limit
# that to global tags. Therefore, use {latesttag} as a substitute when
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH V2] py3: convert os.readlink() path to native strings
On Thu, 27 Sep 2018 22:51:27 -0400, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison > # Date 1537924572 14400 > # Tue Sep 25 21:16:12 2018 -0400 > # Node ID 216114ff8d2bc57d9aa8913cf75f14267a8332f6 > # Parent df02cb5b9b3496aa95cbe754a92d714f4c68262b > py3: convert os.readlink() path to native strings > > Windows insisted that it needs to be str. I skipped the stuff that's > obviously > posix only, and left `tests/f` and `run-tests.py` alone for now. > > diff --git a/mercurial/util.py b/mercurial/util.py > --- a/mercurial/util.py > +++ b/mercurial/util.py > @@ -1841,7 +1841,7 @@ def makelock(info, pathname): > > def readlock(pathname): > try: > -return os.readlink(pathname) > +return pycompat.fsencode(os.readlink(pycompat.fsdecode(pathname))) Well, this is still bad as it goes non-native route on Unix. If we really need to support symlinks on Windows, we'll have to move it to platform module. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 5] pullreport: skip filtered revs instead of obsolete ones
On Fri, 28 Sep 2018 14:36:50 +0200, Boris FELD wrote:
> On 28/09/2018 13:56, Yuya Nishihara wrote:
> > On Fri, 28 Sep 2018 12:22:19 +0200, Boris Feld wrote:
> >> # HG changeset patch
> >> # User Boris Feld
> >> # Date 1538058910 -7200
> >> # Thu Sep 27 16:35:10 2018 +0200
> >> # Node ID 4bd42e72e7ba8c0ee9dc4e153127882e6961602a
> >> # Parent b36914d9928effac212d851c9617de93d6260746
> >> # EXP-Topic obsolete-duplicates
> >> # Available At https://bitbucket.org/octobus/mercurial-devel/
> >> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> >> 4bd42e72e7ba
> >> pullreport: skip filtered revs instead of obsolete ones
> >>
> >> Obsolescence is closely related to visibility but still a distinct
> >> concept. We
> >> can receive changesets that are obsolete but visible (eg: when pulling
> >> orphans). Such changeset should be reported too. In addition, the filtering
> >> level can be anything, we should respect it.
> >>
> >> diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
> >> --- a/mercurial/scmutil.py
> >> +++ b/mercurial/scmutil.py
> >> @@ -1603,13 +1603,11 @@ def registersummarycallback(repo, otr, t
> >> if origrepolen >= len(repo):
> >> return
> >>
> >> -# Compute the bounds of new revisions' range, excluding
> >> obsoletes.
> >> -unfi = repo.unfiltered()
> >> -revs = unfi.revs('%d: and not obsolete()', origrepolen)
> >> +# Compute the bounds of new visible revisions' range.
> >> +revs = list(repo.changelog.revs(start=origrepolen))
> > Use revset? It's probably better to not construct a list of tens of
> > thousands
> > of integers.
>
> The issue here is that we don't have a way to express what we want as a
> revset (to our knowledge). The revnum `origrepolen` could be filtered so
> it cannot be used explicitly in "%d"
>
> We either need:
>
> unfi.revs("%d - _filteredrevs(%s)", origrepolen, repo.filtername)
>
> or
>
> repo.revs("firstrevgreaterthan(%s):", origrepolen)
>
> Should we implement either one of the possibility above or do you have
> another idea?
Good point. Maybe we can instantiate smartset.spanset() directly then?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH V2] py3: convert os.readlink() path to native strings
> On Sep 28, 2018, at 7:32 AM, Yuya Nishihara wrote: > >> On Thu, 27 Sep 2018 22:51:27 -0400, Matt Harbison wrote: >> # HG changeset patch >> # User Matt Harbison >> # Date 1537924572 14400 >> # Tue Sep 25 21:16:12 2018 -0400 >> # Node ID 216114ff8d2bc57d9aa8913cf75f14267a8332f6 >> # Parent df02cb5b9b3496aa95cbe754a92d714f4c68262b >> py3: convert os.readlink() path to native strings >> >> Windows insisted that it needs to be str. I skipped the stuff that's >> obviously >> posix only, and left `tests/f` and `run-tests.py` alone for now. >> >> diff --git a/mercurial/util.py b/mercurial/util.py >> --- a/mercurial/util.py >> +++ b/mercurial/util.py >> @@ -1841,7 +1841,7 @@ def makelock(info, pathname): >> >> def readlock(pathname): >> try: >> -return os.readlink(pathname) >> +return pycompat.fsencode(os.readlink(pycompat.fsdecode(pathname))) > > Well, this is still bad as it goes non-native route on Unix. If we really > need to support symlinks on Windows, we'll have to move it to platform module. Even though I turned off symlinks in tests for now because you need admin rights to create them, I read recently that some new-ish builds of Windows 10 allow them to be created by a normal user with developer mode enabled. So we probably shouldn’t cut too many corners here. I originally coded it using procutil.tonativestr(), but figured an os specific module with a filesystem function would be the right thing. The module seemed out of place for this usage too. I didn’t look to see how much would need to be moved to move that to encoding. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 5] pullreport: skip filtered revs instead of obsolete ones
On Fri, 28 Sep 2018 12:22:19 +0200, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1538058910 -7200
> # Thu Sep 27 16:35:10 2018 +0200
> # Node ID 4bd42e72e7ba8c0ee9dc4e153127882e6961602a
> # Parent b36914d9928effac212d851c9617de93d6260746
> # EXP-Topic obsolete-duplicates
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 4bd42e72e7ba
> pullreport: skip filtered revs instead of obsolete ones
>
> Obsolescence is closely related to visibility but still a distinct concept. We
> can receive changesets that are obsolete but visible (eg: when pulling
> orphans). Such changeset should be reported too. In addition, the filtering
> level can be anything, we should respect it.
>
> diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
> --- a/mercurial/scmutil.py
> +++ b/mercurial/scmutil.py
> @@ -1603,13 +1603,11 @@ def registersummarycallback(repo, otr, t
> if origrepolen >= len(repo):
> return
>
> -# Compute the bounds of new revisions' range, excluding
> obsoletes.
> -unfi = repo.unfiltered()
> -revs = unfi.revs('%d: and not obsolete()', origrepolen)
> +# Compute the bounds of new visible revisions' range.
> +revs = list(repo.changelog.revs(start=origrepolen))
Use revset? It's probably better to not construct a list of tens of thousands
of integers.
> if not revs:
> -# Got only obsoletes.
> return
> -minrev, maxrev = repo[revs.min()], repo[revs.max()]
> +minrev, maxrev = repo[revs[0]], repo[revs[-1]]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH V2] py3: convert os.readlink() path to native strings
On Fri, 28 Sep 2018 07:59:17 -0400, Matt Harbison wrote: > > > On Sep 28, 2018, at 7:32 AM, Yuya Nishihara wrote: > > > >> On Thu, 27 Sep 2018 22:51:27 -0400, Matt Harbison wrote: > >> # HG changeset patch > >> # User Matt Harbison > >> # Date 1537924572 14400 > >> # Tue Sep 25 21:16:12 2018 -0400 > >> # Node ID 216114ff8d2bc57d9aa8913cf75f14267a8332f6 > >> # Parent df02cb5b9b3496aa95cbe754a92d714f4c68262b > >> py3: convert os.readlink() path to native strings > >> > >> Windows insisted that it needs to be str. I skipped the stuff that's > >> obviously > >> posix only, and left `tests/f` and `run-tests.py` alone for now. > >> > >> diff --git a/mercurial/util.py b/mercurial/util.py > >> --- a/mercurial/util.py > >> +++ b/mercurial/util.py > >> @@ -1841,7 +1841,7 @@ def makelock(info, pathname): > >> > >> def readlock(pathname): > >> try: > >> -return os.readlink(pathname) > >> +return pycompat.fsencode(os.readlink(pycompat.fsdecode(pathname))) > > > > Well, this is still bad as it goes non-native route on Unix. If we really > > need to support symlinks on Windows, we'll have to move it to platform > > module. > > Even though I turned off symlinks in tests for now because you need admin > rights to create them, I read recently that some new-ish builds of Windows 10 > allow them to be created by a normal user with developer mode enabled. So we > probably shouldn’t cut too many corners here. > > I originally coded it using procutil.tonativestr(), but figured an os > specific module with a filesystem function would be the right thing. Perhaps. I think that's what the vfs object is meant to be. > The module seemed out of place for this usage too. I didn’t look to see how > much would need to be moved to move that to encoding. I think this can be just posix/windows.readlink() for now. At some point, we might want to make windows vfs directly call windows.readlink"u"() to support unicode filenames. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 5 V2] pullreport: skip filtered revs instead of obsolete ones
# HG changeset patch
# User Boris Feld
# Date 1538058910 -7200
# Thu Sep 27 16:35:10 2018 +0200
# Node ID c2ac56afe1e26620a0c38a0f006902102dae9f19
# Parent 78ef4cda114aabd21835d7bc08b0a7c04040d80f
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
c2ac56afe1e2
pullreport: skip filtered revs instead of obsolete ones
Obsolescence is closely related to visibility but still a distinct concept. We
can receive changesets that are obsolete but visible (eg: when pulling
orphans). Such changeset should be reported too. In addition, the filtering
level can be anything, we should respect it.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -38,6 +38,7 @@ from . import (
pycompat,
revsetlang,
similar,
+smartset,
url,
util,
vfs,
@@ -1605,11 +1606,9 @@ def registersummarycallback(repo, otr, t
if origrepolen >= len(repo):
return
-# Compute the bounds of new revisions' range, excluding obsoletes.
-unfi = repo.unfiltered()
-revs = unfi.revs('%d: and not obsolete()', origrepolen)
+# Compute the bounds of new visible revisions' range.
+revs = smartset.spanset(repo, start=origrepolen)
if not revs:
-# Got only obsoletes.
return
minrev, maxrev = repo[revs.min()], repo[revs.max()]
diff --git a/tests/test-obsolete-distributed.t
b/tests/test-obsolete-distributed.t
--- a/tests/test-obsolete-distributed.t
+++ b/tests/test-obsolete-distributed.t
@@ -537,5 +537,5 @@ Actual test
added 2 changesets with 0 changes to 0 files
1 new obsolescence markers
1 new orphan changesets
- new changesets 95d586532b49 (1 drafts)
+ new changesets 66f7d451a68b:95d586532b49 (2 drafts)
(run 'hg update' to get a working copy)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 5 V2] pullreport: add a test to show misreporting of visible changeset
# HG changeset patch
# User Boris Feld
# Date 1538064373 -7200
# Thu Sep 27 18:06:13 2018 +0200
# Node ID 78ef4cda114aabd21835d7bc08b0a7c04040d80f
# Parent 4e3e9163c676af92f765683958dbdc68b9dc16bd
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
78ef4cda114a
pullreport: add a test to show misreporting of visible changeset
The current code ignores all obsolete changesets including the visible one. We
add a test showing this behavior before fixing the behavior.
diff --git a/tests/test-obsolete-distributed.t
b/tests/test-obsolete-distributed.t
--- a/tests/test-obsolete-distributed.t
+++ b/tests/test-obsolete-distributed.t
@@ -487,3 +487,55 @@ decision is made in that case, so receiv
ef908e42ce65ef57f970d799acaddde26f58a4cc
5ffb9e311b35f6ab6f76f667ca5d6e595645481b 0 (Thu Jan 01 00:00:00 1970 +)
{'ef1': '4', 'operation': 'rebase', 'user': 'bob'}
$ cd ..
+
+Test pull report consistency
+
+
+obsolete but visible should be reported
+---
+
+Setup
+
+ $ hg init repo-a
+ $ cat << EOF >> repo-a/.hg/hgrc
+ > [ui]
+ > username=test
+ > EOF
+ $ cd repo-a
+ $ hg debugbuilddag ..
+ $ hg debugobsolete `getid tip`
+ obsoleted 1 changesets
+ $ cd ../
+ $ hg clone --pull repo-a repo-b
+ requesting all changes
+ adding changesets
+ adding manifests
+ adding file changes
+ added 1 changesets with 0 changes to 0 files
+ new changesets 1ea73414a91b (1 drafts)
+ updating to branch default
+ 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ $ hg -R repo-a up tip --hidden
+ 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ updated to hidden changeset 66f7d451a68b
+ (hidden revision '66f7d451a68b' is pruned)
+ $ hg -R repo-a branch foo
+ marked working directory as branch foo
+ (branches are permanent and global, did you want a bookmark?)
+ $ hg -R repo-a commit -m foo
+ 1 new orphan changesets
+
+Actual test
+(BROKEN)
+
+ $ hg -R repo-b pull
+ pulling from
$TESTTMP/distributed-chain-building/distributed-chain-building/repo-a
+ searching for changes
+ adding changesets
+ adding manifests
+ adding file changes
+ added 2 changesets with 0 changes to 0 files
+ 1 new obsolescence markers
+ 1 new orphan changesets
+ new changesets 95d586532b49 (1 drafts)
+ (run 'hg update' to get a working copy)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 5 V2] pullreport: skip or rework some early return
# HG changeset patch
# User Boris Feld
# Date 1538060400 -7200
# Thu Sep 27 17:00:00 2018 +0200
# Node ID ee865e674f55e4434a0796810872235999c8d438
# Parent c2ac56afe1e26620a0c38a0f006902102dae9f19
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
ee865e674f55
pullreport: skip or rework some early return
We are about to add more logic in this report. Before that, we need it to not
quit so early.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1603,35 +1603,36 @@ def registersummarycallback(repo, otr, t
def reportnewcs(repo, tr):
"""Report the range of new revisions pulled/unbundled."""
origrepolen = tr.changes.get('origrepolen', len(repo))
-if origrepolen >= len(repo):
+unfi = repo.unfiltered()
+if origrepolen >= len(unfi):
return
# Compute the bounds of new visible revisions' range.
revs = smartset.spanset(repo, start=origrepolen)
-if not revs:
-return
-minrev, maxrev = repo[revs.min()], repo[revs.max()]
+if revs:
+minrev, maxrev = repo[revs.min()], repo[revs.max()]
-if minrev == maxrev:
-revrange = minrev
-else:
-revrange = '%s:%s' % (minrev, maxrev)
-draft = len(repo.revs('%ld and draft()', revs))
-secret = len(repo.revs('%ld and secret()', revs))
-if not (draft or secret):
-msg = _('new changesets %s\n') % revrange
-elif draft and secret:
-msg = _('new changesets %s (%d drafts, %d secrets)\n')
-msg %= (revrange, draft, secret)
-elif draft:
-msg = _('new changesets %s (%d drafts)\n')
-msg %= (revrange, draft)
-elif secret:
-msg = _('new changesets %s (%d secrets)\n')
-msg %= (revrange, secret)
-else:
-raise error.ProgrammingError('entered unreachable condition')
-repo.ui.status(msg)
+if minrev == maxrev:
+revrange = minrev
+else:
+revrange = '%s:%s' % (minrev, maxrev)
+draft = len(repo.revs('%ld and draft()', revs))
+secret = len(repo.revs('%ld and secret()', revs))
+if not (draft or secret):
+msg = _('new changesets %s\n') % revrange
+elif draft and secret:
+msg = _('new changesets %s (%d drafts, %d secrets)\n')
+msg %= (revrange, draft, secret)
+elif draft:
+msg = _('new changesets %s (%d drafts)\n')
+msg %= (revrange, draft)
+elif secret:
+msg = _('new changesets %s (%d secrets)\n')
+msg %= (revrange, secret)
+else:
+errormsg = 'entered unreachable condition'
+raise error.ProgrammingError(errormsg)
+repo.ui.status(msg)
@reportsummary
def reportphasechanges(repo, tr):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 5 V2] pullreport: issue a message about "extinct" pulled changesets
# HG changeset patch
# User Boris Feld
# Date 1538059945 -7200
# Thu Sep 27 16:52:25 2018 +0200
# Node ID 4693e5593cc54c6bacc481950e561dcdcaf80a55
# Parent ee865e674f55e4434a0796810872235999c8d438
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
4693e5593cc5
pullreport: issue a message about "extinct" pulled changesets
Changeset pulled from a remote repository while already obsolete locally can
end up hidden after the pull. Hiding obsolete changesets is a good behavior
but silently "skipping" some of the pulled content can get confusing.
We now detect this situation and emit a message about it. The message is
simple and the wording could be improved, however, we focus on the detection
here. Evolution is still an experimental feature, so the output is open to
changes.
In particular, we could point out at the latest successors of the obsolete
changesets, however, it can get tricky is there are many of them. So we delay
these improvements to another adventure.
Another easy improvement would be to merge this message with the previous line
about the new nodes and their phases.
This is a good example of cases where we can only transmit a limited amount of
data to users by default. We need some sort of "transaction journal" we could
point the user to.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1634,6 +1634,17 @@ def registersummarycallback(repo, otr, t
raise error.ProgrammingError(errormsg)
repo.ui.status(msg)
+# search new changesets directly pulled as obsolete
+obsadded = unfi.revs('%d: and obsolete()', origrepolen)
+cl = repo.changelog
+extinctadded = [r for r in obsadded if r not in cl]
+if extinctadded:
+# They are not just obsolete, but obsolete and invisible
+# we call them "extinct" internally but the terms have not been
+# exposed to users.
+msg = '(%d other changesets obsolete on arrival)\n'
+repo.ui.status(msg % len(extinctadded))
+
@reportsummary
def reportphasechanges(repo, tr):
"""Report statistics of phase changes for changesets pre-existing
diff --git a/tests/test-obsolete-bundle-strip.t
b/tests/test-obsolete-bundle-strip.t
--- a/tests/test-obsolete-bundle-strip.t
+++ b/tests/test-obsolete-bundle-strip.t
@@ -170,6 +170,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-A1")'
@@ -248,6 +249,7 @@ Actual testing
# unbundling: added 2 changesets with 2 changes to 2 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with prune children
@@ -339,6 +341,7 @@ problematic)
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files
# unbundling: 1 new obsolescence markers
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg update' to get a working copy)
$ testrevs 'desc("C-A1")'
@@ -437,6 +440,7 @@ bundling multiple revisions
# unbundling: added 3 changesets with 3 changes to 3 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (2 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with precursors also pruned
@@ -503,6 +507,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-A1")'
@@ -578,6 +583,7 @@ Actual testing
# unbundling: added 2 changesets with 2 changes to 2 files (+1 heads)
# unbundling: 3 new obsolescence markers
# unbundling: new changesets cf2c22470d67 (1 drafts)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
chain with missing prune
@@ -836,6 +842,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
# unbundling: added 1 changesets with 1 changes to 1 files (+1 heads)
+ # unbundling: (1 other changesets obsolete on arrival)
# unbundling: (run 'hg heads' to see heads)
$ testrevs 'desc("C-B")'
@@ -864,6 +871,7 @@ Actual testing
# unbundling: adding manifests
# unbundling: adding file changes
#
[PATCH 5 of 5 V2] pullreport: rev duplicated and extinct into account
# HG changeset patch
# User Boris Feld
# Date 1538060106 -7200
# Thu Sep 27 16:55:06 2018 +0200
# Node ID 36064527c3d1f617fe7ec86cdd406aa37222a10c
# Parent 4693e5593cc54c6bacc481950e561dcdcaf80a55
# EXP-Topic obsolete-duplicates
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
36064527c3d1
pullreport: rev duplicated and extinct into account
If we already have some obsolete and hidden nodes locally and the server send
them again to you, it seems useful to point it out instead of being silent about
it.
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1635,7 +1635,9 @@ def registersummarycallback(repo, otr, t
repo.ui.status(msg)
# search new changesets directly pulled as obsolete
-obsadded = unfi.revs('%d: and obsolete()', origrepolen)
+duplicates = tr.changes.get('revduplicates', ())
+obsadded = unfi.revs('(%d: + %ld) and obsolete()',
+ origrepolen, duplicates)
cl = repo.changelog
extinctadded = [r for r in obsadded if r not in cl]
if extinctadded:
diff --git a/tests/test-obsolete.t b/tests/test-obsolete.t
--- a/tests/test-obsolete.t
+++ b/tests/test-obsolete.t
@@ -806,6 +806,7 @@ check hgweb does not explode
adding file changes
added 62 changesets with 63 changes to 9 files (+60 heads)
new changesets 50c51b361e60:c15e9edfca13 (62 drafts)
+ (2 other changesets obsolete on arrival)
(run 'hg heads .' to see heads, 'hg merge' to merge)
$ for node in `hg log -r 'desc(babar_)' --template '{node}\n'`;
> do
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
