D4267: zsh_completion: update all options
av6 added inline comments. INLINE COMMENTS > spectral wrote in zsh_completion:644 > I assume the first section here makes continue and abort mutually exclusive? > I believe that revision arguments are also forbidden if either --continue or > --abort are specified; I don't know if there's a way of representing that. Just having `--rev -r` inside the parentheses should be enough, but I'm not sure if implementing every piece of exclusivity logic here makes sense (I'm also concerned about completions diverging from real behavior). But I've found this in zsh's _git: `(- :)--abort[cancel revert or cherry-pick sequence]` This form disables completion of any other arguments, and I think it's future-proof enough to do this for `--abort` here. I'll do a follow-up to touch just exclusivity when this series lands. > spectral wrote in zsh_completion:828 > Does the order of the first section matter? Should the three of them be > identical? I don't think it matters. All options that the group mentions just disappear from completion when `-l`, `-m` or `-u` is present. > spectral wrote in zsh_completion:1198 > Does this indicate that `hg rebase -d ` should work (without a > `-r`, `-s`, or `-b`?) I don't think that's correct.. You mean giving revisions without `-r`? No, `*` means that `-r` can be specified multiple times (and each would require an argument). To see how it's done for a command that has "optional" `-r` see _hg_cmd_update or _hg_cmd_strip above ^ REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4267 To: av6, #hg-reviewers, spectral Cc: spectral, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4281: branchmap: load branchmap as an iterable
mjpieters created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This avoids reading all the file into memory if the cache turns out to be invalid. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4281 AFFECTED FILES mercurial/branchmap.py CHANGE DETAILS diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py --- a/mercurial/branchmap.py +++ b/mercurial/branchmap.py @@ -40,13 +40,7 @@ def read(repo): try: f = repo.cachevfs(_filename(repo)) -lines = f.read().split('\n') -f.close() -except (IOError, OSError): -return None - -try: -cachekey = lines.pop(0).split(" ", 2) +cachekey = next(f).split(" ", 2) last, lrev = cachekey[:2] last, lrev = bin(last), int(lrev) filteredhash = None @@ -58,7 +52,7 @@ # invalidate the cache raise ValueError(r'tip differs') cl = repo.changelog -for l in lines: +for l in f: if not l: continue node, state, label = l.split(" ", 2) @@ -72,6 +66,10 @@ partial.setdefault(label, []).append(node) if state == 'c': partial._closednodes.add(node) + +except (IOError, OSError): +return None + except Exception as inst: if repo.ui.debugflag: msg = 'invalid branchheads cache' To: mjpieters, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4280: perf: time loading branchmap caches
mjpieters created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4280 AFFECTED FILES contrib/perf.py CHANGE DETAILS diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -1720,6 +1720,29 @@ branchcachewrite.restore() fm.end() +@command('perfbranchmapload', [ + ('f', 'filter', '', 'Specify repoview filter'), + ('', 'list', False, 'List brachmap filter caches'), +] + formatteropts) +def perfbranchmapread(ui, repo, filter='', list=False, **opts): +"""benchmark reading the branchmap""" +if list: +for name, kind, st in repo.cachevfs.readdir(stat=True): +if name.startswith('branch2'): +filtername = name.partition('-')[2] or 'unfiltered' +ui.status('%s - %s\n' % (filtername, util.bytecount(st.st_size))) +return +if filter: +repo = repoview.repoview(repo, filter) +else: +repo = repo.unfiltered() +# try once without timer, the filter may not be cached +if branchmap.read(repo) is None: +raise error.Abort('No brachmap cached for %s repo' % (filter or 'unfiltered')) +timer, fm = gettimer(ui, opts) +timer(lambda: branchmap.read(repo) and None) +fm.end() + @command('perfloadmarkers') def perfloadmarkers(ui, repo): """benchmark the time to parse the on-disk markers for a repo To: mjpieters, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[Bug 5958] New: A case that "shouldn't happen" can happen relatively easily
https://bz.mercurial-scm.org/show_bug.cgi?id=5958 Bug ID: 5958 Summary: A case that "shouldn't happen" can happen relatively easily Product: Mercurial Version: unspecified Hardware: PC OS: Linux Status: UNCONFIRMED Severity: feature Priority: wish Component: evolution Assignee: bugzi...@mercurial-scm.org Reporter: h...@pewpew.net CC: mercurial-devel@mercurial-scm.org, pierre-yves.da...@ens-lyon.org The hint isn't super useful to me (I don't know what it's saying I should do), and one of my users managed to hit this case. Here's the .t test that shows how to reproduce: Show that a case that "shouldn't happen" can happen without too much difficulty. $ hg init shouldnt_happen $ cd shouldnt_happen (Let's make the glog output nicer) $ echo hi > r0 $ hg ci -qAm 'add r0' $ echo hi > foo.txt $ hg ci -qAm 'add foo.txt' $ hg touch -q (Make changes in unrelated files so that we don't have any merge conflicts during the rebase, but the two touched revisions aren't identical) $ echo hi > bar.txt $ hg add -q bar.txt $ hg amend -q $ hg touch -q --allowdivergence -r 1 --hidden 2 new content-divergent changesets $ echo hi > baz.txt $ hg add -q baz.txt $ hg amend -q $ hg rebase -qr tip -d 4 --hidden $ hg glog @ 6:6c25f9b11514 add foo.txt | () [default] draft * 4:b46bf84699a8 add foo.txt | () [default] draft o 0:a24ed8ad918c add r0 () [default] draft $ hg obslog -a -r . @ 6c25f9b11514 (6) add foo.txt | | * b46bf84699a8 (4) add foo.txt | | x | 33e073a9d78a (5) add foo.txt | |rewritten(meta, parent, content) as 6c25f9b11514 using rebase by test (Thu Jan 01 00:00:00 1970 +) | | x | 07137fa7fd35 (3) add foo.txt | |rewritten(content) as 33e073a9d78a using amend by test (Thu Jan 01 00:00:00 1970 +) | | x | 79741c4b51f3 (2) add foo.txt |/ rewritten(content) as 07137fa7fd35 using amend by test (Thu Jan 01 00:00:00 1970 +) | x cc71ffbc7c00 (1) add foo.txt rewritten(meta) as 79741c4b51f3 using touch by test (Thu Jan 01 00:00:00 1970 +) rewritten(meta) as b46bf84699a8 using touch by test (Thu Jan 01 00:00:00 1970 +) $ hg evolve --content-divergent merge:[6] add foo.txt with: [4] add foo.txt base: [1] add foo.txt rebasing "other" content-divergent changeset b46bf84699a8 on b46bf84699a8 abort: tried to relocate a node on top of itself (This shouldn't happen. If you still need to move changesets, please do so manually with nothing to rebase - working directory parent is also destination) [255] -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4262: zsh_completion: drop -w, add -S to _arguments
spectral added a comment. In https://phab.mercurial-scm.org/D4262#65777, @av6 wrote: > Thanks, `‘` and `’`, for making it difficult to phabsend this. I had to > > diff --git a/contrib/phabricator.py b/contrib/phabricator.py > --- a/contrib/phabricator.py > +++ b/contrib/phabricator.py > @@ -117,6 +117,8 @@ def urlencodenested(params): >def process(prefix, obj): >items = {list: enumerate, dict: lambda x: x.items()}.get(type(obj)) >if items is None: > +if isinstance(obj, unicode): > +obj = obj.encode('utf-8') >flatparams[prefix] = obj >else: >for k, v in items(obj): > > > But having word diff for this series makes up for the frustration, I think. Can you send that patch too (maybe you already have, just not to phabricator... I don't generally follow mercurial-devel@)? I've also encountered unicode issues with phabsend, but didn't have the time or ability to track it down and just removed the unicode from my commit description :) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4262 To: av6, #hg-reviewers, spectral Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4267: zsh_completion: update all options
spectral accepted this revision. spectral added inline comments. INLINE COMMENTS > zsh_completion:373 > # Common options > _hg_global_opts=( > +'(--repository -R)'{-R+,--repository=}'[repository root directory or > name of overlay bundle file]:repository:_files -/' I wonder how hard it would be to add tests to make sure these don't diverge from the actual output in the future? (no action required in this change, just curious if you have any ideas). At my job our code review system has a "if this is modified, then you probably want to modify this other place" mechanism, I wonder if something like that, generically as part of one of the existing tests (or on its own) would be easier and make more sense than attempting to match the output, just rely on people doing the right thing when the tests prompt them to (with a way of ignoring it, perhaps via something in the commit description). Example: 1. IfChange @command('^add', walkopts + subrepoopts + dryrunopts, _('[OPTION]... [FILE]...'), inferrrepo=True) 2. ThenChange(contrib/zsh_completion, contrib/bash_completion) def add(ui, repo, *pats, **opts): ... Though, since we'd be doing this for every @command, I'm back to thinking it'd probably be better to hard-code it for the completion stuff - something that says "Oh, you've modified the contents of an @command, make sure you update these other locations as well..." > zsh_completion:644 > + '*'{-r+,--rev=}'[revisions to graft]:revision:_hg_labels' \ > + '(--continue -c --abort -a)'{-c,--continue}'[resume interrupted graft]' \ > + '(--continue -c --abort -a)'{-a,--abort}'[abort interrupted graft]' \ I assume the first section here makes continue and abort mutually exclusive? I believe that revision arguments are also forbidden if either --continue or --abort are specified; I don't know if there's a way of representing that. > zsh_completion:828 >'(--mark -m --list -l --unmark -u)'{-m,--mark}'[mark files as > resolved]:*:unresolved files:_hg_unresolved' \ > - '(--unmark -u --list -l --mark -m)'{-u,--unmark}'[unmark files as > resolved]:*:resolved files:_hg_resolved' \ > + '(--unmark -u --list -l --mark -m)'{-u,--unmark}'[mark files as > unresolved]:*:resolved files:_hg_resolved' \ >'*:file:_hg_unresolved' && ret=0 Does the order of the first section matter? Should the three of them be identical? > zsh_completion:1198 >_arguments -s -S : $_hg_global_opts $_hg_commit_opts $_hg_mergetool_opts \ >'*'{-r+,--rev=}'[rebase these revisions]:revision:_hg_revrange' \ > + '(--source -s --base -b)'{-s+,--source=}'[rebase the specified changeset > and descendants]:revision:_hg_labels' \ Does this indicate that `hg rebase -d ` should work (without a `-r`, `-s`, or `-b`?) I don't think that's correct.. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4267 To: av6, #hg-reviewers, spectral Cc: spectral, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4264: zsh_completion: declare appropriate local parameters for ->string form
spectral accepted this revision. spectral added a comment. I don't understand this aspect super well, I admit I haven't read the docs you linked to, but I can see why this would be more correct :) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4264 To: av6, #hg-reviewers, spectral Cc: spectral, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4261: zsh_completion: rename _hg_style_opts to _hg_template_opts
spectral accepted this revision. spectral added a comment. (Like the description says, this hides --style in tab completion. I think this is an improvement). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4261 To: av6, #hg-reviewers, spectral Cc: spectral, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4277: manifest: add getstorage() to manifestlog and use it globally
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY It is a common pattern to obtain a directory manifest storage instance (a manifestrevlog) by going through manifestlog._revlog.dirlog(). Why access to storage and caching of other manifests is done through manifestrevlog instead of manifestlog, I don't know. This commit establishes a getstorage(tree) API on manifestlog and imanifestlog that provides a public API for accessing manifest storage. All consumers previously using private attributes have been updated to use this new method. .. api:: manifestlog now has a getstorage(tree) method It should be used for obtaining an object representing the manifest's storage implementation. Accessing manifestlog._revlog should be avoided. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4277 AFFECTED FILES hgext/convert/hg.py mercurial/changegroup.py mercurial/cmdutil.py mercurial/debugcommands.py mercurial/manifest.py mercurial/repair.py mercurial/repository.py mercurial/verify.py CHANGE DETAILS diff --git a/mercurial/verify.py b/mercurial/verify.py --- a/mercurial/verify.py +++ b/mercurial/verify.py @@ -45,7 +45,7 @@ self.errors = 0 self.warnings = 0 self.havecl = len(repo.changelog) > 0 -self.havemf = len(repo.manifestlog._revlog) > 0 +self.havemf = len(repo.manifestlog.getstorage(b'')) > 0 self.revlogv1 = repo.changelog.version != revlog.REVLOGV0 self.lrugetctx = util.lrucachefunc(repo.__getitem__) self.refersmf = False @@ -205,7 +205,7 @@ ui = self.ui match = self.match mfl = self.repo.manifestlog -mf = mfl._revlog.dirlog(dir) +mf = mfl.getstorage(dir) if not dir: self.ui.status(_("checking manifests\n")) diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -1023,6 +1023,15 @@ interface. """ +def getstorage(tree): +"""Retrieve an interface to storage for a particular tree. + +If ``tree`` is the empty bytestring, storage for the root manifest will +be returned. Otherwise storage for a tree manifest is returned. + +TODO formalize interface for returned object. +""" + def clearcaches(): """Clear caches associated with this collection.""" diff --git a/mercurial/repair.py b/mercurial/repair.py --- a/mercurial/repair.py +++ b/mercurial/repair.py @@ -81,7 +81,7 @@ return [revlog.linkrev(r) for r in brokenset] def _collectmanifest(repo, striprev): -return _collectrevlog(repo.manifestlog._revlog, striprev) +return _collectrevlog(repo.manifestlog.getstorage(b''), striprev) def _collectbrokencsets(repo, files, striprev): """return the changesets which will be broken by the truncation""" @@ -322,7 +322,7 @@ callback.addnodes(nodelist) def stripmanifest(repo, striprev, tr, files): -revlog = repo.manifestlog._revlog +revlog = repo.manifestlog.getstorage(b'') revlog.strip(striprev, tr) striptrees(repo, tr, striprev, files) @@ -333,7 +333,7 @@ if (unencoded.startswith('meta/') and unencoded.endswith('00manifest.i')): dir = unencoded[5:-12] -repo.manifestlog._revlog.dirlog(dir).strip(striprev, tr) +repo.manifestlog.getstorage(dir).strip(striprev, tr) def rebuildfncache(ui, repo): """Rebuilds the fncache file from repo history. diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1452,7 +1452,7 @@ if tree: if self._revlog._treeondisk: if verify: -dirlog = self._revlog.dirlog(tree) +dirlog = self.getstorage(tree) if node not in dirlog.nodemap: raise LookupError(node, dirlog.indexfile, _('no node')) @@ -1479,6 +1479,9 @@ mancache[node] = m return m +def getstorage(self, tree): +return self._revlog.dirlog(tree) + def clearcaches(self, clear_persisted_data=False): self._dirmancache.clear() self._revlog.clearcaches(clear_persisted_data=clear_persisted_data) @@ -1638,7 +1641,7 @@ if not narrowmatch.always(): if not narrowmatch.visitdir(self._dir[:-1] or '.'): return excludedmanifestrevlog(self._dir) -return self._manifestlog._revlog.dirlog(self._dir) +return self._manifestlog.getstorage(self._dir) def read(self): if self._data is None: diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py --- a/mercurial/debugcommands.py +++ b/mercurial/debugcommands.py @@ -1486,7
D4271: changegroup: rename mfs to manifests
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY In the name of readability. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4271 AFFECTED FILES mercurial/changegroup.py CHANGE DETAILS diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -850,7 +850,7 @@ self._verbosenote(_('%8.i (changelog)\n') % size) clrevorder = clstate['clrevorder'] -mfs = clstate['mfs'] +manifests = clstate['manifests'] changedfiles = clstate['changedfiles'] # We need to make sure that the linkrev in the changegroup refers to @@ -879,7 +879,7 @@ size = 0 it = self.generatemanifests( -commonrevs, clrevorder, fastpathlinkrev, mfs, fnodes, source, +commonrevs, clrevorder, fastpathlinkrev, manifests, fnodes, source, clstate['clrevtomanifestrev']) for tree, deltas in it: @@ -905,9 +905,9 @@ mfdicts = None if self._ellipses and self._isshallow: mfdicts = [(self._repo.manifestlog[n].read(), lr) - for (n, lr) in mfs.iteritems()] + for (n, lr) in manifests.iteritems()] -mfs.clear() +manifests.clear() clrevs = set(cl.rev(x) for x in clnodes) it = self.generatefiles(changedfiles, commonrevs, @@ -944,7 +944,7 @@ chunk stream has been fully consumed. """ clrevorder = {} -mfs = {} # needed manifests +manifests = {} mfl = self._repo.manifestlog changedfiles = set() clrevtomanifestrev = {} @@ -957,7 +957,7 @@ clrevorder[x] = len(clrevorder) if self._ellipses: -# Only update mfs if x is going to be sent. Otherwise we +# Only update manifests if x is going to be sent. Otherwise we # end up with bogus linkrevs specified for manifests and # we skip some manifest nodes that we should otherwise # have sent. @@ -967,7 +967,7 @@ manifestnode = c.manifest # Record the first changeset introducing this manifest # version. -mfs.setdefault(manifestnode, x) +manifests.setdefault(manifestnode, x) # Set this narrow-specific dict so we have the lowest # manifest revnum to look up for this cl revnum. (Part of # mapping changelog ellipsis parents to manifest ellipsis @@ -982,16 +982,16 @@ changedfiles.update(c.files) else: # record the first changeset introducing this manifest version -mfs.setdefault(c.manifest, x) +manifests.setdefault(c.manifest, x) # Record a complete list of potentially-changed files in # this manifest. changedfiles.update(c.files) return x state = { 'clrevorder': clrevorder, -'mfs': mfs, +'manifests': manifests, 'changedfiles': changedfiles, 'clrevtomanifestrev': clrevtomanifestrev, } @@ -1009,25 +1009,25 @@ return state, gen -def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, mfs, - fnodes, source, clrevtolocalrev): +def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, + manifests, fnodes, source, clrevtolocalrev): """Returns an iterator of changegroup chunks containing manifests. `source` is unused here, but is used by extensions like remotefilelog to change what is sent based in pulls vs pushes, etc. """ repo = self._repo mfl = repo.manifestlog dirlog = mfl._revlog.dirlog -tmfnodes = {'': mfs} +tmfnodes = {'': manifests} # Callback for the manifest, used to collect linkrevs for filelog # revisions. # Returns the linkrev node (collected in lookupcl). def makelookupmflinknode(tree, nodes): if fastpathlinkrev: assert not tree -return mfs.__getitem__ +return manifests.__getitem__ def lookupmflinknode(x): """Callback for looking up the linknode for manifests. To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4274: manifest: remove addgroup() from manifestlog and imanifestlog
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY addgroup() is part of the storage interface for manifests. Unfortunately, we don't yet have a formal storage interface for manifests. (One will be established in subsequent commits.) One thing is for sure, addgroup() doesn't belong on imanifestlog - at least not unless we extend that interface to encompass storage. For now, let's access addgroup() on the _revlog attribute, just like we do for tree manifests. Even though this violates visibility, it is consistent. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4274 AFFECTED FILES mercurial/changegroup.py mercurial/manifest.py mercurial/repository.py CHANGE DETAILS diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -1032,22 +1032,6 @@ Raises ``error.LookupError`` if the node is not known. """ -def addgroup(deltas, linkmapper, transaction): -"""Process a series of deltas for storage. - -``deltas`` is an iterable of 7-tuples of -(node, p1, p2, linknode, deltabase, delta, flags) defining revisions -to add. - -The ``delta`` field contains ``mpatch`` data to apply to a base -revision, identified by ``deltabase``. The base node can be -``nullid``, in which case the header from the delta can be ignored -and the delta used as the fulltext. - -Returns a list of nodes that were processed. A node will be in the list -even if it existed in the store previously. -""" - class completelocalrepository(interfaceutil.Interface): """Monolithic interface for local repositories. diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1485,9 +1485,6 @@ def rev(self, node): return self._revlog.rev(node) -def addgroup(self, deltas, linkmapper, transaction): -return self._revlog.addgroup(deltas, linkmapper, transaction) - @interfaceutil.implementer(repository.imanifestrevisionwritable) class memmanifestctx(object): def __init__(self, manifestlog): diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -253,7 +253,8 @@ # be empty during the pull self.manifestheader() deltas = self.deltaiter() -repo.manifestlog.addgroup(deltas, revmap, trp) +# TODO this violates storage abstraction. +repo.manifestlog._revlog.addgroup(deltas, revmap, trp) prog.complete() self.callback = None To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4269: changegroup: call rev() on manifestlog instance
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY rev() is part of the imanifestlog interface and should be used instead of using the private revlog instance, which is an implementation detail. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4269 AFFECTED FILES mercurial/changegroup.py CHANGE DETAILS diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -946,8 +946,6 @@ clrevorder = {} mfs = {} # needed manifests mfl = self._repo.manifestlog -# TODO violates storage abstraction. -mfrevlog = mfl._revlog changedfiles = set() clrevtomanifestrev = {} @@ -973,7 +971,7 @@ # manifest revnum to look up for this cl revnum. (Part of # mapping changelog ellipsis parents to manifest ellipsis # parents) -clrevtomanifestrev.setdefault(cl.rev(x), mfrevlog.rev(n)) +clrevtomanifestrev.setdefault(cl.rev(x), mfl.rev(n)) # We can't trust the changed files list in the changeset if the # client requested a shallow clone. if self._isshallow: To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4279: manifest: use rev() instead of nodemap.__contains__
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY nodemap is an implementation detail of revlogs and isn't appropriate to expose on the manifest storage API. While revlogs don't have a __contains__, they do have lookup() for resolving a value to a node. And this calls rev(), whose API is documented to raise LookupError if a node doesn't exist. And the parameters to LookupError are identical to what was being raised here. So this change should be backwards compatible. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4279 AFFECTED FILES mercurial/manifest.py CHANGE DETAILS diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1453,20 +1453,20 @@ if tree: if self._revlog._treeondisk: if verify: -dirlog = self.getstorage(tree) -if node not in dirlog.nodemap: -raise LookupError(node, dirlog.indexfile, - _('no node')) +# Side-effect is LookupError is raised if node doesn't +# exist. +self.getstorage(tree).rev(node) + m = treemanifestctx(self, tree, node) else: raise error.Abort( _("cannot ask for manifest directory '%s' in a flat " "manifest") % tree) else: if verify: -if node not in self._revlog.nodemap: -raise LookupError(node, self._revlog.indexfile, - _('no node')) +# Side-effect is LookupError is raised if node doesn't exist. +self._revlog.rev(node) + if self._treemanifests: m = treemanifestctx(self, '', node) else: To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4272: changegroup: change topics during generation
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Changegroup generation and consumption currently uses different sets of topics. Generation uses "bundling" and consumption uses the name of the thing being consumed. This commit makes the topic and unit names consistent on both ends. The reason I chose to change the generation side is because when "bundling" is used for 3 different stages, it is unclear which stage of changegroup generation we're in. By making the topics "changesets," "manifests," and "files," one can get a better feel for how far along in the bundling process we are. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4272 AFFECTED FILES mercurial/changegroup.py tests/test-bundle.t tests/test-bundle2-format.t CHANGE DETAILS diff --git a/tests/test-bundle2-format.t b/tests/test-bundle2-format.t --- a/tests/test-bundle2-format.t +++ b/tests/test-bundle2-format.t @@ -873,17 +873,17 @@ bundle2-output-part: "changegroup" (advisory) streamed payload bundle2-output: part 0: "changegroup" bundle2-output: header chunk size: 18 - bundling: 1/4 changesets (25.00%) - bundling: 2/4 changesets (50.00%) - bundling: 3/4 changesets (75.00%) - bundling: 4/4 changesets (100.00%) - bundling: 1/4 manifests (25.00%) - bundling: 2/4 manifests (50.00%) - bundling: 3/4 manifests (75.00%) - bundling: 4/4 manifests (100.00%) - bundling: D 1/3 files (33.33%) - bundling: E 2/3 files (66.67%) - bundling: H 3/3 files (100.00%) + changesets: 1/4 chunks (25.00%) + changesets: 2/4 chunks (50.00%) + changesets: 3/4 chunks (75.00%) + changesets: 4/4 chunks (100.00%) + manifests: 1/4 chunks (25.00%) + manifests: 2/4 chunks (50.00%) + manifests: 3/4 chunks (75.00%) + manifests: 4/4 chunks (100.00%) + files: D 1/3 files (33.33%) + files: E 2/3 files (66.67%) + files: H 3/3 files (100.00%) bundle2-output: payload chunk size: 1555 bundle2-output: closing payload chunk bundle2-output: end of bundle diff --git a/tests/test-bundle.t b/tests/test-bundle.t --- a/tests/test-bundle.t +++ b/tests/test-bundle.t @@ -796,13 +796,13 @@ 057f4db07f61970e1c11e83be79e9d08adc4dc31 bundle2-output-bundle: "HG20", (1 params) 2 parts total bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload - bundling: 1/2 changesets (50.00%) - bundling: 2/2 changesets (100.00%) - bundling: 1/2 manifests (50.00%) - bundling: 2/2 manifests (100.00%) - bundling: b 1/3 files (33.33%) - bundling: b1 2/3 files (66.67%) - bundling: x 3/3 files (100.00%) + changesets: 1/2 chunks (50.00%) + changesets: 2/2 chunks (100.00%) + manifests: 1/2 chunks (50.00%) + manifests: 2/2 chunks (100.00%) + files: b 1/3 files (33.33%) + files: b1 2/3 files (66.67%) + files: x 3/3 files (100.00%) bundle2-output-part: "cache:rev-branch-cache" (advisory) streamed payload #if repobundlerepo diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -658,15 +658,15 @@ def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev, allowreorder, - units=None, + topic=None, ellipses=False, clrevtolocalrev=None, fullclnodes=None, precomputedellipsis=None): """Calculate deltas for a set of revisions. Is a generator of ``revisiondelta`` instances. -If units is not None, progress detail will be generated, units specifies -the type of revlog that is touched (changelog, manifest, etc.). +If topic is not None, progress detail will be generated using this +topic name (e.g. changesets, manifests, etc). """ if not nodes: return @@ -743,8 +743,8 @@ # We expect the first pass to be fast, so we only engage the progress # meter for constructing the revision deltas. progress = None -if units is not None: -progress = repo.ui.makeprogress(_('bundling'), unit=units, +if topic is not None: +progress = repo.ui.makeprogress(topic, unit=_('chunks'), total=len(requests)) for i, delta in enumerate(store.emitrevisiondeltas(requests)): @@ -1002,7 +1002,7 @@ # Reorder settings are currently ignored for changelog. True, ellipses=self._ellipses, -units=_('changesets'), +topic=_('changesets'), clrevtolocalrev={}, fullclnodes=self._fullclnodes, precomputedellipsis=self._precomputedellipsis) @@ -1083,7 +1083,7 @@ self._repo, store, prunednodes, False, lookupfn, self._forcedeltaparentprev, self._reorder, ellipses=self._ellipses, -units=_('manifests'), +topic=_('manifests'),
D4273: repository: clarify role of imanifestlog
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Looking at the implementation of manifest.manifestlog, it is highly tailored towards servicing the root manifest. So clarify that in the interface docstring. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4273 AFFECTED FILES mercurial/repository.py CHANGE DETAILS diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -989,7 +989,13 @@ """ class imanifestlog(interfaceutil.Interface): -"""Interface representing a collection of manifest snapshots.""" +"""Interface representing a collection of manifest snapshots. + +Represents the root manifest in a repository. + +Also serves as a means to access nested tree manifests and to cache +tree manifests. +""" def __getitem__(node): """Obtain a manifest instance for a given binary node. To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4276: manifest: rename dir argument and attribute to tree
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY dir shadows a built-in. We use it throughout the manifest code, which is unfortunate. This commit updates just manifestrevlog to be more well behaved. .. api:: renamed manifest.manifestrevlog.__init__ dir argument to tree REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4276 AFFECTED FILES mercurial/bundlerepo.py mercurial/changegroup.py mercurial/manifest.py mercurial/upgrade.py CHANGE DETAILS diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py --- a/mercurial/upgrade.py +++ b/mercurial/upgrade.py @@ -450,7 +450,7 @@ return changelog.changelog(repo.svfs) elif path.endswith('00manifest.i'): mandir = path[:-len('00manifest.i')] -return manifest.manifestrevlog(repo.svfs, dir=mandir) +return manifest.manifestrevlog(repo.svfs, tree=mandir) else: #reverse of "/".join(("data", path + ".i")) return filelog.filelog(repo.svfs, path[5:-2]) diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1250,7 +1250,7 @@ '''A revlog that stores manifest texts. This is responsible for caching the full-text manifest contents. ''' -def __init__(self, opener, dir='', dirlogcache=None, indexfile=None, +def __init__(self, opener, tree='', dirlogcache=None, indexfile=None, treemanifest=False): """Constructs a new manifest revlog @@ -1276,24 +1276,24 @@ self._fulltextcache = manifestfulltextcache(cachesize) -if dir: +if tree: assert self._treeondisk, 'opts is %r' % opts if indexfile is None: indexfile = '00manifest.i' -if dir: -indexfile = "meta/" + dir + indexfile +if tree: +indexfile = "meta/" + tree + indexfile -self._dir = dir +self._tree = tree # The dirlogcache is kept on the root manifest log -if dir: +if tree: self._dirlogcache = dirlogcache else: self._dirlogcache = {'': self} super(manifestrevlog, self).__init__(opener, indexfile, # only root indexfile is cached - checkambig=not bool(dir), + checkambig=not bool(tree), mmaplargeindex=True) def _setupmanifestcachehooks(self, repo): @@ -1325,7 +1325,7 @@ def clearcaches(self, clear_persisted_data=False): super(manifestrevlog, self).clearcaches() self._fulltextcache.clear(clear_persisted_data=clear_persisted_data) -self._dirlogcache = {self._dir: self} +self._dirlogcache = {self._tree: self} def dirlog(self, d): if d: @@ -1360,8 +1360,8 @@ # process. if self._treeondisk: assert readtree, "readtree must be set for treemanifest writes" -m1 = readtree(self._dir, p1) -m2 = readtree(self._dir, p2) +m1 = readtree(self._tree, p1) +m2 = readtree(self._tree, p2) n = self._addtree(m, transaction, link, m1, m2, readtree) arraytext = None else: @@ -1377,16 +1377,17 @@ def _addtree(self, m, transaction, link, m1, m2, readtree): # If the manifest is unchanged compared to one parent, # don't write a new revision -if self._dir != '' and (m.unmodifiedsince(m1) or m.unmodifiedsince(m2)): +if self._tree != '' and (m.unmodifiedsince(m1) or m.unmodifiedsince( +m2)): return m.node() def writesubtree(subm, subp1, subp2): sublog = self.dirlog(subm.dir()) sublog.add(subm, transaction, link, subp1, subp2, None, None, readtree=readtree) m.writesubtrees(m1, m2, writesubtree) text = m.dirtext() n = None -if self._dir != '': +if self._tree != '': # Double-check whether contents are unchanged to one parent if text == m1.dirtext(): n = m1.node() diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -1068,7 +1068,7 @@ tree, nodes = tmfnodes.popitem() store = dirlog(tree) -if not self._filematcher.visitdir(store._dir[:-1] or '.'): +if not self._filematcher.visitdir(store._tree[:-1] or '.'): prunednodes = [] else: frev, flr = store.rev, store.linkrev diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py --- a/mercurial/bundlerepo.py +++ b/mercurial/bundlerepo.py @@
D4275: manifest: set appropriate cache entry when clearing _dirlogcache()
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY manifestrevlog applies to any manifest, not just the root manifest. Resetting the cache and populating its root entry with self is not correct when the instance does not refer to the root manifest. This has no test fallout. So I'm guessing we only ever call clearcaches() on the root manifest. Or we have no test coverage that are impacted by a bad cache on a non-root manifestrevlog. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4275 AFFECTED FILES mercurial/manifest.py CHANGE DETAILS diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1325,7 +1325,7 @@ def clearcaches(self, clear_persisted_data=False): super(manifestrevlog, self).clearcaches() self._fulltextcache.clear(clear_persisted_data=clear_persisted_data) -self._dirlogcache = {'': self} +self._dirlogcache = {self._dir: self} def dirlog(self, d): if d: To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4278: manifest: rename manifestlog._treeinmem to ._treemanifests
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Not sure what "inmem" was supposed to indicate. This object is an interface to manifest data on disk as well as "in memory" (assuming that's what "inmem" means). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4278 AFFECTED FILES mercurial/manifest.py CHANGE DETAILS diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1418,7 +1418,8 @@ if opts is not None: usetreemanifest = opts.get('treemanifest', usetreemanifest) cachesize = opts.get('manifestcachesize', cachesize) -self._treeinmem = usetreemanifest + +self._treemanifests = usetreemanifest self._revlog = repo._constructmanifest() self._revlog._setupmanifestcachehooks(repo) @@ -1466,7 +1467,7 @@ if node not in self._revlog.nodemap: raise LookupError(node, self._revlog.indexfile, _('no node')) -if self._treeinmem: +if self._treemanifests: m = treemanifestctx(self, '', node) else: m = manifestctx(self, node) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4270: changegroup: clean up changelog callback
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY changelog.read() calls changelog.changelogrevion() then turns it into a tuple. Let's call changelogrevision() directly and used named attributes to make code easier to read. While we're here, also change some variable names to make code easier to read. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4270 AFFECTED FILES mercurial/changegroup.py CHANGE DETAILS diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py --- a/mercurial/changegroup.py +++ b/mercurial/changegroup.py @@ -953,7 +953,7 @@ # manifest nodes. # Returns the linkrev node (identity in the changelog case). def lookupcl(x): -c = cl.read(x) +c = cl.changelogrevision(x) clrevorder[x] = len(clrevorder) if self._ellipses: @@ -963,29 +963,29 @@ # have sent. if (x in self._fullclnodes or cl.rev(x) in self._precomputedellipsis): -n = c[0] + +manifestnode = c.manifest # Record the first changeset introducing this manifest # version. -mfs.setdefault(n, x) +mfs.setdefault(manifestnode, x) # Set this narrow-specific dict so we have the lowest # manifest revnum to look up for this cl revnum. (Part of # mapping changelog ellipsis parents to manifest ellipsis # parents) -clrevtomanifestrev.setdefault(cl.rev(x), mfl.rev(n)) +clrevtomanifestrev.setdefault( +cl.rev(x), mfl.rev(manifestnode)) # We can't trust the changed files list in the changeset if the # client requested a shallow clone. if self._isshallow: -changedfiles.update(mfl[c[0]].read().keys()) +changedfiles.update(mfl[c.manifest].read().keys()) else: -changedfiles.update(c[3]) +changedfiles.update(c.files) else: - -n = c[0] # record the first changeset introducing this manifest version -mfs.setdefault(n, x) +mfs.setdefault(c.manifest, x) # Record a complete list of potentially-changed files in # this manifest. -changedfiles.update(c[3]) +changedfiles.update(c.files) return x To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4268: manifest: rename dir to tree to avoid shadowing built-in
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY And update the argument name in the imanifestlog interface. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4268 AFFECTED FILES mercurial/manifest.py mercurial/repository.py CHANGE DETAILS diff --git a/mercurial/repository.py b/mercurial/repository.py --- a/mercurial/repository.py +++ b/mercurial/repository.py @@ -1000,15 +1000,15 @@ interface. """ -def get(dir, node, verify=True): +def get(tree, node, verify=True): """Retrieve the manifest instance for a given directory and binary node. ``node`` always refers to the node of the root manifest (which will be the only manifest if flat manifests are being used). -If ``dir`` is the empty string, the root manifest is returned. Otherwise -the manifest for the specified directory will be returned (requires -tree manifests). +If ``tree`` is the empty string, the root manifest is returned. +Otherwise the manifest for the specified directory will be returned +(requires tree manifests). If ``verify`` is True, ``LookupError`` is raised if the node is not known. diff --git a/mercurial/manifest.py b/mercurial/manifest.py --- a/mercurial/manifest.py +++ b/mercurial/manifest.py @@ -1435,31 +1435,31 @@ """ return self.get('', node) -def get(self, dir, node, verify=True): +def get(self, tree, node, verify=True): """Retrieves the manifest instance for the given node. Throws a LookupError if not found. `verify` - if True an exception will be thrown if the node is not in the revlog """ -if node in self._dirmancache.get(dir, ()): -return self._dirmancache[dir][node] +if node in self._dirmancache.get(tree, ()): +return self._dirmancache[tree][node] if not self._narrowmatch.always(): -if not self._narrowmatch.visitdir(dir[:-1] or '.'): -return excludeddirmanifestctx(dir, node) -if dir: +if not self._narrowmatch.visitdir(tree[:-1] or '.'): +return excludeddirmanifestctx(tree, node) +if tree: if self._revlog._treeondisk: if verify: -dirlog = self._revlog.dirlog(dir) +dirlog = self._revlog.dirlog(tree) if node not in dirlog.nodemap: raise LookupError(node, dirlog.indexfile, _('no node')) -m = treemanifestctx(self, dir, node) +m = treemanifestctx(self, tree, node) else: raise error.Abort( _("cannot ask for manifest directory '%s' in a flat " - "manifest") % dir) + "manifest") % tree) else: if verify: if node not in self._revlog.nodemap: @@ -1471,10 +1471,10 @@ m = manifestctx(self, node) if node != revlog.nullid: -mancache = self._dirmancache.get(dir) +mancache = self._dirmancache.get(tree) if not mancache: mancache = util.lrucachedict(self._cachesize) -self._dirmancache[dir] = mancache +self._dirmancache[tree] = mancache mancache[node] = m return m To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 5 of 5] mail: convert encoding.encoding to sysstr
> On Aug 11, 2018, at 11:27 PM, Yuya Nishihara wrote: > > # HG changeset patch > # User Yuya Nishihara > # Date 1534043768 -32400 > # Sun Aug 12 12:16:08 2018 +0900 > # Node ID 1c137e0a33afe4d7e61325513d2518af8b93434a > # Parent a160d88772ff1da51160faa5cfdbd7ecdbcd07fa > mail: convert encoding.encoding to sysstr queued, thanks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] byteify-strings: prevent "__name__ == '__main__'" from being transformed
> On Aug 12, 2018, at 1:10 AM, Yuya Nishihara wrote: > > # HG changeset patch > # User Yuya Nishihara > # Date 1534050202 -32400 > # Sun Aug 12 14:03:22 2018 +0900 > # Node ID 6ee5e64e124c87b356874367dbf6355707d00883 > # Parent 1c137e0a33afe4d7e61325513d2518af8b93434a > byteify-strings: prevent "__name__ == '__main__'" from being transformed queued, thanks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel