D2108: infinitepush: drop the `--to` flag to push and use `-B` instead

2018-02-12 Thread durham (Durham Goode) 
durham added a comment.


  > There are things which I am not sure whether to keep or not:
  > 
  > - the --bundle-store flag to push command
  
  This is useful for scripts or tools that want to upload a commit to the cloud 
without having to give it a name. For instance, you can use it to push a commit 
then send that commit hash to some build service which can checkout the commit 
without having to worry about a bookmark name.  But this could always be added 
back later, so it's probably fine to drop it if there's not an immediate need 
in Mozilla's use case.
  
  > - functionality to pull from bundlestore using hg pull
  
  Similar to the points above and below, this is useful for automation that 
already passes hashes around.  Not having to pass around bookmark names as well 
means it's easier for that automation to migrate to infinitepush.
  
  > - functionality to pull changesets from bundlestore if a changeset is not 
found locally on hg update
  
  This is a bit of magic that user's really like.  When combined with automatic 
backup pushes, it makes it feel like everyone is using the same repository.  
I'd highly recommend keeping this just for the eventual PR of saying "I can 
just hg commit, and my friend can do hg checkout HASH"
  
  > - logic around sql store
  
  Without this, would the server always store data in the filesystem?  The sql 
store seems like an important bit of making this robust in enterprise usage.
  
  > - interaction with the hoisting functionality of remotenames extension 
which is also being moved to core
  
  I'm not familiar with how infinitepush plays into hoisting, but I just wanted 
to make sure users never have to type 'remote/' or 'default/'.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2108

To: pulkit, #hg-reviewers, indygreg
Cc: indygreg, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1814: rebase: add experimental.inmemory.nomergedriver to turn off IMM

2018-01-05 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> rebase.py:812
> +elif (ui.config('experimental', 'mergedriver') and
> +  ui.configbool('rebase', 
> 'experimental.inmemory.nomergedriver')):
> +whynotimm = 'mergedriver enabled'

I think we probably want IMM disabled by default if merge drivers are 
specified.  If the merge-driver does IO during the preprocess step, it could 
cause problems in an IMM situation. So I don't think we can enable IMM by 
default in the merge-driver case until merge-driver enforces (or at least 
documents) that there should be no IO during preprocess.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1814

To: phillco, #hg-reviewers
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1806: filemerge: fix backing up an in-memory file to a custom location

2018-01-04 Thread durham (Durham Goode) 
durham accepted this revision.
durham added a comment.


  Accepting, since it seems more correct than before.
  
  Should we even be calling _makebackup in the case of an inmemory merge?  
Like, maybe the makebackup should be conditional based on if the source file 
context is actually a workingctx?

INLINE COMMENTS

> filemerge.py:623
> +Backups are only need to be written for the premerge, and not again 
> during
> +the main merge.
>  """

I know you're just documenting the parameter that already existed, but might be 
nice to explain why this is (if you happen to know why right now).

> filemerge.py:643
> +if premerge:
> +# Otherwise, write to wherever path the user specified the 
> backups
> +# should go. We still need to switch based on whether the source 
> is

s/wherever/whatever/

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1806

To: phillco, #hg-reviewers, durham
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1564: worker: make windows workers daemons

2017-11-30 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> worker.py:286
> +threads.remove(t)
> +except KeyboardInterrupt:
> +trykillworkers()

Why only do it on keyboard interrupt?  What if there's another exception?  If 
you did it for all exceptions, you could drop the trykillworkers() inside the 
loop, and just throw the exception up to here.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1564

To: wlis, #hg-reviewers, ikostia
Cc: durham, ikostia, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D821: unamend: move fb extension unamend to core

2017-11-20 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> pulkit wrote in uncommit.py:265
> Okay while trying to add this condition, I found we cannot refuse to unamend 
> a changeset on the basis of unamend_source, for e.g
> `a -amend-> b -unamend-> a' -amend-> c -unamend-> a''`
> 
> But if we refuse on basis on unamend_source, unamend `c` will refuse. We need 
> to be more smart here but when I think the other way around, I think it's 
> okay to unamend an unamend. I am open to suggestion and will go with what you 
> guys prefer is good.

I'd just let unamend undo an unamend.  Letting unamend toggle back and forth 
between the two states seems like it might grant the user more confidence in 
the command, even.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D821

To: pulkit, #hg-reviewers, durham
Cc: quark, durin42, ryanmce, singhsrb, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1358: remotenames: store journal entry for bookmarks if journal is loaded

2017-11-17 Thread durham (Durham Goode) 
durham added a comment.


  There seems like two use cases for remotenames:  knowledge about the most 
recently seen location of each remote bookmark, and knowledge about where the 
remote bookmarks have been over time.  I think 99% of the benefit of 
remotenames comes from the first part.  Building a storage layer which suits 
both the journal and remotenames seems like massive scope creep to address the 
1% use case.  Even if we did unify the storage models, I still don't believe 
remotenames and the journal could share a common storage format because they 
don't have similar access patterns.  Remotenames is much more of a random 
access dict-like object, while the journal is much more of a sequential log.  
If I want to lookup the value of the latest remote master I don't want to deal 
with scanning some log of recent activity, and if I want to maintain a log of 
recent activity I don't want to build an index that allows for random access.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1358

To: pulkit, #hg-reviewers
Cc: durham, quark, durin42, smf, dlax, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1431: sshpeer: making the ssh error message configurable

2017-11-15 Thread durham (Durham Goode) 
durham added a comment.


  I'd also update the commit summary with an example of what a better message 
might be.  Like "there was an ssh error, please see 
http://company/internalwiki/ssh.html;

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1431

To: zuza, #hg-reviewers, durham, mitrandir
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1431: sshpeer: making the ssh error message configurable

2017-11-15 Thread durham (Durham Goode) 
durham requested changes to this revision.
durham added inline comments.
This revision now requires changes to proceed.

INLINE COMMENTS

> sshpeer.py:210
> +"errormessage",
> +_("no suitable response from remote hg"
>  

The standard Mercurial way to format this would be something like:

  msg = self.ui.config("ssh", "errormessage",
   _("no suitable response from remote hg"))
  self._abort(error.RepoError(msg))

> test-ssh.t:314
> +
> +
>  clone bookmarks

These lines seem unrelated to your change and therefore probably shouldn't have 
changed.  Usually this is caused by your editor trying to remove trailing 
whitespace from a file, then run-tests -i adding the missing new line back in.  
Can you undo these new line changes?  'hg uncommit tests/test-ssh.t && hg 
commit -i' could make it pretty straightforward.

> test-ssh.t:447
>> from mercurial import exchange, extensions
> -  > 
>> def wrappedpush(orig, repo, *args, **kwargs):

Same for this type of change

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1431

To: zuza, #hg-reviewers, durham, mitrandir
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1358: remotenames: store journal entry for bookmarks if journal is loaded

2017-11-15 Thread durham (Durham Goode) 
durham added a comment.


  I'm a little confused here, in the same way Pulkit has expressed.  
remotenames has a couple parts related to storage: one is the storing of the 
remotenames themselves, and the other is tracking how they change over time.  
The remotename storage itself doesn't seem like it overlaps with journal or 
blackbox, as it's not a log.  The tracking of how they change over time seems 
to fit the journal exactly, which is a store for tracking how various 
references to commits (working copy parent, bookmarks, and now remotenames) 
change over time.
  
  So I guess I'm confused about what part of remotenames needs unification?  
They seem to either be completely orthogonal, or a complete match for an 
existing system.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1358

To: pulkit, #hg-reviewers
Cc: durham, quark, durin42, smf, dlax, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D821: unamend: move fb extension unamend to core

2017-11-10 Thread durham (Durham Goode) 
durham accepted this revision.
durham added a comment.


  Overall looks good to me.  My one comment is probably not enough to block 
this going in.

INLINE COMMENTS

> uncommit.py:260
> +prednode = markers[0].prednode()
> +predctx = unfi[prednode]
> +

Might be worth doing the predecessor check in the lock as well, since the 
result of this verification could technically change between now and when the 
lock is taken.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D821

To: pulkit, #hg-reviewers, durham
Cc: ryanmce, singhsrb, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1329: bundle: allow bundlerepo to support alternative manifest implementations

2017-11-09 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa2dfc723b6b5: bundle: allow bundlerepo to support 
alternative manifest implementations (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1329?vs=3314=3385

REVISION DETAIL
  https://phab.mercurial-scm.org/D1329

AFFECTED FILES
  mercurial/bundlerepo.py

CHANGE DETAILS

diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py
--- a/mercurial/bundlerepo.py
+++ b/mercurial/bundlerepo.py
@@ -352,14 +352,31 @@
 self.filestart = self.bundle.tell()
 return m
 
+def _consumemanifest(self):
+"""Consumes the manifest portion of the bundle, setting filestart so 
the
+file portion can be read."""
+self.bundle.seek(self.manstart)
+self.bundle.manifestheader()
+for delta in self.bundle.deltaiter():
+pass
+self.filestart = self.bundle.tell()
+
 @localrepo.unfilteredpropertycache
 def manstart(self):
 self.changelog
 return self.manstart
 
 @localrepo.unfilteredpropertycache
 def filestart(self):
 self.manifestlog
+
+# If filestart was not set by self.manifestlog, that means the
+# manifestlog implementation did not consume the manifests from the
+# changegroup (ex: it might be consuming trees from a separate bundle2
+# part instead). So we need to manually consume it.
+if 'filestart' not in self.__dict__:
+self._consumemanifest()
+
 return self.filestart
 
 def url(self):



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1329: bundle: allow bundlerepo to support alternative manifest implementations

2017-11-07 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  With our treemanifest logic, the manifests are no longer transported as part 
of
  the changegroup and are no longer stored in a revlog. This means the
  self.manifestlog line in bundlerepo.filestart no longer calls
  _constructmanifest, and therefore does not consume the manifest portion of the
  changegroup, which means filestart is not populated and we result in an 
infinite
  loop.
  
  The fix is to make filestart aware that self.manifestlog might not consume the
  changegroup part, and consume it manually if necessary.
  
  There's currently no way to test this in core, but our treemanifest extension
  has tests to cover this.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1329

AFFECTED FILES
  mercurial/bundlerepo.py

CHANGE DETAILS

diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py
--- a/mercurial/bundlerepo.py
+++ b/mercurial/bundlerepo.py
@@ -352,14 +352,31 @@
 self.filestart = self.bundle.tell()
 return m
 
+def _consumemanifest(self):
+"""Consumes the manifest portion of the bundle, setting filestart so 
the
+file portion can be read."""
+self.bundle.seek(self.manstart)
+self.bundle.manifestheader()
+for delta in self.bundle.deltaiter():
+pass
+self.filestart = self.bundle.tell()
+
 @localrepo.unfilteredpropertycache
 def manstart(self):
 self.changelog
 return self.manstart
 
 @localrepo.unfilteredpropertycache
 def filestart(self):
 self.manifestlog
+
+# If filestart was not set by self.manifestlog, that means the
+# manifestlog implementation did not consume the manifests from the
+# changegroup (ex: it might be consuming trees from a separate bundle2
+# part instead). So we need to manually consume it.
+if 'filestart' not in self.__dict__:
+self._consumemanifest()
+
 return self.filestart
 
 def url(self):



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1257: dirstate: remove excess attribute lookups for dirstate.status (issue5714)

2017-10-28 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGffeea2406276: dirstate: remove excess attribute lookups for 
dirstate.status (issue5714) (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1257?vs=3147=3150

REVISION DETAIL
  https://phab.mercurial-scm.org/D1257

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -1053,6 +1053,9 @@
 removed, deleted, clean = [], [], []
 
 dmap = self._map
+dmap.preload()
+dcontains = dmap.__contains__
+dget = dmap.__getitem__
 ladd = lookup.append# aka "unsure"
 madd = modified.append
 aadd = added.append
@@ -1074,7 +1077,7 @@
 full = listclean or match.traversedir is not None
 for fn, st in self.walk(match, subrepos, listunknown, listignored,
 full=full).iteritems():
-if fn not in dmap:
+if not dcontains(fn):
 if (listignored or mexact(fn)) and dirignore(fn):
 if listignored:
 iadd(fn)
@@ -1089,7 +1092,7 @@
 # a list, but falls back to creating a full-fledged iterator in
 # general. That is much slower than simply accessing and storing 
the
 # tuple members one by one.
-t = dmap[fn]
+t = dget(fn)
 state = t[0]
 mode = t[1]
 size = t[2]
@@ -1216,8 +1219,8 @@
 return self.copymap
 
 def clear(self):
-self._map = {}
-self.copymap = {}
+self._map.clear()
+self.copymap.clear()
 self.setparents(nullid, nullid)
 
 def iteritems(self):
@@ -1247,6 +1250,10 @@
 def keys(self):
 return self._map.keys()
 
+def preload(self):
+"""Loads the underlying data, if it's not already loaded"""
+self._map
+
 def nonnormalentries(self):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
@@ -1373,6 +1380,13 @@
 if not self._dirtyparents:
 self.setparents(*p)
 
+# Avoid excess attribute lookups by fast pathing certain checks
+self.__contains__ = self._map.__contains__
+self.__getitem__ = self._map.__getitem__
+self.__setitem__ = self._map.__setitem__
+self.__delitem__ = self._map.__delitem__
+self.get = self._map.get
+
 def write(self, st, now):
 st.write(parsers.pack_dirstate(self._map, self.copymap,
self.parents(), now))



To: durham, #hg-reviewers, yuja
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1257: dirstate: remove excess attribute lookups for dirstate.status (issue5714)

2017-10-28 Thread durham (Durham Goode) 
durham added a comment.


  Fixed by using _map.clear() instead of replacing the _map.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1257

To: durham, #hg-reviewers, yuja
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1257: dirstate: remove excess attribute lookups for dirstate.status (issue5714)

2017-10-28 Thread durham (Durham Goode) 
durham updated this revision to Diff 3147.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1257?vs=3136=3147

REVISION DETAIL
  https://phab.mercurial-scm.org/D1257

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -1053,6 +1053,9 @@
 removed, deleted, clean = [], [], []
 
 dmap = self._map
+dmap.preload()
+dcontains = dmap.__contains__
+dget = dmap.__getitem__
 ladd = lookup.append# aka "unsure"
 madd = modified.append
 aadd = added.append
@@ -1074,7 +1077,7 @@
 full = listclean or match.traversedir is not None
 for fn, st in self.walk(match, subrepos, listunknown, listignored,
 full=full).iteritems():
-if fn not in dmap:
+if not dcontains(fn):
 if (listignored or mexact(fn)) and dirignore(fn):
 if listignored:
 iadd(fn)
@@ -1089,7 +1092,7 @@
 # a list, but falls back to creating a full-fledged iterator in
 # general. That is much slower than simply accessing and storing 
the
 # tuple members one by one.
-t = dmap[fn]
+t = dget(fn)
 state = t[0]
 mode = t[1]
 size = t[2]
@@ -1216,8 +1219,8 @@
 return self.copymap
 
 def clear(self):
-self._map = {}
-self.copymap = {}
+self._map.clear()
+self.copymap.clear()
 self.setparents(nullid, nullid)
 
 def iteritems(self):
@@ -1247,6 +1250,10 @@
 def keys(self):
 return self._map.keys()
 
+def preload(self):
+"""Loads the underlying data, if it's not already loaded"""
+self._map
+
 def nonnormalentries(self):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
@@ -1373,6 +1380,13 @@
 if not self._dirtyparents:
 self.setparents(*p)
 
+# Avoid excess attribute lookups by fast pathing certain checks
+self.__contains__ = self._map.__contains__
+self.__getitem__ = self._map.__getitem__
+self.__setitem__ = self._map.__setitem__
+self.__delitem__ = self._map.__delitem__
+self.get = self._map.get
+
 def write(self, st, now):
 st.write(parsers.pack_dirstate(self._map, self.copymap,
self.parents(), now))



To: durham, #hg-reviewers, yuja
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1253: dirstate: avoid reading the map when possible (issue5713) (issue5717)

2017-10-28 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6e66033f91cc: dirstate: avoid reading the map when possible 
(issue5713) (issue5717) (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1253?vs=3129=3146

REVISION DETAIL
  https://phab.mercurial-scm.org/D1253

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -129,7 +129,7 @@
 def _map(self):
 '''Return the dirstate contents as a map from filename to
 (state, mode, size, time).'''
-self._read()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 return self._map
 
 @property
@@ -353,10 +353,6 @@
 f.discard()
 raise
 
-def _read(self):
-self._map = dirstatemap(self._ui, self._opener, self._root)
-self._map.read()
-
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
 
@@ -1201,14 +1197,24 @@
 self._root = root
 self._filename = 'dirstate'
 
-self._map = {}
-self.copymap = {}
 self._parents = None
 self._dirtyparents = False
 
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
 
+@propertycache
+def _map(self):
+self._map = {}
+self.read()
+return self._map
+
+@propertycache
+def copymap(self):
+self.copymap = {}
+self._map
+return self.copymap
+
 def clear(self):
 self._map = {}
 self.copymap = {}
@@ -1388,7 +1394,7 @@
 
 @propertycache
 def identity(self):
-self.read()
+self._map
 return self.identity
 
 @propertycache



To: durham, #hg-reviewers, lothiraldan, yuja
Cc: yuja, lothiraldan, mharbison72, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1252: dirstate: move clear onto dirstatemap class

2017-10-28 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0217f75b6e59: dirstate: move clear onto dirstatemap class 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1252?vs=3128=3145

REVISION DETAIL
  https://phab.mercurial-scm.org/D1252

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -593,8 +593,7 @@
 return path
 
 def clear(self):
-self._map = dirstatemap(self._ui, self._opener, self._root)
-self._map.setparents(nullid, nullid)
+self._map.clear()
 self._lastnormaltime = 0
 self._updatedfiles.clear()
 self._dirty = True
@@ -1210,6 +1209,11 @@
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
 
+def clear(self):
+self._map = {}
+self.copymap = {}
+self.setparents(nullid, nullid)
+
 def iteritems(self):
 return self._map.iteritems()
 



To: durham, #hg-reviewers, yuja
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1257: dirstate: remove excess attribute lookups for dirstate.status (issue5714)

2017-10-27 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  A recent refactor added a layer of abstraction to the dirstate which makes 
doing
  things like 'foo in dirstate' now require some extra Python attribute lookups.
  This is causing a 100ms slow down in hg status for mozilla-central.
  
  The fix is to hoist the inner dict's functions onto the main class once the 
lazy
  loading it complete, as well as store the actual functions before doing the
  status loop (as is done for other such functions).
  
  In my testing, it seems to address the performance regression, but we'll
  need to see the perf run results to know for sure.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1257

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -1053,6 +1053,9 @@
 removed, deleted, clean = [], [], []
 
 dmap = self._map
+dmap.preload()
+dcontains = dmap.__contains__
+dget = dmap.__getitem__
 ladd = lookup.append# aka "unsure"
 madd = modified.append
 aadd = added.append
@@ -1074,7 +1077,7 @@
 full = listclean or match.traversedir is not None
 for fn, st in self.walk(match, subrepos, listunknown, listignored,
 full=full).iteritems():
-if fn not in dmap:
+if not dcontains(fn):
 if (listignored or mexact(fn)) and dirignore(fn):
 if listignored:
 iadd(fn)
@@ -1089,7 +1092,7 @@
 # a list, but falls back to creating a full-fledged iterator in
 # general. That is much slower than simply accessing and storing 
the
 # tuple members one by one.
-t = dmap[fn]
+t = dget(fn)
 state = t[0]
 mode = t[1]
 size = t[2]
@@ -1247,6 +1250,10 @@
 def keys(self):
 return self._map.keys()
 
+def preload(self):
+"""Loads the underlying data, if it's not already loaded"""
+self._map
+
 def nonnormalentries(self):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
@@ -1373,6 +1380,13 @@
 if not self._dirtyparents:
 self.setparents(*p)
 
+# Avoid excess attribute lookups by fast pathing certain checks
+self.__contains__ = self._map.__contains__
+self.__getitem__ = self._map.__getitem__
+self.__setitem__ = self._map.__setitem__
+self.__delitem__ = self._map.__delitem__
+self.get = self._map.get
+
 def write(self, st, now):
 st.write(parsers.pack_dirstate(self._map, self.copymap,
self.parents(), now))



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1144: directaccess: add support to export and tests to demonstrate things

2017-10-27 Thread durham (Durham Goode) 
durham added a comment.


  @lothiraldan
  Are pinned revs deleted when the cache is rebuilt?  I would kinda not expect 
them to be.  If they are deleted, then yea moving these exceptions to another 
attribute that is persisted across cache clears makes sense.
  
  For your concern about changing from a command-level option to a scope and 
revrange level option, I have a couple comments:
  
  - If we went the current command-level flag, the examples your talking about 
where it's initially ambiguous if the command is actually reading or writing 
the hidden commit (phase, fold --reuse-msg, etc) would default to warn mode and 
the only negative effect would be the user gets an extra warning message 
telling them a hash they passed is hidden. Probably not a common occurrence, 
and not a huge issue.  In the future we could add more logic within the command 
to suppress that warning when doing the read only paths, but for the short term 
this let's us get directaccess out the door without auditing and updating every 
commands logic.
  - As for adding exception context manager type things, I think one of the 
issues with our current visibility code is that it requires the person writing 
a command to be too aware that visibility exists.  If we start requiring 
commands to use this new context appropriately, and to pass the right argument 
to revrange at the right time, I think we just introduce more risk of commands 
being written incorrectly and having more complexity at the business logic 
layer.  If we can avoid all of that at the cost of printing warnings to the 
user slightly more often, I think that's a good trade off.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1144

To: pulkit, #hg-reviewers, lothiraldan
Cc: durham, dlax, lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1201: dirstate: clean up when restoring identical backups

2017-10-27 Thread durham (Durham Goode) 
durham accepted this revision.
durham added a comment.


  lgtm.  Is this a recent regression?  Seems like this would be polluting lots 
of peoples repositories.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1201

To: mbthomas, #hg-reviewers, krbullock, durham
Cc: krbullock, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1253: dirstate: avoid reading the map when possible (issue5713) (issue5717)

2017-10-27 Thread durham (Durham Goode) 
durham added a comment.


  @mharbison72 your test output looks like the sort order changed?  Does it 
consistently repro before and after this patch?  The sort order is from the 
sort -u in the test, so I can't imagine this affecting it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1253

To: durham, #hg-reviewers, lothiraldan
Cc: yuja, lothiraldan, mharbison72, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1253: dirstate: avoid reading the map when possible (issue5713) (issue5717)

2017-10-26 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Before the recent refactor, we would not load the entire map until it was
  accessed. As part of the refactor, that got lost and even just trying to load
  the dirstate parents would load the whole map. This caused a perf regression
  (issue5713) and a regression with static http serving (issue5717).
  
  Making it lazy loaded again fixes both.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1253

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -129,7 +129,7 @@
 def _map(self):
 '''Return the dirstate contents as a map from filename to
 (state, mode, size, time).'''
-self._read()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 return self._map
 
 @property
@@ -353,10 +353,6 @@
 f.discard()
 raise
 
-def _read(self):
-self._map = dirstatemap(self._ui, self._opener, self._root)
-self._map.read()
-
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
 
@@ -1201,14 +1197,24 @@
 self._root = root
 self._filename = 'dirstate'
 
-self._map = {}
-self.copymap = {}
 self._parents = None
 self._dirtyparents = False
 
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
 
+@propertycache
+def _map(self):
+self._map = {}
+self.read()
+return self._map
+
+@propertycache
+def copymap(self):
+self.copymap = {}
+self._map
+return self.copymap
+
 def clear(self):
 self._map = {}
 self.copymap = {}
@@ -1388,7 +1394,7 @@
 
 @propertycache
 def identity(self):
-self.read()
+self._map
 return self.identity
 
 @propertycache



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1252: dirstate: move clear onto dirstatemap class

2017-10-26 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  A future diff will move the lazy loading aspect of dirstate to the dirstatemap
  class. This means it requires a slightly different strategy of clearing than
  just reinstantiating the object (since just reinstantiating the object will
  lazily load the on disk data again later instead of remaining permanently
  empty).
  
  So let's give it it's own clear function.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1252

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -593,8 +593,7 @@
 return path
 
 def clear(self):
-self._map = dirstatemap(self._ui, self._opener, self._root)
-self._map.setparents(nullid, nullid)
+self._map.clear()
 self._lastnormaltime = 0
 self._updatedfiles.clear()
 self._dirty = True
@@ -1210,6 +1209,11 @@
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
 
+def clear(self):
+self._map = {}
+self.copymap = {}
+self.setparents(nullid, nullid)
+
 def iteritems(self):
 return self._map.iteritems()
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D983: dirstate: move the _dirfoldmap to dirstatemap

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe8a89ed7ce96: dirstate: move the _dirfoldmap to dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D983?vs=2513=2713

REVISION DETAIL
  https://phab.mercurial-scm.org/D983

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -132,14 +132,6 @@
 self._read()
 return self._map
 
-@propertycache
-def _dirfoldmap(self):
-f = {}
-normcase = util.normcase
-for name in self._map.dirs:
-f[normcase(name)] = name
-return f
-
 @property
 def _sparsematcher(self):
 """The matcher for the sparse checkout.
@@ -372,8 +364,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_dirfoldmap", "_branch",
-  "_ignore"):
+for a in ("_map", "_branch", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -568,15 +559,15 @@
 normed = util.normcase(path)
 folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
-folded = self._dirfoldmap.get(normed, None)
+folded = self._map.dirfoldmap.get(normed, None)
 if folded is None:
 if isknown:
 folded = path
 else:
 # store discovered result in dirfoldmap so that future
 # normalizefile calls don't start matching directories
 folded = self._discoverpath(path, normed, ignoremissing, 
exists,
-self._dirfoldmap)
+self._map.dirfoldmap)
 return folded
 
 def normalize(self, path, isknown=False, ignoremissing=False):
@@ -875,7 +866,7 @@
 if len(paths) > 1:
 for path in paths:
 folded = self._discoverpath(path, norm, True, None,
-self._dirfoldmap)
+self._map.dirfoldmap)
 if path != folded:
 results[path] = None
 
@@ -1396,3 +1387,10 @@
 self.read()
 return self.identity
 
+@propertycache
+def dirfoldmap(self):
+f = {}
+normcase = util.normcase
+for name in self.dirs:
+f[normcase(name)] = name
+return f
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -560,8 +560,8 @@
 dirstate = repo.dirstate
 'a' in dirstate
 def d():
-dirstate._dirfoldmap.get('a')
-del dirstate._dirfoldmap
+dirstate._map.dirfoldmap.get('a')
+del dirstate._map.dirfoldmap
 del dirstate._map.dirs
 timer(d)
 fm.end()



To: durham, #hg-reviewers, quark, durin42
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D981: dirstate: remove _filefoldmap property cache

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGbfddc3d678ae: dirstate: remove _filefoldmap property cache 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D981?vs=2511=2710

REVISION DETAIL
  https://phab.mercurial-scm.org/D981

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -133,10 +133,6 @@
 return self._map
 
 @propertycache
-def _filefoldmap(self):
-return self._map.filefoldmap()
-
-@propertycache
 def _dirfoldmap(self):
 f = {}
 normcase = util.normcase
@@ -380,7 +376,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_filefoldmap", "_dirfoldmap", "_branch",
+for a in ("_map", "_dirfoldmap", "_branch",
   "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
@@ -412,10 +408,10 @@
 if self[f] not in "?r" and "_dirs" in self.__dict__:
 self._dirs.delpath(f)
 
-if "_filefoldmap" in self.__dict__:
+if "filefoldmap" in self._map.__dict__:
 normed = util.normcase(f)
-if normed in self._filefoldmap:
-del self._filefoldmap[normed]
+if normed in self._map.filefoldmap:
+del self._map.filefoldmap[normed]
 
 self._updatedfiles.add(f)
 
@@ -563,18 +559,18 @@
 
 def _normalizefile(self, path, isknown, ignoremissing=False, exists=None):
 normed = util.normcase(path)
-folded = self._filefoldmap.get(normed, None)
+folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
 if isknown:
 folded = path
 else:
 folded = self._discoverpath(path, normed, ignoremissing, 
exists,
-self._filefoldmap)
+self._map.filefoldmap)
 return folded
 
 def _normalize(self, path, isknown, ignoremissing=False, exists=None):
 normed = util.normcase(path)
-folded = self._filefoldmap.get(normed, None)
+folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
 folded = self._dirfoldmap.get(normed, None)
 if folded is None:
@@ -1270,6 +1266,7 @@
 otherparent.add(fname)
 return nonnorm, otherparent
 
+@propertycache
 def filefoldmap(self):
 """Returns a dictionary mapping normalized case paths to their
 non-normalized versions.
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -549,8 +549,8 @@
 dirstate = repo.dirstate
 'a' in dirstate
 def d():
-dirstate._filefoldmap.get('a')
-del dirstate._filefoldmap
+dirstate._map.filefoldmap.get('a')
+del dirstate._map.filefoldmap
 timer(d)
 fm.end()
 



To: durham, #hg-reviewers, quark, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D982: dirstate: remove _dirs property cache

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG014bd2a555c8: dirstate: remove _dirs property cache 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D982?vs=2512=2711

REVISION DETAIL
  https://phab.mercurial-scm.org/D982

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -136,7 +136,7 @@
 def _dirfoldmap(self):
 f = {}
 normcase = util.normcase
-for name in self._dirs:
+for name in self._map.dirs:
 f[normcase(name)] = name
 return f
 
@@ -166,12 +166,8 @@
 def _pl(self):
 return self._map.parents()
 
-@propertycache
-def _dirs(self):
-return self._map.dirs()
-
 def dirs(self):
-return self._dirs
+return self._map.dirs
 
 @rootcache('.hgignore')
 def _ignore(self):
@@ -377,7 +373,7 @@
 check whether the dirstate has changed before rereading it.'''
 
 for a in ("_map", "_dirfoldmap", "_branch",
-  "_dirs", "_ignore"):
+  "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -405,8 +401,8 @@
 return self._map.copymap
 
 def _droppath(self, f):
-if self[f] not in "?r" and "_dirs" in self.__dict__:
-self._dirs.delpath(f)
+if self[f] not in "?r" and "dirs" in self._map.__dict__:
+self._map.dirs.delpath(f)
 
 if "filefoldmap" in self._map.__dict__:
 normed = util.normcase(f)
@@ -419,18 +415,18 @@
 oldstate = self[f]
 if state == 'a' or oldstate == 'r':
 scmutil.checkfilename(f)
-if f in self._dirs:
+if f in self._map.dirs:
 raise error.Abort(_('directory %r already in dirstate') % f)
 # shadows
 for d in util.finddirs(f):
-if d in self._dirs:
+if d in self._map.dirs:
 break
 entry = self._map.get(d)
 if entry is not None and entry[0] != 'r':
 raise error.Abort(
 _('file %r in dirstate clashes with %r') % (d, f))
-if oldstate in "?r" and "_dirs" in self.__dict__:
-self._dirs.addpath(f)
+if oldstate in "?r" and "dirs" in self._map.__dict__:
+self._map.dirs.addpath(f)
 self._dirty = True
 self._updatedfiles.add(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
@@ -607,8 +603,6 @@
 
 def clear(self):
 self._map = dirstatemap(self._ui, self._opener, self._root)
-if "_dirs" in self.__dict__:
-delattr(self, "_dirs")
 self._map.setparents(nullid, nullid)
 self._lastnormaltime = 0
 self._updatedfiles.clear()
@@ -1287,6 +1281,7 @@
 f['.'] = '.' # prevents useless util.fspath() invocation
 return f
 
+@propertycache
 def dirs(self):
 """Returns a set-like object containing all the directories in the
 current dirstate.
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -519,7 +519,7 @@
 'a' in dirstate
 def d():
 dirstate.dirs()
-del dirstate._dirs
+del dirstate._map.dirs
 timer(d)
 fm.end()
 
@@ -538,8 +538,8 @@
 timer, fm = gettimer(ui, opts)
 "a" in repo.dirstate
 def d():
-"a" in repo.dirstate._dirs
-del repo.dirstate._dirs
+"a" in repo.dirstate._map.dirs
+del repo.dirstate._map.dirs
 timer(d)
 fm.end()
 
@@ -562,7 +562,7 @@
 def d():
 dirstate._dirfoldmap.get('a')
 del dirstate._dirfoldmap
-del dirstate._dirs
+del dirstate._map.dirs
 timer(d)
 fm.end()
 



To: durham, #hg-reviewers, quark, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D979: dirstate: move nonnormal and otherparent sets to dirstatemap

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG60927b19ed65: dirstate: move nonnormal and otherparent sets 
to dirstatemap (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D979?vs=2509=2708

REVISION DETAIL
  https://phab.mercurial-scm.org/D979

AFFECTED FILES
  contrib/dirstatenonnormalcheck.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -138,18 +138,6 @@
 return self._identity
 
 @propertycache
-def _nonnormalset(self):
-nonnorm, otherparents = self._map.nonnormalentries()
-self._otherparentset = otherparents
-return nonnorm
-
-@propertycache
-def _otherparentset(self):
-nonnorm, otherparents = self._map.nonnormalentries()
-self._nonnormalset = nonnorm
-return otherparents
-
-@propertycache
 def _filefoldmap(self):
 return self._map.filefoldmap()
 
@@ -349,7 +337,8 @@
 self._map.setparents(p1, p2)
 copies = {}
 if oldp2 != nullid and p2 == nullid:
-candidatefiles = self._nonnormalset.union(self._otherparentset)
+candidatefiles = self._map.nonnormalset.union(
+self._map.otherparentset)
 for f in candidatefiles:
 s = self._map.get(f)
 if s is None:
@@ -401,8 +390,7 @@
 
 for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
-  "_dirs", "_ignore", "_nonnormalset",
-  "_otherparentset"):
+  "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -460,19 +448,19 @@
 self._updatedfiles.add(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
 if state != 'n' or mtime == -1:
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 if size == -2:
-self._otherparentset.add(f)
+self._map.otherparentset.add(f)
 
 def normal(self, f):
 '''Mark a file normal and clean.'''
 s = os.lstat(self._join(f))
 mtime = s.st_mtime
 self._addpath(f, 'n', s.st_mode,
   s.st_size & _rangemask, mtime & _rangemask)
 self._map.copymap.pop(f, None)
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 if mtime > self._lastnormaltime:
 # Remember the most recent modification timeslot for status(),
 # to make sure we won't miss future size-preserving file content
@@ -500,8 +488,8 @@
 return
 self._addpath(f, 'n', 0, -1, -1)
 self._map.copymap.pop(f, None)
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 
 def otherparent(self, f):
 '''Mark as coming from the other parent, always dirty.'''
@@ -534,9 +522,9 @@
 size = -1
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
-self._otherparentset.add(f)
+self._map.otherparentset.add(f)
 self._map[f] = dirstatetuple('r', 0, size, 0)
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 if size == 0:
 self._map.copymap.pop(f, None)
 
@@ -552,8 +540,8 @@
 self._dirty = True
 self._droppath(f)
 del self._map[f]
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 self._map.copymap.pop(f, None)
 
 def _discoverpath(self, path, normed, ignoremissing, exists, storemap):
@@ -632,8 +620,6 @@
 
 def clear(self):
 self._map = dirstatemap(self._ui, self._opener, self._root)
-self._nonnormalset = set()
-self._otherparentset = set()
 if "_dirs" in self.__dict__:
 delattr(self, "_dirs")
 self._map.setparents(nullid, nullid)
@@ -687,7 +673,7 @@
 e = dmap.get(f)
 if e is not None and e[0] == 'n' and e[3] == now:
 dmap[f] = dirstatetuple(e[0], e[1], e[2], -1)
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 
 # emulate that all 'dirstate.normal' results are written out
 self._lastnormaltime = 0
@@ -740,7 +726,6 @@
 break
 
 self._map.write(st, now)
-self._nonnormalset, self._otherparentset =

D978: dirstate: move write into dirstatemap

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe2214632c3a2: dirstate: move write into dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D978?vs=2508=2707

REVISION DETAIL
  https://phab.mercurial-scm.org/D978

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -739,12 +739,10 @@
 now = end # trust our estimate that the end is near now
 break
 
-st.write(parsers.pack_dirstate(self._map._map, self._map.copymap,
-   self._pl, now))
+self._map.write(st, now)
 self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
-st.close()
 self._lastnormaltime = 0
-self._dirty = self._map._dirtyparents = False
+self._dirty = False
 
 def _dirignore(self, f):
 if f == '.':
@@ -1401,3 +1399,9 @@
 p = parse_dirstate(self._map, self.copymap, st)
 if not self._dirtyparents:
 self.setparents(*p)
+
+def write(self, st, now):
+st.write(parsers.pack_dirstate(self._map, self.copymap,
+   self.parents(), now))
+st.close()
+self._dirtyparents = False



To: durham, #hg-reviewers, quark, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D977: dirstate: move _read into dirstatemap

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe159f217230e: dirstate: move _read into dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D977?vs=2507=2706

REVISION DETAIL
  https://phab.mercurial-scm.org/D977

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -386,53 +386,11 @@
 raise
 
 def _read(self):
-self._map = dirstatemap(self._ui, self._opener, self._root)
-
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
-try:
-fp = self._map._opendirstatefile()
-try:
-st = fp.read()
-finally:
-fp.close()
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return
-if not st:
-return
-
-if util.safehasattr(parsers, 'dict_new_presized'):
-# Make an estimate of the number of files in the dirstate based on
-# its size. From a linear regression on a set of real-world repos,
-# all over 10,000 files, the size of a dirstate entry is 85
-# bytes. The cost of resizing is significantly higher than the cost
-# of filling in a larger presized dict, so subtract 20% from the
-# size.
-#
-# This heuristic is imperfect in many ways, so in a future dirstate
-# format update it makes sense to just record the number of entries
-# on write.
-self._map._map = parsers.dict_new_presized(len(st) / 71)
-
-# Python's garbage collector triggers a GC each time a certain number
-# of container objects (the number being defined by
-# gc.get_threshold()) are allocated. parse_dirstate creates a tuple
-# for each file in the dirstate. The C version then immediately marks
-# them as not to be tracked by the collector. However, this has no
-# effect on when GCs are triggered, only on what objects the GC looks
-# into. This means that O(number of files) GCs are unavoidable.
-# Depending on when in the process's lifetime the dirstate is parsed,
-# this can get very expensive. As a workaround, disable GC while
-# parsing the dirstate.
-#
-# (we cannot decorate the function directly since it is in a C module)
-parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map._map, self._map.copymap, st)
-if not self._map._dirtyparents:
-self._map.setparents(*p)
+self._map = dirstatemap(self._ui, self._opener, self._root)
+self._map.read()
 
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
@@ -1399,3 +1357,47 @@
 def setparents(self, p1, p2):
 self._parents = (p1, p2)
 self._dirtyparents = True
+
+def read(self):
+try:
+fp = self._opendirstatefile()
+try:
+st = fp.read()
+finally:
+fp.close()
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+return
+if not st:
+return
+
+if util.safehasattr(parsers, 'dict_new_presized'):
+# Make an estimate of the number of files in the dirstate based on
+# its size. From a linear regression on a set of real-world repos,
+# all over 10,000 files, the size of a dirstate entry is 85
+# bytes. The cost of resizing is significantly higher than the cost
+# of filling in a larger presized dict, so subtract 20% from the
+# size.
+#
+# This heuristic is imperfect in many ways, so in a future dirstate
+# format update it makes sense to just record the number of entries
+# on write.
+self._map = parsers.dict_new_presized(len(st) / 71)
+
+# Python's garbage collector triggers a GC each time a certain number
+# of container objects (the number being defined by
+# gc.get_threshold()) are allocated. parse_dirstate creates a tuple
+# for each file in the dirstate. The C version then immediately marks
+# them as not to be tracked by the collector. However, this has no
+# effect on when GCs are triggered, only on what objects the GC looks
+# into. This means that O(number of files) GCs are unavoidable.
+# Depending on when in the process's lifetime the dirstate is parsed,
+# this can get very expensive. As a workaround, disable GC while
+# parsing the

D980: dirstate: move identity to dirstatemap

2017-10-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc6ef9a2498a5: dirstate: move identity to dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D980?vs=2510=2709

REVISION DETAIL
  https://phab.mercurial-scm.org/D980

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -133,11 +133,6 @@
 return self._map
 
 @propertycache
-def _identity(self):
-self._read()
-return self._identity
-
-@propertycache
 def _filefoldmap(self):
 return self._map.filefoldmap()
 
@@ -375,9 +370,6 @@
 raise
 
 def _read(self):
-# ignore HG_PENDING because identity is used only for writing
-self._identity = util.filestat.frompath(
-self._opener.join(self._filename))
 self._map = dirstatemap(self._ui, self._opener, self._root)
 self._map.read()
 
@@ -388,8 +380,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_identity",
-  "_filefoldmap", "_dirfoldmap", "_branch",
+for a in ("_map", "_filefoldmap", "_dirfoldmap", "_branch",
   "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
@@ -652,7 +643,7 @@
 If identity of previous dirstate is equal to this, writing
 changes based on the former dirstate out can keep consistency.
 '''
-return self._identity
+return self._map.identity
 
 def write(self, tr):
 if not self._dirty:
@@ -1342,6 +1333,10 @@
 self._dirtyparents = True
 
 def read(self):
+# ignore HG_PENDING because identity is used only for writing
+self.identity = util.filestat.frompath(
+self._opener.join(self._filename))
+
 try:
 fp = self._opendirstatefile()
 try:
@@ -1404,3 +1399,8 @@
 self.nonnormalset = nonnorm
 return otherparents
 
+@propertycache
+def identity(self):
+self.read()
+return self.identity
+



To: durham, #hg-reviewers, quark, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D960: bundle2: immediate exit for ctrl+c (issue5692)

2017-10-11 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> yuja wrote in bundle2.py:380
> SignalInterrupt is a subclass of KeyboardInterrupt, so there might
> be deeper problem.
> 
> FWIW, I think it's probably a good idea to replace this blacklist
> with `isinstance(exc, Exception)`.

Before my refactor there used to be two levels of error handling, 1) inside 
_parthandler which tried to seek to the end if it wasn't systemexit or 
keyboardinterrup, and 2) inside processbundle (which calls _parthandler) which 
only caught type Exception.

It seems like the correct unification of these would be to just keep the 
`isinstance(exc, Exception)` check above, and delete this sub-check entirely.  
I'll send an update.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D960

To: durham, #hg-reviewers, quark
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D945: fsmonitor: update to match new dirstate refactor

2017-10-11 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG7259f0ddfc0f: fsmonitor: update to match new dirstate 
refactor (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D945?vs=2536=2583

REVISION DETAIL
  https://phab.mercurial-scm.org/D945

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -251,7 +251,7 @@
 
 matchfn = match.matchfn
 matchalways = match.always()
-dmap = self._map
+dmap = self._map._map
 nonnormalset = getattr(self, '_nonnormalset', None)
 
 copymap = self._map.copymap



To: durham, #hg-reviewers, quark, yuja, ryanmce
Cc: ryanmce, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D981: dirstate: remove _filefoldmap property cache

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that the filefoldmap is source of truthed on the dirstatemap, let's get 
rid
  of the property cache on the dirstate.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D981

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -133,10 +133,6 @@
 return self._map
 
 @propertycache
-def _filefoldmap(self):
-return self._map.filefoldmap()
-
-@propertycache
 def _dirfoldmap(self):
 f = {}
 normcase = util.normcase
@@ -380,7 +376,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_filefoldmap", "_dirfoldmap", "_branch",
+for a in ("_map", "_dirfoldmap", "_branch",
   "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
@@ -412,10 +408,10 @@
 if self[f] not in "?r" and "_dirs" in self.__dict__:
 self._dirs.delpath(f)
 
-if "_filefoldmap" in self.__dict__:
+if "filefoldmap" in self._map.__dict__:
 normed = util.normcase(f)
-if normed in self._filefoldmap:
-del self._filefoldmap[normed]
+if normed in self._map.filefoldmap:
+del self._map.filefoldmap[normed]
 
 self._updatedfiles.add(f)
 
@@ -563,18 +559,18 @@
 
 def _normalizefile(self, path, isknown, ignoremissing=False, exists=None):
 normed = util.normcase(path)
-folded = self._filefoldmap.get(normed, None)
+folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
 if isknown:
 folded = path
 else:
 folded = self._discoverpath(path, normed, ignoremissing, 
exists,
-self._filefoldmap)
+self._map.filefoldmap)
 return folded
 
 def _normalize(self, path, isknown, ignoremissing=False, exists=None):
 normed = util.normcase(path)
-folded = self._filefoldmap.get(normed, None)
+folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
 folded = self._dirfoldmap.get(normed, None)
 if folded is None:
@@ -1270,6 +1266,7 @@
 otherparent.add(fname)
 return nonnorm, otherparent
 
+@propertycache
 def filefoldmap(self):
 """Returns a dictionary mapping normalized case paths to their
 non-normalized versions.
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -539,8 +539,8 @@
 dirstate = repo.dirstate
 'a' in dirstate
 def d():
-dirstate._filefoldmap.get('a')
-del dirstate._filefoldmap
+dirstate._map.filefoldmap.get('a')
+del dirstate._map.filefoldmap
 timer(d)
 fm.end()
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D982: dirstate: remove _dirs property cache

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that dirs is source of truthed on the dirstatemap, let's get rid of the
  _dirs propertycache on the dirstate.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D982

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -136,7 +136,7 @@
 def _dirfoldmap(self):
 f = {}
 normcase = util.normcase
-for name in self._dirs:
+for name in self._map.dirs:
 f[normcase(name)] = name
 return f
 
@@ -166,12 +166,8 @@
 def _pl(self):
 return self._map.parents()
 
-@propertycache
-def _dirs(self):
-return self._map.dirs()
-
 def dirs(self):
-return self._dirs
+return self._map.dirs
 
 @rootcache('.hgignore')
 def _ignore(self):
@@ -377,7 +373,7 @@
 check whether the dirstate has changed before rereading it.'''
 
 for a in ("_map", "_dirfoldmap", "_branch",
-  "_dirs", "_ignore"):
+  "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -405,8 +401,8 @@
 return self._map.copymap
 
 def _droppath(self, f):
-if self[f] not in "?r" and "_dirs" in self.__dict__:
-self._dirs.delpath(f)
+if self[f] not in "?r" and "dirs" in self._map.__dict__:
+self._map.dirs.delpath(f)
 
 if "filefoldmap" in self._map.__dict__:
 normed = util.normcase(f)
@@ -419,18 +415,18 @@
 oldstate = self[f]
 if state == 'a' or oldstate == 'r':
 scmutil.checkfilename(f)
-if f in self._dirs:
+if f in self._map.dirs:
 raise error.Abort(_('directory %r already in dirstate') % f)
 # shadows
 for d in util.finddirs(f):
-if d in self._dirs:
+if d in self._map.dirs:
 break
 entry = self._map.get(d)
 if entry is not None and entry[0] != 'r':
 raise error.Abort(
 _('file %r in dirstate clashes with %r') % (d, f))
-if oldstate in "?r" and "_dirs" in self.__dict__:
-self._dirs.addpath(f)
+if oldstate in "?r" and "dirs" in self._map.__dict__:
+self._map.dirs.addpath(f)
 self._dirty = True
 self._updatedfiles.add(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
@@ -607,8 +603,6 @@
 
 def clear(self):
 self._map = dirstatemap(self._ui, self._opener, self._root)
-if "_dirs" in self.__dict__:
-delattr(self, "_dirs")
 self._map.setparents(nullid, nullid)
 self._lastnormaltime = 0
 self._updatedfiles.clear()
@@ -1287,6 +1281,7 @@
 f['.'] = '.' # prevents useless util.fspath() invocation
 return f
 
+@propertycache
 def dirs(self):
 """Returns a set-like object containing all the directories in the
 current dirstate.
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -509,7 +509,7 @@
 'a' in dirstate
 def d():
 dirstate.dirs()
-del dirstate._dirs
+del dirstate._map.dirs
 timer(d)
 fm.end()
 
@@ -528,8 +528,8 @@
 timer, fm = gettimer(ui, opts)
 "a" in repo.dirstate
 def d():
-"a" in repo.dirstate._dirs
-del repo.dirstate._dirs
+"a" in repo.dirstate._map.dirs
+del repo.dirstate._map.dirs
 timer(d)
 fm.end()
 
@@ -552,7 +552,7 @@
 def d():
 dirstate._dirfoldmap.get('a')
 del dirstate._dirfoldmap
-del dirstate._dirs
+del dirstate._map.dirs
 timer(d)
 fm.end()
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D983: dirstate: move the _dirfoldmap to dirstatemap

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that dirstatemap is the source of truth for the list of directories, let's
  move _dirfoldmap on to it.
  
  This pattern of moving cached variables onto the dirstate map makes it easier 
to
  invalidate them, as seen by how the cache invalidation functions are slowly
  shrinking to just be recreating the dirstatemap instance.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D983

AFFECTED FILES
  contrib/perf.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -132,14 +132,6 @@
 self._read()
 return self._map
 
-@propertycache
-def _dirfoldmap(self):
-f = {}
-normcase = util.normcase
-for name in self._map.dirs:
-f[normcase(name)] = name
-return f
-
 @property
 def _sparsematcher(self):
 """The matcher for the sparse checkout.
@@ -372,8 +364,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_dirfoldmap", "_branch",
-  "_ignore"):
+for a in ("_map", "_branch", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -568,15 +559,15 @@
 normed = util.normcase(path)
 folded = self._map.filefoldmap.get(normed, None)
 if folded is None:
-folded = self._dirfoldmap.get(normed, None)
+folded = self._map.dirfoldmap.get(normed, None)
 if folded is None:
 if isknown:
 folded = path
 else:
 # store discovered result in dirfoldmap so that future
 # normalizefile calls don't start matching directories
 folded = self._discoverpath(path, normed, ignoremissing, 
exists,
-self._dirfoldmap)
+self._map.dirfoldmap)
 return folded
 
 def normalize(self, path, isknown=False, ignoremissing=False):
@@ -875,7 +866,7 @@
 if len(paths) > 1:
 for path in paths:
 folded = self._discoverpath(path, norm, True, None,
-self._dirfoldmap)
+self._map.dirfoldmap)
 if path != folded:
 results[path] = None
 
@@ -1396,3 +1387,10 @@
 self.read()
 return self.identity
 
+@propertycache
+def dirfoldmap(self):
+f = {}
+normcase = util.normcase
+for name in self.dirs:
+f[normcase(name)] = name
+return f
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -550,8 +550,8 @@
 dirstate = repo.dirstate
 'a' in dirstate
 def d():
-dirstate._dirfoldmap.get('a')
-del dirstate._dirfoldmap
+dirstate._map.dirfoldmap.get('a')
+del dirstate._map.dirfoldmap
 del dirstate._map.dirs
 timer(d)
 fm.end()



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D979: dirstate: move nonnormal and otherparent sets to dirstatemap

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of separating dirstate business logic from storage, let's move the
  nonnormal and otherparent storage to the dirstatemap class. This will allow
  alternative dirstate storage to persist these sets instead of recomputing 
them.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D979

AFFECTED FILES
  contrib/dirstatenonnormalcheck.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -138,18 +138,6 @@
 return self._identity
 
 @propertycache
-def _nonnormalset(self):
-nonnorm, otherparents = self._map.nonnormalentries()
-self._otherparentset = otherparents
-return nonnorm
-
-@propertycache
-def _otherparentset(self):
-nonnorm, otherparents = self._map.nonnormalentries()
-self._nonnormalset = nonnorm
-return otherparents
-
-@propertycache
 def _filefoldmap(self):
 return self._map.filefoldmap()
 
@@ -349,7 +337,8 @@
 self._map.setparents(p1, p2)
 copies = {}
 if oldp2 != nullid and p2 == nullid:
-candidatefiles = self._nonnormalset.union(self._otherparentset)
+candidatefiles = self._map.nonnormalset.union(
+self._map.otherparentset)
 for f in candidatefiles:
 s = self._map.get(f)
 if s is None:
@@ -401,8 +390,7 @@
 
 for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
-  "_dirs", "_ignore", "_nonnormalset",
-  "_otherparentset"):
+  "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
 self._lastnormaltime = 0
@@ -460,19 +448,19 @@
 self._updatedfiles.add(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
 if state != 'n' or mtime == -1:
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 if size == -2:
-self._otherparentset.add(f)
+self._map.otherparentset.add(f)
 
 def normal(self, f):
 '''Mark a file normal and clean.'''
 s = os.lstat(self._join(f))
 mtime = s.st_mtime
 self._addpath(f, 'n', s.st_mode,
   s.st_size & _rangemask, mtime & _rangemask)
 self._map.copymap.pop(f, None)
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 if mtime > self._lastnormaltime:
 # Remember the most recent modification timeslot for status(),
 # to make sure we won't miss future size-preserving file content
@@ -500,8 +488,8 @@
 return
 self._addpath(f, 'n', 0, -1, -1)
 self._map.copymap.pop(f, None)
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 
 def otherparent(self, f):
 '''Mark as coming from the other parent, always dirty.'''
@@ -534,9 +522,9 @@
 size = -1
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
-self._otherparentset.add(f)
+self._map.otherparentset.add(f)
 self._map[f] = dirstatetuple('r', 0, size, 0)
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 if size == 0:
 self._map.copymap.pop(f, None)
 
@@ -552,8 +540,8 @@
 self._dirty = True
 self._droppath(f)
 del self._map[f]
-if f in self._nonnormalset:
-self._nonnormalset.remove(f)
+if f in self._map.nonnormalset:
+self._map.nonnormalset.remove(f)
 self._map.copymap.pop(f, None)
 
 def _discoverpath(self, path, normed, ignoremissing, exists, storemap):
@@ -632,8 +620,6 @@
 
 def clear(self):
 self._map = dirstatemap(self._ui, self._opener, self._root)
-self._nonnormalset = set()
-self._otherparentset = set()
 if "_dirs" in self.__dict__:
 delattr(self, "_dirs")
 self._map.setparents(nullid, nullid)
@@ -687,7 +673,7 @@
 e = dmap.get(f)
 if e is not None and e[0] == 'n' and e[3] == now:
 dmap[f] = dirstatetuple(e[0], e[1], e[2], -1)
-self._nonnormalset.add(f)
+self._map.nonnormalset.add(f)
 
 # emulate that all 'dirstate.normal' results are written out
 self._lastnormaltime = 0
@@ -740,7 +726,6 @@
 break

D980: dirstate: move identity to dirstatemap

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Moving the identity function to the dirstatemap class will allow alternative
  dirstate implementations to replace the implementation.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D980

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -133,11 +133,6 @@
 return self._map
 
 @propertycache
-def _identity(self):
-self._read()
-return self._identity
-
-@propertycache
 def _filefoldmap(self):
 return self._map.filefoldmap()
 
@@ -375,9 +370,6 @@
 raise
 
 def _read(self):
-# ignore HG_PENDING because identity is used only for writing
-self._identity = util.filestat.frompath(
-self._opener.join(self._filename))
 self._map = dirstatemap(self._ui, self._opener, self._root)
 self._map.read()
 
@@ -388,8 +380,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_identity",
-  "_filefoldmap", "_dirfoldmap", "_branch",
+for a in ("_map", "_filefoldmap", "_dirfoldmap", "_branch",
   "_dirs", "_ignore"):
 if a in self.__dict__:
 delattr(self, a)
@@ -652,7 +643,7 @@
 If identity of previous dirstate is equal to this, writing
 changes based on the former dirstate out can keep consistency.
 '''
-return self._identity
+return self._map.identity
 
 def write(self, tr):
 if not self._dirty:
@@ -1342,6 +1333,10 @@
 self._dirtyparents = True
 
 def read(self):
+# ignore HG_PENDING because identity is used only for writing
+self.identity = util.filestat.frompath(
+self._opener.join(self._filename))
+
 try:
 fp = self._opendirstatefile()
 try:
@@ -1404,3 +1399,8 @@
 self.nonnormalset = nonnorm
 return otherparents
 
+@propertycache
+def identity(self):
+self.read()
+return self.identity
+



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D978: dirstate: move write into dirstatemap

2017-10-06 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of separating the dirstate business logic from the dirstate storage
  logic, let's move the serialization code down into dirstatemap.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D978

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -739,12 +739,10 @@
 now = end # trust our estimate that the end is near now
 break
 
-st.write(parsers.pack_dirstate(self._map._map, self._map.copymap,
-   self._pl, now))
+self._map.write(st, now)
 self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
-st.close()
 self._lastnormaltime = 0
-self._dirty = self._map._dirtyparents = False
+self._dirty = False
 
 def _dirignore(self, f):
 if f == '.':
@@ -1401,3 +1399,9 @@
 p = parse_dirstate(self._map, self.copymap, st)
 if not self._dirtyparents:
 self.setparents(*p)
+
+def write(self, st, now):
+st.write(parsers.pack_dirstate(self._map, self.copymap,
+   self.parents(), now))
+st.close()
+self._dirtyparents = False



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D960: bundle2: immediate exit for ctrl+c (issue5692)

2017-10-05 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  https://phab.mercurial-scm.org/rHG21c2df59a1dad534bfac45acc0bbfb6cb2afe9b4 
regressed bundle2 by catching all exceptions and trying to handle
  them. The old behavior was to allow KeyboardInterrupts to throw and not have
  graceful cleanup, which allowed it to exit immediately. Let's go back to that
  behavior.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D960

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -370,7 +370,10 @@
 if not self.iterator:
 return
 
-if exc:
+# Only gracefully abort in a normal exception situation. User aborts
+# like Ctrl+C throw a KeyboardInterrupt which is not a base Exception,
+# and should not gracefully cleanup.
+if isinstance(exc, Exception):
 # If exiting or interrupted, do not attempt to seek the stream in
 # the finally block below. This makes abort faster.
 if (self.current and



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D945: fsmonitor: update to match new dirstate refactor

2017-10-04 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The dirstate was refactored so dirstate._map is now at dirstate._map._map. 
Same
  for _copymap, is not _map.copymap. It seems none of the mercurial tests cover
  this stuff, but it was caught by our Facebook extension tests.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D945

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -228,10 +228,10 @@
 
 matchfn = match.matchfn
 matchalways = match.always()
-dmap = self._map
+dmap = self._map._map
 nonnormalset = getattr(self, '_nonnormalset', None)
 
-copymap = self._copymap
+copymap = self._map.copymap
 getkind = stat.S_IFMT
 dirkind = stat.S_IFDIR
 regkind = stat.S_IFREG



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D759: dirstate: move parents source of truth to dirstatemap

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5fabba9b3d9c: dirstate: move parents source of truth to 
dirstatemap (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D759?vs=2083=2162

REVISION DETAIL
  https://phab.mercurial-scm.org/D759

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -71,7 +71,6 @@
 # UNC path pointing to root share (issue4557)
 self._rootdir = pathutil.normasprefix(root)
 self._dirty = False
-self._dirtypl = False
 self._lastnormaltime = 0
 self._ui = ui
 self._filecache = {}
@@ -184,7 +183,7 @@
 raise
 return "default"
 
-@propertycache
+@property
 def _pl(self):
 return self._map.parents()
 
@@ -343,11 +342,11 @@
 raise ValueError("cannot set dirstate parent without "
  "calling dirstate.beginparentchange")
 
-self._dirty = self._dirtypl = True
+self._dirty = True
 oldp2 = self._pl[1]
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = p1, p2
+self._map.setparents(p1, p2)
 copies = {}
 if oldp2 != nullid and p2 == nullid:
 candidatefiles = self._nonnormalset.union(self._otherparentset)
@@ -432,8 +431,8 @@
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
 p = parse_dirstate(self._map._map, self._map.copymap, st)
-if not self._dirtypl:
-self._pl = p
+if not self._map._dirtyparents:
+self._map.setparents(*p)
 
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
@@ -444,7 +443,7 @@
 
 for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
-  "_pl", "_dirs", "_ignore", "_nonnormalset",
+  "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
 if a in self.__dict__:
 delattr(self, a)
@@ -679,7 +678,7 @@
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
 delattr(self, "_dirs")
-self._pl = [nullid, nullid]
+self._map.setparents(nullid, nullid)
 self._lastnormaltime = 0
 self._updatedfiles.clear()
 self._dirty = True
@@ -694,7 +693,7 @@
 
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = (parent, nullid)
+self._map.setparents(parent, nullid)
 for f in changedfiles:
 if f in allfiles:
 self.normallookup(f)
@@ -787,7 +786,7 @@
 self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
 st.close()
 self._lastnormaltime = 0
-self._dirty = self._dirtypl = False
+self._dirty = self._map._dirtyparents = False
 
 def _dirignore(self, f):
 if f == '.':
@@ -1292,6 +1291,8 @@
 
 self._map = {}
 self.copymap = {}
+self._parents = None
+self._dirtyparents = False
 
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
@@ -1370,16 +1371,28 @@
 return fp
 
 def parents(self):
-try:
-fp = self._opendirstatefile()
-st = fp.read(40)
-fp.close()
+if not self._parents:
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+# File doesn't exist, so the current state is empty
+st = ''
+
 l = len(st)
 if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+self._parents = st[:20], st[20:40]
+elif l == 0:
+self._parents = [nullid, nullid]
+else:
+raise error.Abort(_('working directory state appears '
+'damaged!'))
+
+return self._parents
+
+def setparents(self, p1, p2):
+self._parents = (p1, p2)
+self._dirtyparents = True



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D754: dirstate: move filefoldmap to dirstatemap

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG19a5ad535cb1: dirstate: move filefoldmap to dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D754?vs=2078=2157

REVISION DETAIL
  https://phab.mercurial-scm.org/D754

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -160,21 +160,7 @@
 
 @propertycache
 def _filefoldmap(self):
-try:
-makefilefoldmap = parsers.make_file_foldmap
-except AttributeError:
-pass
-else:
-return makefilefoldmap(self._map._map, util.normcasespec,
-   util.normcasefallback)
-
-f = {}
-normcase = util.normcase
-for name, s in self._map.iteritems():
-if s[0] != 'r':
-f[normcase(name)] = name
-f['.'] = '.' # prevents useless util.fspath() invocation
-return f
+return self._map.filefoldmap()
 
 @propertycache
 def _dirfoldmap(self):
@@ -1370,3 +1356,22 @@
 otherparent.add(fname)
 return nonnorm, otherparent
 
+def filefoldmap(self):
+"""Returns a dictionary mapping normalized case paths to their
+non-normalized versions.
+"""
+try:
+makefilefoldmap = parsers.make_file_foldmap
+except AttributeError:
+pass
+else:
+return makefilefoldmap(self._map, util.normcasespec,
+   util.normcasefallback)
+
+f = {}
+normcase = util.normcase
+for name, s in self._map.iteritems():
+if s[0] != 'r':
+f[normcase(name)] = name
+f['.'] = '.' # prevents useless util.fspath() invocation
+return f



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D758: dirstate: move parent reading to the dirstatemap class

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG949276a8cd76: dirstate: move parent reading to the 
dirstatemap class (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D758?vs=2082=2161

REVISION DETAIL
  https://phab.mercurial-scm.org/D758

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -186,19 +186,7 @@
 
 @propertycache
 def _pl(self):
-try:
-fp = self._map._opendirstatefile()
-st = fp.read(40)
-fp.close()
-l = len(st)
-if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+return self._map.parents()
 
 @propertycache
 def _dirs(self):
@@ -1381,3 +1369,17 @@
 self._pendingmode = mode
 return fp
 
+def parents(self):
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+l = len(st)
+if l == 40:
+return st[:20], st[20:40]
+elif l > 0 and l < 40:
+raise error.Abort(_('working directory state appears 
damaged!'))
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+return [nullid, nullid]



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D752: dirstate: create new dirstatemap class

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG42b68c9c2742: dirstate: create new dirstatemap class 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D752?vs=1947=2155

REVISION DETAIL
  https://phab.mercurial-scm.org/D752

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -57,7 +57,7 @@
 def nonnormalentries(dmap):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
-return parsers.nonnormalotherparententries(dmap)
+return parsers.nonnormalotherparententries(dmap._map)
 except AttributeError:
 nonnorm = set()
 otherparent = set()
@@ -179,7 +179,7 @@
 except AttributeError:
 pass
 else:
-return makefilefoldmap(self._map, util.normcasespec,
+return makefilefoldmap(self._map._map, util.normcasespec,
util.normcasefallback)
 
 f = {}
@@ -238,7 +238,7 @@
 
 @propertycache
 def _dirs(self):
-return util.dirs(self._map, 'r')
+return util.dirs(self._map._map, 'r')
 
 def dirs(self):
 return self._dirs
@@ -444,7 +444,8 @@
 return fp
 
 def _read(self):
-self._map = {}
+self._map = dirstatemap()
+
 self._copymap = {}
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
@@ -473,7 +474,7 @@
 # This heuristic is imperfect in many ways, so in a future dirstate
 # format update it makes sense to just record the number of entries
 # on write.
-self._map = parsers.dict_new_presized(len(st) / 71)
+self._map._map = parsers.dict_new_presized(len(st) / 71)
 
 # Python's garbage collector triggers a GC each time a certain number
 # of container objects (the number being defined by
@@ -488,7 +489,7 @@
 #
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map, self._copymap, st)
+p = parse_dirstate(self._map._map, self._copymap, st)
 if not self._dirtypl:
 self._pl = p
 
@@ -731,7 +732,7 @@
 return path
 
 def clear(self):
-self._map = {}
+self._map = dirstatemap()
 self._nonnormalset = set()
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
@@ -840,7 +841,8 @@
 now = end # trust our estimate that the end is near now
 break
 
-st.write(parsers.pack_dirstate(self._map, self._copymap, self._pl, 
now))
+st.write(parsers.pack_dirstate(self._map._map, self._copymap, self._pl,
+   now))
 self._nonnormalset, self._otherparentset = nonnormalentries(self._map)
 st.close()
 self._lastnormaltime = 0
@@ -979,7 +981,7 @@
 results[nf] = None
 else: # does it match a missing directory?
 if alldirs is None:
-alldirs = util.dirs(dmap)
+alldirs = util.dirs(dmap._map)
 if nf in alldirs:
 if matchedir:
 matchedir(nf)
@@ -1339,3 +1341,31 @@
 def clearbackup(self, tr, backupname):
 '''Clear backup file'''
 self._opener.unlink(backupname)
+
+class dirstatemap(object):
+def __init__(self):
+self._map = {}
+
+def iteritems(self):
+return self._map.iteritems()
+
+def __iter__(self):
+return iter(self._map)
+
+def get(self, key, default=None):
+return self._map.get(key, default)
+
+def __contains__(self, key):
+return key in self._map
+
+def __setitem__(self, key, value):
+self._map[key] = value
+
+def __getitem__(self, key):
+return self._map[key]
+
+def __delitem__(self, key):
+del self._map[key]
+
+def keys(self):
+return self._map.keys()



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D757: dirstate: move opendirstatefile to dirstatemap

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0407c611b7be: dirstate: move opendirstatefile to 
dirstatemap (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D757?vs=2081=2160

REVISION DETAIL
  https://phab.mercurial-scm.org/D757

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -82,9 +82,6 @@
 self._origpl = None
 self._updatedfiles = set()
 
-# for consistent view between _pl() and _read() invocations
-self._pendingmode = None
-
 @contextlib.contextmanager
 def parentchange(self):
 '''Context manager for handling dirstate parents.
@@ -190,7 +187,7 @@
 @propertycache
 def _pl(self):
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 st = fp.read(40)
 fp.close()
 l = len(st)
@@ -401,23 +398,14 @@
 f.discard()
 raise
 
-def _opendirstatefile(self):
-fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
-if self._pendingmode is not None and self._pendingmode != mode:
-fp.close()
-raise error.Abort(_('working directory state may be '
-'changed parallelly'))
-self._pendingmode = mode
-return fp
-
 def _read(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 try:
 st = fp.read()
 finally:
@@ -698,7 +686,7 @@
 return path
 
 def clear(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 self._nonnormalset = set()
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
@@ -1308,10 +1296,18 @@
 self._opener.unlink(backupname)
 
 class dirstatemap(object):
-def __init__(self):
+def __init__(self, ui, opener, root):
+self._ui = ui
+self._opener = opener
+self._root = root
+self._filename = 'dirstate'
+
 self._map = {}
 self.copymap = {}
 
+# for consistent view between _pl() and _read() invocations
+self._pendingmode = None
+
 def iteritems(self):
 return self._map.iteritems()
 
@@ -1375,3 +1371,13 @@
 current dirstate.
 """
 return util.dirs(self._map, 'r')
+
+def _opendirstatefile(self):
+fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
+if self._pendingmode is not None and self._pendingmode != mode:
+fp.close()
+raise error.Abort(_('working directory state may be '
+'changed parallelly'))
+self._pendingmode = mode
+return fp
+



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D756: dirstate: move _copymap to dirstatemap

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG894cd88815ca: dirstate: move _copymap to dirstatemap 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D756?vs=2080=2159

REVISION DETAIL
  https://phab.mercurial-scm.org/D756

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -137,11 +137,6 @@
 return self._map
 
 @propertycache
-def _copymap(self):
-self._read()
-return self._copymap
-
-@propertycache
 def _identity(self):
 self._read()
 return self._identity
@@ -378,13 +373,13 @@
 
 # Discard 'm' markers when moving away from a merge state
 if s[0] == 'm':
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.normallookup(f)
 # Also fix up otherparent markers
 elif s[0] == 'n' and s[2] == -2:
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.add(f)
@@ -418,7 +413,6 @@
 def _read(self):
 self._map = dirstatemap()
 
-self._copymap = {}
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
@@ -461,7 +455,7 @@
 #
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map._map, self._copymap, st)
+p = parse_dirstate(self._map._map, self._map.copymap, st)
 if not self._dirtypl:
 self._pl = p
 
@@ -472,7 +466,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_copymap", "_identity",
+for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
   "_pl", "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
@@ -490,17 +484,17 @@
 return
 self._dirty = True
 if source is not None:
-self._copymap[dest] = source
+self._map.copymap[dest] = source
 self._updatedfiles.add(source)
 self._updatedfiles.add(dest)
-elif self._copymap.pop(dest, None):
+elif self._map.copymap.pop(dest, None):
 self._updatedfiles.add(dest)
 
 def copied(self, file):
-return self._copymap.get(file, None)
+return self._map.copymap.get(file, None)
 
 def copies(self):
-return self._copymap
+return self._map.copymap
 
 def _droppath(self, f):
 if self[f] not in "?r" and "_dirs" in self.__dict__:
@@ -543,7 +537,7 @@
 mtime = s.st_mtime
 self._addpath(f, 'n', s.st_mode,
   s.st_size & _rangemask, mtime & _rangemask)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 if mtime > self._lastnormaltime:
@@ -561,7 +555,7 @@
 entry = self._map.get(f)
 if entry is not None:
 if entry[0] == 'r' and entry[2] in (-1, -2):
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if entry[2] == -1:
 self.merge(f)
 elif entry[2] == -2:
@@ -572,7 +566,7 @@
 if entry[0] == 'm' or entry[0] == 'n' and entry[2] == -2:
 return
 self._addpath(f, 'n', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 
@@ -587,12 +581,12 @@
 else:
 # add-like
 self._addpath(f, 'n', 0, -2, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def add(self, f):
 '''Mark a file added.'''
 self._addpath(f, 'a', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def remove(self, f):
 '''Mark a file removed.'''
@@ -611,7 +605,7 @@
 self._map[f] = dirstatetuple('r', 0, size, 0)
 self._nonnormalset.add(f)
 if size == 0:
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def merge(self, f):
 '''Mark a file merged.'''
@@ -627,7

D753: dirstate: move nonnormalentries to dirstatemap

2017-09-29 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG362ed91ca00c: dirstate: move nonnormalentries to 
dirstatemap (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D753?vs=1948=2156

REVISION DETAIL
  https://phab.mercurial-scm.org/D753

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -54,20 +54,6 @@
 os.close(tmpfd)
 vfs.unlink(tmpname)
 
-def nonnormalentries(dmap):
-'''Compute the nonnormal dirstate entries from the dmap'''
-try:
-return parsers.nonnormalotherparententries(dmap._map)
-except AttributeError:
-nonnorm = set()
-otherparent = set()
-for fname, e in dmap.iteritems():
-if e[0] != 'n' or e[3] == -1:
-nonnorm.add(fname)
-if e[0] == 'n' and e[2] == -2:
-otherparent.add(fname)
-return nonnorm, otherparent
-
 class dirstate(object):
 
 def __init__(self, opener, ui, root, validate, sparsematchfn):
@@ -162,13 +148,13 @@
 
 @propertycache
 def _nonnormalset(self):
-nonnorm, otherparents = nonnormalentries(self._map)
+nonnorm, otherparents = self._map.nonnormalentries()
 self._otherparentset = otherparents
 return nonnorm
 
 @propertycache
 def _otherparentset(self):
-nonnorm, otherparents = nonnormalentries(self._map)
+nonnorm, otherparents = self._map.nonnormalentries()
 self._nonnormalset = nonnorm
 return otherparents
 
@@ -843,7 +829,7 @@
 
 st.write(parsers.pack_dirstate(self._map._map, self._copymap, self._pl,
now))
-self._nonnormalset, self._otherparentset = nonnormalentries(self._map)
+self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
 st.close()
 self._lastnormaltime = 0
 self._dirty = self._dirtypl = False
@@ -1369,3 +1355,18 @@
 
 def keys(self):
 return self._map.keys()
+
+def nonnormalentries(self):
+'''Compute the nonnormal dirstate entries from the dmap'''
+try:
+return parsers.nonnormalotherparententries(self._map)
+except AttributeError:
+nonnorm = set()
+otherparent = set()
+for fname, e in self._map.iteritems():
+if e[0] != 'n' or e[3] == -1:
+nonnorm.add(fname)
+if e[0] == 'n' and e[2] == -2:
+otherparent.add(fname)
+return nonnorm, otherparent
+



To: durham, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D821: unamend: move fb extension unamend to core

2017-09-27 Thread durham (Durham Goode) 
durham requested changes to this revision.
durham added a comment.
This revision now requires changes to proceed.


  Generally looks good.  Just need to fix the transaction thing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D821

To: pulkit, #hg-reviewers, durham
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D821: unamend: move fb extension unamend to core

2017-09-27 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> uncommit.py:260
> +
> +tr = repo.transaction('unamend')
> +with dirstate.parentchange():

This should be in a with statement probably?  Can we just have it be as part of 
the top level lock with statement?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D821

To: pulkit, #hg-reviewers
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D738: directaccess: add support to export and tests to demonstrate things

2017-09-26 Thread durham (Durham Goode) 
durham accepted this revision.
durham added inline comments.

INLINE COMMENTS

> test-directaccess.t:11
> +  > [extensions]
> +  > uncommit =
> +  > amend =

Do we need uncommit? It doesn't seem used.

> test-directaccess.t:64
> +
> +  $ hg exp 7
> +  # HG changeset patch

I'm still not sure we want to support rev numbers.  I think we definitely don't 
want to support them for write commands, even recoverable ones.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D738

To: pulkit, #hg-reviewers, durham
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D737: directaccess: add support for accessing hidden commits if command is read only

2017-09-26 Thread durham (Durham Goode) 
durham requested changes to this revision.
durham added a comment.
This revision now requires changes to proceed.


  Probably needs a simple test, unless that comes later in the series.

INLINE COMMENTS

> dispatch.py:898
> +if cmdtype == readonly:
> +repo = repo.unfiltered()
>  args.insert(0, repo)

This seems overly broad.  'hg log' is a readonly command, and if I do 'hg log 
-r HIDDEN_HASH' I want to see the hidden commit, but if I do 'hg log' I don't 
want to see hidden commits.  I think this change would cause hg log to show all 
hidden commits right?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D737

To: pulkit, #hg-reviewers, durham
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D736: directaccess: add support for storing the type of command in func object

2017-09-26 Thread durham (Durham Goode) 
durham accepted this revision.
durham added a comment.


  Minor comments, but lgtm

INLINE COMMENTS

> registrar.py:143
> +unrecoverablewrite = "unrecoverable"
> +recoverablewrite = "recoverable"
> +readonly = "readonly"

Might be worth a comment describing what these mean, since it's not obvious 
without context.

> registrar.py:150
> +
> +possiblecmdtypes = set(["unrecoverable", "recoverable", "readonly"])
> +if cmdtype not in possiblecmdtypes:

These should probably refer to the class constants instead of hard coding 
duplicate strings.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D736

To: pulkit, #hg-reviewers, durham
Cc: durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D728: rebase: move working parent movement logic to scmutil.cleanupnodes

2017-09-26 Thread durham (Durham Goode) 
durham requested changes to this revision.
durham added inline comments.
This revision now requires changes to proceed.

INLINE COMMENTS

> scmutil.py:603
> +changes will be discarded, the caller is responsible for checking the
> +cleanness of working copy.
>  """

This seems like a bit of a bizarre feature for an api about cleaning up nodes.  
Maybe cleanup nodes should return a mapping of before-to-after and the caller 
can use that to perform an update if they need to?  I'd rather have APIs that 
can be composed, than add new permutations to existing APIs.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D728

To: quark, #hg-reviewers, durham
Cc: durham, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D758: dirstate: move parent reading to the dirstatemap class

2017-09-26 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> indygreg wrote in dirstate.py:1370
> Nit: this should ideally be in a context manager or try..finally block. But 
> it existed this way before, so not worth blocking over.

Left this alone for now to avoid code churn during the refactor.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D758

To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D757: dirstate: move opendirstatefile to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham marked an inline comment as done.
durham added inline comments.

INLINE COMMENTS

> indygreg wrote in dirstate.py:1373-1374
> This (pre-existing) error message is pretty bad because typical end users 
> won't have a clue what it means or what to do if they see it.

Agreed.  I'm not even sure what it's actually saying as a mercurial developer...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D757

To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D756: dirstate: move _copymap to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2080.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D756?vs=1951=2080

REVISION DETAIL
  https://phab.mercurial-scm.org/D756

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -137,11 +137,6 @@
 return self._map
 
 @propertycache
-def _copymap(self):
-self._read()
-return self._copymap
-
-@propertycache
 def _identity(self):
 self._read()
 return self._identity
@@ -378,13 +373,13 @@
 
 # Discard 'm' markers when moving away from a merge state
 if s[0] == 'm':
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.normallookup(f)
 # Also fix up otherparent markers
 elif s[0] == 'n' and s[2] == -2:
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.add(f)
@@ -418,7 +413,6 @@
 def _read(self):
 self._map = dirstatemap()
 
-self._copymap = {}
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
@@ -461,7 +455,7 @@
 #
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map._map, self._copymap, st)
+p = parse_dirstate(self._map._map, self._map.copymap, st)
 if not self._dirtypl:
 self._pl = p
 
@@ -472,7 +466,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_copymap", "_identity",
+for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
   "_pl", "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
@@ -490,17 +484,17 @@
 return
 self._dirty = True
 if source is not None:
-self._copymap[dest] = source
+self._map.copymap[dest] = source
 self._updatedfiles.add(source)
 self._updatedfiles.add(dest)
-elif self._copymap.pop(dest, None):
+elif self._map.copymap.pop(dest, None):
 self._updatedfiles.add(dest)
 
 def copied(self, file):
-return self._copymap.get(file, None)
+return self._map.copymap.get(file, None)
 
 def copies(self):
-return self._copymap
+return self._map.copymap
 
 def _droppath(self, f):
 if self[f] not in "?r" and "_dirs" in self.__dict__:
@@ -543,7 +537,7 @@
 mtime = s.st_mtime
 self._addpath(f, 'n', s.st_mode,
   s.st_size & _rangemask, mtime & _rangemask)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 if mtime > self._lastnormaltime:
@@ -561,7 +555,7 @@
 entry = self._map.get(f)
 if entry is not None:
 if entry[0] == 'r' and entry[2] in (-1, -2):
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if entry[2] == -1:
 self.merge(f)
 elif entry[2] == -2:
@@ -572,7 +566,7 @@
 if entry[0] == 'm' or entry[0] == 'n' and entry[2] == -2:
 return
 self._addpath(f, 'n', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 
@@ -587,12 +581,12 @@
 else:
 # add-like
 self._addpath(f, 'n', 0, -2, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def add(self, f):
 '''Mark a file added.'''
 self._addpath(f, 'a', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def remove(self, f):
 '''Mark a file removed.'''
@@ -611,7 +605,7 @@
 self._map[f] = dirstatetuple('r', 0, size, 0)
 self._nonnormalset.add(f)
 if size == 0:
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def merge(self, f):
 '''Mark a file merged.'''
@@ -627,7 +621,7 @@
 del self._map[f]
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
-

D759: dirstate: move parents source of truth to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2083.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D759?vs=1954=2083

REVISION DETAIL
  https://phab.mercurial-scm.org/D759

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -71,7 +71,6 @@
 # UNC path pointing to root share (issue4557)
 self._rootdir = pathutil.normasprefix(root)
 self._dirty = False
-self._dirtypl = False
 self._lastnormaltime = 0
 self._ui = ui
 self._filecache = {}
@@ -184,7 +183,7 @@
 raise
 return "default"
 
-@propertycache
+@property
 def _pl(self):
 return self._map.parents()
 
@@ -343,11 +342,11 @@
 raise ValueError("cannot set dirstate parent without "
  "calling dirstate.beginparentchange")
 
-self._dirty = self._dirtypl = True
+self._dirty = True
 oldp2 = self._pl[1]
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = p1, p2
+self._map.setparents(p1, p2)
 copies = {}
 if oldp2 != nullid and p2 == nullid:
 candidatefiles = self._nonnormalset.union(self._otherparentset)
@@ -432,8 +431,8 @@
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
 p = parse_dirstate(self._map._map, self._map.copymap, st)
-if not self._dirtypl:
-self._pl = p
+if not self._map._dirtyparents:
+self._map.setparents(*p)
 
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
@@ -444,7 +443,7 @@
 
 for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
-  "_pl", "_dirs", "_ignore", "_nonnormalset",
+  "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
 if a in self.__dict__:
 delattr(self, a)
@@ -679,7 +678,7 @@
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
 delattr(self, "_dirs")
-self._pl = [nullid, nullid]
+self._map.setparents(nullid, nullid)
 self._lastnormaltime = 0
 self._updatedfiles.clear()
 self._dirty = True
@@ -694,7 +693,7 @@
 
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = (parent, nullid)
+self._map.setparents(parent, nullid)
 for f in changedfiles:
 if f in allfiles:
 self.normallookup(f)
@@ -787,7 +786,7 @@
 self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
 st.close()
 self._lastnormaltime = 0
-self._dirty = self._dirtypl = False
+self._dirty = self._map._dirtyparents = False
 
 def _dirignore(self, f):
 if f == '.':
@@ -1292,6 +1291,8 @@
 
 self._map = {}
 self.copymap = {}
+self._parents = None
+self._dirtyparents = False
 
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
@@ -1370,16 +1371,28 @@
 return fp
 
 def parents(self):
-try:
-fp = self._opendirstatefile()
-st = fp.read(40)
-fp.close()
+if not self._parents:
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+# File doesn't exist, so the current state is empty
+st = ''
+
 l = len(st)
 if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+self._parents = st[:20], st[20:40]
+elif l == 0:
+self._parents = [nullid, nullid]
+else:
+raise error.Abort(_('working directory state appears '
+'damaged!'))
+
+return self._parents
+
+def setparents(self, p1, p2):
+self._parents = (p1, p2)
+self._dirtyparents = True



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D757: dirstate: move opendirstatefile to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2081.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D757?vs=1952=2081

REVISION DETAIL
  https://phab.mercurial-scm.org/D757

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -82,9 +82,6 @@
 self._origpl = None
 self._updatedfiles = set()
 
-# for consistent view between _pl() and _read() invocations
-self._pendingmode = None
-
 @contextlib.contextmanager
 def parentchange(self):
 '''Context manager for handling dirstate parents.
@@ -190,7 +187,7 @@
 @propertycache
 def _pl(self):
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 st = fp.read(40)
 fp.close()
 l = len(st)
@@ -401,23 +398,14 @@
 f.discard()
 raise
 
-def _opendirstatefile(self):
-fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
-if self._pendingmode is not None and self._pendingmode != mode:
-fp.close()
-raise error.Abort(_('working directory state may be '
-'changed parallelly'))
-self._pendingmode = mode
-return fp
-
 def _read(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 try:
 st = fp.read()
 finally:
@@ -698,7 +686,7 @@
 return path
 
 def clear(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._ui, self._opener, self._root)
 self._nonnormalset = set()
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
@@ -1308,10 +1296,18 @@
 self._opener.unlink(backupname)
 
 class dirstatemap(object):
-def __init__(self):
+def __init__(self, ui, opener, root):
+self._ui = ui
+self._opener = opener
+self._root = root
+self._filename = 'dirstate'
+
 self._map = {}
 self.copymap = {}
 
+# for consistent view between _pl() and _read() invocations
+self._pendingmode = None
+
 def iteritems(self):
 return self._map.iteritems()
 
@@ -1375,3 +1371,13 @@
 current dirstate.
 """
 return util.dirs(self._map, 'r')
+
+def _opendirstatefile(self):
+fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
+if self._pendingmode is not None and self._pendingmode != mode:
+fp.close()
+raise error.Abort(_('working directory state may be '
+'changed parallelly'))
+self._pendingmode = mode
+return fp
+



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D758: dirstate: move parent reading to the dirstatemap class

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2082.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D758?vs=1953=2082

REVISION DETAIL
  https://phab.mercurial-scm.org/D758

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -186,19 +186,7 @@
 
 @propertycache
 def _pl(self):
-try:
-fp = self._map._opendirstatefile()
-st = fp.read(40)
-fp.close()
-l = len(st)
-if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+return self._map.parents()
 
 @propertycache
 def _dirs(self):
@@ -1381,3 +1369,17 @@
 self._pendingmode = mode
 return fp
 
+def parents(self):
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+l = len(st)
+if l == 40:
+return st[:20], st[20:40]
+elif l > 0 and l < 40:
+raise error.Abort(_('working directory state appears 
damaged!'))
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+return [nullid, nullid]



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D755: dirstate: move _dirs to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2079.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D755?vs=1950=2079

REVISION DETAIL
  https://phab.mercurial-scm.org/D755

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -210,7 +210,7 @@
 
 @propertycache
 def _dirs(self):
-return util.dirs(self._map._map, 'r')
+return self._map.dirs()
 
 def dirs(self):
 return self._dirs
@@ -1375,3 +1375,9 @@
 f[normcase(name)] = name
 f['.'] = '.' # prevents useless util.fspath() invocation
 return f
+
+def dirs(self):
+"""Returns a set-like object containing all the directories in the
+current dirstate.
+"""
+return util.dirs(self._map, 'r')



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D754: dirstate: move filefoldmap to dirstatemap

2017-09-26 Thread durham (Durham Goode) 
durham updated this revision to Diff 2078.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D754?vs=1949=2078

REVISION DETAIL
  https://phab.mercurial-scm.org/D754

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -160,21 +160,7 @@
 
 @propertycache
 def _filefoldmap(self):
-try:
-makefilefoldmap = parsers.make_file_foldmap
-except AttributeError:
-pass
-else:
-return makefilefoldmap(self._map._map, util.normcasespec,
-   util.normcasefallback)
-
-f = {}
-normcase = util.normcase
-for name, s in self._map.iteritems():
-if s[0] != 'r':
-f[normcase(name)] = name
-f['.'] = '.' # prevents useless util.fspath() invocation
-return f
+return self._map.filefoldmap()
 
 @propertycache
 def _dirfoldmap(self):
@@ -1370,3 +1356,22 @@
 otherparent.add(fname)
 return nonnorm, otherparent
 
+def filefoldmap(self):
+"""Returns a dictionary mapping normalized case paths to their
+non-normalized versions.
+"""
+try:
+makefilefoldmap = parsers.make_file_foldmap
+except AttributeError:
+pass
+else:
+return makefilefoldmap(self._map, util.normcasespec,
+   util.normcasefallback)
+
+f = {}
+normcase = util.normcase
+for name, s in self._map.iteritems():
+if s[0] != 'r':
+f[normcase(name)] = name
+f['.'] = '.' # prevents useless util.fspath() invocation
+return f



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D752: dirstate: create new dirstatemap class

2017-09-26 Thread durham (Durham Goode) 
durham added inline comments.

INLINE COMMENTS

> indygreg wrote in dirstate.py:1345
> Do we want this inherit from `collections.MutableMapping`?
> 
> A good reason to say "no" is KISS, especially if we'll be having multiple 
> backends.

For now I'd say no, until we've done at least one other backend implementation 
and have seen that MutableMapping is still appropriate.  In particular, we 
might not want to allow iteration without a matcher, to discourage O(working 
copy) operations.

> indygreg wrote in dirstate.py:1349-1350
> Given that `iteritems()` no longer exists in Python 3, should we change this 
> to `items()` and standardize on the modern convention?

Do we do that anywhere else in the code base already?  My initial tendency is 
to just follow existing patterns, especially during a refactor, to minimize 
churn.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D752

To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D752: dirstate: create new dirstatemap class

2017-09-20 Thread durham (Durham Goode) 
durham added a comment.


  This series is the first 8 commits of a 20 patch series. At the end all the 
storage for the dirstate goes through the dirstatemap class.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D752

To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D758: dirstate: move parent reading to the dirstatemap class

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving dirstate storage logic to a separate class, let's move the
  function that reads the parents from the file. This will allow extensions to
  write dirstate's that store the parents in other ways.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D758

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -186,19 +186,7 @@
 
 @propertycache
 def _pl(self):
-try:
-fp = self._map._opendirstatefile()
-st = fp.read(40)
-fp.close()
-l = len(st)
-if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+return self._map.parents()
 
 @propertycache
 def _dirs(self):
@@ -1375,3 +1363,17 @@
 self._pendingmode = mode
 return fp
 
+def parents(self):
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+l = len(st)
+if l == 40:
+return st[:20], st[20:40]
+elif l > 0 and l < 40:
+raise error.Abort(_('working directory state appears 
damaged!'))
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+return [nullid, nullid]



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D754: dirstate: move filefoldmap to dirstatemap

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving the dirstate storage logic to a separate class, lets move 
the
  filfoldmap computation to that class. This will allow extensions to replace 
the
  dirstate storage with something that persists the filefoldmap.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D754

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -160,21 +160,7 @@
 
 @propertycache
 def _filefoldmap(self):
-try:
-makefilefoldmap = parsers.make_file_foldmap
-except AttributeError:
-pass
-else:
-return makefilefoldmap(self._map._map, util.normcasespec,
-   util.normcasefallback)
-
-f = {}
-normcase = util.normcase
-for name, s in self._map.iteritems():
-if s[0] != 'r':
-f[normcase(name)] = name
-f['.'] = '.' # prevents useless util.fspath() invocation
-return f
+return self._map.filefoldmap()
 
 @propertycache
 def _dirfoldmap(self):
@@ -1370,3 +1356,19 @@
 otherparent.add(fname)
 return nonnorm, otherparent
 
+def filefoldmap(self):
+try:
+makefilefoldmap = parsers.make_file_foldmap
+except AttributeError:
+pass
+else:
+return makefilefoldmap(self._map, util.normcasespec,
+   util.normcasefallback)
+
+f = {}
+normcase = util.normcase
+for name, s in self._map.iteritems():
+if s[0] != 'r':
+f[normcase(name)] = name
+f['.'] = '.' # prevents useless util.fspath() invocation
+return f



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D759: dirstate: move parents source of truth to dirstatemap

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving dirstate storage to its own class, let's move the source of
  truth for the dirstate parents to dirstatemap. This requires that dirstate._pl
  no longer be a cache, and that all sets go through dirstatemap.setparents.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D759

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -71,7 +71,6 @@
 # UNC path pointing to root share (issue4557)
 self._rootdir = pathutil.normasprefix(root)
 self._dirty = False
-self._dirtypl = False
 self._lastnormaltime = 0
 self._ui = ui
 self._filecache = {}
@@ -184,7 +183,7 @@
 raise
 return "default"
 
-@propertycache
+@property
 def _pl(self):
 return self._map.parents()
 
@@ -343,11 +342,11 @@
 raise ValueError("cannot set dirstate parent without "
  "calling dirstate.beginparentchange")
 
-self._dirty = self._dirtypl = True
+self._dirty = True
 oldp2 = self._pl[1]
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = p1, p2
+self._map.setparents(p1, p2)
 copies = {}
 if oldp2 != nullid and p2 == nullid:
 candidatefiles = self._nonnormalset.union(self._otherparentset)
@@ -432,8 +431,8 @@
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
 p = parse_dirstate(self._map._map, self._map.copymap, st)
-if not self._dirtypl:
-self._pl = p
+if not self._map._dirtyparents:
+self._map.setparents(*p)
 
 def invalidate(self):
 '''Causes the next access to reread the dirstate.
@@ -444,7 +443,7 @@
 
 for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
-  "_pl", "_dirs", "_ignore", "_nonnormalset",
+  "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
 if a in self.__dict__:
 delattr(self, a)
@@ -679,7 +678,7 @@
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
 delattr(self, "_dirs")
-self._pl = [nullid, nullid]
+self._map.setparents(nullid, nullid)
 self._lastnormaltime = 0
 self._updatedfiles.clear()
 self._dirty = True
@@ -694,7 +693,7 @@
 
 if self._origpl is None:
 self._origpl = self._pl
-self._pl = (parent, nullid)
+self._map.setparents(parent, nullid)
 for f in changedfiles:
 if f in allfiles:
 self.normallookup(f)
@@ -787,7 +786,7 @@
 self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
 st.close()
 self._lastnormaltime = 0
-self._dirty = self._dirtypl = False
+self._dirty = self._map._dirtyparents = False
 
 def _dirignore(self, f):
 if f == '.':
@@ -1292,6 +1291,8 @@
 
 self._map = {}
 self.copymap = {}
+self._parents = None
+self._dirtyparents = False
 
 # for consistent view between _pl() and _read() invocations
 self._pendingmode = None
@@ -1364,16 +1365,25 @@
 return fp
 
 def parents(self):
-try:
-fp = self._opendirstatefile()
-st = fp.read(40)
-fp.close()
-l = len(st)
-if l == 40:
-return st[:20], st[20:40]
-elif l > 0 and l < 40:
-raise error.Abort(_('working directory state appears 
damaged!'))
-except IOError as err:
-if err.errno != errno.ENOENT:
-raise
-return [nullid, nullid]
+if not self._parents:
+try:
+fp = self._opendirstatefile()
+st = fp.read(40)
+fp.close()
+l = len(st)
+if l == 40:
+self._parents = st[:20], st[20:40]
+elif l > 0 and l < 40:
+raise error.Abort(_('working directory state appears '
+'damaged!'))
+except IOError as err:
+if err.errno != errno.ENOENT:
+raise
+if not self._parents:
+self._parents = [nullid, nullid]
+
+return self._parents
+
+def setparents(self, p1, p2):
+self._parents = (p1, p2)
+self._dirtyparents = True



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list

D752: dirstate: create new dirstatemap class

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is part of a larger refactor to move the dirstate storage logic to a
  separate class, so it's easier to rewrite the dirstate storage layer without
  having to rewrite all the algorithms as well.
  
  Step one it to create the class, and replace dirstate._map with it.  The
  abstraction bleeds through in a few places where the main dirstate class has 
to
  access self._map._map, but those will be cleaned up in future patches.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D752

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -57,7 +57,7 @@
 def nonnormalentries(dmap):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
-return parsers.nonnormalotherparententries(dmap)
+return parsers.nonnormalotherparententries(dmap._map)
 except AttributeError:
 nonnorm = set()
 otherparent = set()
@@ -179,7 +179,7 @@
 except AttributeError:
 pass
 else:
-return makefilefoldmap(self._map, util.normcasespec,
+return makefilefoldmap(self._map._map, util.normcasespec,
util.normcasefallback)
 
 f = {}
@@ -238,7 +238,7 @@
 
 @propertycache
 def _dirs(self):
-return util.dirs(self._map, 'r')
+return util.dirs(self._map._map, 'r')
 
 def dirs(self):
 return self._dirs
@@ -444,7 +444,8 @@
 return fp
 
 def _read(self):
-self._map = {}
+self._map = dirstatemap()
+
 self._copymap = {}
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
@@ -473,7 +474,7 @@
 # This heuristic is imperfect in many ways, so in a future dirstate
 # format update it makes sense to just record the number of entries
 # on write.
-self._map = parsers.dict_new_presized(len(st) / 71)
+self._map._map = parsers.dict_new_presized(len(st) / 71)
 
 # Python's garbage collector triggers a GC each time a certain number
 # of container objects (the number being defined by
@@ -488,7 +489,7 @@
 #
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map, self._copymap, st)
+p = parse_dirstate(self._map._map, self._copymap, st)
 if not self._dirtypl:
 self._pl = p
 
@@ -731,7 +732,7 @@
 return path
 
 def clear(self):
-self._map = {}
+self._map = dirstatemap()
 self._nonnormalset = set()
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
@@ -840,7 +841,8 @@
 now = end # trust our estimate that the end is near now
 break
 
-st.write(parsers.pack_dirstate(self._map, self._copymap, self._pl, 
now))
+st.write(parsers.pack_dirstate(self._map._map, self._copymap, self._pl,
+   now))
 self._nonnormalset, self._otherparentset = nonnormalentries(self._map)
 st.close()
 self._lastnormaltime = 0
@@ -979,7 +981,7 @@
 results[nf] = None
 else: # does it match a missing directory?
 if alldirs is None:
-alldirs = util.dirs(dmap)
+alldirs = util.dirs(dmap._map)
 if nf in alldirs:
 if matchedir:
 matchedir(nf)
@@ -1339,3 +1341,31 @@
 def clearbackup(self, tr, backupname):
 '''Clear backup file'''
 self._opener.unlink(backupname)
+
+class dirstatemap(object):
+def __init__(self):
+self._map = {}
+
+def iteritems(self):
+return self._map.iteritems()
+
+def __iter__(self):
+return iter(self._map)
+
+def get(self, key, default=None):
+return self._map.get(key, default)
+
+def __contains__(self, key):
+return key in self._map
+
+def __setitem__(self, key, value):
+self._map[key] = value
+
+def __getitem__(self, key):
+return self._map[key]
+
+def __delitem__(self, key):
+del self._map[key]
+
+def keys(self):
+return self._map.keys()



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D757: dirstate: move opendirstatefile to dirstatemap

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving the dirstate storage logic to another class, let's move
  opendirstatefile to dirstatemap. This will allow extensions to replace the
  pending abstraction.
  
  Future patches will move the consumers of _opendirstatefile into dirstatemap 
as
  well.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D757

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -82,9 +82,6 @@
 self._origpl = None
 self._updatedfiles = set()
 
-# for consistent view between _pl() and _read() invocations
-self._pendingmode = None
-
 @contextlib.contextmanager
 def parentchange(self):
 '''Context manager for handling dirstate parents.
@@ -190,7 +187,7 @@
 @propertycache
 def _pl(self):
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 st = fp.read(40)
 fp.close()
 l = len(st)
@@ -401,23 +398,14 @@
 f.discard()
 raise
 
-def _opendirstatefile(self):
-fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
-if self._pendingmode is not None and self._pendingmode != mode:
-fp.close()
-raise error.Abort(_('working directory state may be '
-'changed parallelly'))
-self._pendingmode = mode
-return fp
-
 def _read(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._opener, self._ui, self._root)
 
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
 try:
-fp = self._opendirstatefile()
+fp = self._map._opendirstatefile()
 try:
 st = fp.read()
 finally:
@@ -698,7 +686,7 @@
 return path
 
 def clear(self):
-self._map = dirstatemap()
+self._map = dirstatemap(self._opener, self._ui, self._root)
 self._nonnormalset = set()
 self._otherparentset = set()
 if "_dirs" in self.__dict__:
@@ -1308,10 +1296,18 @@
 self._opener.unlink(backupname)
 
 class dirstatemap(object):
-def __init__(self):
+def __init__(self, opener, ui, root):
+self._opener = opener
+self._ui = ui
+self._root = root
+self._filename = 'dirstate'
+
 self._map = {}
 self.copymap = {}
 
+# for consistent view between _pl() and _read() invocations
+self._pendingmode = None
+
 def iteritems(self):
 return self._map.iteritems()
 
@@ -1369,3 +1365,13 @@
 
 def dirs(self):
 return util.dirs(self._map, 'r')
+
+def _opendirstatefile(self):
+fp, mode = txnutil.trypending(self._root, self._opener, self._filename)
+if self._pendingmode is not None and self._pendingmode != mode:
+fp.close()
+raise error.Abort(_('working directory state may be '
+'changed parallelly'))
+self._pendingmode = mode
+return fp
+



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D756: dirstate: move _copymap to dirstatemap

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving all dirstate storage to a new class, let's move the copymap
  onto that class. In a future patch this will let us move the read/write
  functions to the dirstatemap class, and for extensions this lets us replace 
the
  copy storage with alternative storage.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D756

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -137,11 +137,6 @@
 return self._map
 
 @propertycache
-def _copymap(self):
-self._read()
-return self._copymap
-
-@propertycache
 def _identity(self):
 self._read()
 return self._identity
@@ -378,13 +373,13 @@
 
 # Discard 'm' markers when moving away from a merge state
 if s[0] == 'm':
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.normallookup(f)
 # Also fix up otherparent markers
 elif s[0] == 'n' and s[2] == -2:
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if source:
 copies[f] = source
 self.add(f)
@@ -418,7 +413,6 @@
 def _read(self):
 self._map = dirstatemap()
 
-self._copymap = {}
 # ignore HG_PENDING because identity is used only for writing
 self._identity = util.filestat.frompath(
 self._opener.join(self._filename))
@@ -461,7 +455,7 @@
 #
 # (we cannot decorate the function directly since it is in a C module)
 parse_dirstate = util.nogc(parsers.parse_dirstate)
-p = parse_dirstate(self._map._map, self._copymap, st)
+p = parse_dirstate(self._map._map, self._map.copymap, st)
 if not self._dirtypl:
 self._pl = p
 
@@ -472,7 +466,7 @@
 rereads the dirstate. Use localrepo.invalidatedirstate() if you want to
 check whether the dirstate has changed before rereading it.'''
 
-for a in ("_map", "_copymap", "_identity",
+for a in ("_map", "_identity",
   "_filefoldmap", "_dirfoldmap", "_branch",
   "_pl", "_dirs", "_ignore", "_nonnormalset",
   "_otherparentset"):
@@ -490,17 +484,17 @@
 return
 self._dirty = True
 if source is not None:
-self._copymap[dest] = source
+self._map.copymap[dest] = source
 self._updatedfiles.add(source)
 self._updatedfiles.add(dest)
-elif self._copymap.pop(dest, None):
+elif self._map.copymap.pop(dest, None):
 self._updatedfiles.add(dest)
 
 def copied(self, file):
-return self._copymap.get(file, None)
+return self._map.copymap.get(file, None)
 
 def copies(self):
-return self._copymap
+return self._map.copymap
 
 def _droppath(self, f):
 if self[f] not in "?r" and "_dirs" in self.__dict__:
@@ -543,7 +537,7 @@
 mtime = s.st_mtime
 self._addpath(f, 'n', s.st_mode,
   s.st_size & _rangemask, mtime & _rangemask)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 if mtime > self._lastnormaltime:
@@ -561,7 +555,7 @@
 entry = self._map.get(f)
 if entry is not None:
 if entry[0] == 'r' and entry[2] in (-1, -2):
-source = self._copymap.get(f)
+source = self._map.copymap.get(f)
 if entry[2] == -1:
 self.merge(f)
 elif entry[2] == -2:
@@ -572,7 +566,7 @@
 if entry[0] == 'm' or entry[0] == 'n' and entry[2] == -2:
 return
 self._addpath(f, 'n', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 if f in self._nonnormalset:
 self._nonnormalset.remove(f)
 
@@ -587,12 +581,12 @@
 else:
 # add-like
 self._addpath(f, 'n', 0, -2, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def add(self, f):
 '''Mark a file added.'''
 self._addpath(f, 'a', 0, -1, -1)
-self._copymap.pop(f, None)
+self._map.copymap.pop(f, None)
 
 def remove(self, f):
 '''Mark a file removed.'''
@@ -611,7 +605,7 @@
 self._map[f] = dirstatetuple('r', 0, size, 0)
 self._nonnormalset.add(f)
 if size == 0:
-

D753: dirstate: move nonnormalentries to dirstatemap

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving dirstate storage to its own class, let's move the
  nonnormalentries logic onto the dirstatemap class. This will let extensions
  replace the nonnormalentries logic with a persisted cache.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D753

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -54,20 +54,6 @@
 os.close(tmpfd)
 vfs.unlink(tmpname)
 
-def nonnormalentries(dmap):
-'''Compute the nonnormal dirstate entries from the dmap'''
-try:
-return parsers.nonnormalotherparententries(dmap._map)
-except AttributeError:
-nonnorm = set()
-otherparent = set()
-for fname, e in dmap.iteritems():
-if e[0] != 'n' or e[3] == -1:
-nonnorm.add(fname)
-if e[0] == 'n' and e[2] == -2:
-otherparent.add(fname)
-return nonnorm, otherparent
-
 class dirstate(object):
 
 def __init__(self, opener, ui, root, validate, sparsematchfn):
@@ -162,13 +148,13 @@
 
 @propertycache
 def _nonnormalset(self):
-nonnorm, otherparents = nonnormalentries(self._map)
+nonnorm, otherparents = self._map.nonnormalentries()
 self._otherparentset = otherparents
 return nonnorm
 
 @propertycache
 def _otherparentset(self):
-nonnorm, otherparents = nonnormalentries(self._map)
+nonnorm, otherparents = self._map.nonnormalentries()
 self._nonnormalset = nonnorm
 return otherparents
 
@@ -843,7 +829,7 @@
 
 st.write(parsers.pack_dirstate(self._map._map, self._copymap, self._pl,
now))
-self._nonnormalset, self._otherparentset = nonnormalentries(self._map)
+self._nonnormalset, self._otherparentset = self._map.nonnormalentries()
 st.close()
 self._lastnormaltime = 0
 self._dirty = self._dirtypl = False
@@ -1369,3 +1355,18 @@
 
 def keys(self):
 return self._map.keys()
+
+def nonnormalentries(self):
+'''Compute the nonnormal dirstate entries from the dmap'''
+try:
+return parsers.nonnormalotherparententries(self._map)
+except AttributeError:
+nonnorm = set()
+otherparent = set()
+for fname, e in self._map.iteritems():
+if e[0] != 'n' or e[3] == -1:
+nonnorm.add(fname)
+if e[0] == 'n' and e[2] == -2:
+otherparent.add(fname)
+return nonnorm, otherparent
+



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D730: revlog: add revmap back to revlog.addgroup

2017-09-20 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
durham marked an inline comment as done.
Closed by commit rHG1db9abf407c5: revlog: add revmap back to revlog.addgroup 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D730?vs=1888=1942

REVISION DETAIL
  https://phab.mercurial-scm.org/D730

AFFECTED FILES
  mercurial/changegroup.py
  mercurial/revlog.py
  tests/test-revlog-raw.py

CHANGE DETAILS

diff --git a/tests/test-revlog-raw.py b/tests/test-revlog-raw.py
--- a/tests/test-revlog-raw.py
+++ b/tests/test-revlog-raw.py
@@ -119,7 +119,7 @@
 'deltabase': rlog.node(deltaparent),
 'delta': rlog.revdiff(deltaparent, r)}
 
-def deltaiter(self, linkmapper):
+def deltaiter(self):
 chain = None
 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
 node = chunkdata['node']
@@ -130,17 +130,16 @@
 delta = chunkdata['delta']
 flags = chunkdata['flags']
 
-link = linkmapper(cs)
 chain = node
 
-yield (node, p1, p2, link, deltabase, delta, flags)
+yield (node, p1, p2, cs, deltabase, delta, flags)
 
 def linkmap(lnode):
 return rlog.rev(lnode)
 
 dlog = newrevlog(destname, recreate=True)
-dummydeltas = dummychangegroup().deltaiter(linkmap)
-dlog.addgroup(dummydeltas, tr)
+dummydeltas = dummychangegroup().deltaiter()
+dlog.addgroup(dummydeltas, linkmap, tr)
 return dlog
 
 def lowlevelcopy(rlog, tr, destname=b'_destrevlog.i'):
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1910,7 +1910,7 @@
 ifh.write(data[1])
 self.checkinlinesize(transaction, ifh)
 
-def addgroup(self, deltas, transaction, addrevisioncb=None):
+def addgroup(self, deltas, linkmapper, transaction, addrevisioncb=None):
 """
 add a delta group
 
@@ -1944,7 +1944,8 @@
 try:
 # loop through our set of deltas
 for data in deltas:
-node, p1, p2, link, deltabase, delta, flags = data
+node, p1, p2, linknode, deltabase, delta, flags = data
+link = linkmapper(linknode)
 flags = flags or REVIDX_DEFAULT_FLAGS
 
 nodes.append(node)
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -245,8 +245,8 @@
 # no new manifest will be created and the manifest group will
 # be empty during the pull
 self.manifestheader()
-deltas = self.deltaiter(revmap)
-repo.manifestlog._revlog.addgroup(deltas, trp)
+deltas = self.deltaiter()
+repo.manifestlog._revlog.addgroup(deltas, revmap, trp)
 repo.ui.progress(_('manifests'), None)
 self.callback = None
 
@@ -308,8 +308,8 @@
 efiles.update(cl.readfiles(node))
 
 self.changelogheader()
-deltas = self.deltaiter(csmap)
-cgnodes = cl.addgroup(deltas, trp, addrevisioncb=onchangelog)
+deltas = self.deltaiter()
+cgnodes = cl.addgroup(deltas, csmap, trp, 
addrevisioncb=onchangelog)
 efiles = len(efiles)
 
 if not cgnodes:
@@ -430,7 +430,7 @@
 ret = deltaheads + 1
 return ret
 
-def deltaiter(self, linkmapper):
+def deltaiter(self):
 """
 returns an iterator of the deltas in this changegroup
 
@@ -446,10 +446,9 @@
 delta = chunkdata['delta']
 flags = chunkdata['flags']
 
-link = linkmapper(cs)
 chain = node
 
-yield (node, p1, p2, link, deltabase, delta, flags)
+yield (node, p1, p2, cs, deltabase, delta, flags)
 
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.
@@ -491,8 +490,8 @@
 d = chunkdata["filename"]
 repo.ui.debug("adding %s revisions\n" % d)
 dirlog = repo.manifestlog._revlog.dirlog(d)
-deltas = self.deltaiter(revmap)
-if not dirlog.addgroup(deltas, trp):
+deltas = self.deltaiter()
+if not dirlog.addgroup(deltas, revmap, trp):
 raise error.Abort(_("received dir revlog group is empty"))
 
 class headerlessfixup(object):
@@ -983,8 +982,8 @@
 fl = repo.file(f)
 o = len(fl)
 try:
-deltas = source.deltaiter(revmap)
-if not fl.addgroup(deltas, trp):
+deltas = source.deltaiter()
+if not fl.addgroup(deltas, revmap, trp):
 raise error.Abort(_("received file revlog group is empty"))
 except error.CensoredBaseError as e:
 raise error.Abort(_("received delta base is censored: %s") % e)



To: durham, indygreg,

D746: changegroup: remove dictionary creation from deltachunk

2017-09-20 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG05131c963767: changegroup: remove dictionary creation from 
deltachunk (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D746?vs=1934=1945

REVISION DETAIL
  https://phab.mercurial-scm.org/D746

AFFECTED FILES
  mercurial/changegroup.py

CHANGE DETAILS

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -188,8 +188,7 @@
 header = struct.unpack(self.deltaheader, headerdata)
 delta = readexactly(self._stream, l - self.deltaheadersize)
 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, 
prevnode)
-return {'node': node, 'p1': p1, 'p2': p2, 'cs': cs,
-'deltabase': deltabase, 'delta': delta, 'flags': flags}
+return (node, p1, p2, cs, deltabase, delta, flags)
 
 def getchunks(self):
 """returns all the chunks contains in the bundle
@@ -438,17 +437,9 @@
 """
 chain = None
 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags']
-
-chain = node
-
-yield (node, p1, p2, cs, deltabase, delta, flags)
+# Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
+yield chunkdata
+chain = chunkdata[0]
 
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.



To: durham, #hg-reviewers, martinvonz
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D745: bundlerepo: update to use new deltaiter api

2017-09-20 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0fe62d8bdd50: bundlerepo: update to use new deltaiter api 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D745?vs=1933=1944

REVISION DETAIL
  https://phab.mercurial-scm.org/D745

AFFECTED FILES
  mercurial/bundlerepo.py

CHANGE DETAILS

diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py
--- a/mercurial/bundlerepo.py
+++ b/mercurial/bundlerepo.py
@@ -55,25 +55,16 @@
 self.bundle = bundle
 n = len(self)
 self.repotiprev = n - 1
-chain = None
 self.bundlerevs = set() # used by 'bundle()' revset expression
-getchunk = lambda: bundle.deltachunk(chain)
-for chunkdata in iter(getchunk, {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags']
+for deltadata in bundle.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 
 size = len(delta)
 start = bundle.tell() - size
 
 link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
-chain = node
 self.bundlerevs.add(self.nodemap[node])
 continue
 
@@ -93,7 +84,6 @@
 self.index.insert(-1, e)
 self.nodemap[node] = n
 self.bundlerevs.add(n)
-chain = node
 n += 1
 
 def _chunk(self, rev):



To: durham, #hg-reviewers, martinvonz
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D744: debug: update debugbundle to use new deltaiter api

2017-09-20 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG311f6ccf8f23: debug: update debugbundle to use new 
deltaiter api (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D744?vs=1932=1943

REVISION DETAIL
  https://phab.mercurial-scm.org/D744

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -263,18 +263,11 @@
 
 def showchunks(named):
 ui.write("\n%s%s\n" % (indent_string, named))
-chain = None
-for chunkdata in iter(lambda: gen.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
+for deltadata in gen.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 ui.write("%s%s %s %s %s %s %s\n" %
  (indent_string, hex(node), hex(p1), hex(p2),
   hex(cs), hex(deltabase), len(delta)))
-chain = node
 
 chunkdata = gen.changelogheader()
 showchunks("changelog")
@@ -287,11 +280,9 @@
 if isinstance(gen, bundle2.unbundle20):
 raise error.Abort(_('use debugbundle2 for this file'))
 chunkdata = gen.changelogheader()
-chain = None
-for chunkdata in iter(lambda: gen.deltachunk(chain), {}):
-node = chunkdata['node']
+for deltadata in gen.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 ui.write("%s%s\n" % (indent_string, hex(node)))
-chain = node
 
 def _debugobsmarkers(ui, part, indent=0, **opts):
 """display version and markers contained in 'data'"""



To: durham, #hg-reviewers, martinvonz
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D730: revlog: add revmap back to revlog.addgroup

2017-09-20 Thread durham (Durham Goode) 
durham marked an inline comment as done.
durham added a comment.


  I've updated the message and attached 3 more commits to the stack that update 
debugbundle, bundlerepo, and the deltachunk fix you suggested.

INLINE COMMENTS

> martinvonz wrote in changegroup.py:191-192
> I'll repeat my comment from https://phab.mercurial-scm.org/D688: This now 
> seems unnecessary since the first thing you do is to convert it back into a 
> very similar tuple. Do you think the is still useful? Maybe use a named tuple 
> if we want the names? Either way, that's not for this patch, of course.

I've added a diff to the end of the stack to change it to a tuple. No need for 
names I think, since the return value is only consumed internal to the class.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D730

To: durham, indygreg, #hg-reviewers
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D745: bundlerepo: update to use new deltaiter api

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D745

AFFECTED FILES
  mercurial/bundlerepo.py

CHANGE DETAILS

diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py
--- a/mercurial/bundlerepo.py
+++ b/mercurial/bundlerepo.py
@@ -55,25 +55,16 @@
 self.bundle = bundle
 n = len(self)
 self.repotiprev = n - 1
-chain = None
 self.bundlerevs = set() # used by 'bundle()' revset expression
-getchunk = lambda: bundle.deltachunk(chain)
-for chunkdata in iter(getchunk, {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags']
+for deltadata in bundle.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 
 size = len(delta)
 start = bundle.tell() - size
 
 link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
-chain = node
 self.bundlerevs.add(self.nodemap[node])
 continue
 
@@ -93,7 +84,6 @@
 self.index.insert(-1, e)
 self.nodemap[node] = n
 self.bundlerevs.add(n)
-chain = node
 n += 1
 
 def _chunk(self, rev):



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D746: changegroup: remove dictionary creation from deltachunk

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously delta chunk returned a dictionary. Now that we consume deltachunk
  within changegroup (instead of outside in revlog) we can just return a tuple 
and
  have it be returned directly by deltaiter.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D746

AFFECTED FILES
  mercurial/changegroup.py

CHANGE DETAILS

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -188,8 +188,7 @@
 header = struct.unpack(self.deltaheader, headerdata)
 delta = readexactly(self._stream, l - self.deltaheadersize)
 node, p1, p2, deltabase, cs, flags = self._deltaheader(header, 
prevnode)
-return {'node': node, 'p1': p1, 'p2': p2, 'cs': cs,
-'deltabase': deltabase, 'delta': delta, 'flags': flags}
+return (node, p1, p2, cs, deltabase, delta, flags)
 
 def getchunks(self):
 """returns all the chunks contains in the bundle
@@ -438,17 +437,9 @@
 """
 chain = None
 for chunkdata in iter(lambda: self.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags']
-
-chain = node
-
-yield (node, p1, p2, cs, deltabase, delta, flags)
+# Chunkdata: (node, p1, p2, cs, deltabase, delta, flags)
+yield chunkdata
+chain = chunkdata[0]
 
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D744: debug: update debugbundle to use new deltaiter api

2017-09-20 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Changegroup now has a deltaiter api for easy iteration over a series of 
deltas.
  Let's use that in the debugbundle command.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D744

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -263,18 +263,11 @@
 
 def showchunks(named):
 ui.write("\n%s%s\n" % (indent_string, named))
-chain = None
-for chunkdata in iter(lambda: gen.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
+for deltadata in gen.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 ui.write("%s%s %s %s %s %s %s\n" %
  (indent_string, hex(node), hex(p1), hex(p2),
   hex(cs), hex(deltabase), len(delta)))
-chain = node
 
 chunkdata = gen.changelogheader()
 showchunks("changelog")
@@ -287,11 +280,9 @@
 if isinstance(gen, bundle2.unbundle20):
 raise error.Abort(_('use debugbundle2 for this file'))
 chunkdata = gen.changelogheader()
-chain = None
-for chunkdata in iter(lambda: gen.deltachunk(chain), {}):
-node = chunkdata['node']
+for deltadata in gen.deltaiter():
+node, p1, p2, cs, deltabase, delta, flags = deltadata
 ui.write("%s%s\n" % (indent_string, hex(node)))
-chain = node
 
 def _debugobsmarkers(ui, part, indent=0, **opts):
 """display version and markers contained in 'data'"""



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D708: bundle2: remove unnecessary try finally

2017-09-18 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGcc7b37c90616: bundle2: remove unnecessary try finally 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D708?vs=1820=1875

REVISION DETAIL
  https://phab.mercurial-scm.org/D708

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -505,32 +505,28 @@
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-try:
-handler = _gethandler(op, part)
-if handler is None:
-return
+handler = _gethandler(op, part)
+if handler is None:
+return
 
-# handler is called outside the above try block so that we don't
-# risk catching KeyErrors from anything other than the
-# parthandlermapping lookup (any KeyError raised by handler()
-# itself represents a defect of a different variety).
-output = None
-if op.captureoutput and op.reply is not None:
-op.ui.pushbuffer(error=True, subproc=True)
-output = ''
-try:
-handler(op, part)
-finally:
-if output is not None:
-output = op.ui.popbuffer()
-if output:
-outpart = op.reply.newpart('output', data=output,
-   mandatory=False)
-outpart.addparam(
-'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
+# handler is called outside the above try block so that we don't
+# risk catching KeyErrors from anything other than the
+# parthandlermapping lookup (any KeyError raised by handler()
+# itself represents a defect of a different variety).
+output = None
+if op.captureoutput and op.reply is not None:
+op.ui.pushbuffer(error=True, subproc=True)
+output = ''
+try:
+handler(op, part)
 finally:
-pass
-
+if output is not None:
+output = op.ui.popbuffer()
+if output:
+outpart = op.reply.newpart('output', data=output,
+   mandatory=False)
+outpart.addparam(
+'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
 
 def decodecaps(blob):
 """decode a bundle2 caps bytes blob into a dictionary



To: durham, #hg-reviewers, indygreg, krbullock
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D709: bundle2: move part processing to a separate function

2017-09-18 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf010097c885c: bundle2: move part processing to a separate 
function (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D709?vs=1821=1876

REVISION DETAIL
  https://phab.mercurial-scm.org/D709

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -447,12 +447,15 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
+processparts(repo, op, unbundler)
+
+return op
+
+def processparts(repo, op, unbundler):
 with partiterator(repo, op, unbundler) as parts:
 for part in parts:
 _processpart(op, part)
 
-return op
-
 def _processchangegroup(op, cg, tr, source, url, **kwargs):
 ret = cg.apply(op.repo, tr, source, url, **kwargs)
 op.records.add('changegroup', {



To: durham, #hg-reviewers, krbullock
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D707: bundle2: move handler validation out of processpart

2017-09-18 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG07e4170f02f3: bundle2: move handler validation out of 
processpart (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D707?vs=1819=1873

REVISION DETAIL
  https://phab.mercurial-scm.org/D707

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -460,48 +460,55 @@
 })
 return ret
 
+def _gethandler(op, part):
+status = 'unknown' # used by debug output
+try:
+handler = parthandlermapping.get(part.type)
+if handler is None:
+status = 'unsupported-type'
+raise error.BundleUnknownFeatureError(parttype=part.type)
+indebug(op.ui, 'found a handler for part %r' % part.type)
+unknownparams = part.mandatorykeys - handler.params
+if unknownparams:
+unknownparams = list(unknownparams)
+unknownparams.sort()
+status = 'unsupported-params (%s)' % unknownparams
+raise error.BundleUnknownFeatureError(parttype=part.type,
+  params=unknownparams)
+status = 'supported'
+except error.BundleUnknownFeatureError as exc:
+if part.mandatory: # mandatory parts
+raise
+indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
+return # skip to part processing
+finally:
+if op.ui.debugflag:
+msg = ['bundle2-input-part: "%s"' % part.type]
+if not part.mandatory:
+msg.append(' (advisory)')
+nbmp = len(part.mandatorykeys)
+nbap = len(part.params) - nbmp
+if nbmp or nbap:
+msg.append(' (params:')
+if nbmp:
+msg.append(' %i mandatory' % nbmp)
+if nbap:
+msg.append(' %i advisory' % nbmp)
+msg.append(')')
+msg.append(' %s\n' % status)
+op.ui.debug(''.join(msg))
+
+return handler
+
 def _processpart(op, part):
 """process a single part from a bundle
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-status = 'unknown' # used by debug output
 try:
-try:
-handler = parthandlermapping.get(part.type)
-if handler is None:
-status = 'unsupported-type'
-raise error.BundleUnknownFeatureError(parttype=part.type)
-indebug(op.ui, 'found a handler for part %r' % part.type)
-unknownparams = part.mandatorykeys - handler.params
-if unknownparams:
-unknownparams = list(unknownparams)
-unknownparams.sort()
-status = 'unsupported-params (%s)' % unknownparams
-raise error.BundleUnknownFeatureError(parttype=part.type,
-  params=unknownparams)
-status = 'supported'
-except error.BundleUnknownFeatureError as exc:
-if part.mandatory: # mandatory parts
-raise
-indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
-return # skip to part processing
-finally:
-if op.ui.debugflag:
-msg = ['bundle2-input-part: "%s"' % part.type]
-if not part.mandatory:
-msg.append(' (advisory)')
-nbmp = len(part.mandatorykeys)
-nbap = len(part.params) - nbmp
-if nbmp or nbap:
-msg.append(' (params:')
-if nbmp:
-msg.append(' %i mandatory' % nbmp)
-if nbap:
-msg.append(' %i advisory' % nbmp)
-msg.append(')')
-msg.append(' %s\n' % status)
-op.ui.debug(''.join(msg))
+handler = _gethandler(op, part)
+if handler is None:
+return
 
 # handler is called outside the above try block so that we don't
 # risk catching KeyErrors from anything other than the



To: durham, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D477: revlog: add option to mmap revlog index

2017-09-15 Thread durham (Durham Goode) 
durham added a comment.


  Also, the improvements should become more pronounced if we fix some 
algorithms (like phases) that access really old commits.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D477

To: mbthomas, #fbhgext, indygreg, #hg-reviewers, durin42, quark
Cc: durham, quark, durin42, simonfar, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D696: registrar: add a enum 'cmdtype' for the type of the command

2017-09-14 Thread durham (Durham Goode) 
durham accepted this revision.
durham added a comment.


  I think the name could be better, but that can be bikeshed.  Stamping my 
approval for the concept and pattern.

INLINE COMMENTS

> registrar.py:148
>  
> +class cmdtype(object):
> +""" enum for the type of command which will tell whether the command is

`cmdtype` might be overly vague, since I could imagine a number of 
classifications it could mean.  Maybe "cmdwritetype"?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D696

To: pulkit, #hg-reviewers, durham
Cc: durham, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D706: bundle2: move processpart stream maintenance into part iterator

2017-09-14 Thread durham (Durham Goode) 
durham updated this revision to Diff 1818.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D706?vs=1802=1818

REVISION DETAIL
  https://phab.mercurial-scm.org/D706

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -354,21 +354,32 @@
 self.unbundler = unbundler
 self.iterator = None
 self.count = 0
+self.current = None
 
 def __enter__(self):
 def func():
 itr = enumerate(self.unbundler.iterparts())
 for count, p in itr:
 self.count = count
+self.current = p
 yield p
+p.seek(0, 2)
+self.current = None
 self.iterator = func()
 return self.iterator
 
 def __exit__(self, type, exc, tb):
 if not self.iterator:
 return
 
 if exc:
+# If exiting or interrupted, do not attempt to seek the stream in
+# the finally block below. This makes abort faster.
+if (self.current and
+not isinstance(exc, (SystemExit, KeyboardInterrupt))):
+# consume the part content to not corrupt the stream.
+self.current.seek(0, 2)
+
 # Any exceptions seeking to the end of the bundle at this point are
 # almost certainly related to the underlying stream being bad.
 # And, chances are that the exception we're handling is related to
@@ -455,7 +466,6 @@
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
 status = 'unknown' # used by debug output
-hardabort = False
 try:
 try:
 handler = parthandlermapping.get(part.type)
@@ -511,15 +521,8 @@
mandatory=False)
 outpart.addparam(
 'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
-# If exiting or interrupted, do not attempt to seek the stream in the
-# finally block below. This makes abort faster.
-except (SystemExit, KeyboardInterrupt):
-hardabort = True
-raise
 finally:
-# consume the part content to not corrupt the stream.
-if not hardabort:
-part.seek(0, 2)
+pass
 
 
 def decodecaps(blob):
@@ -1143,7 +1146,15 @@
 return
 part = unbundlepart(self.ui, headerblock, self._fp)
 op = interruptoperation(self.ui)
-_processpart(op, part)
+hardabort = False
+try:
+_processpart(op, part)
+except (SystemExit, KeyboardInterrupt):
+hardabort = True
+raise
+finally:
+if not hardabort:
+part.seek(0, 2)
 self.ui.debug('bundle2-input-stream-interrupt:'
   ' closing out of band context\n')
 



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D707: bundle2: move handler validation out of processpart

2017-09-14 Thread durham (Durham Goode) 
durham updated this revision to Diff 1819.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D707?vs=1803=1819

REVISION DETAIL
  https://phab.mercurial-scm.org/D707

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -460,48 +460,55 @@
 })
 return ret
 
+def _gethandler(op, part):
+status = 'unknown' # used by debug output
+try:
+handler = parthandlermapping.get(part.type)
+if handler is None:
+status = 'unsupported-type'
+raise error.BundleUnknownFeatureError(parttype=part.type)
+indebug(op.ui, 'found a handler for part %r' % part.type)
+unknownparams = part.mandatorykeys - handler.params
+if unknownparams:
+unknownparams = list(unknownparams)
+unknownparams.sort()
+status = 'unsupported-params (%s)' % unknownparams
+raise error.BundleUnknownFeatureError(parttype=part.type,
+  params=unknownparams)
+status = 'supported'
+except error.BundleUnknownFeatureError as exc:
+if part.mandatory: # mandatory parts
+raise
+indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
+return # skip to part processing
+finally:
+if op.ui.debugflag:
+msg = ['bundle2-input-part: "%s"' % part.type]
+if not part.mandatory:
+msg.append(' (advisory)')
+nbmp = len(part.mandatorykeys)
+nbap = len(part.params) - nbmp
+if nbmp or nbap:
+msg.append(' (params:')
+if nbmp:
+msg.append(' %i mandatory' % nbmp)
+if nbap:
+msg.append(' %i advisory' % nbmp)
+msg.append(')')
+msg.append(' %s\n' % status)
+op.ui.debug(''.join(msg))
+
+return handler
+
 def _processpart(op, part):
 """process a single part from a bundle
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-status = 'unknown' # used by debug output
 try:
-try:
-handler = parthandlermapping.get(part.type)
-if handler is None:
-status = 'unsupported-type'
-raise error.BundleUnknownFeatureError(parttype=part.type)
-indebug(op.ui, 'found a handler for part %r' % part.type)
-unknownparams = part.mandatorykeys - handler.params
-if unknownparams:
-unknownparams = list(unknownparams)
-unknownparams.sort()
-status = 'unsupported-params (%s)' % unknownparams
-raise error.BundleUnknownFeatureError(parttype=part.type,
-  params=unknownparams)
-status = 'supported'
-except error.BundleUnknownFeatureError as exc:
-if part.mandatory: # mandatory parts
-raise
-indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
-return # skip to part processing
-finally:
-if op.ui.debugflag:
-msg = ['bundle2-input-part: "%s"' % part.type]
-if not part.mandatory:
-msg.append(' (advisory)')
-nbmp = len(part.mandatorykeys)
-nbap = len(part.params) - nbmp
-if nbmp or nbap:
-msg.append(' (params:')
-if nbmp:
-msg.append(' %i mandatory' % nbmp)
-if nbap:
-msg.append(' %i advisory' % nbmp)
-msg.append(')')
-msg.append(' %s\n' % status)
-op.ui.debug(''.join(msg))
+handler = _gethandler(op, part)
+if handler is None:
+return
 
 # handler is called outside the above try block so that we don't
 # risk catching KeyErrors from anything other than the



To: durham, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D708: bundle2: remove unnecessary try finally

2017-09-14 Thread durham (Durham Goode) 
durham updated this revision to Diff 1820.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D708?vs=1804=1820

REVISION DETAIL
  https://phab.mercurial-scm.org/D708

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -505,32 +505,28 @@
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-try:
-handler = _gethandler(op, part)
-if handler is None:
-return
+handler = _gethandler(op, part)
+if handler is None:
+return
 
-# handler is called outside the above try block so that we don't
-# risk catching KeyErrors from anything other than the
-# parthandlermapping lookup (any KeyError raised by handler()
-# itself represents a defect of a different variety).
-output = None
-if op.captureoutput and op.reply is not None:
-op.ui.pushbuffer(error=True, subproc=True)
-output = ''
-try:
-handler(op, part)
-finally:
-if output is not None:
-output = op.ui.popbuffer()
-if output:
-outpart = op.reply.newpart('output', data=output,
-   mandatory=False)
-outpart.addparam(
-'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
+# handler is called outside the above try block so that we don't
+# risk catching KeyErrors from anything other than the
+# parthandlermapping lookup (any KeyError raised by handler()
+# itself represents a defect of a different variety).
+output = None
+if op.captureoutput and op.reply is not None:
+op.ui.pushbuffer(error=True, subproc=True)
+output = ''
+try:
+handler(op, part)
 finally:
-pass
-
+if output is not None:
+output = op.ui.popbuffer()
+if output:
+outpart = op.reply.newpart('output', data=output,
+   mandatory=False)
+outpart.addparam(
+'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
 
 def decodecaps(blob):
 """decode a bundle2 caps bytes blob into a dictionary



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D705: bundle2: move exception handling into part iterator

2017-09-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3d8fb0c37e12: bundle2: move exception handling into part 
iterator (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D705?vs=1801=1809

REVISION DETAIL
  https://phab.mercurial-scm.org/D705

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -348,8 +348,9 @@
 return op
 
 class partiterator(object):
-def __init__(self, repo, unbundler):
+def __init__(self, repo, op, unbundler):
 self.repo = repo
+self.op = op
 self.unbundler = unbundler
 self.iterator = None
 self.count = 0
@@ -363,10 +364,43 @@
 self.iterator = func()
 return self.iterator
 
-def __exit__(self, type, value, tb):
+def __exit__(self, type, exc, tb):
 if not self.iterator:
 return
 
+if exc:
+# Any exceptions seeking to the end of the bundle at this point are
+# almost certainly related to the underlying stream being bad.
+# And, chances are that the exception we're handling is related to
+# getting in that bad state. So, we swallow the seeking error and
+# re-raise the original error.
+seekerror = False
+try:
+for part in self.iterator:
+# consume the bundle content
+part.seek(0, 2)
+except Exception:
+seekerror = True
+
+# Small hack to let caller code distinguish exceptions from bundle2
+# processing from processing the old format. This is mostly needed
+# to handle different return codes to unbundle according tothe type
+# of bundle. We should probably clean up or drop this return code
+# craziness in a future version.
+exc.duringunbundle2 = True
+salvaged = []
+replycaps = None
+if self.op.reply is not None:
+salvaged = self.op.reply.salvageoutput()
+replycaps = self.op.reply.capabilities
+exc._replycaps = replycaps
+exc._bundle2salvagedoutput = salvaged
+
+# Re-raising from a variable loses the original stack. So only use
+# that form if we need to.
+if seekerror:
+raise exc
+
 self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
self.count)
 
@@ -402,45 +436,9 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
-with partiterator(repo, unbundler) as parts:
-part = None
-try:
-for part in parts:
-_processpart(op, part)
-except Exception as exc:
-# Any exceptions seeking to the end of the bundle at this point are
-# almost certainly related to the underlying stream being bad.
-# And, chances are that the exception we're handling is related to
-# getting in that bad state. So, we swallow the seeking error and
-# re-raise the original error.
-seekerror = False
-try:
-for part in parts:
-# consume the bundle content
-part.seek(0, 2)
-except Exception:
-seekerror = True
-
-# Small hack to let caller code distinguish exceptions from bundle2
-# processing from processing the old format. This is mostly needed
-# to handle different return codes to unbundle according to the 
type
-# of bundle. We should probably clean up or drop this return code
-# craziness in a future version.
-exc.duringunbundle2 = True
-salvaged = []
-replycaps = None
-if op.reply is not None:
-salvaged = op.reply.salvageoutput()
-replycaps = op.reply.capabilities
-exc._replycaps = replycaps
-exc._bundle2salvagedoutput = salvaged
-
-# Re-raising from a variable loses the original stack. So only use
-# that form if we need to.
-if seekerror:
-raise exc
-else:
-raise
+with partiterator(repo, op, unbundler) as parts:
+for part in parts:
+_processpart(op, part)
 
 return op
 



To: durham, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D704: bundle2: move part counter to partiterator

2017-09-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG550343626bb2: bundle2: move part counter to partiterator 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D704?vs=1800=1808

REVISION DETAIL
  https://phab.mercurial-scm.org/D704

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -348,14 +348,27 @@
 return op
 
 class partiterator(object):
-def __init__(self, unbundler):
+def __init__(self, repo, unbundler):
+self.repo = repo
 self.unbundler = unbundler
+self.iterator = None
+self.count = 0
 
 def __enter__(self):
-return enumerate(self.unbundler.iterparts())
+def func():
+itr = enumerate(self.unbundler.iterparts())
+for count, p in itr:
+self.count = count
+yield p
+self.iterator = func()
+return self.iterator
 
 def __exit__(self, type, value, tb):
-pass
+if not self.iterator:
+return
+
+self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
+   self.count)
 
 def processbundle(repo, unbundler, transactiongetter=None, op=None):
 """This function process a bundle, apply effect to/from a repo
@@ -389,11 +402,10 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
-with partiterator(unbundler) as parts:
+with partiterator(repo, unbundler) as parts:
 part = None
-nbpart = 0
 try:
-for nbpart, part in parts:
+for part in parts:
 _processpart(op, part)
 except Exception as exc:
 # Any exceptions seeking to the end of the bundle at this point are
@@ -403,7 +415,7 @@
 # re-raise the original error.
 seekerror = False
 try:
-for nbpart, part in parts:
+for part in parts:
 # consume the bundle content
 part.seek(0, 2)
 except Exception:
@@ -429,8 +441,6 @@
 raise exc
 else:
 raise
-finally:
-repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
 
 return op
 



To: durham, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D703: bundle2: move part iterator a separate class

2017-09-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe9e0e1143fc5: bundle2: move part iterator a separate class 
(authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D703?vs=1799=1807

REVISION DETAIL
  https://phab.mercurial-scm.org/D703

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -347,6 +347,16 @@
 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
 return op
 
+class partiterator(object):
+def __init__(self, unbundler):
+self.unbundler = unbundler
+
+def __enter__(self):
+return enumerate(self.unbundler.iterparts())
+
+def __exit__(self, type, value, tb):
+pass
+
 def processbundle(repo, unbundler, transactiongetter=None, op=None):
 """This function process a bundle, apply effect to/from a repo
 
@@ -378,48 +388,49 @@
 msg.append(' with-transaction')
 msg.append('\n')
 repo.ui.debug(''.join(msg))
-iterparts = enumerate(unbundler.iterparts())
-part = None
-nbpart = 0
-try:
-for nbpart, part in iterparts:
-_processpart(op, part)
-except Exception as exc:
-# Any exceptions seeking to the end of the bundle at this point are
-# almost certainly related to the underlying stream being bad.
-# And, chances are that the exception we're handling is related to
-# getting in that bad state. So, we swallow the seeking error and
-# re-raise the original error.
-seekerror = False
+
+with partiterator(unbundler) as parts:
+part = None
+nbpart = 0
 try:
-for nbpart, part in iterparts:
-# consume the bundle content
-part.seek(0, 2)
-except Exception:
-seekerror = True
+for nbpart, part in parts:
+_processpart(op, part)
+except Exception as exc:
+# Any exceptions seeking to the end of the bundle at this point are
+# almost certainly related to the underlying stream being bad.
+# And, chances are that the exception we're handling is related to
+# getting in that bad state. So, we swallow the seeking error and
+# re-raise the original error.
+seekerror = False
+try:
+for nbpart, part in parts:
+# consume the bundle content
+part.seek(0, 2)
+except Exception:
+seekerror = True
 
-# Small hack to let caller code distinguish exceptions from bundle2
-# processing from processing the old format. This is mostly
-# needed to handle different return codes to unbundle according to the
-# type of bundle. We should probably clean up or drop this return code
-# craziness in a future version.
-exc.duringunbundle2 = True
-salvaged = []
-replycaps = None
-if op.reply is not None:
-salvaged = op.reply.salvageoutput()
-replycaps = op.reply.capabilities
-exc._replycaps = replycaps
-exc._bundle2salvagedoutput = salvaged
+# Small hack to let caller code distinguish exceptions from bundle2
+# processing from processing the old format. This is mostly needed
+# to handle different return codes to unbundle according to the 
type
+# of bundle. We should probably clean up or drop this return code
+# craziness in a future version.
+exc.duringunbundle2 = True
+salvaged = []
+replycaps = None
+if op.reply is not None:
+salvaged = op.reply.salvageoutput()
+replycaps = op.reply.capabilities
+exc._replycaps = replycaps
+exc._bundle2salvagedoutput = salvaged
 
-# Re-raising from a variable loses the original stack. So only use
-# that form if we need to.
-if seekerror:
-raise exc
-else:
-raise
-finally:
-repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
+# Re-raising from a variable loses the original stack. So only use
+# that form if we need to.
+if seekerror:
+raise exc
+else:
+raise
+finally:
+repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
 
 return op
 



To: durham, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D706: bundle2: move processpart stream maintenance into part iterator

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The processpart function also did some stream maintenance, so let's move it to
  the part iterator as well, as part of moving all part iteration logic into the
  class.
  
  There is one place processpart is called outside of the normal loop, so we
  manually handle the seek there.
  
  The now-empty try/finally will be removed in a later patch, for ease of 
review.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D706

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -354,21 +354,32 @@
 self.unbundler = unbundler
 self.iterator = None
 self.count = 0
+self.current = None
 
 def __enter__(self):
 def func():
 itr = enumerate(self.unbundler.iterparts())
 for count, p in itr:
 self.count = count
+self.current = p
 yield p
+p.seek(0, 2)
+self.current = None
 self.iterator = func()
 return self.iterator
 
 def __exit__(self, type, exc, tb):
 if not self.iterator:
 return
 
 if exc:
+# If exiting or interrupted, do not attempt to seek the stream in
+# the finally block below. This makes abort faster.
+if (self.current and
+not isinstance(exc, (SystemExit, KeyboardInterrupt))):
+# consume the part content to not corrupt the stream.
+self.current.seek(0, 2)
+
 # Any exceptions seeking to the end of the bundle at this point are
 # almost certainly related to the underlying stream being bad.
 # And, chances are that the exception we're handling is related to
@@ -455,7 +466,6 @@
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
 status = 'unknown' # used by debug output
-hardabort = False
 try:
 try:
 handler = parthandlermapping.get(part.type)
@@ -511,15 +521,8 @@
mandatory=False)
 outpart.addparam(
 'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
-# If exiting or interrupted, do not attempt to seek the stream in the
-# finally block below. This makes abort faster.
-except (SystemExit, KeyboardInterrupt):
-hardabort = True
-raise
 finally:
-# consume the part content to not corrupt the stream.
-if not hardabort:
-part.seek(0, 2)
+pass
 
 
 def decodecaps(blob):
@@ -1143,7 +1146,10 @@
 return
 part = unbundlepart(self.ui, headerblock, self._fp)
 op = interruptoperation(self.ui)
-_processpart(op, part)
+try:
+_processpart(op, part)
+finally:
+part.seek(0, 2)
 self.ui.debug('bundle2-input-stream-interrupt:'
   ' closing out of band context\n')
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D708: bundle2: remove unnecessary try finally

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is no longer needed.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D708

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -505,32 +505,28 @@
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-try:
-handler = _gethandler(op, part)
-if handler is None:
-return
+handler = _gethandler(op, part)
+if handler is None:
+return
 
-# handler is called outside the above try block so that we don't
-# risk catching KeyErrors from anything other than the
-# parthandlermapping lookup (any KeyError raised by handler()
-# itself represents a defect of a different variety).
-output = None
-if op.captureoutput and op.reply is not None:
-op.ui.pushbuffer(error=True, subproc=True)
-output = ''
-try:
-handler(op, part)
-finally:
-if output is not None:
-output = op.ui.popbuffer()
-if output:
-outpart = op.reply.newpart('output', data=output,
-   mandatory=False)
-outpart.addparam(
-'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
+# handler is called outside the above try block so that we don't
+# risk catching KeyErrors from anything other than the
+# parthandlermapping lookup (any KeyError raised by handler()
+# itself represents a defect of a different variety).
+output = None
+if op.captureoutput and op.reply is not None:
+op.ui.pushbuffer(error=True, subproc=True)
+output = ''
+try:
+handler(op, part)
 finally:
-pass
-
+if output is not None:
+output = op.ui.popbuffer()
+if output:
+outpart = op.reply.newpart('output', data=output,
+   mandatory=False)
+outpart.addparam(
+'in-reply-to', pycompat.bytestr(part.id), mandatory=False)
 
 def decodecaps(blob):
 """decode a bundle2 caps bytes blob into a dictionary



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D703: bundle2: move part iterator a separate class

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Currently, the part iterator logic is tightly coupled with the part handling
  logic, which means it's hard to replace the part handling logic without
  duplicating the part iterator bits.
  
  In a future diff we'll want to be able to replace all part handling, so let's
  begin refactoring the part iterator logic to it's own class.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D703

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -347,6 +347,16 @@
 _processchangegroup(op, unbundler, tr, source, url, **kwargs)
 return op
 
+class partiterator(object):
+def __init__(self, unbundler):
+self.unbundler = unbundler
+
+def __enter__(self):
+return enumerate(self.unbundler.iterparts())
+
+def __exit__(self, type, value, tb):
+pass
+
 def processbundle(repo, unbundler, transactiongetter=None, op=None):
 """This function process a bundle, apply effect to/from a repo
 
@@ -378,48 +388,49 @@
 msg.append(' with-transaction')
 msg.append('\n')
 repo.ui.debug(''.join(msg))
-iterparts = enumerate(unbundler.iterparts())
-part = None
-nbpart = 0
-try:
-for nbpart, part in iterparts:
-_processpart(op, part)
-except Exception as exc:
-# Any exceptions seeking to the end of the bundle at this point are
-# almost certainly related to the underlying stream being bad.
-# And, chances are that the exception we're handling is related to
-# getting in that bad state. So, we swallow the seeking error and
-# re-raise the original error.
-seekerror = False
+
+with partiterator(unbundler) as parts:
+part = None
+nbpart = 0
 try:
-for nbpart, part in iterparts:
-# consume the bundle content
-part.seek(0, 2)
-except Exception:
-seekerror = True
+for nbpart, part in parts:
+_processpart(op, part)
+except Exception as exc:
+# Any exceptions seeking to the end of the bundle at this point are
+# almost certainly related to the underlying stream being bad.
+# And, chances are that the exception we're handling is related to
+# getting in that bad state. So, we swallow the seeking error and
+# re-raise the original error.
+seekerror = False
+try:
+for nbpart, part in parts:
+# consume the bundle content
+part.seek(0, 2)
+except Exception:
+seekerror = True
 
-# Small hack to let caller code distinguish exceptions from bundle2
-# processing from processing the old format. This is mostly
-# needed to handle different return codes to unbundle according to the
-# type of bundle. We should probably clean up or drop this return code
-# craziness in a future version.
-exc.duringunbundle2 = True
-salvaged = []
-replycaps = None
-if op.reply is not None:
-salvaged = op.reply.salvageoutput()
-replycaps = op.reply.capabilities
-exc._replycaps = replycaps
-exc._bundle2salvagedoutput = salvaged
+# Small hack to let caller code distinguish exceptions from bundle2
+# processing from processing the old format. This is mostly needed
+# to handle different return codes to unbundle according to the 
type
+# of bundle. We should probably clean up or drop this return code
+# craziness in a future version.
+exc.duringunbundle2 = True
+salvaged = []
+replycaps = None
+if op.reply is not None:
+salvaged = op.reply.salvageoutput()
+replycaps = op.reply.capabilities
+exc._replycaps = replycaps
+exc._bundle2salvagedoutput = salvaged
 
-# Re-raising from a variable loses the original stack. So only use
-# that form if we need to.
-if seekerror:
-raise exc
-else:
-raise
-finally:
-repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
+# Re-raising from a variable loses the original stack. So only use
+# that form if we need to.
+if seekerror:
+raise exc
+else:
+raise
+finally:
+repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
 
 return op
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list

D707: bundle2: move handler validation out of processpart

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of refactoring bundle part processing let's move handler validation to
  its own function.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D707

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -460,48 +460,55 @@
 })
 return ret
 
+def _gethandler(op, part):
+status = 'unknown' # used by debug output
+try:
+handler = parthandlermapping.get(part.type)
+if handler is None:
+status = 'unsupported-type'
+raise error.BundleUnknownFeatureError(parttype=part.type)
+indebug(op.ui, 'found a handler for part %r' % part.type)
+unknownparams = part.mandatorykeys - handler.params
+if unknownparams:
+unknownparams = list(unknownparams)
+unknownparams.sort()
+status = 'unsupported-params (%s)' % unknownparams
+raise error.BundleUnknownFeatureError(parttype=part.type,
+  params=unknownparams)
+status = 'supported'
+except error.BundleUnknownFeatureError as exc:
+if part.mandatory: # mandatory parts
+raise
+indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
+return # skip to part processing
+finally:
+if op.ui.debugflag:
+msg = ['bundle2-input-part: "%s"' % part.type]
+if not part.mandatory:
+msg.append(' (advisory)')
+nbmp = len(part.mandatorykeys)
+nbap = len(part.params) - nbmp
+if nbmp or nbap:
+msg.append(' (params:')
+if nbmp:
+msg.append(' %i mandatory' % nbmp)
+if nbap:
+msg.append(' %i advisory' % nbmp)
+msg.append(')')
+msg.append(' %s\n' % status)
+op.ui.debug(''.join(msg))
+
+return handler
+
 def _processpart(op, part):
 """process a single part from a bundle
 
 The part is guaranteed to have been fully consumed when the function exits
 (even if an exception is raised)."""
-status = 'unknown' # used by debug output
 try:
-try:
-handler = parthandlermapping.get(part.type)
-if handler is None:
-status = 'unsupported-type'
-raise error.BundleUnknownFeatureError(parttype=part.type)
-indebug(op.ui, 'found a handler for part %r' % part.type)
-unknownparams = part.mandatorykeys - handler.params
-if unknownparams:
-unknownparams = list(unknownparams)
-unknownparams.sort()
-status = 'unsupported-params (%s)' % unknownparams
-raise error.BundleUnknownFeatureError(parttype=part.type,
-  params=unknownparams)
-status = 'supported'
-except error.BundleUnknownFeatureError as exc:
-if part.mandatory: # mandatory parts
-raise
-indebug(op.ui, 'ignoring unsupported advisory part %s' % exc)
-return # skip to part processing
-finally:
-if op.ui.debugflag:
-msg = ['bundle2-input-part: "%s"' % part.type]
-if not part.mandatory:
-msg.append(' (advisory)')
-nbmp = len(part.mandatorykeys)
-nbap = len(part.params) - nbmp
-if nbmp or nbap:
-msg.append(' (params:')
-if nbmp:
-msg.append(' %i mandatory' % nbmp)
-if nbap:
-msg.append(' %i advisory' % nbmp)
-msg.append(')')
-msg.append(' %s\n' % status)
-op.ui.debug(''.join(msg))
+handler = _gethandler(op, part)
+if handler is None:
+return
 
 # handler is called outside the above try block so that we don't
 # risk catching KeyErrors from anything other than the



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D704: bundle2: move part counter to partiterator

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of moving the part iterator logic to a separate class, let's move the
  part counting logic and the output for it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D704

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -348,14 +348,27 @@
 return op
 
 class partiterator(object):
-def __init__(self, unbundler):
+def __init__(self, repo, unbundler):
+self.repo = repo
 self.unbundler = unbundler
+self.iterator = None
+self.count = 0
 
 def __enter__(self):
-return enumerate(self.unbundler.iterparts())
+def func():
+itr = enumerate(self.unbundler.iterparts())
+for count, p in itr:
+self.count = count
+yield p
+self.iterator = func()
+return self.iterator
 
 def __exit__(self, type, value, tb):
-pass
+if not self.iterator:
+return
+
+self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
+   self.count)
 
 def processbundle(repo, unbundler, transactiongetter=None, op=None):
 """This function process a bundle, apply effect to/from a repo
@@ -389,11 +402,10 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
-with partiterator(unbundler) as parts:
+with partiterator(repo, unbundler) as parts:
 part = None
-nbpart = 0
 try:
-for nbpart, part in parts:
+for part in parts:
 _processpart(op, part)
 except Exception as exc:
 # Any exceptions seeking to the end of the bundle at this point are
@@ -403,7 +415,7 @@
 # re-raise the original error.
 seekerror = False
 try:
-for nbpart, part in parts:
+for part in parts:
 # consume the bundle content
 part.seek(0, 2)
 except Exception:
@@ -429,8 +441,6 @@
 raise exc
 else:
 raise
-finally:
-repo.ui.debug('bundle2-input-bundle: %i parts total\n' % nbpart)
 
 return op
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D709: bundle2: move part processing to a separate function

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that the part processing loop is tiny, let's move it to a separate 
function.
  This will allow extensions to completely replace the part processing logic,
  without having to replace the overall bundle processing logic or the stream
  maintenance logic.
  
  This will be useful for the infinitepush extension, so it can completely take
  over receiving a bundle and rerouting it to a side store. This will also make 
it
  easier to upstream the infinitepush functionality later.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D709

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -447,12 +447,15 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
+processparts(repo, op, unbundler)
+
+return op
+
+def processparts(repo, op, unbundler):
 with partiterator(repo, op, unbundler) as parts:
 for part in parts:
 _processpart(op, part)
 
-return op
-
 def _processchangegroup(op, cg, tr, source, url, **kwargs):
 ret = cg.apply(op.repo, tr, source, url, **kwargs)
 op.records.add('changegroup', {



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D705: bundle2: move exception handling into part iterator

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of separating the part iteration logic from the part handling logic,
  let's move the exception handling to the part iterator class.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D705

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -348,8 +348,9 @@
 return op
 
 class partiterator(object):
-def __init__(self, repo, unbundler):
+def __init__(self, repo, op, unbundler):
 self.repo = repo
+self.op = op
 self.unbundler = unbundler
 self.iterator = None
 self.count = 0
@@ -363,10 +364,43 @@
 self.iterator = func()
 return self.iterator
 
-def __exit__(self, type, value, tb):
+def __exit__(self, type, exc, tb):
 if not self.iterator:
 return
 
+if exc:
+# Any exceptions seeking to the end of the bundle at this point are
+# almost certainly related to the underlying stream being bad.
+# And, chances are that the exception we're handling is related to
+# getting in that bad state. So, we swallow the seeking error and
+# re-raise the original error.
+seekerror = False
+try:
+for part in self.iterator:
+# consume the bundle content
+part.seek(0, 2)
+except Exception:
+seekerror = True
+
+# Small hack to let caller code distinguish exceptions from bundle2
+# processing from processing the old format. This is mostly needed
+# to handle different return codes to unbundle according tothe type
+# of bundle. We should probably clean up or drop this return code
+# craziness in a future version.
+exc.duringunbundle2 = True
+salvaged = []
+replycaps = None
+if self.op.reply is not None:
+salvaged = self.op.reply.salvageoutput()
+replycaps = self.op.reply.capabilities
+exc._replycaps = replycaps
+exc._bundle2salvagedoutput = salvaged
+
+# Re-raising from a variable loses the original stack. So only use
+# that form if we need to.
+if seekerror:
+raise exc
+
 self.repo.ui.debug('bundle2-input-bundle: %i parts total\n' %
self.count)
 
@@ -402,45 +436,9 @@
 msg.append('\n')
 repo.ui.debug(''.join(msg))
 
-with partiterator(repo, unbundler) as parts:
-part = None
-try:
-for part in parts:
-_processpart(op, part)
-except Exception as exc:
-# Any exceptions seeking to the end of the bundle at this point are
-# almost certainly related to the underlying stream being bad.
-# And, chances are that the exception we're handling is related to
-# getting in that bad state. So, we swallow the seeking error and
-# re-raise the original error.
-seekerror = False
-try:
-for part in parts:
-# consume the bundle content
-part.seek(0, 2)
-except Exception:
-seekerror = True
-
-# Small hack to let caller code distinguish exceptions from bundle2
-# processing from processing the old format. This is mostly needed
-# to handle different return codes to unbundle according to the 
type
-# of bundle. We should probably clean up or drop this return code
-# craziness in a future version.
-exc.duringunbundle2 = True
-salvaged = []
-replycaps = None
-if op.reply is not None:
-salvaged = op.reply.salvageoutput()
-replycaps = op.reply.capabilities
-exc._replycaps = replycaps
-exc._bundle2salvagedoutput = salvaged
-
-# Re-raising from a variable loses the original stack. So only use
-# that form if we need to.
-if seekerror:
-raise exc
-else:
-raise
+with partiterator(repo, op, unbundler) as parts:
+for part in parts:
+_processpart(op, part)
 
 return op
 



To: durham, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D688: changegroup: remove changegroup dependency from revlog.addgroup

2017-09-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc8b6ed51386b: changegroup: remove changegroup dependency 
from revlog.addgroup (authored by durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D688?vs=1788=1790

REVISION DETAIL
  https://phab.mercurial-scm.org/D688

AFFECTED FILES
  mercurial/changegroup.py
  mercurial/revlog.py
  tests/test-revlog-raw.py

CHANGE DETAILS

diff --git a/tests/test-revlog-raw.py b/tests/test-revlog-raw.py
--- a/tests/test-revlog-raw.py
+++ b/tests/test-revlog-raw.py
@@ -119,11 +119,28 @@
 'deltabase': rlog.node(deltaparent),
 'delta': rlog.revdiff(deltaparent, r)}
 
+def deltaiter(self, linkmapper):
+chain = None
+for chunkdata in iter(lambda: self.deltachunk(chain), {}):
+node = chunkdata['node']
+p1 = chunkdata['p1']
+p2 = chunkdata['p2']
+cs = chunkdata['cs']
+deltabase = chunkdata['deltabase']
+delta = chunkdata['delta']
+flags = chunkdata['flags']
+
+link = linkmapper(cs)
+chain = node
+
+yield (node, p1, p2, link, deltabase, delta, flags)
+
 def linkmap(lnode):
 return rlog.rev(lnode)
 
 dlog = newrevlog(destname, recreate=True)
-dlog.addgroup(dummychangegroup(), linkmap, tr)
+dummydeltas = dummychangegroup().deltaiter(linkmap)
+dlog.addgroup(dummydeltas, tr)
 return dlog
 
 def lowlevelcopy(rlog, tr, destname=b'_destrevlog.i'):
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1872,7 +1872,7 @@
 ifh.write(data[1])
 self.checkinlinesize(transaction, ifh)
 
-def addgroup(self, cg, linkmapper, transaction, addrevisioncb=None):
+def addgroup(self, deltas, transaction, addrevisioncb=None):
 """
 add a delta group
 
@@ -1905,20 +1905,12 @@
 ifh.flush()
 try:
 # loop through our set of deltas
-chain = None
-for chunkdata in iter(lambda: cg.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags'] or REVIDX_DEFAULT_FLAGS
+for data in deltas:
+node, p1, p2, link, deltabase, delta, flags = data
+flags = flags or REVIDX_DEFAULT_FLAGS
 
 nodes.append(node)
-chain = node
 
-link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
 continue
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -245,7 +245,8 @@
 # no new manifest will be created and the manifest group will
 # be empty during the pull
 self.manifestheader()
-repo.manifestlog._revlog.addgroup(self, revmap, trp)
+deltas = self.deltaiter(revmap)
+repo.manifestlog._revlog.addgroup(deltas, trp)
 repo.ui.progress(_('manifests'), None)
 self.callback = None
 
@@ -307,7 +308,8 @@
 efiles.update(cl.readfiles(node))
 
 self.changelogheader()
-cgnodes = cl.addgroup(self, csmap, trp, addrevisioncb=onchangelog)
+deltas = self.deltaiter(csmap)
+cgnodes = cl.addgroup(deltas, trp, addrevisioncb=onchangelog)
 efiles = len(efiles)
 
 if not cgnodes:
@@ -428,6 +430,27 @@
 ret = deltaheads + 1
 return ret
 
+def deltaiter(self, linkmapper):
+"""
+returns an iterator of the deltas in this changegroup
+
+Useful for passing to the underlying storage system to be stored.
+"""
+chain = None
+for chunkdata in iter(lambda: self.deltachunk(chain), {}):
+node = chunkdata['node']
+p1 = chunkdata['p1']
+p2 = chunkdata['p2']
+cs = chunkdata['cs']
+deltabase = chunkdata['deltabase']
+delta = chunkdata['delta']
+flags = chunkdata['flags']
+
+link = linkmapper(cs)
+chain = node
+
+yield (node, p1, p2, link, deltabase, delta, flags)
+
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.
 
@@ -468,7 +491,8 @@
 d = chunkdata["filename"]
 repo.ui.debug("adding %s revisions\n" % d)
 dirlog = repo.manifestlog._revlog.dirlog(d)
-if not dirlog.addgroup(self, revmap, trp):
+deltas =

D699: revlog: refactor chain variable

2017-09-13 Thread durham (Durham Goode) 
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb96cfc309ac5: revlog: refactor chain variable (authored by 
durham, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D699?vs=1787=1789

REVISION DETAIL
  https://phab.mercurial-scm.org/D699

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1916,11 +1916,11 @@
 flags = chunkdata['flags'] or REVIDX_DEFAULT_FLAGS
 
 nodes.append(node)
+chain = node
 
 link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
-chain = node
 continue
 
 for p in (p1, p2):
@@ -1954,13 +1954,13 @@
 # We're only using addgroup() in the context of changegroup
 # generation so the revision data can always be handled as raw
 # by the flagprocessor.
-chain = self._addrevision(node, None, transaction, link,
-  p1, p2, flags, (baserev, delta),
-  ifh, dfh,
-  alwayscache=bool(addrevisioncb))
+self._addrevision(node, None, transaction, link,
+  p1, p2, flags, (baserev, delta),
+  ifh, dfh,
+  alwayscache=bool(addrevisioncb))
 
 if addrevisioncb:
-addrevisioncb(self, chain)
+addrevisioncb(self, node)
 
 if not dfh and not self._inline:
 # addrevision switched from inline to conventional



To: durham, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D699: revlog: refactor chain variable

2017-09-13 Thread durham (Durham Goode) 
durham created this revision.
Herald added a reviewer: indygreg.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously the addgroup loop would set chain to be the result of
  self._addrevision(node,...). Since _addrevision now always returns the passed 
in
  node, we can drop that behavior and just always set chain = node in the loop.
  
  This will be useful in a future patch where we refactor the cg.deltachunk 
logic
  to another function and therefore chain disappears entirely from this 
function.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D699

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1916,11 +1916,11 @@
 flags = chunkdata['flags'] or REVIDX_DEFAULT_FLAGS
 
 nodes.append(node)
+chain = node
 
 link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
-chain = node
 continue
 
 for p in (p1, p2):
@@ -1954,13 +1954,13 @@
 # We're only using addgroup() in the context of changegroup
 # generation so the revision data can always be handled as raw
 # by the flagprocessor.
-chain = self._addrevision(node, None, transaction, link,
-  p1, p2, flags, (baserev, delta),
-  ifh, dfh,
-  alwayscache=bool(addrevisioncb))
+self._addrevision(node, None, transaction, link,
+  p1, p2, flags, (baserev, delta),
+  ifh, dfh,
+  alwayscache=bool(addrevisioncb))
 
 if addrevisioncb:
-addrevisioncb(self, chain)
+addrevisioncb(self, node)
 
 if not dfh and not self._inline:
 # addrevision switched from inline to conventional



To: durham, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D688: changegroup: remove changegroup dependency from revlog.addgroup

2017-09-13 Thread durham (Durham Goode) 
durham updated this revision to Diff 1788.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D688?vs=1745=1788

REVISION DETAIL
  https://phab.mercurial-scm.org/D688

AFFECTED FILES
  mercurial/changegroup.py
  mercurial/revlog.py
  tests/test-revlog-raw.py

CHANGE DETAILS

diff --git a/tests/test-revlog-raw.py b/tests/test-revlog-raw.py
--- a/tests/test-revlog-raw.py
+++ b/tests/test-revlog-raw.py
@@ -119,11 +119,28 @@
 'deltabase': rlog.node(deltaparent),
 'delta': rlog.revdiff(deltaparent, r)}
 
+def deltaiter(self, linkmapper):
+chain = None
+for chunkdata in iter(lambda: self.deltachunk(chain), {}):
+node = chunkdata['node']
+p1 = chunkdata['p1']
+p2 = chunkdata['p2']
+cs = chunkdata['cs']
+deltabase = chunkdata['deltabase']
+delta = chunkdata['delta']
+flags = chunkdata['flags']
+
+link = linkmapper(cs)
+chain = node
+
+yield (node, p1, p2, link, deltabase, delta, flags)
+
 def linkmap(lnode):
 return rlog.rev(lnode)
 
 dlog = newrevlog(destname, recreate=True)
-dlog.addgroup(dummychangegroup(), linkmap, tr)
+dummydeltas = dummychangegroup().deltaiter(linkmap)
+dlog.addgroup(dummydeltas, tr)
 return dlog
 
 def lowlevelcopy(rlog, tr, destname=b'_destrevlog.i'):
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1872,7 +1872,7 @@
 ifh.write(data[1])
 self.checkinlinesize(transaction, ifh)
 
-def addgroup(self, cg, linkmapper, transaction, addrevisioncb=None):
+def addgroup(self, deltas, transaction, addrevisioncb=None):
 """
 add a delta group
 
@@ -1905,20 +1905,12 @@
 ifh.flush()
 try:
 # loop through our set of deltas
-chain = None
-for chunkdata in iter(lambda: cg.deltachunk(chain), {}):
-node = chunkdata['node']
-p1 = chunkdata['p1']
-p2 = chunkdata['p2']
-cs = chunkdata['cs']
-deltabase = chunkdata['deltabase']
-delta = chunkdata['delta']
-flags = chunkdata['flags'] or REVIDX_DEFAULT_FLAGS
+for data in deltas:
+node, p1, p2, link, deltabase, delta, flags = data
+flags = flags or REVIDX_DEFAULT_FLAGS
 
 nodes.append(node)
-chain = node
 
-link = linkmapper(cs)
 if node in self.nodemap:
 # this can happen if two branches make the same change
 continue
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -245,7 +245,8 @@
 # no new manifest will be created and the manifest group will
 # be empty during the pull
 self.manifestheader()
-repo.manifestlog._revlog.addgroup(self, revmap, trp)
+deltas = self.deltaiter(revmap)
+repo.manifestlog._revlog.addgroup(deltas, trp)
 repo.ui.progress(_('manifests'), None)
 self.callback = None
 
@@ -307,7 +308,8 @@
 efiles.update(cl.readfiles(node))
 
 self.changelogheader()
-cgnodes = cl.addgroup(self, csmap, trp, addrevisioncb=onchangelog)
+deltas = self.deltaiter(csmap)
+cgnodes = cl.addgroup(deltas, trp, addrevisioncb=onchangelog)
 efiles = len(efiles)
 
 if not cgnodes:
@@ -428,6 +430,27 @@
 ret = deltaheads + 1
 return ret
 
+def deltaiter(self, linkmapper):
+"""
+returns an iterator of the deltas in this changegroup
+
+Useful for passing to the underlying storage system to be stored.
+"""
+chain = None
+for chunkdata in iter(lambda: self.deltachunk(chain), {}):
+node = chunkdata['node']
+p1 = chunkdata['p1']
+p2 = chunkdata['p2']
+cs = chunkdata['cs']
+deltabase = chunkdata['deltabase']
+delta = chunkdata['delta']
+flags = chunkdata['flags']
+
+link = linkmapper(cs)
+chain = node
+
+yield (node, p1, p2, link, deltabase, delta, flags)
+
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.
 
@@ -468,7 +491,8 @@
 d = chunkdata["filename"]
 repo.ui.debug("adding %s revisions\n" % d)
 dirlog = repo.manifestlog._revlog.dirlog(d)
-if not dirlog.addgroup(self, revmap, trp):
+deltas = self.deltaiter(revmap)
+if not dirlog.addgroup(deltas, trp):
 raise error.Abort(_("received dir revlog group is empty"))
 
 class headerlessfixup(object):
@@

  1   2   >