D2010: check-commit: allow foo_bar naming in functions

[indygreg (Gregory Szorc)]

indygreg edited the summary of this revision.
indygreg updated this revision to Diff 16959.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2010?vs=5162=16959

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D2010/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D2010

AFFECTED FILES
  contrib/check-code.py
  contrib/check-commit
  tests/test-contrib-check-commit.t

CHANGE DETAILS

diff --git a/tests/test-contrib-check-commit.t 
b/tests/test-contrib-check-commit.t
--- a/tests/test-contrib-check-commit.t
+++ b/tests/test-contrib-check-commit.t
@@ -130,6 +130,4 @@
This has no topic and ends with a period.
   7: don't add trailing period on summary line
This has no topic and ends with a period.
-  20: adds a function with foo_bar naming
-   + def blah_blah(x):
   [1]
diff --git a/contrib/check-commit b/contrib/check-commit
--- a/contrib/check-commit
+++ b/contrib/check-commit
@@ -39,12 +39,6 @@
  "summary keyword should be most user-relevant one-word command or topic"),
 (afterheader + r".*\.\s*\n", "don't add trailing period on summary line"),
 (afterheader + r".{79,}", "summary line too long (limit is 78)"),
-# Forbid "_" in function name.
-#
-# We skip the check for cffi related functions. They use names mapping the
-# name of the C function. C function names may contain "_".
-(r"\n\+[ \t]+def (?!cffi)[a-z]+_[a-z]",
- "adds a function with foo_bar naming"),
 ]
 
 word = re.compile(r'\S')
diff --git a/contrib/check-code.py b/contrib/check-code.py
--- a/contrib/check-code.py
+++ b/contrib/check-code.py
@@ -340,8 +340,6 @@
 ),
 (r'[^\n]\Z', "no trailing newline"),
 (r'(\S[ \t]+|^[ \t]+)\n', "trailing whitespace"),
-#(r'^\s+[^_ \n][^_. \n]+_[^_\n]+\s*=',
-# "don't use underbars in identifiers"),
 (
 r'^\s+(self\.)?[A-Za-z][a-z0-9]+[A-Z]\w* = ',
 "don't use camelcase in identifiers",



To: indygreg, #hg-reviewers, pulkit, durin42, martinvonz, marmoute
Cc: martinvonz, av6, yuja, durin42, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7021: notify: cast hash to bytes

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is needed to avoid a str/bytes mismatch when interpolating a
  line or 2 later.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7021

AFFECTED FILES
  hgext/notify.py

CHANGE DETAILS

diff --git a/hgext/notify.py b/hgext/notify.py
--- a/hgext/notify.py
+++ b/hgext/notify.py
@@ -161,6 +161,7 @@
 logcmdutil,
 mail,
 patch,
+pycompat,
 registrar,
 util,
 )
@@ -559,7 +560,10 @@
 host = encoding.strtolocal(socket.getfqdn())
 if messageidseed:
 messagehash = hashlib.sha512(ctx.hex() + messageidseed)
-messageid = b'' % (messagehash.hexdigest()[:64], host)
+messageid = b'' % (
+pycompat.sysbytes(messagehash.hexdigest()[:64]),
+host,
+)
 else:
 messageid = b'' % (
 ctx,



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7020: pycompat: implement a shlexquote that properly handles bytes

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  2cc453284d5 
 
introduced this function call to for mail.py. This broke Python
  3 because shlex.quote() expects str, not bytes. In order to unbust it,
  we need to normalize bytes to str then go back to bytes to appease the
  caller.
  
  This is a bit ugly. But I don't see any other obvious solution.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7020

AFFECTED FILES
  mercurial/pycompat.py

CHANGE DETAILS

diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -337,7 +337,11 @@
 ret = shlex.split(s.decode('latin-1'), comments, posix)
 return [a.encode('latin-1') for a in ret]
 
-shlexquote = shlex.quote
+def shlexquote(s):
+s = s.decode('latin-1')
+s = shlex.quote(s)
+return s.encode('latin-1')
+
 iteritems = lambda x: x.items()
 itervalues = lambda x: x.values()
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6876: phabricator: support automatically obsoleting old revisions of pulled commits

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D6876#102925 , @marmoute 
wrote:
  
  > The feature seens pretty usful, but is also a potential foot-gun/data-loss 
engine. I think it is useful to take the feature, but maybe with proper 
documentaiton warning and turned of by efaut. I made a couple of comment about 
the implementation.
  
  I basically agree with the inline comments.  This was an import with minimal 
changes as a baseline, and I didn't want to spend time making a bunch of follow 
ups until I gauged interest in this.
  
  Alternately, if there's concern over obsolete marker growth, should this use 
the archived phase instead?  Since the use case is mostly around stuff a 
developer imports locally, I don't see much value in those markers being 
exchangeable.  The only potential hole I see is if someone has pushed this to a 
non publishing repo before pulling.  Then everyone else could pull the (not 
obsoleted) commit.  But then //they// would archive it on pull too, instead of 
N developers potentially creating N markers for the same commit.

INLINE COMMENTS

> marmoute wrote in phabricator.py:187
> So, this only checks the diff number, right ? So in theory, we could obsolete 
> a later version of this, silently dropping change (instead of detecting 
> potential phase-divergence)

It looks like, based on the loop where it's used, it will obsolete //all// (non 
public) commits that match.  The only way I can think of where that happens is 
if I `phabimport` someone else's patches, they make updates, and then I 
`phabimport` again.  That's why I was wondering aloud about doing this on 
import, but if the eventual pull cleans it all up, that may be sufficient for 
the use case I had in mind.  I don't see how a developer would do that on their 
own commits with amend and friends.

Do you have any ideas to detect phase divergence?  And can that even happen if 
we're ignoring public commits?

> marmoute wrote in phabricator.py:192
> It would probably make sense to wrap the pull exchange logic instead. It 
> would catch more instance and could reuse the same transaction.

I wonder if it would be better in some txn hook.  And then it could (I think) 
share the existing transaction and lock.

> marmoute wrote in phabricator.py:216
> This should be `non public()` instead of `draft()` these could be draft.

Agreed.

I saw mention of the archived phase the other day; how would that (and other 
special phases) play here?

> marmoute wrote in phabricator.py:225
> We should do the computation before locking to avoid other process updating 
> the repo under our feet.

You mean //after// locking, correct?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6876/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6876

To: mharbison72, #hg-reviewers
Cc: pulkit, sheehan, Kwan, mercurial-devel, marmoute
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6987: strip: move strip extension to core as debugstrip

[marmoute (Pierre-Yves David)]

marmoute added inline comments.

INLINE COMMENTS

> strip.py:1-47
>  """strip changesets and their descendants from history
>  
> -This extension allows you to strip changesets and all their descendants from 
> the
> -repository. See the command help for details.
> +strip extension has been renamed to debugstrip and moved to core. However,
> +this extension is to preserve the old `strip` name forusers that are used
> +to that.
>  """
> +

Why do we need that much code to remain into `hgext/strip.py`. What I would 
expect to see is a very small file that only register debugstrip as `strip`.

> strip.py:24-34
> +[
> + ('r', 'rev', [], _('strip specified revision (optional, can specify '
> +'revisions without this option)'), _('REV')),
> + ('f', 'force', None, _('force removal of changesets, discard 
> uncommitted'
> +' changes (no backup)')),
> + ('', 'no-backup', None, _('do not save backup bundle')),
> + ('', 'nobackup', None, _('do not save backup bundle (DEPRECATED)')),

What is going on here? IT seems like the same code with a different formatting.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6987/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6987

To: navaneeth.suresh, durin42, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7003: grep: put --diff in the first line of usage

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  This does not seems to be the first line you are looking for. The first line 
we are looking for is the `usage:` line. It is defined lower in the decorator :
  
_(b'[OPTION]... PATTERN [FILE]...'),
  
  We need to gain a `[--diff]` here.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7003/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7003

To: navaneeth.suresh, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7001: share: unmark --relative as EXPERIMENTAL

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute accepted this revision.


  Looks good to me

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7001/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7001

To: pulkit, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6939: sidedata: apply basic but tight security around exchange

[marmoute (Pierre-Yves David)]

marmoute added inline comments.
marmoute marked an inline comment as done.

INLINE COMMENTS

> martinvonz wrote in bundle2.py:1837-1838
> Isn't `bool()` redundant here?

It is redundant, but helping Raphaël work on Rust got me scared about boolean. 
I'll clean it up (either resend or followup)

> indygreg wrote in bundle2.py:2343
> Shouldn't this by `exp-sidedata`?

Hum, yeah. That would be safer. Updated code coming

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6939/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6939

To: marmoute, #hg-reviewers
Cc: indygreg, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6876: phabricator: support automatically obsoleting old revisions of pulled commits

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  The feature seens pretty usful, but is also a potential foot-gun/data-loss 
engine. I think it is useful to take the feature, but maybe with proper 
documentaiton warning and turned of by efaut. I made a couple of comment about 
the implementation.

INLINE COMMENTS

> phabricator.py:187
> +if m:
> +return b"D%s" % m.group(r'id')
> +

So, this only checks the diff number, right ? So in theory, we could obsolete a 
later version of this, silently dropping change (instead of detecting potential 
phase-divergence)

> phabricator.py:192
> +@eh.wrapcommand(b"pull")
> +def _pull(orig, ui, repo, *args, **opts):
> +if not (obsolete.isenabled(repo, obsolete.createmarkersopt)

It would probably make sense to wrap the pull exchange logic instead. It would 
catch more instance and could reuse the same transaction.

> phabricator.py:216
> +unfiltered = repo.unfiltered()
> +for rev in unfiltered.revs("draft() - obsolete()"):
> +n = unfiltered[rev]

This should be `non public()` instead of `draft()` these could be draft.

> phabricator.py:225
> +
> +with unfiltered.lock(), unfiltered.transaction('phabpullcreatemarkers'):
> +obsolete.createmarkers(unfiltered, tocreate)

We should do the computation before locking to avoid other process updating the 
repo under our feet.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6876/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6876

To: mharbison72, #hg-reviewers
Cc: pulkit, sheehan, Kwan, mercurial-devel, marmoute
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6954: sidedatacopies: move various copies related function to the copies modules

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16955.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6954?vs=16786=16955

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6954/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6954

AFFECTED FILES
  mercurial/changelog.py
  mercurial/context.py
  mercurial/copies.py
  mercurial/scmutil.py

CHANGE DETAILS

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -2218,23 +2218,3 @@
 mark,
 mark,
 )
-
-
-def computechangesetfilesadded(ctx):
-"""return the list of files added in a changeset
-"""
-added = []
-for f in ctx.files():
-if not any(f in p for p in ctx.parents()):
-added.append(f)
-return added
-
-
-def computechangesetfilesremoved(ctx):
-"""return the list of files removed in a changeset
-"""
-removed = []
-for f in ctx.files():
-if f not in ctx:
-removed.append(f)
-return removed
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -14,6 +14,7 @@
 from .i18n import _
 
 from . import (
+error,
 match as matchmod,
 node,
 pathutil,
@@ -854,6 +855,26 @@
 wctx[dst].markcopied(src)
 
 
+def computechangesetfilesadded(ctx):
+"""return the list of files added in a changeset
+"""
+added = []
+for f in ctx.files():
+if not any(f in p for p in ctx.parents()):
+added.append(f)
+return added
+
+
+def computechangesetfilesremoved(ctx):
+"""return the list of files removed in a changeset
+"""
+removed = []
+for f in ctx.files():
+if f not in ctx:
+removed.append(f)
+return removed
+
+
 def computechangesetcopies(ctx):
 """return the copies data for a changeset
 
@@ -878,3 +899,58 @@
 elif src in p2 and p2[src].filenode() == srcnode:
 p2copies[dst] = src
 return p1copies, p2copies
+
+
+def encodecopies(files, copies):
+items = []
+for i, dst in enumerate(files):
+if dst in copies:
+items.append(b'%d\0%s' % (i, copies[dst]))
+if len(items) != len(copies):
+raise error.ProgrammingError(
+b'some copy targets missing from file list'
+)
+return b"\n".join(items)
+
+
+def decodecopies(files, data):
+try:
+copies = {}
+if not data:
+return copies
+for l in data.split(b'\n'):
+strindex, src = l.split(b'\0')
+i = int(strindex)
+dst = files[i]
+copies[dst] = src
+return copies
+except (ValueError, IndexError):
+# Perhaps someone had chosen the same key name (e.g. "p1copies") and
+# used different syntax for the value.
+return None
+
+
+def encodefileindices(files, subset):
+subset = set(subset)
+indices = []
+for i, f in enumerate(files):
+if f in subset:
+indices.append(b'%d' % i)
+return b'\n'.join(indices)
+
+
+def decodefileindices(files, data):
+try:
+subset = []
+if not data:
+return subset
+for strindex in data.split(b'\n'):
+i = int(strindex)
+if i < 0 or i >= len(files):
+return None
+subset.append(files[i])
+return subset
+except (ValueError, IndexError):
+# Perhaps someone had chosen the same key name (e.g. "added") and
+# used different syntax for the value.
+return None
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -544,9 +544,9 @@
 filesadded = []
 elif source == b'compatibility':
 if filesadded is None:
-filesadded = scmutil.computechangesetfilesadded(self)
+filesadded = copies.computechangesetfilesadded(self)
 else:
-filesadded = scmutil.computechangesetfilesadded(self)
+filesadded = copies.computechangesetfilesadded(self)
 return filesadded
 
 def filesremoved(self):
@@ -561,9 +561,9 @@
 filesremoved = []
 elif source == b'compatibility':
 if filesremoved is None:
-filesremoved = scmutil.computechangesetfilesremoved(self)
+filesremoved = copies.computechangesetfilesremoved(self)
 else:
-filesremoved = scmutil.computechangesetfilesremoved(self)
+filesremoved = copies.computechangesetfilesremoved(self)
 return filesremoved
 
 @propertycache
diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -16,6 +16,7 @@
 from .thirdparty import attr
 
 from . import (
+copies,
 encoding,
 error,
 

D6953: sidedatacopies: read rename information from sidedata

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16954.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6953?vs=16785=16954

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6953/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6953

AFFECTED FILES
  mercurial/changelog.py
  mercurial/context.py
  mercurial/copies.py
  tests/test-copies-unrelated.t
  tests/test-copies.t

CHANGE DETAILS

diff --git a/tests/test-copies.t b/tests/test-copies.t
--- a/tests/test-copies.t
+++ b/tests/test-copies.t
@@ -309,7 +309,6 @@
   x -> z
   $ hg debugpathcopies 0 2
   x -> z (filelog !)
-  x -> z (sidedata !)
 
 Copy file that exists on both sides of the merge, different content
   $ newrepo
@@ -338,12 +337,14 @@
  x
   $ hg debugp1copies -r 2
   x -> z (changeset !)
+  x -> z (sidedata !)
   $ hg debugp2copies -r 2
-  x -> z (no-changeset !)
+  x -> z (no-changeset no-sidedata !)
   $ hg debugpathcopies 1 2
   x -> z (changeset !)
+  x -> z (sidedata !)
   $ hg debugpathcopies 0 2
-  x -> z (no-changeset !)
+  x -> z (no-changeset no-sidedata !)
 
 Copy x->y on one side of merge and copy x->z on the other side. Pathcopies 
from one parent
 of the merge to the merge should include the copy from the other side.
@@ -403,7 +404,7 @@
   $ hg debugpathcopies 2 3
   y -> z
   $ hg debugpathcopies 1 3
-  y -> z (no-filelog no-sidedata !)
+  y -> z (no-filelog !)
 
 Create x and y, then rename x to z on one side of merge, and rename y to z and
 modify z on the other side. When storing copies in the changeset, we don't
@@ -448,18 +449,18 @@
   o  0 add x and y
  x y
   $ hg debugpathcopies 1 4
-  y -> z (no-filelog no-sidedata !)
+  y -> z (no-filelog !)
   $ hg debugpathcopies 2 4
-  x -> z (no-filelog no-sidedata !)
+  x -> z (no-filelog !)
   $ hg debugpathcopies 0 4
   x -> z (filelog !)
-  x -> z (sidedata !)
+  y -> z (sidedata !)
   y -> z (compatibility !)
   y -> z (changeset !)
   $ hg debugpathcopies 1 5
-  y -> z (no-filelog no-sidedata !)
+  y -> z (no-filelog !)
   $ hg debugpathcopies 2 5
-  x -> z (no-filelog no-sidedata !)
+  x -> z (no-filelog !)
   $ hg debugpathcopies 0 5
   x -> z
 
diff --git a/tests/test-copies-unrelated.t b/tests/test-copies-unrelated.t
--- a/tests/test-copies-unrelated.t
+++ b/tests/test-copies-unrelated.t
@@ -179,8 +179,8 @@
   o  0 add x
  x
   $ hg debugpathcopies 0 5
-  x -> y (no-filelog no-sidedata !)
-#if no-filelog no-sidedata
+  x -> y (no-filelog !)
+#if no-filelog
   $ hg graft -r 2
   grafting 2:* "modify x again" (glob)
   merging y and x to y
@@ -347,8 +347,8 @@
   o  0 base
  a
   $ hg debugpathcopies 1 5
-  x -> y (no-filelog no-sidedata !)
-#if no-filelog no-sidedata
+  x -> y (no-filelog !)
+#if no-filelog
   $ hg graft -r 2
   grafting 2:* "modify x" (glob)
   merging y and x to y
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -187,6 +187,8 @@
 
 def usechangesetcentricalgo(repo):
 """Checks if we should use changeset-centric copy algorithms"""
+if repo.filecopiesmode == b'changeset-sidedata':
+return True
 readfrom = repo.ui.config(b'experimental', b'copies.read-from')
 changesetsource = (b'changeset-only', b'compatibility')
 return readfrom in changesetsource
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -534,7 +534,10 @@
 
 def filesadded(self):
 filesadded = self._changeset.filesadded
-if True:
+if self._repo.filecopiesmode == b'changeset-sidedata':
+if filesadded is None:
+filesadded = []
+else:
 source = self._repo.ui.config(b'experimental', b'copies.read-from')
 if source == b'changeset-only':
 if filesadded is None:
@@ -548,7 +551,10 @@
 
 def filesremoved(self):
 filesremoved = self._changeset.filesremoved
-if True:
+if self._repo.filecopiesmode == b'changeset-sidedata':
+if filesremoved is None:
+filesremoved = []
+else:
 source = self._repo.ui.config(b'experimental', b'copies.read-from')
 if source == b'changeset-only':
 if filesremoved is None:
@@ -564,7 +570,12 @@
 def _copies(self):
 p1copies = self._changeset.p1copies
 p2copies = self._changeset.p2copies
-if True:
+if self._repo.filecopiesmode == b'changeset-sidedata':
+if p1copies is None:
+p1copies = {}
+if p2copies is None:
+p2copies = {}
+else:
 source = self._repo.ui.config(b'experimental', b'copies.read-from')
 # If config says to get copy metadata only from changeset, then
 # return that, defaulting to {} if there was no copy metadata.  In
diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ 

D6955: sidedatacopies: deal with upgrading and downgrading to that format

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16956.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6955?vs=16787=16956

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6955/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6955

AFFECTED FILES
  mercurial/copies.py
  mercurial/upgrade.py
  tests/test-copies-in-changeset.t

CHANGE DETAILS

diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
--- a/tests/test-copies-in-changeset.t
+++ b/tests/test-copies-in-changeset.t
@@ -450,4 +450,91 @@
   $ hg ci -Aqm 'add a'
   $ hg mv a b
   $ hg ci -m 'remove a' a
+
+#if sidedata
+
+Test upgrading/downgrading to sidedata storage
+==
+
+downgrading (keeping some sidedata)
+
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:  yesyes  no
+  copies-sdc:yesyes  no
+  plain-cl-delta:yesyes yes
+  compression:   zlib   zlibzlib
+  compression-level: default default default
+  $ hg debugsidedata -c -- 0
+  4 sidedata entries
+   entry-0010 size 0
+   entry-0011 size 0
+   entry-0012 size 1
+   entry-0013 size 0
+  $ hg debugsidedata -c -- 1
+  4 sidedata entries
+   entry-0010 size 0
+   entry-0011 size 0
+   entry-0012 size 0
+   entry-0013 size 1
+  $ hg debugsidedata -m -- 0
+  $ cat << EOF > .hg/hgrc
+  > [format]
+  > use-side-data = yes
+  > exp-use-copies-side-data-changeset = no
+  > EOF
+  $ hg debugupgraderepo --run --quiet --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:  yesyes  no
+  copies-sdc: no no  no
+  plain-cl-delta:yesyes yes
+  compression:   zlib   zlibzlib
+  compression-level: default default default
+  $ hg debugsidedata -c -- 0
+  $ hg debugsidedata -c -- 1
+  $ hg debugsidedata -m -- 0
+
+upgrading
+
+  $ cat << EOF > .hg/hgrc
+  > [format]
+  > exp-use-copies-side-data-changeset = yes
+  > EOF
+  $ hg debugupgraderepo --run --quiet --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:  yesyes  no
+  copies-sdc:yesyes  no
+  plain-cl-delta:yesyes yes
+  compression:   zlib   zlibzlib
+  compression-level: default default default
+  $ hg debugsidedata -c -- 0
+  4 sidedata entries
+   entry-0010 size 0
+   entry-0011 size 0
+   entry-0012 size 1
+   entry-0013 size 0
+  $ hg debugsidedata -c -- 1
+  4 sidedata entries
+   entry-0010 size 0
+   entry-0011 size 0
+   entry-0012 size 0
+   entry-0013 size 1
+  $ hg debugsidedata -m -- 0
+
+#endif
+
   $ cd ..
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -13,6 +13,7 @@
 from .pycompat import getattr
 from . import (
 changelog,
+copies,
 error,
 filelog,
 hg,
@@ -31,7 +32,6 @@
 RECLONES_REQUIREMENTS = {
 b'generaldelta',
 localrepo.SPARSEREVLOG_REQUIREMENT,
-localrepo.SIDEDATA_REQUIREMENT,
 }
 
 
@@ -77,6 +77,7 @@
 supported = {
 localrepo.SPARSEREVLOG_REQUIREMENT,
 localrepo.SIDEDATA_REQUIREMENT,
+localrepo.COPIESSDC_REQUIREMENT,
 }
 for name in compression.compengines:
 engine = compression.compengines[name]
@@ -103,6 +104,7 @@
 b'store',
 localrepo.SPARSEREVLOG_REQUIREMENT,
 localrepo.SIDEDATA_REQUIREMENT,
+localrepo.COPIESSDC_REQUIREMENT,
 }
 for name in compression.compengines:
 engine = compression.compengines[name]
@@ -129,6 +131,7 @@
 b'generaldelta',
 localrepo.SPARSEREVLOG_REQUIREMENT,
 localrepo.SIDEDATA_REQUIREMENT,
+localrepo.COPIESSDC_REQUIREMENT,
 }
 for name in compression.compengines:
 engine = compression.compengines[name]
@@ -698,6 +701,7 @@
 def getsidedatacompanion(srcrepo, dstrepo):
 sidedatacompanion = None
 removedreqs = srcrepo.requirements - dstrepo.requirements
+addedreqs = dstrepo.requirements - srcrepo.requirements
 if localrepo.SIDEDATA_REQUIREMENT in removedreqs:
 
 def sidedatacompanion(rl, rev):
@@ -706,6 +710,10 @@
 return True, (), {}
 return False, (), {}
 
+elif localrepo.COPIESSDC_REQUIREMENT in addedreqs:
+sidedatacompanion = copies.getsidedataadder(srcrepo, dstrepo)
+elif localrepo.COPIESSDC_REQUIREMENT in removedreqs:
+

D6950: sidedatacopies: write copies information in sidedata when applicable

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16951.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6950?vs=16911=16951

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6950/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6950

AFFECTED FILES
  mercurial/changelog.py
  mercurial/interfaces/repository.py
  mercurial/localrepo.py
  mercurial/revlogutils/sidedata.py
  tests/test-copies-in-changeset.t

CHANGE DETAILS

diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
--- a/tests/test-copies-in-changeset.t
+++ b/tests/test-copies-in-changeset.t
@@ -75,7 +75,17 @@
   p1copies: 0\x00a (esc)
   1\x00a (esc)
   2\x00a (esc)
-
+#else
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 11
+'0\x00a\n1\x00a\n2\x00a'
+   entry-0011 size 0
+''
+   entry-0012 size 5
+'0\n1\n2'
+   entry-0013 size 0
+''
 #endif
 
   $ hg showcopies
@@ -107,6 +117,17 @@
   
   p1copies: 1\x00b (esc)
 
+#else
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 3
+'1\x00b'
+   entry-0011 size 0
+''
+   entry-0012 size 1
+'1'
+   entry-0013 size 1
+'0'
 #endif
 
   $ hg showcopies
@@ -145,6 +166,17 @@
   
   p1copies: 0\x00b2 (esc)
 
+#else
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 4
+'0\x00b2'
+   entry-0011 size 0
+''
+   entry-0012 size 0
+''
+   entry-0013 size 0
+''
 #endif
 
   $ hg showcopies
@@ -197,6 +229,17 @@
   2\x00f (esc)
   p2copies: 1\x00d (esc)
 
+#else
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 7
+'0\x00a\n2\x00f'
+   entry-0011 size 3
+'1\x00d'
+   entry-0012 size 5
+'0\n1\n2'
+   entry-0013 size 0
+''
 #endif
 
   $ hg showcopies
@@ -218,6 +261,16 @@
   p2copies: 
 #else
   $ hg ci -m 'copy a to j'
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 3
+'0\x00a'
+   entry-0011 size 0
+''
+   entry-0012 size 1
+'0'
+   entry-0013 size 0
+''
 #endif
   $ hg debugdata j 0
   \x01 (esc)
@@ -243,6 +296,16 @@
 #else
   $ hg ci --amend -m 'copy a to j, v2'
   saved backup bundle to $TESTTMP/repo/.hg/strip-backup/*-*-amend.hg (glob)
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 3
+'0\x00a'
+   entry-0011 size 0
+''
+   entry-0012 size 1
+'0'
+   entry-0013 size 0
+''
 #endif
   $ hg showcopies --config experimental.copies.read-from=filelog-only
   a -> j
@@ -260,6 +323,16 @@
   p2copies: 
 #else
   $ hg ci -m 'modify j'
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 0
+''
+   entry-0011 size 0
+''
+   entry-0012 size 0
+''
+   entry-0013 size 0
+''
 #endif
 
 Test writing only to filelog
@@ -273,6 +346,16 @@
   
 #else
   $ hg ci -m 'copy a to k'
+  $ hg debugsidedata -c -v -- -1
+  4 sidedata entries
+   entry-0010 size 3
+'0\x00a'
+   entry-0011 size 0
+''
+   entry-0012 size 1
+'0'
+   entry-0013 size 0
+''
 #endif
 
   $ hg debugdata k 0
diff --git a/mercurial/revlogutils/sidedata.py 
b/mercurial/revlogutils/sidedata.py
--- a/mercurial/revlogutils/sidedata.py
+++ b/mercurial/revlogutils/sidedata.py
@@ -48,6 +48,12 @@
 SD_TEST6 = 6
 SD_TEST7 = 7
 
+# key to store copies related information
+SD_P1COPIES = 8
+SD_P2COPIES = 9
+SD_FILESADDED = 10
+SD_FILESREMOVED = 11
+
 # internal format constant
 SIDEDATA_HEADER = struct.Struct(r'>H')
 SIDEDATA_ENTRY = struct.Struct(r'>HL20s')
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -827,10 +827,13 @@
 else:  # explicitly mark repo as using revlogv0
 options[b'revlogv0'] = True
 
-writecopiesto = ui.config(b'experimental', b'copies.write-to')
-copiesextramode = (b'changeset-only', b'compatibility')
-if writecopiesto in copiesextramode:
-options[b'copies-storage'] = b'extra'
+if COPIESSDC_REQUIREMENT in requirements:
+options[b'copies-storage'] = b'changeset-sidedata'
+else:
+writecopiesto = ui.config(b'experimental', b'copies.write-to')
+copiesextramode = (b'changeset-only', b'compatibility')
+if writecopiesto in copiesextramode:
+options[b'copies-storage'] = b'extra'
 
 return options
 
@@ -1184,6 +1187,10 @@
 
 self._extrafilterid = repoview.extrafilter(ui)
 
+self.filecopiesmode = None
+if COPIESSDC_REQUIREMENT in self.requirements:
+self.filecopiesmode = b'changeset-sidedata'
+
 def _getvfsward(self, origfunc):
 """build a ward for self.vfs"""
 rref = weakref.ref(self)
@@ -2952,12 +2959,17 @@
 p1, p2 = ctx.p1(), ctx.p2()
 user = ctx.user()
 
-writecopiesto = self.ui.config(b'experimental', b'copies.write-to')
-writefilecopymeta = writecopiesto != b'changeset-only'
-writechangesetcopy = writecopiesto in (
-

D6885: relnotes: mention API change from https://phab.mercurial-scm.org/D6884

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  I am a bit confused about this diff, is this still required ?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6885/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6885

To: martinvonz, #hg-reviewers
Cc: marmoute, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6952: sidedatacopies: preindent some copies related code

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16953.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6952?vs=16784=16953

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6952/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6952

AFFECTED FILES
  mercurial/context.py

CHANGE DETAILS

diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -533,55 +533,58 @@
 return sorted(modified)
 
 def filesadded(self):
-source = self._repo.ui.config(b'experimental', b'copies.read-from')
 filesadded = self._changeset.filesadded
-if source == b'changeset-only':
-if filesadded is None:
-filesadded = []
-elif source == b'compatibility':
-if filesadded is None:
+if True:
+source = self._repo.ui.config(b'experimental', b'copies.read-from')
+if source == b'changeset-only':
+if filesadded is None:
+filesadded = []
+elif source == b'compatibility':
+if filesadded is None:
+filesadded = scmutil.computechangesetfilesadded(self)
+else:
 filesadded = scmutil.computechangesetfilesadded(self)
-else:
-filesadded = scmutil.computechangesetfilesadded(self)
 return filesadded
 
 def filesremoved(self):
-source = self._repo.ui.config(b'experimental', b'copies.read-from')
 filesremoved = self._changeset.filesremoved
-if source == b'changeset-only':
-if filesremoved is None:
-filesremoved = []
-elif source == b'compatibility':
-if filesremoved is None:
+if True:
+source = self._repo.ui.config(b'experimental', b'copies.read-from')
+if source == b'changeset-only':
+if filesremoved is None:
+filesremoved = []
+elif source == b'compatibility':
+if filesremoved is None:
+filesremoved = scmutil.computechangesetfilesremoved(self)
+else:
 filesremoved = scmutil.computechangesetfilesremoved(self)
-else:
-filesremoved = scmutil.computechangesetfilesremoved(self)
 return filesremoved
 
 @propertycache
 def _copies(self):
-source = self._repo.ui.config(b'experimental', b'copies.read-from')
 p1copies = self._changeset.p1copies
 p2copies = self._changeset.p2copies
-# If config says to get copy metadata only from changeset, then return
-# that, defaulting to {} if there was no copy metadata.
-# In compatibility mode, we return copy data from the changeset if
-# it was recorded there, and otherwise we fall back to getting it from
-# the filelogs (below).
-if source == b'changeset-only':
-if p1copies is None:
-p1copies = {}
-if p2copies is None:
-p2copies = {}
-elif source == b'compatibility':
-if p1copies is None:
-# we are in compatiblity mode and there is not data in the
-# changeset), we get the copy metadata from the filelogs.
+if True:
+source = self._repo.ui.config(b'experimental', b'copies.read-from')
+# If config says to get copy metadata only from changeset, then
+# return that, defaulting to {} if there was no copy metadata.  In
+# compatibility mode, we return copy data from the changeset if it
+# was recorded there, and otherwise we fall back to getting it from
+# the filelogs (below).
+if source == b'changeset-only':
+if p1copies is None:
+p1copies = {}
+if p2copies is None:
+p2copies = {}
+elif source == b'compatibility':
+if p1copies is None:
+# we are in compatiblity mode and there is not data in the
+# changeset), we get the copy metadata from the filelogs.
+p1copies, p2copies = super(changectx, self)._copies
+else:
+# config said to read only from filelog, we get the copy
+# metadata from the filelogs.
 p1copies, p2copies = super(changectx, self)._copies
-else:
-# config said to read only from filelog, we get the copy metadata
-# from the filelogs.
-p1copies, p2copies = super(changectx, self)._copies
 return p1copies, p2copies
 
 def description(self):



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6946: sidedatacopies: teach upgrade about the new requirement

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16950.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6946?vs=16778=16950

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6946/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6946

AFFECTED FILES
  mercurial/upgrade.py
  tests/test-lfs-serve.t
  tests/test-sidedata.t
  tests/test-upgrade-repo.t

CHANGE DETAILS

diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -58,6 +58,7 @@
   generaldelta:  yes
   sparserevlog:  yes
   sidedata:   no
+  copies-sdc: no
   plain-cl-delta:yes
   compression:   zlib
   compression-level: default
@@ -68,6 +69,7 @@
   generaldelta:  yesyes yes
   sparserevlog:  yesyes yes
   sidedata:   no no  no
+  copies-sdc: no no  no
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
@@ -78,6 +80,7 @@
   generaldelta:  yesyes yes
   sparserevlog:  yesyes yes
   sidedata:   no no  no
+  copies-sdc: no no  no
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
@@ -88,6 +91,7 @@
   [formatvariant.name.uptodate|generaldelta: 
][formatvariant.repo.uptodate| yes][formatvariant.config.default|
yes][formatvariant.default| yes]
   [formatvariant.name.uptodate|sparserevlog: 
][formatvariant.repo.uptodate| yes][formatvariant.config.default|
yes][formatvariant.default| yes]
   [formatvariant.name.uptodate|sidedata: 
][formatvariant.repo.uptodate|  no][formatvariant.config.default| 
no][formatvariant.default|  no]
+  [formatvariant.name.uptodate|copies-sdc:   
][formatvariant.repo.uptodate|  no][formatvariant.config.default| 
no][formatvariant.default|  no]
   [formatvariant.name.uptodate|plain-cl-delta:   
][formatvariant.repo.uptodate| yes][formatvariant.config.default|
yes][formatvariant.default| yes]
   [formatvariant.name.uptodate|compression:  
][formatvariant.repo.uptodate| zlib][formatvariant.config.default|   
zlib][formatvariant.default|zlib]
   
[formatvariant.name.uptodate|compression-level:][formatvariant.repo.uptodate| 
default][formatvariant.config.default| default][formatvariant.default| default]
@@ -124,6 +128,12 @@
 "repo": false
},
{
+"config": false,
+"default": false,
+"name": "copies-sdc",
+"repo": false
+   },
+   {
 "config": true,
 "default": true,
 "name": "plain-cl-delta",
@@ -152,6 +162,9 @@
   sidedata
  Allows storage of extra data alongside a revision.
   
+  copies-sdc
+ Allows to use more efficient algorithm to deal with copy tracing.
+  
   additional optimizations are available by specifying "--optimize ":
   
   re-delta-parent
@@ -179,6 +192,9 @@
   sidedata
  Allows storage of extra data alongside a revision.
   
+  copies-sdc
+ Allows to use more efficient algorithm to deal with copy tracing.
+  
   re-delta-parent
  deltas within internal storage will choose a new base revision if needed
   
@@ -206,6 +222,9 @@
   sidedata
  Allows storage of extra data alongside a revision.
   
+  copies-sdc
+ Allows to use more efficient algorithm to deal with copy tracing.
+  
   re-delta-parent
  deltas within internal storage will choose a new base revision if needed
   
@@ -242,6 +261,7 @@
   generaldelta:   no
   sparserevlog:   no
   sidedata:   no
+  copies-sdc: no
   plain-cl-delta:yes
   compression:   zlib
   compression-level: default
@@ -252,6 +272,7 @@
   generaldelta:   noyes yes
   sparserevlog:   noyes yes
   sidedata:   no no  no
+  copies-sdc: no no  no
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
@@ -262,6 +283,7 @@
   generaldelta:   no no yes
   sparserevlog:   no no yes
   sidedata:   no no  no
+  copies-sdc: no no  no
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
@@ -272,6 +294,7 @@
   [formatvariant.name.mismatchdefault|generaldelta: 
][formatvariant.repo.mismatchdefault|  no][formatvariant.config.special| 
no][formatvariant.default| yes]
   [formatvariant.name.mismatchdefault|sparserevlog: 
][formatvariant.repo.mismatchdefault|  no][formatvariant.config.special| 
no][formatvariant.default| yes]
   [formatvariant.name.uptodate|sidedata: 
][formatvariant.repo.uptodate|  no][formatvariant.config.default| 
no][formatvariant.default|  no]
+  [formatvariant.name.uptodate|copies-sdc:   

D6945: sidedatacopies: add a new requirement for storing copies into sidedata

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16949.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6945?vs=16777=16949

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6945/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6945

AFFECTED FILES
  mercurial/configitems.py
  mercurial/localrepo.py

CHANGE DETAILS

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -442,6 +442,10 @@
 # information for revision without altering their original hashes.
 SIDEDATA_REQUIREMENT = b'exp-sidedata-flag'
 
+# A repository with the the copies-sidedata-changeset requirement will store
+# copies related information in changeset's sidedata.
+COPIESSDC_REQUIREMENT = b'exp-copies-sidedata-changeset'
+
 # Functions receiving (ui, features) that extensions can register to impact
 # the ability to load repositories with custom requirements. Only
 # functions defined in loaded extensions are called.
@@ -999,6 +1003,7 @@
 b'revlogv1',
 b'generaldelta',
 b'treemanifest',
+COPIESSDC_REQUIREMENT,
 REVLOGV2_REQUIREMENT,
 SIDEDATA_REQUIREMENT,
 SPARSEREVLOG_REQUIREMENT,
@@ -3515,6 +3520,10 @@
 # experimental config: format.use-side-data
 if ui.configbool(b'format', b'use-side-data'):
 requirements.add(SIDEDATA_REQUIREMENT)
+# experimental config: format.exp-use-copies-side-data-changeset
+if ui.configbool(b'format', b'exp-use-copies-side-data-changeset'):
+requirements.add(SIDEDATA_REQUIREMENT)
+requirements.add(COPIESSDC_REQUIREMENT)
 if ui.configbool(b'experimental', b'treemanifest'):
 requirements.add(b'treemanifest')
 
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -748,6 +748,12 @@
 b'format', b'usestore', default=True,
 )
 coreconfigitem(
+b'format',
+b'exp-use-copies-side-data-changeset',
+default=False,
+experimental=True,
+)
+coreconfigitem(
 b'format', b'use-side-data', default=False, experimental=True,
 )
 coreconfigitem(



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6951: sidedatacopies: get and store sidedata in the changelogrevision object

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16952.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6951?vs=16783=16952

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6951/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6951

AFFECTED FILES
  mercurial/changelog.py

CHANGE DETAILS

diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -268,9 +268,10 @@
 __slots__ = (
 r'_offsets',
 r'_text',
+r'_sidedata',
 )
 
-def __new__(cls, text):
+def __new__(cls, text, sidedata):
 if not text:
 return _changelogrevision(extra=_defaultextra)
 
@@ -302,6 +303,7 @@
 
 self._offsets = (nl1, nl2, nl3, doublenl)
 self._text = text
+self._sidedata = sidedata
 
 return self
 
@@ -613,12 +615,13 @@
 ``changelogrevision`` instead, as it is faster for partial object
 access.
 """
-c = changelogrevision(self.revision(node))
+c = changelogrevision(*self._revisiondata(node))
 return (c.manifest, c.user, c.date, c.files, c.description, c.extra)
 
 def changelogrevision(self, nodeorrev):
 """Obtain a ``changelogrevision`` for a node or revision."""
-return changelogrevision(self.revision(nodeorrev))
+text, sidedata = self._revisiondata(nodeorrev)
+return changelogrevision(text, sidedata)
 
 def readfiles(self, node):
 """



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6944: debugsidedata: small doc improvement

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16948.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6944?vs=16910=16948

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6944/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6944

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -3212,7 +3212,9 @@
 
 @command(b'debugsidedata', cmdutil.debugrevlogopts, _(b'-c|-m|FILE REV'))
 def debugsidedata(ui, repo, file_, rev=None, **opts):
-"""dump the side data for a cl/manifest/file revision"""
+"""dump the side data for a cl/manifest/file revision
+
+Use --verbose to dump the sidedata content."""
 opts = pycompat.byteskwargs(opts)
 if opts.get(b'changelog') or opts.get(b'manifest') or opts.get(b'dir'):
 if rev is not None:



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6943: upgrade: allow for `sidedata` removal

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16947.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6943?vs=16775=16947

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6943/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6943

AFFECTED FILES
  mercurial/upgrade.py
  tests/test-sidedata.t
  tests/test-upgrade-repo.t

CHANGE DETAILS

diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -1361,3 +1361,53 @@
entry-0001 size 4
entry-0002 size 32
 
+downgrade
+
+  $ hg debugupgraderepo --config format.use-side-data=no --run --no-backup > 
/dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:   no no  no
+  plain-cl-delta:yesyes yes
+  compression:   zstd   zstdzlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+  $ hg debugsidedata -c 0
+
+upgrade from hgrc
+
+  $ cat >> .hg/hgrc << EOF
+  > [format]
+  > use-side-data=yes
+  > EOF
+  $ hg debugupgraderepo --run --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:  yesyes  no
+  plain-cl-delta:yesyes yes
+  compression:   zstd   zstdzlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  exp-sidedata-flag
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+  $ hg debugsidedata -c 0
diff --git a/tests/test-sidedata.t b/tests/test-sidedata.t
--- a/tests/test-sidedata.t
+++ b/tests/test-sidedata.t
@@ -71,8 +71,8 @@
   compression-level: default default default
   $ hg debugupgraderepo -R up-no-side-data --config format.use-side-data=yes > 
/dev/null
 
-Check that we cannot upgrade to sidedata
-
+Check that we can upgrade to sidedata
+-
 
   $ hg init up-side-data --config format.use-side-data=yes
   $ hg debugformat -v -R up-side-data
@@ -95,6 +95,4 @@
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
-  $ hg debugupgraderepo -R up-side-data --config format.use-side-data=no
-  abort: cannot upgrade repository; requirement would be removed: 
exp-sidedata-flag
-  [255]
+  $ hg debugupgraderepo -R up-side-data --config format.use-side-data=no > 
/dev/null
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -76,6 +76,7 @@
 """
 supported = {
 localrepo.SPARSEREVLOG_REQUIREMENT,
+localrepo.SIDEDATA_REQUIREMENT,
 }
 for name in compression.compengines:
 engine = compression.compengines[name]
@@ -679,8 +680,18 @@
 )
 
 
-def getsidedatacompanion(srcrepo, destrepo):
-return None
+def getsidedatacompanion(srcrepo, dstrepo):
+sidedatacompanion = None
+removedreqs = srcrepo.requirements - dstrepo.requirements
+if localrepo.SIDEDATA_REQUIREMENT in removedreqs:
+
+def sidedatacompanion(rl, rev):
+rl = getattr(rl, '_revlog', rl)
+if rl.flags(rev) & revlog.REVIDX_SIDEDATA:
+return True, (), {}
+return False, (), {}
+
+return sidedatacompanion
 
 
 def matchrevlog(revlogfilter, entry):



To: marmoute, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6942: upgrade: allow upgrade to repository using sidedata

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16946.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6942?vs=16773=16946

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6942/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6942

AFFECTED FILES
  mercurial/upgrade.py
  tests/test-sidedata.t
  tests/test-upgrade-repo.t
  tests/testlib/ext-sidedata.py

CHANGE DETAILS

diff --git a/tests/testlib/ext-sidedata.py b/tests/testlib/ext-sidedata.py
--- a/tests/testlib/ext-sidedata.py
+++ b/tests/testlib/ext-sidedata.py
@@ -12,8 +12,10 @@
 
 from mercurial import (
 extensions,
+localrepo,
 node,
 revlog,
+upgrade,
 )
 
 from mercurial.revlogutils import sidedata
@@ -36,6 +38,8 @@
 
 def wraprevision(orig, self, nodeorrev, *args, **kwargs):
 text = orig(self, nodeorrev, *args, **kwargs)
+if getattr(self, 'sidedatanocheck', False):
+return text
 if nodeorrev != node.nullrev and nodeorrev != node.nullid:
 sd = self.sidedata(nodeorrev)
 if len(text) != struct.unpack('>I', sd[sidedata.SD_TEST1])[0]:
@@ -47,6 +51,33 @@
 return text
 
 
+def wrapgetsidedatacompanion(orig, srcrepo, dstrepo):
+sidedatacompanion = orig(srcrepo, dstrepo)
+addedreqs = dstrepo.requirements - srcrepo.requirements
+if localrepo.SIDEDATA_REQUIREMENT in addedreqs:
+assert sidedatacompanion is None  # deal with composition later
+
+def sidedatacompanion(revlog, rev):
+update = {}
+revlog.sidedatanocheck = True
+try:
+text = revlog.revision(rev)
+finally:
+del revlog.sidedatanocheck
+## let's store some arbitrary data just for testing
+# text length
+update[sidedata.SD_TEST1] = struct.pack('>I', len(text))
+# and sha2 hashes
+sha256 = hashlib.sha256(text).digest()
+update[sidedata.SD_TEST2] = struct.pack('>32s', sha256)
+return False, (), update
+
+return sidedatacompanion
+
+
 def extsetup(ui):
 extensions.wrapfunction(revlog.revlog, 'addrevision', wrapaddrevision)
 extensions.wrapfunction(revlog.revlog, 'revision', wraprevision)
+extensions.wrapfunction(
+upgrade, 'getsidedatacompanion', wrapgetsidedatacompanion
+)
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -1329,6 +1329,35 @@
   sparserevlog
   store
 
-  $ cd ..
+#endif
+
+Check upgrading to a side-data revlog
+-
+
+upgrade
 
-#endif
+  $ hg --config format.use-side-data=yes debugupgraderepo --run  --no-backup 
--config "extensions.sidedata=$TESTDIR/testlib/ext-sidedata.py" >/dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  sidedata:  yes no  no
+  plain-cl-delta:yesyes yes
+  compression:   zstd   zstdzlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  exp-sidedata-flag
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+  $ hg debugsidedata -c 0
+  2 sidedata entries
+   entry-0001 size 4
+   entry-0002 size 32
+
diff --git a/tests/test-sidedata.t b/tests/test-sidedata.t
--- a/tests/test-sidedata.t
+++ b/tests/test-sidedata.t
@@ -45,8 +45,8 @@
 
 Right now, sidedata has not upgrade support
 
-Check that we cannot upgrade to sidedata
-
+Check that we can upgrade to sidedata
+-
 
   $ hg init up-no-side-data --config format.use-side-data=no
   $ hg debugformat -v -R up-no-side-data
@@ -69,9 +69,7 @@
   plain-cl-delta:yesyes yes
   compression:   zlib   zlibzlib
   compression-level: default default default
-  $ hg debugupgraderepo -R up-no-side-data --config format.use-side-data=yes
-  abort: cannot upgrade repository; do not support adding requirement: 
exp-sidedata-flag
-  [255]
+  $ hg debugupgraderepo -R up-no-side-data --config format.use-side-data=yes > 
/dev/null
 
 Check that we cannot upgrade to sidedata
 
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -31,6 +31,7 @@
 RECLONES_REQUIREMENTS = {
 b'generaldelta',
 localrepo.SPARSEREVLOG_REQUIREMENT,
+localrepo.SIDEDATA_REQUIREMENT,
 }
 
 
@@ -100,6 +101,7 @@
 b'revlogv1',
 b'store',
 localrepo.SPARSEREVLOG_REQUIREMENT,
+localrepo.SIDEDATA_REQUIREMENT,
 }
 for name in compression.compengines:
 engine = compression.compengines[name]
@@ -125,6 +127,7 @@
 b'fncache',
 b'generaldelta',
 

D6941: revlog: add a way to control sidedata changes during revlog.clone

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16945.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6941?vs=16772=16945

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6941/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6941

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2540,6 +2540,7 @@
 addrevisioncb=None,
 deltareuse=DELTAREUSESAMEREVS,
 forcedeltabothparents=None,
+sidedatacompanion=None,
 ):
 """Copy this revlog to another, possibly with format changes.
 
@@ -2581,6 +2582,20 @@
 In addition to the delta policy, the ``forcedeltabothparents``
 argument controls whether to force compute deltas against both parents
 for merges. By default, the current default is used.
+
+If not None, the `sidedatacompanion` is callable that accept two
+arguments:
+
+(srcrevlog, rev)
+
+and return a triplet that control changes to sidedata content from the
+old revision to the new clone result:
+
+(dropall, filterout, update)
+
+* if `dropall` is True, all sidedata should be dropped
+* `filterout` is a set of sidedata keys that should be dropped
+* `update` is a mapping of additionnal/new key -> value
 """
 if deltareuse not in self.DELTAREUSEALL:
 raise ValueError(
@@ -2615,7 +2630,12 @@
 destrevlog._deltabothparents = forcedeltabothparents or oldamd
 
 self._clone(
-tr, destrevlog, addrevisioncb, deltareuse, 
forcedeltabothparents
+tr,
+destrevlog,
+addrevisioncb,
+deltareuse,
+forcedeltabothparents,
+sidedatacompanion,
 )
 
 finally:
@@ -2624,7 +2644,13 @@
 destrevlog._deltabothparents = oldamd
 
 def _clone(
-self, tr, destrevlog, addrevisioncb, deltareuse, forcedeltabothparents
+self,
+tr,
+destrevlog,
+addrevisioncb,
+deltareuse,
+forcedeltabothparents,
+sidedatacompanion,
 ):
 """perform the core duty of `revlog.clone` after parameter 
processing"""
 deltacomputer = deltautil.deltacomputer(destrevlog)
@@ -2640,12 +2666,24 @@
 p2 = index[entry[6]][7]
 node = entry[7]
 
+sidedataactions = (False, [], {})
+if sidedatacompanion is not None:
+sidedataactions = sidedatacompanion(self, rev)
+
 # (Possibly) reuse the delta from the revlog if allowed and
 # the revlog chunk is a delta.
 cachedelta = None
 rawtext = None
-if deltareuse == self.DELTAREUSEFULLADD:
-text = self.revision(rev)
+if any(sidedataactions) or deltareuse == self.DELTAREUSEFULLADD:
+dropall, filterout, update = sidedataactions
+text, sidedata = self._revisiondata(rev)
+if dropall:
+sidedata = {}
+for key in filterout:
+sidedata.pop(key, None)
+sidedata.update(update)
+if not sidedata:
+sidedata = None
 destrevlog.addrevision(
 text,
 tr,
@@ -2656,6 +2694,7 @@
 node=node,
 flags=flags,
 deltacomputer=deltacomputer,
+sidedata=sidedata,
 )
 else:
 if destrevlog._lazydelta:



To: marmoute, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6939: sidedata: apply basic but tight security around exchange

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16943.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6939?vs=16771=16943

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6939/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6939

AFFECTED FILES
  mercurial/bundle2.py
  mercurial/exchange.py

CHANGE DETAILS

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1049,6 +1049,8 @@
 cgpart.addparam(b'version', version)
 if b'treemanifest' in pushop.repo.requirements:
 cgpart.addparam(b'treemanifest', b'1')
+if b'exp-sidedata-flag' in pushop.repo.requirements:
+cgpart.addparam(b'exp-sidedata', b'1')
 
 def handlereply(op):
 """extract addchangegroup returns from server reply"""
@@ -2512,6 +2514,9 @@
 if b'treemanifest' in repo.requirements:
 part.addparam(b'treemanifest', b'1')
 
+if b'exp-sidedata-flag' in repo.requirements:
+part.addparam(b'exp-sidedata', b'1')
+
 if (
 kwargs.get(r'narrow', False)
 and kwargs.get(r'narrow_acl', False)
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1711,6 +1711,8 @@
 part.addparam(
 b'targetphase', b'%d' % phases.secret, mandatory=False
 )
+if b'exp-sidedata-flag' in repo.requirements:
+part.addparam(b'exp-sidedata', b'1')
 
 if opts.get(b'streamv2', False):
 addpartbundlestream2(bundler, repo, stream=True)
@@ -1930,7 +1932,14 @@
 
 
 @parthandler(
-b'changegroup', (b'version', b'nbchanges', b'treemanifest', b'targetphase')
+b'changegroup',
+(
+b'version',
+b'nbchanges',
+b'exp-sidedata',
+b'treemanifest',
+b'targetphase',
+),
 )
 def handlechangegroup(op, inpart):
 """apply a changegroup part on the repo
@@ -1965,6 +1974,14 @@
 op.repo.ui, op.repo.requirements, op.repo.features
 )
 op.repo._writerequirements()
+
+bundlesidedata = bool(b'exp-sidedata' in inpart.params)
+reposidedata = bool(b'exp-sidedata-flag' in op.repo.requirements)
+if reposidedata and not bundlesidedata:
+msg = b"repository is using sidedata but the bundle source do not"
+hint = b'this is currently unsupported'
+raise error.Abort(msg, hint=hint)
+
 extrakwargs = {}
 targetphase = inpart.params.get(b'targetphase')
 if targetphase is not None:
@@ -2551,5 +2568,7 @@
 part.addparam(b'version', cgversion)
 if b'treemanifest' in repo.requirements:
 part.addparam(b'treemanifest', b'1')
+if b'exp-sidedata-flag' in repo.requirements:
+part.addparam(b'exp-sidedata', b'1')
 
 return bundler



To: marmoute, #hg-reviewers
Cc: indygreg, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6940: sidedata: use only changegroup3 if sidedata is in use

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 16944.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6940?vs=16743=16944

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6940/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6940

AFFECTED FILES
  mercurial/changegroup.py

CHANGE DETAILS

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -1479,6 +1479,11 @@
 #
 # (or even to push subset of history)
 needv03 = True
+if b'exp-sidedata-flag' in repo.requirements:
+needv03 = True
+# don't attempt to use 01/02 until we do sidedata cleaning
+versions.discard(b'01')
+versions.discard(b'02')
 if not needv03:
 versions.discard(b'03')
 return versions



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7015: py3: finish porting iteritems() to pycompat and remove source transformer

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This commit finishes porting .iteritems() to pycompat.iteritems()
  for the mercurial package.
  
  The translation of .iteritems() to .items() was the last conversion
  performed by the source transformer. With the porting to pycompat
  complete, we no longer have a need for the source transformer. So
  the source transformer has been removed. Good riddance! The code
  base is now compatible with Python 2 and Python 3.
  
  For the record, as the person who introduced the source transformer,
  it brings me joy to delete it. It accomplished its goal to facilitate
  a port to Python 3 without overly burdening people on some painful
  low-level differences between Python 2 and 3. It is unfortunate we
  still have to wallpaper over many differences with the pycompat
  shim. But it is what it is.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7015

AFFECTED FILES
  mercurial/__init__.py
  mercurial/archival.py
  mercurial/bookmarks.py
  mercurial/branchmap.py
  mercurial/changegroup.py
  mercurial/chgserver.py
  mercurial/cmdutil.py
  mercurial/commands.py
  mercurial/config.py
  mercurial/context.py
  mercurial/copies.py
  mercurial/debugcommands.py
  mercurial/dirstate.py
  mercurial/discovery.py
  mercurial/dispatch.py
  mercurial/exchange.py
  mercurial/exchangev2.py
  mercurial/extensions.py
  mercurial/exthelper.py
  mercurial/filemerge.py
  mercurial/fileset.py
  mercurial/formatter.py
  mercurial/help.py
  mercurial/hgweb/hgweb_mod.py
  mercurial/hgweb/hgwebdir_mod.py
  mercurial/hgweb/request.py
  mercurial/hgweb/webcommands.py
  mercurial/hgweb/webutil.py
  mercurial/hgweb/wsgicgi.py
  mercurial/hook.py
  mercurial/httpconnection.py
  mercurial/keepalive.py
  mercurial/localrepo.py
  mercurial/logcmdutil.py
  mercurial/logexchange.py
  mercurial/lsprof.py
  mercurial/manifest.py
  mercurial/match.py
  mercurial/merge.py
  mercurial/namespaces.py
  mercurial/obsolete.py
  mercurial/obsutil.py
  mercurial/patch.py
  mercurial/phases.py
  mercurial/pure/parsers.py
  mercurial/pycompat.py
  mercurial/revlog.py
  mercurial/revset.py
  mercurial/revsetlang.py
  mercurial/scmutil.py
  mercurial/similar.py
  mercurial/sparse.py
  mercurial/sshpeer.py
  mercurial/statprof.py
  mercurial/store.py
  mercurial/subrepo.py
  mercurial/subrepoutil.py
  mercurial/tags.py
  mercurial/templatefilters.py
  mercurial/templatefuncs.py
  mercurial/templatekw.py
  mercurial/templater.py
  mercurial/templateutil.py
  mercurial/transaction.py
  mercurial/ui.py
  mercurial/url.py
  mercurial/util.py
  mercurial/utils/cborutil.py
  mercurial/utils/procutil.py
  mercurial/verify.py
  mercurial/wireprotoframing.py
  mercurial/wireprotov1peer.py
  mercurial/wireprotov1server.py
  mercurial/wireprotov2server.py

CHANGE DETAILS

diff --git a/mercurial/wireprotov2server.py b/mercurial/wireprotov2server.py
--- a/mercurial/wireprotov2server.py
+++ b/mercurial/wireprotov2server.py
@@ -965,7 +965,7 @@
 
 @wireprotocommand(b'branchmap', permission=b'pull')
 def branchmapv2(repo, proto):
-yield {encoding.fromlocal(k): v for k, v in repo.branchmap().iteritems()}
+yield {encoding.fromlocal(k): v for k, v in 
pycompat.iteritems(repo.branchmap())}
 
 
 @wireprotocommand(b'capabilities', permission=b'pull')
@@ -1061,7 +1061,7 @@
 # If requested, send bookmarks from nodes that didn't have revision
 # data sent so receiver is aware of any bookmark updates.
 if b'bookmarks' in fields:
-for node, marks in sorted(nodebookmarks.iteritems()):
+for node, marks in sorted(pycompat.iteritems(nodebookmarks)):
 yield {
 b'node': node,
 b'bookmarks': sorted(marks),
@@ -1379,7 +1379,7 @@
 keys = repo.listkeys(encoding.tolocal(namespace))
 keys = {
 encoding.fromlocal(k): encoding.fromlocal(v)
-for k, v in keys.iteritems()
+for k, v in pycompat.iteritems(keys)
 }
 
 yield keys
diff --git a/mercurial/wireprotov1server.py b/mercurial/wireprotov1server.py
--- a/mercurial/wireprotov1server.py
+++ b/mercurial/wireprotov1server.py
@@ -243,7 +243,7 @@
 def branchmap(repo, proto):
 branchmap = repo.branchmap()
 heads = []
-for branch, nodes in branchmap.iteritems():
+for branch, nodes in pycompat.iteritems(branchmap):
 branchname = urlreq.quote(encoding.fromlocal(branch))
 branchnodes = wireprototypes.encodelist(nodes)
 heads.append(b'%s %s' % (branchname, branchnodes))
@@ -440,7 +440,7 @@
 opts = options(
 b'getbundle', wireprototypes.GETBUNDLE_ARGUMENTS.keys(), others
 )
-for k, v in opts.iteritems():
+for k, v in pycompat.iteritems(opts):
 keytype = wireprototypes.GETBUNDLE_ARGUMENTS[k]
 if keytype == b'nodes':
 opts[k] = wireprototypes.decodelist(v)
diff --git 

D7014: py3: define and use pycompat.iteritems() for hgext/

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a reviewer: durin42.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: martinvonz.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  .iteritems() -> .items() is the last source transform being performed.
  But it is also the most widely used.
  
  This commit adds a pycompat.iteritems symbol and imports it in place
  of .iteritems() for usage in hgext/. I chose to stop at just hgext/
  because the patch will be large and it is an easy boundary to stop at
  since we can disable source transformation on a per-package basis.
  
  There are places where the type does implement items() and we could
  call items() directly. However, this would require critical thought
  and I thought it would be easier to just blindly change the code. We
  know which call sites need to be audited in the future because they
  have "pycompat.iteritems."
  
  With this change, we no longer perform source transformation on
  hgext!

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7014

AFFECTED FILES
  hgext/absorb.py
  hgext/convert/bzr.py
  hgext/convert/common.py
  hgext/convert/convcmd.py
  hgext/convert/cvs.py
  hgext/convert/cvsps.py
  hgext/convert/filemap.py
  hgext/convert/hg.py
  hgext/convert/monotone.py
  hgext/convert/subversion.py
  hgext/eol.py
  hgext/fastannotate/context.py
  hgext/fastannotate/protocol.py
  hgext/fix.py
  hgext/fsmonitor/__init__.py
  hgext/githelp.py
  hgext/hgk.py
  hgext/histedit.py
  hgext/infinitepush/__init__.py
  hgext/infinitepush/bundleparts.py
  hgext/infinitepush/sqlindexapi.py
  hgext/journal.py
  hgext/keyword.py
  hgext/largefiles/overrides.py
  hgext/largefiles/remotestore.py
  hgext/lfs/__init__.py
  hgext/lfs/pointer.py
  hgext/lfs/wrapper.py
  hgext/mq.py
  hgext/rebase.py
  hgext/releasenotes.py
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/datapack.py
  hgext/remotefilelog/historypack.py
  hgext/remotefilelog/remotefilelog.py
  hgext/remotefilelog/remotefilelogserver.py
  hgext/remotefilelog/repack.py
  hgext/remotefilelog/shallowrepo.py
  hgext/remotefilelog/shallowutil.py
  hgext/remotenames.py
  hgext/strip.py
  hgext/uncommit.py
  hgext/win32text.py
  mercurial/__init__.py
  mercurial/pycompat.py

CHANGE DETAILS

diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -338,6 +338,7 @@
 return [a.encode('latin-1') for a in ret]
 
 shlexquote = shlex.quote
+iteritems = lambda x: x.items()
 itervalues = lambda x: x.values()
 
 else:
@@ -417,6 +418,7 @@
 ziplist = zip
 rawinput = raw_input
 getargspec = inspect.getargspec
+iteritems = lambda x: x.iteritems()
 itervalues = lambda x: x.itervalues()
 
 isjython = sysplatform.startswith(b'java')
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -31,7 +31,7 @@
 
 def find_spec(self, fullname, path, target=None):
 # Only handle Mercurial-related modules.
-if not fullname.startswith(('mercurial.', 'hgext.')):
+if not fullname.startswith('mercurial.'):
 return None
 # don't try to parse binary
 if fullname.startswith('mercurial.cext.'):
@@ -46,9 +46,6 @@
 # don't try and mangle it
 if fullname.startswith('mercurial.rustext'):
 return None
-# pywatchman is already dual-version clean, don't try and mangle it
-if fullname.startswith('hgext.fsmonitor.pywatchman'):
-return None
 
 # Try to find the module using other registered finders.
 spec = None
@@ -131,7 +128,7 @@
 # ``replacetoken`` or any mechanism that changes semantics of module
 # loading is changed. Otherwise cached bytecode may get loaded without
 # the new transformation mechanisms applied.
-BYTECODEHEADER = b'HG\x00\x15'
+BYTECODEHEADER = b'HG\x00\x16'
 
 class hgloader(importlib.machinery.SourceFileLoader):
 """Custom module loader that transforms source code.
diff --git a/hgext/win32text.py b/hgext/win32text.py
--- a/hgext/win32text.py
+++ b/hgext/win32text.py
@@ -209,7 +209,7 @@
 def reposetup(ui, repo):
 if not repo.local():
 return
-for name, fn in _filters.iteritems():
+for name, fn in pycompat.iteritems(_filters):
 repo.adddatafilter(name, fn)
 
 
diff --git a/hgext/uncommit.py b/hgext/uncommit.py
--- a/hgext/uncommit.py
+++ b/hgext/uncommit.py
@@ -78,7 +78,7 @@
 files = initialfiles - exclude
 # Filter copies
 copied = copiesmod.pathcopies(base, ctx)
-copied = dict((dst, src) for dst, src in copied.iteritems() if dst in 
files)
+copied = dict((dst, src) for dst, src in pycompat.iteritems(copied) if dst 
in files)
 
 def filectxfn(repo, memctx, path, contentctx=ctx, redirect=()):
  

D7018: tests: allow warning about file

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The code is protected by a block that makes it Python 2 only, so
  use of file is acceptable.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7018

AFFECTED FILES
  tests/test-check-pyflakes.t

CHANGE DETAILS

diff --git a/tests/test-check-pyflakes.t b/tests/test-check-pyflakes.t
--- a/tests/test-check-pyflakes.t
+++ b/tests/test-check-pyflakes.t
@@ -23,4 +23,5 @@
   > | xargs pyflakes 2>/dev/null | "$TESTDIR/filterpyflakes.py"
   contrib/perf.py:*: undefined name 'xrange' (glob) (?)
   mercurial/hgweb/server.py:*: undefined name 'reload' (glob) (?)
+  mercurial/util.py:*: undefined name 'file' (glob) (?)
   



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7019: tests: use range() in generate-churning-module.py

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is a test-only script. Performance on Python 2 for creating a
  full list instead of a generator doesn't matter.
  
  With this change, test-check-pyflakes.t passes on Python 3!

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7019

AFFECTED FILES
  tests/artifacts/scripts/generate-churning-bundle.py

CHANGE DETAILS

diff --git a/tests/artifacts/scripts/generate-churning-bundle.py 
b/tests/artifacts/scripts/generate-churning-bundle.py
--- a/tests/artifacts/scripts/generate-churning-bundle.py
+++ b/tests/artifacts/scripts/generate-churning-bundle.py
@@ -62,7 +62,7 @@
 else:
 current = str(iteridx)
 
-for idx in xrange(NB_LINES):
+for idx in range(NB_LINES):
 do_change_line = True
 if oldcontent is not None and ALWAYS_CHANGE_LINES < idx:
 do_change_line = not ((idx - iteridx) % OTHER_CHANGES)
@@ -113,7 +113,7 @@
 hg('init')
 updatefile(FILENAME, None)
 hg('commit', '--addremove', '--message', 'initial commit')
-for idx in xrange(1, NB_CHANGESET + 1):
+for idx in range(1, NB_CHANGESET + 1):
 if sys.stdout.isatty():
 print("generating commit #%d/%d" % (idx, NB_CHANGESET))
 if (idx % PERIOD_BRANCHING) == 0:



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7016: tests: use proper Python 3.8 feature

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Fix typo introduced in 830eacef67f8 
.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7016

AFFECTED FILES
  tests/test-archive.t

CHANGE DETAILS

diff --git a/tests/test-archive.t b/tests/test-archive.t
--- a/tests/test-archive.t
+++ b/tests/test-archive.t
@@ -132,7 +132,7 @@
   transfer-encoding: chunked
   
   body: size=408, sha1=8fa06531bddecc365a9f5edb0f88b65974bfe505 (no-py38 !)
-  body: size=506, sha1=70926a04cb8887d0bcccf5380488100a10222def (py-38 !)
+  body: size=506, sha1=70926a04cb8887d0bcccf5380488100a10222def (py38 !)
   % tar.bz2 and zip disallowed should both give 403
   403 Archive type not allowed: bz2
   content-type: text/html; charset=ascii



To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7017: lsprof: remove __main__ functionality

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I'm pretty sure nobody uses this. I noticed it because Python 3
  linting is complaining about execfile.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7017

AFFECTED FILES
  mercurial/lsprof.py

CHANGE DETAILS

diff --git a/mercurial/lsprof.py b/mercurial/lsprof.py
--- a/mercurial/lsprof.py
+++ b/mercurial/lsprof.py
@@ -143,16 +143,3 @@
 res = res.encode('latin-1')
 
 return res
-
-
-if __name__ == '__main__':
-import os
-
-sys.argv = sys.argv[1:]
-if not sys.argv:
-print(b"usage: lsprof.py 

D7013: py3: define and use pycompat.itervalues()

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a reviewer: martinvonz.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  .itervalues() only exists on Python 2. Python 3's equivalent is
  .values(). But we don't want to blindly use .values() everywhere
  because on Python 2, it will create a list, which will have performance
  implications.
  
  This commit introduces pycompat.itervalues() which will call the appropriate
  method on the passed object. We update all callers of obj.itervalues()
  to pycompat.itervalues(obj) instead.
  
  With this commit, the only source tranforming remaining is for
  iteritems(). Victory is near...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7013

AFFECTED FILES
  contrib/synthrepo.py
  hgext/journal.py
  hgext/rebase.py
  hgext/remotefilelog/connectionpool.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/repack.py
  hgext/transplant.py
  mercurial/__init__.py
  mercurial/branchmap.py
  mercurial/exchangev2.py
  mercurial/localrepo.py
  mercurial/merge.py
  mercurial/patch.py
  mercurial/pycompat.py
  mercurial/statprof.py
  mercurial/ui.py
  tests/test-pathencode.py

CHANGE DETAILS

diff --git a/tests/test-pathencode.py b/tests/test-pathencode.py
--- a/tests/test-pathencode.py
+++ b/tests/test-pathencode.py
@@ -67,7 +67,7 @@
 counts[c] += 1
 for c in '\r/\n':
 counts.pop(c, None)
-t = sum(counts.itervalues()) / 100.0
+t = sum(pycompat.itervalues(counts)) / 100.0
 fp.write('probtable = (')
 for i, (k, v) in enumerate(
 sorted(counts.items(), key=lambda x: x[1], reverse=True)
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1966,7 +1966,7 @@
 if not self._loggers:
 return
 activeloggers = [
-l for l in self._loggers.itervalues() if l.tracked(event)
+l for l in pycompat.itervalues(self._loggers) if l.tracked(event)
 ]
 if not activeloggers:
 return
diff --git a/mercurial/statprof.py b/mercurial/statprof.py
--- a/mercurial/statprof.py
+++ b/mercurial/statprof.py
@@ -475,7 +475,7 @@
 if i == 0:
 sitestat.addself()
 
-return [s for s in stats.itervalues()]
+return [s for s in pycompat.itervalues(stats)]
 
 
 class DisplayFormats:
@@ -744,7 +744,7 @@
 site = node.site
 visiblechildren = [
 c
-for c in node.children.itervalues()
+for c in pycompat.itervalues(node.children)
 if c.count >= (limit * root.count)
 ]
 if site:
@@ -752,7 +752,7 @@
 filename = b''
 function = b''
 if len(node.children) > 0:
-childsite = list(node.children.itervalues())[0].site
+childsite = list(pycompat.itervalues(node.children))[0].site
 filename = (childsite.filename() + b':').ljust(15)
 function = childsite.function
 
@@ -777,7 +777,7 @@
 )
 
 finalstring = liststring + codestring
-childrensamples = sum([c.count for c in 
node.children.itervalues()])
+childrensamples = sum([c.count for c in 
pycompat.itervalues(node.children)])
 # Make frames that performed more than 10% of the operation red
 if node.count - childrensamples > (0.1 * root.count):
 finalstring = b'\033[91m' + finalstring + b'\033[0m'
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -338,6 +338,7 @@
 return [a.encode('latin-1') for a in ret]
 
 shlexquote = shlex.quote
+itervalues = lambda x: x.values()
 
 else:
 import cStringIO
@@ -416,6 +417,7 @@
 ziplist = zip
 rawinput = raw_input
 getargspec = inspect.getargspec
+itervalues = lambda x: x.itervalues()
 
 isjython = sysplatform.startswith(b'java')
 
diff --git a/mercurial/patch.py b/mercurial/patch.py
--- a/mercurial/patch.py
+++ b/mercurial/patch.py
@@ -1326,7 +1326,7 @@
 fixoffset += chunk.removed - chunk.added
 return (
 sum(
-[h for h in applied.itervalues() if h[0].special() or len(h) > 1],
+[h for h in pycompat.itervalues(applied) if h[0].special() or 
len(h) > 1],
 [],
 ),
 {},
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -702,7 +702,7 @@
 """return counts for updated, merged and removed files in this
 session"""
 updated, merged, removed = 0, 0, 0
-for r, action in self._results.itervalues():
+for r, action in pycompat.itervalues(self._results):
 if r is None:
 updated += 1
 elif r == 0:
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- 

D7012: py3: stop normalizing 2nd argument of *attr() to unicode

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that we don't byteify strings, we can stop normalizing the 2nd
  string argument to getattr() and remove explicit overrides we were
  using in the code base.
  
  We no longer use some helper functions in the source transformer,
  so we remove those as well.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7012

AFFECTED FILES
  mercurial/__init__.py
  mercurial/keepalive.py
  mercurial/policy.py
  mercurial/pycompat.py
  mercurial/testing/storage.py
  mercurial/util.py

CHANGE DETAILS

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -623,7 +623,7 @@
 def _fillbuffer(self):
 res = super(observedbufferedinputpipe, self)._fillbuffer()
 
-fn = getattr(self._input._observer, r'osread', None)
+fn = getattr(self._input._observer, 'osread', None)
 if fn:
 fn(res, _chunksize)
 
@@ -634,7 +634,7 @@
 def read(self, size):
 res = super(observedbufferedinputpipe, self).read(size)
 
-fn = getattr(self._input._observer, r'bufferedread', None)
+fn = getattr(self._input._observer, 'bufferedread', None)
 if fn:
 fn(res, size)
 
@@ -643,7 +643,7 @@
 def readline(self, *args, **kwargs):
 res = super(observedbufferedinputpipe, self).readline(*args, **kwargs)
 
-fn = getattr(self._input._observer, r'bufferedreadline', None)
+fn = getattr(self._input._observer, 'bufferedreadline', None)
 if fn:
 fn(res)
 
diff --git a/mercurial/testing/storage.py b/mercurial/testing/storage.py
--- a/mercurial/testing/storage.py
+++ b/mercurial/testing/storage.py
@@ -24,7 +24,7 @@
 
 
 class basetestcase(unittest.TestCase):
-if not getattr(unittest.TestCase, r'assertRaisesRegex', False):
+if not getattr(unittest.TestCase, 'assertRaisesRegex', False):
 assertRaisesRegex = (  # camelcase-required
 unittest.TestCase.assertRaisesRegexp
 )
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -270,7 +270,7 @@
 def getdoc(obj):
 """Get docstring as bytes; may be None so gettext() won't confuse it
 with _('')"""
-doc = getattr(obj, u'__doc__', None)
+doc = getattr(obj, '__doc__', None)
 if doc is None:
 return doc
 return sysbytes(doc)
diff --git a/mercurial/policy.py b/mercurial/policy.py
--- a/mercurial/policy.py
+++ b/mercurial/policy.py
@@ -70,7 +70,7 @@
 except AttributeError:
 raise ImportError(r'cannot import name %s' % modname)
 # force import; fakelocals[modname] may be replaced with the real module
-getattr(mod, r'__doc__', None)
+getattr(mod, '__doc__', None)
 return fakelocals[modname]
 
 
@@ -94,7 +94,7 @@
 
 def _checkmod(pkgname, modname, mod):
 expected = _cextversions.get((pkgname, modname))
-actual = getattr(mod, r'version', None)
+actual = getattr(mod, 'version', None)
 if actual != expected:
 raise ImportError(
 r'cannot import module %s.%s '
diff --git a/mercurial/keepalive.py b/mercurial/keepalive.py
--- a/mercurial/keepalive.py
+++ b/mercurial/keepalive.py
@@ -255,7 +255,7 @@
 # If not a persistent connection, don't try to reuse it. Look
 # for this using getattr() since vcr doesn't define this
 # attribute, and in that case always close the connection.
-if getattr(r, r'will_close', True):
+if getattr(r, 'will_close', True):
 self._cm.remove(h)
 
 if DEBUG:
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -110,65 +110,14 @@
 except IndexError:
 return False
 
-def _findargnofcall(n):
-"""Find arg n of a call expression (start at 0)
-
-Returns index of the first token of that argument, or None if
-there is not that many arguments.
-
-Assumes that token[i + 1] is '('.
-
-"""
-nested = 0
-for j in range(i + 2, len(tokens)):
-if _isop(j, ')', ']', '}'):
-# end of call, tuple, subscription or dict / set
-nested -= 1
-if nested < 0:
-return None
-elif n == 0:
-# this is the starting position of arg
-return j
-elif _isop(j, '(', '[', '{'):
-nested += 1
-elif _isop(j, ',') and nested == 0:
-n -= 1
-
-return None
-
-def _ensureunicode(j):
-"""Make sure the token at j is a unicode string
-
-This rewrites a string token to include the unicode 

Re: black and byteify-strings breaking your patches

[Pierre-Yves David]

SPOILER WARNING: do not use the "old" example for fix configuration, it 
eats your data. Use the version currently in `hg-committed` (with the 
trailing `-` for the command).


On 10/6/19 6:13 PM, Augie Fackler wrote:

Folks, we just landed my mass-reformatting patches that start unwinding the 
source mangler in mercurial/__init__.py. As a result, your patches are probably 
all unable to apply right now. To fix up your code, you'll probably want to do 
something like this:

1) rebase your changes to 9cc55b743713 (the last change before the formatting)
2) rebase again to 687b865b95ad (the second of the two changes that rewrote 
everything)
3) When you get merge conflicts in (2), resolve them all with your edits, and 
follow the instructions in 687b865b95ad's commit message on how the change was 
generated. Look before you leap on accepting edits to make sure things are 
reasonable.


This solution worked fairly okay, for getting over the black formatting. 
I "just" needed to run `hg fix` on full my series after rebasing on 
687b865b95ad.


However, getting past 687b865b95ad (byteifying everything) is a larger 
issue since `hg fix` and `byteify-strings.py` cannot work together (pipe 
input/ouput vs file input/output).


To work around this, I created a temporary draft changeset in my 
repository for format-source to deal with the conflict (format-source 
has support for file input/output).


You can get this changeset using:

hg pull --rev "FORMAT-WORK-AROUND-DO-NOT-PUBLISH" 
https://www.mercurial-scm.org/repo/users/marmoute/blackgnarock/

then run:
hg up "FORMAT-WORK-AROUND-DO-NOT-PUBLISH"
hg rebase --dest "FORMAT-WORK-AROUND-DO-NOT-PUBLISH" --keepbranch 
--rev YOURREVS
This will rebase your changes past both the blackgnarock and the 
bytemageddon. Once this is done, you can rebase your changes back to the 
hg-committed content (to remove the format source thing)
 hg rebase  --config format-source.run-mode=off --dest 9002f4a3dde6 
--keepbranch --rev 'children("FORMAT-WORK-AROUND-DO-NOT-PUBLISH")::'



install format source from PyPI::

pip install --user hg-format-source

You need to have a python install with a development version of black::

virtualenv --python python3 ~/blackgnarock/
~/blackgnarock/bin/pip install git+https://github.com/psf/black/

Then add the following to your repository config::

[extensions]
formatsource=
[format-source]
# run format-source more aggressively, might slow down merges
run-mode=on
# configure the various formatters
byteify = python3 PATH/TO/contrib/byteify-strings.py -i
byteify:mode = file
grey = ~/blackgnarock/bin/python3 PATH/TO/contrib/grey.py --quiet 
--skip-string-normalization -

grey:mode = pipe
[fix]
black:command = ~/blackgnarock/bin/python3 PATH/TO/contrib/grey.py 
-S --quiet --skip-string-normalization -
black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
mercurial/thirdparty/** - "contrib/python-zstandard/** - contrib/grey.py"



Enjoy your formatted source code,

Cheers.


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 BLACKGNAROCK-V2] formatting: introduce a `test-check-format-black.t` that enforce formatting

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570463578 14400
#  Mon Oct 07 11:52:58 2019 -0400
# Node ID c5294d305666f3b5de9a001c9327ec8eab97eee5
# Parent  3c5fc04c5cbc3b146105b0e6764634c77067dbd5
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
c5294d305666
formatting: introduce a `test-check-format-black.t` that enforce formatting

This should prevent use to drift away from the expect format.

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -1,4 +1,4 @@
-from __future__ import absolute_import
+from __future__ import absolute_import, print_function
 
 import os
 import re
@@ -972,3 +972,14 @@ def has_emacs():
 # 24.4, so we allow emacs 24.4, 24.5, and 25+ (24.5 was the last
 # 24 release)
 return matchoutput('emacs --version', b'GNU Emacs 2(4.4|4.5|5|6|7|8|9)')
+
+
+# @check('black', 'the black formatter for python')
+@check('grey', 'grey, the fork of the black formatter for python')
+def has_black():
+# use that to actual black as soon as possible
+# blackcmd = b'black --version'
+blackcmd = b'python3 $RUNTESTDIR/../contrib/grey.py --version'
+# version_regex = b'black, version \d'
+version_regex = b'grey.py, version \d'
+return matchoutput(blackcmd, version_regex)
diff --git a/tests/test-check-format.t b/tests/test-check-format.t
new file mode 100644
--- /dev/null
+++ b/tests/test-check-format.t
@@ -0,0 +1,7 @@
+#require grey
+
+(this should use the actual black as soon as possible)
+
+  $ cd $RUNTESTDIR/..
+  $ python3 contrib/grey.py -S --quiet --check --diff `hg files 'set:**.py - 
hgext/fsmonitor/pywatchman/** - mercurial/thirdparty/** - 
"contrib/python-zstandard/**" - contrib/grey.py'`
+
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 BLACKGNAROCK-V2] formatting: run black on all file again

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570460331 14400
#  Mon Oct 07 10:58:51 2019 -0400
# Node ID 3c5fc04c5cbc3b146105b0e6764634c77067dbd5
# Parent  9002f4a3dde63518d1467f3934a1fe226f3b2297
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
3c5fc04c5cbc
formatting: run black on all file again

Apparently, since the blackgnarok, we divergence from the expected formatting.

Formatted using::

  grey.py -S $(hg files 'set:**.py - mercurial/thirdparty/** - 
"contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/** - 
contrib/grey.py')

# skip-blame mass-reformatting only

# no-check-commit reformats foo_bar functions

diff --git a/doc/check-seclevel.py b/doc/check-seclevel.py
--- a/doc/check-seclevel.py
+++ b/doc/check-seclevel.py
@@ -70,7 +70,9 @@ def checkseclevel(ui, doc, name, initlev
 continue
 nextlevel = mark2level[mark]
 if curlevel < nextlevel and curlevel + 1 != nextlevel:
-ui.warnnoi18n('gap of section level at "%s" of %s\n' % (title, 
name))
+ui.warnnoi18n(
+'gap of section level at "%s" of %s\n' % (title, name)
+)
 showavailables(ui, initlevel)
 errorcnt += 1
 continue
@@ -88,7 +90,9 @@ def checkcmdtable(ui, cmdtable, namefmt,
 for k, entry in cmdtable.items():
 name = k.split(b"|")[0].lstrip(b"^")
 if not entry[0].__doc__:
-ui.notenoi18n('skip checking %s: no help document\n' % (namefmt % 
name))
+ui.notenoi18n(
+'skip checking %s: no help document\n' % (namefmt % name)
+)
 continue
 errorcnt += checkseclevel(
 ui, entry[0].__doc__, namefmt % name, initlevel
@@ -113,7 +117,9 @@ def checkhghelps(ui):
 ):
 mod = extensions.load(ui, name, None)
 if not mod.__doc__:
-ui.notenoi18n('skip checking %s extension: no help document\n' % 
name)
+ui.notenoi18n(
+'skip checking %s extension: no help document\n' % name
+)
 continue
 errorcnt += checkseclevel(
 ui, mod.__doc__, '%s extension' % name, initlevel_ext
diff --git a/hgext/phabricator.py b/hgext/phabricator.py
--- a/hgext/phabricator.py
+++ b/hgext/phabricator.py
@@ -775,7 +775,9 @@ def phabsend(ui, repo, *revs, **opts):
 try:
 writediffproperties(unfi[newnode], diffmap[old.node()])
 except util.urlerr.urlerror:
-ui.warnnoi18n(b'Failed to update metadata for D%s\n' % 
drevid)
+ui.warnnoi18n(
+b'Failed to update metadata for D%s\n' % drevid
+)
 # Remove local tags since it's no longer necessary
 tagname = b'D%d' % drevid
 if tagname in repo.tags():
diff --git a/hgext/win32mbcs.py b/hgext/win32mbcs.py
--- a/hgext/win32mbcs.py
+++ b/hgext/win32mbcs.py
@@ -216,4 +216,6 @@ def extsetup(ui):
 # command line options is not yet applied when
 # extensions.loadall() is called.
 if b'--debug' in sys.argv:
-ui.writenoi18n(b"[win32mbcs] activated with encoding: %s\n" % 
_encoding)
+ui.writenoi18n(
+b"[win32mbcs] activated with encoding: %s\n" % _encoding
+)
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -1250,11 +1250,17 @@ def debugformat(ui, repo, **opts):
 def debugfsinfo(ui, path=b"."):
 """show information detected about current filesystem"""
 ui.writenoi18n(b'path: %s\n' % path)
-ui.writenoi18n(b'mounted on: %s\n' % (util.getfsmountpoint(path) or 
b'(unknown)'))
+ui.writenoi18n(
+b'mounted on: %s\n' % (util.getfsmountpoint(path) or b'(unknown)')
+)
 ui.writenoi18n(b'exec: %s\n' % (util.checkexec(path) and b'yes' or b'no'))
 ui.writenoi18n(b'fstype: %s\n' % (util.getfstype(path) or b'(unknown)'))
-ui.writenoi18n(b'symlink: %s\n' % (util.checklink(path) and b'yes' or 
b'no'))
-ui.writenoi18n(b'hardlink: %s\n' % (util.checknlink(path) and b'yes' or 
b'no'))
+ui.writenoi18n(
+b'symlink: %s\n' % (util.checklink(path) and b'yes' or b'no')
+)
+ui.writenoi18n(
+b'hardlink: %s\n' % (util.checknlink(path) and b'yes' or b'no')
+)
 casesensitive = b'(unknown)'
 try:
 with pycompat.namedtempfile(prefix=b'.debugfsinfo', dir=path) as f:
@@ -1938,7 +1944,9 @@ def debugmergestate(ui, repo, *args):
 ui.writenoi18n(b'other: %s\n' % record)
 elif rtype == b'm':
 driver, mdstate = record.split(b'\0', 1)
-ui.writenoi18n(b'merge driver: %s (state "%s")\n' % (driver, 
mdstate))
+

Re: [PATCH 4 of 5 BLACKGNAROK] formatting: make black --quiet in the example `hg fix` config

[Augie Fackler]

queued patches 2 and 4, thanks

> On Oct 7, 2019, at 12:15, Pierre-Yves David  
> wrote:
> 
> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1570463494 14400
> #  Mon Oct 07 11:51:34 2019 -0400
> # Node ID 423493e0127d6328bc948025886a8c8fe4150957
> # Parent  176c569d499555cc64522bf8bd1e5acddc92a991
> # EXP-Topic blackgnarok-bytemageddon
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 423493e0127d
> formatting: make black --quiet in the example `hg fix` config
> 
> I do like cake, but I cannot have so many of them.
> 
> diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
> --- a/contrib/examples/fix.hgrc
> +++ b/contrib/examples/fix.hgrc
> @@ -11,5 +11,5 @@ rustfmt:pattern = set:**.rs
> # 
> git+https://github.com/python/black/@d9e71a75ccfefa3d9156a64c03313a0d4ad981e5
> # to have the dependencies for grey.
> #
> -# black:command = python3.7 contrib/grey.py -l 80 -t py33 
> --skip-string-normalization -
> +# black:command = python3.7 contrib/grey.py -l 80 -t py33 --quiet 
> --skip-string-normalization -
> # black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
> mercurial/thirdparty/** - "contrib/python-zstandard/**"

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 5 of 5 BLACKGNAROK] formatting: introduce a `test-check-format-black.t` that enforce formatting

[Augie Fackler]

> On Oct 7, 2019, at 12:15, Pierre-Yves David  
> wrote:
> 
> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1570463578 14400
> #  Mon Oct 07 11:52:58 2019 -0400
> # Node ID 3dd70df24ffa47698aa456f7640393537bc266b2
> # Parent  423493e0127d6328bc948025886a8c8fe4150957
> # EXP-Topic blackgnarok-bytemageddon
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 3dd70df24ffa
> formatting: introduce a `test-check-format-black.t` that enforce formatting
> 
> This should prevent use to drift away from the expect format.
> 
> diff --git a/tests/hghave.py b/tests/hghave.py
> --- a/tests/hghave.py
> +++ b/tests/hghave.py
> @@ -1,4 +1,4 @@
> -from __future__ import absolute_import
> +from __future__ import absolute_import, print_function
> 
> import os
> import re
> @@ -972,3 +972,14 @@ def has_emacs():
> # 24.4, so we allow emacs 24.4, 24.5, and 25+ (24.5 was the last
> # 24 release)
> return matchoutput('emacs --version', b'GNU Emacs 2(4.4|4.5|5|6|7|8|9)')
> +
> +
> +# @check('black', 'the black formatter for python')
> +@check('grey', 'grey, the fork of the black formatter for python')
> +def has_black():
> +# use that to actual black as soon as possible
> +# blackcmd = b'black --version'
> +blackcmd = b'python3 $RUNTESTDIR/../contrib/grey.py --version'
> +# version_regex = b'black, version \d'
> +version_regex = b'grey.py, version \d'
> +return matchoutput(blackcmd, version_regex)
> diff --git a/tests/test-check-format.t b/tests/test-check-format.t
> new file mode 100644
> --- /dev/null
> +++ b/tests/test-check-format.t
> @@ -0,0 +1,7 @@
> +#require grey
> +
> +(this should use the actual black as soon as possible)
> +
> +  $ cd $RUNTESTDIR/..
> +  $ python3 contrib/grey.py -l 80 -t py33 -S --quiet --check --diff `hg 
> files 'set:**.py - hgext/fsmonitor/pywatchman/** - mercurial/thirdparty/** - 
> "contrib/python-zstandard/**"'`

omit -l and -t options so pyproject.toml is authoritative

> +

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 5 BLACKGNAROK] formatting: update example config to match the setting we uses

[Augie Fackler]

> On Oct 7, 2019, at 12:15, Pierre-Yves David  
> wrote:
> 
> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1570463432 14400
> #  Mon Oct 07 11:50:32 2019 -0400
> # Node ID 176c569d499555cc64522bf8bd1e5acddc92a991
> # Parent  99946ed8e49657d0678426b9fda3958bd102ed67
> # EXP-Topic blackgnarok-bytemageddon
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 176c569d4995
> formatting: update example config to match the setting we uses
> 
> The commit message doing formatting mention this invocation and it seems to
> match what we have in the repo. So we update the example config.

We have a pyproject.toml that (in theory) should take care of the line length 
param. I don't think the -t py33 is required at all.

> 
> diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
> --- a/contrib/examples/fix.hgrc
> +++ b/contrib/examples/fix.hgrc
> @@ -11,5 +11,5 @@ rustfmt:pattern = set:**.rs
> # 
> git+https://github.com/python/black/@d9e71a75ccfefa3d9156a64c03313a0d4ad981e5
> # to have the dependencies for grey.
> #
> -# black:command = python3.7 contrib/grey.py --skip-string-normalization -
> +# black:command = python3.7 contrib/grey.py -l 80 -t py33 
> --skip-string-normalization -
> # black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
> mercurial/thirdparty/** - "contrib/python-zstandard/**"

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 5 BLACKGNAROK] formatting: run black on all file again

[Augie Fackler]

> On Oct 7, 2019, at 12:15, Pierre-Yves David  
> wrote:
> 
> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1570460331 14400
> #  Mon Oct 07 10:58:51 2019 -0400
> # Node ID f243703025e2ba5a53feb6f8ca8f029b6e186194
> # Parent  7054fd370430ae76b07ff5f68d0ab8df9de70fc5
> # EXP-Topic blackgnarok-bytemageddon
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> f243703025e2
> formatting: run black on all file again
> 
> Apparently, since the blackgnarok, we divergence from the expected formatting.
> 
> Formatted using::
> 
>  grey.py -l 80 -t py33 -S $(hg files 'set:**.py - mercurial/thirdparty/** - 
> "contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**')

we should also exclude grey.py, so that it's easy to pick up updates from black 
if we have to rebase the patch...

> 
> # skip-blame mass-reformatting only
> 
> # no-check-commit reformats foo_bar functions
> 
> diff --git a/contrib/grey.py b/contrib/grey.py
> --- a/contrib/grey.py
> +++ b/contrib/grey.py
> @@ -55,9 +55,7 @@ from blib2to3.pgen2.parse import ParseEr
> __version__ = '19.3b1.dev95+gdc1add6.d20191005'
> 
> DEFAULT_LINE_LENGTH = 88
> -DEFAULT_EXCLUDES = (
> -
> r"/(\.eggs|\.git|\.hg|\.mypy_cache|\.nox|\.tox|\.venv|_build|buck-out|build|dist)/"
> -)
> +DEFAULT_EXCLUDES = 
> r"/(\.eggs|\.git|\.hg|\.mypy_cache|\.nox|\.tox|\.venv|_build|buck-out|build|dist)/"
> DEFAULT_INCLUDES = r"\.pyi?$"
> CACHE_DIR = Path(user_cache_dir("black", version=__version__))
> 
> @@ -196,7 +194,9 @@ class FileMode:
> if self.target_versions:
> version_str = ",".join(
> str(version.value)
> -for version in sorted(self.target_versions, key=lambda v: 
> v.value)
> +for version in sorted(
> +self.target_versions, key=lambda v: v.value
> +)
> )
> else:
> version_str = "-"
> @@ -209,12 +209,18 @@ class FileMode:
> return ".".join(parts)
> 
> 
> -def supports_feature(target_versions: Set[TargetVersion], feature: Feature) 
> -> bool:
> -return all(feature in VERSION_TO_FEATURES[version] for version in 
> target_versions)
> +def supports_feature(
> +target_versions: Set[TargetVersion], feature: Feature
> +) -> bool:
> +return all(
> +feature in VERSION_TO_FEATURES[version] for version in 
> target_versions
> +)
> 
> 
> def read_pyproject_toml(
> -ctx: click.Context, param: click.Parameter, value: Union[str, int, bool, 
> None]
> +ctx: click.Context,
> +param: click.Parameter,
> +value: Union[str, int, bool, None],
> ) -> Optional[str]:
> """Inject Black configuration from "pyproject.toml" into defaults in 
> `ctx`.
> 
> @@ -250,7 +256,9 @@ def read_pyproject_toml(
> 
> 
> @click.command(context_settings=dict(help_option_names=["-h", "--help"]))
> -@click.option("-c", "--code", type=str, help="Format the code passed in as a 
> string.")
> +@click.option(
> +"-c", "--code", type=str, help="Format the code passed in as a string."
> +)
> @click.option(
> "-l",
> "--line-length",
> @@ -361,14 +369,22 @@ def read_pyproject_toml(
> "src",
> nargs=-1,
> type=click.Path(
> -exists=True, file_okay=True, dir_okay=True, readable=True, 
> allow_dash=True
> +exists=True,
> +file_okay=True,
> +dir_okay=True,
> +readable=True,
> +allow_dash=True,
> ),
> is_eager=True,
> )
> @click.option(
> "--config",
> type=click.Path(
> -exists=False, file_okay=True, dir_okay=False, readable=True, 
> allow_dash=False
> +exists=False,
> +file_okay=True,
> +dir_okay=False,
> +readable=True,
> +allow_dash=False,
> ),
> is_eager=True,
> callback=read_pyproject_toml,
> @@ -439,7 +455,9 @@ def main(
> p = Path(s)
> if p.is_dir():
> sources.update(
> -gen_python_files_in_dir(p, root, include_regex, 
> exclude_regex, report)
> +gen_python_files_in_dir(
> +p, root, include_regex, exclude_regex, report
> +)
> )
> elif p.is_file() or s == "-":
> # if a file was explicitly given, we don't care about its 
> extension
> @@ -461,7 +479,11 @@ def main(
> )
> else:
> reformat_many(
> -sources=sources, fast=fast, write_back=write_back, mode=mode, 
> report=report
> +sources=sources,
> +fast=fast,
> +write_back=write_back,
> +mode=mode,
> +report=report,
> )
> 
> if verbose or not quiet:
> @@ -470,7 +492,9 @@ def main(
> ctx.exit(report.return_code)
> 
> 
> -def path_empty(src: Tuple[str], quiet: bool, verbose: bool, ctx: 
> click.Context) -> None:
> +def path_empty(
> +src: Tuple[str], quiet: bool, verbose: bool, ctx: click.Context
> +) -> 

[PATCH 5 of 5 BLACKGNAROK] formatting: introduce a `test-check-format-black.t` that enforce formatting

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570463578 14400
#  Mon Oct 07 11:52:58 2019 -0400
# Node ID 3dd70df24ffa47698aa456f7640393537bc266b2
# Parent  423493e0127d6328bc948025886a8c8fe4150957
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
3dd70df24ffa
formatting: introduce a `test-check-format-black.t` that enforce formatting

This should prevent use to drift away from the expect format.

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -1,4 +1,4 @@
-from __future__ import absolute_import
+from __future__ import absolute_import, print_function
 
 import os
 import re
@@ -972,3 +972,14 @@ def has_emacs():
 # 24.4, so we allow emacs 24.4, 24.5, and 25+ (24.5 was the last
 # 24 release)
 return matchoutput('emacs --version', b'GNU Emacs 2(4.4|4.5|5|6|7|8|9)')
+
+
+# @check('black', 'the black formatter for python')
+@check('grey', 'grey, the fork of the black formatter for python')
+def has_black():
+# use that to actual black as soon as possible
+# blackcmd = b'black --version'
+blackcmd = b'python3 $RUNTESTDIR/../contrib/grey.py --version'
+# version_regex = b'black, version \d'
+version_regex = b'grey.py, version \d'
+return matchoutput(blackcmd, version_regex)
diff --git a/tests/test-check-format.t b/tests/test-check-format.t
new file mode 100644
--- /dev/null
+++ b/tests/test-check-format.t
@@ -0,0 +1,7 @@
+#require grey
+
+(this should use the actual black as soon as possible)
+
+  $ cd $RUNTESTDIR/..
+  $ python3 contrib/grey.py -l 80 -t py33 -S --quiet --check --diff `hg files 
'set:**.py - hgext/fsmonitor/pywatchman/** - mercurial/thirdparty/** - 
"contrib/python-zstandard/**"'`
+
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 5 BLACKGNAROK] formatting: make black --quiet in the example `hg fix` config

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570463494 14400
#  Mon Oct 07 11:51:34 2019 -0400
# Node ID 423493e0127d6328bc948025886a8c8fe4150957
# Parent  176c569d499555cc64522bf8bd1e5acddc92a991
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
423493e0127d
formatting: make black --quiet in the example `hg fix` config

I do like cake, but I cannot have so many of them.

diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
--- a/contrib/examples/fix.hgrc
+++ b/contrib/examples/fix.hgrc
@@ -11,5 +11,5 @@ rustfmt:pattern = set:**.rs
 # git+https://github.com/python/black/@d9e71a75ccfefa3d9156a64c03313a0d4ad981e5
 # to have the dependencies for grey.
 #
-# black:command = python3.7 contrib/grey.py -l 80 -t py33 
--skip-string-normalization -
+# black:command = python3.7 contrib/grey.py -l 80 -t py33 --quiet 
--skip-string-normalization -
 # black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
mercurial/thirdparty/** - "contrib/python-zstandard/**"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 5 BLACKGNAROK] formatting: update example config to match the setting we uses

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570463432 14400
#  Mon Oct 07 11:50:32 2019 -0400
# Node ID 176c569d499555cc64522bf8bd1e5acddc92a991
# Parent  99946ed8e49657d0678426b9fda3958bd102ed67
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
176c569d4995
formatting: update example config to match the setting we uses

The commit message doing formatting mention this invocation and it seems to
match what we have in the repo. So we update the example config.

diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
--- a/contrib/examples/fix.hgrc
+++ b/contrib/examples/fix.hgrc
@@ -11,5 +11,5 @@ rustfmt:pattern = set:**.rs
 # git+https://github.com/python/black/@d9e71a75ccfefa3d9156a64c03313a0d4ad981e5
 # to have the dependencies for grey.
 #
-# black:command = python3.7 contrib/grey.py --skip-string-normalization -
+# black:command = python3.7 contrib/grey.py -l 80 -t py33 
--skip-string-normalization -
 # black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
mercurial/thirdparty/** - "contrib/python-zstandard/**"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5 BLACKGNAROK] formatting: remove the data-ogre from the config example

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570463314 14400
#  Mon Oct 07 11:48:34 2019 -0400
# Node ID 99946ed8e49657d0678426b9fda3958bd102ed67
# Parent  f243703025e2ba5a53feb6f8ca8f029b6e186194
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
99946ed8e496
formatting: remove the data-ogre from the config example

Without he final `-`, black ignore the stdin and just looks at file provided as
command line argument.

Since `hg fix` feeds the file content through stdin and does not pass file
argument, this meant black happily exited successful (all files passed as
argument were formatted) without any output. Fix picked this "no output" as the
new file content, deleting all previous content.

I appreciate the fact this effectively removed all buggy code in any files
processing that way, but this also ate all my data.

The example config is now fixed in that regards.

diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
--- a/contrib/examples/fix.hgrc
+++ b/contrib/examples/fix.hgrc
@@ -11,5 +11,5 @@ rustfmt:pattern = set:**.rs
 # git+https://github.com/python/black/@d9e71a75ccfefa3d9156a64c03313a0d4ad981e5
 # to have the dependencies for grey.
 #
-# black:command = python3.7 contrib/grey.py --skip-string-normalization
+# black:command = python3.7 contrib/grey.py --skip-string-normalization -
 # black:pattern = set:**.py - hgext/fsmonitor/pywatchman/** - 
mercurial/thirdparty/** - "contrib/python-zstandard/**"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5 BLACKGNAROK] formatting: run black on all file again

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1570460331 14400
#  Mon Oct 07 10:58:51 2019 -0400
# Node ID f243703025e2ba5a53feb6f8ca8f029b6e186194
# Parent  7054fd370430ae76b07ff5f68d0ab8df9de70fc5
# EXP-Topic blackgnarok-bytemageddon
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
f243703025e2
formatting: run black on all file again

Apparently, since the blackgnarok, we divergence from the expected formatting.

Formatted using::

  grey.py -l 80 -t py33 -S $(hg files 'set:**.py - mercurial/thirdparty/** - 
"contrib/python-zstandard/**" - hgext/fsmonitor/pywatchman/**')

# skip-blame mass-reformatting only

# no-check-commit reformats foo_bar functions

diff --git a/contrib/grey.py b/contrib/grey.py
--- a/contrib/grey.py
+++ b/contrib/grey.py
@@ -55,9 +55,7 @@ from blib2to3.pgen2.parse import ParseEr
 __version__ = '19.3b1.dev95+gdc1add6.d20191005'
 
 DEFAULT_LINE_LENGTH = 88
-DEFAULT_EXCLUDES = (
-
r"/(\.eggs|\.git|\.hg|\.mypy_cache|\.nox|\.tox|\.venv|_build|buck-out|build|dist)/"
-)
+DEFAULT_EXCLUDES = 
r"/(\.eggs|\.git|\.hg|\.mypy_cache|\.nox|\.tox|\.venv|_build|buck-out|build|dist)/"
 DEFAULT_INCLUDES = r"\.pyi?$"
 CACHE_DIR = Path(user_cache_dir("black", version=__version__))
 
@@ -196,7 +194,9 @@ class FileMode:
 if self.target_versions:
 version_str = ",".join(
 str(version.value)
-for version in sorted(self.target_versions, key=lambda v: 
v.value)
+for version in sorted(
+self.target_versions, key=lambda v: v.value
+)
 )
 else:
 version_str = "-"
@@ -209,12 +209,18 @@ class FileMode:
 return ".".join(parts)
 
 
-def supports_feature(target_versions: Set[TargetVersion], feature: Feature) -> 
bool:
-return all(feature in VERSION_TO_FEATURES[version] for version in 
target_versions)
+def supports_feature(
+target_versions: Set[TargetVersion], feature: Feature
+) -> bool:
+return all(
+feature in VERSION_TO_FEATURES[version] for version in target_versions
+)
 
 
 def read_pyproject_toml(
-ctx: click.Context, param: click.Parameter, value: Union[str, int, bool, 
None]
+ctx: click.Context,
+param: click.Parameter,
+value: Union[str, int, bool, None],
 ) -> Optional[str]:
 """Inject Black configuration from "pyproject.toml" into defaults in `ctx`.
 
@@ -250,7 +256,9 @@ def read_pyproject_toml(
 
 
 @click.command(context_settings=dict(help_option_names=["-h", "--help"]))
-@click.option("-c", "--code", type=str, help="Format the code passed in as a 
string.")
+@click.option(
+"-c", "--code", type=str, help="Format the code passed in as a string."
+)
 @click.option(
 "-l",
 "--line-length",
@@ -361,14 +369,22 @@ def read_pyproject_toml(
 "src",
 nargs=-1,
 type=click.Path(
-exists=True, file_okay=True, dir_okay=True, readable=True, 
allow_dash=True
+exists=True,
+file_okay=True,
+dir_okay=True,
+readable=True,
+allow_dash=True,
 ),
 is_eager=True,
 )
 @click.option(
 "--config",
 type=click.Path(
-exists=False, file_okay=True, dir_okay=False, readable=True, 
allow_dash=False
+exists=False,
+file_okay=True,
+dir_okay=False,
+readable=True,
+allow_dash=False,
 ),
 is_eager=True,
 callback=read_pyproject_toml,
@@ -439,7 +455,9 @@ def main(
 p = Path(s)
 if p.is_dir():
 sources.update(
-gen_python_files_in_dir(p, root, include_regex, exclude_regex, 
report)
+gen_python_files_in_dir(
+p, root, include_regex, exclude_regex, report
+)
 )
 elif p.is_file() or s == "-":
 # if a file was explicitly given, we don't care about its extension
@@ -461,7 +479,11 @@ def main(
 )
 else:
 reformat_many(
-sources=sources, fast=fast, write_back=write_back, mode=mode, 
report=report
+sources=sources,
+fast=fast,
+write_back=write_back,
+mode=mode,
+report=report,
 )
 
 if verbose or not quiet:
@@ -470,7 +492,9 @@ def main(
 ctx.exit(report.return_code)
 
 
-def path_empty(src: Tuple[str], quiet: bool, verbose: bool, ctx: 
click.Context) -> None:
+def path_empty(
+src: Tuple[str], quiet: bool, verbose: bool, ctx: click.Context
+) -> None:
 """
 Exit if there is no `src` provided for formatting
 """
@@ -481,7 +505,11 @@ def path_empty(src: Tuple[str], quiet: b
 
 
 def reformat_one(
-src: Path, fast: bool, write_back: WriteBack, mode: FileMode, report: 
"Report"
+src: Path,
+fast: bool,
+write_back: WriteBack,
+mode: FileMode,
+report: "Report",
 ) -> None:
 """Reformat a single file under `src` without spawning child 

D6989: push: support config option to require revs be specified when running push

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D6989#102437 , @indygreg 
wrote:
  
  > Question: should this checking be performed in `exchange.push` or at the 
command layer?
  > (I'm not sure of the answer.)
  
  A problem with moving it to exchange is that subrepos implicitly (AFAICT) 
push all revisions with `exchange.push`, even when a subset of revisions in the 
parent is pushed.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6989/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6989

To: spectral, #hg-reviewers
Cc: mharbison72, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7009: py3: manually import getattr where it is needed

[indygreg (Gregory Szorc)]

Closed by commit rHGc59eb1560c44: py3: manually import getattr where it is 
needed (authored by indygreg).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7009?vs=16924=16932

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7009/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7009

AFFECTED FILES
  hgext/convert/cvs.py
  hgext/convert/transport.py
  hgext/fastannotate/context.py
  hgext/fastannotate/support.py
  hgext/histedit.py
  hgext/infinitepush/__init__.py
  hgext/keyword.py
  hgext/largefiles/storefactory.py
  hgext/lfs/blobstore.py
  hgext/lfs/wrapper.py
  hgext/mq.py
  hgext/phabricator.py
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/contentstore.py
  hgext/strip.py
  hgext/win32mbcs.py
  mercurial/__init__.py
  mercurial/bookmarks.py
  mercurial/chgserver.py
  mercurial/cmdutil.py
  mercurial/color.py
  mercurial/commandserver.py
  mercurial/config.py
  mercurial/context.py
  mercurial/crecord.py
  mercurial/debugcommands.py
  mercurial/dispatch.py
  mercurial/encoding.py
  mercurial/extensions.py
  mercurial/filemerge.py
  mercurial/fileset.py
  mercurial/filesetlang.py
  mercurial/help.py
  mercurial/hg.py
  mercurial/hgweb/common.py
  mercurial/hgweb/hgweb_mod.py
  mercurial/hgweb/server.py
  mercurial/hgweb/webcommands.py
  mercurial/hgweb/wsgicgi.py
  mercurial/hook.py
  mercurial/httppeer.py
  mercurial/i18n.py
  mercurial/keepalive.py
  mercurial/localrepo.py
  mercurial/lock.py
  mercurial/lsprof.py
  mercurial/mail.py
  mercurial/manifest.py
  mercurial/mdiff.py
  mercurial/narrowspec.py
  mercurial/obsolete.py
  mercurial/phases.py
  mercurial/policy.py
  mercurial/posix.py
  mercurial/profiling.py
  mercurial/pure/osutil.py
  mercurial/pycompat.py
  mercurial/repoview.py
  mercurial/revlog.py
  mercurial/revlogutils/deltas.py
  mercurial/revset.py
  mercurial/revsetlang.py
  mercurial/scmposix.py
  mercurial/scmutil.py
  mercurial/smartset.py
  mercurial/sshpeer.py
  mercurial/sslutil.py
  mercurial/store.py
  mercurial/subrepoutil.py
  mercurial/templater.py
  mercurial/templateutil.py
  mercurial/testing/storage.py
  mercurial/ui.py
  mercurial/unionrepo.py
  mercurial/upgrade.py
  mercurial/url.py
  mercurial/urllibcompat.py
  mercurial/util.py
  mercurial/utils/compression.py
  mercurial/utils/procutil.py
  mercurial/vfs.py
  mercurial/windows.py
  mercurial/wireprotoframing.py
  mercurial/wireprototypes.py
  mercurial/wireprotov1peer.py
  mercurial/wireprotov1server.py

CHANGE DETAILS

diff --git a/mercurial/wireprotov1server.py b/mercurial/wireprotov1server.py
--- a/mercurial/wireprotov1server.py
+++ b/mercurial/wireprotov1server.py
@@ -15,6 +15,7 @@
 hex,
 nullid,
 )
+from .pycompat import getattr
 
 from . import (
 bundle2,
diff --git a/mercurial/wireprotov1peer.py b/mercurial/wireprotov1peer.py
--- a/mercurial/wireprotov1peer.py
+++ b/mercurial/wireprotov1peer.py
@@ -13,7 +13,10 @@
 
 from .i18n import _
 from .node import bin
-from .pycompat import setattr
+from .pycompat import (
+getattr,
+setattr,
+)
 from . import (
 bundle2,
 changegroup as changegroupmod,
diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -10,6 +10,7 @@
 hex,
 )
 from .i18n import _
+from .pycompat import getattr
 from .thirdparty import attr
 from . import (
 error,
diff --git a/mercurial/wireprotoframing.py b/mercurial/wireprotoframing.py
--- a/mercurial/wireprotoframing.py
+++ b/mercurial/wireprotoframing.py
@@ -15,6 +15,7 @@
 import struct
 
 from .i18n import _
+from .pycompat import getattr
 from .thirdparty import attr
 from . import (
 encoding,
diff --git a/mercurial/windows.py b/mercurial/windows.py
--- a/mercurial/windows.py
+++ b/mercurial/windows.py
@@ -16,6 +16,7 @@
 import sys
 
 from .i18n import _
+from .pycompat import getattr
 from . import (
 encoding,
 error,
diff --git a/mercurial/vfs.py b/mercurial/vfs.py
--- a/mercurial/vfs.py
+++ b/mercurial/vfs.py
@@ -14,7 +14,10 @@
 import threading
 
 from .i18n import _
-from .pycompat import setattr
+from .pycompat import (
+getattr,
+setattr,
+)
 from . import (
 encoding,
 error,
diff --git a/mercurial/utils/procutil.py b/mercurial/utils/procutil.py
--- a/mercurial/utils/procutil.py
+++ b/mercurial/utils/procutil.py
@@ -20,7 +20,10 @@
 import time
 
 from ..i18n import _
-from ..pycompat import open
+from ..pycompat import (
+getattr,
+open,
+)
 
 from .. import (
 encoding,
diff --git a/mercurial/utils/compression.py b/mercurial/utils/compression.py
--- a/mercurial/utils/compression.py
+++ b/mercurial/utils/compression.py
@@ -10,6 +10,7 @@
 import collections
 import zlib
 
+from ..pycompat import getattr
 from .. import (
 error,
 i18n,

D7005: py3: manually import pycompat.open into files that need it

[indygreg (Gregory Szorc)]

Closed by commit rHGeef9a2d67051: py3: manually import pycompat.open into files 
that need it (authored by indygreg).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7005?vs=16920=16928

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7005/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7005

AFFECTED FILES
  hgext/churn.py
  hgext/convert/common.py
  hgext/convert/convcmd.py
  hgext/convert/cvs.py
  hgext/convert/cvsps.py
  hgext/convert/hg.py
  hgext/convert/monotone.py
  hgext/convert/subversion.py
  hgext/fastannotate/context.py
  hgext/fastannotate/protocol.py
  hgext/fastannotate/revmap.py
  hgext/fsmonitor/__init__.py
  hgext/histedit.py
  hgext/infinitepush/__init__.py
  hgext/infinitepush/store.py
  hgext/largefiles/lfutil.py
  hgext/largefiles/localstore.py
  hgext/largefiles/overrides.py
  hgext/largefiles/proto.py
  hgext/mq.py
  hgext/patchbomb.py
  hgext/rebase.py
  hgext/releasenotes.py
  hgext/relink.py
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/remotefilelogserver.py
  hgext/remotefilelog/shallowutil.py
  hgext/transplant.py
  mercurial/__init__.py
  mercurial/archival.py
  mercurial/changegroup.py
  mercurial/cmdutil.py
  mercurial/commands.py
  mercurial/context.py
  mercurial/crecord.py
  mercurial/debugcommands.py
  mercurial/extensions.py
  mercurial/filemerge.py
  mercurial/hgweb/common.py
  mercurial/hgweb/server.py
  mercurial/httpconnection.py
  mercurial/mail.py
  mercurial/match.py
  mercurial/patch.py
  mercurial/posix.py
  mercurial/profiling.py
  mercurial/pycompat.py
  mercurial/server.py
  mercurial/shelve.py
  mercurial/statprof.py
  mercurial/streamclone.py
  mercurial/subrepo.py
  mercurial/ui.py
  mercurial/util.py
  mercurial/utils/procutil.py

CHANGE DETAILS

diff --git a/mercurial/utils/procutil.py b/mercurial/utils/procutil.py
--- a/mercurial/utils/procutil.py
+++ b/mercurial/utils/procutil.py
@@ -20,6 +20,7 @@
 import time
 
 from ..i18n import _
+from ..pycompat import open
 
 from .. import (
 encoding,
diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -35,6 +35,7 @@
 import warnings
 
 from .thirdparty import attr
+from .pycompat import open
 from hgdemandimport import tracing
 from . import (
 encoding,
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -22,6 +22,7 @@
 
 from .i18n import _
 from .node import hex
+from .pycompat import open
 
 from . import (
 color,
diff --git a/mercurial/subrepo.py b/mercurial/subrepo.py
--- a/mercurial/subrepo.py
+++ b/mercurial/subrepo.py
@@ -19,6 +19,7 @@
 import xml.dom.minidom
 
 from .i18n import _
+from .pycompat import open
 from . import (
 cmdutil,
 encoding,
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -12,6 +12,7 @@
 import struct
 
 from .i18n import _
+from .pycompat import open
 from .interfaces import repository
 from . import (
 cacheutil,
diff --git a/mercurial/statprof.py b/mercurial/statprof.py
--- a/mercurial/statprof.py
+++ b/mercurial/statprof.py
@@ -114,6 +114,7 @@
 import threading
 import time
 
+from .pycompat import open
 from . import (
 encoding,
 pycompat,
diff --git a/mercurial/shelve.py b/mercurial/shelve.py
--- a/mercurial/shelve.py
+++ b/mercurial/shelve.py
@@ -28,6 +28,7 @@
 import stat
 
 from .i18n import _
+from .pycompat import open
 from . import (
 bookmarks,
 bundle2,
diff --git a/mercurial/server.py b/mercurial/server.py
--- a/mercurial/server.py
+++ b/mercurial/server.py
@@ -10,6 +10,7 @@
 import os
 
 from .i18n import _
+from .pycompat import open
 
 from . import (
 chgserver,
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -354,6 +354,7 @@
 sysstr = identity
 strurl = identity
 bytesurl = identity
+open = open
 
 # this can't be parsed on Python 3
 exec(b'def raisewithtb(exc, tb):\n' b'raise exc, None, tb\n')
diff --git a/mercurial/profiling.py b/mercurial/profiling.py
--- a/mercurial/profiling.py
+++ b/mercurial/profiling.py
@@ -10,6 +10,7 @@
 import contextlib
 
 from .i18n import _
+from .pycompat import open
 from . import (
 encoding,
 error,
diff --git a/mercurial/posix.py b/mercurial/posix.py
--- a/mercurial/posix.py
+++ b/mercurial/posix.py
@@ -21,6 +21,7 @@
 import unicodedata
 
 from .i18n import _
+from .pycompat import open
 from . import (
 encoding,
 error,
diff --git a/mercurial/patch.py b/mercurial/patch.py
--- a/mercurial/patch.py
+++ b/mercurial/patch.py
@@ -24,6 +24,7 @@
 hex,
 short,
 

D7010: py3: manually import pycompat.delattr where it is needed

[indygreg (Gregory Szorc)]

Closed by commit rHG1f339b503a40: py3: manually import pycompat.delattr where 
it is needed (authored by indygreg).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7010?vs=16925=16933

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7010/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7010

AFFECTED FILES
  hgext/mq.py
  mercurial/__init__.py
  mercurial/dirstate.py
  mercurial/localrepo.py
  mercurial/merge.py
  mercurial/pycompat.py
  mercurial/repoview.py
  mercurial/util.py
  mercurial/vfs.py

CHANGE DETAILS

diff --git a/mercurial/vfs.py b/mercurial/vfs.py
--- a/mercurial/vfs.py
+++ b/mercurial/vfs.py
@@ -15,6 +15,7 @@
 
 from .i18n import _
 from .pycompat import (
+delattr,
 getattr,
 setattr,
 )
diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -36,6 +36,7 @@
 
 from .thirdparty import attr
 from .pycompat import (
+delattr,
 getattr,
 open,
 setattr,
diff --git a/mercurial/repoview.py b/mercurial/repoview.py
--- a/mercurial/repoview.py
+++ b/mercurial/repoview.py
@@ -13,6 +13,7 @@
 
 from .node import nullrev
 from .pycompat import (
+delattr,
 getattr,
 setattr,
 )
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -355,6 +355,7 @@
 strurl = identity
 bytesurl = identity
 open = open
+delattr = delattr
 getattr = getattr
 hasattr = hasattr
 setattr = setattr
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -23,6 +23,7 @@
 nullid,
 nullrev,
 )
+from .pycompat import delattr
 from .thirdparty import attr
 from . import (
 copies,
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -23,7 +23,10 @@
 nullrev,
 short,
 )
-from .pycompat import getattr
+from .pycompat import (
+delattr,
+getattr,
+)
 from . import (
 bookmarks,
 branchmap,
diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -15,6 +15,7 @@
 
 from .i18n import _
 from .node import nullid
+from .pycompat import delattr
 from . import (
 encoding,
 error,
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -101,8 +101,6 @@
 REMEMBER TO CHANGE ``BYTECODEHEADER`` WHEN CHANGING THIS FUNCTION
 OR CACHED FILES WON'T GET INVALIDATED PROPERLY.
 """
-futureimpline = False
-
 # The following utility functions access the tokens list and i index of
 # the for i, t enumerate(tokens) loop below
 def _isop(j, *o):
@@ -153,34 +151,6 @@
 tokens[j] = st._replace(string='u%s' % st.string)
 
 for i, t in enumerate(tokens):
-# Insert compatibility imports at "from __future__ import" line.
-# No '\n' should be added to preserve line numbers.
-if (
-t.type == token.NAME
-and t.string == 'import'
-and all(u.type == token.NAME for u in tokens[i - 2 : i])
-and [u.string for u in tokens[i - 2 : i]]
-== ['from', '__future__']
-):
-futureimpline = True
-if t.type == token.NEWLINE and futureimpline:
-futureimpline = False
-if fullname == 'mercurial.pycompat':
-yield t
-continue
-r, c = t.start
-l = (
-b'; from mercurial.pycompat import '
-b'delattr\n'
-)
-for u in tokenize.tokenize(io.BytesIO(l).readline):
-if u.type in (tokenize.ENCODING, token.ENDMARKER):
-continue
-yield u._replace(
-start=(r, c + u.start[1]), end=(r, c + u.end[1])
-)
-continue
-
 # This looks like a function call.
 if t.type == token.NAME and _isop(i + 1, '('):
 fn = t.string
@@ -220,7 +190,7 @@
 # ``replacetoken`` or any mechanism that changes semantics of module
 # loading is changed. Otherwise cached bytecode may get loaded without
 # the new transformation mechanisms applied.
-BYTECODEHEADER = b'HG\x00\x11'
+BYTECODEHEADER = b'HG\x00\x12'
 
 class hgloader(importlib.machinery.SourceFileLoader):
 """Custom module loader that transforms source code.
diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -77,6 +77,7 @@
 short,
 )
 from mercurial.pycompat import (
+delattr,
  

D7011: py3: stop normalizing .encode()/.decode() arguments to unicode

[indygreg (Gregory Szorc)]

Closed by commit rHG127cc1f72e70: py3: stop normalizing .encode()/.decode() 
arguments to unicode (authored by indygreg).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7011?vs=16926=16934

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7011/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7011

AFFECTED FILES
  contrib/testparseutil.py
  hgext/lfs/blobstore.py
  mercurial/__init__.py
  mercurial/pycompat.py

CHANGE DETAILS

diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -206,7 +206,7 @@
 ) and not hasattr(  # hasattr-py3-only
 s, u'__bytes__'
 ):
-s = str(s).encode(u'ascii')
+s = str(s).encode('ascii')
 return bytes.__new__(cls, s)
 
 def __getitem__(self, key):
@@ -237,7 +237,7 @@
 This never raises UnicodeEncodeError, but only ASCII characters
 can be round-trip by sysstr(sysbytes(s)).
 """
-return s.encode(u'utf-8')
+return s.encode('utf-8')
 
 def sysstr(s):
 """Return a keyword str to be passed to Python functions such as
@@ -249,18 +249,18 @@
 """
 if isinstance(s, builtins.str):
 return s
-return s.decode(u'latin-1')
+return s.decode('latin-1')
 
 def strurl(url):
 """Converts a bytes url back to str"""
 if isinstance(url, bytes):
-return url.decode(u'ascii')
+return url.decode('ascii')
 return url
 
 def bytesurl(url):
 """Converts a str url to bytes by encoding in ascii"""
 if isinstance(url, str):
-return url.encode(u'ascii')
+return url.encode('ascii')
 return url
 
 def raisewithtb(exc, tb):
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -166,14 +166,6 @@
 if arg1idx is not None:
 _ensureunicode(arg1idx)
 
-# .encode() and .decode() on str/bytes/unicode don't accept
-# byte strings on Python 3.
-elif fn in ('encode', 'decode') and _isop(i - 1, '.'):
-for argn in range(2):
-argidx = _findargnofcall(argn)
-if argidx is not None:
-_ensureunicode(argidx)
-
 # It changes iteritems/values to items/values as they are not
 # present in Python 3 world.
 elif fn in ('iteritems', 'itervalues') and not (
@@ -190,7 +182,7 @@
 # ``replacetoken`` or any mechanism that changes semantics of module
 # loading is changed. Otherwise cached bytecode may get loaded without
 # the new transformation mechanisms applied.
-BYTECODEHEADER = b'HG\x00\x12'
+BYTECODEHEADER = b'HG\x00\x13'
 
 class hgloader(importlib.machinery.SourceFileLoader):
 """Custom module loader that transforms source code.
diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
--- a/hgext/lfs/blobstore.py
+++ b/hgext/lfs/blobstore.py
@@ -394,7 +394,7 @@
 
 def encodestr(x):
 if isinstance(x, pycompat.unicode):
-return x.encode(u'utf-8')
+return x.encode('utf-8')
 return x
 
 return pycompat.rapply(encodestr, response)
diff --git a/contrib/testparseutil.py b/contrib/testparseutil.py
--- a/contrib/testparseutil.py
+++ b/contrib/testparseutil.py
@@ -49,7 +49,7 @@
 def sysstr(s):
 if isinstance(s, builtins.str):
 return s
-return s.decode(u'latin-1')
+return s.decode('latin-1')
 
 def opentext(f):
 return open(f, 'r')



To: indygreg, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7007: py3: manually import pycompat.setattr where it is needed

[indygreg (Gregory Szorc)]

Closed by commit rHG66f2cc210a29: py3: manually import pycompat.setattr where 
it is needed (authored by indygreg).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7007?vs=16922=16930

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7007/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7007

AFFECTED FILES
  hgext/factotum.py
  hgext/fastannotate/context.py
  hgext/fix.py
  hgext/lfs/wrapper.py
  hgext/sparse.py
  hgext/win32mbcs.py
  mercurial/__init__.py
  mercurial/chgserver.py
  mercurial/cmdutil.py
  mercurial/extensions.py
  mercurial/hgweb/webutil.py
  mercurial/mdiff.py
  mercurial/phases.py
  mercurial/pycompat.py
  mercurial/repoview.py
  mercurial/ui.py
  mercurial/util.py
  mercurial/vfs.py
  mercurial/wireprotov1peer.py

CHANGE DETAILS

diff --git a/mercurial/wireprotov1peer.py b/mercurial/wireprotov1peer.py
--- a/mercurial/wireprotov1peer.py
+++ b/mercurial/wireprotov1peer.py
@@ -13,6 +13,7 @@
 
 from .i18n import _
 from .node import bin
+from .pycompat import setattr
 from . import (
 bundle2,
 changegroup as changegroupmod,
diff --git a/mercurial/vfs.py b/mercurial/vfs.py
--- a/mercurial/vfs.py
+++ b/mercurial/vfs.py
@@ -14,6 +14,7 @@
 import threading
 
 from .i18n import _
+from .pycompat import setattr
 from . import (
 encoding,
 error,
diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -35,7 +35,10 @@
 import warnings
 
 from .thirdparty import attr
-from .pycompat import open
+from .pycompat import (
+open,
+setattr,
+)
 from hgdemandimport import tracing
 from . import (
 encoding,
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -22,7 +22,10 @@
 
 from .i18n import _
 from .node import hex
-from .pycompat import open
+from .pycompat import (
+open,
+setattr,
+)
 
 from . import (
 color,
diff --git a/mercurial/repoview.py b/mercurial/repoview.py
--- a/mercurial/repoview.py
+++ b/mercurial/repoview.py
@@ -12,6 +12,7 @@
 import weakref
 
 from .node import nullrev
+from .pycompat import setattr
 from . import (
 obsolete,
 phases,
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -355,6 +355,7 @@
 strurl = identity
 bytesurl = identity
 open = open
+setattr = setattr
 
 # this can't be parsed on Python 3
 exec(b'def raisewithtb(exc, tb):\n' b'raise exc, None, tb\n')
diff --git a/mercurial/phases.py b/mercurial/phases.py
--- a/mercurial/phases.py
+++ b/mercurial/phases.py
@@ -113,6 +113,7 @@
 nullrev,
 short,
 )
+from .pycompat import setattr
 from . import (
 error,
 pycompat,
diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py
--- a/mercurial/mdiff.py
+++ b/mercurial/mdiff.py
@@ -12,6 +12,7 @@
 import zlib
 
 from .i18n import _
+from .pycompat import setattr
 from . import (
 encoding,
 error,
diff --git a/mercurial/hgweb/webutil.py b/mercurial/hgweb/webutil.py
--- a/mercurial/hgweb/webutil.py
+++ b/mercurial/hgweb/webutil.py
@@ -15,6 +15,7 @@
 
 from ..i18n import _
 from ..node import hex, nullid, short
+from ..pycompat import setattr
 
 from .common import (
 ErrorResponse,
diff --git a/mercurial/extensions.py b/mercurial/extensions.py
--- a/mercurial/extensions.py
+++ b/mercurial/extensions.py
@@ -18,7 +18,10 @@
 _,
 gettext,
 )
-from .pycompat import open
+from .pycompat import (
+open,
+setattr,
+)
 
 from . import (
 cmdutil,
diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -19,7 +19,10 @@
 nullrev,
 short,
 )
-from .pycompat import open
+from .pycompat import (
+open,
+setattr,
+)
 
 from . import (
 bookmarks,
diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py
--- a/mercurial/chgserver.py
+++ b/mercurial/chgserver.py
@@ -51,6 +51,7 @@
 import time
 
 from .i18n import _
+from .pycompat import setattr
 
 from . import (
 commandserver,
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -171,7 +171,7 @@
 r, c = t.start
 l = (
 b'; from mercurial.pycompat import '
-b'delattr, getattr, hasattr, setattr\n'
+b'delattr, getattr, hasattr\n'
 )
 for u in tokenize.tokenize(io.BytesIO(l).readline):
 if u.type in (tokenize.ENCODING, token.ENDMARKER):
@@ -220,7 +220,7 @@
 # ``replacetoken`` or any mechanism that changes semantics of module
 # loading is changed. Otherwise cached bytecode may get loaded without
 # the new transformation mechanisms applied.
-

D7008: py3: stop injecting pycompat.hasattr into modules

[indygreg (Gregory Szorc)]

Closed by commit rHG0d612db7047c: py3: stop injecting pycompat.hasattr into 
modules (authored by indygreg).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7008?vs=16923=16931

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7008/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7008

AFFECTED FILES
  mercurial/__init__.py
  mercurial/pycompat.py
  mercurial/statprof.py

CHANGE DETAILS

diff --git a/mercurial/statprof.py b/mercurial/statprof.py
--- a/mercurial/statprof.py
+++ b/mercurial/statprof.py
@@ -168,7 +168,7 @@
 # a float
 if frequency:
 self.sample_interval = 1.0 / frequency
-elif not hasattr(self, 'sample_interval'):
+elif not pycompat.hasattr(self, 'sample_interval'):
 # default to 1000 Hz
 self.sample_interval = 1.0 / 1000.0
 else:
diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -355,6 +355,7 @@
 strurl = identity
 bytesurl = identity
 open = open
+hasattr = hasattr
 setattr = setattr
 
 # this can't be parsed on Python 3
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -171,7 +171,7 @@
 r, c = t.start
 l = (
 b'; from mercurial.pycompat import '
-b'delattr, getattr, hasattr\n'
+b'delattr, getattr\n'
 )
 for u in tokenize.tokenize(io.BytesIO(l).readline):
 if u.type in (tokenize.ENCODING, token.ENDMARKER):
@@ -220,7 +220,7 @@
 # ``replacetoken`` or any mechanism that changes semantics of module
 # loading is changed. Otherwise cached bytecode may get loaded without
 # the new transformation mechanisms applied.
-BYTECODEHEADER = b'HG\x00\x0f'
+BYTECODEHEADER = b'HG\x00\x10'
 
 class hgloader(importlib.machinery.SourceFileLoader):
 """Custom module loader that transforms source code.



To: indygreg, #hg-reviewers
Cc: durin42, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7006: py3: stop implicitly importing unicode

[indygreg (Gregory Szorc)]

Closed by commit rHGbbcbb82e3589: py3: stop implicitly importing unicode 
(authored by indygreg).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7006?vs=16921=16929

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7006/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7006

AFFECTED FILES
  mercurial/__init__.py
  mercurial/templatefilters.py

CHANGE DETAILS

diff --git a/mercurial/templatefilters.py b/mercurial/templatefilters.py
--- a/mercurial/templatefilters.py
+++ b/mercurial/templatefilters.py
@@ -367,7 +367,7 @@
 """Any text. Returns the input text rendered as a sequence of
 XML entities.
 """
-text = unicode(text, pycompat.sysstr(encoding.encoding), r'replace')
+text = pycompat.unicode(text, pycompat.sysstr(encoding.encoding), 
r'replace')
 return 

D7005: py3: manually import pycompat.open into files that need it

[durin42 (Augie Fackler)]

durin42 added a comment.


  In D7005#102780 , @martinvonz 
wrote:
  
  > Can we import `io.open` regardless of Python version instead?
  
  That won't work because pycompat.open accepts bytestrings as the mode 
argument. I'll take this now, and then we can do a cleanup to unwind the mode 
string back to a sysstr everywhere (I'll take that on if Pulkit doesn't).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7005/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7005

To: indygreg, durin42, martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel