mercurial@45710: 33 new changesets

2020-10-14 Thread Mercurial Commits 
33 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/bd2df58366b1
changeset:   45678:bd2df58366b1
user:Martin von Zweigbergk 
date:Tue Oct 06 20:37:35 2020 -0700
summary: errors: name arguments to CommandError constructor

https://www.mercurial-scm.org/repo/hg/rev/65e2b64670b5
changeset:   45679:65e2b64670b5
user:Martin von Zweigbergk 
date:Tue Oct 06 20:45:52 2020 -0700
summary: errors: name arguments to AmbiguousCommand constructor

https://www.mercurial-scm.org/repo/hg/rev/bb1a988ef4a5
changeset:   45680:bb1a988ef4a5
user:Martin von Zweigbergk 
date:Tue Oct 06 21:06:18 2020 -0700
summary: errors: name arguments to UnknownCommand constructor

https://www.mercurial-scm.org/repo/hg/rev/a736ab681b78
changeset:   45681:a736ab681b78
user:Martin von Zweigbergk 
date:Thu Oct 08 15:35:44 2020 -0700
summary: errors: stop passing non-strings to Abort's constructor

https://www.mercurial-scm.org/repo/hg/rev/d2e1dcd4490d
changeset:   45682:d2e1dcd4490d
user:Martin von Zweigbergk 
date:Thu Oct 08 13:37:31 2020 -0700
summary: errors: name arguments to Abort constructor

https://www.mercurial-scm.org/repo/hg/rev/04aa48afab99
changeset:   45683:04aa48afab99
user:Martin von Zweigbergk 
date:Thu Oct 08 13:10:16 2020 -0700
summary: tests: use `git init` instead of unusual synonym `git init-db`

https://www.mercurial-scm.org/repo/hg/rev/0c18493287f5
changeset:   45684:0c18493287f5
user:Martin von Zweigbergk 
date:Thu Oct 08 13:19:18 2020 -0700
summary: tests: set git config using `git config` for simplicity

https://www.mercurial-scm.org/repo/hg/rev/57b5452a55d5
changeset:   45685:57b5452a55d5
user:Gregory Szorc 
date:Sun Oct 04 22:32:41 2020 -0700
summary: pyoxidizer: produce working Python 3 Windows installers (issue6366)

https://www.mercurial-scm.org/repo/hg/rev/17a12f53dd72
changeset:   45686:17a12f53dd72
user:Pierre-Yves David 
date:Thu Oct 08 16:14:06 2020 +0200
summary: revset: add a `node` key for sorting

https://www.mercurial-scm.org/repo/hg/rev/223296268c4e
changeset:   45687:223296268c4e
user:Martin von Zweigbergk 
date:Fri Oct 09 08:08:54 2020 -0700
summary: tests: fix test-url.py on py3, broken by D9179

https://www.mercurial-scm.org/repo/hg/rev/ed0ded64a8a9
changeset:   45688:ed0ded64a8a9
user:Martin von Zweigbergk 
date:Fri Oct 09 09:17:47 2020 -0700
summary: py3: convert an exception message to bytes

https://www.mercurial-scm.org/repo/hg/rev/ef8eccefe0c3
changeset:   45689:ef8eccefe0c3
user:Gregory Szorc 
date:Thu Oct 08 18:02:47 2020 -0700
summary: contrib: stop installing Python 3.5 and 3.6 in Windows environment

https://www.mercurial-scm.org/repo/hg/rev/7a907388a4a5
changeset:   45690:7a907388a4a5
user:Gregory Szorc 
date:Thu Oct 08 18:07:34 2020 -0700
summary: contrib: install Python 3.9.0

https://www.mercurial-scm.org/repo/hg/rev/eceebe3f9baf
changeset:   45691:eceebe3f9baf
user:Gregory Szorc 
date:Thu Oct 08 18:17:20 2020 -0700
summary: packaging: upgrade packages in Windows environment

https://www.mercurial-scm.org/repo/hg/rev/9934920af5f7
changeset:   45692:9934920af5f7
user:Gregory Szorc 
date:Fri Oct 09 09:22:59 2020 -0700
summary: automation: upgrade packages in Linux environment

https://www.mercurial-scm.org/repo/hg/rev/64a9423450ef
changeset:   45693:64a9423450ef
user:Gregory Szorc 
date:Fri Oct 09 09:46:03 2020 -0700
summary: automation: support running against Python 3.9

https://www.mercurial-scm.org/repo/hg/rev/d1c10d33a85c
changeset:   45694:d1c10d33a85c
user:Pulkit Goyal <7895pul...@gmail.com>
date:Thu Oct 08 17:29:51 2020 +0530
summary: upgrade: improve documentation of matchrevlog()

https://www.mercurial-scm.org/repo/hg/rev/760bb4d74aad
changeset:   45695:760bb4d74aad
user:Yuya Nishihara 
date:Wed Sep 09 15:17:26 2020 +0900
summary: grep: explicitly pass regexp to closure functions

https://www.mercurial-scm.org/repo/hg/rev/de6f2afc0247
changeset:   45696:de6f2afc0247
user:Yuya Nishihara 
date:Wed Sep 09 15:23:49 2020 +0900
summary: grep: move match and diff logic to new module

https://www.mercurial-scm.org/repo/hg/rev/494642ed3c50
changeset:   45697:494642ed3c50
user:Yuya Nishihara 
date:Wed Sep 09 15:56:40 2020 +0900
summary: grep: add stub class that maintains cache and states of grep 
operation

https://www.mercurial-scm.org/repo/hg/rev/41e0cbccb260
changeset:   45698:41e0cbccb260
user:Yuya Nishihara 
date:Wed Sep 09 16:00:03 2020 +0900
summary: grep: move getbody() to grepsearcher class

https://www.mercurial-scm.org/repo/hg/rev/888e633f0c1c
changeset:   45699:888e633f0c1c
user:Yuya Nishihara 
date:Wed Sep 09 16:04:39 2020

D9208: clonebundles: move a bundle of clone bundle related code to a new module

2020-10-14 Thread marmoute (Pierre-Yves David) 
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  In the process on general clone bundle automatically, we need to make some
  function available more widely. This is a good opportunity to extract a
  significant amount of code from `mercurial.exchange` into a new
  `mercurial.bundlecaches`. This make `mercurial.exchange` move under the 3K 
line
  range (hooray…).
  
  The module is called `bundlecaches` because I expect it to be eventually 
useful
  for more than just clone bundle (like pull bunbles).

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9208

AFFECTED FILES
  hgext/lfs/__init__.py
  mercurial/bundlecaches.py
  mercurial/commands.py
  mercurial/exchange.py
  mercurial/wireprotov1server.py
  tests/flagprocessorext.py
  tests/test-pull-bundle.t

CHANGE DETAILS

diff --git a/tests/test-pull-bundle.t b/tests/test-pull-bundle.t
--- a/tests/test-pull-bundle.t
+++ b/tests/test-pull-bundle.t
@@ -52,7 +52,7 @@
   > 1.hg BUNDLESPEC=bzip2-v2 heads=ed1b79f46b9a29f5a6efa59cf12fcfca43bead5a 
bases=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
   > 0.hg BUNDLESPEC=gzip-v2 heads=bbd179dfa0a71671c253b3ae0aa1513b60d199fa
   > EOF
-  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid
+  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid -E ../error.txt
   listening at http://*:$HGPORT2/ (bound to $LOCALIP:$HGPORT2) (glob) (?)
   $ cat ../repo.pid >> $DAEMON_PIDS
   $ cd ..
@@ -64,6 +64,7 @@
   new changesets bbd179dfa0a7 (1 drafts)
   updating to branch default
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cat error.txt
   $ cd repo.pullbundle
   $ hg pull -r 1
   pulling from http://localhost:$HGPORT2/
diff --git a/tests/flagprocessorext.py b/tests/flagprocessorext.py
--- a/tests/flagprocessorext.py
+++ b/tests/flagprocessorext.py
@@ -6,8 +6,8 @@
 import zlib
 
 from mercurial import (
+bundlecaches,
 changegroup,
-exchange,
 extensions,
 revlog,
 util,
@@ -134,8 +134,8 @@
 revlog.REVIDX_FLAGS_ORDER.extend(flags)
 
 # Teach exchange to use changegroup 3
-for k in exchange._bundlespeccontentopts.keys():
-exchange._bundlespeccontentopts[k][b"cg.version"] = b"03"
+for k in bundlecaches._bundlespeccontentopts.keys():
+bundlecaches._bundlespeccontentopts[k][b"cg.version"] = b"03"
 
 # Register flag processors for each extension
 flagutil.addflagprocessor(
diff --git a/mercurial/wireprotov1server.py b/mercurial/wireprotov1server.py
--- a/mercurial/wireprotov1server.py
+++ b/mercurial/wireprotov1server.py
@@ -19,6 +19,7 @@
 
 from . import (
 bundle2,
+bundlecaches,
 changegroup as changegroupmod,
 discovery,
 encoding,
@@ -387,8 +388,8 @@
 manifest = repo.vfs.tryread(b'pullbundles.manifest')
 if not manifest:
 return None
-res = exchange.parseclonebundlesmanifest(repo, manifest)
-res = exchange.filterclonebundleentries(repo, res)
+res = bundlecaches.parseclonebundlesmanifest(repo, manifest)
+res = bundlecaches.filterclonebundleentries(repo, res)
 if not res:
 return None
 cl = repo.unfiltered().changelog
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -16,10 +16,10 @@
 nullid,
 nullrev,
 )
-from .thirdparty import attr
 from . import (
 bookmarks as bookmod,
 bundle2,
+bundlecaches,
 changegroup,
 discovery,
 error,
@@ -34,7 +34,6 @@
 pycompat,
 requirements,
 scmutil,
-sslutil,
 streamclone,
 url as urlmod,
 util,
@@ -50,202 +49,6 @@
 
 _NARROWACL_SECTION = b'narrowacl'
 
-# Maps bundle version human names to changegroup versions.
-_bundlespeccgversions = {
-b'v1': b'01',
-b'v2': b'02',
-b'packed1': b's1',
-b'bundle2': b'02',  # legacy
-}
-
-# Maps bundle version with content opts to choose which part to bundle
-_bundlespeccontentopts = {
-b'v1': {
-b'changegroup': True,
-b'cg.version': b'01',
-b'obsolescence': False,
-b'phases': False,
-b'tagsfnodescache': False,
-b'revbranchcache': False,
-},
-b'v2': {
-b'changegroup': True,
-b'cg.version': b'02',
-b'obsolescence': False,
-b'phases': False,
-b'tagsfnodescache': True,
-b'revbranchcache': True,
-},
-b'packed1': {b'cg.version': b's1'},
-}
-_bundlespeccontentopts[b'bundle2'] = _bundlespeccontentopts[b'v2']
-
-_bundlespecvariants = {
-b"streamv2": {
-b"changegroup": False,
-b"streamv2": True,
-b"tagsfnodescache": False,
-b"revbranchcache": False,
-}
-}
-
-# Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
-_bundlespecv1compengines = {b'gzip', b'bzip2', b'none'}
-
-
-@attr.s
-class

D9209: clonebundle: move the manifest filename to a constant

2020-10-14 Thread marmoute (Pierre-Yves David) 
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  I am about to add more reference to it, so I would rather have it an explicit
  constant. This allow to unify various call too.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9209

AFFECTED FILES
  hgext/clonebundles.py
  mercurial/bundlecaches.py
  mercurial/localrepo.py
  mercurial/wireprotov1server.py

CHANGE DETAILS

diff --git a/mercurial/wireprotov1server.py b/mercurial/wireprotov1server.py
--- a/mercurial/wireprotov1server.py
+++ b/mercurial/wireprotov1server.py
@@ -273,7 +273,7 @@
 data center given the client's IP address.
 """
 return wireprototypes.bytesresponse(
-repo.vfs.tryread(b'clonebundles.manifest')
+repo.vfs.tryread(bundlecaches.CB_MANIFEST_FILE)
 )
 
 
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -31,6 +31,7 @@
 bookmarks,
 branchmap,
 bundle2,
+bundlecaches,
 changegroup,
 color,
 commit,
@@ -299,7 +300,7 @@
 return self._caps
 
 def clonebundles(self):
-return self._repo.tryread(b'clonebundles.manifest')
+return self._repo.tryread(bundlecaches.CB_MANIFEST_FILE)
 
 def debugwireargs(self, one, two, three=None, four=None, five=None):
 """Used to test argument passing over the wire"""
diff --git a/mercurial/bundlecaches.py b/mercurial/bundlecaches.py
--- a/mercurial/bundlecaches.py
+++ b/mercurial/bundlecaches.py
@@ -16,6 +16,8 @@
 
 urlreq = util.urlreq
 
+CB_MANIFEST_FILE = b'clonebundles.manifest'
+
 
 @attr.s
 class bundlespec(object):
diff --git a/hgext/clonebundles.py b/hgext/clonebundles.py
--- a/hgext/clonebundles.py
+++ b/hgext/clonebundles.py
@@ -205,6 +205,7 @@
 from __future__ import absolute_import
 
 from mercurial import (
+bundlecaches,
 extensions,
 wireprotov1server,
 )
@@ -218,7 +219,7 @@
 # Only advertise if a manifest exists. This does add some I/O to requests.
 # But this should be cheaper than a wasted network round trip due to
 # missing file.
-if repo.vfs.exists(b'clonebundles.manifest'):
+if repo.vfs.exists(bundlecaches.CB_MANIFEST_FILE):
 caps.append(b'clonebundles')
 
 return caps



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9212: procutil: allow to specify arbitrary stdin bytes to runbgcommand

2020-10-14 Thread marmoute (Pierre-Yves David) 
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  For automatic clonebundles generation I need to pass arbitrary large amount of
  data to the process (eg: common nodes, target nodes).
  
  I am updating the `runbgcommand` to allow for this. Previously not stdin input
  was possible, now, one can provide raw bytes and they will be feed to the
  command through an unnamed temporary files.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9212

AFFECTED FILES
  mercurial/utils/procutil.py

CHANGE DETAILS

diff --git a/mercurial/utils/procutil.py b/mercurial/utils/procutil.py
--- a/mercurial/utils/procutil.py
+++ b/mercurial/utils/procutil.py
@@ -635,21 +635,35 @@
 stderr=None,
 ensurestart=True,
 record_wait=None,
+stdin_bytes=None,
 ):
 '''Spawn a command without waiting for it to finish.'''
 # we can't use close_fds *and* redirect stdin. I'm not sure that we
 # need to because the detached process has no console connection.
-p = subprocess.Popen(
-tonativestr(script),
-shell=shell,
-env=tonativeenv(env),
-close_fds=True,
-creationflags=_creationflags,
-stdout=stdout,
-stderr=stderr,
-)
-if record_wait is not None:
-record_wait(p.wait)
+
+try:
+stdin = None
+if stdin_bytes is not None:
+stdin = pycompat.unnamedtempfile()
+stdin.write(stdin_bytes)
+stdin.flush()
+stdin.seek(0)
+
+p = subprocess.Popen(
+tonativestr(script),
+shell=shell,
+env=tonativeenv(env),
+close_fds=True,
+creationflags=_creationflags,
+stdin=stdin,
+stdout=stdout,
+stderr=stderr,
+)
+if record_wait is not None:
+record_wait(p.wait)
+finally:
+if stdin is not None:
+stdin.close()
 
 
 else:
@@ -662,6 +676,7 @@
 stderr=None,
 ensurestart=True,
 record_wait=None,
+stdin_bytes=None,
 ):
 '''Spawn a command without waiting for it to finish.
 
@@ -722,15 +737,21 @@
 if record_wait is None:
 # Start a new session
 os.setsid()
+# connect stdin to devnull to make sure the subprocess can't
+# muck up that stream for mercurial.
+if stdin_bytes is None:
+stdin = open(os.devnull, b'r')
+else:
+stdin = pycompat.unnamedtempfile()
+stdin.write(stdin_bytes)
+stdin.flush()
+stdin.seek(0)
 
-stdin = open(os.devnull, b'r')
 if stdout is None:
 stdout = open(os.devnull, b'w')
 if stderr is None:
 stderr = open(os.devnull, b'w')
 
-# connect stdin to devnull to make sure the subprocess can't
-# muck up that stream for mercurial.
 p = subprocess.Popen(
 cmd,
 shell=shell,
@@ -754,5 +775,6 @@
 finally:
 # mission accomplished, this child needs to exit and not
 # continue the hg process here.
+stdin.close()
 if record_wait is None:
 os._exit(returncode)



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9211: pycompat: add an entry for unnamedtmpfile

2020-10-14 Thread marmoute (Pierre-Yves David) 
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  I am going to use unnamed temporary files to pass arbitrarily large input data
  to worker creating bundles. To do so, I need a unified API that work on py2 
and
  py3.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9211

AFFECTED FILES
  mercurial/pycompat.py

CHANGE DETAILS

diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -506,6 +506,17 @@
 return tempfile.mkstemp(suffix, prefix, dir)
 
 
+# TemporaryFile does not support an "encoding=" argument on python2.
+# This wrapper file are always open in byte mode.
+def unnamedtempfile(mode=None, *args, **kwargs):
+if mode is None:
+mode = b'w+b'
+else:
+mode = sysstr(mode)
+assert 'b' in mode
+return tempfile.TemporaryFile(mode, *args, **kwargs)
+
+
 # NamedTemporaryFile does not support an "encoding=" argument on python2.
 # This wrapper file are always open in byte mode.
 def namedtempfile(



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9210: pycompat: update comment about unnamedtempfile

2020-10-14 Thread marmoute (Pierre-Yves David) 
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  I found the comment clearer. I end up having to think about this for
  `TemporaryFile` and I update that one in the process.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9210

AFFECTED FILES
  mercurial/pycompat.py

CHANGE DETAILS

diff --git a/mercurial/pycompat.py b/mercurial/pycompat.py
--- a/mercurial/pycompat.py
+++ b/mercurial/pycompat.py
@@ -506,7 +506,8 @@
 return tempfile.mkstemp(suffix, prefix, dir)
 
 
-# mode must include 'b'ytes as encoding= is not supported
+# NamedTemporaryFile does not support an "encoding=" argument on python2.
+# This wrapper file are always open in byte mode.
 def namedtempfile(
 mode=b'w+b', bufsize=-1, suffix=b'', prefix=b'tmp', dir=None, delete=True
 ):



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9207: record: when backing up, avoid generating very long filenames

2020-10-14 Thread spectral (Kyle Lippincott) 
spectral created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  If the original file's path is longer than the individual filename maximum
  length (256 on Linux, I believe?), then this mechanism of "replace slashes 
with
  underscores" causes an error.
  
  Now, we'll produce just the "basename" of the file, plus some stuff to ensure
  it's unique. This can be potentially confusing for users if there's a file 
with
  the same name in multiple directories, but I suspect that this is better than
  just breaking.
  
  Example:
  `/a/long/path/to/somefile.txt` used to be backed up as
  `/.hg/record-backups/a_long_path_to_somefile.txt.abcdefgh`, it will
  now be backed up as `/.hg/record-backups/somefile.txt.abcdefgh`
  
  We could do the naive thing (what we were doing before) and have it to doing
  something with either subdirectories
  (`/a/long/path/to/somefile.txt.abcdefgh` or minimize #dirs with
  `/a_long_path/to_somefile.txt.abcdefgh`), prefix-truncated paths
  (such as `/__ath_to_somefile.txt.abcdefgh`, where that `__` elides
  enough to get us under 255 chars (counting the +9 we need to add!)), or
  hash-of-dirname (`//somefile.txt.abcdefgh`), 
but
  ultimately every option felt over engineered and that it would be more likely 
to
  cause problems than it would be to solve any, especially if it was conditional
  on directory length.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9207

AFFECTED FILES
  mercurial/cmdutil.py

CHANGE DETAILS

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -558,7 +558,7 @@
 # backup continues
 for f in tobackup:
 fd, tmpname = pycompat.mkstemp(
-prefix=f.replace(b'/', b'_') + b'.', dir=backupdir
+prefix=os.path.basename(f) + b'.', dir=backupdir
 )
 os.close(fd)
 ui.debug(b'backup %r as %r\n' % (f, tmpname))



To: spectral, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9206: posix: avoid a leaked file descriptor in a unix domain socket exception case

2020-10-14 Thread mharbison72 (Matt Harbison) 
mharbison72 created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9206

AFFECTED FILES
  mercurial/posix.py

CHANGE DETAILS

diff --git a/mercurial/posix.py b/mercurial/posix.py
--- a/mercurial/posix.py
+++ b/mercurial/posix.py
@@ -764,10 +764,14 @@
 # platforms (see sys/un.h)
 dirname, basename = os.path.split(path)
 bakwdfd = None
-if dirname:
-bakwdfd = os.open(b'.', os.O_DIRECTORY)
-os.chdir(dirname)
-sock.bind(basename)
-if bakwdfd:
-os.fchdir(bakwdfd)
-os.close(bakwdfd)
+
+try:
+if dirname:
+bakwdfd = os.open(b'.', os.O_DIRECTORY)
+os.chdir(dirname)
+sock.bind(basename)
+if bakwdfd:
+os.fchdir(bakwdfd)
+finally:
+if bakwdfd:
+os.close(bakwdfd)



To: mharbison72, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9205: posix: use context managers in a couple of places

2020-10-14 Thread mharbison72 (Matt Harbison) 
mharbison72 created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9205

AFFECTED FILES
  mercurial/posix.py

CHANGE DETAILS

diff --git a/mercurial/posix.py b/mercurial/posix.py
--- a/mercurial/posix.py
+++ b/mercurial/posix.py
@@ -144,26 +144,24 @@
 if l:
 if not stat.S_ISLNK(s):
 # switch file to link
-fp = open(f, b'rb')
-data = fp.read()
-fp.close()
+with open(f, b'rb') as fp:
+data = fp.read()
 unlink(f)
 try:
 os.symlink(data, f)
 except OSError:
 # failed to make a link, rewrite file
-fp = open(f, b"wb")
-fp.write(data)
-fp.close()
+with open(f, b"wb") as fp:
+fp.write(data)
+
 # no chmod needed at this point
 return
 if stat.S_ISLNK(s):
 # switch link to file
 data = os.readlink(f)
 unlink(f)
-fp = open(f, b"wb")
-fp.write(data)
-fp.close()
+with open(f, b"wb") as fp:
+fp.write(data)
 s = 0o666 & ~umask  # avoid restatting for chmod
 
 sx = s & 0o100



To: mharbison72, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D9204: tests: add notes about broken `hg log --follow ` with copies in extras

2020-10-14 Thread martinvonz (Martin von Zweigbergk) 
martinvonz created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  I also removed some unnecessary `#if no-changeset` where the `#else`
  was the same :P

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9204

AFFECTED FILES
  tests/test-copies-chain-merge.t

CHANGE DETAILS

diff --git a/tests/test-copies-chain-merge.t b/tests/test-copies-chain-merge.t
--- a/tests/test-copies-chain-merge.t
+++ b/tests/test-copies-chain-merge.t
@@ -402,29 +402,15 @@
 
 Log output should not include a merge commit as it did not happen
 
-#if no-changeset
-  $ hg log -Gfr 'desc("mBDm-0")' d
-  o  8 d-2 re-add d
-  |
-  ~
-#else
   $ hg log -Gfr 'desc("mBDm-0")' d
   o  8 d-2 re-add d
   |
   ~
-#endif
 
-#if no-changeset
   $ hg log -Gfr 'desc("mDBm-0")' d
   o  8 d-2 re-add d
   |
   ~
-#else
-  $ hg log -Gfr 'desc("mDBm-0")' d
-  o  8 d-2 re-add d
-  |
-  ~
-#endif
 
   $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mBDm-0")'
   M b
@@ -669,6 +655,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mBFm-0")' d
   o  22 f-2: rename i -> d
   |
@@ -684,6 +671,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mFBm-0")' d
   o  22 f-2: rename i -> d
   |
@@ -773,6 +761,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mDGm-0")' d
   o26 mDGm-0 simple merge - one way
   |\
@@ -801,6 +790,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mDGm-0")' d
   o26 mDGm-0 simple merge - one way
   |\
@@ -909,6 +899,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mFGm-0")' d
   o28 mFGm-0 simple merge - one way
   |\
@@ -938,6 +929,7 @@
   o  0 i-0 initial commit: a b h
   
 #else
+BROKEN: `hg log --follow ` relies on filelog metadata to work
   $ hg log -Gfr 'desc("mGFm-0")' d
   @29 mGFm-0 simple merge - the other way
   |\



To: martinvonz, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 7] revset: add diff(pattern) predicate for "grep --diff"

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1599556584 -32400
#  Tue Sep 08 18:16:24 2020 +0900
# Node ID 86c34dc393abb0cd636ec03a2e81a26f4b0694d5
# Parent  c71d4b4018af62936283b28ce4ed70cc9ea33840
revset: add diff(pattern) predicate for "grep --diff"

I find this is useful in GUI log viewer since the tool only needs to support
"log -rREV" command.

This is basic implementation. Windowed search is not implemented since it
wouldn't work pretty well with the smartset API. And filename matcher is
not supported because the syntax isn't determined. My idea is to add handling
of diff(pattern, file(..)) and diff(pattern, follow(..)), which will then be
evolved to a full revset+matcher combinator support:

  x & diff(pattern, y & z)
=
y & z builds (revs(y) & revs(z), matcher(y) & matcher(z))
pair, and narrows the search space of diff()
  
  diff() returns matched (revs, matcher) pair
  
  revs and matcher will be combined respectively by &-operator, and the matcher
  will optionally be used to filter "hg log -p" output

The predicate name "diff()" wouldn't be great, but grep() is already used.
Another options I can think of are "grepdiff()" and "containsdiff()".
Naming suggestions are welcome.

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -17,6 +17,7 @@ from . import (
 diffutil,
 encoding,
 error,
+grep as grepmod,
 hbisect,
 match as matchmod,
 node,
@@ -993,6 +994,45 @@ def destination(repo, subset, x):
 )
 
 
+@predicate(b'diff(pattern)', weight=110)
+def diff(repo, subset, x):
+"""Search revision differences for when the pattern was added or removed.
+
+The pattern may be a substring literal or a regular expression. See
+:hg:`help revisions.patterns`.
+"""
+args = getargsdict(x, b'diff', b'pattern')
+if b'pattern' not in args:
+# i18n: "diff" is a keyword
+raise error.ParseError(_(b'diff takes at least 1 argument'))
+
+pattern = getstring(args[b'pattern'], _(b'diff requires a string pattern'))
+regexp = stringutil.substringregexp(pattern, re.M)
+
+# TODO: add support for file pattern and --follow. For example,
+# diff(pattern[, set]) where set may be file(pattern) or follow(pattern),
+# and we'll eventually add a support for narrowing files by revset?
+fmatch = matchmod.always()
+
+def makefilematcher(ctx):
+return fmatch
+
+# TODO: search in a windowed way
+searcher = grepmod.grepsearcher(repo.ui, repo, regexp, diff=True)
+
+def testdiff(rev):
+# consume the generator to discard revfiles/matches cache
+found = False
+for fn, ctx, pstates, states in searcher.searchfiles(
+baseset([rev]), makefilematcher
+):
+if next(grepmod.difflinestates(pstates, states), None):
+found = True
+return found
+
+return subset.filter(testdiff, condrepr=(b'', pattern))
+
+
 @predicate(b'contentdivergent()', safe=True)
 def contentdivergent(repo, subset, x):
 """
diff --git a/tests/test-grep.t b/tests/test-grep.t
--- a/tests/test-grep.t
+++ b/tests/test-grep.t
@@ -21,6 +21,18 @@ pattern error
   grep: invalid match pattern: nothing to repeat* (glob)
   [1]
 
+invalid revset syntax
+
+  $ hg log -r 'diff()'
+  hg: parse error: diff takes at least 1 argument
+  [255]
+  $ hg log -r 'diff(:)'
+  hg: parse error: diff requires a string pattern
+  [255]
+  $ hg log -r 'diff("re:**test**")'
+  hg: parse error: invalid regular expression: nothing to repeat* (glob)
+  [255]
+
 simple
 
   $ hg grep -r tip:0 '.*'
@@ -553,6 +565,18 @@ Test wdir
   color:2:-:orange
   color:1:+:orange
 
+revset predicate for "grep --diff"
+
+  $ hg log -qr 'diff("re:^bl...$")'
+  0:203191eb5e21
+  $ hg log -qr 'diff("orange")'
+  1:7c585a21e0d1
+  2:11bd8bc8d653
+  3:e0116d3829f8
+  $ hg log -qr '2:0 & diff("orange")'
+  2:11bd8bc8d653
+  1:7c585a21e0d1
+
 test substring match: '^' should only match at the beginning
 
   $ hg grep -r tip:0 '^.' --config extensions.color= --color debug
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 7] stringutil: add function to compile stringmatcher pattern into regexp

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1601898039 -32400
#  Mon Oct 05 20:40:39 2020 +0900
# Node ID c71d4b4018af62936283b28ce4ed70cc9ea33840
# Parent  18bb913c45acf3f8d1fcbc90e22853f8e5f8e767
stringutil: add function to compile stringmatcher pattern into regexp

Prepares for adding a revset predicate for "grep --diff". The grep logic
needs a regexp object instead of a match function.

diff --git a/mercurial/utils/stringutil.py b/mercurial/utils/stringutil.py
--- a/mercurial/utils/stringutil.py
+++ b/mercurial/utils/stringutil.py
@@ -376,6 +376,58 @@ def stringmatcher(pattern, casesensitive
 raise error.ProgrammingError(b'unhandled pattern kind: %s' % kind)
 
 
+def substringregexp(pattern, flags=0):
+"""Build a regexp object from a string pattern possibly starting with
+'re:' or 'literal:' prefix.
+
+helper for tests:
+>>> def test(pattern, *tests):
+... regexp = substringregexp(pattern)
+... return [bool(regexp.search(t)) for t in tests]
+>>> def itest(pattern, *tests):
+... regexp = substringregexp(pattern, remod.I)
+... return [bool(regexp.search(t)) for t in tests]
+
+substring matching (no prefix):
+>>> test(b'bcde', b'abc', b'def', b'abcdefg')
+[False, False, True]
+
+substring pattern should be escaped:
+>>> substringregexp(b'.bc').pattern
+'.bc'
+>>> test(b'.bc', b'abc', b'def', b'abcdefg')
+[False, False, False]
+
+regex matching ('re:' prefix)
+>>> test(b're:a.+b', b'nomatch', b'fooadef', b'fooadefbar')
+[False, False, True]
+
+force substring matches ('literal:' prefix)
+>>> test(b'literal:re:foobar', b'foobar', b're:foobar')
+[False, True]
+
+case insensitive literal matches
+>>> itest(b'BCDE', b'abc', b'def', b'abcdefg')
+[False, False, True]
+
+case insensitive regex matches
+>>> itest(b're:A.+b', b'nomatch', b'fooadef', b'fooadefBar')
+[False, False, True]
+"""
+kind, pattern = _splitpattern(pattern)
+if kind == b're':
+try:
+return remod.compile(pattern, flags)
+except remod.error as e:
+raise error.ParseError(
+_(b'invalid regular expression: %s') % forcebytestr(e)
+)
+elif kind == b'literal':
+return remod.compile(remod.escape(pattern), flags)
+
+raise error.ProgrammingError(b'unhandled pattern kind: %s' % kind)
+
+
 def shortuser(user):
 """Return a short representation of a user name or email address."""
 f = user.find(b'@')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 7] py3: fix stringmatcher() to byte-stringify exception message

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1602681048 -32400
#  Wed Oct 14 22:10:48 2020 +0900
# Node ID 18bb913c45acf3f8d1fcbc90e22853f8e5f8e767
# Parent  b6c30689b9fe9e5f53bceb7913c71ae98109c804
py3: fix stringmatcher() to byte-stringify exception message

Spotted while writing regexp variant of stringmatcher().

diff --git a/mercurial/utils/stringutil.py b/mercurial/utils/stringutil.py
--- a/mercurial/utils/stringutil.py
+++ b/mercurial/utils/stringutil.py
@@ -361,7 +361,9 @@ def stringmatcher(pattern, casesensitive
 flags = remod.I
 regex = remod.compile(pattern, flags)
 except remod.error as e:
-raise error.ParseError(_(b'invalid regular expression: %s') % e)
+raise error.ParseError(
+_(b'invalid regular expression: %s') % forcebytestr(e)
+)
 return kind, pattern, regex.search
 elif kind == b'literal':
 if casesensitive:
diff --git a/tests/test-revset.t b/tests/test-revset.t
--- a/tests/test-revset.t
+++ b/tests/test-revset.t
@@ -1448,6 +1448,9 @@ test author
 (string '('))
   hg: parse error: invalid match pattern: (unbalanced parenthesis|missing 
\),.*) (re)
   [255]
+  $ log 'desc("re:(")'
+  hg: parse error: invalid regular expression: (unbalanced parenthesis|missing 
\),.*) (re)
+  [255]
   $ try 'grep("\bissue\d+")'
   (func
 (symbol 'grep')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 7] stringutil: extract helper function that splits stringmatcher() pattern

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1601898814 -32400
#  Mon Oct 05 20:53:34 2020 +0900
# Node ID b6c30689b9fe9e5f53bceb7913c71ae98109c804
# Parent  5b3013b8e40a903e7d31196c1e00877120339242
stringutil: extract helper function that splits stringmatcher() pattern

diff --git a/mercurial/utils/stringutil.py b/mercurial/utils/stringutil.py
--- a/mercurial/utils/stringutil.py
+++ b/mercurial/utils/stringutil.py
@@ -307,6 +307,14 @@ def binary(s):
 return bool(s and b'\0' in s)
 
 
+def _splitpattern(pattern):
+if pattern.startswith(b're:'):
+return b're', pattern[3:]
+elif pattern.startswith(b'literal:'):
+return b'literal', pattern[8:]
+return b'literal', pattern
+
+
 def stringmatcher(pattern, casesensitive=True):
 """
 accepts a string, possibly starting with 're:' or 'literal:' prefix.
@@ -345,8 +353,8 @@ def stringmatcher(pattern, casesensitive
 >>> itest(b'ABCDEFG', b'abc', b'def', b'abcdefg')
 ('literal', 'ABCDEFG', [False, False, True])
 """
-if pattern.startswith(b're:'):
-pattern = pattern[3:]
+kind, pattern = _splitpattern(pattern)
+if kind == b're':
 try:
 flags = 0
 if not casesensitive:
@@ -354,16 +362,16 @@ def stringmatcher(pattern, casesensitive
 regex = remod.compile(pattern, flags)
 except remod.error as e:
 raise error.ParseError(_(b'invalid regular expression: %s') % e)
-return b're', pattern, regex.search
-elif pattern.startswith(b'literal:'):
-pattern = pattern[8:]
+return kind, pattern, regex.search
+elif kind == b'literal':
+if casesensitive:
+match = pattern.__eq__
+else:
+ipat = encoding.lower(pattern)
+match = lambda s: ipat == encoding.lower(s)
+return kind, pattern, match
 
-match = pattern.__eq__
-
-if not casesensitive:
-ipat = encoding.lower(pattern)
-match = lambda s: ipat == encoding.lower(s)
-return b'literal', pattern, match
+raise error.ProgrammingError(b'unhandled pattern kind: %s' % kind)
 
 
 def shortuser(user):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 7] grep: extract main search loop as searcher method

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1599639458 -32400
#  Wed Sep 09 17:17:38 2020 +0900
# Node ID 5b3013b8e40a903e7d31196c1e00877120339242
# Parent  09a598f9d9790918f47f66b6ce5e51f198510289
grep: extract main search loop as searcher method

Still displayer part is in commands.grep(), the core grep logic is now
reusable. I'll revisit the displayer stuff later since it will be another
long series.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -3403,8 +3403,6 @@ def grep(ui, repo, pattern, *pats, **opt
 )
 
 getfile = searcher._getfile
-matches = searcher._matches
-copies = searcher._copies
 
 uipathfn = scmutil.getuipathfn(repo)
 
@@ -3514,8 +3512,6 @@ def grep(ui, repo, pattern, *pats, **opt
 fm.data(matched=False)
 fm.end()
 
-skip = searcher._skip
-revfiles = searcher._revfiles
 found = False
 
 wopts = logcmdutil.walkopts(
@@ -3532,29 +3528,11 @@ def grep(ui, repo, pattern, *pats, **opt
 
 ui.pager(b'grep')
 fm = ui.formatter(b'grep', opts)
-for ctx in scmutil.walkchangerevs(
-repo, revs, makefilematcher, searcher._prep
-):
-rev = ctx.rev()
-parent = ctx.p1().rev()
-for fn in sorted(revfiles.get(rev, [])):
-states = matches[rev][fn]
-copy = copies.get(rev, {}).get(fn)
-if fn in skip:
-if copy:
-skip.add(copy)
-continue
-pstates = matches.get(parent, {}).get(copy or fn, [])
-if pstates or states:
-r = display(fm, fn, ctx, pstates, states)
-found = found or r
-if r and not diff and not all_files:
-searcher.skipfile(fn, rev)
-del revfiles[rev]
-# We will keep the matches dict for the duration of the window
-# clear the matches dict once the window is over
-if not revfiles:
-matches.clear()
+for fn, ctx, pstates, states in searcher.searchfiles(revs, 
makefilematcher):
+r = display(fm, fn, ctx, pstates, states)
+found = found or r
+if r and not diff and not all_files:
+searcher.skipfile(fn, ctx.rev())
 fm.end()
 
 return not found
diff --git a/mercurial/grep.py b/mercurial/grep.py
--- a/mercurial/grep.py
+++ b/mercurial/grep.py
@@ -115,6 +115,34 @@ class grepsearcher(object):
 if copy:
 self._skip.add(copy)
 
+def searchfiles(self, revs, makefilematcher):
+"""Walk files and revisions to yield (fn, ctx, pstates, states)
+matches
+
+states is a list of linestate objects. pstates may be empty unless
+diff is True.
+"""
+for ctx in scmutil.walkchangerevs(
+self._repo, revs, makefilematcher, self._prep
+):
+rev = ctx.rev()
+parent = ctx.p1().rev()
+for fn in sorted(self._revfiles.get(rev, [])):
+states = self._matches[rev][fn]
+copy = self._copies.get(rev, {}).get(fn)
+if fn in self._skip:
+if copy:
+self._skip.add(copy)
+continue
+pstates = self._matches.get(parent, {}).get(copy or fn, [])
+if pstates or states:
+yield fn, ctx, pstates, states
+del self._revfiles[rev]
+# We will keep the matches dict for the duration of the window
+# clear the matches dict once the window is over
+if not self._revfiles:
+self._matches.clear()
+
 def _grepbody(self, fn, rev, body):
 self._matches[rev].setdefault(fn, [])
 m = self._matches[rev][fn]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 7] scmutil: move walkchangerevs() from cmdutil

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1601785077 -32400
#  Sun Oct 04 13:17:57 2020 +0900
# Node ID 09a598f9d9790918f47f66b6ce5e51f198510289
# Parent  d36d703c291e45670245384440993ff70ad17da4
scmutil: move walkchangerevs() from cmdutil

It's no longer a command-level function, but a pure helper to walk revisions
in a windowed way. This change will help eliminate reverse dependency of
revset.py -> grep.py -> cmdutil.py in future patches.

diff --git a/hgext/churn.py b/hgext/churn.py
--- a/hgext/churn.py
+++ b/hgext/churn.py
@@ -23,6 +23,7 @@ from mercurial import (
 patch,
 pycompat,
 registrar,
+scmutil,
 )
 
 cmdtable = {}
@@ -98,7 +99,7 @@ def countrate(ui, repo, amap, *pats, **o
 exclude_pats=opts[b'exclude'],
 )
 revs, makefilematcher = logcmdutil.makewalker(repo, wopts)
-for ctx in cmdutil.walkchangerevs(repo, revs, makefilematcher, prep):
+for ctx in scmutil.walkchangerevs(repo, revs, makefilematcher, prep):
 continue
 
 progress.complete()
diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -2240,55 +2240,6 @@ def finddate(ui, repo, date):
 return b'%d' % rev
 
 
-def increasingwindows(windowsize=8, sizelimit=512):
-while True:
-yield windowsize
-if windowsize < sizelimit:
-windowsize *= 2
-
-
-def walkchangerevs(repo, revs, makefilematcher, prepare):
-'''Iterate over files and the revs in a "windowed" way.
-
-Callers most commonly need to iterate backwards over the history
-in which they are interested. Doing so has awful (quadratic-looking)
-performance, so we use iterators in a "windowed" way.
-
-We walk a window of revisions in the desired order.  Within the
-window, we first walk forwards to gather data, then in the desired
-order (usually backwards) to display it.
-
-This function returns an iterator yielding contexts. Before
-yielding each context, the iterator will first call the prepare
-function on each context in the window in forward order.'''
-
-if not revs:
-return []
-change = repo.__getitem__
-
-def iterate():
-it = iter(revs)
-stopiteration = False
-for windowsize in increasingwindows():
-nrevs = []
-for i in pycompat.xrange(windowsize):
-rev = next(it, None)
-if rev is None:
-stopiteration = True
-break
-nrevs.append(rev)
-for rev in sorted(nrevs):
-ctx = change(rev)
-prepare(ctx, makefilematcher(ctx))
-for rev in nrevs:
-yield change(rev)
-
-if stopiteration:
-break
-
-return iterate()
-
-
 def add(ui, repo, match, prefix, uipathfn, explicitonly, **opts):
 bad = []
 
diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -3532,7 +3532,7 @@ def grep(ui, repo, pattern, *pats, **opt
 
 ui.pager(b'grep')
 fm = ui.formatter(b'grep', opts)
-for ctx in cmdutil.walkchangerevs(
+for ctx in scmutil.walkchangerevs(
 repo, revs, makefilematcher, searcher._prep
 ):
 rev = ctx.rev()
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -760,6 +760,55 @@ def revrange(repo, specs, localalias=Non
 return repo.anyrevs(allspecs, user=True, localalias=localalias)
 
 
+def increasingwindows(windowsize=8, sizelimit=512):
+while True:
+yield windowsize
+if windowsize < sizelimit:
+windowsize *= 2
+
+
+def walkchangerevs(repo, revs, makefilematcher, prepare):
+'''Iterate over files and the revs in a "windowed" way.
+
+Callers most commonly need to iterate backwards over the history
+in which they are interested. Doing so has awful (quadratic-looking)
+performance, so we use iterators in a "windowed" way.
+
+We walk a window of revisions in the desired order.  Within the
+window, we first walk forwards to gather data, then in the desired
+order (usually backwards) to display it.
+
+This function returns an iterator yielding contexts. Before
+yielding each context, the iterator will first call the prepare
+function on each context in the window in forward order.'''
+
+if not revs:
+return []
+change = repo.__getitem__
+
+def iterate():
+it = iter(revs)
+stopiteration = False
+for windowsize in increasingwindows():
+nrevs = []
+for i in pycompat.xrange(windowsize):
+rev = next(it, None)
+if rev is None:
+stopiteration = True
+break
+nrevs.append(rev)
+for rev in sorted(nrevs):
+ctx = change(rev)
+prepare(ctx,

[PATCH 1 of 7] grep: extract public function to register file to be skipped

2020-10-14 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1599638684 -32400
#  Wed Sep 09 17:04:44 2020 +0900
# Node ID d36d703c291e45670245384440993ff70ad17da4
# Parent  0428978bca22dc1173577ca6247d588182805b78
grep: extract public function to register file to be skipped

The main grep loop will be extracted to a searcher method, but this skipping
condition depends on the result of display() function.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -3549,9 +3549,7 @@ def grep(ui, repo, pattern, *pats, **opt
 r = display(fm, fn, ctx, pstates, states)
 found = found or r
 if r and not diff and not all_files:
-skip.add(fn)
-if copy:
-skip.add(copy)
+searcher.skipfile(fn, rev)
 del revfiles[rev]
 # We will keep the matches dict for the duration of the window
 # clear the matches dict once the window is over
diff --git a/mercurial/grep.py b/mercurial/grep.py
--- a/mercurial/grep.py
+++ b/mercurial/grep.py
@@ -107,6 +107,14 @@ class grepsearcher(object):
 self._skip = set()
 self._revfiles = {}
 
+def skipfile(self, fn, rev):
+"""Exclude the given file (and the copy at the specified revision)
+from future search"""
+copy = self._copies.get(rev, {}).get(fn)
+self._skip.add(fn)
+if copy:
+self._skip.add(copy)
+
 def _grepbody(self, fn, rev, body):
 self._matches[rev].setdefault(fn, [])
 m = self._matches[rev][fn]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH V2] mergestate: add `allextras()` to get all extras

2020-10-14 Thread Yuya Nishihara 
On Wed, 14 Oct 2020 13:47:17 +0530, Pulkit Goyal wrote:
> # HG changeset patch
> # User Pulkit Goyal <7895pul...@gmail.com>
> # Date 1602313984 -19800
> #  Sat Oct 10 12:43:04 2020 +0530
> # Node ID 4884022d4ad9b918f863946f6557fac47c664af7
> # Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
> # EXP-Topic merge-newnode-final
> mergestate: add `allextras()` to get all extras

Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH V2] mergestate: document `o` merge record state in _mergestate_base docs

2020-10-14 Thread Yuya Nishihara 
On Wed, 14 Oct 2020 13:40:43 +0530, Pulkit Goyal wrote:
> # HG changeset patch
> # User Pulkit Goyal <7895pul...@gmail.com>
> # Date 1601991694 -19800
> #  Tue Oct 06 19:11:34 2020 +0530
> # Node ID c3ffbf50856b954e3a8be968b9a93000b03b1843
> # Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
> mergestate: document `o` merge record state in _mergestate_base docs

Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH V2] mergestate: make filename argument optional in _mergestate_base.extras()

2020-10-14 Thread Pulkit Goyal 
On Wed, Oct 14, 2020 at 1:46 PM Pulkit Goyal <7895pul...@gmail.com> wrote:
>
> # HG changeset patch
> # User Pulkit Goyal <7895pul...@gmail.com>
> # Date 1602313984 -19800
> #  Sat Oct 10 12:43:04 2020 +0530
> # Node ID 16290c7c8eea7f859aade5c8c2f9af357da97164
> # Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
> # EXP-Topic merge-newnode-final
> mergestate: make filename argument optional in _mergestate_base.extras()
>
> Earlier `extras()` can only be used for getting extra for a file. However at
> couple of places in code, we wanted to iterate over all the extras stored with
> the mergestate and they were accessing the private `_stateextras`.
>
> Now, if filename is not passed, we return all the extras.
>
> Differential Revision: https://phab.mercurial-scm.org/D9190

Kindly ignore this. Forgot to update commit message before sending.
Sent another one with updated commit message.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH V2] mergestate: add `allextras()` to get all extras

2020-10-14 Thread Pulkit Goyal 
# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1602313984 -19800
#  Sat Oct 10 12:43:04 2020 +0530
# Node ID 4884022d4ad9b918f863946f6557fac47c664af7
# Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
# EXP-Topic merge-newnode-final
mergestate: add `allextras()` to get all extras

`extras()` can only be used for getting extra for a file. However at
couple of places in code, we wanted to iterate over all the extras stored with
the mergestate and they were accessing the private `_stateextras`.

We add a new function for this.

Differential Revision: https://phab.mercurial-scm.org/D9190

diff --git a/mercurial/commit.py b/mercurial/commit.py
--- a/mercurial/commit.py
+++ b/mercurial/commit.py
@@ -155,7 +155,7 @@ def _get_salvaged(repo, ms, ctx):
 copy_sd = repo.filecopiesmode == b'changeset-sidedata'
 if copy_sd and len(ctx.parents()) > 1:
 if ms.active():
-for fname in sorted(ms._stateextras.keys()):
+for fname in sorted(ms.allextras().keys()):
 might_removed = 
ms.extras(fname).get(b'merge-removal-candidate')
 if might_removed == b'yes':
 if fname in ctx:
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2107,7 +2107,7 @@ def debugmergestate(ui, repo, *args, **o
 fm_files.end()
 
 fm_extras = fm.nested(b'extras')
-for f, d in sorted(pycompat.iteritems(ms._stateextras)):
+for f, d in sorted(pycompat.iteritems(ms.allextras())):
 if f in ms:
 # If file is in mergestate, we have already processed it's extras
 continue
diff --git a/mercurial/mergestate.py b/mercurial/mergestate.py
--- a/mercurial/mergestate.py
+++ b/mercurial/mergestate.py
@@ -305,7 +305,12 @@ class _mergestate_base(object):
 ):
 yield f
 
+def allextras(self):
+""" return all extras information stored with the mergestate """
+return self._stateextras
+
 def extras(self, filename):
+""" return extras stored with the mergestate for the given filename """
 return self._stateextras[filename]
 
 def _resolve(self, preresolve, dfile, wctx):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH V2] mergestate: make filename argument optional in _mergestate_base.extras()

2020-10-14 Thread Pulkit Goyal 
# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1602313984 -19800
#  Sat Oct 10 12:43:04 2020 +0530
# Node ID 16290c7c8eea7f859aade5c8c2f9af357da97164
# Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
# EXP-Topic merge-newnode-final
mergestate: make filename argument optional in _mergestate_base.extras()

Earlier `extras()` can only be used for getting extra for a file. However at
couple of places in code, we wanted to iterate over all the extras stored with
the mergestate and they were accessing the private `_stateextras`.

Now, if filename is not passed, we return all the extras.

Differential Revision: https://phab.mercurial-scm.org/D9190

diff --git a/mercurial/commit.py b/mercurial/commit.py
--- a/mercurial/commit.py
+++ b/mercurial/commit.py
@@ -155,7 +155,7 @@ def _get_salvaged(repo, ms, ctx):
 copy_sd = repo.filecopiesmode == b'changeset-sidedata'
 if copy_sd and len(ctx.parents()) > 1:
 if ms.active():
-for fname in sorted(ms._stateextras.keys()):
+for fname in sorted(ms.allextras().keys()):
 might_removed = 
ms.extras(fname).get(b'merge-removal-candidate')
 if might_removed == b'yes':
 if fname in ctx:
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2107,7 +2107,7 @@ def debugmergestate(ui, repo, *args, **o
 fm_files.end()
 
 fm_extras = fm.nested(b'extras')
-for f, d in sorted(pycompat.iteritems(ms._stateextras)):
+for f, d in sorted(pycompat.iteritems(ms.allextras())):
 if f in ms:
 # If file is in mergestate, we have already processed it's extras
 continue
diff --git a/mercurial/mergestate.py b/mercurial/mergestate.py
--- a/mercurial/mergestate.py
+++ b/mercurial/mergestate.py
@@ -305,7 +305,12 @@ class _mergestate_base(object):
 ):
 yield f
 
+def allextras(self):
+""" return all extras information stored with the mergestate """
+return self._stateextras
+
 def extras(self, filename):
+""" return extras stored with the mergestate for the given filename """
 return self._stateextras[filename]
 
 def _resolve(self, preresolve, dfile, wctx):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH V2] mergestate: document `o` merge record state in _mergestate_base docs

2020-10-14 Thread Pulkit Goyal 
# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1601991694 -19800
#  Tue Oct 06 19:11:34 2020 +0530
# Node ID c3ffbf50856b954e3a8be968b9a93000b03b1843
# Parent  04de8a1ec08f380fda794dda3b3f2ed1ccfd442b
mergestate: document `o` merge record state in _mergestate_base docs

_mergestate_base documentation serves as a nice documentation for mergestate.
This also documents known merge records and known merge record states.

I missed adding `o` state to it when I introduced it. Let's add it now.

Differential Revision: https://phab.mercurial-scm.org/D9156

diff --git a/mercurial/mergestate.py b/mercurial/mergestate.py
--- a/mercurial/mergestate.py
+++ b/mercurial/mergestate.py
@@ -160,6 +160,7 @@ class _mergestate_base(object):
 r: resolved conflict
 pu: unresolved path conflict (file conflicts with directory)
 pr: resolved path conflict
+o: file was merged in favor of other parent of merge (DEPRECATED)
 
 The resolve command transitions between 'u' and 'r' for conflicts and
 'pu' and 'pr' for path conflicts.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel