D7941: py3: wallpaper over demand importer differences on Python 3.5

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  For some reason, Python 3.5 and only Python 3.5's demand importer
  is importing extra parent modules. This is causing test output
  on 3.5 to diverge.
  
  While I'm not thrilled about the divergence in behavior, I don't
  think it is critical enough to warrant fixing in code. So this
  commit teaches our test to recognize the divergent behavior on
  Python 3.5.
  
  Because the test format doesn't support conditional exit codes and
  we were already inside an #if (it also doesn't allow nested #ifs),
  I added a `|| false` to normalize the exit code. The actual exit
  code shouldn't matter here, as we have tests for that elsewhere.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7941

AFFECTED FILES
  tests/test-extension.t

CHANGE DETAILS

diff --git a/tests/test-extension.t b/tests/test-extension.t
--- a/tests/test-extension.t
+++ b/tests/test-extension.t
@@ -574,25 +574,35 @@
   > NO_CHECK_EOF
 
 Examine module importing.
-
-  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config 
extensions.absextroot=$TESTTMP/absextroot showabsolute)
-  LIB: this is extlibroot.lsub1.lsub2.used
-  LIB: this is extlibroot.lsub1.lsub2.called.func()
-  LIB: this is extlibroot.recursedown.abs.used
-  LIB: this is extlibroot.recursedown.legacy.used
-  LIB: this is extlibroot.shadowing.used
-  ABS: this is absextroot.xsub1.xsub2.used
-  ABS: this is absextroot.xsub1.xsub2.called.func()
+Python 3.5's lazy module importer varies in behavior from ours and the
+one in the standard library at 3.6+. This is not desirable. But it is
+a corner case in behavior and we don't expect many people to be running
+into it.
 
-  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config 
extensions.absextroot=$TESTTMP/absextroot showrelative)
-  LIB: this is extlibroot.lsub1.lsub2.used
-  LIB: this is extlibroot.lsub1.lsub2.called.func()
-  LIB: this is extlibroot.recursedown.abs.used
-  LIB: this is extlibroot.recursedown.legacy.used
-  LIB: this is extlibroot.shadowing.used
-  REL: this is absextroot.xsub1.xsub2.used
-  REL: this is absextroot.xsub1.xsub2.called.func()
-  REL: this relimporter imports 'this is absextroot.relimportee'
+  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config 
extensions.absextroot=$TESTTMP/absextroot showabsolute) || true
+  LIB: this is extlibroot.lsub1.lsub2.used (no-pyexact35 !)
+  LIB: this is extlibroot.lsub1.lsub2.called.func() (no-pyexact35 !)
+  LIB: this is extlibroot.recursedown.abs.used (no-pyexact35 !)
+  LIB: this is extlibroot.recursedown.legacy.used (no-pyexact35 !)
+  LIB: this is extlibroot.shadowing.used (no-pyexact35 !)
+  ABS: this is absextroot.xsub1.xsub2.used (no-pyexact35 !)
+  ABS: this is absextroot.xsub1.xsub2.called.func() (no-pyexact35 !)
+  *** failed to import extension absextroot from $TESTTMP/absextroot: 
extlibroot.lsub1.lsub2.unused is loaded unintentionally (pyexact35 !)
+  hg: unknown command 'showabsolute' (pyexact35 !)
+  (use 'hg help' for a list of commands) (pyexact35 !)
+
+  $ (PYTHONPATH=${PYTHONPATH}${PATHSEP}${TESTTMP}; hg --config 
extensions.absextroot=$TESTTMP/absextroot showrelative) || true
+  LIB: this is extlibroot.lsub1.lsub2.used (no-pyexact35 !)
+  LIB: this is extlibroot.lsub1.lsub2.called.func() (no-pyexact35 !)
+  LIB: this is extlibroot.recursedown.abs.used (no-pyexact35 !)
+  LIB: this is extlibroot.recursedown.legacy.used (no-pyexact35 !)
+  LIB: this is extlibroot.shadowing.used (no-pyexact35 !)
+  REL: this is absextroot.xsub1.xsub2.used (no-pyexact35 !)
+  REL: this is absextroot.xsub1.xsub2.called.func() (no-pyexact35 !)
+  REL: this relimporter imports 'this is absextroot.relimportee' (no-pyexact35 
!)
+  *** failed to import extension absextroot from $TESTTMP/absextroot: 
extlibroot.lsub1.lsub2.unused is loaded unintentionally (pyexact35 !)
+  hg: unknown command 'showrelative' (pyexact35 !)
+  (did you mean shelve?) (pyexact35 !)
 
 Examine whether sub-module is imported relatively as expected.
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7939: py3: glob over exception in test-check-py3-compat.t

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Python 3.6+ raise ModuleNotFoundError and older versions raise
  ImportError. Glob over the exception differences.
  
  For whatever reason, we were already doing this for one failure.
  But not all occurrences of ModuleNotFoundError were changed.
  Who knows.
  
  This test should now pass on all Python versions (although I didn't
  check Windows).

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7939

AFFECTED FILES
  tests/test-check-py3-compat.t

CHANGE DETAILS

diff --git a/tests/test-check-py3-compat.t b/tests/test-check-py3-compat.t
--- a/tests/test-check-py3-compat.t
+++ b/tests/test-check-py3-compat.t
@@ -41,9 +41,9 @@
   hgext/infinitepush/sqlindexapi.py: error importing: <*Error> No module named 
'mysql' (error at sqlindexapi.py:*) (glob) (?)
   mercurial/scmwindows.py: error importing:  _type_ 'v' not 
supported (error at win32.py:*) (no-windows !)
   mercurial/win32.py: error importing:  _type_ 'v' not supported 
(error at win32.py:*) (no-windows !)
-  mercurial/windows.py: error importing:  No module named 
'msvcrt' (error at windows.py:*) (no-windows !)
-  mercurial/posix.py: error importing:  No module named 
'fcntl' (error at posix.py:*) (windows !)
-  mercurial/scmposix.py: error importing:  No module 
named 'fcntl' (error at scmposix.py:*) (windows !)
+  mercurial/windows.py: error importing: <*Error> No module named 'msvcrt' 
(error at windows.py:*) (glob) (no-windows !)
+  mercurial/posix.py: error importing: <*Error> No module named 'fcntl' (error 
at posix.py:*) (glob) (windows !)
+  mercurial/scmposix.py: error importing: <*Error> No module named 'fcntl' 
(error at scmposix.py:*) (glob) (windows !)
 #endif
 
 #if py3 pygments



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7938: py3: string normalization and I/O tweaks in test-lfs.t

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The print was inserting b'' on Python 3. In addition, since we
  weren't writing to the ui instance (which isn't readily available
  in this function), output order could get mixed up.
  
  We add some pycompat casts and a stdout flush to make the test
  happy on all Python versions.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7938

AFFECTED FILES
  tests/test-lfs.t

CHANGE DETAILS

diff --git a/tests/test-lfs.t b/tests/test-lfs.t
--- a/tests/test-lfs.t
+++ b/tests/test-lfs.t
@@ -810,8 +810,11 @@
 blob, and the output shows that it isn't fetched.
 
   $ cat > $TESTTMP/lfsrename.py < import sys
+  > 
   > from mercurial import (
   > exthelper,
+  > pycompat,
   > )
   > 
   > from hgext.lfs import (
@@ -829,7 +832,10 @@
   > rawtext = self._revlog.rawdata(node)
   > metadata = pointer.deserialize(rawtext)
   > print('lfs blob %s renamed %s -> %s'
-  >   % (metadata[b'oid'], ret[0], self._revlog.filename))
+  >   % (pycompat.sysstr(metadata[b'oid']),
+  >  pycompat.sysstr(ret[0]),
+  >  pycompat.fsdecode(self._revlog.filename)))
+  > sys.stdout.flush()
   > return ret
   > EOF
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44109: 7 new changesets

[Mercurial Commits]

7 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/2077ffede71f
changeset:   44103:2077ffede71f
user:Martin von Zweigbergk 
date:Thu Jan 16 08:41:38 2020 -0800
summary: examples: refer to nightly rustfmt in Windows-compatible way

https://www.mercurial-scm.org/repo/hg/rev/85c4cd73996b
changeset:   44104:85c4cd73996b
user:Pierre-Yves David 
date:Sun Nov 17 14:39:28 2019 +0100
summary: localrepo: also fastpath access to working copy parents when 
possible

https://www.mercurial-scm.org/repo/hg/rev/d86dede17392
changeset:   44105:d86dede17392
user:Pierre-Yves David 
date:Sun Nov 17 14:47:29 2019 +0100
summary: localrepo: fastpath access to "."

https://www.mercurial-scm.org/repo/hg/rev/bfaf4c673bec
changeset:   44106:bfaf4c673bec
user:Pierre-Yves David 
date:Sun Nov 17 14:54:41 2019 +0100
summary: localrepo: recognize trivial request for '.'

https://www.mercurial-scm.org/repo/hg/rev/5a012404503b
changeset:   44107:5a012404503b
user:Pierre-Yves David 
date:Thu Nov 21 23:25:08 2019 +0100
summary: localrepo: also fast past the parents of working copies parents

https://www.mercurial-scm.org/repo/hg/rev/c472970339d2
changeset:   44108:c472970339d2
user:Pierre-Yves David 
date:Thu Nov 21 23:46:51 2019 +0100
summary: changectx: use unfiltered changelog to walk ancestors in annotate

https://www.mercurial-scm.org/repo/hg/rev/98349eddceef
changeset:   44109:98349eddceef
bookmark:@
tag: tip
user:Pierre-Yves David 
date:Thu Nov 21 17:27:44 2019 +0100
summary: changectx: mark parent of changesets as non filtered

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7937: python-zstandard: blacken at 80 characters

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I made this change upstream and it will make it into the next
  release of python-zstandard. I figured I'd send it Mercurial's
  way because it will allow us to drop this directory from the black
  exclusion list.
  
  1. skip-blame blackening

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7937

AFFECTED FILES
  black.toml
  contrib/python-zstandard/make_cffi.py
  contrib/python-zstandard/setup.py
  contrib/python-zstandard/setup_zstd.py
  contrib/python-zstandard/tests/common.py
  contrib/python-zstandard/tests/test_buffer_util.py
  contrib/python-zstandard/tests/test_compressor.py
  contrib/python-zstandard/tests/test_compressor_fuzzing.py
  contrib/python-zstandard/tests/test_data_structures.py
  contrib/python-zstandard/tests/test_data_structures_fuzzing.py
  contrib/python-zstandard/tests/test_decompressor.py
  contrib/python-zstandard/tests/test_decompressor_fuzzing.py
  contrib/python-zstandard/tests/test_train_dictionary.py
  contrib/python-zstandard/zstandard/cffi.py
  tests/test-check-format.t

CHANGE DETAILS

diff --git a/tests/test-check-format.t b/tests/test-check-format.t
--- a/tests/test-check-format.t
+++ b/tests/test-check-format.t
@@ -1,5 +1,5 @@
 #require black
 
   $ cd $RUNTESTDIR/..
-  $ black --config=black.toml --check --diff `hg files 'set:(**.py + 
grep("^#!.*python")) - mercurial/thirdparty/** - "contrib/python-zstandard/**"'`
+  $ black --config=black.toml --check --diff `hg files 'set:(**.py + 
grep("^#!.*python")) - mercurial/thirdparty/**'`
 
diff --git a/contrib/python-zstandard/zstandard/cffi.py 
b/contrib/python-zstandard/zstandard/cffi.py
--- a/contrib/python-zstandard/zstandard/cffi.py
+++ b/contrib/python-zstandard/zstandard/cffi.py
@@ -299,10 +299,14 @@
 _set_compression_parameter(params, lib.ZSTD_c_chainLog, chain_log)
 _set_compression_parameter(params, lib.ZSTD_c_searchLog, search_log)
 _set_compression_parameter(params, lib.ZSTD_c_minMatch, min_match)
-_set_compression_parameter(params, lib.ZSTD_c_targetLength, 
target_length)
+_set_compression_parameter(
+params, lib.ZSTD_c_targetLength, target_length
+)
 
 if strategy != -1 and compression_strategy != -1:
-raise ValueError("cannot specify both compression_strategy and 
strategy")
+raise ValueError(
+"cannot specify both compression_strategy and strategy"
+)
 
 if compression_strategy != -1:
 strategy = compression_strategy
@@ -313,12 +317,16 @@
 _set_compression_parameter(
 params, lib.ZSTD_c_contentSizeFlag, write_content_size
 )
-_set_compression_parameter(params, lib.ZSTD_c_checksumFlag, 
write_checksum)
+_set_compression_parameter(
+params, lib.ZSTD_c_checksumFlag, write_checksum
+)
 _set_compression_parameter(params, lib.ZSTD_c_dictIDFlag, 
write_dict_id)
 _set_compression_parameter(params, lib.ZSTD_c_jobSize, job_size)
 
 if overlap_log != -1 and overlap_size_log != -1:
-raise ValueError("cannot specify both overlap_log and 
overlap_size_log")
+raise ValueError(
+"cannot specify both overlap_log and overlap_size_log"
+)
 
 if overlap_size_log != -1:
 overlap_log = overlap_size_log
@@ -326,12 +334,16 @@
 overlap_log = 0
 
 _set_compression_parameter(params, lib.ZSTD_c_overlapLog, overlap_log)
-_set_compression_parameter(params, lib.ZSTD_c_forceMaxWindow, 
force_max_window)
+_set_compression_parameter(
+params, lib.ZSTD_c_forceMaxWindow, force_max_window
+)
 _set_compression_parameter(
 params, lib.ZSTD_c_enableLongDistanceMatching, enable_ldm
 )
 _set_compression_parameter(params, lib.ZSTD_c_ldmHashLog, ldm_hash_log)
-_set_compression_parameter(params, lib.ZSTD_c_ldmMinMatch, 
ldm_min_match)
+_set_compression_parameter(
+params, lib.ZSTD_c_ldmMinMatch, ldm_min_match
+)
 _set_compression_parameter(
 params, lib.ZSTD_c_ldmBucketSizeLog, ldm_bucket_size_log
 )
@@ -346,7 +358,9 @@
 elif ldm_hash_rate_log == -1:
 ldm_hash_rate_log = 0
 
-_set_compression_parameter(params, lib.ZSTD_c_ldmHashRateLog, 
ldm_hash_rate_log)
+_set_compression_parameter(
+params, lib.ZSTD_c_ldmHashRateLog, ldm_hash_rate_log
+)
 
 @property
 def format(self):
@@ -354,7 +368,9 @@
 
 @property
 def compression_level(self):
-return _get_compression_parameter(self._params, 
lib.ZSTD_c_compressionLevel)
+return _get_compression_parameter(
+self._params, lib.ZSTD_c_compressionLevel
+)
 
 @property
 

D7790: rust-node: handling binary Node prefix

[martinvonz (Martin von Zweigbergk)]

martinvonz added inline comments.

INLINE COMMENTS

> node.rs:62-64
> +/// Since it can potentially come from an hexadecimal representation with
> +/// odd length, it needs to carry around whether the last 4 bits are relevant
> +/// or not.

Did you consider using a full u8 for each nibble instead? It seems like that 
would be simpler. I'd appreciate it if you could spend a few minutes to try 
that (if you haven't already).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7790/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7790

To: gracinet, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7936: help: minor copy editing to the `config.format` section

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7936

AFFECTED FILES
  mercurial/helptext/config.txt

CHANGE DETAILS

diff --git a/mercurial/helptext/config.txt b/mercurial/helptext/config.txt
--- a/mercurial/helptext/config.txt
+++ b/mercurial/helptext/config.txt
@@ -827,18 +827,18 @@
 --
 
 Configuration that controls the repository format. Newer format options are 
more
-powerful but incompatible with some older versions of Mercurial. Format options
+powerful, but incompatible with some older versions of Mercurial. Format 
options
 are considered at repository initialization only. You need to make a new clone
-for config change to be taken into account.
+for config changes to be taken into account.
 
 For more details about repository format and version compatibility, see
 https://www.mercurial-scm.org/wiki/MissingRequirement
 
 ``usegeneraldelta``
 Enable or disable the "generaldelta" repository format which improves
-repository compression by allowing "revlog" to store delta against 
arbitrary
-revision instead of the previous stored one. This provides significant
-improvement for repositories with branches.
+repository compression by allowing "revlog" to store deltas against
+arbitrary revisions instead of the previously stored one. This provides
+significant improvement for repositories with branches.
 
 Repositories with this on-disk format require Mercurial version 1.9.
 
@@ -847,7 +847,7 @@
 ``dotencode``
 Enable or disable the "dotencode" repository format which enhances
 the "fncache" repository format (which has to be enabled to use
-dotencode) to avoid issues with filenames starting with ._ on
+dotencode) to avoid issues with filenames starting with "._" on
 Mac OS X and spaces on Windows.
 
 Repositories with this on-disk format require Mercurial version 1.7.
@@ -885,12 +885,14 @@
 Enabled by default.
 
 ``revlog-compression``
-Compression algorithm used by revlog. Supported value are `zlib` and 
`zstd`.
-The `zlib` engine is the historical default of Mercurial. `zstd` is a newer
-format that is usually a net win over `zlib` operating faster at better
-compression rate. Use `zstd` to reduce CPU usage.
-
-On some system, Mercurial installation may lack `zstd` supports. Default 
is `zlib`.
+Compression algorithm used by revlog. Supported values are `zlib` and
+`zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is
+a newer format that is usually a net win over `zlib`, operating faster at
+better compression rates. Use `zstd` to reduce CPU usage.
+
+On some systems, the Mercurial installation may lack `zstd` support.
+
+Default is `zlib`.
 
 ``bookmarks-in-store``
 Store bookmarks in .hg/store/. This means that bookmarks are shared when



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7920: tests: restore phabricator tests and regenerate the recordings

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D7920#116596 , @mharbison72 
wrote:
  
  > I did `hg phabread D7920 | hg import --bypass -` on another Windows system, 
and it gives the following error when running the test (omitting the cascading 
failures):
  
  Nevermind this.  If I set HGENCODING=utf-8 before doing the phabread and 
import, it runs fine, so it's got the right data in phabricator.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7920/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7920

To: mharbison72, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7789: rust-revlog: a trait for the revlog index

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  > The None return cases in node() match what the index_node()
  > C function does.
  
  You mean the code on 
https://www.mercurial-scm.org/repo/hg/file/fdaa4233dc18/mercurial/cext/revlog.c#l388?
 Line 394 there returns nullid, so does that really match? Why not return the 
nullid when given nullrev as input?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7789/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7789

To: gracinet, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7788: rust-node: binary Node and conversion utilities

[martinvonz (Martin von Zweigbergk)]

This revision now requires changes to proceed.
martinvonz added inline comments.
martinvonz requested changes to this revision.

INLINE COMMENTS

> dirs_multiset.rs:111
>  path.as_ref().to_owned(),
> -))
> +));
>  }

please revert unrelated change

> kevincox wrote in node.rs:31
> I find progressing through the string easier to understand than this slicing. 
> WDYT.
> 
>   for byte in node.iter_mut() {
>   *byte = u8::from_str_radix([..2], 16)?;
>   hex = [2..];
>   }

Or even use `hex::decode()` from the `hex` crate? Can also use `hex::encode()` 
in `node_to_hex` if we're okay with that.

> node.rs:41-51
> +/// Retrieve the `i`th half-byte from a bytes slice
> +///
> +/// This is also the `i`th hexadecimal digit in numeric form,
> +/// also called a [nybble](https://en.wikipedia.org/wiki/Nibble).
> +pub fn get_nybble(i: usize, s: &[u8]) -> u8 {
> +if i % 2 == 0 {
> +s[i / 2] >> 4

This feels like something that will only be used in nodemap.rs, so put it there 
instead? Or do you think (or know) that it will be used elsewhere soon?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7788/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7788

To: gracinet, #hg-reviewers, kevincox, martinvonz
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44102: 2 new changesets

[Mercurial Commits]

2 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/25097b4d2c6f
changeset:   44101:25097b4d2c6f
user:Inada Naoki 
date:Tue Jan 14 17:57:15 2020 +0900
summary: remotefilelog: fix opening validatecachelog in text mode

https://www.mercurial-scm.org/repo/hg/rev/fdaa4233dc18
changeset:   44102:fdaa4233dc18
bookmark:@
tag: tip
user:Joerg Sonnenberger 
date:Thu Dec 26 19:05:55 2019 +0100
summary: convert: refactor authormap into separate function for outside use

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7920: tests: restore phabricator tests and regenerate the recordings

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D7920#116596 , @mharbison72 
wrote:
  
  > I did `hg phabread D7920 | hg import --bypass -` on another Windows system, 
and it gives the following error when running the test (omitting the cascading 
failures):
  >
  >   --- d:/mercurial/tests/test-phabricator.t
  >   +++ d:/mercurial/tests/test-phabricator.t.err
  >   @@ -56,7 +56,10 @@
  >  $ echo alpha > alpha
  >  $ hg ci --addremove -m 'create alpha for phabricator test Ç'
  >  adding alpha
  >   -
  >   +  transaction abort!
  >   +  rollback completed
  >   +  abort: decoding near 'ator test \x80': 'utf8' codec can't decode byte 
0x80 in position 34: invalid start byte! (esc)
  >   +  [255]
  >A bad .arcconfig doesn't error out
  >  $ echo 'garbage' > .arcconfig
  >  $ hg config phabricator --debug
  >   ERROR: test-phabricator.t output changed
  >
  > So I'm not sure if there is a `phabsend` or `phabread` issue.
  
  It looks like it only complains on py2.7.15, but both py27 and py37 fail the 
import checker on this system:
  
--- d:/mercurial/tests/test-check-module-imports.t
+++ d:/mercurial/tests/test-check-module-imports.t.err
@@ -37,3 +37,18 @@
   > -X tests/test-imports-checker.t \
   > -X tests/test-verify-repo-operations.py \
   > | sed 's-\\-/-g' | "$PYTHON" "$import_checker" -
+  Traceback (most recent call last):\r (esc)
+File "d:\\mercurial\\tests/../contrib/import-checker.py", line 820, in 
\r (esc)
+  sys.exit(int(main(sys.argv)))\r (esc)
+File "d:\\mercurial\\tests/../contrib/import-checker.py", line 787, in 
main\r (esc)
+  for src, modname, name, line in sources(source_path, 
localmodname):\r (esc)
+File "d:\\mercurial\\tests/../contrib/import-checker.py", line 765, in 
sources\r (esc)
+  for script, modname, t, line in embedded(f, modname, src):\r (esc)
+File "d:\\mercurial\\tests/../contrib/import-checker.py", line 731, in 
embedded\r (esc)
+  for name, starts, ends, code in testparseutil.pyembedded(f, src, 
errors):\r (esc)
+File "d:\\mercurial\\contrib\\testparseutil.py", line 186, in 
embedded\r (esc)
+  for lineno, line in enumerate(lines, 1):\r (esc)
+File "c:\\Program Files\\Python37\\lib\\codecs.py", line 322, in 
decode\r (esc)
+  (result, consumed) = self._buffer_decode(data, self.errors, final)\r 
(esc)
+  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 
2164: invalid start byte\r (esc)
+  [1]
  
  All tests ran cleanly on the original system.  The *.t was resurrected by 
reverting to the parent of the revision that deleted it.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7920/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7920

To: mharbison72, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7935: phabricator: use .arcconfig for `phabricator.url` if not set locally

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added subscribers: mercurial-devel, Kwan.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This setting is also per repo; see the previous commit for details.
  
  The existing `conduit_uri` setting is the previous name of 
`phabricator.uri`[1]
  and while it could easily be queried before the latter for compatibility, the
  config in this repo has '/api' appended. That's already done in 
`callconduit()`,
  which would clearly end up giving the wrong result. It looks like the path of
  the URL is now ignored in user configs[2], so add the modern setting without 
it
  to this repo's .arcconfig.
  
  Sadly, we still need to have contributors configure `auth.hg.phabtoken` (and
  therefore `auth.hg.prefix` to link it to `phabricator.url`) in order to submit
  patches, but at least now it's localized to a single section.
  
  [1] 
https://secure.phabricator.com/book/phabricator/article/arcanist_new_project/
  [2] 
https://github.com/phacility/arcanist/blob/cc850163f30c4697e925df0d6212469679600a2c/scripts/arcanist.php#L271

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7935

AFFECTED FILES
  .arcconfig
  hgext/phabricator.py
  tests/test-phabricator.t

CHANGE DETAILS

diff --git a/tests/test-phabricator.t b/tests/test-phabricator.t
--- a/tests/test-phabricator.t
+++ b/tests/test-phabricator.t
@@ -228,8 +228,8 @@
   $ mv .hg/hgrc .hg/hgrc.bak
   $ hg config phabricator --debug
   read config from: */.hgrc (glob)
-  */.hgrc:*: phabricator.url=global (glob)
   $TESTTMP/repo/.arcconfig: phabricator.callsign=HG
+  $TESTTMP/repo/.arcconfig: phabricator.url=https://phab.mercurial-scm.org/
 
 But it doesn't override local config
   $ cat >> .hg/hgrc << EOF
diff --git a/hgext/phabricator.py b/hgext/phabricator.py
--- a/hgext/phabricator.py
+++ b/hgext/phabricator.py
@@ -178,11 +178,16 @@
 except IOError:
 pass
 
+cfg = util.sortdict()
+
 if b"repository.callsign" in arcconfig:
-ui.applyconfig(
-{(b"phabricator", b"callsign"): arcconfig[b"repository.callsign"]},
-source=wdirvfs.join(b".arcconfig"),
-)
+cfg[(b"phabricator", b"callsign")] = arcconfig[b"repository.callsign"]
+
+if b"phabricator.uri" in arcconfig:
+cfg[(b"phabricator", b"url")] = arcconfig[b"phabricator.uri"]
+
+if cfg:
+ui.applyconfig(cfg, source=wdirvfs.join(b".arcconfig"))
 
 return orig(ui, wdirvfs, hgvfs, requirements) or result  # Load .hg/hgrc
 
diff --git a/.arcconfig b/.arcconfig
--- a/.arcconfig
+++ b/.arcconfig
@@ -1,5 +1,6 @@
 {
 "conduit_uri": "https://phab.mercurial-scm.org/api;,
+"phabricator.uri": "https://phab.mercurial-scm.org/;,
 "repository.callsign": "HG",
 "arc.land.onto.default": "@",
 "base": "hg:.^"



To: mharbison72, #hg-reviewers
Cc: Kwan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7934: phabricator: use .arcconfig for the callsign if not set locally (issue6243)

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added subscribers: mercurial-devel, Kwan.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This makes things easier for people working with more than one repository
  because this file can be committed to each repository.  The bug report asks to
  read /.arcrc, but AFAICT, that file lives in ~/ and holds the 
credentials.
  And we already track an .arcconfig file.  Any callsign set globally is still
  used if that is all that is present, but .arcconfig will override it if
  available.  The idea behind letting the local hgrc override .arcconfig is that
  the developer may need to do testing against another server, and not dirty the
  working directory.
  
  Originally I was going to just try to read the callsign in `getrepophid()` if 
it
  wasn't present in the hg config.  That works fine, but I think it also makes
  sense to read the URL from this file too.  That would have worked less well
  because `readurltoken()` doesn't have access to the repo object to know where 
to
  find the file.  Supplimenting the config mechanism is less magical because it
  reports the source and value of the properties used, and it doesn't need to 
read
  the file twice.
  
  Invalid hgrc files generally cause the program to abort.  I only flagged it 
as a
  warning here because it's not our config file, not crucial to the whole 
program
  operating, and really shouldn't be corrupt in the typical case where it is
  checked into the repo.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7934

AFFECTED FILES
  hgext/phabricator.py
  tests/test-phabricator.t

CHANGE DETAILS

diff --git a/tests/test-phabricator.t b/tests/test-phabricator.t
--- a/tests/test-phabricator.t
+++ b/tests/test-phabricator.t
@@ -210,5 +210,37 @@
extensions.loadall(self.ui)

   
+A bad .arcconfig doesn't error out
+  $ echo 'garbage' > .arcconfig
+  $ hg config phabricator --debug
+  invalid JSON in $TESTTMP/repo/.arcconfig
+  read config from: */.hgrc (glob)
+  $TESTTMP/repo/.hg/hgrc:*: phabricator.url=https://phab.mercurial-scm.org/ 
(glob)
+  $TESTTMP/repo/.hg/hgrc:*: phabricator.callsign=HG (glob)
+
+The .arcconfig content overrides global config
+  $ cat >> $HGRCPATH << EOF
+  > [phabricator]
+  > url = global
+  > callsign = global
+  > EOF
+  $ cp $TESTDIR/../.arcconfig .
+  $ mv .hg/hgrc .hg/hgrc.bak
+  $ hg config phabricator --debug
+  read config from: */.hgrc (glob)
+  */.hgrc:*: phabricator.url=global (glob)
+  $TESTTMP/repo/.arcconfig: phabricator.callsign=HG
+
+But it doesn't override local config
+  $ cat >> .hg/hgrc << EOF
+  > [phabricator]
+  > url = local
+  > callsign = local
+  > EOF
+  $ hg config phabricator --debug
+  read config from: */.hgrc (glob)
+  $TESTTMP/repo/.hg/hgrc:*: phabricator.url=local (glob)
+  $TESTTMP/repo/.hg/hgrc:*: phabricator.callsign=local (glob)
+  $ mv .hg/hgrc.bak .hg/hgrc
 
   $ cd ..
diff --git a/hgext/phabricator.py b/hgext/phabricator.py
--- a/hgext/phabricator.py
+++ b/hgext/phabricator.py
@@ -66,6 +66,7 @@
 exthelper,
 graphmod,
 httpconnection as httpconnectionmod,
+localrepo,
 logcmdutil,
 match,
 mdiff,
@@ -101,6 +102,7 @@
 command = eh.command
 configtable = eh.configtable
 templatekeyword = eh.templatekeyword
+uisetup = eh.finaluisetup
 
 # developer config: phabricator.batchsize
 eh.configitem(
@@ -152,6 +154,39 @@
 ]
 
 
+@eh.wrapfunction(localrepo, "loadhgrc")
+def _loadhgrc(orig, ui, wdirvfs, hgvfs, requirements):
+"""Load ``.arcconfig`` content into a ui instance on repository open.
+"""
+result = False
+arcconfig = {}
+
+try:
+# json.loads only accepts bytes from 3.6+
+rawparams = encoding.unifromlocal(wdirvfs.read(b".arcconfig"))
+# json.loads only returns unicode strings
+arcconfig = pycompat.rapply(
+lambda x: encoding.unitolocal(x)
+if isinstance(x, pycompat.unicode)
+else x,
+pycompat.json_loads(rawparams),
+)
+
+result = True
+except ValueError:
+ui.warn(_(b"invalid JSON in %s\n") % wdirvfs.join(b".arcconfig"))
+except IOError:
+pass
+
+if b"repository.callsign" in arcconfig:
+ui.applyconfig(
+{(b"phabricator", b"callsign"): arcconfig[b"repository.callsign"]},
+source=wdirvfs.join(b".arcconfig"),
+)
+
+return orig(ui, wdirvfs, hgvfs, requirements) or result  # Load .hg/hgrc
+
+
 def vcrcommand(name, flags, spec, helpcategory=None, optionalrepo=False):
 fullflags = flags + _VCR_FLAGS
 



To: mharbison72, #hg-reviewers
Cc: Kwan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7933: config: add a function to insert non-file based, but overridable settings

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This will be used in the next patch.
  
  Until relatively recently (473510bf0575 
), 
there was no official way for
  extensions to inject per-repo config data, so it probably makes sense that
  `ui.setconfig()` items are sticky, and not affected by loading more config
  files.  But that makes it cumbersome if the extension wants to allow the data 
it
  might add to be overridden by any data in the local hgrc file.  The only 
thing I
  could get to work was to load the local hgrc first, and then check if the 
source
  for the config item that should be overridden was *not* the local hgrc file
  name.  But that's brittle because in addition to the file name, the source
  contains the line number, there are the usual '\' vs '/' platform differences,
  etc.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7933

AFFECTED FILES
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -467,6 +467,25 @@
 raise
 self.warn(_(b'ignored: %s\n') % stringutil.forcebytestr(inst))
 
+self._applyconfig(cfg, trusted, root)
+
+def applyconfig(self, configitems, source=b"", root=None):
+"""Add configitems from a non-file source.  Unlike with 
``setconfig()``,
+they can be overridden by subsequent config file reads.  The items are
+in the same format as ``configoverride()``, namely a dict of the
+following structures: {(section, name) : value}
+
+Typically this is used by extensions that inject themselves into the
+config file load procedure by monkeypatching ``localrepo.loadhgrc()``.
+"""
+cfg = config.config()
+
+for (section, name), value in configitems.items():
+cfg.set(section, name, value, source)
+
+self._applyconfig(cfg, True, root)
+
+def _applyconfig(self, cfg, trusted, root):
 if self.plain():
 for k in (
 b'debug',



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7920: tests: restore phabricator tests and regenerate the recordings

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  I did `hg phabread D7920 | hg import --bypass -` on another Windows system, 
and it gives the following error when running the test (omitting the cascading 
failures):
  
--- d:/mercurial/tests/test-phabricator.t
+++ d:/mercurial/tests/test-phabricator.t.err
@@ -56,7 +56,10 @@
   $ echo alpha > alpha
   $ hg ci --addremove -m 'create alpha for phabricator test Ç'
   adding alpha
-
+  transaction abort!
+  rollback completed
+  abort: decoding near 'ator test \x80': 'utf8' codec can't decode byte 
0x80 in position 34: invalid start byte! (esc)
+  [255]
 A bad .arcconfig doesn't error out
   $ echo 'garbage' > .arcconfig
   $ hg config phabricator --debug

ERROR: test-phabricator.t output changed
  
  So I'm not sure if there is a `phabsend` or `phabread` issue.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7920/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7920

To: mharbison72, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7932: [RFC]debugbackups: introduce command to interact with strip backups

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This vendors backups extension from hg-experimental.
  
  Listing backups and having some utility to apply them is nice. I know we have
  obsmarkers now, but this will help a lot of end users who still uses strip 
until
  we get evolve out of experimental.
  
  This is RFC because:
  
  - I am not sure whether we want this in core
  - I copied the code from hg-experimental and looks like there is some code
  
  deduplication which needs to be done

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7932

AFFECTED FILES
  mercurial/debugcommands.py
  tests/test-debugbackup.t

CHANGE DETAILS

diff --git a/tests/test-debugbackup.t b/tests/test-debugbackup.t
new file mode 100644
--- /dev/null
+++ b/tests/test-debugbackup.t
@@ -0,0 +1,36 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > strip=
+  > EOF
+
+Setup repo
+
+  $ hg init repo
+  $ cd repo
+
+Test backups list and recover
+
+  $ mkcommit() {
+  >echo "$1" > "$1"
+  >hg add "$1"
+  >hg ci -l $1
+  > }
+  $ mkcommit a
+  $ mkcommit b
+  $ hg strip .
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/d2ae7f538514-2953539b-backup.hg (glob)
+  $ hg debugbackups
+  Recover changesets using: hg debugbackups --recover 
+  
+  Available backup changesets:
+  * (glob)
+  d2ae7f538514 b
+
+  $ hg debugbackups --recover d2ae7f538514
+  Unbundling d2ae7f538514
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets d2ae7f538514 (1 drafts)
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -11,6 +11,7 @@
 import collections
 import difflib
 import errno
+import glob
 import operator
 import os
 import random
@@ -38,6 +39,7 @@
 )
 from . import (
 bundle2,
+bundlerepo,
 changegroup,
 cmdutil,
 color,
@@ -130,6 +132,137 @@
 
 
 @command(
+"debugbackups",
+[
+(
+"",
+"recover",
+"",
+"brings the specified changeset back into the repository",
+)
+]
++ cmdutil.logopts,
+_("hg debugbackups [--recover HASH]"),
+)
+def debugbackups(ui, repo, *pats, **opts):
+"""lists the changesets available in backup bundles
+
+Without any arguments, this command prints a list of the changesets in each
+backup bundle.
+
+--recover takes a changeset hash and unbundles the first bundle that
+contains that hash, which puts that changeset back in your repository.
+
+--verbose will print the entire commit message and the bundle path for that
+backup.
+"""
+backuppath = repo.vfs.join("strip-backup")
+backups = filter(os.path.isfile, glob.glob(backuppath + "/*.hg"))
+backups.sort(key=lambda x: os.path.getmtime(x), reverse=True)
+
+opts["bundle"] = ""
+opts["force"] = None
+
+def display(other, chlist, displayer):
+limit = logcmdutil.getlimit(opts)
+if opts.get("newest_first"):
+chlist.reverse()
+count = 0
+for n in chlist:
+if limit is not None and count >= limit:
+break
+parents = [p for p in other.changelog.parents(n) if p != nullid]
+if opts.get("no_merges") and len(parents) == 2:
+continue
+count += 1
+displayer.show(other[n])
+
+recovernode = opts.get("recover")
+if recovernode:
+if scmutil.isrevsymbol(repo, recovernode):
+ui.warn(_("%s already exists in the repo\n") % recovernode)
+return
+else:
+msg = _(
+"Recover changesets using: hg debugbackups --recover "
+"\n\nAvailable backup changesets:"
+)
+ui.status(msg, label="status.removed")
+
+for backup in backups:
+# Much of this is copied from the hg incoming logic
+source = os.path.relpath(backup, encoding.getcwd())
+source = ui.expandpath(source)
+source, branches = hg.parseurl(source, opts.get("branch"))
+try:
+other = hg.peer(repo, opts, source)
+except error.LookupError as ex:
+msg = _("\nwarning: unable to open bundle %s") % source
+hint = _("\n(missing parent rev %s)\n") % short(ex.name)
+ui.warn(msg)
+ui.warn(hint)
+continue
+revs, checkout = hg.addbranchrevs(
+repo, other, branches, opts.get("rev")
+)
+
+if revs:
+revs = [other.lookup(rev) for rev in revs]
+
+quiet = ui.quiet
+try:
+ui.quiet = True
+other, chlist, cleanupfn = bundlerepo.getremotechanges(
+ui, repo, other, revs, opts["bundle"], opts["force"]
+  

Heptapod 0.8.0 (SSH) released

[Georges Racinet]

We're glad to announce version 0.8.0 of Heptapod, the project to
integrate Mercurial in GitLab.

What follows is an abridged version of the online announcement published
earlier today [1].

It's been a crowded development cycle, as we are readying ourselves for
public hosting at Clever Cloud [2], thanks to a partnership with Octobus.

Building on the groundwork layed down in Heptapod 0.7.0, several major
targets have been attained in this new version:

- SSH support
- Install from source
- Full import of Bitbucket Pull Requests

Further important foundational work has been done: we're hoping to offer
contributors a much better experience in the near future, and finally,
work on the HGitaly project has started for good.

It's a perfect time to start contributing - well, can it be ever a bad
time?

More on what's going on and upgrade notes are available from the full
announcement [1].

Cheers, and happy 2020 everyone.


[1] https://heptapod.net/heptapod-080-released.html

[2] https://www.clever-cloud.com/en/heptapod

-- 
Georges Racinet
https://octobus.net, https://heptapod.net
GPG: BF5456F4DC625443849B6E58EE20CA44EF691D39, sur serveurs publics




signature.asc
Description: OpenPGP digital signature
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7894: nodemap: introduce an option to use mmap to read the nodemap mapping

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19435.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7894?vs=19312=19435

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7894/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7894

AFFECTED FILES
  mercurial/configitems.py
  mercurial/debugcommands.py
  mercurial/localrepo.py
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -84,3 +84,37 @@
   $ hg debugnodemap --check
   revision in index:   5002
   revision in nodemap: 5002
+
+Test code path without mmap
+---
+
+  $ echo bar > bar
+  $ hg add bar
+  $ hg ci -m 'bar' --config experimental.exp-persistent-nodemap.mmap=no
+
+  $ hg debugnodemap --check --config 
experimental.exp-persistent-nodemap.mmap=yes
+  revision in index:   5003
+  revision in nodemap: 5003
+  $ hg debugnodemap --check --config 
experimental.exp-persistent-nodemap.mmap=no
+  revision in index:   5003
+  revision in nodemap: 5003
+
+
+#if pure
+  $ hg debugnodemap --metadata
+  uid:  (glob)
+  tip-rev: 5002
+  data-length: 246656
+  data-unused: 768
+  $ f --sha256 .hg/store/00changelog-*.nd --size
+  .hg/store/00changelog-.nd: size=246656, 
sha256=8221807a0860a7a65002d2d3e0d33512d28aa6db2433db966e56aa17dcf6329f (glob)
+
+#else
+  $ hg debugnodemap --metadata
+  uid:  (glob)
+  tip-rev: 5002
+  data-length: 245888
+  data-unused: 0
+  $ f --sha256 .hg/store/00changelog-*.nd --size
+  .hg/store/00changelog-.nd: size=245888, 
sha256=42233b63e5567fd362fb7847fa7a9f4d40ad93c28f8571197b356a69fe8bd271 (glob)
+#endif
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -8,6 +8,7 @@
 
 from __future__ import absolute_import
 
+import errno
 import os
 import re
 import struct
@@ -46,10 +47,17 @@
 
 filename = _rawdata_filepath(revlog, docket)
 data = revlog.opener.tryread(filename)
+try:
+with revlog.opener(filename) as fd:
+if revlog.opener.options.get("exp-persistent-nodemap.mmap"):
+data = fd.read(data_length)
+else:
+data = util.buffer(util.mmapread(fd, data_length))
+except OSError as e:
+if e.errno != errno.ENOENT:
+raise
 if len(data) < data_length:
 return None
-elif len(data) > data_length:
-data = data[:data_length]
 return docket, data
 
 
@@ -81,6 +89,8 @@
 
 can_incremental = util.safehasattr(revlog.index, 
"nodemap_data_incremental")
 ondisk_docket = revlog._nodemap_docket
+feed_data = util.safehasattr(revlog.index, "update_nodemap_data")
+use_mmap = revlog.opener.options.get("exp-persistent-nodemap.mmap")
 
 data = None
 # first attemp an incremental update of the data
@@ -97,12 +107,18 @@
 datafile = _rawdata_filepath(revlog, target_docket)
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
 # store vfs
+new_length = target_docket.data_length + len(data)
 with revlog.opener(datafile, 'r+') as fd:
 fd.seek(target_docket.data_length)
 fd.write(data)
-fd.seek(0)
-new_data = fd.read(target_docket.data_length + len(data))
-target_docket.data_length += len(data)
+if feed_data:
+if use_mmap:
+fd.seek(0)
+new_data = fd.read(new_length)
+else:
+fd.flush()
+new_data = util.buffer(util.mmapread(fd, new_length))
+target_docket.data_length = new_length
 target_docket.data_unused += data_changed_count
 
 if data is None:
@@ -115,9 +131,14 @@
 data = persistent_data(revlog.index)
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
 # store vfs
-new_data = data
-with revlog.opener(datafile, 'w') as fd:
+with revlog.opener(datafile, 'w+') as fd:
 fd.write(data)
+if feed_data:
+if use_mmap:
+new_data = data
+else:
+fd.flush()
+new_data = util.buffer(util.mmapread(fd, len(data)))
 target_docket.data_length = len(data)
 target_docket.tip_rev = revlog.tiprev()
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
@@ -125,7 +146,7 @@
 with revlog.opener(revlog.nodemap_file, 'w', atomictemp=True) as fp:
 fp.write(target_docket.serialize())
 revlog._nodemap_docket = target_docket
-if util.safehasattr(revlog.index, 

D7888: nodemap: track the maximum revision tracked in the nodemap

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19433.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7888?vs=19306=19433

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7888/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7888

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/pure/parsers.py
  mercurial/revlog.py
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -14,8 +14,9 @@
   $ hg debugbuilddag .+5000
   $ hg debugnodemap --metadata
   uid:  (glob)
+  tip-rev: 5000
   $ f --size .hg/store/00changelog.n
-  .hg/store/00changelog.n: size=18
+  .hg/store/00changelog.n: size=26
   $ f --sha256 .hg/store/00changelog-*.nd
   .hg/store/00changelog-.nd: 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c (glob)
   $ hg debugnodemap --dump-new | f --sha256 --size
@@ -51,8 +52,9 @@
   $ hg ci -m 'foo'
   $ hg debugnodemap --metadata
   uid:  (glob)
+  tip-rev: 5001
   $ f --size .hg/store/00changelog.n
-  .hg/store/00changelog.n: size=18
+  .hg/store/00changelog.n: size=26
 
 (The pure code use the debug code that perform incremental update, the C code 
reencode from scratch)
 
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -36,9 +36,11 @@
 if version != ONDISK_VERSION:
 return None
 offset += S_VERSION.size
-(uid_size,) = S_HEADER.unpack(pdata[offset : offset + S_HEADER.size])
+headers = S_HEADER.unpack(pdata[offset : offset + S_HEADER.size])
+uid_size, tip_rev = headers
 offset += S_HEADER.size
 docket = NodeMapDocket(pdata[offset : offset + uid_size])
+docket.tip_rev = tip_rev
 
 filename = _rawdata_filepath(revlog, docket)
 return docket, revlog.opener.tryread(filename)
@@ -94,6 +96,7 @@
 # store vfs
 with revlog.opener(datafile, 'w') as fd:
 fd.write(data)
+target_docket.tip_rev = revlog.tiprev()
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
 # store vfs
 with revlog.opener(revlog.nodemap_file, 'w', atomictemp=True) as fp:
@@ -142,7 +145,7 @@
 ONDISK_VERSION = 0
 
 S_VERSION = struct.Struct(">B")
-S_HEADER = struct.Struct(">B")
+S_HEADER = struct.Struct(">BQ")
 
 ID_SIZE = 8
 
@@ -164,15 +167,19 @@
 if uid is None:
 uid = _make_uid()
 self.uid = uid
+self.tip_rev = None
 
 def copy(self):
-return NodeMapDocket(uid=self.uid)
+new = NodeMapDocket(uid=self.uid)
+new.tip_rev = self.tip_rev
+return new
 
 def serialize(self):
 """return serialized bytes for a docket using the passed uid"""
 data = []
 data.append(S_VERSION.pack(ONDISK_VERSION))
-data.append(S_HEADER.pack(len(self.uid)))
+headers = (len(self.uid), self.tip_rev)
+data.append(S_HEADER.pack(*headers))
 data.append(self.uid)
 return b''.join(data)
 
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -639,7 +639,7 @@
 if use_nodemap:
 nodemap_data = nodemaputil.persisted_data(self)
 if nodemap_data is not None:
-index.update_nodemap_data(nodemap_data[1])
+index.update_nodemap_data(*nodemap_data)
 except (ValueError, IndexError):
 raise error.RevlogError(
 _(b"index %s is corrupted") % self.indexfile
diff --git a/mercurial/pure/parsers.py b/mercurial/pure/parsers.py
--- a/mercurial/pure/parsers.py
+++ b/mercurial/pure/parsers.py
@@ -170,7 +170,7 @@
 self._nm_root = self._nm_max_idx = self._nm_rev = None
 return data
 
-def update_nodemap_data(self, nm_data):
+def update_nodemap_data(self, docket, nm_data):
 """provide full serialiazed data from a nodemap
 
 The data are expected to come from disk. See `nodemap_data_all` for a
@@ -178,7 +178,7 @@
 if nm_data is not None:
 self._nm_root, self._nm_max_idx = nodemaputil.parse_data(nm_data)
 if self._nm_root:
-self._nm_rev = len(self) - 1
+self._nm_rev = docket.tip_rev
 else:
 self._nm_root = self._nm_max_idx = self._nm_rev = None
 
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2130,6 +2130,7 @@
 if nm_data is not None:
 docket, data = nm_data
 ui.write((b"uid: %s\n") % docket.uid)
+ui.write((b"tip-rev: %d\n") % docket.tip_rev)
 
 
 @command(



To: marmoute, indygreg, #hg-reviewers
Cc: 

D7889: nodemap: track the total and unused amount of data in the rawdata file

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19434.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7889?vs=19307=19434

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7889/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7889

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/pure/parsers.py
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -15,8 +15,10 @@
   $ hg debugnodemap --metadata
   uid:  (glob)
   tip-rev: 5000
+  data-length: 245760
+  data-unused: 0
   $ f --size .hg/store/00changelog.n
-  .hg/store/00changelog.n: size=26
+  .hg/store/00changelog.n: size=42
   $ f --sha256 .hg/store/00changelog-*.nd
   .hg/store/00changelog-.nd: 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c (glob)
   $ hg debugnodemap --dump-new | f --sha256 --size
@@ -50,11 +52,22 @@
   $ echo foo > foo
   $ hg add foo
   $ hg ci -m 'foo'
+
+#if pure
   $ hg debugnodemap --metadata
   uid:  (glob)
   tip-rev: 5001
+  data-length: 246144
+  data-unused: 384
+#else
+  $ hg debugnodemap --metadata
+  uid:  (glob)
+  tip-rev: 5001
+  data-length: 245760
+  data-unused: 0
+#endif
   $ f --size .hg/store/00changelog.n
-  .hg/store/00changelog.n: size=26
+  .hg/store/00changelog.n: size=42
 
 (The pure code use the debug code that perform incremental update, the C code 
reencode from scratch)
 
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -37,10 +37,12 @@
 return None
 offset += S_VERSION.size
 headers = S_HEADER.unpack(pdata[offset : offset + S_HEADER.size])
-uid_size, tip_rev = headers
+uid_size, tip_rev, data_length, data_unused = headers
 offset += S_HEADER.size
 docket = NodeMapDocket(pdata[offset : offset + uid_size])
 docket.tip_rev = tip_rev
+docket.data_length = data_length
+docket.data_unused = data_unused
 
 filename = _rawdata_filepath(revlog, docket)
 return docket, revlog.opener.tryread(filename)
@@ -78,12 +80,14 @@
 # first attemp an incremental update of the data
 if can_incremental and ondisk_docket is not None:
 target_docket = revlog._nodemap_docket.copy()
-data = revlog.index.nodemap_data_incremental()
+data_changed_count, data = revlog.index.nodemap_data_incremental()
 datafile = _rawdata_filepath(revlog, target_docket)
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
 # store vfs
 with revlog.opener(datafile, 'a') as fd:
 fd.write(data)
+target_docket.data_length += len(data)
+target_docket.data_unused += data_changed_count
 else:
 # otherwise fallback to a full new export
 target_docket = NodeMapDocket()
@@ -96,6 +100,7 @@
 # store vfs
 with revlog.opener(datafile, 'w') as fd:
 fd.write(data)
+target_docket.data_length = len(data)
 target_docket.tip_rev = revlog.tiprev()
 # EXP-TODO: if this is a cache, this should use a cache vfs, not a
 # store vfs
@@ -143,9 +148,8 @@
 
 # version 0 is experimental, no BC garantee, do no use outside of tests.
 ONDISK_VERSION = 0
-
 S_VERSION = struct.Struct(">B")
-S_HEADER = struct.Struct(">BQ")
+S_HEADER = struct.Struct(">BQQQ")
 
 ID_SIZE = 8
 
@@ -168,17 +172,26 @@
 uid = _make_uid()
 self.uid = uid
 self.tip_rev = None
+self.data_length = None
+self.data_unused = 0
 
 def copy(self):
 new = NodeMapDocket(uid=self.uid)
 new.tip_rev = self.tip_rev
+new.data_length = self.data_length
+new.data_unused = self.data_unused
 return new
 
 def serialize(self):
 """return serialized bytes for a docket using the passed uid"""
 data = []
 data.append(S_VERSION.pack(ONDISK_VERSION))
-headers = (len(self.uid), self.tip_rev)
+headers = (
+len(self.uid),
+self.tip_rev,
+self.data_length,
+self.data_unused,
+)
 data.append(S_HEADER.pack(*headers))
 data.append(self.uid)
 return b''.join(data)
@@ -236,8 +249,8 @@
 def update_persistent_data(index, root, max_idx, last_rev):
 """return the serialised data of a nodemap for a given index
 """
-trie = _update_trie(index, root, last_rev)
-return _dump_trie(trie, existing_idx=max_idx)
+changed_block, trie = _update_trie(index, root, last_rev)
+return changed_block * S_BLOCK.size, _dump_trie(trie, existing_idx=max_idx)
 
 
 S_BLOCK = struct.Struct(">" + ("q" * 16))
@@ -293,10 +306,11 @@
 
 def _update_trie(index, root, last_rev):
 

D7887: nodemap: add a flag to dump the details of the docket

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19432.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7887?vs=19305=19432

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7887/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7887

AFFECTED FILES
  mercurial/debugcommands.py
  tests/test-completion.t
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -12,6 +12,8 @@
   > persistent-nodemap=yes
   > EOF
   $ hg debugbuilddag .+5000
+  $ hg debugnodemap --metadata
+  uid:  (glob)
   $ f --size .hg/store/00changelog.n
   .hg/store/00changelog.n: size=18
   $ f --sha256 .hg/store/00changelog-*.nd
@@ -47,6 +49,8 @@
   $ echo foo > foo
   $ hg add foo
   $ hg ci -m 'foo'
+  $ hg debugnodemap --metadata
+  uid:  (glob)
   $ f --size .hg/store/00changelog.n
   .hg/store/00changelog.n: size=18
 
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -290,7 +290,7 @@
   debugmanifestfulltextcache: clear, add
   debugmergestate: 
   debugnamecomplete: 
-  debugnodemap: dump-new, dump-disk, check
+  debugnodemap: dump-new, dump-disk, check, metadata
   debugobsolete: flags, record-parents, rev, exclusive, index, delete, date, 
user, template
   debugp1copies: rev
   debugp2copies: rev
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2093,6 +2093,12 @@
 False,
 _(b'check that the data on disk data are correct.'),
 ),
+(
+'',
+b'metadata',
+False,
+_(b'display the on disk meta data for the nodemap'),
+),
 ],
 )
 def debugnodemap(ui, repo, **opts):
@@ -2117,6 +2123,13 @@
 if nm_data is not None:
 docket, data = nm_data
 return nodemap.check_data(ui, cl.index, data)
+elif opts['metadata']:
+unfi = repo.unfiltered()
+cl = unfi.changelog
+nm_data = nodemap.persisted_data(cl)
+if nm_data is not None:
+docket, data = nm_data
+ui.write((b"uid: %s\n") % docket.uid)
 
 
 @command(



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7885: nodemap: keep track of the docket for loaded data

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19431.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7885?vs=19303=19431

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7885/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7885

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/revlog.py
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -41,7 +41,7 @@
 docket = NodeMapDocket(pdata[offset : offset + uid_size])
 
 filename = _rawdata_filepath(revlog, docket)
-return revlog.opener.tryread(filename)
+return docket, revlog.opener.tryread(filename)
 
 
 def setup_persistent_nodemap(tr, revlog):
@@ -93,6 +93,7 @@
 # store vfs
 with revlog.opener(revlog.nodemap_file, 'w', atomictemp=True) as fp:
 fp.write(target_docket.serialize())
+revlog._nodemap_docket = target_docket
 # EXP-TODO: if the transaction abort, we should remove the new data and
 # reinstall the old one.
 
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -455,6 +455,7 @@
 self._maxchainlen = None
 self._deltabothparents = True
 self.index = None
+self._nodemap_docket = None
 # Mapping of partial identifiers to full nodes.
 self._pcache = {}
 # Mapping of revision integer to full node.
@@ -544,6 +545,9 @@
 indexdata = b''
 self._initempty = True
 try:
+nodemap_data = nodemaputil.persisted_data(self)
+if nodemap_data is not None:
+self._nodemap_docket = nodemap_data[0]
 with self._indexfp() as f:
 if (
 mmapindexthreshold is not None
@@ -635,7 +639,7 @@
 if use_nodemap:
 nodemap_data = nodemaputil.persisted_data(self)
 if nodemap_data is not None:
-index.update_nodemap_data(nodemap_data)
+index.update_nodemap_data(nodemap_data[1])
 except (ValueError, IndexError):
 raise error.RevlogError(
 _(b"index %s is corrupted") % self.indexfile
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2106,13 +2106,17 @@
 elif opts['dump_disk']:
 unfi = repo.unfiltered()
 cl = unfi.changelog
-data = nodemap.persisted_data(cl)
-ui.write(data)
+nm_data = nodemap.persisted_data(cl)
+if nm_data is not None:
+docket, data = nm_data
+ui.write(data)
 elif opts['check']:
 unfi = repo.unfiltered()
 cl = unfi.changelog
-data = nodemap.persisted_data(cl)
-return nodemap.check_data(ui, cl.index, data)
+nm_data = nodemap.persisted_data(cl)
+if nm_data is not None:
+docket, data = nm_data
+return nodemap.check_data(ui, cl.index, data)
 
 
 @command(



To: marmoute, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7840: nodemap: add a (python) index class for persistent nodemap testing

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19428.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7840?vs=19186=19428

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7840/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7840

AFFECTED FILES
  mercurial/configitems.py
  mercurial/localrepo.py
  mercurial/pure/parsers.py
  mercurial/revlog.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -8,6 +8,8 @@
   $ cat << EOF >> .hg/hgrc
   > [experimental]
   > exp-persistent-nodemap=yes
+  > [devel]
+  > persistent-nodemap=yes
   > EOF
   $ hg debugbuilddag .+5000
   $ f --size .hg/store/00changelog.n
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -352,6 +352,21 @@
 return p
 
 
+NodemapRevlogIO = None
+
+if util.safehasattr(parsers, 'parse_index_devel_nodemap'):
+
+class NodemapRevlogIO(revlogio):
+"""A debug oriented IO class that return a PersistentNodeMapIndexObject
+
+The PersistentNodeMapIndexObject object is meant to test the 
persistent nodemap feature.
+"""
+
+def parseindex(self, data, inline):
+index, cache = parsers.parse_index_devel_nodemap(data, inline)
+return index, cache
+
+
 class rustrevlogio(revlogio):
 def parseindex(self, data, inline):
 index, cache = super(rustrevlogio, self).parseindex(data, inline)
@@ -596,9 +611,17 @@
 
 self._storedeltachains = True
 
+devel_nodemap = (
+self.nodemap_file
+and opts.get(b'devel-force-nodemap', False)
+and NodemapRevlogIO is not None
+)
+
 self._io = revlogio()
 if self.version == REVLOGV0:
 self._io = revlogoldio()
+elif devel_nodemap:
+self._io = NodemapRevlogIO()
 elif rustrevlog is not None and self.opener.options.get(b'rust.index'):
 self._io = rustrevlogio()
 try:
diff --git a/mercurial/pure/parsers.py b/mercurial/pure/parsers.py
--- a/mercurial/pure/parsers.py
+++ b/mercurial/pure/parsers.py
@@ -141,6 +141,15 @@
 self._extra = self._extra[: i - self._lgt]
 
 
+class PersistentNodeMapIndexObject(IndexObject):
+"""a Debug oriented class to test persistent nodemap
+
+We need a simple python object to test API and higher level behavior. See
+the Rust implementation for  more serious usage. This should be used only
+through the dedicated `devel.persistent-nodemap` config.
+"""
+
+
 class InlinedIndexObject(BaseIndexObject):
 def __init__(self, data, inline=0):
 self._data = data
@@ -188,6 +197,12 @@
 return InlinedIndexObject(data, inline), (0, data)
 
 
+def parse_index_devel_nodemap(data, inline):
+"""like parse_index2, but alway return a PersistentNodeMapIndexObject
+"""
+return PersistentNodeMapIndexObject(data), None
+
+
 def parse_dirstate(dmap, copymap, st):
 parents = [st[:20], st[20:40]]
 # dereference fields so they will be local in loop
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -931,6 +931,8 @@
 options[b'rust.index'] = True
 if ui.configbool('experimental', 'exp-persistent-nodemap'):
 options[b'exp-persistent-nodemap'] = True
+if ui.configbool('devel', 'persistent-nodemap'):
+options[b'devel-force-nodemap'] = True
 
 return options
 
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -406,6 +406,9 @@
 b'devel', b'legacy.exchange', default=list,
 )
 coreconfigitem(
+b'devel', b'persistent-nodemap', default=False,
+)
+coreconfigitem(
 b'devel', b'servercafile', default=b'',
 )
 coreconfigitem(



To: marmoute, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7845: nodemap: add basic checking of the on disk nodemap content

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19429.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7845?vs=19299=19429

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7845/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7845

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/revlogutils/nodemap.py
  tests/test-completion.t
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -36,6 +36,9 @@
   00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
   00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
   00f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  $ hg debugnodemap --check
+  revision in index:   5001
+  revision in nodemap: 5001
 
 add a new commit
 
@@ -48,3 +51,6 @@
   .hg/store/00changelog.n: size=18
   $ f --sha256 .hg/store/00changelog-*.nd --size
   .hg/store/00changelog-.nd: size=245760, 
sha256=e6ee5d59afaab2cb1afae1077715be280578d29df508bd3dd9d74a994bc555e7 (glob)
+  $ hg debugnodemap --check
+  revision in index:   5002
+  revision in nodemap: 5002
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -290,7 +290,7 @@
   debugmanifestfulltextcache: clear, add
   debugmergestate: 
   debugnamecomplete: 
-  debugnodemap: dump-new, dump-disk
+  debugnodemap: dump-new, dump-disk, check
   debugobsolete: flags, record-parents, rev, exclusive, index, delete, date, 
user, template
   debugp1copies: rev
   debugp2copies: rev
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -341,3 +341,37 @@
 else:
 block[idx] = _transform_rev(v)
 return block
+
+
+# debug utility
+
+
+def check_data(ui, index, data):
+"""verify that the provided nodemap data are valid for the given idex"""
+ret = 0
+ui.status((b"revision in index:   %d\n") % len(index))
+root = parse_data(data)
+all_revs = set(_all_revisions(root))
+ui.status((b"revision in nodemap: %d\n") % len(all_revs))
+for r in range(len(index)):
+if r not in all_revs:
+msg = b"  revision missing from nodemap: %d\n" % r
+ui.write_err(msg)
+ret = 1
+else:
+all_revs.remove(r)
+if all_revs:
+for r in sorted(all_revs):
+msg = b"  extra revision in  nodemap: %d\n" % r
+ui.write_err(msg)
+ret = 1
+return ret
+
+
+def _all_revisions(root):
+"""return all revisions stored in a Trie"""
+for block in _walk_trie(root):
+for v in block:
+if v is None or isinstance(v, Block):
+continue
+yield v
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2087,6 +2087,12 @@
 _(b'write a (binary) serialised (new) nodemap on stdin'),
 ),
 ('', b'dump-disk', False, _(b'dump on-disk data on stdin')),
+(
+'',
+b'check',
+False,
+_(b'check that the data on disk data are correct.'),
+),
 ],
 )
 def debugnodemap(ui, repo, **opts):
@@ -2102,6 +2108,11 @@
 cl = unfi.changelog
 data = nodemap.persisted_data(cl)
 ui.write(data)
+elif opts['check']:
+unfi = repo.unfiltered()
+cl = unfi.changelog
+data = nodemap.persisted_data(cl)
+return nodemap.check_data(ui, cl.index, data)
 
 
 @command(



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7847: nodemap: provide the on disk data to indexes who support it

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19430.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7847?vs=19184=19430

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7847/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7847

AFFECTED FILES
  mercurial/pure/parsers.py
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -626,6 +626,16 @@
 self._io = rustrevlogio()
 try:
 d = self._io.parseindex(indexdata, self._inline)
+index, _chunkcache = d
+use_nodemap = (
+not self._inline
+and self.nodemap_file is not None
+and util.safehasattr(index, 'update_nodemap_data')
+)
+if use_nodemap:
+nodemap_data = nodemaputil.persisted_data(self)
+if nodemap_data is not None:
+index.update_nodemap_data(nodemap_data)
 except (ValueError, IndexError):
 raise error.RevlogError(
 _(b"index %s is corrupted") % self.indexfile
diff --git a/mercurial/pure/parsers.py b/mercurial/pure/parsers.py
--- a/mercurial/pure/parsers.py
+++ b/mercurial/pure/parsers.py
@@ -156,6 +156,14 @@
 index."""
 return nodemaputil.persistent_data(self)
 
+def update_nodemap_data(self, nm_data):
+"""provide full serialiazed data from a nodemap
+
+The data are expected to come from disk. See `nodemap_data_all` for a
+produceur of such data."""
+if nm_data is not None:
+nodemaputil.parse_data(nm_data)
+
 
 class InlinedIndexObject(BaseIndexObject):
 def __init__(self, data, inline=0):



To: marmoute, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7835: nodemap: write nodemap data on disk

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19425.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7835?vs=19185=19425

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7835/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7835

AFFECTED FILES
  mercurial/changelog.py
  mercurial/configitems.py
  mercurial/localrepo.py
  mercurial/revlog.py
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -5,8 +5,14 @@
 
   $ hg init test-repo
   $ cd test-repo
+  $ cat << EOF >> .hg/hgrc
+  > [experimental]
+  > exp-persistent-nodemap=yes
+  > EOF
   $ hg debugbuilddag .+5000
-  $ hg debugnodemap --dump | f --sha256 --bytes=256 --hexdump --size
+  $ hg debugnodemap --dump | f --sha256 --size
+  size=245760, 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c
+  $ f --sha256 --bytes=256 --hexdump --size < .hg/store/00changelog.n
   size=245760, 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c
   : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
   0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -21,6 +21,39 @@
 raise error.RevlogError(b'unknown node: %s' % x)
 
 
+def setup_persistent_nodemap(tr, revlog):
+"""Install whatever is needed transaction side to persist a nodemap on disk
+
+(only actually persist the nodemap if this is relevant for this revlog)
+"""
+if revlog.nodemap_file is None:
+return  # we do not use persistent_nodemap on this revlog
+callback_id = b"revlog-persistent-nodemap-%s" % revlog.nodemap_file
+if tr.hasfinalize(callback_id):
+return  # no need to register again
+tr.addfinalize(callback_id, lambda tr: _persist_nodemap(tr, revlog))
+
+
+def _persist_nodemap(tr, revlog):
+"""Write nodemap data on disk for a given revlog
+"""
+if getattr(revlog, 'filteredrevs', ()):
+raise error.ProgrammingError(
+"cannot persist nodemap of a filtered changelog"
+)
+if revlog.nodemap_file is None:
+msg = "calling persist nodemap on a revlog without the feature enableb"
+raise error.ProgrammingError(msg)
+data = persistent_data(revlog.index)
+# EXP-TODO: if this is a cache, this should use a cache vfs, not a
+# store vfs
+with revlog.opener(revlog.nodemap_file, 'w') as f:
+f.write(data)
+# EXP-TODO: if the transaction abort, we should remove the new data and
+# reinstall the old one. (This will be simpler when the file format get a
+# bit more advanced)
+
+
 ### Nodemap Trie
 #
 # This is a simple reference implementation to compute and serialise a nodemap
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -407,6 +407,7 @@
 mmaplargeindex=False,
 censorable=False,
 upperboundcomp=None,
+persistentnodemap=False,
 ):
 """
 create a revlog object
@@ -418,6 +419,10 @@
 self.upperboundcomp = upperboundcomp
 self.indexfile = indexfile
 self.datafile = datafile or (indexfile[:-2] + b".d")
+self.nodemap_file = None
+if persistentnodemap:
+self.nodemap_file = indexfile[:-2] + b".n"
+
 self.opener = opener
 #  When True, indexfile is opened with checkambig=True at writing, to
 #  avoid file stat ambiguity.
@@ -2286,6 +2291,7 @@
 ifh.write(data[0])
 ifh.write(data[1])
 self._enforceinlinesize(transaction, ifh)
+nodemaputil.setup_persistent_nodemap(transaction, self)
 
 def addgroup(self, deltas, linkmapper, transaction, addrevisioncb=None):
 """
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -929,6 +929,8 @@
 
 if ui.configbool(b'experimental', b'rust.index'):
 options[b'rust.index'] = True
+if ui.configbool('experimental', 'exp-persistent-nodemap'):
+options[b'exp-persistent-nodemap'] = True
 
 return options
 
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -660,6 +660,9 @@
 b'experimental', b'rust.index', default=False,
 )
 coreconfigitem(
+b'experimental', b'exp-persistent-nodemap', default=False,
+)
+coreconfigitem(
 b'experimental', b'server.filesdata.recommended-batch-size', default=5,
 )
 coreconfigitem(
diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -385,6 +385,9 @@
   

D7834: nodemap: have some python code serializing a nodemap

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19424.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7834?vs=19171=19424

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7834/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7834

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/revlogutils/nodemap.py
  tests/test-completion.t
  tests/test-help.t
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
new file mode 100644
--- /dev/null
+++ b/tests/test-persistent-nodemap.t
@@ -0,0 +1,26 @@
+===
+Test the persistent on-disk nodemap
+===
+
+
+  $ hg init test-repo
+  $ cd test-repo
+  $ hg debugbuilddag .+5000
+  $ hg debugnodemap --dump | f --sha256 --bytes=256 --hexdump --size
+  size=245760, 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c
+  : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0030: ff ff ff ff ff ff fa c2 ff ff ff ff ff ff ff ff ||
+  0040: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0050: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0060: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ed b3 ||
+  0070: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  0080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ee 34 |...4|
+  0090: 00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ||
+  00a0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  00b0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  00c0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
+  00f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -1017,6 +1017,7 @@
  print merge state
debugnamecomplete
  complete "names" - tags, open branch names, bookmark names
+   debugnodemap  write and inspect on disk nodemap
debugobsolete
  create arbitrary obsolete marker
debugoptADV   (no help text available)
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -107,6 +107,7 @@
   debugmanifestfulltextcache
   debugmergestate
   debugnamecomplete
+  debugnodemap
   debugobsolete
   debugp1copies
   debugp2copies
@@ -289,6 +290,7 @@
   debugmanifestfulltextcache: clear, add
   debugmergestate: 
   debugnamecomplete: 
+  debugnodemap: dump
   debugobsolete: flags, record-parents, rev, exclusive, index, delete, date, 
user, template
   debugp1copies: rev
   debugp2copies: rev
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -7,9 +7,156 @@
 # GNU General Public License version 2 or any later version.
 
 from __future__ import absolute_import
-from .. import error
+
+import struct
+
+from .. import (
+error,
+node as nodemod,
+)
 
 
 class NodeMap(dict):
 def __missing__(self, x):
 raise error.RevlogError(b'unknown node: %s' % x)
+
+
+### Nodemap Trie
+#
+# This is a simple reference implementation to compute and serialise a nodemap
+# trie. This reference implementation is write only. The python version of this
+# is not expected to be actually used, since it wont provide performance
+# improvement over existing non-persistent C implementation.
+#
+# The nodemap is serialized as Trie using 4bits-address/16-entries block. each
+# revision can be adressed using its node shortest prefix.
+#
+# The trie is stored as a sequence of block. Each block contains 16 entries
+# (signed 64bit integer, big endian). Each entry can be one of the following:
+#
+#  * value >=  0 -> index of sub-block
+#  * value == -1 -> no value
+#  * value <  -1 -> a revision value: rev = -(value+10)
+#
+# The implementation focus on simplicity, not on performance. A Rust
+# implementation should provide a efficient version of the same serialization.
+# This reference python implementation is never meant to be extensively use in
+# production.
+
+
+def persistent_data(index):
+"""return the serialised data of a nodemap for a given index
+"""
+trie = _build_trie(index)
+return _dump_trie(trie)
+
+
+S_BLOCK = struct.Struct(">" + ("q" * 16))
+
+NO_ENTRY = -1
+# rev-0 need to be -2 because 0 is used by block, -1 is a special value.
+REV_OFFSET = 2
+
+
+def 

D7836: nodemap: add a function to read the data from disk

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19426.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7836?vs=19173=19426

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7836/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7836

AFFECTED FILES
  mercurial/debugcommands.py
  mercurial/revlogutils/nodemap.py
  tests/test-completion.t
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -10,9 +10,9 @@
   > exp-persistent-nodemap=yes
   > EOF
   $ hg debugbuilddag .+5000
-  $ hg debugnodemap --dump | f --sha256 --size
+  $ hg debugnodemap --dump-new | f --sha256 --size
   size=245760, 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c
-  $ f --sha256 --bytes=256 --hexdump --size < .hg/store/00changelog.n
+  $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
   size=245760, 
sha256=bc400bf49f11e83bbd25630439feee6628a80a8602d2e38972eac44cc3efe10c
   : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
   0010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ||
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -290,7 +290,7 @@
   debugmanifestfulltextcache: clear, add
   debugmergestate: 
   debugnamecomplete: 
-  debugnodemap: dump
+  debugnodemap: dump-new, dump-disk
   debugobsolete: flags, record-parents, rev, exclusive, index, delete, date, 
user, template
   debugp1copies: rev
   debugp2copies: rev
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -21,6 +21,13 @@
 raise error.RevlogError(b'unknown node: %s' % x)
 
 
+def persisted_data(revlog):
+"""read the nodemap for a revlog from disk"""
+if revlog.nodemap_file is None:
+return None
+return revlog.opener.tryread(revlog.nodemap_file)
+
+
 def setup_persistent_nodemap(tr, revlog):
 """Install whatever is needed transaction side to persist a nodemap on disk
 
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2079,16 +2079,29 @@
 
 @command(
 b'debugnodemap',
-[('', b'dump', False, _(b'write a (binary) serialised nodemap on stdin'))],
+[
+(
+'',
+b'dump-new',
+False,
+_(b'write a (binary) serialised (new) nodemap on stdin'),
+),
+('', b'dump-disk', False, _(b'dump on-disk data on stdin')),
+],
 )
 def debugnodemap(ui, repo, **opts):
 """write and inspect on disk nodemap
 """
-if opts['dump']:
+if opts['dump_new']:
 unfi = repo.unfiltered()
 cl = unfi.changelog
 data = nodemap.persistent_data(cl.index)
 ui.write(data)
+elif opts['dump_disk']:
+unfi = repo.unfiltered()
+cl = unfi.changelog
+data = nodemap.persisted_data(cl)
+ui.write(data)
 
 
 @command(



To: marmoute, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7837: nodemap: only use persistent nodemap for non-inlined revlog

[marmoute (Pierre-Yves David)]

marmoute updated this revision to Diff 19427.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7837?vs=19174=19427

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7837/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7837

AFFECTED FILES
  mercurial/revlog.py
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -33,6 +33,8 @@
 
 (only actually persist the nodemap if this is relevant for this revlog)
 """
+if revlog._inline:
+return  # inlined revlog are too small for this to be relevant
 if revlog.nodemap_file is None:
 return  # we do not use persistent_nodemap on this revlog
 callback_id = b"revlog-persistent-nodemap-%s" % revlog.nodemap_file
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1965,6 +1965,7 @@
 # manager
 
 tr.replace(self.indexfile, trindex * self._io.size)
+nodemaputil.setup_persistent_nodemap(tr, self)
 self._chunkclear()
 
 def _nodeduplicatecallback(self, transaction, node):



To: marmoute, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7834: nodemap: have some python code serializing a nodemap

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  As stated yesterday, I'll have limited screen exposure time available for the 
next couple of day. I am going to ignore any minor nits during that time. (Am I 
happy to look at them afterward and follow up).
  
  Right now, I have only identified the `s/**args/**opts/` as non nits. I am 
sending an updated version that update that part. If any remains, please 
highlight any other major complains about this patch that is worth blocking the 
rest of the series.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7834/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7834

To: marmoute, #hg-reviewers, martinvonz
Cc: martinvonz, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7914: rust-matchers: implement `visit_children_set` for `FileMatcher`

[martinvonz (Martin von Zweigbergk)]

martinvonz added inline comments.

INLINE COMMENTS

> Alphare wrote in matchers.rs:166
> I don't feel like this is an issue of code quality. This is (probably) an 
> issue of performance, and I have a good incentive for making this code go 
> faster in the future. I don't think this is necessary for now since this is a 
> minor optimization compared to the other ones that are pending.

How about we just replace the implementation by 
`VisitChildrenSet::Recursive`then? That's easy to maintain and makes it 
functionally correct. It's usually fast enough too.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7914/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7914

To: Alphare, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7931: rust-status: use bare hg status fastpath from Python

[Raphaël Gomès]

Alphare created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This change also adds a test case for subrepos.
  Repeating the benchmark information from the `hg-core` commit:
  
  On the Netbeans repository:
  C: 840ms
  Rust+C: 556ms
  
  Mozilla Central with the one pattern that causes a fallback removed:
  C: 2.315s
  Rust+C: 1.700s

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7931

AFFECTED FILES
  mercurial/dirstate.py
  mercurial/match.py
  tests/test-subrepo-deep-nested-change.t

CHANGE DETAILS

diff --git a/tests/test-subrepo-deep-nested-change.t 
b/tests/test-subrepo-deep-nested-change.t
--- a/tests/test-subrepo-deep-nested-change.t
+++ b/tests/test-subrepo-deep-nested-change.t
@@ -355,6 +355,11 @@
   R sub1/sub2/folder/test.txt
   ! sub1/.hgsub
   ? sub1/x.hgsub
+  $ hg status -R sub1
+  warning: subrepo spec file 'sub1/.hgsub' not found
+  R .hgsubstate
+  ! .hgsub
+  ? x.hgsub
   $ mv sub1/x.hgsub sub1/.hgsub
   $ hg update -Cq
   $ touch sub1/foo
diff --git a/mercurial/match.py b/mercurial/match.py
--- a/mercurial/match.py
+++ b/mercurial/match.py
@@ -45,6 +45,7 @@
 
 propertycache = util.propertycache
 
+rustmod = policy.importrust('dirstate')
 
 def _rematcher(regex):
 '''compile the regexp with the best available regexp engine and return a
@@ -666,7 +667,10 @@
 class includematcher(basematcher):
 def __init__(self, root, kindpats, badfn=None):
 super(includematcher, self).__init__(badfn)
-
+if rustmod is not None:
+# We need to pass the patterns to Rust because they can contain
+# patterns from the user interface
+self._kindpats = kindpats
 self._pats, self.matchfn = _buildmatch(kindpats, b'(?:/|$)', root)
 self._prefix = _prefix(kindpats)
 roots, dirs, parents = _rootsdirsandparents(kindpats)
diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -27,6 +27,7 @@
 policy,
 pycompat,
 scmutil,
+sparse,
 txnutil,
 util,
 )
@@ -1083,7 +1084,7 @@
 results[next(iv)] = st
 return results
 
-def _rust_status(self, matcher, list_clean):
+def _rust_status(self, matcher, list_clean, list_ignored, list_unknown):
 # Force Rayon (Rust parallelism library) to respect the number of
 # workers. This is a temporary workaround until Rust code knows
 # how to read the config file.
@@ -1101,16 +1102,45 @@
 added,
 removed,
 deleted,
+clean,
+ignored,
 unknown,
-clean,
+warnings,
+bad,
 ) = rustmod.status(
 self._map._rustmap,
 matcher,
 self._rootdir,
-bool(list_clean),
+self._ignorefiles(),
+self._checkexec,
 self._lastnormaltime,
-self._checkexec,
+bool(list_clean),
+bool(list_ignored),
+bool(list_unknown),
 )
+if self._ui.warn:
+for item in warnings:
+if isinstance(item, tuple):
+file_path, syntax = item
+msg = _(b"%s: ignoring invalid syntax '%s'\n") % (
+file_path,
+syntax,
+)
+self._ui.warn(msg)
+else:
+msg = _(b"skipping unreadable pattern file '%s': %s\n")
+self._ui.warn(
+msg
+% (
+pathutil.canonpath(
+self._rootdir, self._rootdir, item
+),
+b"No such file or directory",
+)
+)
+
+for (fn, message) in bad:
+matcher.bad(fn, encoding.strtolocal(message))
 
 status = scmutil.status(
 modified=modified,
@@ -1118,7 +1148,7 @@
 removed=removed,
 deleted=deleted,
 unknown=unknown,
-ignored=[],
+ignored=ignored,
 clean=clean,
 )
 return (lookup, status)
@@ -1148,26 +1178,32 @@
 
 use_rust = True
 
-allowed_matchers = (matchmod.alwaysmatcher, matchmod.exactmatcher)
+allowed_matchers = (
+matchmod.alwaysmatcher,
+matchmod.exactmatcher,
+matchmod.includematcher,
+)
 
 if rustmod is None:
 use_rust = False
 elif subrepos:
 use_rust = False
-elif bool(listunknown):
-# Pathauditor does not exist yet in Rust, unknown files
-# can't be trusted.
+elif sparse.enabled:
 use_rust = False
-elif self._ignorefiles() 

D7930: rust-status: update rust-cpython bridge to account for the changes in core

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7930

AFFECTED FILES
  rust/hg-cpython/src/dirstate.rs
  rust/hg-cpython/src/dirstate/status.rs
  rust/hg-cpython/src/exceptions.rs

CHANGE DETAILS

diff --git a/rust/hg-cpython/src/exceptions.rs 
b/rust/hg-cpython/src/exceptions.rs
--- a/rust/hg-cpython/src/exceptions.rs
+++ b/rust/hg-cpython/src/exceptions.rs
@@ -40,5 +40,6 @@
 }
 
 py_exception!(rustext, HgPathPyError, RuntimeError);
+py_exception!(rustext, FallbackError, RuntimeError);
 
 py_exception!(shared_ref, AlreadyBorrowed, RuntimeError);
diff --git a/rust/hg-cpython/src/dirstate/status.rs 
b/rust/hg-cpython/src/dirstate/status.rs
--- a/rust/hg-cpython/src/dirstate/status.rs
+++ b/rust/hg-cpython/src/dirstate/status.rs
@@ -9,26 +9,28 @@
 //! `hg-core` crate. From Python, this will be seen as
 //! `rustext.dirstate.status`.
 
-use crate::dirstate::DirstateMap;
-use cpython::exc::ValueError;
+use crate::{dirstate::DirstateMap, exceptions::FallbackError};
 use cpython::{
-ObjectProtocol, PyBytes, PyErr, PyList, PyObject, PyResult, PyTuple,
-Python, PythonObject, ToPyObject,
+exc::ValueError, ObjectProtocol, PyBytes, PyErr, PyList, PyObject,
+PyResult, PyTuple, Python, PythonObject, ToPyObject,
 };
-use hg::utils::hg_path::HgPathBuf;
 use hg::{
-matchers::{AlwaysMatcher, FileMatcher},
-status,
-utils::{files::get_path_from_bytes, hg_path::HgPath},
-DirstateStatus,
+matchers::{AlwaysMatcher, FileMatcher, IncludeMatcher},
+parse_pattern_syntax, status,
+utils::{
+files::{get_bytes_from_path, get_path_from_bytes},
+hg_path::{HgPath, HgPathBuf},
+},
+BadMatch, BadType, DirstateStatus, IgnorePattern, PatternFileWarning,
+StatusError, StatusOptions,
 };
-use std::borrow::Borrow;
+use std::borrow::{Borrow, Cow};
 
 /// This will be useless once trait impls for collection are added to `PyBytes`
 /// upstream.
-fn collect_pybytes_list>(
+fn collect_pybytes_list(
 py: Python,
-collection: &[P],
+collection: &[impl AsRef],
 ) -> PyList {
 let list = PyList::new(py, &[]);
 
@@ -43,34 +45,105 @@
 list
 }
 
+fn collect_bad_matches(
+py: Python,
+collection: &[(impl AsRef, BadMatch)],
+) -> PyResult {
+let list = PyList::new(py, &[]);
+
+let os = py.import("os")?;
+let get_error_message = |code: i32| -> PyResult<_> {
+os.call(
+py,
+"strerror",
+PyTuple::new(py, &[code.to_py_object(py).into_object()]),
+None,
+)
+};
+
+for (i, (path, bad_match)) in collection.iter().enumerate() {
+let message = match bad_match {
+BadMatch::OsError(code) => get_error_message(*code)?,
+BadMatch::BadType(bad_type) => format!(
+"unsupported file type (type is {})",
+match bad_type {
+BadType::CharacterDevice => "character device",
+BadType::BlockDevice => "block device",
+BadType::FIFO => "fifo",
+BadType::Socket => "socket",
+BadType::Directory => "directory",
+BadType::Unknown => "unknown",
+}
+)
+.to_py_object(py)
+.into_object(),
+};
+list.insert_item(
+py,
+i,
+(PyBytes::new(py, path.as_ref().as_bytes()), message)
+.to_py_object(py)
+.into_object(),
+)
+}
+
+Ok(list)
+}
+
+fn handle_fallback(py: Python, err: StatusError) -> PyErr {
+match err {
+StatusError::Pattern(e) => {
+PyErr::new::(py, e.to_string())
+}
+e => PyErr::new::(py, e.to_string()),
+}
+}
+
 pub fn status_wrapper(
 py: Python,
 dmap: DirstateMap,
 matcher: PyObject,
 root_dir: PyObject,
-list_clean: bool,
+ignore_files: PyList,
+check_exec: bool,
 last_normal_time: i64,
-check_exec: bool,
-) -> PyResult<(PyList, PyList, PyList, PyList, PyList, PyList, PyList)> {
+list_clean: bool,
+list_ignored: bool,
+list_unknown: bool,
+) -> PyResult {
 let bytes = root_dir.extract::(py)?;
 let root_dir = get_path_from_bytes(bytes.data(py));
 
 let dmap: DirstateMap = dmap.to_py_object(py);
 let dmap = dmap.get_inner(py);
 
+let ignore_files: PyResult> = ignore_files
+.iter(py)
+.map(|b| {
+let file = b.extract::(py)?;
+Ok(get_path_from_bytes(file.data(py)).to_owned())
+})
+.collect();
+let ignore_files = ignore_files?;
+
 match matcher.get_type(py).name(py).borrow() {
 "alwaysmatcher" => {
 let matcher = AlwaysMatcher;
-let (lookup, status_res) = status(
+let ((lookup, status_res), 

D7929: rust-status: add bare `hg status` support in hg-core

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  A lot of performance remains to be gained, most notably by doing more things
  in parallel, but also by caching, not falling back to Python but switching
  to another regex engine, etc..
  
  I have measured on multiple repositories that this change, when in combination
  with the next two patches, improve bare `hg status` performance, and has no
  observable impact when falling back (because it does so early).
  
  On the Netbeans repository:
  C: 840ms
  Rust+C: 556ms
  
  Mozilla Central with the one pattern that causes a fallback removed:
  C: 2.315s
  Rust+C: 1.700 s

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7929

AFFECTED FILES
  rust/hg-core/src/dirstate/status.rs
  rust/hg-core/src/lib.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/lib.rs b/rust/hg-core/src/lib.rs
--- a/rust/hg-core/src/lib.rs
+++ b/rust/hg-core/src/lib.rs
@@ -13,7 +13,9 @@
 dirs_multiset::{DirsMultiset, DirsMultisetIter},
 dirstate_map::DirstateMap,
 parsers::{pack_dirstate, parse_dirstate, PARENT_SIZE},
-status::{status, DirstateStatus},
+status::{
+status, BadMatch, BadType, DirstateStatus, StatusError, StatusOptions,
+},
 CopyMap, CopyMapIter, DirstateEntry, DirstateParents, EntryState,
 StateMap, StateMapIter,
 };
diff --git a/rust/hg-core/src/dirstate/status.rs 
b/rust/hg-core/src/dirstate/status.rs
--- a/rust/hg-core/src/dirstate/status.rs
+++ b/rust/hg-core/src/dirstate/status.rs
@@ -11,22 +11,31 @@
 
 use crate::{
 dirstate::SIZE_FROM_OTHER_PARENT,
-matchers::{Matcher, VisitChildrenSet},
+filepatterns::PatternFileWarning,
+matchers::{get_ignore_function, Matcher, VisitChildrenSet},
 utils::{
-files::HgMetadata,
+files::{find_dirs, HgMetadata},
 hg_path::{
 hg_path_to_path_buf, os_string_to_hg_path_buf, HgPath, HgPathBuf,
+HgPathError,
 },
+path_auditor::PathAuditor,
 },
 CopyMap, DirstateEntry, DirstateMap, EntryState, FastHashMap,
+PatternError,
 };
+use lazy_static::lazy_static;
 use rayon::prelude::*;
-use std::borrow::Cow;
-use std::collections::{HashSet, VecDeque};
-use std::fs::{read_dir, DirEntry};
-use std::io::ErrorKind;
-use std::ops::Deref;
-use std::path::Path;
+use std::collections::VecDeque;
+use std::{
+borrow::Cow,
+collections::HashSet,
+fs::{read_dir, DirEntry},
+io::ErrorKind,
+ops::Deref,
+path::Path,
+sync::mpsc,
+};
 
 #[derive(Debug)]
 pub enum BadType {
@@ -47,6 +56,7 @@
 /// Marker enum used to dispatch new status entries into the right collections.
 /// Is similar to `crate::EntryState`, but represents the transient state of
 /// entries during the lifetime of a command.
+#[derive(Debug)]
 enum Dispatch {
 Unsure,
 Modified,
@@ -145,7 +155,7 @@
 } else if options.list_clean {
 Dispatch::Clean
 } else {
-Dispatch::Unknown
+Dispatch::None
 }
 }
 EntryState::Merged => Dispatch::Modified,
@@ -169,57 +179,91 @@
 }
 }
 
+lazy_static! {
+static ref DEFAULT_WORK: HashSet<&'static HgPath> = {
+let mut h = HashSet::new();
+h.insert(HgPath::new(b""));
+h
+};
+}
+
 /// Get stat data about the files explicitly specified by match.
 /// TODO subrepos
 fn walk_explicit<'a>(
-files: &'a HashSet<>,
+files: Option<&'a HashSet<>>,
 dmap: &'a DirstateMap,
 root_dir: impl AsRef + Sync + Send,
+work: mpsc::Sender<&'a HgPath>,
 options: StatusOptions,
-) -> impl ParallelIterator> {
-files.par_iter().filter_map(move |filename| {
-// TODO normalization
-let normalized = filename.as_ref();
+) -> impl ParallelIterator, Dispatch)>> {
+files
+.unwrap_or(_WORK)
+.par_iter()
+.map_with(work, move |work, filename| {
+// TODO normalization
+let normalized = filename.as_ref();
 
-let buf = match hg_path_to_path_buf(normalized) {
-Ok(x) => x,
-Err(e) => return Some(Err(e.into())),
-};
-let target = root_dir.as_ref().join(buf);
-let st = target.symlink_metadata();
-match st {
-Ok(meta) => {
-let file_type = meta.file_type();
-if file_type.is_file() || file_type.is_symlink() {
-if let Some(entry) = dmap.get(normalized) {
+let buf = match hg_path_to_path_buf(normalized) {
+Ok(x) => x,
+Err(e) => return Some(Err(e.into())),
+};
+let target = root_dir.as_ref().join(buf);
+let st = target.symlink_metadata();
+let in_dmap = dmap.get(normalized);
+match st {
+Ok(meta) => {
+let 

D7925: rust-matchers: add `IgnoreMatcher`

[Raphaël Gomès]

Alphare updated this revision to Diff 19420.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7925?vs=19404=19420

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7925/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7925

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,14 +10,25 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::{build_single_regex, PatternResult},
-utils::hg_path::{HgPath, HgPathBuf},
-DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
+dirstate::dirs_multiset::DirsChildrenMultiset,
+filepatterns::{
+build_single_regex, filter_subincludes, get_patterns_from_file,
+PatternFileWarning, PatternResult, SubInclude,
+},
+utils::{
+files::find_dirs,
+hg_path::{HgPath, HgPathBuf},
+Escaped,
+},
+DirsMultiset, DirstateMapError, FastHashMap, IgnorePattern, PatternError,
 PatternSyntax,
 };
+
 use std::collections::HashSet;
+use std::fmt::{Display, Error, Formatter};
 use std::iter::FromIterator;
 use std::ops::Deref;
+use std::path::Path;
 
 #[derive(Debug, PartialEq)]
 pub enum VisitChildrenSet<'a> {
@@ -223,6 +234,88 @@
 }
 }
 
+/// Matches files that are included in the ignore rules.
+///
+#[cfg_attr(
+feature = "with-re2",
+doc = r##"
+```
+use hg::{
+matchers::{IncludeMatcher, Matcher},
+IgnorePattern,
+PatternSyntax,
+utils::hg_path::HgPath
+};
+use std::path::Path;
+///
+let ignore_patterns =
+vec![IgnorePattern::new(PatternSyntax::RootGlob, b"this*", Path::new(""))];
+let (matcher, _) = IncludeMatcher::new(ignore_patterns, "").unwrap();
+///
+assert_eq!(matcher.matches(HgPath::new(b"testing")), false);
+assert_eq!(matcher.matches(HgPath::new(b"this should work")), true);
+assert_eq!(matcher.matches(HgPath::new(b"this also")), true);
+assert_eq!(matcher.matches(HgPath::new(b"but not this")), false);
+```
+"##
+)]
+pub struct IncludeMatcher<'a> {
+patterns: Vec,
+match_fn: Box Fn(&'r HgPath) -> bool + 'a + Sync>,
+/// Whether all the patterns match a prefix (i.e. recursively)
+prefix: bool,
+roots: HashSet,
+dirs: HashSet,
+parents: HashSet,
+}
+
+impl<'a> Matcher for IncludeMatcher<'a> {
+fn file_set() -> Option<<>> {
+None
+}
+
+fn exact_match(, _filename: impl AsRef) -> bool {
+false
+}
+
+fn matches(, filename: impl AsRef) -> bool {
+(self.match_fn)(filename.as_ref())
+}
+
+fn visit_children_set(
+,
+directory: impl AsRef,
+) -> VisitChildrenSet {
+let dir = directory.as_ref();
+if self.prefix && self.roots.contains(dir) {
+return VisitChildrenSet::Recursive;
+}
+if self.roots.contains(HgPath::new(b""))
+|| self.roots.contains(dir)
+|| self.dirs.contains(dir)
+|| find_dirs(dir).any(|parent_dir| self.roots.contains(parent_dir))
+{
+return VisitChildrenSet::This;
+}
+
+if self.parents.contains(directory.as_ref()) {
+let multiset = self.get_all_parents_children();
+if let Some(children) = multiset.get(dir) {
+return VisitChildrenSet::Set(children.to_owned());
+}
+}
+VisitChildrenSet::Empty
+}
+
+fn matches_everything() -> bool {
+false
+}
+
+fn is_exact() -> bool {
+false
+}
+}
+
 const MAX_RE_SIZE: usize = 2;
 
 #[cfg(feature = "with-re2")]
@@ -395,6 +488,175 @@
 })
 }
 
+/// Returns a function that checks whether a given file (in the general sense)
+/// should be matched.
+fn build_match<'a, 'b>(
+ignore_patterns: &'a [IgnorePattern],
+root_dir: impl AsRef,
+) -> PatternResult<(
+Vec,
+Box bool + 'b + Sync>,
+Vec,
+)> {
+let mut match_funcs: Vec bool + Sync>> = vec![];
+// For debugging and printing
+let mut patterns = vec![];
+let mut all_warnings = vec![];
+
+let (subincludes, ignore_patterns) =
+filter_subincludes(ignore_patterns, root_dir)?;
+
+if !subincludes.is_empty() {
+// Build prefix-based matcher functions for subincludes
+let mut submatchers = FastHashMap::default();
+let mut prefixes = vec![];
+
+for SubInclude { prefix, root, path } in subincludes.into_iter() {
+let (match_fn, warnings) = get_ignore_function(&[path], root)?;
+all_warnings.extend(warnings);
+prefixes.push(prefix.to_owned());
+submatchers.insert(prefix.to_owned(), match_fn);
+}
+
+let match_subinclude = move |filename: | {
+for prefix in prefixes.iter() {
+if let Some(rel) = filename.relative_to(prefix) {
+if 

D7924: rust-matchers: add `build_regex_match` function

[Raphaël Gomès]

Alphare updated this revision to Diff 19419.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7924?vs=19403=19419

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7924/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7924

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,7 +10,7 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::PatternResult,
+filepatterns::{build_single_regex, PatternResult},
 utils::hg_path::{HgPath, HgPathBuf},
 DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
 PatternSyntax,
@@ -244,6 +244,57 @@
 Err(PatternError::Re2NotInstalled)
 }
 
+/// Returns a function that matches an `HgPath` against the regex formed by
+/// the given patterns.
+fn build_regex_match<'a>(
+ignore_patterns: &'a [&'a IgnorePattern],
+) -> PatternResult<(Vec, Box bool + Sync>)> {
+let mut all_groups = vec![];
+let regexps: Result, PatternError> = ignore_patterns
+.into_iter()
+.map(|k| build_single_regex(*k))
+.collect();
+let regexps = regexps?;
+let full_regex = regexps.join('|');
+
+let mut start_index = 0;
+let mut group_size = 0;
+
+for (index, re) in regexps.iter().enumerate() {
+let piece_size = re.len();
+if piece_size > MAX_RE_SIZE {
+return Err(PatternError::TooLong(piece_size));
+}
+if (group_size + piece_size) > MAX_RE_SIZE {
+let group = [start_index..index];
+all_groups.push(group.join('|'));
+start_index = index;
+group_size = 0
+}
+group_size += piece_size + 1;
+}
+
+let func = if start_index == 0 {
+let matcher = re_matcher(_regex)?;
+Box::new(move |filename: | matcher(filename))
+as Box bool + Sync>
+} else {
+let group = [start_index..];
+all_groups.push(group.join('|'));
+
+let all_matchers: PatternResult> =
+all_groups.iter().map(|group| re_matcher(group)).collect();
+let all_matchers = all_matchers?;
+
+let ret = Box::new(move |filename: | {
+all_matchers.iter().any(|f| f(filename))
+});
+ret as Box bool + Sync>
+};
+
+Ok((full_regex, func))
+}
+
 /// Returns roots and directories corresponding to each pattern.
 ///
 /// This calculates the roots and directories exactly matching the patterns and



To: Alphare, #hg-reviewers
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7923: rust-matchers: add functions to get roots, dirs and parents from patterns

[Raphaël Gomès]

Alphare updated this revision to Diff 19418.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7923?vs=19402=19418

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7923/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7923

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,8 +10,10 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::PatternResult, utils::hg_path::HgPath, DirsMultiset,
-DirstateMapError, PatternError,
+filepatterns::PatternResult,
+utils::hg_path::{HgPath, HgPathBuf},
+DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
+PatternSyntax,
 };
 use std::collections::HashSet;
 use std::iter::FromIterator;
@@ -242,10 +244,155 @@
 Err(PatternError::Re2NotInstalled)
 }
 
+/// Returns roots and directories corresponding to each pattern.
+///
+/// This calculates the roots and directories exactly matching the patterns and
+/// returns a tuple of (roots, dirs). It does not return other directories
+/// which may also need to be considered, like the parent directories.
+fn roots_and_dirs(
+ignore_patterns: &[IgnorePattern],
+) -> (Vec, Vec) {
+let mut roots = Vec::new();
+let mut dirs = Vec::new();
+
+for ignore_pattern in ignore_patterns {
+let IgnorePattern {
+syntax, pattern, ..
+} = ignore_pattern;
+match syntax {
+PatternSyntax::RootGlob | PatternSyntax::Glob => {
+let mut root = vec![];
+
+for p in pattern.split(|c| *c == b'/') {
+if p.iter().any(|c| match *c {
+b'[' | b'{' | b'*' | b'?' => true,
+_ => false,
+}) {
+break;
+}
+root.push(HgPathBuf::from_bytes(p));
+}
+let buf =
+root.iter().fold(HgPathBuf::new(), |acc, r| acc.join(r));
+roots.push(buf);
+}
+PatternSyntax::Path | PatternSyntax::RelPath => {
+let pat = HgPath::new(if pattern == b"." {
+&[] as &[u8]
+} else {
+pattern
+});
+roots.push(pat.to_owned());
+}
+PatternSyntax::RootFiles => {
+let pat = if pattern == b"." {
+&[] as &[u8]
+} else {
+pattern
+};
+dirs.push(HgPathBuf::from_bytes(pat));
+}
+_ => {
+roots.push(HgPathBuf::new());
+}
+}
+}
+(roots, dirs)
+}
+
+/// Paths extracted from patterns
+struct RootsDirsAndParents {
+/// Directories to match recursively
+pub roots: HashSet,
+/// Directories to match non-recursively
+pub dirs: HashSet,
+/// Implicitly required directories to go to items in either roots or dirs
+pub parents: HashSet,
+}
+
+/// Extract roots, dirs and parents from patterns.
+fn roots_dirs_and_parents(
+ignore_patterns: &[IgnorePattern],
+) -> PatternResult {
+let (roots, dirs) = roots_and_dirs(ignore_patterns);
+
+let mut parents = HashSet::new();
+
+parents.extend(
+DirsMultiset::from_manifest()
+.map_err(|e| match e {
+DirstateMapError::InvalidPath(e) => e,
+_ => unreachable!(),
+})?
+.iter()
+.map(|k| k.to_owned()),
+);
+parents.extend(
+DirsMultiset::from_manifest()
+.map_err(|e| match e {
+DirstateMapError::InvalidPath(e) => e,
+_ => unreachable!(),
+})?
+.iter()
+.map(|k| k.to_owned()),
+);
+
+Ok(RootsDirsAndParents {
+roots: HashSet::from_iter(roots),
+dirs: HashSet::from_iter(dirs),
+parents,
+})
+}
+
 #[cfg(test)]
 mod tests {
 use super::*;
 use pretty_assertions::assert_eq;
+use std::path::Path;
+
+#[test]
+fn test_roots_and_dirs() {
+let pats = vec![
+IgnorePattern::new(PatternSyntax::Glob, b"g/h/*", Path::new("")),
+IgnorePattern::new(PatternSyntax::Glob, b"g/h", Path::new("")),
+IgnorePattern::new(PatternSyntax::Glob, b"g*", Path::new("")),
+];
+let (roots, dirs) = roots_and_dirs();
+
+assert_eq!(
+roots,
+vec!(
+HgPathBuf::from_bytes(b"g/h"),
+HgPathBuf::from_bytes(b"g/h"),
+HgPathBuf::new()
+),
+);
+assert_eq!(dirs, vec!());
+}
+
+#[test]
+fn test_roots_dirs_and_parents() {
+let pats = vec![
+

D7914: rust-matchers: implement `visit_children_set` for `FileMatcher`

[Raphaël Gomès]

Alphare added inline comments.

INLINE COMMENTS

> martinvonz wrote in matchers.rs:166
> Yes, code quality is never urgent. The problem is that there isn't much 
> incentive for people to clean it up after. So I'd really prefer to not queue 
> this without the cleanup.

I don't feel like this is an issue of code quality. This is (probably) an issue 
of performance, and I have a good incentive for making this code go faster in 
the future. I don't think this is necessary for now since this is a minor 
optimization compared to the other ones that are pending.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7914/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7914

To: Alphare, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7894: nodemap: introduce an option to use mmap to read the nodemap mapping

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  In D7894#116540 , @martinvonz 
wrote:
  
  > In D7894#116407 , @marmoute 
wrote:
  >
  >> In D7894#116397 , @martinvonz 
wrote:
  >>
  >>> How much does this patch help performance?
  >>> I would also like to see performance numbers (even just rough ones) for 
the Rust version. Sorry about a possibly stupid question, but why will this 
on-disk nodemap be faster than building it from the index? Is it that the file 
is smaller and thus faster to read? Or is it more the building of the tree than 
the reading that's slow? You mentioned you use some private repo for testing 
this. How large is the `00changelog.n` file in that repo and how large is 
`00changelog.i`?
  >>
  >> This save hundreds of milli second at initialization of large 
repositories. The repositories we are looking at are about 2 millions 
revisions. (but this will help smaller repository too). Mozilla try is a public 
repository in that range. It 00changelog.i is 103MB
  >
  > And 00changelog.n?
  
  
  
106973952 bytes for changelog.i
 83123200 bytes for the nodemap rawfiles
  
  
  
  >> The information in the .i files is just a flat list of node. So anything 
that need a mapping needs to build it. Building a mapping for millions of 
revision is slow. (I think Georges mentionned 300ms to build the mozilla-try 
nodemap). The nodemap we write on disk is directly usage as such. So we just 
need to mmap the files (mostly instant is the repository have been busy 
recently, eg: on server) and directly query the data from disk.
  >
  > Ah, that's what I was wondering. I was wondering while reviewing this 
series if your plan was to lazily from disk but I didn't see any mention of 
that. I guess this mmap business should have been that a hint :)
  
  Yes, so everythign we are doing is not so really "serialization" since we 
never actualy "deserialize" it in practice.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7894/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7894

To: marmoute, #hg-reviewers
Cc: gracinet, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7926: rust-dependencies: update rayon

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  I have no problem with the patch itself, but could you add a few words about 
the motivation? Even if that's just "no particular reason, I just like us to 
stay up to date".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7926/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7926

To: Alphare, #hg-reviewers
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7894: nodemap: introduce an option to use mmap to read the nodemap mapping

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In D7894#116407 , @marmoute 
wrote:
  
  > In D7894#116397 , @martinvonz 
wrote:
  >
  >> How much does this patch help performance?
  >> I would also like to see performance numbers (even just rough ones) for 
the Rust version. Sorry about a possibly stupid question, but why will this 
on-disk nodemap be faster than building it from the index? Is it that the file 
is smaller and thus faster to read? Or is it more the building of the tree than 
the reading that's slow? You mentioned you use some private repo for testing 
this. How large is the `00changelog.n` file in that repo and how large is 
`00changelog.i`?
  >
  > This save hundreds of milli second at initialization of large repositories. 
The repositories we are looking at are about 2 millions revisions. (but this 
will help smaller repository too). Mozilla try is a public repository in that 
range. It 00changelog.i is 103MB
  
  And 00changelog.n?
  
  > The information in the .i files is just a flat list of node. So anything 
that need a mapping needs to build it. Building a mapping for millions of 
revision is slow. (I think Georges mentionned 300ms to build the mozilla-try 
nodemap). The nodemap we write on disk is directly usage as such. So we just 
need to mmap the files (mostly instant is the repository have been busy 
recently, eg: on server) and directly query the data from disk.
  
  Ah, that's what I was wondering. I was wondering while reviewing this series 
if your plan was to lazily from disk but I didn't see any mention of that. I 
guess this mmap business should have been that a hint :)

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7894/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7894

To: marmoute, #hg-reviewers
Cc: gracinet, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44100: 2 new changesets

[Mercurial Commits]

2 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/b4420cea45e8
changeset:   44099:b4420cea45e8
user:Matt Harbison 
date:Wed Jan 15 23:34:04 2020 -0500
summary: sha1dc: avoid including the nonexistent stdint.h with Visual 
Studio 2008

https://www.mercurial-scm.org/repo/hg/rev/969527ac7b44
changeset:   44100:969527ac7b44
bookmark:@
tag: tip
user:Kyle Lippincott 
date:Thu Jan 16 12:27:15 2020 -0800
summary: cext: fix compiler warning about sign changing

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7914: rust-matchers: implement `visit_children_set` for `FileMatcher`

[martinvonz (Martin von Zweigbergk)]

martinvonz added inline comments.

INLINE COMMENTS

> Alphare wrote in matchers.rs:166
> Agreed. In much of this series there exist opportunities for caching/making 
> things run in parallel, etc. With the freeze approaching really fast, I 
> prefer to prioritize getting correct - albeit sub-optimal - code in rather 
> than risking missing the deadline. 
> My benchmarks of the entire series show an improvement in bare `hg status`  
> in all supported repositories (read: that don't have back-references in their 
> `.hgignore patterns), and no measurable slowdown in others.

Yes, code quality is never urgent. The problem is that there isn't much 
incentive for people to clean it up after. So I'd really prefer to not queue 
this without the cleanup.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7914/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7914

To: Alphare, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7923: rust-matchers: add functions to get roots, dirs and parents from patterns

[kevincox (Kevin Cox)]

This revision now requires changes to proceed.
kevincox added inline comments.
kevincox requested changes to this revision.

INLINE COMMENTS

> matchers.rs:250
> +/// This calculates the roots and directories exactly matching the patterns 
> and
> +/// returns a tuple of (roots, dirs) for each. It does not return other
> +/// directories which may also need to be considered, like the parent

I'm confused by the "for each" since this only returns one tuple.

> matchers.rs:311
> +ignore_patterns: &[IgnorePattern],
> +) -> PatternResult<(HashSet, HashSet, 
> HashSet)>
> +{

Please describe the return value.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7923/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7923

To: Alphare, #hg-reviewers, kevincox
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7834: nodemap: have some python code serializing a nodemap

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In D7834#116533 , @marmoute 
wrote:
  
  > Can you clarify which change you are actually requesting (I see mostly 
nits).
  > So far I see the `**args` → `**opts` change but I might be missing others.
  
  I'd prefer if you go through them all.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7834/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7834

To: marmoute, #hg-reviewers, martinvonz
Cc: martinvonz, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7834: nodemap: have some python code serializing a nodemap

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  Can you clarify which change you are actually requesting (I see mostly nits).
  
  So far I see the `**args` → `**opts` change but I might be missing others.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7834/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7834

To: marmoute, #hg-reviewers, martinvonz
Cc: martinvonz, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7928: rust-status: add function for sequential traversal of the working directory

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This change also introduces helper structs to make things clearer.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7928

AFFECTED FILES
  rust/hg-core/src/dirstate/status.rs
  rust/hg-core/src/lib.rs
  rust/hg-cpython/src/dirstate/status.rs

CHANGE DETAILS

diff --git a/rust/hg-cpython/src/dirstate/status.rs 
b/rust/hg-cpython/src/dirstate/status.rs
--- a/rust/hg-cpython/src/dirstate/status.rs
+++ b/rust/hg-cpython/src/dirstate/status.rs
@@ -20,7 +20,7 @@
 matchers::{AlwaysMatcher, FileMatcher},
 status,
 utils::{files::get_path_from_bytes, hg_path::HgPath},
-StatusResult,
+DirstateStatus,
 };
 use std::borrow::Borrow;
 
@@ -114,7 +114,7 @@
 
 fn build_response(
 lookup: Vec<>,
-status_res: StatusResult,
+status_res: DirstateStatus,
 py: Python,
 ) -> PyResult<(PyList, PyList, PyList, PyList, PyList, PyList, PyList)> {
 let modified = collect_pybytes_list(py, status_res.modified.as_ref());
diff --git a/rust/hg-core/src/lib.rs b/rust/hg-core/src/lib.rs
--- a/rust/hg-core/src/lib.rs
+++ b/rust/hg-core/src/lib.rs
@@ -13,7 +13,7 @@
 dirs_multiset::{DirsMultiset, DirsMultisetIter},
 dirstate_map::DirstateMap,
 parsers::{pack_dirstate, parse_dirstate, PARENT_SIZE},
-status::{status, StatusResult},
+status::{status, DirstateStatus},
 CopyMap, CopyMapIter, DirstateEntry, DirstateParents, EntryState,
 StateMap, StateMapIter,
 };
diff --git a/rust/hg-core/src/dirstate/status.rs 
b/rust/hg-core/src/dirstate/status.rs
--- a/rust/hg-core/src/dirstate/status.rs
+++ b/rust/hg-core/src/dirstate/status.rs
@@ -11,20 +11,39 @@
 
 use crate::{
 dirstate::SIZE_FROM_OTHER_PARENT,
-matchers::Matcher,
+matchers::{Matcher, VisitChildrenSet},
 utils::{
 files::HgMetadata,
 hg_path::{
 hg_path_to_path_buf, os_string_to_hg_path_buf, HgPath, HgPathBuf,
 },
 },
-CopyMap, DirstateEntry, DirstateMap, EntryState,
+CopyMap, DirstateEntry, DirstateMap, EntryState, FastHashMap,
 };
 use rayon::prelude::*;
-use std::collections::HashSet;
+use std::borrow::Cow;
+use std::collections::{HashSet, VecDeque};
 use std::fs::{read_dir, DirEntry};
+use std::io::ErrorKind;
+use std::ops::Deref;
 use std::path::Path;
 
+#[derive(Debug)]
+pub enum BadType {
+CharacterDevice,
+BlockDevice,
+FIFO,
+Socket,
+Directory,
+Unknown,
+}
+
+#[derive(Debug)]
+pub enum BadMatch {
+OsError(i32),
+BadType(BadType),
+}
+
 /// Marker enum used to dispatch new status entries into the right collections.
 /// Is similar to `crate::EntryState`, but represents the transient state of
 /// entries during the lifetime of a command.
@@ -36,6 +55,11 @@
 Deleted,
 Clean,
 Unknown,
+Ignored,
+/// Empty dispatch, the file is not worth listing
+None,
+/// Was explicitly matched but cannot be found/accessed
+Bad(BadMatch),
 }
 
 type IoResult = std::io::Result;
@@ -81,9 +105,7 @@
 entry: DirstateEntry,
 metadata: HgMetadata,
 copy_map: ,
-check_exec: bool,
-list_clean: bool,
-last_normal_time: i64,
+options: StatusOptions,
 ) -> Dispatch {
 let DirstateEntry {
 state,
@@ -103,7 +125,7 @@
 EntryState::Normal => {
 let size_changed = mod_compare(size, st_size as i32);
 let mode_changed =
-(mode ^ st_mode as i32) & 0o100 != 0o000 && check_exec;
+(mode ^ st_mode as i32) & 0o100 != 0o000 && options.check_exec;
 let metadata_changed = size >= 0 && (size_changed || mode_changed);
 let other_parent = size == SIZE_FROM_OTHER_PARENT;
 if metadata_changed
@@ -113,14 +135,14 @@
 Dispatch::Modified
 } else if mod_compare(mtime, st_mtime as i32) {
 Dispatch::Unsure
-} else if st_mtime == last_normal_time {
+} else if st_mtime == options.last_normal_time {
 // the file may have just been marked as normal and
 // it may have changed in the same second without
 // changing its size. This can happen if we quickly
 // do multiple commits. Force lookup, so we don't
 // miss such a racy file change.
 Dispatch::Unsure
-} else if list_clean {
+} else if options.list_clean {
 Dispatch::Clean
 } else {
 Dispatch::Unknown
@@ -153,9 +175,7 @@
 files: &'a HashSet<>,
 dmap: &'a DirstateMap,
 root_dir: impl AsRef + Sync + Send,
-check_exec: bool,
-list_clean: bool,
-last_normal_time: i64,
+options: StatusOptions,
 ) -> impl ParallelIterator> {
 files.par_iter().filter_map(move |filename| {
 // TODO normalization
@@ 

D7927: rust-status: add util for listing a directory

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I debated moving it to utils, but it is not used anywhere else for now, and
  its skip behavior is pretty specific to status.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7927

AFFECTED FILES
  rust/hg-core/src/dirstate/status.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/dirstate/status.rs 
b/rust/hg-core/src/dirstate/status.rs
--- a/rust/hg-core/src/dirstate/status.rs
+++ b/rust/hg-core/src/dirstate/status.rs
@@ -14,12 +14,15 @@
 matchers::Matcher,
 utils::{
 files::HgMetadata,
-hg_path::{hg_path_to_path_buf, HgPath},
+hg_path::{
+hg_path_to_path_buf, os_string_to_hg_path_buf, HgPath, HgPathBuf,
+},
 },
 CopyMap, DirstateEntry, DirstateMap, EntryState,
 };
 use rayon::prelude::*;
 use std::collections::HashSet;
+use std::fs::{read_dir, DirEntry};
 use std::path::Path;
 
 /// Marker enum used to dispatch new status entries into the right collections.
@@ -48,6 +51,30 @@
 a & i32::max_value() != b & i32::max_value()
 }
 
+/// Return a sorted list containing information about the entries
+/// in the directory.
+fn list_directory(
+path: impl AsRef,
+skip_dot_hg: bool,
+) -> std::io::Result> {
+let mut results = vec![];
+let entries = read_dir(path.as_ref())?;
+
+for entry in entries {
+let entry = entry?;
+let filename = os_string_to_hg_path_buf(entry.file_name())?;
+let file_type = entry.file_type()?;
+if skip_dot_hg && filename.as_bytes() == b".hg" && file_type.is_dir() {
+return Ok(vec![]);
+} else {
+results.push((HgPathBuf::from(filename), entry))
+}
+}
+
+results.sort_by(|a, b| a.0.cmp());
+Ok(results)
+}
+
 /// The file corresponding to the dirstate entry was found on the filesystem.
 fn dispatch_found(
 filename: impl AsRef,



To: Alphare, #hg-reviewers
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7926: rust-dependencies: update rayon

[Raphaël Gomès]

Alphare created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7926

AFFECTED FILES
  rust/Cargo.lock
  rust/hg-core/Cargo.toml

CHANGE DETAILS

diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -16,7 +16,7 @@
 memchr = "2.2.0"
 rand = "0.6.5"
 rand_pcg = "0.1.1"
-rayon = "1.2.0"
+rayon = "1.3.0"
 regex = "1.1.0"
 twox-hash = "1.5.0"
 same-file = "1.0.6"
diff --git a/rust/Cargo.lock b/rust/Cargo.lock
--- a/rust/Cargo.lock
+++ b/rust/Cargo.lock
@@ -5,7 +5,7 @@
 version = "0.7.6"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
- "memchr 2.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "memchr 2.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -17,16 +17,13 @@
 ]
 
 [[package]]
-name = "arrayvec"
-version = "0.4.12"
+name = "autocfg"
+version = "0.1.7"
 source = "registry+https://github.com/rust-lang/crates.io-index;
-dependencies = [
- "nodrop 0.1.14 (registry+https://github.com/rust-lang/crates.io-index)",
-]
 
 [[package]]
 name = "autocfg"
-version = "0.1.6"
+version = "1.0.0"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 
 [[package]]
@@ -41,11 +38,10 @@
 
 [[package]]
 name = "c2-chacha"
-version = "0.2.2"
+version = "0.2.3"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
- "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "ppv-lite86 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)",
+ "ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -72,46 +68,48 @@
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
  "libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
- "num-traits 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)",
+ "num-traits 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)",
  "python27-sys 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "python3-sys 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "crossbeam-deque"
-version = "0.7.1"
+version = "0.7.2"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
- "crossbeam-epoch 0.7.2 
(registry+https://github.com/rust-lang/crates.io-index)",
- "crossbeam-utils 0.6.6 
(registry+https://github.com/rust-lang/crates.io-index)",
+ "crossbeam-epoch 0.8.0 
(registry+https://github.com/rust-lang/crates.io-index)",
+ "crossbeam-utils 0.7.0 
(registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "crossbeam-epoch"
-version = "0.7.2"
+version = "0.8.0"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
- "arrayvec 0.4.12 (registry+https://github.com/rust-lang/crates.io-index)",
+ "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
  "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
- "crossbeam-utils 0.6.6 
(registry+https://github.com/rust-lang/crates.io-index)",
+ "crossbeam-utils 0.7.0 
(registry+https://github.com/rust-lang/crates.io-index)",
  "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "memoffset 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "memoffset 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)",
  "scopeguard 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "crossbeam-queue"
-version = "0.1.2"
+version = "0.2.1"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
- "crossbeam-utils 0.6.6 
(registry+https://github.com/rust-lang/crates.io-index)",
+ "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
+ "crossbeam-utils 0.7.0 
(registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
 name = "crossbeam-utils"
-version = "0.6.6"
+version = "0.7.0"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
+ "autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
  "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
  "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
@@ -142,12 +140,20 @@
 
 [[package]]
 name = "getrandom"
-version = "0.1.12"
+version = "0.1.14"
 source = "registry+https://github.com/rust-lang/crates.io-index;
 dependencies = [
  "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
- "wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "wasi 0.9.0+wasi-snapshot-preview1 
(registry+https://github.com/rust-lang/crates.io-index)",
+]
+

D7492: localrepo: also fastpath access to working copy parents when possible

[marmoute (Pierre-Yves David)]

Closed by commit rHG85c4cd73996b: localrepo: also fastpath access to working 
copy parents when possible (authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7492?vs=19361=19409

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7492/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7492

AFFECTED FILES
  mercurial/context.py
  mercurial/localrepo.py
  mercurial/scmutil.py
  tests/test-repo-filters-tiptoe.t

CHANGE DETAILS

diff --git a/tests/test-repo-filters-tiptoe.t b/tests/test-repo-filters-tiptoe.t
--- a/tests/test-repo-filters-tiptoe.t
+++ b/tests/test-repo-filters-tiptoe.t
@@ -62,7 +62,6 @@
 Getting status of working copy
 
   $ hg status
-  debug.filters: computing revision filter for "visible"
   M c
   A d
   R a
@@ -78,7 +77,6 @@
 Getting working copy diff
 
   $ hg diff
-  debug.filters: computing revision filter for "visible"
   diff -r c2932ca7786be30b67154d541a8764fae5532261 a
   --- a/a  Thu Jan 01 00:00:00 1970 +
   +++ /dev/nullThu Jan 01 00:00:00 1970 +
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1466,6 +1466,7 @@
 if src not in newctx or dst in newctx or ds[dst] != b'a':
 src = None
 ds.copy(src, dst)
+repo._quick_access_changeid_invalidate()
 
 
 def writerequires(opener, requirements):
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1517,18 +1517,51 @@
 narrowspec.save(self, newincludes, newexcludes)
 self.invalidate(clearfilecache=True)
 
-@util.propertycache
+@unfilteredpropertycache
+def _quick_access_changeid_null(self):
+return {
+b'null': (nullrev, nullid),
+nullrev: (nullrev, nullid),
+nullid: (nullrev, nullid),
+}
+
+@unfilteredpropertycache
+def _quick_access_changeid_wc(self):
+# also fast path access to the working copy parents
+# however, only do it for filter that ensure wc is visible.
+quick = {}
+cl = self.unfiltered().changelog
+for node in self.dirstate.parents():
+if node == nullid:
+continue
+rev = cl.index.get_rev(node)
+if rev is None:
+# unknown working copy parent case:
+#
+#   skip the fast path and let higher code deal with it
+continue
+pair = (rev, node)
+quick[rev] = pair
+quick[node] = pair
+return quick
+
+@unfilteredmethod
+def _quick_access_changeid_invalidate(self):
+if '_quick_access_changeid_wc' in vars(self):
+del self.__dict__['_quick_access_changeid_wc']
+
+@property
 def _quick_access_changeid(self):
 """an helper dictionnary for __getitem__ calls
 
 This contains a list of symbol we can recognise right away without
 further processing.
 """
-return {
-b'null': (nullrev, nullid),
-nullrev: (nullrev, nullid),
-nullid: (nullrev, nullid),
-}
+mapping = self._quick_access_changeid_null
+if self.filtername in repoview.filter_has_wc:
+mapping = mapping.copy()
+mapping.update(self._quick_access_changeid_wc)
+return mapping
 
 def __getitem__(self, changeid):
 # dealing with special cases
@@ -1887,6 +1920,7 @@
 
 def setparents(self, p1, p2=nullid):
 self[None].setparents(p1, p2)
+self._quick_access_changeid_invalidate()
 
 def filectx(self, path, changeid=None, fileid=None, changectx=None):
 """changeid must be a changeset revision, if specified.
@@ -2484,6 +2518,7 @@
 def invalidatevolatilesets(self):
 self.filteredrevcache.clear()
 obsolete.clearobscaches(self)
+self._quick_access_changeid_invalidate()
 
 def invalidatedirstate(self):
 '''Invalidates the dirstate, causing the next call to dirstate
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -1989,6 +1989,7 @@
 for f in self.removed():
 self._repo.dirstate.drop(f)
 self._repo.dirstate.setparents(node)
+self._repo._quick_access_changeid_invalidate()
 
 # write changes out explicitly, because nesting wlock at
 # runtime may prevent 'wlock.release()' in 'repo.commit()'



To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7502: changectx: mark parent of changesets as non filtered

[marmoute (Pierre-Yves David)]

Closed by commit rHG98349eddceef: changectx: mark parent of changesets as non 
filtered (authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7502?vs=19366=19414

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7502/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7502

AFFECTED FILES
  mercurial/context.py

CHANGE DETAILS

diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -533,8 +533,11 @@
 
 p1, p2 = cl.parentrevs(self._rev)
 if p2 == nullrev:
-return [repo[p1]]
-return [repo[p1], repo[p2]]
+return [changectx(repo, p1, cl.node(p1), maybe_filtered=False)]
+return [
+changectx(repo, p1, cl.node(p1), maybe_filtered=False),
+changectx(repo, p2, cl.node(p2), maybe_filtered=False),
+]
 
 def changeset(self):
 c = self._changeset



To: marmoute, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7501: changectx: use unfiltered changelog to walk ancestors in annotate

[marmoute (Pierre-Yves David)]

Closed by commit rHGc472970339d2: changectx: use unfiltered changelog to walk 
ancestors in annotate (authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7501?vs=19365=19413

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7501/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7501

AFFECTED FILES
  mercurial/context.py
  tests/test-repo-filters-tiptoe.t

CHANGE DETAILS

diff --git a/tests/test-repo-filters-tiptoe.t b/tests/test-repo-filters-tiptoe.t
--- a/tests/test-repo-filters-tiptoe.t
+++ b/tests/test-repo-filters-tiptoe.t
@@ -136,13 +136,11 @@
 - file with a single change
 
   $ hg annotate a
-  debug.filters: computing revision filter for "visible"
   0: a
 
 - file with multiple change
 
   $ hg annotate z
-  debug.filters: computing revision filter for "visible"
   0: some line
   1: in a
   2: file
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -1161,7 +1161,9 @@
 # use linkrev to find the first changeset where self appeared
 base = self.introfilectx()
 if getattr(base, '_ancestrycontext', None) is None:
-cl = self._repo.changelog
+# it is safe to use an unfiltered repository here because we are
+# walking ancestors only.
+cl = self._repo.unfiltered().changelog
 if base.rev() is None:
 # wctx is not inclusive, but works because _ancestrycontext
 # is used to test filelog revisions



To: marmoute, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7495: localrepo: recognize trivial request for '.'

[marmoute (Pierre-Yves David)]

Closed by commit rHGbfaf4c673bec: localrepo: recognize trivial request for 
. (authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7495?vs=19363=19411

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7495/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7495

AFFECTED FILES
  mercurial/localrepo.py
  tests/test-repo-filters-tiptoe.t

CHANGE DETAILS

diff --git a/tests/test-repo-filters-tiptoe.t b/tests/test-repo-filters-tiptoe.t
--- a/tests/test-repo-filters-tiptoe.t
+++ b/tests/test-repo-filters-tiptoe.t
@@ -70,7 +70,6 @@
 Getting data about the working copy parent
 
   $ hg log -r '.' -T "{node}\n{date}\n"
-  debug.filters: computing revision filter for "visible"
   c2932ca7786be30b67154d541a8764fae5532261
   0.00
 
@@ -111,8 +110,8 @@
 exporting the current changeset
 
   $ hg export
+  exporting patch:
   debug.filters: computing revision filter for "visible"
-  exporting patch:
   # HG changeset patch
   # User test
   # Date 0 0
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1708,6 +1708,10 @@
 '''
 if specs == [b'null']:
 return revset.baseset([nullrev])
+if specs == [b'.']:
+quick_data = self._quick_access_changeid.get(b'.')
+if quick_data is not None:
+return revset.baseset([quick_data[0]])
 if user:
 m = revset.matchany(
 self.ui,



To: marmoute, #hg-reviewers, pulkit
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7498: localrepo: also fast past the parents of working copies parents

[marmoute (Pierre-Yves David)]

Closed by commit rHG5a012404503b: localrepo: also fast past the parents of 
working copies parents (authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7498?vs=19364=19412

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7498/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7498

AFFECTED FILES
  mercurial/localrepo.py
  tests/test-repo-filters-tiptoe.t

CHANGE DETAILS

diff --git a/tests/test-repo-filters-tiptoe.t b/tests/test-repo-filters-tiptoe.t
--- a/tests/test-repo-filters-tiptoe.t
+++ b/tests/test-repo-filters-tiptoe.t
@@ -93,7 +93,6 @@
   @@ -0,0 +1,1 @@
   +d
   $ hg diff --change .
-  debug.filters: computing revision filter for "visible"
   diff -r 05293e5dd8d1ae4f84a8520a11c6f97cad26deca -r 
c2932ca7786be30b67154d541a8764fae5532261 c
   --- /dev/nullThu Jan 01 00:00:00 1970 +
   +++ b/c  Thu Jan 01 00:00:00 1970 +
@@ -111,7 +110,6 @@
 
   $ hg export
   exporting patch:
-  debug.filters: computing revision filter for "visible"
   # HG changeset patch
   # User test
   # Date 0 0
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1543,6 +1543,14 @@
 pair = (rev, node)
 quick[rev] = pair
 quick[node] = pair
+# also add the parents of the parents
+for r in cl.parentrevs(rev):
+if r == nullrev:
+continue
+n = cl.node(r)
+pair = (r, n)
+quick[r] = pair
+quick[n] = pair
 p1node = self.dirstate.p1()
 if p1node != nullid:
 quick[b'.'] = quick[p1node]



To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7494: localrepo: fastpath access to "."

[marmoute (Pierre-Yves David)]

Closed by commit rHGd86dede17392: localrepo: fastpath access to . 
(authored by marmoute).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7494?vs=19362=19410

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7494/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7494

AFFECTED FILES
  mercurial/localrepo.py

CHANGE DETAILS

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1543,6 +1543,9 @@
 pair = (rev, node)
 quick[rev] = pair
 quick[node] = pair
+p1node = self.dirstate.p1()
+if p1node != nullid:
+quick[b'.'] = quick[p1node]
 return quick
 
 @unfilteredmethod



To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7911: examples: refer to nightly rustfmt in Windows-compatible way

[martinvonz (Martin von Zweigbergk)]

Closed by commit rHG2077ffede71f: examples: refer to nightly rustfmt in 
Windows-compatible way (authored by martinvonz).
This revision was automatically updated to reflect the committed changes.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7911?vs=19376=19408

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7911/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7911

AFFECTED FILES
  contrib/examples/fix.hgrc

CHANGE DETAILS

diff --git a/contrib/examples/fix.hgrc b/contrib/examples/fix.hgrc
--- a/contrib/examples/fix.hgrc
+++ b/contrib/examples/fix.hgrc
@@ -2,7 +2,7 @@
 clang-format:command = clang-format --style file
 clang-format:pattern = set:(**.c or **.cc or **.h) and not 
"include:contrib/clang-format-ignorelist"
 
-rustfmt:command = $(rustup which --toolchain nightly rustfmt)
+rustfmt:command = rustfmt +nightly
 rustfmt:pattern = set:**.rs
 
 black:command = black --config=black.toml -



To: martinvonz, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7732: convert: refactor authormap into separate function for outside use

[joerg.sonnenberger (Joerg Sonnenberger)]

Closed by commit rHGfdaa4233dc18: convert: refactor authormap into separate 
function for outside use (authored by joerg.sonnenberger).
This revision was automatically updated to reflect the committed changes.

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D7732?vs=19398=19407#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7732?vs=19398=19407

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7732/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7732

AFFECTED FILES
  hgext/convert/convcmd.py

CHANGE DETAILS

diff --git a/hgext/convert/convcmd.py b/hgext/convert/convcmd.py
--- a/hgext/convert/convcmd.py
+++ b/hgext/convert/convcmd.py
@@ -56,6 +56,36 @@
 orig_encoding = b'ascii'
 
 
+def readauthormap(ui, authorfile, authors=None):
+if authors is None:
+authors = {}
+with open(authorfile, b'rb') as afile:
+for line in afile:
+
+line = line.strip()
+if not line or line.startswith(b'#'):
+continue
+
+try:
+srcauthor, dstauthor = line.split(b'=', 1)
+except ValueError:
+msg = _(b'ignoring bad line in author map file %s: %s\n')
+ui.warn(msg % (authorfile, line.rstrip()))
+continue
+
+srcauthor = srcauthor.strip()
+dstauthor = dstauthor.strip()
+if authors.get(srcauthor) in (None, dstauthor):
+msg = _(b'mapping author %s to %s\n')
+ui.debug(msg % (srcauthor, dstauthor))
+authors[srcauthor] = dstauthor
+continue
+
+m = _(b'overriding mapping for author %s, was %s, will be %s\n')
+ui.status(m % (srcauthor, authors[srcauthor], dstauthor))
+return authors
+
+
 def recode(s):
 if isinstance(s, pycompat.unicode):
 return s.encode(pycompat.sysstr(orig_encoding), 'replace')
@@ -448,32 +478,7 @@
 ofile.close()
 
 def readauthormap(self, authorfile):
-afile = open(authorfile, b'rb')
-for line in afile:
-
-line = line.strip()
-if not line or line.startswith(b'#'):
-continue
-
-try:
-srcauthor, dstauthor = line.split(b'=', 1)
-except ValueError:
-msg = _(b'ignoring bad line in author map file %s: %s\n')
-self.ui.warn(msg % (authorfile, line.rstrip()))
-continue
-
-srcauthor = srcauthor.strip()
-dstauthor = dstauthor.strip()
-if self.authors.get(srcauthor) in (None, dstauthor):
-msg = _(b'mapping author %s to %s\n')
-self.ui.debug(msg % (srcauthor, dstauthor))
-self.authors[srcauthor] = dstauthor
-continue
-
-m = _(b'overriding mapping for author %s, was %s, will be %s\n')
-self.ui.status(m % (srcauthor, self.authors[srcauthor], dstauthor))
-
-afile.close()
+self.authors = readauthormap(self.ui, authorfile, self.authors)
 
 def cachecommit(self, rev):
 commit = self.source.getcommit(rev)



To: joerg.sonnenberger, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7732: convert: refactor authormap into separate function for outside use

[pulkit (Pulkit Goyal)]

pulkit added a comment.


  Amending the following in flight to make `test-check-format.t` (or black) 
happy:
  
diff --git a/hgext/convert/convcmd.py b/hgext/convert/convcmd.py
--- a/hgext/convert/convcmd.py
+++ b/hgext/convert/convcmd.py
@@ -55,6 +55,7 @@ svn_source = subversion.svn_source
 
 orig_encoding = b'ascii'
 
+
 def readauthormap(ui, authorfile, authors=None):
 if authors is None:
 authors = {}
@@ -84,6 +85,7 @@ def readauthormap(ui, authorfile, author
 ui.status(m % (srcauthor, authors[srcauthor], dstauthor))
 return authors
 
+
 def recode(s):
 if isinstance(s, pycompat.unicode):
 return s.encode(pycompat.sysstr(orig_encoding), 'replace')

REPOSITORY
  rHG Mercurial

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7732/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7732

To: joerg.sonnenberger, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7492: localrepo: also fastpath access to working copy parents when possible

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  Thank you, much appreciated.

REPOSITORY
  rHG Mercurial

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7492/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7492

To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7498: localrepo: also fast past the parents of working copies parents

[pulkit (Pulkit Goyal)]

This revision is now accepted and ready to land.
pulkit added a comment.
pulkit accepted this revision.


  > There are a descent odds that their will be needed too.
  
  There are descent odds that they will be needed too.
  
  made the above edit in flight.

REPOSITORY
  rHG Mercurial

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7498/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7498

To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7494: localrepo: fastpath access to "."

[pulkit (Pulkit Goyal)]

pulkit added a comment.


  > "." is just an aliast for `p1(wdir()`,
  
  "." is just an alias for `p1(wdir())`,
  
  in flight.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7494/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7494

To: marmoute, #hg-reviewers
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7492: localrepo: also fastpath access to working copy parents when possible

[pulkit (Pulkit Goyal)]

This revision is now accepted and ready to land.
pulkit added a comment.
pulkit accepted this revision.


  Amending the perf numbers in commit message and fixed some spellings.

REPOSITORY
  rHG Mercurial

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7492/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7492

To: marmoute, #hg-reviewers, pulkit
Cc: pulkit, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] remotefilelog: fix opening validatecachelog in text mode

[Yuya Nishihara]

On Tue, 14 Jan 2020 18:02:47 +0900, Inada Naoki wrote:
> # HG changeset patch
> # User Inada Naoki 
> # Date 1578992235 -32400
> #  Tue Jan 14 17:57:15 2020 +0900
> # Node ID b8231c6f3cdd3dd56eb0b9aa24b70644ec3d0026
> # Parent  8e09551206f54bb102cdcbf4c57b0b10284735ce
> remotefilelog: fix opening validatecachelog in text mode

Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] remotefilelog: fix opening validatecachelog in text mode

[Inada Naoki]

# HG changeset patch
# User Inada Naoki 
# Date 1578992235 -32400
#  Tue Jan 14 17:57:15 2020 +0900
# Node ID b8231c6f3cdd3dd56eb0b9aa24b70644ec3d0026
# Parent  8e09551206f54bb102cdcbf4c57b0b10284735ce
remotefilelog: fix opening validatecachelog in text mode

diff -r 8e09551206f5 -r b8231c6f3cdd hgext/remotefilelog/basestore.py
--- a/hgext/remotefilelog/basestore.py  Sat Jan 11 05:44:58 2020 +0100
+++ b/hgext/remotefilelog/basestore.py  Tue Jan 14 17:57:15 2020 +0900
@@ -225,7 +225,7 @@ class basestore(object):
 data = shallowutil.readfile(filepath)
 if self._validatecache and not self._validatedata(data, filepath):
 if self._validatecachelog:
-with open(self._validatecachelog, b'a+') as f:
+with open(self._validatecachelog, b'ab+') as f:
 f.write(b"corrupt %s during read\n" % filepath)
 os.rename(filepath, filepath + b".corrupt")
 raise KeyError(b"corrupt local cache file %s" % filepath)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44098: new changeset

[Mercurial Commits]

New changeset in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/19533e4c3450
changeset:   44098:19533e4c3450
bookmark:@
tag: tip
user:Kyle Lippincott 
date:Thu Jan 16 12:17:03 2020 -0800
summary: py3: fix curses chunkselector fallback (when diffs are too large) 
on py3

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7913: cext: fix compiler warning about sign changing

[spectral (Kyle Lippincott)]

Closed by commit rHG969527ac7b44: cext: fix compiler warning about sign 
changing (authored by spectral).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7913?vs=19380=19406

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7913/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7913

AFFECTED FILES
  mercurial/cext/manifest.c

CHANGE DETAILS

diff --git a/mercurial/cext/manifest.c b/mercurial/cext/manifest.c
--- a/mercurial/cext/manifest.c
+++ b/mercurial/cext/manifest.c
@@ -42,17 +42,17 @@
 #define MANIFEST_TOO_SHORT_LINE -5
 
 /* get the length of the path for a line */
-static size_t pathlen(line *l)
+static Py_ssize_t pathlen(line *l)
 {
const char *end = memchr(l->start, '\0', l->len);
-   return (end) ? (size_t)(end - l->start) : l->len;
+   return (end) ? (Py_ssize_t)(end - l->start) : l->len;
 }
 
 /* get the node value of a single line */
 static PyObject *nodeof(line *l)
 {
char *s = l->start;
-   ssize_t llen = pathlen(l);
+   Py_ssize_t llen = pathlen(l);
PyObject *hash;
if (llen + 1 + 40 + 1 > l->len) { /* path '\0' hash '\n' */
PyErr_SetString(PyExc_ValueError, "manifest line too short");
@@ -76,7 +76,7 @@
 static PyObject *hashflags(line *l)
 {
char *s = l->start;
-   size_t plen = pathlen(l);
+   Py_ssize_t plen = pathlen(l);
PyObject *hash = nodeof(l);
 
/* 40 for hash, 1 for null byte, 1 for newline */
@@ -270,7 +270,7 @@
 
 static PyObject *lmiter_iterentriesnext(PyObject *o)
 {
-   size_t pl;
+   Py_ssize_t pl;
line *l;
Py_ssize_t consumed;
PyObject *ret = NULL, *path = NULL, *hash = NULL, *flags = NULL;
@@ -337,7 +337,7 @@
 
 static PyObject *lmiter_iterkeysnext(PyObject *o)
 {
-   size_t pl;
+   Py_ssize_t pl;
line *l = lmiter_nextline((lmIter *)o);
if (!l) {
return NULL;



To: spectral, #hg-reviewers, mharbison72
Cc: yuja, mharbison72, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7903: sha1dc: avoid including the nonexistent stdint.h with Visual Studio 2008

[mharbison72 (Matt Harbison)]

Closed by commit rHGb4420cea45e8: sha1dc: avoid including the nonexistent 
stdint.h with Visual Studio 2008 (authored by mharbison72).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7903?vs=19389=19405

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7903/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7903

AFFECTED FILES
  mercurial/thirdparty/sha1dc/lib/ubc_check.c
  mercurial/thirdparty/sha1dc/lib/ubc_check.h

CHANGE DETAILS

diff --git a/mercurial/thirdparty/sha1dc/lib/ubc_check.h 
b/mercurial/thirdparty/sha1dc/lib/ubc_check.h
--- a/mercurial/thirdparty/sha1dc/lib/ubc_check.h
+++ b/mercurial/thirdparty/sha1dc/lib/ubc_check.h
@@ -28,7 +28,12 @@
 #endif
 
 #ifndef SHA1DC_NO_STANDARD_INCLUDES
+#if !defined(_MSC_VER) || _MSC_VER >= 1600
 #include 
+#else
+/* prior to Visual Studio 2010 */
+typedef unsigned __int32 uint32_t;
+#endif
 #endif
 
 #define DVMASKSIZE 1
diff --git a/mercurial/thirdparty/sha1dc/lib/ubc_check.c 
b/mercurial/thirdparty/sha1dc/lib/ubc_check.c
--- a/mercurial/thirdparty/sha1dc/lib/ubc_check.c
+++ b/mercurial/thirdparty/sha1dc/lib/ubc_check.c
@@ -25,8 +25,10 @@
 */
 
 #ifndef SHA1DC_NO_STANDARD_INCLUDES
+#if !defined(_MSC_VER) || _MSC_VER >= 1600
 #include 
 #endif
+#endif
 #ifdef SHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C
 #include SHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C
 #endif



To: mharbison72, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7913: cext: fix compiler warning about sign changing

[yuja (Yuya Nishihara)]

yuja added a comment.


  > static PyObject *hashflags(line *l)
  >  {
  >
  > char *s = l->start;
  >
  > - size_t plen = pathlen(l);
  >
  > +   Py_ssize_t plen = pathlen(l);
  
  We'll probably want to change `hplen` to `Py_ssize_t` as well.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7913/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7913

To: spectral, #hg-reviewers, mharbison72
Cc: yuja, mharbison72, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D7913: cext: fix compiler warning about sign changing

[Yuya Nishihara]

>  static PyObject *hashflags(line *l)
>  {
>   char *s = l->start;
> - size_t plen = pathlen(l);
> + Py_ssize_t plen = pathlen(l);

We'll probably want to change `hplen` to `Py_ssize_t` as well.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7925: rust-matchers: add `IgnoreMatcher`

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is a big change but all of the pieces call each other, so it makes sense
  to have this all in one patch.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7925

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,14 +10,25 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::{build_single_regex, PatternResult},
-utils::hg_path::{HgPath, HgPathBuf},
-DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
+dirstate::dirs_multiset::DirsChildrenMultiset,
+filepatterns::{
+build_single_regex, filter_subincludes, get_patterns_from_file,
+PatternFileWarning, PatternResult, SubInclude,
+},
+utils::{
+files::find_dirs,
+hg_path::{HgPath, HgPathBuf},
+Escaped,
+},
+DirsMultiset, DirstateMapError, FastHashMap, IgnorePattern, PatternError,
 PatternSyntax,
 };
+
 use std::collections::HashSet;
+use std::fmt::{Display, Error, Formatter};
 use std::iter::FromIterator;
 use std::ops::Deref;
+use std::path::Path;
 
 #[derive(Debug, PartialEq)]
 pub enum VisitChildrenSet<'a> {
@@ -223,6 +234,88 @@
 }
 }
 
+/// Matches files that are included in the ignore rules.
+///
+#[cfg_attr(
+feature = "with-re2",
+doc = r##"
+```
+use hg::{
+matchers::{IncludeMatcher, Matcher},
+IgnorePattern,
+PatternSyntax,
+utils::hg_path::HgPath
+};
+use std::path::Path;
+///
+let ignore_patterns =
+vec![IgnorePattern::new(PatternSyntax::RootGlob, b"this*", Path::new(""))];
+let (matcher, _) = IncludeMatcher::new(ignore_patterns, "").unwrap();
+///
+assert_eq!(matcher.matches(HgPath::new(b"testing")), false);
+assert_eq!(matcher.matches(HgPath::new(b"this should work")), true);
+assert_eq!(matcher.matches(HgPath::new(b"this also")), true);
+assert_eq!(matcher.matches(HgPath::new(b"but not this")), false);
+```
+"##
+)]
+pub struct IncludeMatcher<'a> {
+patterns: Vec,
+match_fn: Box Fn(&'r HgPath) -> bool + 'a + Sync>,
+/// Whether all the patterns match a prefix (i.e. recursively)
+prefix: bool,
+roots: HashSet,
+dirs: HashSet,
+parents: HashSet,
+}
+
+impl<'a> Matcher for IncludeMatcher<'a> {
+fn file_set() -> Option<<>> {
+None
+}
+
+fn exact_match(, _filename: impl AsRef) -> bool {
+false
+}
+
+fn matches(, filename: impl AsRef) -> bool {
+(self.match_fn)(filename.as_ref())
+}
+
+fn visit_children_set(
+,
+directory: impl AsRef,
+) -> VisitChildrenSet {
+let dir = directory.as_ref();
+if self.prefix && self.roots.contains(dir) {
+return VisitChildrenSet::Recursive;
+}
+if self.roots.contains(HgPath::new(b""))
+|| self.roots.contains(dir)
+|| self.dirs.contains(dir)
+|| find_dirs(dir).any(|parent_dir| self.roots.contains(parent_dir))
+{
+return VisitChildrenSet::This;
+}
+
+if self.parents.contains(directory.as_ref()) {
+let multiset = self.get_all_parents_children();
+if let Some(children) = multiset.get(dir) {
+return VisitChildrenSet::Set(children.to_owned());
+}
+}
+VisitChildrenSet::Empty
+}
+
+fn matches_everything() -> bool {
+false
+}
+
+fn is_exact() -> bool {
+false
+}
+}
+
 const MAX_RE_SIZE: usize = 2;
 
 #[cfg(feature = "with-re2")]
@@ -387,6 +480,171 @@
 Ok((HashSet::from_iter(roots), HashSet::from_iter(dirs), parents))
 }
 
+/// Returns a function that checks whether a given file (in the general sense)
+/// should be matched.
+fn build_match<'a, 'b>(
+ignore_patterns: &'a [IgnorePattern],
+root_dir: impl AsRef,
+) -> PatternResult<(
+Vec,
+Box bool + 'b + Sync>,
+Vec,
+)> {
+let mut match_funcs: Vec bool + Sync>> = vec![];
+// For debugging and printing
+let mut patterns = vec![];
+let mut all_warnings = vec![];
+
+let (subincludes, ignore_patterns) =
+filter_subincludes(ignore_patterns, root_dir)?;
+
+if !subincludes.is_empty() {
+// Build prefix-based matcher functions for subincludes
+let mut submatchers = FastHashMap::default();
+let mut prefixes = vec![];
+
+for SubInclude { prefix, root, path } in subincludes.into_iter() {
+let (match_fn, warnings) = get_ignore_function(&[path], root)?;
+all_warnings.extend(warnings);
+prefixes.push(prefix.to_owned());
+submatchers.insert(prefix.to_owned(), match_fn);
+}
+
+let match_subinclude = move |filename: | {
+  

D7910: rust-re2: add wrapper for calling Re2 from Rust

[Raphaël Gomès]

Alphare updated this revision to Diff 19399.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7910?vs=19386=19399

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7910/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7910

AFFECTED FILES
  rust/Cargo.lock
  rust/hg-core/Cargo.toml
  rust/hg-core/build.rs
  rust/hg-core/src/lib.rs
  rust/hg-core/src/re2/mod.rs
  rust/hg-core/src/re2/re2.rs
  rust/hg-core/src/re2/rust_re2.cpp
  rust/hg-cpython/Cargo.toml
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1351,10 +1351,19 @@
 env['HOME'] = pwd.getpwuid(os.getuid()).pw_dir
 
 cargocmd = ['cargo', 'rustc', '-vv', '--release']
+
+feature_flags = []
+
 if sys.version_info[0] == 3 and self.py3_features is not None:
-cargocmd.extend(
-('--features', self.py3_features, '--no-default-features')
-)
+feature_flags.append(self.py3_features)
+cargocmd.append('--no-default-features')
+
+rust_features = env.get("HG_RUST_FEATURES")
+if rust_features:
+feature_flags.append(rust_features)
+
+cargocmd.extend(('--features', " ".join(feature_flags)))
+
 cargocmd.append('--')
 if sys.platform == 'darwin':
 cargocmd.extend(
diff --git a/rust/hg-cpython/Cargo.toml b/rust/hg-cpython/Cargo.toml
--- a/rust/hg-cpython/Cargo.toml
+++ b/rust/hg-cpython/Cargo.toml
@@ -10,6 +10,7 @@
 
 [features]
 default = ["python27"]
+with-re2 = ["hg-core/with-re2"]
 
 # Features to build an extension module:
 python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"]
@@ -21,7 +22,7 @@
 python3-bin = ["cpython/python3-sys"]
 
 [dependencies]
-hg-core = { path = "../hg-core" }
+hg-core = { path = "../hg-core"}
 libc = '*'
 
 [dependencies.cpython]
diff --git a/rust/hg-core/src/re2/rust_re2.cpp 
b/rust/hg-core/src/re2/rust_re2.cpp
new file mode 100644
--- /dev/null
+++ b/rust/hg-core/src/re2/rust_re2.cpp
@@ -0,0 +1,49 @@
+/*
+rust_re2.cpp
+
+C ABI export of Re2's C++ interface for Rust FFI.
+
+Copyright 2020 Valentin Gatien-Baron
+
+This software may be used and distributed according to the terms of the
+GNU General Public License version 2 or any later version.
+*/
+
+#include 
+using namespace re2;
+
+extern "C" {
+   RE2* rust_re2_create(const char* data, size_t len) {
+   RE2::Options o;
+   o.set_encoding(RE2::Options::Encoding::EncodingLatin1);
+   o.set_log_errors(false);
+   o.set_max_mem(5000);
+
+   return new RE2(StringPiece(data, len), o);
+   }
+
+   void rust_re2_destroy(RE2* re) {
+   delete re;
+   }
+
+   bool rust_re2_ok(RE2* re) {
+   return re->ok();
+   }
+
+   void rust_re2_error(RE2* re, const char** outdata, size_t* outlen) {
+   const std::string& e = re->error();
+   *outdata = e.data();
+   *outlen = e.length();
+   }
+
+   bool rust_re2_match(RE2* re, char* data, size_t len, int ianchor) {
+   const StringPiece sp = StringPiece(data, len);
+
+   RE2::Anchor anchor =
+   ianchor == 0 ? RE2::Anchor::UNANCHORED :
+   (ianchor == 1 ? RE2::Anchor::ANCHOR_START :
+RE2::Anchor::ANCHOR_BOTH);
+
+   return re->Match(sp, 0, len, anchor, NULL, 0);
+   }
+}
diff --git a/rust/hg-core/src/re2/re2.rs b/rust/hg-core/src/re2/re2.rs
new file mode 100644
--- /dev/null
+++ b/rust/hg-core/src/re2/re2.rs
@@ -0,0 +1,66 @@
+/*
+re2.rs
+
+Rust FFI bindings to Re2.
+
+Copyright 2020 Valentin Gatien-Baron
+
+This software may be used and distributed according to the terms of the
+GNU General Public License version 2 or any later version.
+*/
+use libc::{c_int, c_void};
+
+type Re2Ptr = *const c_void;
+
+pub struct Re2(Re2Ptr);
+
+/// `re2.h` says:
+/// "An "RE2" object is safe for concurrent use by multiple threads."
+unsafe impl Sync for Re2 {}
+
+/// These bind to the C ABI in `rust_re2.cpp`.
+extern "C" {
+fn rust_re2_create(data: *const u8, len: usize) -> Re2Ptr;
+fn rust_re2_destroy(re2: Re2Ptr);
+fn rust_re2_ok(re2: Re2Ptr) -> bool;
+fn rust_re2_error(
+re2: Re2Ptr,
+outdata: *mut *const u8,
+outlen: *mut usize,
+) -> bool;
+fn rust_re2_match(
+re2: Re2Ptr,
+data: *const u8,
+len: usize,
+anchor: c_int,
+) -> bool;
+}
+
+impl Re2 {
+pub fn new(pattern: &[u8]) -> Result {
+unsafe {
+let re2 = rust_re2_create(pattern.as_ptr(), pattern.len());
+if rust_re2_ok(re2) {
+Ok(Re2(re2))
+} else {
+let mut data: *const u8 = std::ptr::null();
+let mut len: usize = 0;
+rust_re2_error(re2,  data,  len);
+

D7924: rust-matchers: add `build_regex_match` function

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This function will be used to help build the upcoming `IncludeMatcher`.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7924

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,7 +10,7 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::PatternResult,
+filepatterns::{build_single_regex, PatternResult},
 utils::hg_path::{HgPath, HgPathBuf},
 DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
 PatternSyntax,
@@ -244,6 +244,57 @@
 Err(PatternError::Re2NotInstalled)
 }
 
+/// Returns a function that matches an `HgPath` against the regex formed by
+/// the given patterns.
+fn build_regex_match<'a>(
+ignore_patterns: &'a [&'a IgnorePattern],
+) -> PatternResult<(Vec, Box bool + Sync>)> {
+let mut all_groups = vec![];
+let regexps: Result, PatternError> = ignore_patterns
+.into_iter()
+.map(|k| build_single_regex(*k))
+.collect();
+let regexps = regexps?;
+let full_regex = regexps.join('|');
+
+let mut start_index = 0;
+let mut group_size = 0;
+
+for (index, re) in regexps.iter().enumerate() {
+let piece_size = re.len();
+if piece_size > MAX_RE_SIZE {
+return Err(PatternError::TooLong(piece_size));
+}
+if (group_size + piece_size) > MAX_RE_SIZE {
+let group = [start_index..index];
+all_groups.push(group.join('|'));
+start_index = index;
+group_size = 0
+}
+group_size += piece_size + 1;
+}
+
+let func = if start_index == 0 {
+let matcher = re_matcher(_regex)?;
+Box::new(move |filename: | matcher(filename))
+as Box bool + Sync>
+} else {
+let group = [start_index..];
+all_groups.push(group.join('|'));
+
+let all_matchers: PatternResult> =
+all_groups.iter().map(|group| re_matcher(group)).collect();
+let all_matchers = all_matchers?;
+
+let ret = Box::new(move |filename: | {
+all_matchers.iter().any(|f| f(filename))
+});
+ret as Box bool + Sync>
+};
+
+Ok((full_regex, func))
+}
+
 /// Returns roots and directories corresponding to each pattern.
 ///
 /// This calculates the roots and directories exactly matching the patterns and



To: Alphare, #hg-reviewers
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7921: rust-dirs-multiset: improve temporary error message

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  While we wait on a future patch that could verify that the paths passed to
  `DirsMultiset` have been audited, we still need to handle this error.
  This patch makes it easier to bubble up and makes the error clearer.
  
  Also, this patch introduces the `subslice_index` function that could be useful
  for other - albeit niche - purposes.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7921

AFFECTED FILES
  rust/hg-core/src/dirstate/dirs_multiset.rs
  rust/hg-core/src/lib.rs
  rust/hg-core/src/utils.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/utils.rs b/rust/hg-core/src/utils.rs
--- a/rust/hg-core/src/utils.rs
+++ b/rust/hg-core/src/utils.rs
@@ -61,6 +61,36 @@
 }
 }
 
+/// Find the offset of the subslice relative to the original collection
+///
+/// This function panics for zero-sized types.
+/// # Examples:
+///
+/// ```
+/// use crate::hg::utils::subslice_offset;
+/// let mut line = b"Subslice me!".to_vec();
+/// assert_eq!(subslice_offset(, [8..]), Some(8));
+///
+/// assert_eq!(subslice_offset(, b"hahaha"), None);
+///
+/// // Empty array
+/// let v: [u8; 0] = [];
+/// assert_eq!(subslice_offset(, ), Some(0));
+/// assert_eq!(subslice_offset(, b"hehe"), None);
+/// ```
+pub fn subslice_offset(outer: &[T], inner: &[T]) -> Option {
+let pointee_size = std::mem::size_of::();
+assert!(0 < pointee_size && pointee_size <= isize::max_value() as usize);
+
+let outer_start = outer.as_ptr() as usize;
+let inner = inner.as_ptr() as usize;
+if inner < outer_start || inner > outer_start.wrapping_add(outer.len()) {
+None
+} else {
+Some(inner.wrapping_sub(outer_start))
+}
+}
+
 pub trait SliceExt {
 fn trim_end() -> 
 fn trim_start() -> 
diff --git a/rust/hg-core/src/lib.rs b/rust/hg-core/src/lib.rs
--- a/rust/hg-core/src/lib.rs
+++ b/rust/hg-core/src/lib.rs
@@ -82,18 +82,17 @@
 pub enum DirstateMapError {
 PathNotFound(HgPathBuf),
 EmptyPath,
-ConsecutiveSlashes,
+InvalidPath(HgPathError),
 }
 
 impl ToString for DirstateMapError {
 fn to_string() -> String {
-use crate::DirstateMapError::*;
 match self {
-PathNotFound(_) => "expected a value, found none".to_string(),
-EmptyPath => "Overflow in dirstate.".to_string(),
-ConsecutiveSlashes => {
-"found invalid consecutive slashes in path".to_string()
+DirstateMapError::PathNotFound(_) => {
+"expected a value, found none".to_string()
 }
+DirstateMapError::EmptyPath => "Overflow in dirstate.".to_string(),
+DirstateMapError::InvalidPath(e) => e.to_string(),
 }
 }
 }
diff --git a/rust/hg-core/src/dirstate/dirs_multiset.rs 
b/rust/hg-core/src/dirstate/dirs_multiset.rs
--- a/rust/hg-core/src/dirstate/dirs_multiset.rs
+++ b/rust/hg-core/src/dirstate/dirs_multiset.rs
@@ -8,6 +8,8 @@
 //! A multiset of directory names.
 //!
 //! Used to counts the references to directories in a manifest or dirstate.
+use crate::utils::hg_path::HgPathError;
+use crate::utils::subslice_offset;
 use crate::{
 dirstate::EntryState,
 utils::{
@@ -78,7 +80,18 @@
 if subpath.as_bytes().last() == Some('/') {
 // TODO Remove this once PathAuditor is certified
 // as the only entrypoint for path data
-return Err(DirstateMapError::ConsecutiveSlashes);
+let offset = subslice_offset(
+path.as_ref().as_bytes(),
+subpath.as_bytes(),
+);
+let second_slash_index = offset.unwrap() + subpath.len() - 1;
+
+return Err(DirstateMapError::InvalidPath(
+HgPathError::ConsecutiveSlashes {
+bytes: path.as_ref().as_bytes().to_owned(),
+second_slash_index,
+},
+));
 }
 if let Some(val) = self.inner.get_mut(subpath) {
 *val += 1;



To: Alphare, #hg-reviewers
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7922: rust-matchers: add function to generate a regex matcher function

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This function will be used to help build the upcoming `IncludeMatcher`. While
  Re2 is still used and behind a feature flag, this function returns an error
  meant for fallback in the default case.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7922

AFFECTED FILES
  rust/hg-core/src/lib.rs
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -7,7 +7,12 @@
 
 //! Structs and types for matching files and directories.
 
-use crate::{utils::hg_path::HgPath, DirsMultiset, DirstateMapError};
+#[cfg(feature = "with-re2")]
+use crate::re2::Re2;
+use crate::{
+filepatterns::PatternResult, utils::hg_path::HgPath, DirsMultiset,
+DirstateMapError, PatternError,
+};
 use std::collections::HashSet;
 use std::iter::FromIterator;
 use std::ops::Deref;
@@ -215,6 +220,28 @@
 true
 }
 }
+
+const MAX_RE_SIZE: usize = 2;
+
+#[cfg(feature = "with-re2")]
+/// Returns a function that matches an `HgPath` against the given regex
+/// pattern.
+///
+/// This can fail when the pattern is invalid or not supported by the
+/// underlying engine `Re2`, for instance anything with back-references.
+fn re_matcher(
+pattern: &[u8],
+) -> PatternResult bool + Sync> {
+let regex = Re2::new(pattern);
+let regex = regex.map_err(|e| PatternError::UnsupportedSyntax(e))?;
+Ok(move |path: | regex.is_match(path.as_bytes()))
+}
+
+#[cfg(not(feature = "with-re2"))]
+fn re_matcher(_: &[u8]) -> PatternResult bool + Sync>> {
+Err(PatternError::Re2NotInstalled)
+}
+
 #[cfg(test)]
 mod tests {
 use super::*;
diff --git a/rust/hg-core/src/lib.rs b/rust/hg-core/src/lib.rs
--- a/rust/hg-core/src/lib.rs
+++ b/rust/hg-core/src/lib.rs
@@ -126,6 +126,9 @@
 /// Needed a pattern that can be turned into a regex but got one that can't
 /// This should only happen through programmer error.
 NonRegexPattern(IgnorePattern),
+/// This is temporary, see `re2/mod.rs`.
+/// This will cause a fallback to Python.
+Re2NotInstalled,
 }
 
 impl ToString for PatternError {
@@ -148,6 +151,10 @@
 PatternError::NonRegexPattern(pattern) => {
 format!("'{:?}' cannot be turned into a regex", pattern)
 }
+PatternError::Re2NotInstalled => {
+"Re2 is not installed, cannot use regex functionality."
+.to_string()
+}
 }
 }
 }



To: Alphare, #hg-reviewers
Cc: durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7923: rust-matchers: add functions to get roots, dirs and parents from patterns

[Raphaël Gomès]

Alphare created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  These functions will be used to help build the upcoming `IncludeMatcher`.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D7923

AFFECTED FILES
  rust/hg-core/src/matchers.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/matchers.rs b/rust/hg-core/src/matchers.rs
--- a/rust/hg-core/src/matchers.rs
+++ b/rust/hg-core/src/matchers.rs
@@ -10,8 +10,10 @@
 #[cfg(feature = "with-re2")]
 use crate::re2::Re2;
 use crate::{
-filepatterns::PatternResult, utils::hg_path::HgPath, DirsMultiset,
-DirstateMapError, PatternError,
+filepatterns::PatternResult,
+utils::hg_path::{HgPath, HgPathBuf},
+DirsMultiset, DirstateMapError, IgnorePattern, PatternError,
+PatternSyntax,
 };
 use std::collections::HashSet;
 use std::iter::FromIterator;
@@ -242,10 +244,147 @@
 Err(PatternError::Re2NotInstalled)
 }
 
+/// Returns roots and directories corresponding to each pattern.
+///
+/// This calculates the roots and directories exactly matching the patterns and
+/// returns a tuple of (roots, dirs) for each. It does not return other
+/// directories which may also need to be considered, like the parent
+/// directories.
+fn roots_and_dirs(
+ignore_patterns: &[IgnorePattern],
+) -> (Vec, Vec) {
+let mut roots = Vec::new();
+let mut dirs = Vec::new();
+
+for ignore_pattern in ignore_patterns {
+let IgnorePattern {
+syntax, pattern, ..
+} = ignore_pattern;
+match syntax {
+PatternSyntax::RootGlob | PatternSyntax::Glob => {
+let mut root = vec![];
+
+for p in pattern.split(|c| *c == b'/') {
+if p.iter().any(|c| match *c {
+b'[' | b'{' | b'*' | b'?' => true,
+_ => false,
+}) {
+break;
+}
+root.push(HgPathBuf::from_bytes(p));
+}
+let buf =
+root.iter().fold(HgPathBuf::new(), |acc, r| acc.join(r));
+roots.push(buf);
+}
+PatternSyntax::Path | PatternSyntax::RelPath => {
+let pat = HgPath::new(if pattern == b"." {
+&[] as &[u8]
+} else {
+pattern
+});
+roots.push(pat.to_owned());
+}
+PatternSyntax::RootFiles => {
+let pat = if pattern == b"." {
+&[] as &[u8]
+} else {
+pattern
+};
+dirs.push(HgPathBuf::from_bytes(pat));
+}
+_ => {
+roots.push(HgPathBuf::new());
+}
+}
+}
+(roots, dirs)
+}
+
+/// Returns roots and exact directories from patterns.
+///
+/// `roots` are directories to match recursively, `dirs` should
+/// be matched non-recursively, and `parents` are the implicitly required
+/// directories to walk to items in either roots or dirs.
+fn roots_dirs_and_parents(
+ignore_patterns: &[IgnorePattern],
+) -> PatternResult<(HashSet, HashSet, 
HashSet)>
+{
+let (roots, dirs) = roots_and_dirs(ignore_patterns);
+
+let mut parents = HashSet::new();
+
+parents.extend(
+DirsMultiset::from_manifest()
+.map_err(|e| match e {
+DirstateMapError::InvalidPath(e) => e,
+_ => unreachable!(),
+})?
+.iter()
+.map(|k| k.to_owned()),
+);
+parents.extend(
+DirsMultiset::from_manifest()
+.map_err(|e| match e {
+DirstateMapError::InvalidPath(e) => e,
+_ => unreachable!(),
+})?
+.iter()
+.map(|k| k.to_owned()),
+);
+
+Ok((HashSet::from_iter(roots), HashSet::from_iter(dirs), parents))
+}
+
 #[cfg(test)]
 mod tests {
 use super::*;
 use pretty_assertions::assert_eq;
+use std::path::Path;
+
+#[test]
+fn test_roots_and_dirs() {
+let pats = vec![
+IgnorePattern::new(PatternSyntax::Glob, b"g/h/*", Path::new("")),
+IgnorePattern::new(PatternSyntax::Glob, b"g/h", Path::new("")),
+IgnorePattern::new(PatternSyntax::Glob, b"g*", Path::new("")),
+];
+let (roots, dirs) = roots_and_dirs();
+
+assert_eq!(
+roots,
+vec!(
+HgPathBuf::from_bytes(b"g/h"),
+HgPathBuf::from_bytes(b"g/h"),
+HgPathBuf::new()
+),
+);
+assert_eq!(dirs, vec!());
+}
+
+#[test]
+fn test_roots_dirs_and_parents() {
+let pats = vec![
+IgnorePattern::new(PatternSyntax::Glob, b"g/h/*", Path::new("")),
+

D7894: nodemap: introduce an option to use mmap to read the nodemap mapping

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute added a subscriber: gracinet.


  In D7894#116397 , @martinvonz 
wrote:
  
  > How much does this patch help performance?
  > I would also like to see performance numbers (even just rough ones) for the 
Rust version. Sorry about a possibly stupid question, but why will this on-disk 
nodemap be faster than building it from the index? Is it that the file is 
smaller and thus faster to read? Or is it more the building of the tree than 
the reading that's slow? You mentioned you use some private repo for testing 
this. How large is the `00changelog.n` file in that repo and how large is 
`00changelog.i`?
  
  This save hundreds of milli second at initialization of large repositories. 
The repositories we are looking at are about 2 millions revisions. (but this 
will help smaller repository too). Mozilla try is a public repository in that 
range. It 00changelog.i is 103MB
  
  The information in the .i files is just a flat list of node. So anything that 
need a mapping needs to build it. Building a mapping for millions of revision 
is slow. (I think Georges mentionned 300ms to build the mozilla-try nodemap). 
The nodemap we write on disk is directly usage as such. So we just need to mmap 
the files (mostly instant is the repository have been busy recently, eg: on 
server) and directly query the data from disk. We will get more data once the 
latest version of this series and the latest version of the Rust series (from 
@gracinet) are connected again. But this greatly boost many operation (small 
operation, discovery, etc).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7894/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7894

To: marmoute, #hg-reviewers
Cc: gracinet, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7886: nodemap: introduce append-only incremental update of the persisten data

[marmoute (Pierre-Yves David)]

marmoute added inline comments.

INLINE COMMENTS

> martinvonz wrote in test-persistent-nodemap.t:53
> Which patch introduced the C code? I have not looked at all patches yet, but 
> I thought the idea was that this entire stack was just pure Python and then 
> you'd add a Rust version (not C) in a separate series.

There are no C code (and is non planned). This should be `the C path` or `the 
cext policy`.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7886/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7886

To: marmoute, #hg-reviewers
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7834: nodemap: have some python code serializing a nodemap

[marmoute (Pierre-Yves David)]

marmoute added inline comments.

INLINE COMMENTS

> martinvonz wrote in debugcommands.py:2084
> could you also rename `args` to `opts`? or is there a reason to not follow 
> the same convention we follow everywhere (?) else?

No good reason for this to not be `opts`. I'll rename it.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7834/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7834

To: marmoute, #hg-reviewers, martinvonz
Cc: martinvonz, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7914: rust-matchers: implement `visit_children_set` for `FileMatcher`

[Raphaël Gomès]

Alphare added inline comments.

INLINE COMMENTS

> martinvonz wrote in matchers.rs:166
> This will often be called repeatedly, so isn't it better to calculate a map 
> of each parent directory to its `VisitChildrenSet` value upfront (in 
> `FilesMatcher::new()`)?

Agreed. In much of this series there exist opportunities for caching/making 
things run in parallel, etc. With the freeze approaching really fast, I prefer 
to prioritize getting correct - albeit sub-optimal - code in rather than 
risking missing the deadline. 
My benchmarks of the entire series show an improvement in bare `hg status`  in 
all supported repositories (read: that don't have back-references in their 
`.hgignore patterns), and no measurable slowdown in others.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7914/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7914

To: Alphare, #hg-reviewers
Cc: martinvonz, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel