[PATCH 3 of 5] python3.13: address deprecation of re.sub positional argument 'count'

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1705006735 -3600
#  Thu Jan 11 21:58:55 2024 +0100
# Branch stable
# Node ID 8e16bc622b04e2eabb3a47138aa3bdffba03e142
# Parent  a06a7677696d8fa4fc3e33923425ef3fadd6f441
python3.13: address deprecation of re.sub positional argument 'count'

Python 3.13 introduced:

  DeprecationWarning: 'count' is passed as positional argument

Making it mandatory to pass 'count' as named argument reduces the risk of
passing 'flags' to it. That is exactly what happened in test-doctest.py .

diff --git a/mercurial/subrepoutil.py b/mercurial/subrepoutil.py
--- a/mercurial/subrepoutil.py
+++ b/mercurial/subrepoutil.py
@@ -112,7 +112,7 @@ def state(ctx, ui):
 # extra escapes are needed because re.sub string decodes.
 repl = re.sub(br'([0-9]+)', br'\\\1', repl)
 try:
-src = re.sub(pattern, repl, src, 1)
+src = re.sub(pattern, repl, src, count=1)
 except re.error as e:
 raise error.Abort(
 _(b"bad subrepository pattern in %s: %s")
diff --git a/tests/test-doctest.py b/tests/test-doctest.py
--- a/tests/test-doctest.py
+++ b/tests/test-doctest.py
@@ -21,9 +21,9 @@ class py3docchecker(doctest.OutputChecke
 r'''^mercurial\.\w+\.(\w+): (['"])(.*?)\2''',
 r'\1: \3',
 got2,
-re.MULTILINE,
+flags=re.MULTILINE,
 )
-got2 = re.sub(r'^mercurial\.\w+\.(\w+): ', r'\1: ', got2, re.MULTILINE)
+got2 = re.sub(r'^mercurial\.\w+\.(\w+): ', r'\1: ', got2, 
flags=re.MULTILINE)
 return any(
 doctest.OutputChecker.check_output(self, w, g, optionflags)
 for w, g in [(want, got), (want2, got2)]

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5] rust: update to toml 0.8

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1704999033 -3600
#  Thu Jan 11 19:50:33 2024 +0100
# Branch stable
# Node ID 646c00200241229c23dad41b1d086a610aff08b2
# Parent  f2e61759ac0e0125e275acc72bda8a00258762b9
rust: update to toml 0.8

This is needed for Fedora packaging.

diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -31,7 +31,7 @@ sha-1 = "0.10.0"
 twox-hash = "1.6.3"
 same-file = "1.0.6"
 tempfile = "3.3.0"
-toml = "0.6"
+toml = "0.8"
 thread_local = "1.1.4"
 crossbeam-channel = "0.5.6"
 log = "0.4.17"

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5] python3.13: fix resourceutil for removed deprecated importlib.resources

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1705001574 -3600
#  Thu Jan 11 20:32:54 2024 +0100
# Branch stable
# Node ID a06a7677696d8fa4fc3e33923425ef3fadd6f441
# Parent  ab3021e9b0012db64e5bdc70e3f5a36324925d8c
python3.13: fix resourceutil for removed deprecated importlib.resources

The old functionality was deprecated in 3.11 and is on track to be removed in
3.13 . The documentation on
https://docs.python.org/3.12/library/importlib.resources.html recommends using
the new .files() that was introduced in 3.9.

The pytype annotation is probably not preserved correctly.

diff --git a/mercurial/utils/resourceutil.py b/mercurial/utils/resourceutil.py
--- a/mercurial/utils/resourceutil.py
+++ b/mercurial/utils/resourceutil.py
@@ -60,8 +60,10 @@ try:
 
 # Force loading of the resources module
 if hasattr(resources, 'files'):
+# New in Python 3.9
 resources.files  # pytype: disable=module-attr
 else:
+# Deprecated in Python 3.11
 resources.open_binary  # pytype: disable=module-attr
 
 # py2exe raises an AssertionError if uses importlib.resources
@@ -109,12 +111,20 @@ else:
 )
 
 def is_resource(package, name):
-return resources.is_resource(  # pytype: disable=module-attr
-pycompat.sysstr(package), encoding.strfromlocal(name)
-)
+if hasattr(resources, 'files'):
+return 
resources.files(pycompat.sysstr(package)).joinpath(encoding.strfromlocal(name)).is_file()
+else:
+return resources.is_resource(  # pytype: disable=module-attr
+pycompat.sysstr(package), encoding.strfromlocal(name)
+)
 
 def contents(package):
 # pytype: disable=module-attr
-for r in resources.contents(pycompat.sysstr(package)):
-# pytype: enable=module-attr
-yield encoding.strtolocal(r)
+if hasattr(resources, 'files'):
+for resource in 
resources.files(pycompat.sysstr(package)).iterdir():
+if resource.is_file():
+yield encoding.strtolocal(path.name)
+else:
+for r in resources.contents(pycompat.sysstr(package)):
+# pytype: enable=module-attr
+yield encoding.strtolocal(r)

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5] python3.13: use sys.executable instead of removed Py_GetProgramFullPath

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1705001527 -3600
#  Thu Jan 11 20:32:07 2024 +0100
# Branch stable
# Node ID ab3021e9b0012db64e5bdc70e3f5a36324925d8c
# Parent  3f87e0d305cda6e66139a1969cd2cedd45477139
python3.13: use sys.executable instead of removed Py_GetProgramFullPath

I could not make it work with the PyConfig API from the extension. But fetching
sys.executable seems to work fine and isn't that verbose.

diff --git a/mercurial/cext/parsers.c b/mercurial/cext/parsers.c
--- a/mercurial/cext/parsers.c
+++ b/mercurial/cext/parsers.c
@@ -1232,6 +1232,15 @@ static int check_python_version(void)
 * should only occur in unusual circumstances (e.g. if sys.hexversion
 * is manually set to an invalid value). */
if ((hexversion == -1) || (hexversion >> 16 != PY_VERSION_HEX >> 16)) {
+   PyObject *sys = PyImport_ImportModule("sys"), *executable;
+   if (!sys) {
+   return -1;
+   }
+   executable = PyObject_GetAttrString(sys, "executable");
+   Py_DECREF(sys);
+   if (!executable) {
+   return -1;
+   }
PyErr_Format(PyExc_ImportError,
 "%s: The Mercurial extension "
 "modules were compiled with Python " PY_VERSION
@@ -1240,7 +1249,8 @@ static int check_python_version(void)
 "sys.hexversion=%ld: "
 "Python %s\n at: %s",
 versionerrortext, hexversion, Py_GetVersion(),
-Py_GetProgramFullPath());
+PyUnicode_AsUTF8(executable));
+   Py_DECREF(executable);
return -1;
}
return 0;

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 6] utils: stop using datetime.utcfromtimestamp() deprecated in Python 3.12

[Mads Kiilerich]

On 28/06/2023 16:43, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1687866710 -7200
#  Tue Jun 27 13:51:50 2023 +0200
# Branch stable
# Node ID ec46e9b39b2a6f5d0c59aa803e2eb5d9dfc44cc3
# Parent  2b0598121a71fa19c2174e4eee3400ec3a3b1c26
utils: stop using datetime.utcfromtimestamp() deprecated in Python 3.12

Python3.12 made tests fail with warnings:
   DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled 
for removal in a future version. Use timezone-aware objects to represent 
datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).

Computing the diff while in timestamp seconds seems to preserve to the original
intent from ae04af1ce78d.

It would be nice to have some doctest coverage of this, with the problematic
corner cases that has popped up over time...



Some potential test coverage that verified that the change preserve the 
fix from ae04af1ce78d:


    """Return a unix timestamp (or the current time) as a (unixtime,
    offset) tuple based off the local timezone.

    >>> import os, time
    >>> os.environ['TZ'] = 'Asia/Novokuznetsk'
    >>> time.tzset()

    >>> def dtu(*a):
    ...    return datetime.datetime(*a, tzinfo=datetime.timezone.utc)

    # Old winter timezone, +7
    >>> makedate(dtu(2010,  1,  1,  5,  0, 0).timestamp())
    (1262322000.0, -25200)

    # Same timezone in summer, +7, so no DST
    >>> makedate(dtu(2010,  7,  1,  5,  0, 0).timestamp())
    (1277960400.0, -25200)

    # Changing to new winter timezone, from +7 to +6 (ae04af1ce78d 
testcase)

    >>> makedate(dtu(2010, 10, 30, 20,  0, 0).timestamp() - 1)
    (1288468799.0, -25200)
    >>> makedate(dtu(2010, 10, 30, 20,  0, 0).timestamp())
    (1288468800.0, -21600)
    >>> makedate(dtu(2011,  1,  1,  5,  0, 0).timestamp())
    (1293858000.0, -21600)

    # Introducing DST, changing +6 to +7
    >>> makedate(dtu(2011,  3, 26, 20,  0, 0).timestamp() - 1)
    (1301169599.0, -21600)
    >>> makedate(dtu(2011,  3, 26, 20,  0, 0).timestamp())
    (1301169600.0, -25200)
    """

but it relies heavily on a corner case in the time database and setting 
the timezone globally, so it might be fragile and not a good candidate 
for adding to the test suite. The functions are already accidentally 
covered by the suite.


/Mads
___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 6] extensions: imp module is removed in Python 3.12 - use importlib to load files

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687954993 -7200
#  Wed Jun 28 14:23:13 2023 +0200
# Branch stable
# Node ID 24d8509e44785f94047efebbcbb38986323913cd
# Parent  f45267a4d61f7687727efa7fe571d4f2522a3bb6
extensions: imp module is removed in Python 3.12 - use importlib to load files

imp has been deprecated for a long time, and has finally been removed in Python
3.12 .

imp was only used for loading extensions that has been specified with direct
.py path or path to a package directory. The same use cases can be achieved
quite simple with importlib, , possiby with small changes in corner cases with
undefined behaviour, such as extensions without .py source.

There might also be corner cases and undefined behaviour around use of
sys.modules and reloading.

diff --git a/mercurial/extensions.py b/mercurial/extensions.py
--- a/mercurial/extensions.py
+++ b/mercurial/extensions.py
@@ -9,9 +9,10 @@
 import ast
 import collections
 import functools
-import imp
+import importlib
 import inspect
 import os
+import sys
 
 from .i18n import (
 _,
@@ -89,20 +90,17 @@ def loadpath(path, module_name):
 path = pycompat.fsdecode(path)
 if os.path.isdir(path):
 # module/__init__.py style
-d, f = os.path.split(path)
-fd, fpath, desc = imp.find_module(f, [d])
-# When https://github.com/python/typeshed/issues/3466 is fixed
-# and in a pytype release we can drop this disable.
-return imp.load_module(
-module_name, fd, fpath, desc  # pytype: disable=wrong-arg-types
-)
-else:
-try:
-return imp.load_source(module_name, path)
-except IOError as exc:
-if not exc.filename:
-exc.filename = path  # python does not fill this
-raise
+init_py_path = os.path.join(path, '__init__.py')
+if not os.path.exists(init_py_path):
+raise ImportError("No module named '%s'" % os.path.basename(path))
+path = init_py_path
+
+loader = importlib.machinery.SourceFileLoader(module_name, path)
+spec = importlib.util.spec_from_file_location(module_name, loader=loader)
+module = importlib.util.module_from_spec(spec)
+sys.modules[module_name] = module
+spec.loader.exec_module(module)
+return module
 
 
 def _importh(name):

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 6] utils: imp module is removed in Python 3.12 - get is_frozen() from _imp

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687863903 -7200
#  Tue Jun 27 13:05:03 2023 +0200
# Branch stable
# Node ID f45267a4d61f7687727efa7fe571d4f2522a3bb6
# Parent  85a1bdb7d945c647670761676c97ce716fe3796d
utils: imp module is removed in Python 3.12 - get is_frozen() from _imp

imp has been deprecated for a long time, and has finally been removed in Python
3.12 .

The successor importlib is using the same internal _imp module as imp, but
doesn't expose it's is_frozen. Using the internal function directly seems like
the cleanest solution.

Another alternative to
  imp.is_frozen("__main__")
is
  sys.modules['__main__'].__spec__.origin == 'frozen'
but that seems even more internal and fragile.

diff --git a/mercurial/utils/resourceutil.py b/mercurial/utils/resourceutil.py
--- a/mercurial/utils/resourceutil.py
+++ b/mercurial/utils/resourceutil.py
@@ -8,7 +8,7 @@
 # GNU General Public License version 2 or any later version.
 
 
-import imp
+import _imp
 import os
 import sys
 
@@ -24,7 +24,7 @@ def mainfrozen():
 return (
 pycompat.safehasattr(sys, "frozen")  # new py2exe
 or pycompat.safehasattr(sys, "importers")  # old py2exe
-or imp.is_frozen("__main__")  # tools/freeze
+or _imp.is_frozen("__main__")  # tools/freeze
 )
 
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 6] extensions: address ast deprecations introduced in Python 3.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687897904 -7200
#  Tue Jun 27 22:31:44 2023 +0200
# Branch stable
# Node ID 85a1bdb7d945c647670761676c97ce716fe3796d
# Parent  b6633799949e428d27f7704636da0da31a599e6b
extensions: address ast deprecations introduced in Python 3.12

Tests would fail with:
  .../mercurial/extensions.py:910: DeprecationWarning: ast.Str is deprecated 
and will be removed in Python 3.14; use ast.Constant instead
if isinstance(a, ast.Str):
  .../mercurial/extensions.py:912: DeprecationWarning: ast.Bytes is deprecated 
and will be removed in Python 3.14; use ast.Constant instead
elif isinstance(a, ast.Bytes):
  .../mercurial/extensions.py:913: DeprecationWarning: Attribute s is 
deprecated and will be removed in Python 3.14; use value instead
name = a.s

diff --git a/mercurial/extensions.py b/mercurial/extensions.py
--- a/mercurial/extensions.py
+++ b/mercurial/extensions.py
@@ -885,16 +885,31 @@ def _disabledcmdtable(path):
 with open(path, b'rb') as src:
 root = ast.parse(src.read(), path)
 cmdtable = {}
+
+# Python 3.12 started removing Bytes and Str and deprecate harder
+use_constant = 'Bytes' not in vars(ast)
+
 for node in _walkcommand(root):
 if not node.args:
 continue
 a = node.args[0]
-if isinstance(a, ast.Str):
-name = pycompat.sysbytes(a.s)
-elif isinstance(a, ast.Bytes):
-name = a.s
-else:
-continue
+if use_constant:  # Valid since Python 3.8
+if isinstance(a, ast.Constant):
+if isinstance(a.value, str):
+name = pycompat.sysbytes(a.value)
+elif isinstance(a.value, bytes):
+name = a.value
+else:
+continue
+else:
+continue
+else:  # Valid until 3.11
+if isinstance(a, ast.Str):
+name = pycompat.sysbytes(a.s)
+elif isinstance(a, ast.Bytes):
+name = a.s
+else:
+continue
 cmdtable[name] = (None, [], b'')
 return cmdtable
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 6] vfs: handle shutil.rmtree deprecation of onerror in Python 3.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687847952 -7200
#  Tue Jun 27 08:39:12 2023 +0200
# Branch stable
# Node ID b6633799949e428d27f7704636da0da31a599e6b
# Parent  27c7a6d21b9dae4d465a569480331d2467b423f2
vfs: handle shutil.rmtree deprecation of onerror in Python 3.12

Tests would fail with warnings:

  .../mercurial/vfs.py:289: DeprecationWarning: onerror argument is deprecated, 
use onexc instead

The excinfo changed slightly, but we don't use it anyway.

diff --git a/mercurial/vfs.py b/mercurial/vfs.py
--- a/mercurial/vfs.py
+++ b/mercurial/vfs.py
@@ -274,7 +274,7 @@ class abstractvfs:
 """
 if forcibly:
 
-def onerror(function, path, excinfo):
+def onexc(function, path, excinfo):
 if function is not os.remove:
 raise
 # read-only files cannot be unlinked under Windows
@@ -285,10 +285,15 @@ class abstractvfs:
 os.remove(path)
 
 else:
-onerror = None
-return shutil.rmtree(
-self.join(path), ignore_errors=ignore_errors, onerror=onerror
-)
+onexc = None
+try:
+return shutil.rmtree(
+self.join(path), ignore_errors=ignore_errors, onexc=onexc
+)
+except TypeError:  # onexc was introduced in Python 3.12
+return shutil.rmtree(
+self.join(path), ignore_errors=ignore_errors, onerror=onexc
+)
 
 def setflags(self, path: bytes, l: bool, x: bool):
 return util.setflags(self.join(path), l, x)

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 6] tests: fix sortdict doctest with Python 3.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687853351 -7200
#  Tue Jun 27 10:09:11 2023 +0200
# Branch stable
# Node ID 27c7a6d21b9dae4d465a569480331d2467b423f2
# Parent  ec46e9b39b2a6f5d0c59aa803e2eb5d9dfc44cc3
tests: fix sortdict doctest with Python 3.12

The output of OrderedDict changed to use plain dict syntax:

  $ python3.11 -c "import collections;print(collections.OrderedDict([('a', 0), 
('b', 1)]))"
  OrderedDict([('a', 0), ('b', 1)])

  $ python3.12 -c "import collections;print(collections.OrderedDict([('a', 0), 
('b', 1)]))"
  OrderedDict({'a': 0, 'b': 1})

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -1277,14 +1277,14 @@ class sortdict(collections.OrderedDict):
 
 >>> d1 = sortdict([(b'a', 0), (b'b', 1)])
 >>> d2 = d1.copy()
->>> d2
-sortdict([('a', 0), ('b', 1)])
+>>> list(d2.items())
+[('a', 0), ('b', 1)]
 >>> d2.update([(b'a', 2)])
 >>> list(d2.keys()) # should still be in last-set order
 ['b', 'a']
 >>> d1.insert(1, b'a.5', 0.5)
->>> d1
-sortdict([('a', 0), ('a.5', 0.5), ('b', 1)])
+>>> list(d1.items())
+[('a', 0), ('a.5', 0.5), ('b', 1)]
 """
 
 def __setitem__(self, key, value):

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 6] utils: stop using datetime.utcfromtimestamp() deprecated in Python 3.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687866710 -7200
#  Tue Jun 27 13:51:50 2023 +0200
# Branch stable
# Node ID ec46e9b39b2a6f5d0c59aa803e2eb5d9dfc44cc3
# Parent  2b0598121a71fa19c2174e4eee3400ec3a3b1c26
utils: stop using datetime.utcfromtimestamp() deprecated in Python 3.12

Python3.12 made tests fail with warnings:
  DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled 
for removal in a future version. Use timezone-aware objects to represent 
datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).

Computing the diff while in timestamp seconds seems to preserve to the original
intent from ae04af1ce78d.

It would be nice to have some doctest coverage of this, with the problematic
corner cases that has popped up over time...

diff --git a/hgext/convert/common.py b/hgext/convert/common.py
--- a/hgext/convert/common.py
+++ b/hgext/convert/common.py
@@ -567,8 +567,10 @@ class mapfile(dict):
 
 def makedatetimestamp(t):
 """Like dateutil.makedate() but for time t instead of current time"""
-delta = datetime.datetime.utcfromtimestamp(
+tz = round(
 t
-) - datetime.datetime.fromtimestamp(t)
-tz = delta.days * 86400 + delta.seconds
+- datetime.datetime.fromtimestamp(t)
+.replace(tzinfo=datetime.timezone.utc)
+.timestamp()
+)
 return t, tz
diff --git a/mercurial/utils/dateutil.py b/mercurial/utils/dateutil.py
--- a/mercurial/utils/dateutil.py
+++ b/mercurial/utils/dateutil.py
@@ -83,10 +83,14 @@ def makedate(timestamp=None):
 raise error.InputError(
 _(b"negative timestamp: %d") % timestamp, hint=hint
 )
-delta = datetime.datetime.utcfromtimestamp(
+tz = round(
 timestamp
-) - datetime.datetime.fromtimestamp(timestamp)
-tz = delta.days * 86400 + delta.seconds
+- datetime.datetime.fromtimestamp(
+timestamp,
+)
+.replace(tzinfo=datetime.timezone.utc)
+.timestamp()
+)
 return timestamp, tz
 
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 10] hgweb: drop references to deprecated cgitb

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687795228 -7200
#  Mon Jun 26 18:00:28 2023 +0200
# Branch stable
# Node ID 6ca3c985d3fe233bde88057e1a04f6c2c5ab71fd
# Parent  5a32d6ab784657d51dc02c9e86ad92d4efd21ee8
hgweb: drop references to deprecated cgitb

cgitb is going away and gives warnings when importing, and that make tests
fail:
  $TESTTMP/hgweb.cgi:5: DeprecationWarning: 'cgitb' is deprecated and slated 
for removal in Python 3.13

The lack of a "nice" high level error handler is not a huge problem, neither
for users (where it is disabled anyway) or for tests (where we don't use a
browser and the plain tracebacks often are more readable). It is inevitable
that it is going away, and there is no obvious alternative. Remove it and move
on.

diff --git a/contrib/hgweb.fcgi b/contrib/hgweb.fcgi
--- a/contrib/hgweb.fcgi
+++ b/contrib/hgweb.fcgi
@@ -9,9 +9,6 @@ config = b"/path/to/repo/or/config"
 # (consult "installed modules" path from 'hg debuginstall'):
 # import sys; sys.path.insert(0, "/path/to/python/lib")
 
-# Uncomment to send python tracebacks to the browser if an error occurs:
-# import cgitb; cgitb.enable()
-
 from mercurial import demandimport
 
 demandimport.enable()
diff --git a/contrib/hgweb.wsgi b/contrib/hgweb.wsgi
--- a/contrib/hgweb.wsgi
+++ b/contrib/hgweb.wsgi
@@ -8,9 +8,6 @@ config = b"/path/to/repo/or/config"
 # (consult "installed modules" path from 'hg debuginstall'):
 #import sys; sys.path.insert(0, "/path/to/python/lib")
 
-# Uncomment to send python tracebacks to the browser if an error occurs:
-#import cgitb; cgitb.enable()
-
 # enable demandloading to reduce startup time
 from mercurial import demandimport; demandimport.enable()
 
diff --git a/hgweb.cgi b/hgweb.cgi
--- a/hgweb.cgi
+++ b/hgweb.cgi
@@ -10,9 +10,6 @@ config = b"/path/to/repo/or/config"
 # (consult "installed modules" path from 'hg debuginstall'):
 # import sys; sys.path.insert(0, "/path/to/python/lib")
 
-# Uncomment to send python tracebacks to the browser if an error occurs:
-# import cgitb; cgitb.enable()
-
 from mercurial import demandimport
 
 demandimport.enable()
diff --git a/tests/test-clone-cgi.t b/tests/test-clone-cgi.t
--- a/tests/test-clone-cgi.t
+++ b/tests/test-clone-cgi.t
@@ -12,8 +12,6 @@ initialize repository
   $ cat >hgweb.cgi < #
   > # An example CGI script to use hgweb, edit as necessary
-  > import cgitb
-  > cgitb.enable()
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgweb
   > from mercurial.hgweb import wsgicgi
diff --git a/tests/test-mq.t b/tests/test-mq.t
--- a/tests/test-mq.t
+++ b/tests/test-mq.t
@@ -1581,8 +1581,6 @@ Test that secret mq patch does not break
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgweb
   > from mercurial.hgweb import wsgicgi
-  > import cgitb
-  > cgitb.enable()
   > app = hgweb(b'.', b'test')
   > wsgicgi.launch(app)
   > HGWEB
diff --git a/tests/test-newcgi.t b/tests/test-newcgi.t
--- a/tests/test-newcgi.t
+++ b/tests/test-newcgi.t
@@ -9,9 +9,6 @@ before d74fc8dec2b4 still work.
   > #
   > # An example CGI script to use hgweb, edit as necessary
   > 
-  > import cgitb
-  > cgitb.enable()
-  > 
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgweb
   > from mercurial.hgweb import wsgicgi
@@ -35,9 +32,6 @@ before d74fc8dec2b4 still work.
   > #
   > # An example CGI script to export multiple hgweb repos, edit as necessary
   > 
-  > import cgitb
-  > cgitb.enable()
-  > 
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgwebdir
   > from mercurial.hgweb import wsgicgi
diff --git a/tests/test-newercgi.t b/tests/test-newercgi.t
--- a/tests/test-newercgi.t
+++ b/tests/test-newercgi.t
@@ -9,9 +9,6 @@ This is a rudimentary test of the CGI fi
   > #
   > # An example CGI script to use hgweb, edit as necessary
   > 
-  > import cgitb
-  > cgitb.enable()
-  > 
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgweb
   > from mercurial.hgweb import wsgicgi
@@ -32,9 +29,6 @@ This is a rudimentary test of the CGI fi
   > #
   > # An example CGI script to export multiple hgweb repos, edit as necessary
   > 
-  > import cgitb
-  > cgitb.enable()
-  > 
   > from mercurial import demandimport; demandimport.enable()
   > from mercurial.hgweb import hgwebdir
   > from mercurial.hgweb import wsgicgi
diff --git a/tests/test-oldcgi.t b/tests/test-oldcgi.t
--- a/tests/test-oldcgi.t
+++ b/tests/test-oldcgi.t
@@ -8,8 +8,7 @@ This tests if CGI files from before d0db
   > #
   > # An example CGI script to use hgweb, edit as necessary
   > 
-  > import cgitb, os,

[PATCH 09 of 10] setup: fall back to setuptools setup if distutils isn't available

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687860557 -7200
#  Tue Jun 27 12:09:17 2023 +0200
# Branch stable
# Node ID 5a32d6ab784657d51dc02c9e86ad92d4efd21ee8
# Parent  7d0800b9c059349f6ad373215e718ddc7455ee91
setup: fall back to setuptools setup if distutils isn't available

The setuptools comments around this seems slightly outdated. Setuptools is
improving and distutils is being deprecated, so it should perhaps be the
default. But at least, it is a fair fallback.

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -112,7 +112,10 @@ issetuptools = os.name == 'nt' or 'FORCE
 if issetuptools:
 from setuptools import setup
 else:
-from distutils.core import setup
+try:
+from distutils.core import setup
+except ModuleNotFoundError:
+from setuptools import setup
 from distutils.ccompiler import new_compiler
 from distutils.core import Command, Extension
 from distutils.dist import Distribution

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10] tests: fix dummysmtpd argument check

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687790713 -7200
#  Mon Jun 26 16:45:13 2023 +0200
# Branch stable
# Node ID 0fc04b7dda3c22b6eda5385e59be8307f883a88e
# Parent  22a8aa225c38766ddd29c51348dd4484b5e58f59
tests: fix dummysmtpd argument check

diff --git a/tests/dummysmtpd.py b/tests/dummysmtpd.py
--- a/tests/dummysmtpd.py
+++ b/tests/dummysmtpd.py
@@ -99,8 +99,8 @@ def main():
 op.add_option('--logfile', metavar='FILE')
 
 opts, args = op.parse_args()
-if opts.tls == 'smtps' and not opts.certificate:
-op.error('--certificate must be specified')
+if (opts.tls == 'smtps') != bool(opts.certificate):
+op.error('--certificate must be specified with --tls=smtps')
 
 addr = (opts.address, opts.port)
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 10] tests: use simple mock smtp server instead of deprecated asyncore smtpd

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679586312 -3600
#  Thu Mar 23 16:45:12 2023 +0100
# Branch stable
# Node ID 7d0800b9c059349f6ad373215e718ddc7455ee91
# Parent  0fc04b7dda3c22b6eda5385e59be8307f883a88e
tests: use simple mock smtp server instead of deprecated asyncore smtpd

test-patchbomb-tls.t would fail with:
  .../hg/tests/dummysmtpd.py:6: DeprecationWarning: The asyncore module is 
deprecated and will be removed in Python 3.12. The recommended replacement is 
asyncio
import asyncore
  .../hg/tests/dummysmtpd.py:8: DeprecationWarning: The smtpd module is 
deprecated and unmaintained and will be removed in Python 3.12.  Please see 
aiosmtpd (https://aiosmtpd.readthedocs.io/) for the recommended replacement.
import smtpd

The recommended migration path to the standalone asiosmtpd would be overkill.

The tests do not need a full smtp server - we can just use a very simple mock
hack to preserve the existing test coverage.

diff --git a/tests/dummysmtpd.py b/tests/dummysmtpd.py
--- a/tests/dummysmtpd.py
+++ b/tests/dummysmtpd.py
@@ -3,12 +3,11 @@
 """dummy SMTP server for use in tests"""
 
 
-import asyncore
 import optparse
-import smtpd
+import os
+import socket
 import ssl
 import sys
-import traceback
 
 from mercurial import (
 pycompat,
@@ -18,57 +17,97 @@ from mercurial import (
 )
 
 
+if os.environ.get('HGIPV6', '0') == '1':
+family = socket.AF_INET6
+else:
+family = socket.AF_INET
+
+
 def log(msg):
 sys.stdout.write(msg)
 sys.stdout.flush()
 
 
-class dummysmtpserver(smtpd.SMTPServer):
-def __init__(self, localaddr):
-smtpd.SMTPServer.__init__(self, localaddr, remoteaddr=None)
+def mocksmtpserversession(conn, addr):
+conn.send(b'220 smtp.example.com ESMTP\r\n')
+
+line = conn.recv(1024)
+if not line.lower().startswith(b'ehlo '):
+log('no hello: %s\n' % line)
+return
+
+conn.send(b'250 Hello\r\n')
+
+line = conn.recv(1024)
+if not line.lower().startswith(b'mail from:'):
+log('no mail from: %s\n' % line)
+return
+mailfrom = line[10:].decode().rstrip()
+if mailfrom.startswith('<') and mailfrom.endswith('>'):
+mailfrom = mailfrom[1:-1]
+
+conn.send(b'250 Ok\r\n')
 
-def process_message(self, peer, mailfrom, rcpttos, data, **kwargs):
-log(
-'%s from=%s to=%s\n%s\n'
-% (peer[0], mailfrom, ', '.join(rcpttos), data.decode())
-)
+rcpttos = []
+while True:
+line = conn.recv(1024)
+if not line.lower().startswith(b'rcpt to:'):
+break
+rcptto = line[8:].decode().rstrip()
+if rcptto.startswith('<') and rcptto.endswith('>'):
+rcptto = rcptto[1:-1]
+rcpttos.append(rcptto)
+
+conn.send(b'250 Ok\r\n')
+
+if not line.lower().strip() == b'data':
+log('no rcpt to or data: %s' % line)
+
+conn.send(b'354 Go ahead\r\n')
 
-def handle_error(self):
-# On Windows, a bad SSL connection sometimes generates a WSAECONNRESET.
-# The default handler will shutdown this server, and then both the
-# current connection and subsequent ones fail on the client side with
-# "No connection could be made because the target machine actively
-# refused it".  If we eat the error, then the client properly aborts in
-# the expected way, and the server is available for subsequent 
requests.
-traceback.print_exc()
+data = b''
+while True:
+line = conn.recv(1024)
+if not line:
+log('connection closed before end of data')
+break
+data += line
+if data.endswith(b'\r\n.\r\n'):
+data = data[:-5]
+break
+
+conn.send(b'250 Ok\r\n')
+
+log(
+'%s from=%s to=%s\n%s\n'
+% (addr[0], mailfrom, ', '.join(rcpttos), data.decode())
+)
 
 
-class dummysmtpsecureserver(dummysmtpserver):
-def __init__(self, localaddr, certfile):
-dummysmtpserver.__init__(self, localaddr)
-self._certfile = certfile
-
-def handle_accept(self):
-pair = self.accept()
-if not pair:
-return
-conn, addr = pair
-ui = uimod.ui.load()
+def run(host, port, certificate):
+ui = uimod.ui.load()
+with socket.socket(family, socket.SOCK_STREAM) as s:
+s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+s.bind((host, port))
+# log('listening at %s:%d\n' % (host, port))
+s.listen(1)
 try:
-# wrap_socket() would block, but we don't care
-conn = sslutil.wrapserversocket(conn, ui, certfile=self._certfile)
-except ssl.SSLError as e:
-log('%s ssl error: %s\n' % (addr[0], e))
-conn.close()
-return
-smtpd.SMTPChannel(self, conn, addr)
-
-
-def run():
-try:
-asyncore.loop()
-except Keyb

[PATCH 05 of 10] tests: show test-patchbomb-tls.t smtp server log

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679586524 -3600
#  Thu Mar 23 16:48:44 2023 +0100
# Branch stable
# Node ID 54a4e277ff2c32c7fd67bebfb04cac25034cc2b2
# Parent  8687d17528a1fe764572c402f2d0198d5a6c8fbc
tests: show test-patchbomb-tls.t smtp server log

Improve test coverage by exposing what the smtp server actually receives.
Make dummystmtpd redirect stderr to a log file.

diff --git a/tests/dummysmtpd.py b/tests/dummysmtpd.py
--- a/tests/dummysmtpd.py
+++ b/tests/dummysmtpd.py
@@ -93,6 +93,7 @@ def main():
 op.add_option('--pid-file', metavar='FILE')
 op.add_option('--tls', choices=['none', 'smtps'], default='none')
 op.add_option('--certificate', metavar='FILE')
+op.add_option('--logfile', metavar='FILE')
 
 opts, args = op.parse_args()
 if opts.tls == 'smtps' and not opts.certificate:
@@ -113,6 +114,7 @@ def main():
 runfn=run,
 runargs=[pycompat.sysexecutable, pycompat.fsencode(__file__)]
 + pycompat.sysargv[1:],
+logfile=opts.logfile,
 )
 
 
diff --git a/tests/test-patchbomb-tls.t b/tests/test-patchbomb-tls.t
--- a/tests/test-patchbomb-tls.t
+++ b/tests/test-patchbomb-tls.t
@@ -5,7 +5,7 @@ Set up SMTP server:
   $ CERTSDIR="$TESTDIR/sslcerts"
   $ cat "$CERTSDIR/priv.pem" "$CERTSDIR/pub.pem" >> server.pem
 
-  $ "$PYTHON" "$TESTDIR/dummysmtpd.py" -p $HGPORT --pid-file a.pid -d \
+  $ "$PYTHON" "$TESTDIR/dummysmtpd.py" -p $HGPORT --pid-file a.pid --logfile 
log -d \
   > --tls smtps --certificate `pwd`/server.pem
   listening at localhost:$HGPORT (?)
   $ cat a.pid >> $DAEMON_PIDS
@@ -75,6 +75,10 @@ Without certificates:
   (see https://mercurial-scm.org/wiki/SecureConnections for how to configure 
Mercurial to avoid this error or set 
hostsecurity.localhost:fingerprints=sha256:20:de:b3:ad:b4:cd:a5:42:f0:74:41:1c:a2:70:1e:da:6e:c0:5c:16:9e:e7:22:0f:f1:b7:e5:6e:e4:92:af:7e
 to trust this server)
   [150]
 
+  $ cat ../log
+  * ssl error (glob)
+  $ : > ../log
+
 With global certificates:
 
   $ try --debug --config web.cacerts="$CERTSDIR/pub.pem"
@@ -86,6 +90,10 @@ With global certificates:
   (verifying remote certificate)
   sending [PATCH] a ...
 
+  $ cat ../log
+  * from=quux to=foo, bar (glob)
+  $ : > ../log
+
 With invalid certificates:
 
   $ try --config web.cacerts="$CERTSDIR/pub-other.pem"
@@ -96,4 +104,8 @@ With invalid certificates:
   (?i)abort: .*?certificate.verify.failed.* (re)
   [255]
 
+  $ cat ../log
+  * ssl error (glob)
+  $ : > ../log
+
   $ cd ..

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10] tests: improve test-patchbomb-tls.t by by logging errors and data

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687787499 -7200
#  Mon Jun 26 15:51:39 2023 +0200
# Branch stable
# Node ID 22a8aa225c38766ddd29c51348dd4484b5e58f59
# Parent  54a4e277ff2c32c7fd67bebfb04cac25034cc2b2
tests: improve test-patchbomb-tls.t by by logging errors and data

The actual SSL error might be like:
  ::1 ssl error: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca 
(_ssl.c:1002)
and will probably vary so much that it can't be checked in the test. It is
however very useful when debugging failures.

diff --git a/tests/dummysmtpd.py b/tests/dummysmtpd.py
--- a/tests/dummysmtpd.py
+++ b/tests/dummysmtpd.py
@@ -28,7 +28,10 @@ class dummysmtpserver(smtpd.SMTPServer):
 smtpd.SMTPServer.__init__(self, localaddr, remoteaddr=None)
 
 def process_message(self, peer, mailfrom, rcpttos, data, **kwargs):
-log('%s from=%s to=%s\n' % (peer[0], mailfrom, ', '.join(rcpttos)))
+log(
+'%s from=%s to=%s\n%s\n'
+% (peer[0], mailfrom, ', '.join(rcpttos), data.decode())
+)
 
 def handle_error(self):
 # On Windows, a bad SSL connection sometimes generates a WSAECONNRESET.
@@ -54,8 +57,8 @@ class dummysmtpsecureserver(dummysmtpser
 try:
 # wrap_socket() would block, but we don't care
 conn = sslutil.wrapserversocket(conn, ui, certfile=self._certfile)
-except ssl.SSLError:
-log('%s ssl error\n' % addr[0])
+except ssl.SSLError as e:
+log('%s ssl error: %s\n' % (addr[0], e))
 conn.close()
 return
 smtpd.SMTPChannel(self, conn, addr)
diff --git a/tests/test-patchbomb-tls.t b/tests/test-patchbomb-tls.t
--- a/tests/test-patchbomb-tls.t
+++ b/tests/test-patchbomb-tls.t
@@ -76,7 +76,7 @@ Without certificates:
   [150]
 
   $ cat ../log
-  * ssl error (glob)
+  * ssl error: * (glob)
   $ : > ../log
 
 With global certificates:
@@ -92,6 +92,35 @@ With global certificates:
 
   $ cat ../log
   * from=quux to=foo, bar (glob)
+  MIME-Version: 1.0
+  Content-Type: text/plain; charset="us-ascii"
+  Content-Transfer-Encoding: 7bit
+  Subject: [PATCH] a
+  X-Mercurial-Node: 8580ff50825a50c8f716709acdf8de0deddcd6ab
+  X-Mercurial-Series-Index: 1
+  X-Mercurial-Series-Total: 1
+  Message-Id: <*@test-hostname> (glob)
+  X-Mercurial-Series-Id: <*@test-hostname> (glob)
+  User-Agent: Mercurial-patchbomb* (glob)
+  Date: * (glob)
+  From: quux
+  To: foo
+  Cc: bar
+  
+  # HG changeset patch
+  # User test
+  # Date 1 0
+  #  Thu Jan 01 00:00:01 1970 +
+  # Node ID 8580ff50825a50c8f716709acdf8de0deddcd6ab
+  # Parent  
+  a
+  
+  diff -r  -r 
8580ff50825a50c8f716709acdf8de0deddcd6ab a
+  --- /dev/nullThu Jan 01 00:00:00 1970 +
+  +++ b/a  Thu Jan 01 00:00:01 1970 +
+  @@ -0,0 +1,1 @@
+  +a
+  
   $ : > ../log
 
 With invalid certificates:
@@ -105,7 +134,7 @@ With invalid certificates:
   [255]
 
   $ cat ../log
-  * ssl error (glob)
+  * ssl error: * (glob)
   $ : > ../log
 
   $ cd ..

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 10] demandimport: don't delay _distutils_hack import

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687908670 -7200
#  Wed Jun 28 01:31:10 2023 +0200
# Branch stable
# Node ID 8687d17528a1fe764572c402f2d0198d5a6c8fbc
# Parent  deb45f7a7f7bdc36c36f7bfb63c36701b44972ab
demandimport: don't delay _distutils_hack import

test-demandimport.py would fail on 'import distutils.msvc9compiler' because
warnings:
  /usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:18: 
UserWarning: Distutils was imported before Setuptools, but importing Setuptools 
also replaces the `distutils` module in `sys.modules`. This may lead to 
undesirable behaviors or errors. To avoid these issues, avoid using distutils 
directly, ensure that setuptools is installed in the traditional way (e.g. not 
an editable install), and/or make sure that setuptools is always imported 
before distutils.
warnings.warn(
  /usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: 
UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")

Telling demandimport to ignore this module will allow the hack to work as
intended.

Note:

The test for distutils.msvc9compiler comes from 2205d00b6d2b. But since then,
distutils is going away, and setuptools has moved forward and is replacing it.
It is unclear exactly what is being tested here and how setuptools should
depended on msvc9compiler. The test might no longer be relevant.

diff --git a/hgdemandimport/__init__.py b/hgdemandimport/__init__.py
--- a/hgdemandimport/__init__.py
+++ b/hgdemandimport/__init__.py
@@ -55,6 +55,9 @@ IGNORES = {
 'builtins',
 'urwid.command_map',  # for pudb
 'lzma',
+# setuptools uses this hack to inject it's own distutils at import time
+'setuptools',
+'_distutils_hack.override',
 }
 
 _pypy = '__pypy__' in sys.builtin_module_names

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 10] tests: use grep -F instead of obsolescent fgrep

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1687784040 -7200
#  Mon Jun 26 14:54:00 2023 +0200
# Branch stable
# Node ID 141a32ae6e30dc9bdf0e742f240734afa37f3054
# Parent  2790b07cd5da2be40fd5299ae114a49eb6196e55
tests: use grep -F instead of obsolescent fgrep

Testing on Fedora 38 failed with:
  fgrep: warning: fgrep is obsolescent; using grep -F

The warning comes from
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1
. For further anecdotal evidence of the change, see
https://www.phoronix.com/news/GNU-Grep-3.8-Stop-egrep-fgrep .

grep -F is POSIX, but there is a risk that it doesn't work the same on all
platforms - especially older Unix versions. It should however always be
possible to put a GNU grep in $PATH before running the tests.

diff --git a/contrib/check-code.py b/contrib/check-code.py
--- a/contrib/check-code.py
+++ b/contrib/check-code.py
@@ -147,6 +147,7 @@ testpats = [
 '[ foo == bar ] is a bashism, use [ foo = bar ] instead',
 ),
 (r'(^|\|\s*)egrep', "use grep -E for extended grep syntax"),
+(r'(^|\|\s*)fgrep', "use grep -F for fixed string grepping"),
 (r'(^|\|\s*)e?grep .*\\S', "don't use \\S in regular expression"),
 (r'(?> test
-  $ hg pull -u 2>&1 | fgrep -v TESTTMP| fgrep -v "searching for changes" | 
fgrep -v adding
+  $ hg pull -u 2>&1 | grep -F -v TESTTMP| grep -F -v "searching for changes" | 
grep -F -v adding
   pulling from $TESTTMP/a
   updating bookmark X
   added 1 changesets with 0 changes to 0 files (+1 heads)

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 10] tests: update test-remotefilelog-gc.t for Python 3.11

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679497559 -3600
#  Wed Mar 22 16:05:59 2023 +0100
# Branch stable
# Node ID deb45f7a7f7bdc36c36f7bfb63c36701b44972ab
# Parent  141a32ae6e30dc9bdf0e742f240734afa37f3054
tests: update test-remotefilelog-gc.t for Python 3.11

The test output changed because test coverage changed because normpath changed:

  $ python3.10 -c 'import os; print(repr(os.path.normpath("asdas\0das")))'
  'asdas\x00das'

  $ python3.11 -c 'import os; print(repr(os.path.normpath("asdas\0das")))'
  'asdas'

diff --git a/tests/test-remotefilelog-gc.t b/tests/test-remotefilelog-gc.t
--- a/tests/test-remotefilelog-gc.t
+++ b/tests/test-remotefilelog-gc.t
@@ -106,6 +106,11 @@
 # Test that warning is displayed when the repo path is malformed
 
   $ printf "asdas\0das" >> $CACHEDIR/repos
+#if py311
+  $ hg gc
+  finished: removed 0 of 4 files (0.00 GB to 0.00 GB)
+#else
   $ hg gc
   abort: invalid path asdas\x00da: .*(null|NULL).* (re)
   [255]
+#endif

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5 stable] tests: update test-remotefilelog-gc.t for Python 3.11

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679497559 -3600
#  Wed Mar 22 16:05:59 2023 +0100
# Branch stable
# Node ID 2ba2e699e4c38398ab4d408e7cd3d3f148ea11c1
# Parent  7c544bc71aaca594998649ae02d35fb4dd7606b4
tests: update test-remotefilelog-gc.t for Python 3.11

The test output changed because test coverage changed because normpath changed:

  $ python3.10 -c 'import os; print(repr(os.path.normpath("asdas\0das")))'
  'asdas\x00das'

  $ python3.11 -c 'import os; print(repr(os.path.normpath("asdas\0das")))'
  'asdas'

diff --git a/tests/test-remotefilelog-gc.t b/tests/test-remotefilelog-gc.t
--- a/tests/test-remotefilelog-gc.t
+++ b/tests/test-remotefilelog-gc.t
@@ -106,6 +106,11 @@
 # Test that warning is displayed when the repo path is malformed
 
   $ printf "asdas\0das" >> $CACHEDIR/repos
+#if py311
+  $ hg gc
+  finished: removed 0 of 4 files (0.00 GB to 0.00 GB)
+#else
   $ hg gc
   abort: invalid path asdas\x00da: .*(null|NULL).* (re)
   [255]
+#endif

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 5 stable] tests: skip test-https.t TLSv1 testing when system doesn't support it

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679500739 -3600
#  Wed Mar 22 16:58:59 2023 +0100
# Branch stable
# Node ID 7c544bc71aaca594998649ae02d35fb4dd7606b4
# Parent  d641581ee136281971555adc05049b826e995fed
tests: skip test-https.t TLSv1 testing when system doesn't support it

The test failed on Fedora with the default security policy, unless degrading
system with:

  # update-crypto-policies --set LEGACY

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -701,6 +701,14 @@ def has_defaultcacertsloaded():
 return len(ctx.get_ca_certs()) > 0
 
 
+@check("tls1.0", "TLS 1 protocol support")
+def has_tls1_0():
+import ssl
+
+ctx = ssl.create_default_context()
+return ctx.minimum_version <= ssl.TLSVersion.TLSv1
+
+
 @check("tls1.2", "TLS 1.2 protocol support")
 def has_tls1_2():
 from mercurial import sslutil
diff --git a/tests/test-https.t b/tests/test-https.t
--- a/tests/test-https.t
+++ b/tests/test-https.t
@@ -356,10 +356,12 @@ Start servers running supported TLS vers
 
 Clients talking same TLS versions work
 
+#if tls1.0
   $ P="$CERTSDIR" hg --config hostsecurity.minimumprotocol=tls1.0 --config 
hostsecurity.ciphers=DEFAULT id https://localhost:$HGPORT/
   5fed3813f7f5
   $ P="$CERTSDIR" hg --config hostsecurity.minimumprotocol=tls1.1 --config 
hostsecurity.ciphers=DEFAULT id https://localhost:$HGPORT1/
   5fed3813f7f5
+#endif
   $ P="$CERTSDIR" hg --config hostsecurity.minimumprotocol=tls1.2 id 
https://localhost:$HGPORT2/
   5fed3813f7f5
 
@@ -391,6 +393,8 @@ Clients requiring newer TLS version than
   abort: error: .*(unsupported protocol|wrong ssl version|alert protocol 
version).* (re)
   [100]
 
+#if tls1.0
+
 --insecure will allow TLS 1.0 connections and override configs
 
   $ hg --config hostsecurity.minimumprotocol=tls1.2 id --insecure 
https://localhost:$HGPORT1/
@@ -405,6 +409,8 @@ The per-host config option overrides the
   > --config hostsecurity.localhost:minimumprotocol=tls1.0
   5fed3813f7f5
 
+#endif
+
 The per-host config option by itself works
 
   $ P="$CERTSDIR" hg id https://localhost:$HGPORT/ \

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 5 stable] tests: update test-serve.t to use $EACCES$

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679494603 -3600
#  Wed Mar 22 15:16:43 2023 +0100
# Branch stable
# Node ID d641581ee136281971555adc05049b826e995fed
# Parent  0a2a580319267132acd4ab558b0bfcd4f995a50f
tests: update test-serve.t to use $EACCES$

diff --git a/tests/test-serve.t b/tests/test-serve.t
--- a/tests/test-serve.t
+++ b/tests/test-serve.t
@@ -55,7 +55,7 @@ With -v and -p daytime
 #if no-windows
   $ KILLQUIETLY=Y
   $ hgserve -p daytime
-  abort: cannot start server at 'localhost:13': Permission denied (?)
+  abort: cannot start server at 'localhost:13': $EACCES$ (?)
   abort: child process failed to start (?)
   abort: no port number associated with service 'daytime' (?)
   listening at http://localhost/ (bound to $LOCALIP:13) (?)

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5 stable] tests: update test-transaction-rollback-on-revlog-split.t after e2ba2234bf1c

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679494192 -3600
#  Wed Mar 22 15:09:52 2023 +0100
# Branch stable
# Node ID 0a2a580319267132acd4ab558b0bfcd4f995a50f
# Parent  34c33da95b1aede3c9e63d1508570e2d8c620d9c
tests: update test-transaction-rollback-on-revlog-split.t after e2ba2234bf1c

Do like e9d92faa08f8

diff --git a/tests/test-transaction-rollback-on-revlog-split.t 
b/tests/test-transaction-rollback-on-revlog-split.t
--- a/tests/test-transaction-rollback-on-revlog-split.t
+++ b/tests/test-transaction-rollback-on-revlog-split.t
@@ -130,7 +130,7 @@ Reference size:
 #else
   $ hg pull ../troffset-computation
   pulling from ../troffset-computation
-  Killed
+  *Killed* (glob) (?)
   [137]
 #endif
 
@@ -210,7 +210,7 @@ Reference size:
   adding changesets
   adding manifests
   adding file changes
-  Killed
+  *Killed* (glob) (?)
   [137]
 #endif
 
@@ -275,7 +275,7 @@ Reference size:
   adding changesets
   adding manifests
   adding file changes
-  Killed
+  *Killed* (glob) (?)
   [137]
 #endif
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5 stable] tests: avoid using black >= 21

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679416596 -3600
#  Tue Mar 21 17:36:36 2023 +0100
# Branch stable
# Node ID 34c33da95b1aede3c9e63d1508570e2d8c620d9c
# Parent  4be9ecc982e19362747497276fa0841941787d08
tests: avoid using black >= 21

Different formatting will make test-check-format.t fail.

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -1115,7 +1115,9 @@ def has_black():
 version_regex = b'black, (?:version )?([0-9a-b.]+)'
 version = matchoutput(blackcmd, version_regex)
 sv = distutils.version.StrictVersion
-return version and sv(_bytes2sys(version.group(1))) >= sv('20.8b1')
+return version and (
+sv('20.8b1') <= sv(_bytes2sys(version.group(1))) < sv('21.0')
+)
 
 
 @check('pytype', 'the pytype type checker')

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences

[Mads Kiilerich]

Thanks for landing the previous 3.12 fixes. But I missed one fix ...

The remaining test failures on 3.12 are mostly from the removal 
(asyncore, smtpd and distutils) and deprecation (cgitb) in standard library.


/Mads
___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] tests: avoid using black >= 21

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679416596 -3600
#  Tue Mar 21 17:36:36 2023 +0100
# Branch stable
# Node ID c6a1fc106a05b608983caced38a1849b2f4b476f
# Parent  812482250f8d49159bb3024f08e0db38d0bad565
tests: avoid using black >= 21

Different formatting will make test-check-format.t fail.

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -1115,7 +1115,7 @@ def has_black():
 version_regex = b'black, (?:version )?([0-9a-b.]+)'
 version = matchoutput(blackcmd, version_regex)
 sv = distutils.version.StrictVersion
-return version and sv(_bytes2sys(version.group(1))) >= sv('20.8b1')
+return version and sv('20.8b1') <= sv(_bytes2sys(version.group(1))) < 
sv('21')
 
 
 @check('pytype', 'the pytype type checker')

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1679414842 -3600
#  Tue Mar 21 17:07:22 2023 +0100
# Branch stable
# Node ID 812482250f8d49159bb3024f08e0db38d0bad565
# Parent  87f0155d68aa56dcba2326692fff893c283cca96
py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences

Missed in 805d4a462abb:

  $ python3.12 mercurial/store.py
  mercurial/store.py:406: SyntaxWarning: invalid escape sequence '\.'
EXCLUDED = re.compile(b'.*undo\.[^/]+\.(nd?|i)$')

diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -403,7 +403,7 @@ REVLOG_FILES_VOLATILE_EXT = (b'.n', b'.n
 # some exception to the above matching
 #
 # XXX This is currently not in use because of issue6542
-EXCLUDED = re.compile(b'.*undo\.[^/]+\.(nd?|i)$')
+EXCLUDED = re.compile(br'.*undo\.[^/]+\.(nd?|i)$')
 
 
 def is_revlog(f, kind, st):

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 4 stable] hg-core: upgrade `zstd` dependency to 0.12

[Mads Kiilerich]

On 13/03/2023 15:44, Raphaël Gomès wrote:
The idea is to have everything run through the CI. Contributions going 
through the repo and thus Heptapod makes it much easier for reviewers.


Email remains as the lowest bar of entry (since you don't need an 
account and two out-of-core extensions) and for people truly allergic 
to the new workflow.


I get so few email patches I forget how to queue patches that need 
amending, heh. Maybe the mbox extension would help, but it's pretty 
low priority for me ATM.



I agree that email patches are the simplest way to contribute - 
especially when you already have it setup and working.


Also, I consider it important that Mercurial is fully self hosted when 
using email patches, and it support all client side workflows. It seems 
like contributing through heptapod depends on using the evolve extension 
and changing the development workflow. Thanks, but no thanks.


https://www.mercurial-scm.org/wiki/ContributingChanges still mentions 
email as a valid option. I will rely on that. And figure out what to do 
if it is removed without a good alternative.


I hope you the patches can land for 6.4 . There will still be many test 
failures for Python 3.12 , but it will be possible to get far enough to 
see and fix them individually.


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 4 stable] hg-core: upgrade `zstd` dependency to 0.12

[Mads Kiilerich]

Thanks.

I still maintain my local copy of 
https://www.mercurial-scm.org/wiki/MboxExtension . Something like that 
could be perhaps be patched into heptapod if the goal is to have 
everything running through the web interface.


/Mads


On 09/03/2023 18:33, Augie Fackler wrote:

Series LGTM for stable, but last time I queued something from the ML I think I 
angered the heptapod gods, so I’ll tag a couple of people that can hopefully 
push this on our behalf.

AF


On Mar 7, 2023, at 13:23, Mads Kiilerich  wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1678212582 -3600
#  Tue Mar 07 19:09:42 2023 +0100
# Branch stable
# Node ID 98085f80f5b0c786dad342ad1eb9335dda653496
# Parent  6ab12a75c936e0f1bf4305b9fdf9960219d1a39c
hg-core: upgrade `zstd` dependency to 0.12

zstd was recently upgraded from 0.5 to 0.11 . Now, take it one step further to
0.12 .

There is no need to stay in the past. And 0.12 is available in Fedora - 0.11
isn't.

diff --git a/rust/Cargo.lock b/rust/Cargo.lock
--- a/rust/Cargo.lock
+++ b/rust/Cargo.lock
@@ -1425,18 +1425,18 @@ checksum = "09041cd90cf85f7f8b2df60c646f

[[package]]
name = "zstd"
-version = "0.11.2+zstd.1.5.2"
+version = "0.12.3+zstd.1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "20cc960326ece64f010d2d2107537f26dc589a6573a316bd5b1dba685fa5fde4"
+checksum = "76eea132fb024e0e13fd9c2f5d5d595d8a967aa72382ac2f9d39fcc95afd0806"
dependencies = [
  "zstd-safe",
]

[[package]]
name = "zstd-safe"
-version = "5.0.2+zstd.1.5.2"
+version = "6.0.4+zstd.1.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "1d2a5585e04f9eea4b2a3d1eca508c4dee9592a89ef6f450c11719da0726f4db"
+checksum = "7afb4b54b8910cf5447638cb54bf4e8a65cbedd783af98b98c62ffe91f185543"
dependencies = [
  "libc",
  "zstd-sys",
@@ -1444,10 +1444,11 @@ dependencies = [

[[package]]
name = "zstd-sys"
-version = "2.0.1+zstd.1.5.2"
+version = "2.0.7+zstd.1.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "9fd07cbbc53846d9145dbffdf6dd09a7a0aa52be46741825f5c97bdd4f73f12b"
+checksum = "94509c3ba2fe55294d752b79842c530ccfab760192521df74a081a78d2b3c7f5"
dependencies = [
  "cc",
  "libc",
+ "pkg-config",
]
diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -34,7 +34,7 @@ thread_local = "1.1.4"
crossbeam-channel = "0.5.6"
log = "0.4.17"
memmap2 = { version = "0.5.8", features = ["stable_deref_trait"] }
-zstd = "0.11.2"
+zstd = "0.12"
format-bytes = "0.3.0"
# once_cell 1.15 uses edition 2021, while the heptapod CI
# uses an old version of Cargo that doesn't support it.

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel



___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 4 stable] hg-core: upgrade `zstd` dependency to 0.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1678212582 -3600
#  Tue Mar 07 19:09:42 2023 +0100
# Branch stable
# Node ID 98085f80f5b0c786dad342ad1eb9335dda653496
# Parent  6ab12a75c936e0f1bf4305b9fdf9960219d1a39c
hg-core: upgrade `zstd` dependency to 0.12

zstd was recently upgraded from 0.5 to 0.11 . Now, take it one step further to
0.12 .

There is no need to stay in the past. And 0.12 is available in Fedora - 0.11
isn't.

diff --git a/rust/Cargo.lock b/rust/Cargo.lock
--- a/rust/Cargo.lock
+++ b/rust/Cargo.lock
@@ -1425,18 +1425,18 @@ checksum = "09041cd90cf85f7f8b2df60c646f
 
 [[package]]
 name = "zstd"
-version = "0.11.2+zstd.1.5.2"
+version = "0.12.3+zstd.1.5.2"
 source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "20cc960326ece64f010d2d2107537f26dc589a6573a316bd5b1dba685fa5fde4"
+checksum = "76eea132fb024e0e13fd9c2f5d5d595d8a967aa72382ac2f9d39fcc95afd0806"
 dependencies = [
  "zstd-safe",
 ]
 
 [[package]]
 name = "zstd-safe"
-version = "5.0.2+zstd.1.5.2"
+version = "6.0.4+zstd.1.5.4"
 source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "1d2a5585e04f9eea4b2a3d1eca508c4dee9592a89ef6f450c11719da0726f4db"
+checksum = "7afb4b54b8910cf5447638cb54bf4e8a65cbedd783af98b98c62ffe91f185543"
 dependencies = [
  "libc",
  "zstd-sys",
@@ -1444,10 +1444,11 @@ dependencies = [
 
 [[package]]
 name = "zstd-sys"
-version = "2.0.1+zstd.1.5.2"
+version = "2.0.7+zstd.1.5.4"
 source = "registry+https://github.com/rust-lang/crates.io-index;
-checksum = "9fd07cbbc53846d9145dbffdf6dd09a7a0aa52be46741825f5c97bdd4f73f12b"
+checksum = "94509c3ba2fe55294d752b79842c530ccfab760192521df74a081a78d2b3c7f5"
 dependencies = [
  "cc",
  "libc",
+ "pkg-config",
 ]
diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -34,7 +34,7 @@ thread_local = "1.1.4"
 crossbeam-channel = "0.5.6"
 log = "0.4.17"
 memmap2 = { version = "0.5.8", features = ["stable_deref_trait"] }
-zstd = "0.11.2"
+zstd = "0.12"
 format-bytes = "0.3.0"
 # once_cell 1.15 uses edition 2021, while the heptapod CI
 # uses an old version of Cargo that doesn't support it.

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 4 stable] statprof: with Python 3.12, lineno is (more) often None

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1678205618 -3600
#  Tue Mar 07 17:13:38 2023 +0100
# Branch stable
# Node ID 6ab12a75c936e0f1bf4305b9fdf9960219d1a39c
# Parent  72ba5a6dbb52570fbdfa07ce15ac6ad88e35f63c
statprof: with Python 3.12, lineno is (more) often None

test-profile.t failed with errors like:
  TypeError: %d format: a real number is required, not NoneType

statprof.py already handled None values as -1 in some cases. Do the same in
more cases.

diff --git a/mercurial/statprof.py b/mercurial/statprof.py
--- a/mercurial/statprof.py
+++ b/mercurial/statprof.py
@@ -540,7 +540,7 @@ def display_by_line(data, fp):
 
 for stat in stats:
 site = stat.site
-sitelabel = b'%s:%d:%s' % (site.filename(), site.lineno, site.function)
+sitelabel = b'%s:%d:%s' % (site.filename(), site.lineno or -1, 
site.function)
 fp.write(
 b'%6.2f %9.2f %9.2f  %s\n'
 % (
@@ -613,7 +613,7 @@ def display_by_method(data, fp):
 stattuple = (
 stat.selfpercent(),
 stat.selfseconds(),
-stat.site.lineno,
+stat.site.lineno or -1,
 source,
 )
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4 stable] py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1678203954 -3600
#  Tue Mar 07 16:45:54 2023 +0100
# Branch stable
# Node ID 72ba5a6dbb52570fbdfa07ce15ac6ad88e35f63c
# Parent  1efa4e96f6461bf071b28d66b13bdb67fdc91fc6
py3: fix for Python 3.12 emitting SyntaxWarning on invalid escape sequences

Mercurial became very noisy after 
https://github.com/python/cpython/commit/a60ddd31be7ff96a8189e7483bf1eb2071d2bddf
 ,
for example:

  $ python3.12 mercurial/store.py
  mercurial/store.py:406: SyntaxWarning: invalid escape sequence '\.'
EXCLUDED = re.compile(b'.*undo\.[^/]+\.(nd?|i)$')

This verbosity made some tests fail.

The problems were mostly insufficiently escaped regexps, relying on the Python
parser/scanner preserving invalid escape sequences.

diff --git a/mercurial/match.py b/mercurial/match.py
--- a/mercurial/match.py
+++ b/mercurial/match.py
@@ -196,14 +196,14 @@ def match(
 ... return match(util.localpath(root), *args, **kwargs)
 
 Usually a patternmatcher is returned:
->>> _match(b'/foo', b'.', [b're:.*\.c$', b'path:foo/a', b'*.py'])
+>>> _match(b'/foo', b'.', [br're:.*\.c$', b'path:foo/a', b'*.py'])
 
 
 Combining 'patterns' with 'include' (resp. 'exclude') gives an
 intersectionmatcher (resp. a differencematcher):
->>> type(_match(b'/foo', b'.', [b're:.*\.c$'], include=[b'path:lib']))
+>>> type(_match(b'/foo', b'.', [br're:.*\.c$'], include=[b'path:lib']))
 
->>> type(_match(b'/foo', b'.', [b're:.*\.c$'], exclude=[b'path:build']))
+>>> type(_match(b'/foo', b'.', [br're:.*\.c$'], exclude=[b'path:build']))
 
 
 Notice that, if 'patterns' is empty, an alwaysmatcher is returned:
@@ -212,7 +212,7 @@ def match(
 
 The 'default' argument determines which kind of pattern is assumed if a
 pattern has no prefix:
->>> _match(b'/foo', b'.', [b'.*\.c$'], default=b're')
+>>> _match(b'/foo', b'.', [br'.*\.c$'], default=b're')
 
 >>> _match(b'/foo', b'.', [b'main.py'], default=b'relpath')
 
@@ -223,7 +223,7 @@ def match(
 name) matches againset one of the patterns given at initialization. There
 are two ways of doing this check.
 
->>> m = _match(b'/foo', b'', [b're:.*\.c$', b'relpath:a'])
+>>> m = _match(b'/foo', b'', [br're:.*\.c$', b'relpath:a'])
 
 1. Calling the matcher with a file name returns True if any pattern
 matches that file name:
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -1988,7 +1988,7 @@ such str.lower().
 
   $ "$PYTHON" < def escape(s):
-  > return b''.join(b'\\u%x' % ord(uc) for uc in s.decode('cp932'))
+  > return b''.join(br'\\u%x' % ord(uc) for uc in s.decode('cp932'))
   > # translation of "record" in ja_JP.cp932
   > upper = b"\x8bL\x98^"
   > # str.lower()-ed section name should be treated as different one
diff --git a/tests/test-minirst.py b/tests/test-minirst.py
--- a/tests/test-minirst.py
+++ b/tests/test-minirst.py
@@ -154,7 +154,7 @@ marker after the option. It is treated a
 
 debugformats('options', options)
 
-fields = b"""
+fields = br"""
 :a: First item.
 :ab: Second item. Indentation and wrapping
  is handled automatically.
diff --git a/tests/testlib/persistent-nodemap-race-ext.py 
b/tests/testlib/persistent-nodemap-race-ext.py
--- a/tests/testlib/persistent-nodemap-race-ext.py
+++ b/tests/testlib/persistent-nodemap-race-ext.py
@@ -1,4 +1,4 @@
-"""Create the race condition for issue6554
+r"""Create the race condition for issue6554
 
 The persistent nodemap issues had an issue where a second writer could
 overwrite the data that a previous write just wrote. The would break the append

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 4 stable] cext: fix for PyLong refactoring in CPython 3.12

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1678202751 -3600
#  Tue Mar 07 16:25:51 2023 +0100
# Branch stable
# Node ID 1efa4e96f6461bf071b28d66b13bdb67fdc91fc6
# Parent  8a65b43457aba02741bedabe100823d3041af0b5
cext: fix for PyLong refactoring in CPython 3.12

Compiling Mercurial with Python 3.12 a5 would fail with:

mercurial/cext/dirs.c: In function '_addpath':
mercurial/cext/dirs.c:19:44: error: 'PyLongObject' {aka 'struct _longobject'} 
has no member named 'ob_digit'
   19 | #define PYLONG_VALUE(o) ((PyLongObject *)o)->ob_digit[0]
  |^~
mercurial/cext/dirs.c:97:25: note: in expansion of macro 'PYLONG_VALUE'
   97 | PYLONG_VALUE(val) += 1;
  | ^~~~
mercurial/cext/dirs.c:19:44: error: 'PyLongObject' {aka 'struct _longobject'} 
has no member named 'ob_digit'
   19 | #define PYLONG_VALUE(o) ((PyLongObject *)o)->ob_digit[0]
  |^~
mercurial/cext/dirs.c:108:17: note: in expansion of macro 'PYLONG_VALUE'
  108 | PYLONG_VALUE(val) = 1;
  | ^~~~
mercurial/cext/dirs.c: In function '_delpath':
mercurial/cext/dirs.c:19:44: error: 'PyLongObject' {aka 'struct _longobject'} 
has no member named 'ob_digit'
   19 | #define PYLONG_VALUE(o) ((PyLongObject *)o)->ob_digit[0]
  |^~
mercurial/cext/dirs.c:145:23: note: in expansion of macro 'PYLONG_VALUE'
  145 | if (--PYLONG_VALUE(val) <= 0) {
  |   ^~~~

This was caused by
https://github.com/python/cpython/commit/c1b1f51cd1632f0b77dacd43092fb44ed5e053a9
 .

diff --git a/mercurial/cext/dirs.c b/mercurial/cext/dirs.c
--- a/mercurial/cext/dirs.c
+++ b/mercurial/cext/dirs.c
@@ -13,7 +13,11 @@
 
 #include "util.h"
 
+#if PY_VERSION_HEX >= 0x030C00A5
+#define PYLONG_VALUE(o) ((PyLongObject *)o)->long_value.ob_digit[0]
+#else
 #define PYLONG_VALUE(o) ((PyLongObject *)o)->ob_digit[0]
+#endif
 
 /*
  * This is a multiset of directory names, built from the files that

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH stable] rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0

[Mads Kiilerich]

Raphaël, could you please have a look at this?

/Mads


On 7/11/22 23:03, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1657572476 -7200
#  Mon Jul 11 22:47:56 2022 +0200
# Branch stable
# Node ID f1713e81437e894fab0658e4f410184e10d35e5e
# Parent  55adff8105464f6247983940ba109684d36b689d
rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0

The merge in 12adf8c695ed had conflicts in rust/Cargo.lock and
rust/hg-core/Cargo.toml . Let's ignore rust/Cargo.lock - it is regenerated.

For rust/hg-core/Cargo.toml, stable had dd6b67d5c256 "rust: fix unsound
`OwningDirstateMap`" which introduced ouroboros (and dropped
stable_deref_trait).

Default had ec8d9b5a5e7c "rust-hg-core: upgrade dependencies" which had a lot
of churn bumping minimum versions - also patch versions. It is indeed a good
idea to bump to *allow* use of latest package. That means that major versions
should be bumped for packages after 1.0, and for packages below 1.0 minor
versions should be bumped too. But it doesn't work to try enforce a policy of
using latest patch by bumping versions at arbitrary times.

For good or bad, the merge doesn't seem to have resolved the conflicts
correctly, and many of the minor "upgrade dependencies" were lost again.

Unfortunately, it also lost the bump of memmap2 to 0.5.3, which is needed for
Fedora packaging where 0.4 isn't available. Same with micro-timer bump to 0.4
(which already is used in rhg). crossbeam-channel bump was also lost.

This change fixes that regression by redoing these "important" lines of the
merge "correctly".

I propose this for stable, even though dependency changes on stable branches
are annoying.

diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -29,10 +29,10 @@ sha-1 = "0.10.0"
  twox-hash = "1.6.2"
  same-file = "1.0.6"
  tempfile = "3.1.0"
-crossbeam-channel = "0.4"
-micro-timer = "0.3.0"
+crossbeam-channel = "0.5.0"
+micro-timer = "0.4.0"
  log = "0.4.8"
-memmap2 = {version = "0.4", features = ["stable_deref_trait"]}
+memmap2 = { version = "0.5.3", features = ["stable_deref_trait"] }
  zstd = "0.5.3"
  format-bytes = "0.3.0"
  



___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1657572476 -7200
#  Mon Jul 11 22:47:56 2022 +0200
# Branch stable
# Node ID f1713e81437e894fab0658e4f410184e10d35e5e
# Parent  55adff8105464f6247983940ba109684d36b689d
rust: bump to memmap2 0.5.3, micro-timer 0.4.0, and crossbeam-channel 0.5.0

The merge in 12adf8c695ed had conflicts in rust/Cargo.lock and
rust/hg-core/Cargo.toml . Let's ignore rust/Cargo.lock - it is regenerated.

For rust/hg-core/Cargo.toml, stable had dd6b67d5c256 "rust: fix unsound
`OwningDirstateMap`" which introduced ouroboros (and dropped
stable_deref_trait).

Default had ec8d9b5a5e7c "rust-hg-core: upgrade dependencies" which had a lot
of churn bumping minimum versions - also patch versions. It is indeed a good
idea to bump to *allow* use of latest package. That means that major versions
should be bumped for packages after 1.0, and for packages below 1.0 minor
versions should be bumped too. But it doesn't work to try enforce a policy of
using latest patch by bumping versions at arbitrary times.

For good or bad, the merge doesn't seem to have resolved the conflicts
correctly, and many of the minor "upgrade dependencies" were lost again.

Unfortunately, it also lost the bump of memmap2 to 0.5.3, which is needed for
Fedora packaging where 0.4 isn't available. Same with micro-timer bump to 0.4
(which already is used in rhg). crossbeam-channel bump was also lost.

This change fixes that regression by redoing these "important" lines of the
merge "correctly".

I propose this for stable, even though dependency changes on stable branches
are annoying.

diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -29,10 +29,10 @@ sha-1 = "0.10.0"
 twox-hash = "1.6.2"
 same-file = "1.0.6"
 tempfile = "3.1.0"
-crossbeam-channel = "0.4"
-micro-timer = "0.3.0"
+crossbeam-channel = "0.5.0"
+micro-timer = "0.4.0"
 log = "0.4.8"
-memmap2 = {version = "0.4", features = ["stable_deref_trait"]}
+memmap2 = { version = "0.5.3", features = ["stable_deref_trait"] }
 zstd = "0.5.3"
 format-bytes = "0.3.0"
 

___
Mercurial-devel mailing list
Mercurial-devel@lists.mercurial-scm.org
https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] rust: relax im-rc dependency to allow minor updates

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1653395384 -7200
#  Tue May 24 14:29:44 2022 +0200
# Branch stable
# Node ID 32de89308ead51d42d631c208ab17c866218e1c5
# Parent  1d257c4c15683ee998edcc7dd6caf5a4cb52c820
rust: relax im-rc dependency to allow minor updates

This "15.0.*" requirement came from 0d99778af68a and is now replaced with plain
"15.0".

AFAICS, it really should allow (but not necessarily require) im-rc 15.1 .

Narrow requirement requirements with wildcard in the version is not used in
other places.

diff --git a/rust/hg-core/Cargo.toml b/rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml
+++ b/rust/hg-core/Cargo.toml
@@ -14,7 +14,7 @@ bytes-cast = "0.2"
 byteorder = "1.3.4"
 derive_more = "0.99"
 home = "0.5"
-im-rc = "15.0.*"
+im-rc = "15.0"
 itertools = "0.9"
 lazy_static = "1.4.0"
 libc = "0.2"

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] docs: use proper rst markup for preformatted blocks

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1651574481 -7200
#  Tue May 03 12:41:21 2022 +0200
# Branch stable
# Node ID 1d257c4c15683ee998edcc7dd6caf5a4cb52c820
# Parent  f2ef6a4f918f59f2ff01e5d4dc01c94ea35bd32f
docs: use proper rst markup for preformatted blocks

The multiple lines were re-flowed to a single line, both in man page and html.

diff --git a/mercurial/helptext/rust.txt b/mercurial/helptext/rust.txt
--- a/mercurial/helptext/rust.txt
+++ b/mercurial/helptext/rust.txt
@@ -28,7 +28,8 @@ in progress. For more experimental work 
 Checking for Rust
 =
 
-You may already have the Rust extensions depending on how you install 
Mercurial.
+You may already have the Rust extensions depending on how you install
+Mercurial::
 
   $ hg debuginstall | grep -i rust
   checking Rust extensions (installed)
@@ -46,7 +47,7 @@ version to use.
 Using pip
 -
 
-Users of `pip` can install the Rust extensions with the following command:
+Users of `pip` can install the Rust extensions with the following command::
 
   $ pip install mercurial --global-option --rust --no-use-pep517
 

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable, v2] doc: inspect.getargspec has been removed in Python 3.11

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1644245153 -3600
#  Mon Feb 07 15:45:53 2022 +0100
# Branch stable
# Node ID 937db8e65dd5614cc4595cced61f2e21026d3be3
# Parent  01fde63b4eded708802bfd0d0d4cb4ecc5ff6e1c
doc: inspect.getargspec has been removed in Python 3.11

Fixed by dropping the inspection introduced in cdda48c93676. The 2nd "reporter"
parameter to docutils.languages.get_language has been available since 0.8 more
than 10 years ago.

Reported for Fedora on https://bugzilla.redhat.com/show_bug.cgi?id=2022252#c2 .

diff --git a/doc/hgmanpage.py b/doc/hgmanpage.py
--- a/doc/hgmanpage.py
+++ b/doc/hgmanpage.py
@@ -45,7 +45,6 @@ from __future__ import absolute_import
 
 __docformat__ = 'reStructuredText'
 
-import inspect
 import re
 
 from docutils import (
@@ -177,13 +176,7 @@ class Translator(nodes.NodeVisitor):
 nodes.NodeVisitor.__init__(self, document)
 self.settings = settings = document.settings
 lcode = settings.language_code
-arglen = len(inspect.getargspec(languages.get_language)[0])
-if arglen == 2:
-self.language = languages.get_language(
-lcode, self.document.reporter
-)
-else:
-self.language = languages.get_language(lcode)
+self.language = languages.get_language(lcode, self.document.reporter)
 self.head = []
 self.body = []
 self.foot = []

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] doc: inspect.getargspec has been removed in Python 3.11

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1644066813 -3600
#  Sat Feb 05 14:13:33 2022 +0100
# Branch stable
# Node ID e3c50cc1facafbd13120b7693155add1af9f94ed
# Parent  01fde63b4eded708802bfd0d0d4cb4ecc5ff6e1c
doc: inspect.getargspec has been removed in Python 3.11

Fix problem left over from 646002338365.

Reported for Fedora on https://bugzilla.redhat.com/show_bug.cgi?id=2022252#c2 .

diff --git a/doc/hgmanpage.py b/doc/hgmanpage.py
--- a/doc/hgmanpage.py
+++ b/doc/hgmanpage.py
@@ -177,7 +177,11 @@ class Translator(nodes.NodeVisitor):
 nodes.NodeVisitor.__init__(self, document)
 self.settings = settings = document.settings
 lcode = settings.language_code
-arglen = len(inspect.getargspec(languages.get_language)[0])
+try:
+getfullargspec = inspect.getfullargspec
+except:  # Python 2
+getfullargspec = inspect.getargspec
+arglen = len(getfullargspec(languages.get_language)[0])
 if arglen == 2:
 self.language = languages.get_language(
 lcode, self.document.reporter

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] cext: fix Python 3.11 compatibility - Py_SIZE is not an lvalue (issue6610)

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1637235097 -3600
#  Thu Nov 18 12:31:37 2021 +0100
# Branch stable
# Node ID 9c41494c5dd2fa8816bc7b70e76b28c4d3941af0
# Parent  6e576e4665f43101ef9bb98b0f5234f85c5db2ea
cext: fix Python 3.11 compatibility - Py_SIZE is not an lvalue (issue6610)

Py_SIZE was made a static inline function during Python 3.10 development, as
described on https://vstinner.github.io/c-api-opaque-structures.html .
e92ca942ddca updated the Mercurial code base accordingly, but somehow missed a
couple of cases introduced long time ago in a8c948ee3668.

The Python change was dropped for 3.10, but is coming back again in 3.11 .

diff --git a/mercurial/cext/pathencode.c b/mercurial/cext/pathencode.c
--- a/mercurial/cext/pathencode.c
+++ b/mercurial/cext/pathencode.c
@@ -176,7 +176,7 @@ PyObject *encodedir(PyObject *self, PyOb
 
if (newobj) {
assert(PyBytes_Check(newobj));
-   Py_SIZE(newobj)--;
+   Py_SET_SIZE(newobj, Py_SIZE(newobj) - 1);
_encodedir(PyBytes_AS_STRING(newobj), newlen, path, len + 1);
}
 
@@ -791,7 +791,7 @@ PyObject *pathencode(PyObject *self, PyO
 
if (newobj) {
assert(PyBytes_Check(newobj));
-   Py_SIZE(newobj)--;
+   Py_SET_SIZE(newobj, Py_SIZE(newobj) - 1);
basicencode(PyBytes_AS_STRING(newobj), newlen, path,
len + 1);
}

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] bugzilla: fix reporting of exceptions with py3

[Mads Kiilerich]

These patches could be considered for stable. They could qualify, as 
they fix some obvious blockers and make the bugzilla extension work 
better with py3.


But there seems to be many other problems with bugzilla extension on 
py3. The extension is messing with internals of xmlrpclib, and the py3 
xmlrpm.client changed a lot. Porting will be a non-trivial development 
effort ... and that require someone who actually use the extension and 
have a test setup.


These changes are a spin-off from 
https://bugzilla.redhat.com/show_bug.cgi?id=1859043 .


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] bugzilla: pass the url to xmlrpclib.ServerProxy as str

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1607032141 -3600
#  Thu Dec 03 22:49:01 2020 +0100
# Branch stable
# Node ID 6473ac822f918193dd39f43f054695d5d6321736
# Parent  a141d1da33433998590be226b6788fb53e8c90e0
bugzilla: pass the url to xmlrpclib.ServerProxy as str

diff --git a/hgext/bugzilla.py b/hgext/bugzilla.py
--- a/hgext/bugzilla.py
+++ b/hgext/bugzilla.py
@@ -759,7 +759,9 @@ class bzxmlrpc(bzaccess):
 self.fixstatus = self.ui.config(b'bugzilla', b'fixstatus')
 self.fixresolution = self.ui.config(b'bugzilla', b'fixresolution')
 
-self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
+self.bzproxy = xmlrpclib.ServerProxy(
+pycompat.strurl(bzweb), self.transport(bzweb)
+)
 ver = self.bzproxy.Bugzilla.version()[b'version'].split(b'.')
 self.bzvermajor = int(ver[0])
 self.bzverminor = int(ver[1])

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] bugzilla: fix reporting of exceptions with py3

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1607032141 -3600
#  Thu Dec 03 22:49:01 2020 +0100
# Branch stable
# Node ID a141d1da33433998590be226b6788fb53e8c90e0
# Parent  dadbd01f939379a2eea76eab313868fcde48138a
bugzilla: fix reporting of exceptions with py3

diff --git a/hgext/bugzilla.py b/hgext/bugzilla.py
--- a/hgext/bugzilla.py
+++ b/hgext/bugzilla.py
@@ -1211,4 +1211,4 @@ def hook(ui, repo, hooktype, node=None, 
 bz.update(bug, bugs[bug], ctx)
 bz.notify(bugs, stringutil.email(ctx.user()))
 except Exception as e:
-raise error.Abort(_(b'Bugzilla error: %s') % e)
+raise error.Abort(_(b'Bugzilla error: %s') % 
stringutil.forcebytestr(e))

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 8 stable] packaging: update built-in Fedora support to Fedora 31

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572616448 -3600
#  Fri Nov 01 14:54:08 2019 +0100
# Branch stable
# Node ID 86a486daf51bb3a38623ee0635d81aec5c7008b0
# Parent  8c57b4472c1acd7bf11f83dccb29c6cc064455a0
packaging: update built-in Fedora support to Fedora 31

This is now quite easy ...

diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -11,7 +11,7 @@ UBUNTU_CODENAMES := \
   cosmic \
   disco
 
-FEDORA_RELEASE := 29
+FEDORA_RELEASE := 31
 
 CENTOS_RELEASES := \
   5 \
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8 stable] packaging: refactor "fedora29" target to a single more generic "fedora" target

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572612704 -3600
#  Fri Nov 01 13:51:44 2019 +0100
# Branch stable
# Node ID 8c57b4472c1acd7bf11f83dccb29c6cc064455a0
# Parent  0b31ec6e6e7e815a75dc465e133043582b3e4cb7
packaging: refactor "fedora29" target to a single more generic "fedora" target

Fedora moves fast in version numbers, and often with Mercurial packaging being
backwards compatible. Also, most people use the system package. There is thus
much work and tech debt and little value in providing explicit built-in support
for several versions. Thus, only aim for providing built-in support for latest
Fedora version, and make it easy to update.

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -189,7 +189,7 @@ packaging_targets := \
   docker-centos7 \
   docker-debian-jessie \
   docker-debian-stretch \
-  docker-fedora29 \
+  docker-fedora \
   docker-ubuntu-trusty \
   docker-ubuntu-trusty-ppa \
   docker-ubuntu-xenial \
@@ -198,7 +198,7 @@ packaging_targets := \
   docker-ubuntu-artful-ppa \
   docker-ubuntu-bionic \
   docker-ubuntu-bionic-ppa \
-  fedora29 \
+  fedora \
   linux-wheels \
   linux-wheels-x86_64 \
   linux-wheels-i686 \
diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -11,8 +11,7 @@ UBUNTU_CODENAMES := \
   cosmic \
   disco
 
-FEDORA_RELEASES := \
-  29
+FEDORA_RELEASE := 29
 
 CENTOS_RELEASES := \
   5 \
@@ -31,8 +30,8 @@ help:
@echo 'docker-debian-{$(strip $(DEBIAN_CODENAMES))}'
@echo '   Build Debian packages specific to a Debian distro using 
Docker.'
@echo ''
-   @echo 'docker-fedora{$(strip $(FEDORA_RELEASES))}'
-   @echo '   Build an RPM for a specific Fedora version using Docker.'
+   @echo 'docker-fedora'
+   @echo '   Build an RPM for a Fedora $(FEDORA_RELEASE) using Docker.'
@echo ''
@echo 'docker-ubuntu-{$(strip $(UBUNTU_CODENAMES))}'
@echo '   Build Debian package specific to an Ubuntu distro using 
Docker.'
@@ -56,8 +55,8 @@ help:
@echo 'centos{$(strip $(CENTOS_RELEASES))}'
@echo '   Build an RPM for a specific CentOS version locally'
@echo ''
-   @echo 'fedora{$(strip $(FEDORA_RELEASES))}'
-   @echo '   Build an RPM for a specific Fedora version locally'
+   @echo 'fedora'
+   @echo '   Build an RPM for Fedora $(FEDORA_RELEASE) locally'
 
 .PHONY: help
 
@@ -94,22 +93,17 @@ endef
 $(foreach codename,$(UBUNTU_CODENAMES),$(eval $(call 
ubuntu_targets,$(codename
 
 # Fedora targets.
-define fedora_targets
-.PHONY: fedora$(1)
-fedora$(1):
-   mkdir -p $$(HGROOT)/packages/fedora$(1)
+.PHONY: fedora
+fedora:
+   mkdir -p $(HGROOT)/packages/fedora$(FEDORA_RELEASE)
./buildrpm
-   cp $$(HGROOT)/contrib/packaging/rpmbuild/RPMS/*/* 
$$(HGROOT)/packages/fedora$(1)
-   cp $$(HGROOT)/contrib/packaging/rpmbuild/SRPMS/* 
$$(HGROOT)/packages/fedora$(1)
+   cp $(HGROOT)/contrib/packaging/rpmbuild/RPMS/*/* 
$(HGROOT)/packages/fedora$(FEDORA_RELEASE)
+   cp $(HGROOT)/contrib/packaging/rpmbuild/SRPMS/* 
$(HGROOT)/packages/fedora$(FEDORA_RELEASE)
rm -rf $(HGROOT)/rpmbuild
 
-.PHONY: docker-fedora$(1)
-docker-fedora$(1):
-   ./dockerrpm fedora$(1)
-
-endef
-
-$(foreach release,$(FEDORA_RELEASES),$(eval $(call fedora_targets,$(release
+.PHONY: docker-fedora
+docker-fedora:
+   ./dockerrpm fedora$(FEDORA_RELEASE)
 
 # CentOS targets.
 define centos_targets
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 8 stable] packaging: make dockerrpm fedora target more generic

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572618554 -3600
#  Fri Nov 01 15:29:14 2019 +0100
# Branch stable
# Node ID 0b31ec6e6e7e815a75dc465e133043582b3e4cb7
# Parent  dd589d02eba31ae40369d24bf3620863b4c86008
packaging: make dockerrpm fedora target more generic

Fedora moves fast in version numbers, and often with Mercurial packaging being
backwards compatible. Thus, only aim for providing built-in support for latest
Fedora version, and make it easy to update.

With this refactoring, 'dockerrpm fedora31' also works.

'dockerrpm fedora' will use the 'fedora:latest' Docker image.

diff --git a/contrib/packaging/docker/fedora29 
b/contrib/packaging/docker/fedora.template
rename from contrib/packaging/docker/fedora29
rename to contrib/packaging/docker/fedora.template
--- a/contrib/packaging/docker/fedora29
+++ b/contrib/packaging/docker/fedora.template
@@ -1,4 +1,4 @@
-FROM fedora:29
+FROM fedora:%OS_RELEASE%
 
 RUN groupadd -g 1000 build && \
 useradd -u 1000 -g 1000 -s /bin/bash -d /build -m build
diff --git a/contrib/packaging/dockerrpm b/contrib/packaging/dockerrpm
--- a/contrib/packaging/dockerrpm
+++ b/contrib/packaging/dockerrpm
@@ -6,6 +6,14 @@ export ROOTDIR=$(cd $BUILDDIR/../..; pwd
 PLATFORM="$1"
 shift # extra params are passed to buildrpm
 
+DOCKERFILE="$PLATFORM"
+OS_RELEASE="${PLATFORM//[a-z]/}"
+case "$PLATFORM" in
+fedora*)
+DOCKERFILE="${PLATFORM//[0-9]/}.template"
+;;
+esac
+
 DOCKER=$($BUILDDIR/hg-docker docker-path)
 
 CONTAINER=hg-docker-$PLATFORM
@@ -18,7 +26,11 @@ else
 DOCKERGID=$(id -g)
 fi
 
-$BUILDDIR/hg-docker build --build-arg UID=$DOCKERUID --build-arg 
GID=$DOCKERGID $BUILDDIR/docker/$PLATFORM $CONTAINER
+$BUILDDIR/hg-docker build \
+--build-arg UID=$DOCKERUID \
+--build-arg GID=$DOCKERGID \
+--build-arg OS_RELEASE=${OS_RELEASE:-latest} \
+$BUILDDIR/docker/$DOCKERFILE $CONTAINER
 
 RPMBUILDDIR=$ROOTDIR/packages/$PLATFORM
 mkdir -p $RPMBUILDDIR
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8 stable] packaging: use "python3" for fedora29 ... and as buildrpm default

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572609562 -3600
#  Fri Nov 01 12:59:22 2019 +0100
# Branch stable
# Node ID dd589d02eba31ae40369d24bf3620863b4c86008
# Parent  27f849825e3a2fe7bd08a79659cfddaa95851ff9
packaging: use "python3" for fedora29 ... and as buildrpm default

Change the buidrpm default. The CentOS targets explicitly use "python", and
changing the default will only influence Fedora 29.

A Python 3 package needs python3 dependencies, so pythonexe (and pythonver) is
used for specifying dependencies. Other OS versions will keep using "python" as
before ... or potentially change to explicit "python2". Fedora 29 packages can
thus also still be built for Python 2 - just not in the docker image that is
updated for Python 3.

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -6,7 +6,7 @@
 
 BUILD=1
 RPMBUILDDIR="$PWD/rpmbuild"
-PYTHONEXE=python2
+PYTHONEXE=python3
 
 while [ "$1" ]; do
 case "$1" in
diff --git a/contrib/packaging/docker/fedora29 
b/contrib/packaging/docker/fedora29
--- a/contrib/packaging/docker/fedora29
+++ b/contrib/packaging/docker/fedora29
@@ -7,8 +7,8 @@ RUN dnf install -y \
gcc \
gettext \
make \
-   python-devel \
-   python-docutils \
+   python3-devel \
+   python3-docutils \
rpm-build
 
 # For creating repo meta data
diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -2,7 +2,7 @@
 
 %define withpython %{nil}
 
-%global pythonexe python2
+%global pythonexe python3
 
 %if "%{?withpython}"
 
@@ -39,8 +39,8 @@ BuildRequires: make, gcc, gettext
 %if "%{?withpython}"
 BuildRequires: readline-devel, openssl-devel, ncurses-devel, zlib-devel, 
bzip2-devel
 %else
-BuildRequires: python >= %{pythonver}, python-devel, python-docutils >= 0.5
-Requires: python >= %{pythonver}
+BuildRequires: %{pythonexe} >= %{pythonver}, %{pythonexe}-devel, 
%{pythonexe}-docutils >= 0.5
+Requires: %{pythonexe} >= %{pythonver}
 %endif
 # The hgk extension uses the wish tcl interpreter, but we don't enforce it
 #Requires: tk
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 8 stable] packaging: use "--python python" for centos7 to avoid explicit "python2"

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572608858 -3600
#  Fri Nov 01 12:47:38 2019 +0100
# Branch stable
# Node ID 27f849825e3a2fe7bd08a79659cfddaa95851ff9
# Parent  0039dc1d76f454f86a9cbfb9064ca6b3cac664d6
packaging: use "--python python" for centos7 to avoid explicit "python2"

This is a partial backout of 92a51a45d44c.

We will need to be able to control whether package dependencies are python2 or
python3. Generally (at least in recent Fedora), the package prefix match the
name of the python executable ...  but CentOS 7 doesn't use the python2 prefix
in package name or alias for python-docutils yet, so just keep centos7 in the
unversioned "python" world.

Change the new (unused) buildrpm "--python3" option (introduced in
a6dcac6454c1) to "--python python3" to get a more generic method for explicit
control over whether we use python, python2 or python3.

diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -116,13 +116,13 @@ define centos_targets
 .PHONY: centos$(1)
 centos$(1):
mkdir -p $$(HGROOT)/packages/centos$(1)
-   ./buildrpm $$(if $$(filter 
$(1),$$(CENTOS_WITH_PYTHON_RELEASES)),--withpython)
+   ./buildrpm $$(if $$(filter 
$(1),$$(CENTOS_WITH_PYTHON_RELEASES)),--withpython,--python python)
cp $$(HGROOT)/contrib/packaging/rpmbuild/RPMS/*/* 
$$(HGROOT)/packages/centos$(1)
cp $$(HGROOT)/contrib/packaging/rpmbuild/SRPMS/* 
$$(HGROOT)/packages/centos$(1)
 
 .PHONY: docker-centos$(1)
 docker-centos$(1):
-   ./dockerrpm centos$(1) $$(if $$(filter 
$(1),$$(CENTOS_WITH_PYTHON_RELEASES)),--withpython)
+   ./dockerrpm centos$(1) $$(if $$(filter 
$(1),$$(CENTOS_WITH_PYTHON_RELEASES)),--withpython,--python python)
 
 endef
 
diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -14,9 +14,10 @@ while [ "$1" ]; do
 shift
 BUILD=
 ;;
---python3)
+--python)
 shift
-PYTHONEXE=python3
+PYTHONEXE=$1
+shift
 ;;
 --withpython | --with-python)
 shift
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8 stable] packaging: fix docker-centos5 - use pythonexe and set to "python" as before

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572608048 -3600
#  Fri Nov 01 12:34:08 2019 +0100
# Branch stable
# Node ID 0039dc1d76f454f86a9cbfb9064ca6b3cac664d6
# Parent  b6c904e28f35efa5fcb6162a72e4acb19899f183
packaging: fix docker-centos5 - use pythonexe and set to "python" as before

Fix 92a51a45d44c .

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -22,6 +22,7 @@ while [ "$1" ]; do
 shift
 PYTHONVER=2.7.16
 PYTHONMD5=f1a2ace631068444831d01485466ece0
+PYTHONEXE=python
 ;;
 --rpmbuilddir )
 shift
diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -2,6 +2,8 @@
 
 %define withpython %{nil}
 
+%global pythonexe python2
+
 %if "%{?withpython}"
 
 %global pythonver %{withpython}
@@ -15,7 +17,6 @@
 
 %else
 
-%global pythonexe python2
 %global pythonver %(%{pythonexe} -c 'import sys;print(".".join(map(str, 
sys.version_info[:2])))')
 
 %endif
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8 stable] packaging: move dockerrpm output directory creation to dockerrpm

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572607097 -3600
#  Fri Nov 01 12:18:17 2019 +0100
# Branch stable
# Node ID b6c904e28f35efa5fcb6162a72e4acb19899f183
# Parent  a29e42f5492266b779ab24ac4496211c947c6601
packaging: move dockerrpm output directory creation to dockerrpm

Avoid having to compute the directory in two places in different environments.

diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -105,7 +105,6 @@ fedora$(1):
 
 .PHONY: docker-fedora$(1)
 docker-fedora$(1):
-   mkdir -p $$(HGROOT)/packages/fedora$(1)
./dockerrpm fedora$(1)
 
 endef
@@ -123,7 +122,6 @@ centos$(1):
 
 .PHONY: docker-centos$(1)
 docker-centos$(1):
-   mkdir -p $$(HGROOT)/packages/centos$(1)
./dockerrpm centos$(1) $$(if $$(filter 
$(1),$$(CENTOS_WITH_PYTHON_RELEASES)),--withpython)
 
 endef
diff --git a/contrib/packaging/dockerrpm b/contrib/packaging/dockerrpm
--- a/contrib/packaging/dockerrpm
+++ b/contrib/packaging/dockerrpm
@@ -21,6 +21,7 @@ fi
 $BUILDDIR/hg-docker build --build-arg UID=$DOCKERUID --build-arg 
GID=$DOCKERGID $BUILDDIR/docker/$PLATFORM $CONTAINER
 
 RPMBUILDDIR=$ROOTDIR/packages/$PLATFORM
+mkdir -p $RPMBUILDDIR
 $ROOTDIR/contrib/packaging/buildrpm --rpmbuilddir $RPMBUILDDIR --prepare $*
 
 DSHARED=/mnt/shared
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 8 stable] packaging: drop "support" for unsupported Fedora versions

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572519191 -3600
#  Thu Oct 31 11:53:11 2019 +0100
# Branch stable
# Node ID a29e42f5492266b779ab24ac4496211c947c6601
# Parent  fdc3af52305b805863d0178b9e8c7150b2495479
packaging: drop "support" for unsupported Fedora versions

Fedora 31 has just been released, and Fedora 29 will be EOL in a month. Don't
spend any time thinking about dead stuff.

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -189,9 +189,6 @@ packaging_targets := \
   docker-centos7 \
   docker-debian-jessie \
   docker-debian-stretch \
-  docker-fedora20 \
-  docker-fedora21 \
-  docker-fedora28 \
   docker-fedora29 \
   docker-ubuntu-trusty \
   docker-ubuntu-trusty-ppa \
@@ -201,9 +198,6 @@ packaging_targets := \
   docker-ubuntu-artful-ppa \
   docker-ubuntu-bionic \
   docker-ubuntu-bionic-ppa \
-  fedora20 \
-  fedora21 \
-  fedora28 \
   fedora29 \
   linux-wheels \
   linux-wheels-x86_64 \
diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -12,9 +12,6 @@ UBUNTU_CODENAMES := \
   disco
 
 FEDORA_RELEASES := \
-  20 \
-  21 \
-  28 \
   29
 
 CENTOS_RELEASES := \
diff --git a/contrib/packaging/docker/fedora20 
b/contrib/packaging/docker/fedora20
deleted file mode 100644
--- a/contrib/packaging/docker/fedora20
+++ /dev/null
@@ -1,15 +0,0 @@
-FROM fedora:20
-
-RUN groupadd -g 1000 build && \
-useradd -u 1000 -g 1000 -s /bin/bash -d /build -m build
-
-RUN yum install -y \
-   gcc \
-   gettext \
-   make \
-   python-devel \
-   python-docutils \
-   rpm-build
-
-# For creating repo meta data
-RUN yum install -y createrepo
diff --git a/contrib/packaging/docker/fedora21 
b/contrib/packaging/docker/fedora21
deleted file mode 100644
--- a/contrib/packaging/docker/fedora21
+++ /dev/null
@@ -1,15 +0,0 @@
-FROM fedora:21
-
-RUN groupadd -g 1000 build && \
-useradd -u 1000 -g 1000 -s /bin/bash -d /build -m build
-
-RUN yum install -y \
-   gcc \
-   gettext \
-   make \
-   python-devel \
-   python-docutils \
-   rpm-build
-
-# For creating repo meta data
-RUN yum install -y createrepo
diff --git a/contrib/packaging/docker/fedora28 
b/contrib/packaging/docker/fedora28
deleted file mode 100644
--- a/contrib/packaging/docker/fedora28
+++ /dev/null
@@ -1,15 +0,0 @@
-FROM fedora:28
-
-RUN groupadd -g 1000 build && \
-useradd -u 1000 -g 1000 -s /bin/bash -d /build -m build
-
-RUN dnf install -y \
-   gcc \
-   gettext \
-   make \
-   python-devel \
-   python-docutils \
-   rpm-build
-
-# For creating repo meta data
-RUN dnf install -y createrepo
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 7 stable] packaging: introduce generic local rpm build target using python3

[Mads Kiilerich]

Actually ... testing shows so many small differences between versions 
that break something. Even the very first patch in series seems to break 
docker-centos5.


Pleasedrop everything except the last 3 trivial patches (hgweb.wsgi, 
"tested on", and whitespace). I will resend.


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 7 stable] packaging: introduce generic local rpm build target using python3

[Mads Kiilerich]

On 10/28/19 12:37 AM, Mads Kiilerich wrote:

packaging: introduce generic local rpm build target using python3



I realize this change is bad. It has not been pushed yet - please drop 
it. The rest of the series still looks good.


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 7 of 7 stable] packaging: fix buildrpm whitespace

[Mads Kiilerich]

On 10/30/19 9:14 PM, Augie Fackler wrote:

On Mon, Oct 28, 2019 at 12:37:15AM +0100, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1572203819 -3600
#  Sun Oct 27 20:16:59 2019 +0100
# Branch stable
# Node ID c84f1465c44ebc539b803b876206712e0ebd78b4
# Parent  8c18adcd0177f3ca35f7f20f52f27f5a13ac9f90
packaging: fix buildrpm whitespace

Queued, many thanks - I have been meaning to try and fix our in-tree
RPM building for a while. Let's plan to switch this to py3 by default
for 5.2.1?



I can see Debian packaging already did switched to py3. So perhaps just 
do the same already now for 5.2.0? Change the buildrpm default to py3 
and provide a --python2 option? While it is late and risky for 5.2.0, it 
seems even more risky to do it for 5.2.1.


I don't know if it makes sense to switch over the old Fedora targets. 
Fedora 31 is out now, 28 was EOL 5 months ago, and 29 is EOL in a month. 
Perhaps just delete them and rely on the generic `make rpm` target instead?


The docker targets are a bit more tricky to make more durable and 
low-maintenance. They can't as easily be generic. But it could perhaps 
be done with something like `make docker-fedora FEDORA=31` .


For centos, we should perhaps keep 5 and 6 on py2 for now ... also 
because we don't yet have means for building our own py3 for these RPMs. 
Building Python is a bit more tricky than building Mercurial, and it 
would require more testing.


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 7 stable] packaging: introduce Python3 support as buildrpm --python3

[Mads Kiilerich]

Some of the changes in this series can perhaps wait, but it could 
perhaps also be nice to take it step further and match how deb packaging 
already moved everything to py3 ...


/Mads
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 7 stable] packaging: drop outdated buildrpm "tested on" comment

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572203798 -3600
#  Sun Oct 27 20:16:38 2019 +0100
# Branch stable
# Node ID 8c18adcd0177f3ca35f7f20f52f27f5a13ac9f90
# Parent  066475204d43572137083c64a02a813e1d78ac5a
packaging: drop outdated buildrpm "tested on" comment

Packaging usually works on other versions too, and it is not efficient to
maintain the list in repo. It is already out of sync with the Makefile targets.

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -1,11 +1,6 @@
 #!/bin/bash -e
 #
-# Build a Mercurial RPM from the current repo
-#
-# Tested on
-# - Fedora 20
-# - CentOS 5
-# - centOS 6
+# Build a Mercurial RPM from the current repo, mainly for Fedora/CentOS/RHEL
 
 . $(dirname $0)/packagelib.sh
 

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 7 stable] packaging: also include hgweb.wsgi in rpms

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572218982 -3600
#  Mon Oct 28 00:29:42 2019 +0100
# Branch stable
# Node ID 066475204d43572137083c64a02a813e1d78ac5a
# Parent  467d582886f688e01fe59b1d6f2de244d7be1458
packaging: also include hgweb.wsgi in rpms

diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -142,7 +142,7 @@ rm -rf $RPM_BUILD_ROOT
 
 %files
 %defattr(-,root,root,-)
-%doc CONTRIBUTORS COPYING doc/README doc/hg*.txt doc/hg*.html *.cgi 
contrib/*.fcgi
+%doc CONTRIBUTORS COPYING doc/README doc/hg*.txt doc/hg*.html *.cgi 
contrib/*.fcgi contrib/*.wsgi
 %doc %attr(644,root,root) %{_mandir}/man?/hg*
 %doc %attr(644,root,root) contrib/*.svg
 %dir %{_datadir}/zsh/

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 7 stable] packaging: introduce Python3 support as buildrpm --python3

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572208106 -3600
#  Sun Oct 27 21:28:26 2019 +0100
# Branch stable
# Node ID 32dbdaffb2ae030119890e05b87c4677706e7c6a
# Parent  6ce451ed79a3132bb6390210ca460a106d40b6db
packaging: introduce Python3 support as buildrpm --python3

Just overrule the HGPYTHON3 warning.

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -11,6 +11,7 @@
 
 BUILD=1
 RPMBUILDDIR="$PWD/rpmbuild"
+PYTHONEXE=python2
 
 while [ "$1" ]; do
 case "$1" in
@@ -18,6 +19,10 @@ while [ "$1" ]; do
 shift
 BUILD=
 ;;
+--python3)
+shift
+PYTHONEXE=python3
+;;
 --withpython | --with-python)
 shift
 PYTHONVER=2.7.16
@@ -96,6 +101,7 @@ rpmspec=$RPMBUILDDIR/SPECS/mercurial.spe
 
 sed -e "s,^Version:.*,Version: $version," \
 -e "s,^Release:.*,Release: $release," \
+-e "s/^%global pythonexe .*/%global pythonexe $PYTHONEXE/" \
 $specfile > $rpmspec
 
 echo >> $rpmspec
diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -38,8 +38,8 @@ BuildRequires: make, gcc, gettext
 %if "%{?withpython}"
 BuildRequires: readline-devel, openssl-devel, ncurses-devel, zlib-devel, 
bzip2-devel
 %else
-BuildRequires: python >= 2.7, python-devel, python-docutils >= 0.5
-Requires: python >= 2.7
+BuildRequires: python >= %{pythonver}, python-devel, python-docutils >= 0.5
+Requires: python >= %{pythonver}
 %endif
 # The hgk extension uses the wish tcl interpreter, but we don't enforce it
 #Requires: tk
@@ -60,6 +60,8 @@ sed -i '1c#! /usr/bin/env %{pythonexe}' 
 
 %build
 
+export HGPYTHON3=1
+
 %if "%{?withpython}"
 
 PYPATH=$PWD/%{pythonname}
@@ -91,6 +93,8 @@ sed -i -e '1s|#!/usr/bin/env python$|#!/
 %install
 rm -rf $RPM_BUILD_ROOT
 
+export HGPYTHON3=1
+
 %if "%{?withpython}"
 
 PYPATH=$PWD/%{pythonname}

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 7 stable] packaging: fix buildrpm whitespace

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572203819 -3600
#  Sun Oct 27 20:16:59 2019 +0100
# Branch stable
# Node ID c84f1465c44ebc539b803b876206712e0ebd78b4
# Parent  8c18adcd0177f3ca35f7f20f52f27f5a13ac9f90
packaging: fix buildrpm whitespace

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -51,7 +51,7 @@ fi
 gethgversion
 
 if [ -z "$type" ] ; then
-   release=1
+release=1
 else
 release=0.9_$type
 fi

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 7 stable] packaging: introduce generic local rpm build target using python3

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572198921 -3600
#  Sun Oct 27 18:55:21 2019 +0100
# Branch stable
# Node ID 467d582886f688e01fe59b1d6f2de244d7be1458
# Parent  32dbdaffb2ae030119890e05b87c4677706e7c6a
packaging: introduce generic local rpm build target using python3

It is unnecessary extra work to explicitly add Makefile targets for a random
selection of Fedora versions. Generally, it just works. If not, the root cause
should be fixed.

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -208,7 +208,8 @@ packaging_targets := \
   linux-wheels \
   linux-wheels-x86_64 \
   linux-wheels-i686 \
-  ppa
+  ppa \
+  rpm
 
 # Forward packaging targets for convenience.
 $(packaging_targets):
diff --git a/contrib/packaging/Makefile b/contrib/packaging/Makefile
--- a/contrib/packaging/Makefile
+++ b/contrib/packaging/Makefile
@@ -56,6 +56,9 @@ help:
@echo 'ppa'
@echo '   Build a Debian source package locally targeting the current 
system'
@echo ''
+   @echo 'rpm'
+   @echo '   Build a RPM targeting the current system using Python 3'
+   @echo ''
@echo 'centos{$(strip $(CENTOS_RELEASES))}'
@echo '   Build an RPM for a specific CentOS version locally'
@echo ''
@@ -96,6 +99,14 @@ endef
 
 $(foreach codename,$(UBUNTU_CODENAMES),$(eval $(call 
ubuntu_targets,$(codename
 
+# Generic RPM target, mainly for Fedora/CentOS/RHEL.
+rpm:
+   mkdir -p $$(HGROOT)/packages/rpm
+   ./buildrpm --python3
+   cp $$(HGROOT)/contrib/packaging/rpmbuild/RPMS/*/* 
$$(HGROOT)/packages/rpm
+   cp $$(HGROOT)/contrib/packaging/rpmbuild/SRPMS/* $$(HGROOT)/packages/rpm
+   rm -rf $(HGROOT)/rpmbuild
+
 # Fedora targets.
 define fedora_targets
 .PHONY: fedora$(1)

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 7 stable] packaging: be explicit about Python version in rpm spec

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572208821 -3600
#  Sun Oct 27 21:40:21 2019 +0100
# Branch stable
# Node ID 6ce451ed79a3132bb6390210ca460a106d40b6db
# Parent  496bbefb8fa0f24c789ae98f0b37e98046acb83c
packaging: be explicit about Python version in rpm spec

Fedora 31 has Python3 at /usr/bin/python ... but expect everybody to not just
find python in $PATH but be explicit about whether they want python2 or
python3. mercurial.spec just used 'python' and would fail when it unknowingly
used Python 3 and ended up with Mercurial setup.py reporting "Python 3.7
detected." and talking about the HGPYTHON3 environment variable.

For now, just be explicit about using system python2 as python executable when
building rpms.

diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -15,7 +15,8 @@
 
 %else
 
-%global pythonver %(python -c 'import sys;print(".".join(map(str, 
sys.version_info[:2])))')
+%global pythonexe python2
+%global pythonver %(%{pythonexe} -c 'import sys;print(".".join(map(str, 
sys.version_info[:2])))')
 
 %endif
 
@@ -52,7 +53,7 @@ for efficient handling of very large dis
 %if "%{?withpython}"
 %setup -q -n mercurial-%{version}-%{release} -a1 -a2
 # despite the comments in cgi.py, we do this to prevent rpmdeps from picking 
/usr/local/bin/python up
-sed -i '1c#! /usr/bin/env python' %{pythonname}/Lib/cgi.py
+sed -i '1c#! /usr/bin/env %{pythonexe}' %{pythonname}/Lib/cgi.py
 %else
 %setup -q -n mercurial-%{version}-%{release}
 %endif
@@ -82,9 +83,11 @@ export PYTHONPATH=$PWD/%{docutilsname}
 
 %endif
 
-make all
+make all PYTHON=%{pythonexe}
 make -C contrib/chg
 
+sed -i -e '1s|#!/usr/bin/env python$|#!/usr/bin/env %{pythonexe}|' 
contrib/hg-ssh
+
 %install
 rm -rf $RPM_BUILD_ROOT
 
@@ -101,14 +104,14 @@ cd %{docutilsname}
 LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install 
--root="$RPM_BUILD_ROOT"
 cd -
 
-PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install 
DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir}
+PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install PYTHON=%{pythonexe} 
DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir}
 mkdir -p $RPM_BUILD_ROOT%{_bindir}
 ( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . )
 ( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.? 
%{pythonhg} )
 
 %else
 
-make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir}
+make install PYTHON=%{pythonexe} DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} 
MANDIR=%{_mandir}
 
 %endif
 

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 7 stable] packaging: make python snippets in rpm building python3 compatible

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1572203853 -3600
#  Sun Oct 27 20:17:33 2019 +0100
# Branch stable
# Node ID 496bbefb8fa0f24c789ae98f0b37e98046acb83c
# Parent  a2ff3aff81d244685cf2f25a1ee92f379d1d115c
packaging: make python snippets in rpm building python3 compatible

Fedora 31 has Python3 at /usr/bin/python, and buildrpm would fail on snippets
that use python2 syntax. Instead of forcing python2, just accept for the future
while staying backwards compatible.

diff --git a/contrib/packaging/buildrpm b/contrib/packaging/buildrpm
--- a/contrib/packaging/buildrpm
+++ b/contrib/packaging/buildrpm
@@ -121,8 +121,8 @@ for l in sorted(changelog, reverse=True)
 if prevtitle != title:
 prevtitle = title
 print
-print title
-print "- %s" % l[3].strip()
+print(title)
+print("- %s" % l[3].strip())
 ' >> $rpmspec
 
 else
@@ -138,7 +138,7 @@ def datestr(date, format):
 for l in sys.stdin.readlines():
 tok = l.split("\t")
 hgdate = tuple(int(v) for v in tok[0].split())
-print "* %s %s\n- %s" % (datestr(hgdate, "%a %b %d %Y"), tok[1], tok[2])
+print("* %s %s\n- %s" % (datestr(hgdate, "%a %b %d %Y"), tok[1], tok[2]))
 ' >> $rpmspec
 
 fi
diff --git a/contrib/packaging/mercurial.spec b/contrib/packaging/mercurial.spec
--- a/contrib/packaging/mercurial.spec
+++ b/contrib/packaging/mercurial.spec
@@ -15,7 +15,7 @@
 
 %else
 
-%global pythonver %(python -c 'import sys;print ".".join(map(str, 
sys.version_info[:2]))')
+%global pythonver %(python -c 'import sys;print(".".join(map(str, 
sys.version_info[:2])))')
 
 %endif
 

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC] changegroup: leave out all parent file revisions when creating bundles

[Mads Kiilerich]

On 10/11/19 7:57 PM, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1570804632 -7200
#  Fri Oct 11 16:37:12 2019 +0200
# Node ID 72d12ee773795edc163f73b9160e5d29022878dd
# Parent  52781d57313d512efb7150603104bea3ca11d0eb
changegroup: leave out all parent file revisions when creating bundles


I'm not sure the analysis nailed the root cause. It is tricky to 
reproduce good test cases.


But one problem we could fix by pruning ancestors instead of using linkrevs:

Create a repo with aliasing:

  $ hg init repo1
  $ cd repo1
  $ touch f
  $ hg ci -Aqm 0
  $ echo 1 > f
  $ hg ci -m 1f1
  $ hg up -cqr 0
  $ hg branch -q b
  $ echo 1 > f  # linkrev aliasing to rev 1
  $ hg ci -m 2f1

When bundling rev 2 for a repo that has rev 1, f will be skipped even 
though it

isn't an ancestor:

  $ hg bundle -v -r 2 --base 1 bundle.hg
  1 changesets found
  uncompressed size of bundle content:
   185 (changelog)
   163 (manifests)

A bundle with missing ancestor revisions would fail unbundling with "abort:
00changelog.i@d681519c3ea7: unknown parent!". But if using 
fastpathlinkrev and
the ancestors are present but the file revs are not, the bundle can be 
applied

but will break on use:

  $ hg clone -qU . -r 0 repo2
  $ hg -R repo2 pull bundle.hg
  pulling from bundle.hg
  searching for changes
  adding changesets
  adding manifests
  adding file changes
  added 1 changesets with 0 changes to 0 files
  new changesets 5e690c649d09 (1 drafts)
  (run 'hg update' to get a working copy)
  $ hg -R repo2 up -r tip
  abort: data/f.i@d0c79e1d3309: no match found!
  [255]


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 8] eol: don't fallback to use .hgeol from tip (BC)

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570925426 -7200
#  Sun Oct 13 02:10:26 2019 +0200
# Node ID 74c8ac8124723f06a612ff560403d4bd288ec5f4
# Parent  68dde01bb05c782d484d96e424512b89e0673bec
eol: don't fallback to use .hgeol from tip (BC)

If no .hgeol were found in the current working directory, eol would fallback to
use the one in tip. That could in some cases give very confusing or wrong
behaviour when it applied wrong filters.

It might be convenient to have plain 'clone' immediately apply 'native'
encoding patterns in the cloned repo. But it is wrong to assume that this
revision is tip, and even more wrong to also apply it when not cloning - for
example when updating between history revisions. The encoding should always
match the content of the current .hgeol . It should never use anything else.

diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -387,7 +387,7 @@ def reposetup(ui, repo):
 return eol.match
 
 def _hgcleardirstate(self):
-self._eolmatch = self.loadeol([None, b'tip'])
+self._eolmatch = self.loadeol([None])
 if not self._eolmatch:
 self._eolmatch = util.never
 return
diff --git a/tests/test-eol-clone.t b/tests/test-eol-clone.t
--- a/tests/test-eol-clone.t
+++ b/tests/test-eol-clone.t
@@ -21,9 +21,8 @@ setup repository
   adding .hgeol
   adding a.txt
 
-Test commit of removed .hgeol - currently it seems to live on as zombie
-(causing "filtering a.txt through tolf") after being removed ... but actually
-it is just confusing use of tip revision.
+Test commit of removed .hgeol and how it immediately makes the automatic
+changes explicit and committable.
 
   $ cd ..
   $ hg clone repo repo-2
@@ -41,7 +40,7 @@ it is just confusing use of tip revision
   $ hg remove .hgeol
   $ touch a.txt *  # ensure consistent st dirtyness checks, ignoring dirstate 
timing
   $ hg st -v --debug
-  filtering a.txt through tolf
+  M a.txt
   R .hgeol
   $ hg commit -m 'remove eol'
   $ hg exp
@@ -49,16 +48,26 @@ it is just confusing use of tip revision
   # User test
   # Date 0 0
   #  Thu Jan 01 00:00:00 1970 +
-  # Node ID c60b96c20c7de8c821127b548c34e5b170bcf9fe
+  # Node ID 3c20c2d90333b6ecdc8f7aa8f9b73223c7c7a608
   # Parent  90f94e2cf4e24628afddd641688dfe4cd476d6e4
   remove eol
   
-  diff -r 90f94e2cf4e2 -r c60b96c20c7d .hgeol
+  diff -r 90f94e2cf4e2 -r 3c20c2d90333 .hgeol
   --- a/.hgeol Thu Jan 01 00:00:00 1970 +
   +++ /dev/nullThu Jan 01 00:00:00 1970 +
   @@ -1,2 +0,0 @@
   -[patterns]
   -**.txt = native
+  diff -r 90f94e2cf4e2 -r 3c20c2d90333 a.txt
+  --- a/a.txt  Thu Jan 01 00:00:00 1970 +
+  +++ b/a.txt  Thu Jan 01 00:00:00 1970 +
+  @@ -1,3 +1,3 @@
+  -first
+  -second
+  -third
+  +first\r (esc)
+  +second\r (esc)
+  +third\r (esc)
   $ hg push --quiet
   $ cd ..
 
@@ -75,7 +84,7 @@ the source repo:
   updating to branch default
   resolving manifests
branchmerge: False, force: False, partial: False
-   ancestor: , local: +, remote: c60b96c20c7d
+   ancestor: , local: +, remote: 3c20c2d90333
   calling hook preupdate.eol: hgext.eol.preupdate
a.txt: remote created -> g
   getting a.txt
@@ -83,9 +92,9 @@ the source repo:
   $ cd repo-3
 
   $ cat a.txt
-  first
-  second
-  third
+  first\r (esc)
+  second\r (esc)
+  third\r (esc)
 
 Test clone of revision with .hgeol
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8] eol: tweak test-eol-clone.t with better descriptions and logging

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1571010144 -7200
#  Mon Oct 14 01:42:24 2019 +0200
# Node ID 68dde01bb05c782d484d96e424512b89e0673bec
# Parent  950c89a71e9837dc81da80ed34eee9b602a808d9
eol: tweak test-eol-clone.t with better descriptions and logging

Expose impact of changes coming next ...

diff --git a/tests/test-eol-clone.t b/tests/test-eol-clone.t
--- a/tests/test-eol-clone.t
+++ b/tests/test-eol-clone.t
@@ -21,7 +21,9 @@ setup repository
   adding .hgeol
   adding a.txt
 
-Clone
+Test commit of removed .hgeol - currently it seems to live on as zombie
+(causing "filtering a.txt through tolf") after being removed ... but actually
+it is just confusing use of tip revision.
 
   $ cd ..
   $ hg clone repo repo-2
@@ -37,14 +39,46 @@ Clone
   second
   third
   $ hg remove .hgeol
+  $ touch a.txt *  # ensure consistent st dirtyness checks, ignoring dirstate 
timing
+  $ hg st -v --debug
+  filtering a.txt through tolf
+  R .hgeol
   $ hg commit -m 'remove eol'
+  $ hg exp
+  # HG changeset patch
+  # User test
+  # Date 0 0
+  #  Thu Jan 01 00:00:00 1970 +
+  # Node ID c60b96c20c7de8c821127b548c34e5b170bcf9fe
+  # Parent  90f94e2cf4e24628afddd641688dfe4cd476d6e4
+  remove eol
+  
+  diff -r 90f94e2cf4e2 -r c60b96c20c7d .hgeol
+  --- a/.hgeol Thu Jan 01 00:00:00 1970 +
+  +++ /dev/nullThu Jan 01 00:00:00 1970 +
+  @@ -1,2 +0,0 @@
+  -[patterns]
+  -**.txt = native
   $ hg push --quiet
   $ cd ..
 
-Test clone of repo with .hgeol in working dir, but no .hgeol in tip
+Test clone of repo with .hgeol in working dir, but no .hgeol in default
+checkout revision tip. The repo is correctly updated to be consistent and have
+the exact content checked out without filtering, ignoring the current .hgeol in
+the source repo:
 
-  $ hg clone repo repo-3
+  $ cat repo/.hgeol
+  [patterns]
+  **.txt = native
+  $ hg clone repo repo-3 -v --debug
+  linked 7 files
   updating to branch default
+  resolving manifests
+   branchmerge: False, force: False, partial: False
+   ancestor: , local: +, remote: c60b96c20c7d
+  calling hook preupdate.eol: hgext.eol.preupdate
+   a.txt: remote created -> g
+  getting a.txt
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cd repo-3
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 8] eol: fix update - don't use and apply removed .hgeol patterns

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570925707 -7200
#  Sun Oct 13 02:15:07 2019 +0200
# Node ID 950c89a71e9837dc81da80ed34eee9b602a808d9
# Parent  844fa54a1ffbc052fbcd76f103b3db6b02a84690
eol: fix update - don't use and apply removed .hgeol patterns

'hg up -C' to revisions with different .hgeol patterns could leave dirty
changes in the working directory. That could make deployment of new .hgeol
filters tricky: they would "occasionally" apply also in branches where they
shouldn't.

Fixed by dropping all old patterns before applying new ones.

diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -221,6 +221,12 @@ class eolfile(object):
 self.match = match.match(root, b'', [], include, exclude)
 
 def copytoui(self, ui):
+newpatterns = set(pattern for pattern, key, m in self.patterns)
+for section in (b'decode', b'encode'):
+for oldpattern, _filter in ui.configitems(section):
+if oldpattern not in newpatterns:
+if ui.configsource(section, oldpattern) == b'eol':
+ui.setconfig(section, oldpattern, b'!', b'eol')
 for pattern, key, m in self.patterns:
 try:
 ui.setconfig(b'decode', pattern, self._decode[key], b'eol')
diff --git a/tests/test-eol-update.t b/tests/test-eol-update.t
--- a/tests/test-eol-update.t
+++ b/tests/test-eol-update.t
@@ -120,12 +120,6 @@ Test EOL update
first
   -second
third
-  diff --git a/f b/f
-  --- a/f
-  +++ b/f
-  @@ -1,1 +1,1 @@
-  -f\r (esc)
-  +f
   $ dotest CRLF
   
   % hg clone repo repo-CRLF
@@ -159,12 +153,6 @@ Test EOL update
first
   -second
third
-  diff --git a/f b/f
-  --- a/f
-  +++ b/f
-  @@ -1,1 +1,1 @@
-  -f\r (esc)
-  +f
 
 Test in repo using eol extension, while keeping an eye on how filters are
 applied:
@@ -177,8 +165,8 @@ applied:
   > eol =
   > EOF
 
-Update to revision 0 which has no .hgeol . Unfortunately, it uses the filter
-from tip ... which evidently is wrong:
+Update to revision 0 which has no .hgeol, shouldn't use any filters, and
+obviously should leave things as tidy as they were before the clean update.
 
   $ hg up -c -r 0 -v --debug
   resolving manifests
@@ -193,22 +181,8 @@ from tip ... which evidently is wrong:
   filtering a.txt through tolf
f: remote created -> g
   getting f
-  filtering f through tolf
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg st
-  M f
-  $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
-  $ hg up -C -r 0 -v --debug
-  eol: detected change in .hgeol
-  filtering .hgeol through isbinary
-  filtering a.txt through tolf
-  resolving manifests
-   branchmerge: False, force: True, partial: False
-   ancestor: 15cbdf8ca3db+, local: 15cbdf8ca3db+, remote: 15cbdf8ca3db
-  calling hook preupdate.eol: hgext.eol.preupdate
-   f: remote is newer -> g
-  getting f
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ hg branch b
   marked working directory as branch b
@@ -250,9 +224,9 @@ Merge changes that apply a filter to f:
   -f\r (esc)
   +f
 
-Abort the merge with up -C to revision 0 ... but notice how .hgeol changes are
-not detected correctly: f is filtered with tolf even though there is no filter
-for f in revision 0, and it thus ends up with working directory changes.
+Abort the merge with up -C to revision 0.
+Note that files are filtered correctly for revision 0: f is not filtered, a.txt
+is filtered with tolf, and everything is left tidy.
 
   $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
   $ hg up -C -r 0 -v --debug
@@ -269,7 +243,6 @@ for f in revision 0, and it thus ends up
   filtering a.txt through tolf
f: remote is newer -> g
   getting f
-  filtering f through tolf
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ touch .hgeol *
@@ -277,16 +250,9 @@ for f in revision 0, and it thus ends up
   eol: detected change in .hgeol
   filtering .hgeol through isbinary
   filtering a.txt through tolf
-  M f
   $ hg diff
-  diff --git a/f b/f
-  --- a/f
-  +++ b/f
-  @@ -1,1 +1,1 @@
-  -f\r (esc)
-  +f
 
-Workaround: Update again - this will read the right .hgeol:
+Things were clean, and updating again will not change anything:
 
   $ touch .hgeol *
   $ hg up -C -r 0 -v --debug
@@ -297,9 +263,7 @@ Workaround: Update again - this will rea
branchmerge: False, force: True, partial: False
ancestor: 15cbdf8ca3db+, local: 15cbdf8ca3db+, remote: 15cbdf8ca3db
   calling hook preupdate.eol: hgext.eol.preupdate
-   f: remote is newer -> g
-  getting f
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ touch .hgeol *
   $ hg st --debug
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://w

[PATCH 4 of 8] eol: update isbinary filter to work without compat wrapper

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1571009598 -7200
#  Mon Oct 14 01:33:18 2019 +0200
# Node ID 5670699747c92d725927e25754817d5aa498d728
# Parent  1e663f4a658dbf6669c9bfd53918bf1daa734dc6
eol: update isbinary filter to work without compat wrapper

diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -165,7 +165,7 @@ def tocrlf(s, params, ui, **kwargs):
 return util.tocrlf(s)
 
 
-def isbinary(s, params):
+def isbinary(s, params, ui, **kwargs):
 """Filter to do nothing with the file."""
 return s
 
diff --git a/tests/test-eol-update.t b/tests/test-eol-update.t
--- a/tests/test-eol-update.t
+++ b/tests/test-eol-update.t
@@ -187,7 +187,7 @@ from tip ... which evidently is wrong:
   calling hook preupdate.eol: hgext.eol.preupdate
.hgeol: remote created -> g
   getting .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
a.txt: remote created -> g
   getting a.txt
   filtering a.txt through tolf
@@ -200,7 +200,7 @@ from tip ... which evidently is wrong:
   $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
   $ hg up -C -r 0 -v --debug
   eol: detected change in .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
   filtering a.txt through tolf
   resolving manifests
branchmerge: False, force: True, partial: False
@@ -263,7 +263,7 @@ for f in revision 0, and it thus ends up
   calling hook preupdate.eol: hgext.eol.preupdate
.hgeol: remote is newer -> g
   getting .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
a.txt: remote is newer -> g
   getting a.txt
   filtering a.txt through tolf
@@ -275,7 +275,7 @@ for f in revision 0, and it thus ends up
   $ touch .hgeol *
   $ hg st --debug
   eol: detected change in .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
   filtering a.txt through tolf
   M f
   $ hg diff
@@ -291,7 +291,7 @@ Workaround: Update again - this will rea
   $ touch .hgeol *
   $ hg up -C -r 0 -v --debug
   eol: detected change in .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
   filtering a.txt through tolf
   resolving manifests
branchmerge: False, force: True, partial: False
@@ -304,7 +304,7 @@ Workaround: Update again - this will rea
   $ touch .hgeol *
   $ hg st --debug
   eol: detected change in .hgeol
-  filtering .hgeol through compat-isbinary
+  filtering .hgeol through isbinary
   filtering a.txt through tolf
 
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8] localrepo: fix variable binding in handling of old filters

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570925119 -7200
#  Sun Oct 13 02:05:19 2019 +0200
# Node ID 1e663f4a658dbf6669c9bfd53918bf1daa734dc6
# Parent  2b91375a812ce3c694efa35a98a109387962
localrepo: fix variable binding in handling of old filters

The lambda was referencing oldfn in outer scope without binding the current
value. If oldfn function were reassigned before use, wrong filters could be
used.

Fixed by having oldfn as named parameter default value of the lambda.

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1900,7 +1900,7 @@ class localrepository(object):
 # Wrap old filters not supporting keyword arguments
 if not pycompat.getargspec(fn)[2]:
 oldfn = fn
-fn = lambda s, c, **kwargs: oldfn(s, c)
+fn = lambda s, c, oldfn=oldfn, **kwargs: oldfn(s, c)
 fn.__name__ = 'compat-' + oldfn.__name__
 l.append((mf, fn, params))
 self._filterpats[filter] = l
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8] eol: cache needs update, also if it has same timestamp as the source

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570925493 -7200
#  Sun Oct 13 02:11:33 2019 +0200
# Node ID 844fa54a1ffbc052fbcd76f103b3db6b02a84690
# Parent  5670699747c92d725927e25754817d5aa498d728
eol: cache needs update, also if it has same timestamp as the source

Ignoring same timestamp could (in theory?) cause changes to not be detected.

It might happen quite often that the cache is populated right after .hgeol has
been updated and they thus have the same time stamp second. But we want
correctness, and if it populates the cache so fast, then it can also not be a
big problem to run it again next time when the timestamp has moved on.

diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -401,7 +401,7 @@ def reposetup(ui, repo):
 except OSError:
 eolmtime = 0
 
-if eolmtime > cachemtime:
+if eolmtime >= cachemtime and eolmtime > 0:
 self.ui.debug(b"eol: detected change in .hgeol\n")
 
 hgeoldata = self.wvfs.read(b'.hgeol')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8] localrepo: debug log of filter name when filtering through a function

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570970400 -7200
#  Sun Oct 13 14:40:00 2019 +0200
# Node ID 2b91375a812ce3c694efa35a98a109387962
# Parent  57f88532f7005eeb13cb06418ae3a3b156085adf
localrepo: debug log of filter name when filtering through a function

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1896,10 +1896,12 @@ class localrepository(object):
 break
 if not fn:
 fn = lambda s, c, **kwargs: procutil.filter(s, c)
+fn.__name__ = 'commandfilter'
 # Wrap old filters not supporting keyword arguments
 if not pycompat.getargspec(fn)[2]:
 oldfn = fn
 fn = lambda s, c, **kwargs: oldfn(s, c)
+fn.__name__ = 'compat-' + oldfn.__name__
 l.append((mf, fn, params))
 self._filterpats[filter] = l
 return self._filterpats[filter]
@@ -1907,7 +1909,8 @@ class localrepository(object):
 def _filter(self, filterpats, filename, data):
 for mf, fn, cmd in filterpats:
 if mf(filename):
-self.ui.debug(b"filtering %s through %s\n" % (filename, cmd))
+self.ui.debug(b"filtering %s through %s\n" %
+  (filename, cmd or pycompat.bytestr(fn.__name__)))
 data = fn(data, cmd, ui=self.ui, repo=self, filename=filename)
 break
 
diff --git a/tests/test-eol-update.t b/tests/test-eol-update.t
--- a/tests/test-eol-update.t
+++ b/tests/test-eol-update.t
@@ -187,21 +187,21 @@ from tip ... which evidently is wrong:
   calling hook preupdate.eol: hgext.eol.preupdate
.hgeol: remote created -> g
   getting .hgeol
-  filtering .hgeol through 
+  filtering .hgeol through compat-isbinary
a.txt: remote created -> g
   getting a.txt
-  filtering a.txt through 
+  filtering a.txt through tolf
f: remote created -> g
   getting f
-  filtering f through 
+  filtering f through tolf
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg st
   M f
   $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
   $ hg up -C -r 0 -v --debug
   eol: detected change in .hgeol
-  filtering .hgeol through 
-  filtering a.txt through 
+  filtering .hgeol through compat-isbinary
+  filtering a.txt through tolf
   resolving manifests
branchmerge: False, force: True, partial: False
ancestor: 15cbdf8ca3db+, local: 15cbdf8ca3db+, remote: 15cbdf8ca3db
@@ -263,20 +263,20 @@ for f in revision 0, and it thus ends up
   calling hook preupdate.eol: hgext.eol.preupdate
.hgeol: remote is newer -> g
   getting .hgeol
-  filtering .hgeol through 
+  filtering .hgeol through compat-isbinary
a.txt: remote is newer -> g
   getting a.txt
-  filtering a.txt through 
+  filtering a.txt through tolf
f: remote is newer -> g
   getting f
-  filtering f through 
+  filtering f through tolf
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ touch .hgeol *
   $ hg st --debug
   eol: detected change in .hgeol
-  filtering .hgeol through 
-  filtering a.txt through 
+  filtering .hgeol through compat-isbinary
+  filtering a.txt through tolf
   M f
   $ hg diff
   diff --git a/f b/f
@@ -291,8 +291,8 @@ Workaround: Update again - this will rea
   $ touch .hgeol *
   $ hg up -C -r 0 -v --debug
   eol: detected change in .hgeol
-  filtering .hgeol through 
-  filtering a.txt through 
+  filtering .hgeol through compat-isbinary
+  filtering a.txt through tolf
   resolving manifests
branchmerge: False, force: True, partial: False
ancestor: 15cbdf8ca3db+, local: 15cbdf8ca3db+, remote: 15cbdf8ca3db
@@ -304,8 +304,8 @@ Workaround: Update again - this will rea
   $ touch .hgeol *
   $ hg st --debug
   eol: detected change in .hgeol
-  filtering .hgeol through 
-  filtering a.txt through 
+  filtering .hgeol through compat-isbinary
+  filtering a.txt through tolf
 
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 8] eol: test-eol-update.t coverage around update --clean using filters ... badly

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1571004565 -7200
#  Mon Oct 14 00:09:25 2019 +0200
# Node ID 57f88532f7005eeb13cb06418ae3a3b156085adf
# Parent  52781d57313d512efb7150603104bea3ca11d0eb
eol: test-eol-update.t coverage around update --clean using filters ... badly

This will reveal problems and track their fixes.

diff --git a/tests/test-eol-update.t b/tests/test-eol-update.t
--- a/tests/test-eol-update.t
+++ b/tests/test-eol-update.t
@@ -26,14 +26,17 @@ Test EOL update
   > EOF
   > 
   > printf "first\nsecond\nthird\n" > a.txt
+  > printf "f\r\n" > f
   > hg commit --addremove -m 'LF commit'
   > 
   > cat > .hgeol < [patterns]
   > **.txt = CRLF
+  > f = LF
   > EOF
   > 
   > printf "first\r\nsecond\r\nthird\r\n" > a.txt
+  > printf "f\n" > f
   > hg commit -m 'CRLF commit'
   > 
   > cd ..
@@ -83,10 +86,11 @@ Test EOL update
   % hg init
   adding .hgeol
   adding a.txt
+  adding f
   $ dotest LF
   
   % hg clone repo repo-LF
-  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   % a.txt (before)
   first\r (esc)
   second\r (esc)
@@ -104,7 +108,7 @@ Test EOL update
third\r (esc)
   % hg update 0
   merging a.txt
-  1 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  2 files updated, 1 files merged, 0 files removed, 0 files unresolved
   % a.txt
   first
   third
@@ -116,10 +120,16 @@ Test EOL update
first
   -second
third
+  diff --git a/f b/f
+  --- a/f
+  +++ b/f
+  @@ -1,1 +1,1 @@
+  -f\r (esc)
+  +f
   $ dotest CRLF
   
   % hg clone repo repo-CRLF
-  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
   % a.txt (before)
   first\r (esc)
   second\r (esc)
@@ -137,7 +147,7 @@ Test EOL update
third\r (esc)
   % hg update 0
   merging a.txt
-  1 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  2 files updated, 1 files merged, 0 files removed, 0 files unresolved
   % a.txt
   first
   third
@@ -149,4 +159,154 @@ Test EOL update
first
   -second
third
+  diff --git a/f b/f
+  --- a/f
+  +++ b/f
+  @@ -1,1 +1,1 @@
+  -f\r (esc)
+  +f
+
+Test in repo using eol extension, while keeping an eye on how filters are
+applied:
+
+  $ cd repo
+
+  $ hg up -q -c -r null
+  $ cat > .hg/hgrc < [extensions]
+  > eol =
+  > EOF
+
+Update to revision 0 which has no .hgeol . Unfortunately, it uses the filter
+from tip ... which evidently is wrong:
+
+  $ hg up -c -r 0 -v --debug
+  resolving manifests
+   branchmerge: False, force: False, partial: False
+   ancestor: , local: +, remote: 15cbdf8ca3db
+  calling hook preupdate.eol: hgext.eol.preupdate
+   .hgeol: remote created -> g
+  getting .hgeol
+  filtering .hgeol through 
+   a.txt: remote created -> g
+  getting a.txt
+  filtering a.txt through 
+   f: remote created -> g
+  getting f
+  filtering f through 
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg st
+  M f
+  $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
+  $ hg up -C -r 0 -v --debug
+  eol: detected change in .hgeol
+  filtering .hgeol through 
+  filtering a.txt through 
+  resolving manifests
+   branchmerge: False, force: True, partial: False
+   ancestor: 15cbdf8ca3db+, local: 15cbdf8ca3db+, remote: 15cbdf8ca3db
+  calling hook preupdate.eol: hgext.eol.preupdate
+   f: remote is newer -> g
+  getting f
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+  $ hg branch b
+  marked working directory as branch b
+  (branches are permanent and global, did you want a bookmark?)
+  $ hg ci -m b
+
+Merge changes that apply a filter to f:
+
+  $ hg merge 1
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ hg st
+  M .hgeol
+  M a.txt
+  M f
+  $ hg diff
+  diff --git a/.hgeol b/.hgeol
+  --- a/.hgeol
+  +++ b/.hgeol
+  @@ -1,2 +1,3 @@
+   [patterns]
+  -**.txt = LF
+  +**.txt = CRLF
+  +f = LF
+  diff --git a/a.txt b/a.txt
+  --- a/a.txt
+  +++ b/a.txt
+  @@ -1,3 +1,3 @@
+  -first
+  -second
+  -third
+  +first\r (esc)
+  +second\r (esc)
+  +third\r (esc)
+  diff --git a/f b/f
+  --- a/f
+  +++ b/f
+  @@ -1,1 +1,1 @@
+  -f\r (esc)
+  +f
+
+Abort the merge with up -C to revision 0 ... but notice how .hgeol changes are
+not detected correctly: f is filtered with tolf even though there is no filter
+for f in revision 0, and it thus ends up with working directory changes.
+
+  $ touch .hgeol *  # ensure consistent dirtyness checks ignoring dirstate
+  $ hg up -C -r 0 -v --debug
+  eol: detected change in .hgeol
+  resolving manifests
+   branchmerge: False, force: True, partial: False
+   ancestor: 1db78bdd3bd

[PATCH RFC] changegroup: leave out all parent file revisions when creating bundles

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1570804632 -7200
#  Fri Oct 11 16:37:12 2019 +0200
# Node ID 72d12ee773795edc163f73b9160e5d29022878dd
# Parent  52781d57313d512efb7150603104bea3ca11d0eb
changegroup: leave out all parent file revisions when creating bundles

When creating a bundle, we get the revisions to bundle, and also the set of
revisions that have been discovered to be common between sender and receiver of
the bundle - which by definition will include all the ancestor revisions. File
revisions where linkrev points at common revisions will be pruned from the
bundle. The common revisions can be represented lazy and efficiently.

That is making the optimistic assumption that the linkrev revision is the
common one. Sometimes it isn't. In that case, the bundle will contain file
revisions that already are common. The increased size of the bundle might be a
problem, but it also makes the bundle size a useless approximation of
information content, and of how much it actually will increase repository size.

The linkrev based pruning does, however, have the advantage that it can leave
out common file revisions, even without being ancestors. Such aggressive
pruning will create bundles that only apply in a specific context, and
effectively make them depend on additional parents than the actual bundle
parents. It is debatable if that is a good feature and how much it actually
gains, but let's leave that out of scope and keep it for now.

We want to avoid that bundles contain filelog entries that are present in
parent revisions of the bundle content. We will do that by enumerating the
files that seem to be changed anywhere in the bundled set, find all parent
revisions of the bundled changesets, for the changed files find the set of file
revisions in the parent changesets, and leave these out when bundling.

To avoid visiting all base revisions for each file, we start by collecting
revisions of all touched files in all parents and keep that in memory. That
will be O(files_changed_in_set * revisions) ... but while bundled changesets
might be complex internally and all could have two unique parents outside the
bundle, they rarely have a lot of parents outside the set. It is also rare to
change a lot of files ... but even then it seems reasonable to be able to hold
the names of all files and some hashes in memory.) (Alternatively, we could
keep all manifests around and look files up on demand ... but manifests might
be big and changes are usually sparse, so it seems more efficient to compute
the sets upfront.)

Testing with the current verbose logging shows where this change makes a
difference ... including a few "because parents, not linkrev" places where this
change will have a positive impact. The changes generally seem reasonable to me
...

TODO:
gather feedback
more testing
investigate the
test-remotefilelog-bgprefetch.t instability
reduce the current logging

diff --git a/hgext/remotefilelog/shallowbundle.py 
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -71,7 +71,8 @@ class shallowcg1packer(changegroup.cgpac
 try:
 linknodes, commonrevs, source = args
 except ValueError:
-commonrevs, source, mfdicts, fastpathlinkrev, fnodes, clrevs = args
+(commonrevs, source, mfdicts, fastpathlinkrev, fnodes, clrevs,
+ setparents) = args
 if shallowutil.isenabled(self._repo):
 repo = self._repo
 if isinstance(repo, bundlerepo.bundlerepository):
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -931,6 +931,9 @@ class cgpacker(object):
 clrevorder = clstate[b'clrevorder']
 manifests = clstate[b'manifests']
 changedfiles = clstate[b'changedfiles']
+setparents = sorted(set(clstate[b'parentrevs'])
+.difference(clstate[b'changerevs'])
+.difference([-1]))
 
 # We need to make sure that the linkrev in the changegroup refers to
 # the first changeset that introduced the manifest or file revision.
@@ -1005,6 +1008,7 @@ class cgpacker(object):
 fastpathlinkrev,
 fnodes,
 clrevs,
+setparents,
 )
 
 for path, deltas in it:
@@ -1044,12 +1048,16 @@ class cgpacker(object):
 mfl = self._repo.manifestlog
 changedfiles = set()
 clrevtomanifestrev = {}
+changerevs = set()
+parentrevs = set()
 
 state = {
 b'clrevorder': clrevorder,
 b'manifests': manifests,
 b'changedfiles': changedfiles,
 b'clrevtomanifestrev': clrevtomanifestrev,
+b'changerevs': changerevs,
+b'parentrevs': parentrevs,
 }
 
 if not (generate or self._ellipses):
@@ -1106,6 +1114,9 @@ class cgpac

Re: [PATCH v2] graft: introduce --base option for using custom base revision while merging

[Mads Kiilerich]

On 10/23/18 4:59 PM, Augie Fackler wrote:

On Oct 14, 2018, at 11:15, Mads Kiilerich  wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1539529698 -7200
#  Sun Oct 14 17:08:18 2018 +0200
# Node ID 258029c642d97ef663396476c63ce34dbef89b13
# Parent  38ac525b44c93fcadb3680d4ded56f1e5a0029b2
graft: introduce --base option for using custom base revision while merging

queued



Thanks.



At the sprint you had mentioned maybe being able to use this to help backing 
out a merge - if you know the incantation for that maybe send that as a 
follow-up for the verbose help?


IIRC, graft has other safety mechanisms that prevent using a descendant 
as base. The primary use of this is to "graft merges".


Backout of merges should be possible with existing backout 
functionality. The change help hinted in that direction:



+
+  For using non-ancestors as the base to backout changes, see the backout
+  command and the hidden --parent option.



(But ok, backout could use more helpful help too.)

/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Extra files in changelog "changed files" and bundles

[Mads Kiilerich]

Hi

I see some odd and potentially related behaviour:

The files list in changelog entries for merges sometimes contain files 
that didn't change, but where the manifest use the same filelog entry as 
one of the parents.

It seems to often be related to merge of file moves.

Also, sometimes there are different filelog entries, but 
debugrevlogindex shows rename entries (p1=null) with length 0 and where 
debugdata confirms that the content is the same as their p2.


And because of these extra "changedfiles" entries, bundles sometimes 
contains too many files. Linkrev is used for pruning, but is too fragile 
to handle it correctly. (I have a PoC for pruning all filelog entries 
that are reused from the parents "common" and outside the bundled set.) 
(It also doesn't seem to use an optimal delta parent when bundling - 
that could perhaps mitigate it.)


Sorry for the vague description. I have not been fully able to reproduce 
the problems - perhaps because bugs have been fixed. Are you aware of 
changes/fixes that can explain this? Or hints about where to look or 
related problems?


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH v2] graft: introduce --base option for using custom base revision while merging

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1539529698 -7200
#  Sun Oct 14 17:08:18 2018 +0200
# Node ID 258029c642d97ef663396476c63ce34dbef89b13
# Parent  38ac525b44c93fcadb3680d4ded56f1e5a0029b2
graft: introduce --base option for using custom base revision while merging

The graft command usually performs an internal merge of the current parent
revision with the graft revision, using p1 of the grafted revision as base for
the merge.

As a trivial extension of this, we introduce the --base option to allow for
using another base revision.

This can be used as a building block for grafting and collapsing multiple
changesets at once, or for grafting the resulting change from a merge as a
single simple change. (This is kind of similar to backout --parent ... only
different: this graft base must be an ancestor, but is usually *not* a parent.)

This is probably an advanced use case, and we do thus not show it in the
non-verbose help.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -2223,6 +2223,8 @@ def forget(ui, repo, *pats, **opts):
 @command(
 'graft',
 [('r', 'rev', [], _('revisions to graft'), _('REV')),
+ ('', 'base', '',
+  _('base revision when doing the graft merge (ADVANCED)'), _('REV')),
  ('c', 'continue', False, _('resume interrupted graft')),
  ('', 'stop', False, _('stop interrupted graft')),
  ('', 'abort', False, _('abort interrupted graft')),
@@ -2267,6 +2269,35 @@ def graft(ui, repo, *revs, **opts):
 
 .. container:: verbose
 
+  The --base option exposes more of how graft internally uses merge with a
+  custom base revision. --base can be used to specify another ancestor than
+  the first and only parent.
+
+  The command::
+
+hg graft -r 345 --base 234
+
+  is thus pretty much the same as::
+
+hg diff -r 234 -r 345 | hg import
+
+  but using merge to resolve conflicts and track moved files.
+
+  The result of a merge can thus be backported as a single commit by
+  specifying one of the merge parents as base, and thus effectively
+  grafting the changes from the other side.
+
+  It is also possible to collapse multiple changesets and clean up history
+  by specifying another ancestor as base, much like rebase --collapse
+  --keep.
+
+  The commit message can be tweaked after the fact using commit --amend .
+
+  For using non-ancestors as the base to backout changes, see the backout
+  command and the hidden --parent option.
+
+.. container:: verbose
+
   Examples:
 
   - copy a single change to the stable branch and edit its description::
@@ -2290,6 +2321,15 @@ def graft(ui, repo, *revs, **opts):
 
   hg log -r "sort(all(), date)"
 
+  - backport the result of a merge as a single commit::
+
+  hg graft -r 123 --base 123^
+
+  - land a feature branch as one changeset::
+
+  hg up -cr default
+  hg graft -r featureX --base "ancestor('featureX', 'default')"
+
 See :hg:`help revisions` for more about specifying revisions.
 
 Returns 0 on successful completion.
@@ -2305,6 +2345,9 @@ def _dograft(ui, repo, *revs, **opts):
 
 revs = list(revs)
 revs.extend(opts.get('rev'))
+basectx = None
+if opts.get('base'):
+basectx = scmutil.revsingle(repo, opts['base'], None)
 # a dict of data to be stored in state file
 statedata = {}
 # list of new nodes created by ongoing graft
@@ -2384,13 +2427,16 @@ def _dograft(ui, repo, *revs, **opts):
 revs = scmutil.revrange(repo, revs)
 
 skipped = set()
-# check for merges
-for rev in repo.revs('%ld and merge()', revs):
-ui.warn(_('skipping ungraftable merge revision %d\n') % rev)
-skipped.add(rev)
+if basectx is None:
+# check for merges
+for rev in repo.revs('%ld and merge()', revs):
+ui.warn(_('skipping ungraftable merge revision %d\n') % rev)
+skipped.add(rev)
 revs = [r for r in revs if r not in skipped]
 if not revs:
 return -1
+if basectx is not None and len(revs) != 1:
+raise error.Abort(_('only one revision allowed with --base '))
 
 # Don't check in the --continue case, in effect retaining --force across
 # --continues. That's because without --force, any revisions we decided to
@@ -2398,7 +2444,7 @@ def _dograft(ui, repo, *revs, **opts):
 # way to the graftstate. With --force, any revisions we would have 
otherwise
 # skipped would not have been filtered out, and if they hadn't been applied
 # already, they'd have been in the graftstate.
-if not (cont or opts.get('force')):
+if not (cont or opts.get('force')) and basectx is None:
 # check for ancestors of dest branch
 crev = repo['.'].rev()
 ancestors = repo.changelog.ancestors([crev], inclusive=True)
@@ -2494,8 +2540,9 @@ def _dograft(ui,

[PATCH] graft: introduce --base option for using custom base revision for merge

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1539424786 -7200
#  Sat Oct 13 11:59:46 2018 +0200
# Node ID 167e7e7bbead47fadfe4072143cc53b409e3d8b9
# Parent  38ac525b44c93fcadb3680d4ded56f1e5a0029b2
graft: introduce --base option for using custom base revision for merge

The graft command usually merges with the graft revision using its p1 as base
for the merge.

As a trivial extension of this, we introduce the --base option to allow for
using another base revision.

This can be used as a building block for grafting and collapsing multiple
changesets at once, or for grafting the resulting change from a merge as a
single simple change. (This is kind of similar to backout --parent ... only
different: it must be an ancestor, but is usually *not* a parent.)

This is probably an advanced use case, and we do thus not show it in the
non-verbose help.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -2223,6 +2223,7 @@ def forget(ui, repo, *pats, **opts):
 @command(
 'graft',
 [('r', 'rev', [], _('revisions to graft'), _('REV')),
+ ('', 'base', '', _('base revision when doing the graft merge'), _('REV')),
  ('c', 'continue', False, _('resume interrupted graft')),
  ('', 'stop', False, _('stop interrupted graft')),
  ('', 'abort', False, _('abort interrupted graft')),
@@ -2267,6 +2268,12 @@ def graft(ui, repo, *revs, **opts):
 
 .. container:: verbose
 
+  The --base option can be used to specify another merge base than the
+  first parent, thus allowing graft of the impact of a merge, or
+  collapsing of multiple changesets.
+
+.. container:: verbose
+
   Examples:
 
   - copy a single change to the stable branch and edit its description::
@@ -2305,6 +2312,9 @@ def _dograft(ui, repo, *revs, **opts):
 
 revs = list(revs)
 revs.extend(opts.get('rev'))
+basectx = None
+if opts.get('base'):
+basectx = scmutil.revsingle(repo, opts['base'], None)
 # a dict of data to be stored in state file
 statedata = {}
 # list of new nodes created by ongoing graft
@@ -2384,13 +2394,16 @@ def _dograft(ui, repo, *revs, **opts):
 revs = scmutil.revrange(repo, revs)
 
 skipped = set()
-# check for merges
-for rev in repo.revs('%ld and merge()', revs):
-ui.warn(_('skipping ungraftable merge revision %d\n') % rev)
-skipped.add(rev)
+if basectx is None:
+# check for merges
+for rev in repo.revs('%ld and merge()', revs):
+ui.warn(_('skipping ungraftable merge revision %d\n') % rev)
+skipped.add(rev)
 revs = [r for r in revs if r not in skipped]
 if not revs:
 return -1
+if basectx is not None and len(revs) != 1:
+raise error.Abort(_('only one revision allowed with --base '))
 
 # Don't check in the --continue case, in effect retaining --force across
 # --continues. That's because without --force, any revisions we decided to
@@ -2398,7 +2411,7 @@ def _dograft(ui, repo, *revs, **opts):
 # way to the graftstate. With --force, any revisions we would have 
otherwise
 # skipped would not have been filtered out, and if they hadn't been applied
 # already, they'd have been in the graftstate.
-if not (cont or opts.get('force')):
+if not (cont or opts.get('force')) and basectx is None:
 # check for ancestors of dest branch
 crev = repo['.'].rev()
 ancestors = repo.changelog.ancestors([crev], inclusive=True)
@@ -2494,8 +2507,9 @@ def _dograft(ui, repo, *revs, **opts):
 if not cont:
 # perform the graft merge with p1(rev) as 'ancestor'
 overrides = {('ui', 'forcemerge'): opts.get('tool', '')}
+base = ctx.p1() if basectx is None else basectx
 with ui.configoverride(overrides, 'graft'):
-stats = mergemod.graft(repo, ctx, ctx.p1(), ['local', 'graft'])
+stats = mergemod.graft(repo, ctx, base, ['local', 'graft'])
 # report any conflicts
 if stats.unresolvedcount > 0:
 # write out state for --continue
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -318,7 +318,7 @@ Show all commands + options
   debugwireargs: three, four, five, ssh, remotecmd, insecure
   debugwireproto: localssh, peer, noreadstderr, nologhandshake, ssh, 
remotecmd, insecure
   files: rev, print0, include, exclude, template, subrepos
-  graft: rev, continue, stop, abort, edit, log, no-commit, force, currentdate, 
currentuser, date, user, tool, dry-run
+  graft: rev, base, continue, stop, abort, edit, log, no-commit, force, 
currentdate, currentuser, date, user, tool, dry-run
   grep: print0, all, diff, text, follow, ignore-case, files-with-matches, 
line-number, rev, all-files, user, date, template, include, exclude
   heads: rev, topo, active, closed, style, templ

D785: context: also consider path conflicts when clearing unknown files

[kiilerix (Mads Kiilerich)]

kiilerix added inline comments.

INLINE COMMENTS

> test-pathconflicts-basic.t:38
>1 files updated, 0 files merged, 1 files removed, 0 files unresolved
> +  $ rm a~853701544ac3+
>  

I don't know about these `~hash` files, but including the `+` in the name 
definitely seems wrong.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D785

To: mbthomas, #hg-reviewers
Cc: kiilerix, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D778: merge: backup conflicting directories when getting files

[kiilerix (Mads Kiilerich)]

kiilerix added inline comments.

INLINE COMMENTS

> merge.py:1175
> +absf = repo.wjoin(p)
> +break
>  orig = scmutil.origpath(ui, repo, absf)

This seems quite a bit slower. But I guess it never will happen in tight loops? 
If we have to backup a lot of files, then we have lost anyway?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D778

To: mbthomas, #hg-reviewers, ryanmce
Cc: kiilerix, ryanmce, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D680: scmutil: handle conflicting files and dirs in origbackuppath

[kiilerix (Mads Kiilerich)]

kiilerix added a comment.


  I am concerned that this seems to rely even more on not having conflicts 
inside origbackuppath from files/dirs with same name but in different 
directories. If we have a backup file handling, it must be 100% reliable. That 
seems to be what this series is trying to fix - then it seems unfortunate it 
just moves the problem elsewhere.

INLINE COMMENTS

> scmutil.py:576
> +if not origbackuppath:
>  return filepath + ".orig"
>  

Now, this seems like a separate trivial change, fixing the problem that 
origbackuppath couldn't be overridden.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D680

To: mbthomas, ryanmce, #hg-reviewers, durham, yuja, ikostia
Cc: kiilerix, quark, ikostia, yuja, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 stable] mq: test coverage of how [diff] configuration influence can break mq patches

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1505083342 -7200
#  Mon Sep 11 00:42:22 2017 +0200
# Branch stable
# Node ID e05e50fbdeaf7eb52a2936e1dfe98643d68c334e
# Parent  3c3066367d72344935aabf9606a5b40e9950b5e7
mq: test coverage of how [diff] configuration influence can break mq patches

diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -208,5 +208,40 @@ git=no: regular patch after qrefresh wit
   @@ -0,0 +1,1 @@
   +a
 
+Test how [diff] configuration influence and cause invalid or lossy patches:
+
+  $ cat <> .hg/hgrc
+  > [mq]
+  > git = AUTO
+  > [diff]
+  > nobinary = True
+  > noprefix = True
+  > showfunc = True
+  > ignorews = True
+  > ignorewsamount = True
+  > ignoreblanklines = True
+  > unified = 1
+  > EOF
+
+  $ echo ' a' > a
+  $ hg qnew prepare -d '0 0'
+  $ echo '  a' > a
+  $ printf '\0' > b
+  $ echo >> c
+  $ hg qnew diff -d '0 0'
+
+  $ cat .hg/patches/prepare
+  # HG changeset patch
+  # Date 0 0
+  # Parent  cf0bfe72686a47d8d7d7b4529a3adb8b0b449a9f
+  
+  $ cat .hg/patches/diff
+  # HG changeset patch
+  # Date 0 0
+  # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
+  
+  diff --git b b
+  Binary file b has changed
+
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 stable] mq: create non-lossy patches, also with custom global diff configuration

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1505083344 -7200
#  Mon Sep 11 00:42:24 2017 +0200
# Branch stable
# Node ID 987a85c42b08ab2a82cce39b004e00b708320d0e
# Parent  e05e50fbdeaf7eb52a2936e1dfe98643d68c334e
mq: create non-lossy patches, also with custom global diff configuration

Users with custom [diff] configuration most certainly didn't intend it to make
mq lose changes. It could:

 * git is handled perfectly fine.

 * nobinary could make mq leave some files out from the patches.

 * noprefix could make mq itself (and probably also other tools) fail to apply
   patches without the usual a/b prefix.

 * ignorews, ignorewsamount, or ignoreblanklines could create patches with
   missing whitespace that could fail to apply correctly.

Thus, when refreshing patches, use patch.difffeatureopts, optionally with git
as before, but without the config options for whitespace and format changing
that most likely will cause loss or problems.

(patch.diffopts is just patch.difffeatureopts with all options enabled and can
be replaced with that.)

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -503,8 +503,11 @@ class queue(object):
 self.guardsdirty = False
 self.activeguards = None
 
-def diffopts(self, opts=None, patchfn=None):
-diffopts = patchmod.diffopts(self.ui, opts)
+def diffopts(self, opts=None, patchfn=None, plain=False):
+"""Return diff options tweaked for this mq use, possibly upgrading to
+git format, and possibly plain and without lossy options."""
+diffopts = patchmod.difffeatureopts(self.ui, opts,
+git=True, whitespace=not plain, formatchanging=not plain)
 if self.gitmode == 'auto':
 diffopts.upgrade = True
 elif self.gitmode == 'keep':
@@ -1177,7 +1180,7 @@ class queue(object):
 date = opts.get('date')
 if date:
 date = util.parsedate(date)
-diffopts = self.diffopts({'git': opts.get('git')})
+diffopts = self.diffopts({'git': opts.get('git')}, plain=True)
 if opts.get('checkname', True):
 self.checkpatchname(patchfn)
 inclsubs = checksubstate(repo)
@@ -1642,7 +1645,8 @@ class queue(object):
 substatestate = repo.dirstate['.hgsubstate']
 
 ph = patchheader(self.join(patchfn), self.plainmode)
-diffopts = self.diffopts({'git': opts.get('git')}, patchfn)
+diffopts = self.diffopts({'git': opts.get('git')}, patchfn,
+ plain=True)
 if newuser:
 ph.setuser(newuser)
 if newdate:
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -235,13 +235,35 @@ Test how [diff] configuration influence 
   # Date 0 0
   # Parent  cf0bfe72686a47d8d7d7b4529a3adb8b0b449a9f
   
+  diff -r cf0bfe72686a -r fb9c4422b0f3 a
+  --- a/a
+  +++ b/a
+  @@ -1,1 +1,1 @@
+  -a
+  + a
   $ cat .hg/patches/diff
   # HG changeset patch
   # Date 0 0
   # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
   
-  diff --git b b
-  Binary file b has changed
+  diff --git a/a b/a
+  --- a/a
+  +++ b/a
+  @@ -1,1 +1,1 @@
+  - a
+  +  a
+  diff --git a/b b/b
+  index 
78981922613b2afb6025042ff6bd878ac1994e85..f76dd238ade08917e6712764a16a22005a50573d
+  GIT binary patch
+  literal 1
+  Ic${MZ000310RR91
+  
+  diff --git a/c b/c
+  --- a/c
+  +++ b/c
+  @@ -1,1 +1,2 @@
+   a
+  +
 
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 6 stable] mq: create non-lossy patches, also with [diff] nobinary=True

[Mads Kiilerich]

On 06/18/2017 09:03 AM, Yuya Nishihara wrote:

The direction seems good, but can't we use patchmod.difffeatureopts() ?


Right, thanks. Didn't know about that one.

/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 6 stable] mq: test coverage of how [diff] configuration influence and break mq patches

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID 1968bc846c16d0cf59a573daf879bb842daaf7f1
# Parent  d3ab31bf9c0e9cecef77f18d9648716be5fcfdb8
mq: test coverage of how [diff] configuration influence and break mq patches

diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -208,5 +208,40 @@ git=no: regular patch after qrefresh wit
   @@ -0,0 +1,1 @@
   +a
 
+Test how [diff] configuration influence and cause invalid or lossy patches:
+
+  $ cat <> .hg/hgrc
+  > [mq]
+  > git = AUTO
+  > [diff]
+  > nobinary = True
+  > noprefix = True
+  > showfunc = True
+  > ignorews = True
+  > ignorewsamount = True
+  > ignoreblanklines = True
+  > unified = 1
+  > EOF
+
+  $ echo ' a' > a
+  $ hg qnew prepare -d '0 0'
+  $ echo '  a' > a
+  $ printf '\0' > b
+  $ echo >> c
+  $ hg qnew diff -d '0 0'
+
+  $ cat .hg/patches/prepare
+  # HG changeset patch
+  # Date 0 0
+  # Parent  cf0bfe72686a47d8d7d7b4529a3adb8b0b449a9f
+  
+  $ cat .hg/patches/diff
+  # HG changeset patch
+  # Date 0 0
+  # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
+  
+  diff --git b b
+  Binary file b has changed
+
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 6 stable] mq: create non-lossy patches, also with [diff] nobinary=True

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID 2dae66ed58c0e2440a344ab8de645436a4c1f1a6
# Parent  1968bc846c16d0cf59a573daf879bb842daaf7f1
mq: create non-lossy patches, also with [diff] nobinary=True

Users with custom diff configuration most certainly didn't intend it to make mq
drop changes.

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -503,7 +503,7 @@ class queue(object):
 self.guardsdirty = False
 self.activeguards = None
 
-def diffopts(self, opts=None, patchfn=None):
+def diffopts(self, opts=None, patchfn=None, stable=False):
 diffopts = patchmod.diffopts(self.ui, opts)
 if self.gitmode == 'auto':
 diffopts.upgrade = True
@@ -516,6 +516,11 @@ class queue(object):
' got %s') % self.gitmode)
 if patchfn:
 diffopts = self.patchopts(diffopts, patchfn)
+if stable:
+# disable custom diff configuration - we need a stable format
+diffopts.nobinary = False
+# note: diff options showfunc and unified might influence diffs
+# but don't do any harm
 return diffopts
 
 def patchopts(self, diffopts, *patches):
@@ -1178,7 +1183,7 @@ class queue(object):
 date = opts.get('date')
 if date:
 date = util.parsedate(date)
-diffopts = self.diffopts({'git': opts.get('git')})
+diffopts = self.diffopts({'git': opts.get('git')}, stable=True)
 if opts.get('checkname', True):
 self.checkpatchname(patchfn)
 inclsubs = checksubstate(repo)
@@ -1644,7 +1649,8 @@ class queue(object):
 substatestate = repo.dirstate['.hgsubstate']
 
 ph = patchheader(self.join(patchfn), self.plainmode)
-diffopts = self.diffopts({'git': opts.get('git')}, patchfn)
+diffopts = self.diffopts({'git': opts.get('git')}, patchfn,
+ stable=True)
 if newuser:
 ph.setuser(newuser)
 if newdate:
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -241,7 +241,11 @@ Test how [diff] configuration influence 
   # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
   
   diff --git b b
-  Binary file b has changed
+  index 
78981922613b2afb6025042ff6bd878ac1994e85..f76dd238ade08917e6712764a16a22005a50573d
+  GIT binary patch
+  literal 1
+  Ic${MZ000310RR91
+  
 
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 6 stable] mq: don't lose whitespace changes with [diff] ignorewsamount=True

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID feb176a4bd5186488abb27ed9ed4d5fcd9df6f12
# Parent  5dfc12c8d1fa53baf7b9d0099bbac1e05b90f75f
mq: don't lose whitespace changes with [diff] ignorewsamount=True

Patches are fragile - missing whitespace can make them fail to apply correctly.

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -521,6 +521,7 @@ class queue(object):
 diffopts.nobinary = False
 diffopts.noprefix = False
 diffopts.ignorews = False
+diffopts.ignorewsamount = False
 # note: diff options showfunc and unified might influence diffs
 # but don't do any harm
 return diffopts
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -246,6 +246,12 @@ Test how [diff] configuration influence 
   # Date 0 0
   # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
   
+  diff --git a/a b/a
+  --- a/a
+  +++ b/a
+  @@ -1,1 +1,1 @@
+  - a
+  +  a
   diff --git a/b b/b
   index 
78981922613b2afb6025042ff6bd878ac1994e85..f76dd238ade08917e6712764a16a22005a50573d
   GIT binary patch
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 6 stable] mq: don't lose whitespace changes with [diff] ignoreblanklines=True

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID 863bb364f8201dda46c6a3d51fb57bbbdc7cb69b
# Parent  feb176a4bd5186488abb27ed9ed4d5fcd9df6f12
mq: don't lose whitespace changes with [diff] ignoreblanklines=True

Patches are fragile - missing whitespace can make them fail to apply correctly.

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -522,6 +522,7 @@ class queue(object):
 diffopts.noprefix = False
 diffopts.ignorews = False
 diffopts.ignorewsamount = False
+diffopts.ignoreblanklines = False
 # note: diff options showfunc and unified might influence diffs
 # but don't do any harm
 return diffopts
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -258,6 +258,12 @@ Test how [diff] configuration influence 
   literal 1
   Ic${MZ000310RR91
   
+  diff --git a/c b/c
+  --- a/c
+  +++ b/c
+  @@ -1,1 +1,2 @@
+   a
+  +
 
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 6 stable] mq: create patches with a/b prefix, also with [diff] noprefix=True

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID e086df12576d6829b61d760636af2a189c385136
# Parent  2dae66ed58c0e2440a344ab8de645436a4c1f1a6
mq: create patches with a/b prefix, also with [diff] noprefix=True

mq itself (and probably also other tools) fail to apply patches without the a/b
prefix.

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -519,6 +519,7 @@ class queue(object):
 if stable:
 # disable custom diff configuration - we need a stable format
 diffopts.nobinary = False
+diffopts.noprefix = False
 # note: diff options showfunc and unified might influence diffs
 # but don't do any harm
 return diffopts
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -240,7 +240,7 @@ Test how [diff] configuration influence 
   # Date 0 0
   # Parent  fb9c4422b0f37dd576522dd9a3f99b825c177efe
   
-  diff --git b b
+  diff --git a/b b/b
   index 
78981922613b2afb6025042ff6bd878ac1994e85..f76dd238ade08917e6712764a16a22005a50573d
   GIT binary patch
   literal 1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 6 stable] mq: don't lose whitespace changes with [diff] ignorews=True

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1497727704 -7200
#  Sat Jun 17 21:28:24 2017 +0200
# Branch stable
# Node ID 5dfc12c8d1fa53baf7b9d0099bbac1e05b90f75f
# Parent  e086df12576d6829b61d760636af2a189c385136
mq: don't lose whitespace changes with [diff] ignorews=True

Patches are fragile - missing whitespace can make them fail to apply correctly.

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -520,6 +520,7 @@ class queue(object):
 # disable custom diff configuration - we need a stable format
 diffopts.nobinary = False
 diffopts.noprefix = False
+diffopts.ignorews = False
 # note: diff options showfunc and unified might influence diffs
 # but don't do any harm
 return diffopts
diff --git a/tests/test-mq-git.t b/tests/test-mq-git.t
--- a/tests/test-mq-git.t
+++ b/tests/test-mq-git.t
@@ -235,6 +235,12 @@ Test how [diff] configuration influence 
   # Date 0 0
   # Parent  cf0bfe72686a47d8d7d7b4529a3adb8b0b449a9f
   
+  diff -r cf0bfe72686a -r fb9c4422b0f3 a
+  --- a/a
+  +++ b/a
+  @@ -1,1 +1,1 @@
+  -a
+  + a
   $ cat .hg/patches/diff
   # HG changeset patch
   # Date 0 0
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 stable v2] graft: test coverage of grafts and how merges can break duplicate detection

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1494281490 -7200
#  Tue May 09 00:11:30 2017 +0200
# Branch stable
# Node ID 6710017995b4e8b361d6ad5b897ff7d0cc658285
# Parent  247bb7a2c492d8a946cc7ed61a627d53566b7c16
graft: test coverage of grafts and how merges can break duplicate detection

This demonstrates unfortunate behaviour: extending the graft range cause the
graft to behave differently. When the graft range includes a merge, we fail to
detect duplicates that are ancestors of the merge.

diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -1309,4 +1309,49 @@ Graft a change into a new file previousl
   $ hg status --change .
   M b/x
 
+Prepare for test of skipped changesets and how merges can influence it:
+
+  $ hg merge -q -r 1 --tool :local
+  $ hg ci -m m
+  $ echo xx >> b/x
+  $ hg ci -m xx
+
+  $ hg log -G -T '{rev} {desc|firstline}'
+  @  7 xx
+  |
+  o6 m
+  |\
+  | o  5 y
+  | |
+  +---o  4 y
+  | |
+  | o  3 x
+  | |
+  | o  2 b
+  | |
+  o |  1 x
+  |/
+  o  0 a
+  
+Grafting of plain changes correctly detects that 3 and 5 should be skipped:
+
+  $ hg up -qCr 4
+  $ hg graft --tool :local -r 2::5
+  skipping already grafted revision 3:ca093ca2f1d9 (was grafted from 
1:13ec5badbf2a)
+  skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
+  grafting 2:42127f193bcd "b"
+
+Extending the graft range to include a merge will unfortunately make us miss
+that 3 and 5 should be skipped:
+
+  $ hg up -qCr 4
+  $ hg graft --tool :local -r 2::7
+  skipping ungraftable merge revision 6
+  skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
+  grafting 2:42127f193bcd "b"
+  grafting 3:ca093ca2f1d9 "x"
+  note: graft of 3:ca093ca2f1d9 created no changes to commit
+  grafting 7:d3c3f2b38ecc "xx"
+  note: graft of 7:d3c3f2b38ecc created no changes to commit
+
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 stable v2] graft: fix graft across merges of duplicates of grafted changes

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1494515920 -7200
#  Thu May 11 17:18:40 2017 +0200
# Branch stable
# Node ID cf6482170698ea9cd590f6e72745348bef5703b4
# Parent  6710017995b4e8b361d6ad5b897ff7d0cc658285
graft: fix graft across merges of duplicates of grafted changes

Graft used findmissingrevs to find the candidates for graft duplicates in the
destination. That function operates with the constraint:

  1. N is an ancestor of some node in 'heads'
  2. N is not an ancestor of any node in 'common'

For our purpose, we do however have to work correctly in cases where the graft
set has multiple roots or where merges between graft ranges are skipped. The
only changesets we can be sure doesn't have ancestors that are grafts of any
changeset in the graftset, are the ones that are common ancestors of *all*
changesets in the graftset. We thus need:

  2. N is not an ancestor of all nodes in 'common'

This change will graft more correctly, but it will also in some cases make
graft slower by making it search through a bigger and unnecessary large sets of
changes to find duplicates. In the general case of grafting individual or
linear sets, we do the same amount of work as before.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -2295,7 +2295,9 @@ def _dograft(ui, repo, *revs, **opts):
 # check ancestors for earlier grafts
 ui.debug('scanning for duplicate grafts\n')
 
-for rev in repo.changelog.findmissingrevs(revs, [crev]):
+# The only changesets we can be sure doesn't contain grafts of any
+# revs, are the ones that are common ancestors of *all* revs:
+for rev in repo.revs('only(%d,ancestor(%ld))', crev, revs):
 ctx = repo[rev]
 n = ctx.extra().get('source')
 if n in ids:
diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -1341,16 +1341,15 @@ Grafting of plain changes correctly dete
   skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
   grafting 2:42127f193bcd "b"
 
-Extending the graft range to include a merge will unfortunately make us miss
-that 3 and 5 should be skipped:
+Extending the graft range to include a (skipped) merge of 3 will not prevent 
us from
+also detecting that both 3 and 5 should be skipped:
 
   $ hg up -qCr 4
   $ hg graft --tool :local -r 2::7
   skipping ungraftable merge revision 6
+  skipping already grafted revision 3:ca093ca2f1d9 (was grafted from 
1:13ec5badbf2a)
   skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
   grafting 2:42127f193bcd "b"
-  grafting 3:ca093ca2f1d9 "x"
-  note: graft of 3:ca093ca2f1d9 created no changes to commit
   grafting 7:d3c3f2b38ecc "xx"
   note: graft of 7:d3c3f2b38ecc created no changes to commit
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2 stable] graft: fix graft across merges of duplicates of grafted changes

[Mads Kiilerich]

On 05/10/2017 03:42 PM, Yuya Nishihara wrote:

On Tue, 09 May 2017 00:45:02 +0200, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1494283376 -7200
#  Tue May 09 00:42:56 2017 +0200
# Branch stable
# Node ID dd256de590dfd363fa5d497d566d456f471b8d52
# Parent  6710017995b4e8b361d6ad5b897ff7d0cc658285
graft: fix graft across merges of duplicates of grafted changes

Looks good to me. A couple of nits inline.


Graft used findmissingrevs to find the candidates for graft duplicates in the
destination. That function operates with the constraint:

   2. N is not an ancestor of any node in 'common'

For our purpose, we do however need:

   2. There are nodes in 'common' that doesn't have N as ancestor

The right candiates for graft duplicates could be computed with a revset:

   only(destination,transitiveroots(graftset))


I guess it actually can be computed as

only(destination,roots(graftset+roots(graftset)::heads(graftset)))

BUT I realize it also is wrong. There could be criss-cross-ish cases 
where multiple graftset roots have been merged to different branches 
that have grafts of other roots as ancestor. My proposed patch using 
min(graftset) would also fail that.


Instead, the only changesets we can be sure doesn't contain grafts of 
any changeset in the graftset, are the ones that are common ancestors of 
*all* changesets in the graftset:


only(destination,ancestor(graftset))

It will exclude one ancestor. In criss-cross cases where there will be 
more ancestors, it might be inefficient but still correct.


Resending ...

/Mads




where transitiveroots would be a revset function for 'transitive root' and
return changesets in set with no ancestor changeset in set. There doesn't seem
to be any such function readily available, and we thus use the approximation of
just using the smallest revision in the graft set.

Can you copy this message as a code comment? It will help future readers.


This change will graft more correctly, but it will also in some cases make
graft slower by making it search through a bigger and unnecessary large sets of
changes to find duplicates.

Suppose revisions to be grafted are linear in general, I think this is
acceptable.


@@ -2295,7 +2295,8 @@ def _dograft(ui, repo, *revs, **opts):
  # check ancestors for earlier grafts
  ui.debug('scanning for duplicate grafts\n')
  
-for rev in repo.changelog.findmissingrevs(revs, [crev]):

+expr = revsetlang.formatspec('only(%d,min(%ld))', crev, revs)
+for rev in scmutil.revrange(repo, [expr]):

scmutil.revrange() may expand user aliases. Please use repo.revs() instead.
Alternatively, maybe we could use findmissingrevs(min(revs), ...) to minimize
the change?



___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 stable] graft: fix graft across merges of duplicates of grafted changes

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1494283376 -7200
#  Tue May 09 00:42:56 2017 +0200
# Branch stable
# Node ID dd256de590dfd363fa5d497d566d456f471b8d52
# Parent  6710017995b4e8b361d6ad5b897ff7d0cc658285
graft: fix graft across merges of duplicates of grafted changes

Graft used findmissingrevs to find the candidates for graft duplicates in the
destination. That function operates with the constraint:

  2. N is not an ancestor of any node in 'common'

For our purpose, we do however need:

  2. There are nodes in 'common' that doesn't have N as ancestor

The right candiates for graft duplicates could be computed with a revset:

  only(destination,transitiveroots(graftset))

where transitiveroots would be a revset function for 'transitive root' and
return changesets in set with no ancestor changeset in set. There doesn't seem
to be any such function readily available, and we thus use the approximation of
just using the smallest revision in the graft set.

This change will graft more correctly, but it will also in some cases make
graft slower by making it search through a bigger and unnecessary large sets of
changes to find duplicates.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -2295,7 +2295,8 @@ def _dograft(ui, repo, *revs, **opts):
 # check ancestors for earlier grafts
 ui.debug('scanning for duplicate grafts\n')
 
-for rev in repo.changelog.findmissingrevs(revs, [crev]):
+expr = revsetlang.formatspec('only(%d,min(%ld))', crev, revs)
+for rev in scmutil.revrange(repo, [expr]):
 ctx = repo[rev]
 n = ctx.extra().get('source')
 if n in ids:
diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -1341,16 +1341,15 @@ Grafting of plain changes correctly dete
   skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
   grafting 2:42127f193bcd "b"
 
-Extending the graft range to include a merge will unfortunately make us miss
-that 3 and 5 should be skipped:
+Extending the graft range to include a (skipped) merge of 3 will not prevent 
us from
+also detecting that both 3 and 5 should be skipped:
 
   $ hg up -qCr 4
   $ hg graft --tool :local -r 2::7
   skipping ungraftable merge revision 6
+  skipping already grafted revision 3:ca093ca2f1d9 (was grafted from 
1:13ec5badbf2a)
   skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
   grafting 2:42127f193bcd "b"
-  grafting 3:ca093ca2f1d9 "x"
-  note: graft of 3:ca093ca2f1d9 created no changes to commit
   grafting 7:d3c3f2b38ecc "xx"
   note: graft of 7:d3c3f2b38ecc created no changes to commit
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 stable] graft: test coverage of grafts and how merges can break duplicate detection

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1494281490 -7200
#  Tue May 09 00:11:30 2017 +0200
# Branch stable
# Node ID 6710017995b4e8b361d6ad5b897ff7d0cc658285
# Parent  247bb7a2c492d8a946cc7ed61a627d53566b7c16
graft: test coverage of grafts and how merges can break duplicate detection

This demonstrates unfortunate behaviour: extending the graft range cause the
graft to behave differently. When the graft range includes a merge, we fail to
detect duplicates that are ancestors of the merge.

diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -1309,4 +1309,49 @@ Graft a change into a new file previousl
   $ hg status --change .
   M b/x
 
+Prepare for test of skipped changesets and how merges can influence it:
+
+  $ hg merge -q -r 1 --tool :local
+  $ hg ci -m m
+  $ echo xx >> b/x
+  $ hg ci -m xx
+
+  $ hg log -G -T '{rev} {desc|firstline}'
+  @  7 xx
+  |
+  o6 m
+  |\
+  | o  5 y
+  | |
+  +---o  4 y
+  | |
+  | o  3 x
+  | |
+  | o  2 b
+  | |
+  o |  1 x
+  |/
+  o  0 a
+  
+Grafting of plain changes correctly detects that 3 and 5 should be skipped:
+
+  $ hg up -qCr 4
+  $ hg graft --tool :local -r 2::5
+  skipping already grafted revision 3:ca093ca2f1d9 (was grafted from 
1:13ec5badbf2a)
+  skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
+  grafting 2:42127f193bcd "b"
+
+Extending the graft range to include a merge will unfortunately make us miss
+that 3 and 5 should be skipped:
+
+  $ hg up -qCr 4
+  $ hg graft --tool :local -r 2::7
+  skipping ungraftable merge revision 6
+  skipping already grafted revision 5:43e9eb70dab0 (was grafted from 
4:6c9a1289e5f1)
+  grafting 2:42127f193bcd "b"
+  grafting 3:ca093ca2f1d9 "x"
+  note: graft of 3:ca093ca2f1d9 created no changes to commit
+  grafting 7:d3c3f2b38ecc "xx"
+  note: graft of 7:d3c3f2b38ecc created no changes to commit
+
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] exewrapper: add an ability to always load python from hg-python subdir

[Mads Kiilerich]

On 03/12/2017 04:02 PM, Kostia Balytskyi wrote:

# HG changeset patch
# User Kostia Balytskyi 
# Date 1489359573 25200
#  Sun Mar 12 15:59:33 2017 -0700
# Node ID bdd61a3470df08b09bd18bbb40e33a8f7cabe188
# Parent  7548522742b5f4f9f5c0881ae4a2783ecda2f969
exewrapper: add an ability to always load python from hg-python subdir

Currently hg.exe will only try to load python27.dll from hg-python
subdir if PYTHONHOME environment variable is not set and if this
approach fails, proceed to load it from whereever possible. I want
to be able to compile a version of hg.exe which will only use
hg-python and not other options, regardless of its environment. This
patch makes it so running 'python setup.py build_hgexe --usehgpython'
builds such version.


It would be nice if we didn't need this compile flag.

I guess the opposite search order would be perfectly fine: *if* there is 
a hg-python sub folder, then use it. If not, use PYTHONHOME ... or 
search in $PATH.


Do you think that would work?


It also breaks test-check-commit.t becuase it introduces a function
named 'initialize_options', but that is a function from parent class,
so there's not much we can do about it.


Breaks ... how?

And if it breaks it, then it must be fixed or "fixed". That doesn't seem 
to be a part of this.



diff --git a/mercurial/exewrapper.c b/mercurial/exewrapper.c
--- a/mercurial/exewrapper.c
+++ b/mercurial/exewrapper.c
@@ -29,6 +29,14 @@ static char pyhome[MAX_PATH + 10];
  static char envpyhome[MAX_PATH + 10];
  static char pydllfile[MAX_PATH + 10];
  
+/* Compiling with /DUSEHGPYTHON makes Mercurial load Python from hg-python

+subdir regardless of environment in which hg.exe is ran. */
+#ifdef USEHGPYTHON
+static int usehgpython = 1;
+#else
+static int usehgpython = 0;
+#endif
+
  int main(int argc, char *argv[])
  {
char *p;
@@ -71,15 +79,13 @@ int main(int argc, char *argv[])
We first check, that environment variable PYTHONHOME is *not* set.
This just mimicks the behavior of the regular python.exe, which uses
PYTHONHOME to find its installation directory (if it has been set).
-   Note: Users of HackableMercurial are expected to *not* set PYTHONHOME!
+   Note: Users of HackableMercurial are expected to *not* set PYTHONHOME
+   or compile exewrapper.c with /DUSEHGPYTHON.
*/
-   if (GetEnvironmentVariable("PYTHONHOME", envpyhome,
-  sizeof(envpyhome)) == 0)
+   if (usehgpython || (GetEnvironmentVariable("PYTHONHOME", envpyhome,
+  sizeof(envpyhome)) == 0))
{
-   /*
-   Environment var PYTHONHOME is *not* set. Let's see if we are
-   running inside a HackableMercurial.
-   */
+   /* We should try to load Python from hg-python subdir */
  
  		p = strrchr(pyhome, '\\');

if (p == NULL) {
@@ -112,6 +118,14 @@ int main(int argc, char *argv[])
}
Py_SetPythonHome(pyhome);
}
+   else
+   {
+   if (pydll == NULL && usehgpython)
+   {
+   err = "can't find hg-python subdir in Mercurial 
dir";
+   goto bail;
+   }
+   }
}
  
  	if (pydll == NULL) {

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -405,6 +405,17 @@ class buildhgextindex(Command):
  class buildhgexe(build_ext):
  description = 'compile hg.exe from mercurial/exewrapper.c'
  
+user_options = build_ext.user_options + [

+('usehgpython', None, 'always load python dll from hg-python subdir'),
+]
+
+boolean_options = build_ext.boolean_options + ['usehgpython']
+
+def initialize_options(self):
+self.usehgpython = False
+return build_ext.initialize_options(self)
+
+
  def build_extensions(self):
  if os.name != 'nt':
  return
@@ -442,8 +453,10 @@ class buildhgexe(build_ext):
  with open('mercurial/hgpythonlib.h', 'wb') as f:
  f.write('/* this file is autogenerated by setup.py */\n')
  f.write('#define HGPYTHONLIB "%s"\n' % pythonlib)
+extra_preargs = [] if not self.usehgpython else ["/DUSEHGPYTHON"]
  objects = self.compiler.compile(['mercurial/exewrapper.c'],
- output_dir=self.build_temp)
+ output_dir=self.build_temp,
+ extra_preargs=extra_preargs)
  dir = os.path.dirname(self.get_ext_fullpath('dummy'))
  target = os.path.join(dir, 'hg')
  self.compiler.link_executable(objects, target,
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

[PATCH 2 of 2] rebase: allow rebasing children of wd to wd if a new branch has been set

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1489362241 25200
#  Sun Mar 12 16:44:01 2017 -0700
# Node ID e3ac848788e45a0530abfa8705740824db6f3444
# Parent  42083ac9c3ed1b3be205e0f3a621787071587ae5
rebase: allow rebasing children of wd to wd if a new branch has been set

The named branch of the leaf changeset can be changed by updating to it,
setting the branch, and amending.

But previously, there was no good way to *just* change the branch of several
linear changes. If rebasing changes with another parent to '.', it would pick
up a pending branch change up. But when rebasing changes that have the same
parent, it would fail with 'nothing to rebase', even when the branch name was
set differently.

To fix this, allow rebasing to same parent when a branch has been set.

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -1224,7 +1224,12 @@ def buildstate(repo, dest, rebaseset, co
 if commonbase == root:
 raise error.Abort(_('source is ancestor of destination'))
 if commonbase == dest:
-samebranch = root.branch() == dest.branch()
+wctx = repo[None]
+if dest == wctx.p1():
+# when rebasing to '.', it will use the current wd branch name
+samebranch = root.branch() == wctx.branch()
+else:
+samebranch = root.branch() == dest.branch()
 if not collapse and samebranch and root in dest.children():
 repo.ui.debug('source is a child of destination\n')
 return None
diff --git a/tests/test-rebase-named-branches.t 
b/tests/test-rebase-named-branches.t
--- a/tests/test-rebase-named-branches.t
+++ b/tests/test-rebase-named-branches.t
@@ -387,4 +387,23 @@ rebase 'c1' to the branch head 'c2' that
   o  0: '0'
   
 
+  $ hg up -cr 1
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg branch x
+  marked working directory as branch x
+  $ hg rebase -r 3:: -d .
+  rebasing 3:76abc1c6f8c7 "b1"
+  rebasing 4:8427af5d86f2 "c2 closed" (tip)
+  note: rebase of 4:8427af5d86f2 created no changes to commit
+  saved backup bundle to 
$TESTTMP/case2/.hg/strip-backup/76abc1c6f8c7-cd698d13-backup.hg (glob)
+  $ hg tglog
+  o  3: 'b1' x
+  |
+  | o  2: 'c1' c
+  | |
+  @ |  1: 'b2' b
+  |/
+  o  0: '0'
+  
+
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] rbc: empty (and invalid) rbc-names file should give an empty name list

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <m...@kiilerich.com>
# Date 1489346250 25200
#  Sun Mar 12 12:17:30 2017 -0700
# Node ID 1b5144a87e936fc8071463956816a89041478126
# Parent  023b6f7456b3622cc81332ae9d6e30b7ecc37415
rbc: empty (and invalid) rbc-names file should give an empty name list

An empty file (if it somehow should exist) used to give a list with an empty
name. That didn't do any harm, but it was "wrong". Fix that.

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -362,7 +362,9 @@ class revbranchcache(object):
 try:
 bndata = repo.vfs.read(_rbcnames)
 self._rbcsnameslen = len(bndata) # for verification before writing
-self._names = [encoding.tolocal(bn) for bn in bndata.split('\0')]
+if bndata:
+self._names = [encoding.tolocal(bn)
+   for bn in bndata.split('\0')]
 except (IOError, OSError):
 if readonly:
 # don't try to use cache - fall back to the slow path
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] rbc: use struct unpack_from and pack_into instead of unpack and pack

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich <mad...@unity3d.com>
# Date 1476837995 -7200
#  Wed Oct 19 02:46:35 2016 +0200
# Node ID 023b6f7456b3622cc81332ae9d6e30b7ecc37415
# Parent  1c3352d7eaf24533ad52d4b8a024211e9189fb0b
rbc: use struct unpack_from and pack_into instead of unpack and pack

These functions were introduced in Python 2.5 and are faster and simpler than
the old ones ...  mainly because we can avoid intermediate buffers:

  $ python -m timeit -s "_rbcrecfmt='>4sI'" -s 's = "x"*1' -s 'from struct 
import unpack' 'unpack(_rbcrecfmt, buffer(s, 16, 8))'
  100 loops, best of 3: 0.543 usec per loop
  $ python -m timeit -s "_rbcrecfmt='>4sI'" -s 's = "x"*1' -s 'from struct 
import unpack_from' 'unpack_from(_rbcrecfmt, s, 16)'
  100 loops, best of 3: 0.323 usec per loop

  $ python -m timeit -s "from array import array" -s "_rbcrecfmt='>4sI'" -s "s 
= array('c')" -s 's.fromstring("x"*1)' -s 'from struct import pack' -s "rec 
= array('c')" 'rec.fromstring(pack(_rbcrecfmt, "asdf", 7))'
  100 loops, best of 3: 0.364 usec per loop
  $ python -m timeit -s "from array import array" -s "_rbcrecfmt='>4sI'" -s "s 
= array('c')" -s 's.fromstring("x"*1)' -s 'from struct import pack_into' -s 
"rec = array('c')" -s 'rec.fromstring("x"*100)' 'pack_into(_rbcrecfmt, rec, 0, 
"asdf", 7)'
  100 loops, best of 3: 0.229 usec per loop

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -25,8 +25,8 @@ from . import (
 
 array = array.array
 calcsize = struct.calcsize
-pack = struct.pack
-unpack = struct.unpack
+pack_into = struct.pack_into
+unpack_from = struct.unpack_from
 
 def _filename(repo):
 """name of a branchcache file for a given repo or repoview"""
@@ -409,8 +409,7 @@ class revbranchcache(object):
 
 # fast path: extract data from cache, use it if node is matching
 reponode = changelog.node(rev)[:_rbcnodelen]
-cachenode, branchidx = unpack(
-_rbcrecfmt, buffer(self._rbcrevs, rbcrevidx, _rbcrecsize))
+cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, 
rbcrevidx)
 close = bool(branchidx & _rbccloseflag)
 if close:
 branchidx &= _rbcbranchidxmask
@@ -454,13 +453,11 @@ class revbranchcache(object):
 def _setcachedata(self, rev, node, branchidx):
 """Writes the node's branch data to the in-memory cache data."""
 rbcrevidx = rev * _rbcrecsize
-rec = array('c')
-rec.fromstring(pack(_rbcrecfmt, node, branchidx))
 if len(self._rbcrevs) < rbcrevidx + _rbcrecsize:
 self._rbcrevs.extend('\0' *
  (len(self._repo.changelog) * _rbcrecsize -
   len(self._rbcrevs)))
-self._rbcrevs[rbcrevidx:rbcrevidx + _rbcrecsize] = rec
+pack_into(_rbcrecfmt, self._rbcrevs, rbcrevidx, node, branchidx)
 self._rbcrevslen = min(self._rbcrevslen, rev)
 
 tr = self._repo.currenttransaction()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 7 of 7 V3] util: enable hardlink for copyfile

[Mads Kiilerich]

On 03/12/2017 11:03 AM, Augie Fackler wrote:

On Sun, Mar 12, 2017 at 11:00:05AM -0700, Mads Kiilerich wrote:

On 03/12/2017 01:23 AM, Jun Wu wrote:

# HG changeset patch
# User Jun Wu <qu...@fb.com>
# Date 1489309403 28800
#  Sun Mar 12 01:03:23 2017 -0800
# Node ID 9da40a9e54c419490a2ff23b9dda7acc109f81cd
# Parent  de28b62236a7d47a896bc4aba2bd95dcd8defc87
# Available At https://bitbucket.org/quark-zju/hg-draft
#  hg pull https://bitbucket.org/quark-zju/hg-draft -r 9da40a9e54c4
util: enable hardlink for copyfile

Because why not, if we are sure that the filesystem has good hardlink
support (and we are - see "posix: implement a method to get filesystem type
for Linux").

This patch removes the global variable "allowhardlinks" added by "util: add
allowhardlinks module variable". Third party extensions wanting to enable
hardlink support unconditionally can replace "_hardlinkfswhitelist".

How about using the approximation that all case sensitive filesystems are
safe to try to use hardlinks? Or to be kind to macOS: All filesystems that
support symlinks?

e5ce49a30146 was a last minute change and *very* safe, disabling it on all
platforms.

It seems like the problem only is seen when running Windows on the client
side - perhaps only disable these hardlinks on Windows? It seems like there
really is no need for detecting any file systems on Linux?

It's actually believed the problem is in the Windows CIFS *server*, if
you read the bug attached to the previous change.


I only see mentioning of problems with Windows on the client side. 
Matt's theory of the source of the cache coherency issue suggested that 
it was interaction between client and server side caches. Non-windows 
client side implementations may or may not have the same problem, but I 
see nothing suggesting they have.


That might of course be because most users of repos on CIFS are Windows 
users. The problem is serious when it happens, but considering the 
non-Windows uncertainty, the small amount of non-Windows users using 
CIFS repos, and the negative impact on all non-Windows users, it might 
be justified to be less conservative for non-Windows.


/Mads



I'm hesitant to turn on hardlinks at all because of the way problems
manifest (unrecoverable data corruption for users, at some random
point in the future), so if we're going to do it we need to be very
conservative to avoid the bug again.


The infrastructure for detecting file system seems cool, but also quite low
level and high-maintenance for each platform. If we need to detect CIFS,
could we perhaps just detect CIFS through generic file system operations
instead of whitelisting everything that isn't CIFS?

/Mads


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 7 of 7 V3] util: enable hardlink for copyfile

[Mads Kiilerich]

On 03/12/2017 01:23 AM, Jun Wu wrote:

# HG changeset patch
# User Jun Wu 
# Date 1489309403 28800
#  Sun Mar 12 01:03:23 2017 -0800
# Node ID 9da40a9e54c419490a2ff23b9dda7acc109f81cd
# Parent  de28b62236a7d47a896bc4aba2bd95dcd8defc87
# Available At https://bitbucket.org/quark-zju/hg-draft
#  hg pull https://bitbucket.org/quark-zju/hg-draft -r 9da40a9e54c4
util: enable hardlink for copyfile

Because why not, if we are sure that the filesystem has good hardlink
support (and we are - see "posix: implement a method to get filesystem type
for Linux").

This patch removes the global variable "allowhardlinks" added by "util: add
allowhardlinks module variable". Third party extensions wanting to enable
hardlink support unconditionally can replace "_hardlinkfswhitelist".


How about using the approximation that all case sensitive filesystems 
are safe to try to use hardlinks? Or to be kind to macOS: All 
filesystems that support symlinks?


e5ce49a30146 was a last minute change and *very* safe, disabling it on 
all platforms.
It seems like the problem only is seen when running Windows on the 
client side - perhaps only disable these hardlinks on Windows? It seems 
like there really is no need for detecting any file systems on Linux?


The infrastructure for detecting file system seems cool, but also quite 
low level and high-maintenance for each platform. If we need to detect 
CIFS, could we perhaps just detect CIFS through generic file system 
operations instead of whitelisting everything that isn't CIFS?


/Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel