mercurial@40940: 4 new changesets (4 on stable)
4 new changesets (4 on stable) in mercurial: https://www.mercurial-scm.org/repo/hg/rev/e11e03f72baf changeset: 40937:e11e03f72baf branch: stable parent: 40926:21f5810df848 user:Matt Harbison date:Sat Dec 15 01:26:18 2018 -0500 summary: py3: ensure the proxied Windows fd doesn't escape by entering context manager https://www.mercurial-scm.org/repo/hg/rev/9ae4aed27930 changeset: 40938:9ae4aed27930 branch: stable user:Matt Harbison date:Sat Dec 15 13:41:34 2018 -0500 summary: windows: ensure mixedfilemodewrapper fd doesn't escape by entering context mgr https://www.mercurial-scm.org/repo/hg/rev/8d9f366b7f19 changeset: 40939:8d9f366b7f19 branch: stable user:Matt Harbison date:Sat Dec 15 13:54:37 2018 -0500 summary: vfs: ensure closewrapbase fh doesn't escape by entering context manager https://www.mercurial-scm.org/repo/hg/rev/120ecb17242b changeset: 40940:120ecb17242b branch: stable tag: tip user:Matt Harbison date:Sat Dec 15 14:55:06 2018 -0500 summary: windows: ensure pure posixfile fd doesn't escape by entering context manager -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 5] py3: stop subscripting socket.error
On Sun, 16 Dec 2018 01:19:47 -0500, Matt Harbison wrote: > On Sun, 16 Dec 2018 00:55:20 -0500, Yuya Nishihara wrote: > > > On Sun, 16 Dec 2018 00:36:45 -0500, Matt Harbison wrote: > >> > I'm not sure what to do with this info yet, but I just noticed that > >> > pager is also messed up on py3- the debug message about spawning the > >> > pager prints, but no output for the diff. Use --pager=no, and it > >> shows > >> > up. So maybe there's a general problem with stdio of spawned > >> children. > >> > >> To be specific, this: > >> > >> diff --git a/mercurial/ui.py b/mercurial/ui.py > >> --- a/mercurial/ui.py > >> +++ b/mercurial/ui.py > >> @@ -1206,6 +1207,14 @@ class ui(object): > >> pager.stdin.close() > >> pager.wait() > >> > >> +try: > >> + self.write('test output\n') > >> + self.flush() > >> +except Exception: > >> +killpager() > >> +self.traceback(force=True) > >> +raise > >> + > >> return True > >> > >> @property > >> > >> > >> Results in this: > >> > >> $ py -3 ../hg diff -r .^^ --debug > >> *** failed to import extension evolve: No module named 'evolve' > >> obsolete feature not enabled but 177518 markers found! > >> starting pager for command 'diff' > >> > >> Traceback (most recent call last): > >>File "c:\Users\Matt\hg\mercurial\ui.py", line 1024, in _writenobuf > >> color.win32print(self, dest.write, msg, **opts) > >>File "c:\Users\Matt\hg\mercurial\color.py", line 528, in win32print > >> writefunc(m.group(2)) > >>File "c:\Users\Matt\hg\mercurial\windows.py", line 202, in write > >> self.fp.write(s[start:end]) > >> OSError: [WinError 1] Incorrect function > > > > That might be because the stdio files are backed by the Windows-ish type. > > Can you try PYTHONLEGACYWINDOWSSTDIO=1? > > > > https://www.python.org/dev/peps/pep-0528/ > > > > Python 3 shifted away from being Unix scripting language in many ways. > > Yep, that fixed both pager and the strange server problems, thanks. I > need more sleep before digesting that PEP, but skimming it seems to imply > that we need to explicitly encode to stdio on Windows? How will that > interact with the existing encoding functions? I don't think we'll need the new Windows wrapper to get the Py2-level Windows support. Instead, we'll have to make our exewrapper enable all the "Legacy" options. https://docs.python.org/3/c-api/init.html#global-configuration-variables ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 5] py3: stop subscripting socket.error
On Sun, 16 Dec 2018 00:55:20 -0500, Yuya Nishihara wrote: On Sun, 16 Dec 2018 00:36:45 -0500, Matt Harbison wrote: > I'm not sure what to do with this info yet, but I just noticed that > pager is also messed up on py3- the debug message about spawning the > pager prints, but no output for the diff. Use --pager=no, and it shows > up. So maybe there's a general problem with stdio of spawned children. To be specific, this: diff --git a/mercurial/ui.py b/mercurial/ui.py --- a/mercurial/ui.py +++ b/mercurial/ui.py @@ -1206,6 +1207,14 @@ class ui(object): pager.stdin.close() pager.wait() +try: + self.write('test output\n') + self.flush() +except Exception: +killpager() +self.traceback(force=True) +raise + return True @property Results in this: $ py -3 ../hg diff -r .^^ --debug *** failed to import extension evolve: No module named 'evolve' obsolete feature not enabled but 177518 markers found! starting pager for command 'diff' Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1024, in _writenobuf color.win32print(self, dest.write, msg, **opts) File "c:\Users\Matt\hg\mercurial\color.py", line 528, in win32print writefunc(m.group(2)) File "c:\Users\Matt\hg\mercurial\windows.py", line 202, in write self.fp.write(s[start:end]) OSError: [WinError 1] Incorrect function That might be because the stdio files are backed by the Windows-ish type. Can you try PYTHONLEGACYWINDOWSSTDIO=1? https://www.python.org/dev/peps/pep-0528/ Python 3 shifted away from being Unix scripting language in many ways. Yep, that fixed both pager and the strange server problems, thanks. I need more sleep before digesting that PEP, but skimming it seems to imply that we need to explicitly encode to stdio on Windows? How will that interact with the existing encoding functions? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 5] py3: stop subscripting socket.error
On Sun, 16 Dec 2018 00:36:45 -0500, Matt Harbison wrote: > > I'm not sure what to do with this info yet, but I just noticed that > > pager is also messed up on py3- the debug message about spawning the > > pager prints, but no output for the diff. Use --pager=no, and it shows > > up. So maybe there's a general problem with stdio of spawned children. > > To be specific, this: > > diff --git a/mercurial/ui.py b/mercurial/ui.py > --- a/mercurial/ui.py > +++ b/mercurial/ui.py > @@ -1206,6 +1207,14 @@ class ui(object): > pager.stdin.close() > pager.wait() > > +try: > + self.write('test output\n') > + self.flush() > +except Exception: > +killpager() > +self.traceback(force=True) > +raise > + > return True > > @property > > > Results in this: > > $ py -3 ../hg diff -r .^^ --debug > *** failed to import extension evolve: No module named 'evolve' > obsolete feature not enabled but 177518 markers found! > starting pager for command 'diff' > > Traceback (most recent call last): >File "c:\Users\Matt\hg\mercurial\ui.py", line 1024, in _writenobuf > color.win32print(self, dest.write, msg, **opts) >File "c:\Users\Matt\hg\mercurial\color.py", line 528, in win32print > writefunc(m.group(2)) >File "c:\Users\Matt\hg\mercurial\windows.py", line 202, in write > self.fp.write(s[start:end]) > OSError: [WinError 1] Incorrect function That might be because the stdio files are backed by the Windows-ish type. Can you try PYTHONLEGACYWINDOWSSTDIO=1? https://www.python.org/dev/peps/pep-0528/ Python 3 shifted away from being Unix scripting language in many ways. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 4] py3: ensure the proxied Windows fd doesn't escape by entering context manager
On Sat, 15 Dec 2018 21:06:37 -0500, Matt Harbison wrote: > On Sat, 15 Dec 2018 20:02:08 -0500, Yuya Nishihara wrote: > > > On Sat, 15 Dec 2018 15:04:24 -0500, Matt Harbison wrote: > >> # HG changeset patch > >> # User Matt Harbison > >> # Date 1544855178 18000 > >> # Sat Dec 15 01:26:18 2018 -0500 > >> # Node ID 068910232e124a50a13fd7444844d9151db48d6b > >> # Parent 5817c3b186a7799ecc3130493ba134b55cd4ba07 > >> py3: ensure the proxied Windows fd doesn't escape by entering context > >> manager > > > > Good catch. Queued these for stable because some of them will break > > things > > if "with" statement is involved, and I can't be sure we have no such > > invocation. > > There are a couple of other odd ones. fsmonitor.state_update[1], > _AcquireFutures[2], and extensions.wrappedfunction[3] don't return > anything from __enter__(), and bundle2.partiterator returns an iterator > [4]. I thought not explicitly returning something means None is returned, > and I *assume* something would be broke if partiterator is wrong. > > > [1] > https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/hgext/fsmonitor/__init__.py#l661 > [2] > https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/thirdparty/concurrent/futures/_base.py#l149 > [3] > https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/extensions.py#l530 > [4] > https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/bundle2.py#l369 They are correct. What was wrong with the file proxies is that the caller expects a wrapped file object to be returned. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2] py3: quote $PYTHON in test-patchbomb.t for Windows
On Sat, 15 Dec 2018 23:29:44 -0500, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison > # Date 1544931114 18000 > # Sat Dec 15 22:31:54 2018 -0500 > # Node ID 0c1bac5efa4fe85778e5e6aa23380d9a7d73f57a > # Parent f8dc10fa7ca7415ee95bfe6ba50d728b2f58eab1 > py3: quote $PYTHON in test-patchbomb.t for Windows Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 5] py3: stop subscripting socket.error
On Sat, 15 Dec 2018 23:58:13 -0500, Matt Harbison wrote: On Mon, 10 Dec 2018 07:12:41 -0500, Yuya Nishihara wrote: On Sun, 09 Dec 2018 23:39:33 -0500, Matt Harbison wrote: I ended up with similar stdio errors trying to track down the bad http status failures: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1033, in _writenobuf dest.flush() OSError: [WinError 1] Incorrect function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\hgweb\server.py", line 192, in do_hgweb for chunk in self.server.application(env, self._start_response): File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 310, in run_wsgi for r in self._runwsgi(req, res, repo): File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 430, in _runwsgi return getattr(webcommands, cmd)(rctx) File "c:\Users\Matt\hg\mercurial\hgweb\webcommands.py", line 861, in comparison context = parsecontext(web.config('web', 'comparisoncontext', '5')) File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 117, in config untrusted=untrusted) File "c:\Users\Matt\hg\mercurial\ui.py", line 517, in config untrusted=untrusted) File "c:\Users\Matt\hg\mercurial\ui.py", line 536, in _config self.develwarn(msg, 2, 'warn-config-unknown') File "c:\Users\Matt\hg\mercurial\ui.py", line 1790, in develwarn % (msg, fname, lineno, fmsg)) File "c:\Users\Matt\hg\mercurial\ui.py", line 996, in write_err self._write(self._ferr, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1006, in _write self._writenobuf(dest, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1039, in _writenobuf raise error.StdioError(err) mercurial.error.StdioError: [Errno 22] Incorrect function I see that there's a TODO for pycompat.{stdin,stdout,stderr} to use a silly wrapper instead of .buffer. But this feeble attempt didn't seem to help: That's unrelated issue. A pseudo file object might not have .buffer, but a real stdio object should have one. Perhaps, the underlying file descriptor would be messed up in Windows way. If we can know that prior to calling write(), stdio fds can be reattached to NUL. But I don't know if that's possible. I'm not sure what to do with this info yet, but I just noticed that pager is also messed up on py3- the debug message about spawning the pager prints, but no output for the diff. Use --pager=no, and it shows up. So maybe there's a general problem with stdio of spawned children. To be specific, this: diff --git a/mercurial/ui.py b/mercurial/ui.py --- a/mercurial/ui.py +++ b/mercurial/ui.py @@ -1206,6 +1207,14 @@ class ui(object): pager.stdin.close() pager.wait() +try: + self.write('test output\n') + self.flush() +except Exception: +killpager() +self.traceback(force=True) +raise + return True @property Results in this: $ py -3 ../hg diff -r .^^ --debug *** failed to import extension evolve: No module named 'evolve' obsolete feature not enabled but 177518 markers found! starting pager for command 'diff' Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1024, in _writenobuf color.win32print(self, dest.write, msg, **opts) File "c:\Users\Matt\hg\mercurial\color.py", line 528, in win32print writefunc(m.group(2)) File "c:\Users\Matt\hg\mercurial\windows.py", line 202, in write self.fp.write(s[start:end]) OSError: [WinError 1] Incorrect function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1211, in _runpager self.write('test output\n') File "c:\Users\Matt\hg\mercurial\ui.py", line 993, in write self._write(self._fout, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1006, in _write self._writenobuf(dest, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1039, in _writenobuf raise error.StdioError(err) mercurial.error.StdioError: [Errno 22] Incorrect function abort: Incorrect function Absolutely no problem running this code under py2. Changing self.write() to self.error() results in the text making it to the screen, followed by an exception: test output Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1061, in flush self._ferr.flush() OSError: [WinError 1] Incorrect function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1024, in _writenobuf color.win32print(self, dest.write, msg, **opts) File "c:\Users\Matt\hg\mercurial\color.py", line 532, in win32print ui.flush() File "c:\Users\Matt\hg\mercurial\ui.py", line
Re: [PATCH 4 of 5] py3: stop subscripting socket.error
On Mon, 10 Dec 2018 07:12:41 -0500, Yuya Nishihara wrote: On Sun, 09 Dec 2018 23:39:33 -0500, Matt Harbison wrote: I ended up with similar stdio errors trying to track down the bad http status failures: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\ui.py", line 1033, in _writenobuf dest.flush() OSError: [WinError 1] Incorrect function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\Users\Matt\hg\mercurial\hgweb\server.py", line 192, in do_hgweb for chunk in self.server.application(env, self._start_response): File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 310, in run_wsgi for r in self._runwsgi(req, res, repo): File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 430, in _runwsgi return getattr(webcommands, cmd)(rctx) File "c:\Users\Matt\hg\mercurial\hgweb\webcommands.py", line 861, in comparison context = parsecontext(web.config('web', 'comparisoncontext', '5')) File "c:\Users\Matt\hg\mercurial\hgweb\hgweb_mod.py", line 117, in config untrusted=untrusted) File "c:\Users\Matt\hg\mercurial\ui.py", line 517, in config untrusted=untrusted) File "c:\Users\Matt\hg\mercurial\ui.py", line 536, in _config self.develwarn(msg, 2, 'warn-config-unknown') File "c:\Users\Matt\hg\mercurial\ui.py", line 1790, in develwarn % (msg, fname, lineno, fmsg)) File "c:\Users\Matt\hg\mercurial\ui.py", line 996, in write_err self._write(self._ferr, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1006, in _write self._writenobuf(dest, *args, **opts) File "c:\Users\Matt\hg\mercurial\ui.py", line 1039, in _writenobuf raise error.StdioError(err) mercurial.error.StdioError: [Errno 22] Incorrect function I see that there's a TODO for pycompat.{stdin,stdout,stderr} to use a silly wrapper instead of .buffer. But this feeble attempt didn't seem to help: That's unrelated issue. A pseudo file object might not have .buffer, but a real stdio object should have one. Perhaps, the underlying file descriptor would be messed up in Windows way. If we can know that prior to calling write(), stdio fds can be reattached to NUL. But I don't know if that's possible. I'm not sure what to do with this info yet, but I just noticed that pager is also messed up on py3- the debug message about spawning the pager prints, but no output for the diff. Use --pager=no, and it shows up. So maybe there's a general problem with stdio of spawned children. I've been wondering why we aren't spawning the server child process with stdout and stderr handles set to NUL. I know on Unix, the child prints a bit of info before it starts listening. But that never worked on Windows, and we've been abusing the lock file to convey error messages back to the parent. Another Py3 issue is that sys.stdin/stdout/stderr is None on pythonw.exe. IIRC, this can be worked around by attaching NUL to fd=0-2, and open them. nullfd = os.open(os.devnull, O_RDWR | O_BINARY) os.dup2(nullfd, 0) os.dup2(nullfd, 1) os.dup2(nullfd, 2) os.close(nullfd) stdin = os.fdopen(0, 'rb') stdout = os.fdopen(1, 'wb') stderr = os.fdopen(2, 'wb') But it might break ui.isatty() since NUL is reported as a tty on Windows. About NUL being a tty, maybe this will help? https://stackoverflow.com/a/3660125 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5442: rust-cpython: using the new bindings from Python
yuja added a comment. > +try: > +from . import rustext > +except ImportError: > +rustext = None Need to access to e.g. rustext.__doc__ to trigger ImportError here. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5442 To: gracinet, indygreg, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5443: ancestor: uniformity of calling lazyancestors classes
yuja added a comment. > +def parentsfunc(index): > +def parentrevs(rev): > +try: > +entry = index[rev] > +except IndexError: > +if rev == wdirrev: > +raise error.WdirUnsupported > +raise > + > +return entry[5], entry[6] > +return parentrevs It doesn't seem correct that the ancestor module handles the revlog's internals. Instead, can't we add a factory function which takes both `index` and `pfunc`? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5443 To: gracinet, indygreg, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5442: rust-cpython: using the new bindings from Python
> +try: > +from . import rustext > +except ImportError: > +rustext = None Need to access to e.g. rustext.__doc__ to trigger ImportError here. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5443: ancestor: uniformity of calling lazyancestors classes
> +def parentsfunc(index): > +def parentrevs(rev): > +try: > +entry = index[rev] > +except IndexError: > +if rev == wdirrev: > +raise error.WdirUnsupported > +raise > + > +return entry[5], entry[6] > +return parentrevs It doesn't seem correct that the ancestor module handles the revlog's internals. Instead, can't we add a factory function which takes both `index` and `pfunc`? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5441: rust-cpython: binding for LazyAncestors
yuja added a comment. > m.add_class::(py)?; > > +m.add_class::(py)?; Nit: While it's correct per our naming convention, I prefer calling it as `LazyAncestors` in Rust, and export as `lazyancestors`. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5441 To: gracinet, #hg-reviewers, kevincox Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5440: rust: core implementation for lazyancestors
yuja added a comment. > - a/rust/hg-core/src/lib.rs +++ b/rust/hg-core/src/lib.rs @@ -2,8 +2,10 @@ // // This software may be used and distributed according to the terms of the // GNU General Public License version 2 or any later version. +use std::clone::Clone; Nit: it's in prelude. > /// The simplest expression of what we need of Mercurial DAGs. > > -pub trait Graph { > +pub trait Graph: Clone { I think it's the requirement of the caller, i.e. `G: Graph + Clone`. Not all graphs have to be clone-able. Alternatively, maybe the LazyAncestor can be moved to the cpython crate, where PyClone is available. > +/// Tell if the iterator is about an empty set > +/// > +/// The result does not depend whether the iterator has been consumed > +/// or not. > +/// This is mostly meant for iterators backing a lazy ancestors set > +pub fn empty() -> bool { Nit: s/empty/is_empty/ because `empty()` can be a verb. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5440 To: gracinet, #hg-reviewers, kevincox Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5441: rust-cpython: binding for LazyAncestors
> m.add_class::(py)?; > +m.add_class::(py)?; Nit: While it's correct per our naming convention, I prefer calling it as `LazyAncestors` in Rust, and export as `lazyancestors`. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5440: rust: core implementation for lazyancestors
> --- a/rust/hg-core/src/lib.rs > +++ b/rust/hg-core/src/lib.rs > @@ -2,8 +2,10 @@ > // > // This software may be used and distributed according to the terms of the > // GNU General Public License version 2 or any later version. > +use std::clone::Clone; Nit: it's in prelude. > /// The simplest expression of what we need of Mercurial DAGs. > -pub trait Graph { > +pub trait Graph: Clone { I think it's the requirement of the caller, i.e. `G: Graph + Clone`. Not all graphs have to be clone-able. Alternatively, maybe the LazyAncestor can be moved to the cpython crate, where PyClone is available. > +/// Tell if the iterator is about an empty set > +/// > +/// The result does not depend whether the iterator has been consumed > +/// or not. > +/// This is mostly meant for iterators backing a lazy ancestors set > +pub fn empty() -> bool { Nit: s/empty/is_empty/ because `empty()` can be a verb. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] color: fix a documentation typo
# HG changeset patch # User Matt Harbison # Date 1544933643 18000 # Sat Dec 15 23:14:03 2018 -0500 # Node ID 4e1802a4834c9bf0a04eac3a981e503491016c3e # Parent 0c1bac5efa4fe85778e5e6aa23380d9a7d73f57a color: fix a documentation typo diff --git a/mercurial/color.py b/mercurial/color.py --- a/mercurial/color.py +++ b/mercurial/color.py @@ -245,7 +245,7 @@ def _modesetup(ui): # Since "ansi" could result in terminal gibberish, we error on the # side of selecting "win32". However, if w32effects is not defined, # we almost certainly don't support "win32", so don't even try. -# w32ffects is not populated when stdout is redirected, so checking +# w32effects is not populated when stdout is redirected, so checking # it first avoids win32 calls in a state known to error out. if ansienviron or not w32effects or win32.enablevtmode(): realmode = 'ansi' ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2] py3: quote $PYTHON in test-patchbomb.t for Windows
# HG changeset patch # User Matt Harbison # Date 1544931114 18000 # Sat Dec 15 22:31:54 2018 -0500 # Node ID 0c1bac5efa4fe85778e5e6aa23380d9a7d73f57a # Parent f8dc10fa7ca7415ee95bfe6ba50d728b2f58eab1 py3: quote $PYTHON in test-patchbomb.t for Windows I couldn't get the quoting right in the environment variable, so now it's a function. diff --git a/tests/test-patchbomb.t b/tests/test-patchbomb.t --- a/tests/test-patchbomb.t +++ b/tests/test-patchbomb.t @@ -22,7 +22,9 @@ Mercurial-patchbomb/.* -> Mercurial-patc > skipblank = False > print(l, end='') > EOF - $ FILTERBOUNDARY="$PYTHON `pwd`/prune-blank-after-boundary.py" + $ filterboundary() { + > "$PYTHON" "$TESTTMP/prune-blank-after-boundary.py" + > } $ echo "[extensions]" >> $HGRCPATH $ echo "patchbomb=" >> $HGRCPATH @@ -357,7 +359,7 @@ Test breaking format changes aren't test bundle and description: $ hg email --date '1970-1-1 0:3' -n -f quux -t foo \ - > -c bar -s test -r tip -b --desc description | $FILTERBOUNDARY + > -c bar -s test -r tip -b --desc description | filterboundary searching for changes 1 changesets found @@ -403,7 +405,7 @@ with a specific bundle type $ hg email --date '1970-1-1 0:3' -n -f quux -t foo \ > -c bar -s test -r tip -b --desc description \ - > --config patchbomb.bundletype=gzip-v1 | $FILTERBOUNDARY + > --config patchbomb.bundletype=gzip-v1 | filterboundary searching for changes 1 changesets found @@ -884,7 +886,7 @@ test diffstat for multiple patches: test inline for single patch: - $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i -r 2 | $FILTERBOUNDARY + $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i -r 2 | filterboundary this patch series consists of 1 patches. @@ -927,7 +929,7 @@ test inline for single patch: test inline for single patch (quoted-printable): - $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i -r 4 | $FILTERBOUNDARY + $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i -r 4 | filterboundary this patch series consists of 1 patches. @@ -986,7 +988,7 @@ test inline for single patch (quoted-pri test inline for multiple patches: $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i \ - > -r 0:1 -r 4 | $FILTERBOUNDARY + > -r 0:1 -r 4 | filterboundary this patch series consists of 3 patches. @@ -1138,7 +1140,7 @@ test inline for multiple patches: --===*=-- (glob) test attach for single patch: - $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a -r 2 | $FILTERBOUNDARY + $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a -r 2 | filterboundary this patch series consists of 1 patches. @@ -1189,7 +1191,7 @@ test attach for single patch: --===*=-- (glob) test attach for single patch (quoted-printable): - $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a -r 4 | $FILTERBOUNDARY + $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a -r 4 | filterboundary this patch series consists of 1 patches. @@ -1256,7 +1258,7 @@ test attach for single patch (quoted-pri --===*=-- (glob) test attach and body for single patch: - $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a --body -r 2 | $FILTERBOUNDARY + $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a --body -r 2 | filterboundary this patch series consists of 1 patches. @@ -1318,7 +1320,7 @@ test attach and body for single patch: test attach for multiple patches: $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -a \ - > -r 0:1 -r 4 | $FILTERBOUNDARY + > -r 0:1 -r 4 | filterboundary this patch series consists of 3 patches. @@ -1775,7 +1777,7 @@ tagging csets: test inline for single named patch: $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i \ - > -r 2 | $FILTERBOUNDARY + > -r 2 | filterboundary this patch series consists of 1 patches. @@ -1818,7 +1820,7 @@ test inline for single named patch: test inline for multiple named/unnamed patches: $ hg email --date '1970-1-1 0:1' -n -f quux -t foo -c bar -s test -i \ - >-r 0:1 | $FILTERBOUNDARY + >-r 0:1 | filterboundary this patch series consists of 2 patches. @@ -2124,7 +2126,7 @@ test single flag for single patch (and n $ hg up -qr1 $ echo dirt > a $ hg email --date '1970-1-1 0:1' -n --flag fooFlag -f quux -t foo -c bar -s test \ - > -r 2 | $FILTERBOUNDARY + > -r 2 | filterboundary this patch series consists of 1 patches. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5439: rust-cpython: binding for AncestorsIterator
yuja added a comment. > // GNU General Public License version 2 or any later version. > > - //! Bindings for the hg::ancestors module provided by the Nit: perhaps the empty line was removed by mistake. > +fn reviter_to_revvec(py: Python, revs: PyObject) -> PyResult> { > +let revs_iter = revs.iter(py)?; > +// we need to convert to a vector, because Python iterables can > +// raise errors at each step. > +let cap = match revs.len(py) { > +Ok(l) => l, > +Err(_) => 0, // unknown > +}; > +let mut initvec: Vec = Vec::with_capacity(cap); > +for result_revpy in revs_iter { > +let as_pyint: PyInt = match result_revpy { > +Err(e) => { > +return Err(e); > +} > +Ok(revpy) => revpy.extract(py)?, > +}; > +initvec.push(as_pyint.value(py) as Revision); > +} > +Ok(initvec) revs_iter.map(|r| r.and_then(|o| o.extract::(py))).collect() `PyInt::value()` isn't supported on Python 3, and it should be better to extract int directly with bounds checking. https://dgrunwald.github.io/rust-cpython/doc/cpython/struct.PyInt.html#method.value > +impl AncestorsIterator { > +pub fn from_inner(py: Python, ait: CoreIterator) -> PyResult { Nit: I slightly prefer spelling it as `hg::AncestorsIterator`. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5439 To: gracinet, #hg-reviewers, kevincox Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5439: rust-cpython: binding for AncestorsIterator
> // GNU General Public License version 2 or any later version. > - > //! Bindings for the hg::ancestors module provided by the Nit: perhaps the empty line was removed by mistake. > +fn reviter_to_revvec(py: Python, revs: PyObject) -> PyResult> { > +let revs_iter = revs.iter(py)?; > +// we need to convert to a vector, because Python iterables can > +// raise errors at each step. > +let cap = match revs.len(py) { > +Ok(l) => l, > +Err(_) => 0, // unknown > +}; > +let mut initvec: Vec = Vec::with_capacity(cap); > +for result_revpy in revs_iter { > +let as_pyint: PyInt = match result_revpy { > +Err(e) => { > +return Err(e); > +} > +Ok(revpy) => revpy.extract(py)?, > +}; > +initvec.push(as_pyint.value(py) as Revision); > +} > +Ok(initvec) ``` revs_iter.map(|r| r.and_then(|o| o.extract::(py))).collect() ``` `PyInt::value()` isn't supported on Python 3, and it should be better to extract int directly with bounds checking. https://dgrunwald.github.io/rust-cpython/doc/cpython/struct.PyInt.html#method.value > +impl AncestorsIterator { > +pub fn from_inner(py: Python, ait: CoreIterator) -> > PyResult { Nit: I slightly prefer spelling it as `hg::AncestorsIterator`. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5438: rust-cpython: implementing Graph using C parents function
yuja added a comment. > +type IndexParentsFn = unsafe extern "C" fn( > +index: *mut python_sys::PyObject, > +rev: ssize_t, > +ps: *mut [c_int; 2], > +max_rev: c_int, > +) -> c_int; The type differs from the private function. It's `int HgRevlogIndex_GetParents(PyObject *op, int rev, int *ps)`. > +impl Graph for Index { > +/// wrap a call to the C extern parents function > +fn parents(, rev: Revision) -> Result<[Revision; 2], GraphError> { > +let mut res: [c_int; 2] = [0; 2]; > +let code = unsafe { > +(self.parents)( > +self.index.as_ptr(), > +rev as ssize_t, > + res as *mut [c_int; 2], > +rev, > +) > +}; > +match code { > +0 => Ok(res), > +_ => Err(GraphError::ParentOutOfRange(rev)), > +} I think `Python<'p>` is needed here to make it safe, but I have no idea how to express that clearly. Do we need a wrapper that implements `Graph` and translates `Graph::parents(rev)` to `Index::parents(py, rev)`? > - a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -2878,6 +2878,12 @@if (nullentry) PyObject_GC_UnTrack(nullentry); > > + void *caps = PyCapsule_New( Please move `void *caps` to top. We still need to conform to C89. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5438 To: gracinet, #hg-reviewers Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5438: rust-cpython: implementing Graph using C parents function
> +type IndexParentsFn = unsafe extern "C" fn( > +index: *mut python_sys::PyObject, > +rev: ssize_t, > +ps: *mut [c_int; 2], > +max_rev: c_int, > +) -> c_int; The type differs from the private function. It's `int HgRevlogIndex_GetParents(PyObject *op, int rev, int *ps)`. > +impl Graph for Index { > +/// wrap a call to the C extern parents function > +fn parents(, rev: Revision) -> Result<[Revision; 2], GraphError> { > +let mut res: [c_int; 2] = [0; 2]; > +let code = unsafe { > +(self.parents)( > +self.index.as_ptr(), > +rev as ssize_t, > + res as *mut [c_int; 2], > +rev, > +) > +}; > +match code { > +0 => Ok(res), > +_ => Err(GraphError::ParentOutOfRange(rev)), > +} I think `Python<'p>` is needed here to make it safe, but I have no idea how to express that clearly. Do we need a wrapper that implements `Graph` and translates `Graph::parents(rev)` to `Index::parents(py, rev)`? > --- a/mercurial/cext/revlog.c > +++ b/mercurial/cext/revlog.c > @@ -2878,6 +2878,12 @@ > if (nullentry) > PyObject_GC_UnTrack(nullentry); > > + void *caps = PyCapsule_New( Please move `void *caps` to top. We still need to conform to C89. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5434: rust-cpython: started cpython crate bindings
This revision was automatically updated to reflect the committed changes. Closed by commit rHGa3ba080b3118: rust-cpython: start cpython crate bindings (authored by gracinet, committed by ). CHANGED PRIOR TO COMMIT https://phab.mercurial-scm.org/D5434?vs=12871=12883#toc REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5434?vs=12871=12883 REVISION DETAIL https://phab.mercurial-scm.org/D5434 AFFECTED FILES rust/Cargo.lock rust/Cargo.toml rust/hg-cpython/Cargo.toml rust/hg-cpython/rustfmt.toml rust/hg-cpython/src/ancestors.rs rust/hg-cpython/src/exceptions.rs rust/hg-cpython/src/lib.rs CHANGE DETAILS diff --git a/rust/hg-cpython/src/lib.rs b/rust/hg-cpython/src/lib.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/lib.rs @@ -0,0 +1,40 @@ +// lib.rs +// +// Copyright 2018 Georges Racinet +// +// This software may be used and distributed according to the terms of the +// GNU General Public License version 2 or any later version. + +//! Python bindings of `hg-core` objects using the `cpython` crate. +//! Once compiled, the resulting single shared library object can be placed in +//! the `mercurial` package directly as `rustext.so` or `rustext.dll`. +//! It holds several modules, so that from the point of view of Python, +//! it behaves as the `cext` package. +//! +//! Example: +//! ``` +//! >>> from mercurial.rustext import ancestor +//! >>> ancestor.__doc__ +//! 'Generic DAG ancestor algorithms - Rust implementation' +//! ``` + +#[macro_use] +extern crate cpython; +extern crate hg; + +mod ancestors; +mod exceptions; + +py_module_initializer!(rustext, initrustext, PyInit_rustext, |py, m| { +m.add( +py, +"__doc__", +"Mercurial core concepts - Rust implementation", +)?; + +let dotted_name: String = m.get(py, "__name__")?.extract(py)?; +m.add(py, "__package__", "mercurial")?; +m.add(py, "ancestor", ancestors::init_module(py, _name)?)?; +m.add(py, "GraphError", py.get_type::())?; +Ok(()) +}); diff --git a/rust/hg-cpython/src/exceptions.rs b/rust/hg-cpython/src/exceptions.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/exceptions.rs @@ -0,0 +1,15 @@ +use cpython::exc::ValueError; +use cpython::{PyErr, Python}; +use hg; + +py_exception!(rustext, GraphError, ValueError); + +impl GraphError { +pub fn pynew(py: Python, inner: hg::GraphError) -> PyErr { +match inner { +hg::GraphError::ParentOutOfRange(r) => { +GraphError::new(py, ("ParentOutOfRange", r)) +} +} +} +} diff --git a/rust/hg-cpython/src/ancestors.rs b/rust/hg-cpython/src/ancestors.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/ancestors.rs @@ -0,0 +1,30 @@ +// ancestors.rs +// +// Copyright 2018 Georges Racinet +// +// This software may be used and distributed according to the terms of the +// GNU General Public License version 2 or any later version. + +//! Bindings for the hg::ancestors module provided by the +//! `hg-core` crate. From Python, this will be seen as `rustext.ancestor` +use cpython::{PyDict, PyModule, PyResult, Python}; + +/// Create the module, with __package__ given from parent +pub fn init_module(py: Python, package: ) -> PyResult { +let dotted_name = !("{}.ancestor", package); +let m = PyModule::new(py, dotted_name)?; +m.add(py, "__package__", package)?; +m.add( +py, +"__doc__", +"Generic DAG ancestor algorithms - Rust implementation", +)?; + +let sys = PyModule::import(py, "sys")?; +let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; +sys_modules.set_item(py, dotted_name, )?; +// Example C code (see pyexpat.c and import.c) will "give away the +// reference", but we won't because it will be consumed once the +// Rust PyObject is dropped. +Ok(m) +} diff --git a/rust/hg-cpython/rustfmt.toml b/rust/hg-cpython/rustfmt.toml new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/rustfmt.toml @@ -0,0 +1,3 @@ +max_width = 79 +wrap_comments = true +error_on_line_overflow = true diff --git a/rust/hg-cpython/Cargo.toml b/rust/hg-cpython/Cargo.toml new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/Cargo.toml @@ -0,0 +1,29 @@ +[package] +name = "hg-cpython" +version = "0.1.0" +authors = ["Georges Racinet "] + +[lib] +name='rusthg' +crate-type = ["cdylib"] + +[features] +default = ["python27", "python27-sys"] + +python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"] + +[dependencies] +hg-core = { path = "../hg-core" } +libc = '*' + +[dependencies.cpython] +version = "*" +default-features = false + +[dependencies.python27-sys] +version = "0.2.1" +optional = true + +[dependencies.python3-sys] +version = "0.2.1" +optional = true diff --git a/rust/Cargo.toml b/rust/Cargo.toml --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -1,3 +1,3 @@ [workspace] -members = ["hg-core", "hg-direct-ffi"] +members = ["hg-core", "hg-direct-ffi",
D5433: rust-cpython: excluded hgcli from workspace
This revision was automatically updated to reflect the committed changes. Closed by commit rHG6e815adf91de: rust-cpython: exclude hgcli from workspace (authored by gracinet, committed by ). CHANGED PRIOR TO COMMIT https://phab.mercurial-scm.org/D5433?vs=12870=12882#toc REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5433?vs=12870=12882 REVISION DETAIL https://phab.mercurial-scm.org/D5433 AFFECTED FILES rust/Cargo.lock rust/Cargo.toml rust/hgcli/Cargo.lock CHANGE DETAILS diff --git a/rust/Cargo.lock b/rust/hgcli/Cargo.lock copy from rust/Cargo.lock copy to rust/hgcli/Cargo.lock --- a/rust/Cargo.lock +++ b/rust/hgcli/Cargo.lock @@ -11,33 +11,21 @@ version = "0.1.0" source = "git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52#c90d65cf84abfffce7ef54476bbfed56017a2f52; dependencies = [ - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", - "num-traits 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", + "num-traits 0.1.43 (registry+https://github.com/rust-lang/crates.io-index)", "python27-sys 0.1.2 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", ] [[package]] -name = "hg-core" -version = "0.1.0" - -[[package]] name = "hgcli" version = "0.1.0" dependencies = [ "cpython 0.1.0 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", "python27-sys 0.1.2 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", ] [[package]] -name = "hgdirectffi" -version = "0.1.0" -dependencies = [ - "hg-core 0.1.0", - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] name = "kernel32-sys" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index; @@ -48,28 +36,36 @@ [[package]] name = "libc" -version = "0.2.35" +version = "0.2.45" source = "registry+https://github.com/rust-lang/crates.io-index; [[package]] name = "memchr" version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index; dependencies = [ - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] name = "num-traits" -version = "0.1.41" +version = "0.1.43" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "num-traits" +version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index; [[package]] name = "python27-sys" version = "0.1.2" source = "git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52#c90d65cf84abfffce7ef54476bbfed56017a2f52; dependencies = [ - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", ] @@ -96,7 +92,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index; dependencies = [ "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", ] [[package]] @@ -126,9 +122,10 @@ "checksum aho-corasick 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ca972c2ea5f742bfce5687b9aef75506a764f61d37f8f649047846a9686ddb66" "checksum cpython 0.1.0 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)" = "" "checksum kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7507624b29483431c0ba2d82aece8ca6cdba9382bff4ddd0f7490560c056098d" -"checksum libc 0.2.35 (registry+https://github.com/rust-lang/crates.io-index)" = "96264e9b293e95d25bfcbbf8a88ffd1aedc85b754eba8b7d78012f638ba220eb" +"checksum libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)" = "2d2857ec59fadc0773853c664d2d18e7198e83883e7060b63c924cb077bd5c74" "checksum memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "d8b629fb514376c675b98c1421e80b151d3817ac42d7c667717d282761418d20" -"checksum num-traits 0.1.41 (registry+https://github.com/rust-lang/crates.io-index)" = "cacfcab5eb48250ee7d0c7896b51a2c5eec99c1feea5f32025635f5ae4b00070" +"checksum num-traits 0.1.43 (registry+https://github.com/rust-lang/crates.io-index)" = "92e5113e9fd4cc14ded8e499429f396a20f98c772a47cc8622a736e1ec843c31" +"checksum num-traits 0.2.6
D5436: rust-cpython: build via HGWITHRUSTEXT=cpython
This revision was automatically updated to reflect the committed changes. Closed by commit rHG9e755c16f05d: rust-cpython: build via HGWITHRUSTEXT=cpython (authored by gracinet, committed by ). CHANGED PRIOR TO COMMIT https://phab.mercurial-scm.org/D5436?vs=12873=12885#toc REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5436?vs=12873=12885 REVISION DETAIL https://phab.mercurial-scm.org/D5436 AFFECTED FILES Makefile setup.py CHANGE DETAILS diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -132,7 +132,11 @@ ispypy = "PyPy" in sys.version -iswithrustextensions = 'HGWITHRUSTEXT' in os.environ +hgrustext = os.environ.get('HGWITHRUSTEXT') +# TODO record it for proper rebuild upon changes +# (see mercurial/__modulepolicy__.py) +if hgrustext != 'cpython' and hgrustext is not None: +hgrustext = 'direct-ffi' import ctypes import errno @@ -458,11 +462,18 @@ return build_ext.initialize_options(self) def build_extensions(self): +ruststandalones = [e for e in self.extensions + if isinstance(e, RustStandaloneExtension)] +self.extensions = [e for e in self.extensions + if e not in ruststandalones] # Filter out zstd if disabled via argument. if not self.zstd: self.extensions = [e for e in self.extensions if e.name != 'mercurial.zstd'] +for rustext in ruststandalones: +rustext.build('' if self.inplace else self.build_lib) + return build_ext.build_extensions(self) def build_extension(self, ext): @@ -903,20 +914,16 @@ """Exception class for Rust compilation errors.""" class RustExtension(Extension): -"""A C Extension, conditionnally enhanced with Rust code. - -if iswithrustextensions is False, does nothing else than plain Extension +"""Base classes for concrete Rust Extension classes. """ rusttargetdir = os.path.join('rust', 'target', 'release') def __init__(self, mpath, sources, rustlibname, subcrate, **kw): Extension.__init__(self, mpath, sources, **kw) -if not iswithrustextensions: +if hgrustext is None: return srcdir = self.rustsrcdir = os.path.join('rust', subcrate) -self.libraries.append(rustlibname) -self.extra_compile_args.append('-DWITH_RUST') # adding Rust source and control files to depends so that the extension # gets rebuilt if they've changed @@ -930,7 +937,7 @@ if os.path.splitext(fname)[1] == '.rs') def rustbuild(self): -if not iswithrustextensions: +if hgrustext is None: return env = os.environ.copy() if 'HGTEST_RESTOREENV' in env: @@ -962,6 +969,40 @@ "Cargo failed. Working directory: %r, " "command: %r, environment: %r" % (self.rustsrcdir, cmd, env)) +class RustEnhancedExtension(RustExtension): +"""A C Extension, conditionally enhanced with Rust code. + +If the HGRUSTEXT environment variable is set to something else +than 'cpython', the Rust sources get compiled and linked within the +C target shared library object. +""" + +def __init__(self, mpath, sources, rustlibname, subcrate, **kw): +RustExtension.__init__(self, mpath, sources, rustlibname, subcrate, + **kw) +if hgrustext != 'direct-ffi': +return +self.extra_compile_args.append('-DWITH_RUST') +self.libraries.append(rustlibname) +self.library_dirs.append(self.rusttargetdir) + +class RustStandaloneExtension(RustExtension): + +def __init__(self, pydottedname, rustcrate, dylibname, **kw): +RustExtension.__init__(self, pydottedname, [], dylibname, rustcrate, + **kw) +self.dylibname = dylibname + +def build(self, target_dir): +self.rustbuild() +target = [target_dir] +target.extend(self.name.split('.')) +ext = '.so' # TODO Unix only +target[-1] += ext +shutil.copy2(os.path.join(self.rusttargetdir, self.dylibname + ext), + os.path.join(*target)) + + extmodules = [ Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'], include_dirs=common_include_dirs, @@ -974,19 +1015,20 @@ 'mercurial/cext/mpatch.c'], include_dirs=common_include_dirs, depends=common_depends), -RustExtension('mercurial.cext.parsers', ['mercurial/cext/charencode.c', - 'mercurial/cext/dirs.c', - 'mercurial/cext/manifest.c', - 'mercurial/cext/parsers.c', - 'mercurial/cext/pathencode.c', -
D5437: rust-cpython: testing the bindings from Python
This revision was automatically updated to reflect the committed changes. Closed by commit rHG57e3dfeb3a5d: rust-cpython: testing the bindings from Python (authored by gracinet, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5437?vs=12874=12886 REVISION DETAIL https://phab.mercurial-scm.org/D5437 AFFECTED FILES tests/test-rust-ancestor.py CHANGE DETAILS diff --git a/tests/test-rust-ancestor.py b/tests/test-rust-ancestor.py new file mode 100644 --- /dev/null +++ b/tests/test-rust-ancestor.py @@ -0,0 +1,39 @@ +from __future__ import absolute_import +import unittest + +try: +from mercurial import rustext +except ImportError: +rustext = None + +try: +from mercurial.cext import parsers as cparsers +except ImportError: +cparsers = None + +@unittest.skipIf(rustext is None or cparsers is None, + "rustext.ancestor or the C Extension parsers module " + "it relies on is not available") +class rustancestorstest(unittest.TestCase): +"""Test the correctness of binding to Rust code. + +This test is merely for the binding to Rust itself: extraction of +Python variable, giving back the results etc. + +It is not meant to test the algorithmic correctness of the operations +on ancestors it provides. Hence the very simple embedded index data is +good enough. + +Algorithmic correctness is asserted by the Rust unit tests. +""" + +def testmodule(self): +self.assertTrue('DAG' in rustext.ancestor.__doc__) + +def testgrapherror(self): +self.assertTrue('GraphError' in dir(rustext)) + + +if __name__ == '__main__': +import silenttestrunner +silenttestrunner.main(__name__) To: gracinet, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5435: rust: better treatment of cargo/rustc errors
This revision was automatically updated to reflect the committed changes. Closed by commit rHG5955cf85ed74: rust: better treatment of cargo/rustc errors (authored by gracinet, committed by ). CHANGED PRIOR TO COMMIT https://phab.mercurial-scm.org/D5435?vs=12872=12884#toc REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5435?vs=12872=12884 REVISION DETAIL https://phab.mercurial-scm.org/D5435 AFFECTED FILES setup.py CHANGE DETAILS diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -135,6 +135,7 @@ iswithrustextensions = 'HGWITHRUSTEXT' in os.environ import ctypes +import errno import stat, subprocess, time import re import shutil @@ -898,6 +899,9 @@ 'mercurial/thirdparty/xdiff/xutils.h', ] +class RustCompilationError(CCompilerError): +"""Exception class for Rust compilation errors.""" + class RustExtension(Extension): """A C Extension, conditionnally enhanced with Rust code. @@ -942,9 +946,21 @@ import pwd env['HOME'] = pwd.getpwuid(os.getuid()).pw_dir -subprocess.check_call(['cargo', 'build', '-vv', '--release'], - env=env, cwd=self.rustsrcdir) -self.library_dirs.append(self.rusttargetdir) +cargocmd = ['cargo', 'build', '-vv', '--release'] +try: +subprocess.check_call(cargocmd, env=env, cwd=self.rustsrcdir) +except OSError as exc: +if exc.errno == errno.ENOENT: +raise RustCompilationError("Cargo not found") +elif exc.errno == errno.EACCES: +raise RustCompilationError( +"Cargo found, but permisssion to execute it is denied") +else: +raise +except subprocess.CalledProcessError: +raise RustCompilationError( +"Cargo failed. Working directory: %r, " +"command: %r, environment: %r" % (self.rustsrcdir, cmd, env)) extmodules = [ Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'], To: gracinet, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5434: rust-cpython: started cpython crate bindings
yuja added a comment. I've queued the first 5 patches, thanks. Please send a follow up to fix nits and minor issues. > +py_module_initializer!(rustext, initrustext, PyInit_rustext, |py, m| { > +m.add( > +py, > +"__doc__", > +"Mercurial core concepts - Rust implementation", > +)?; > + > +let dotted_name: String = m.get(py, "__name__")?.extract(py)?; > +m.add(py, "__package__", "mercurial")?; IIUC, __package__ should be set by the importer, not by the module. And if set, I think it should be "mercurial.rustext" since this is a package module. https://stackoverflow.com/a/2124/10435339 > +impl GraphError { > +pub fn pynew(py: Python, inner: hg::GraphError) -> PyErr { > +match inner { > +hg::GraphError::ParentOutOfRange(r) => { > +GraphError::new(py, ("ParentOutOfRange", r)) > +} > +} > +} > +} Maybe this can be a trait impl or helper function that maps `hg::GraphError` to `PyErr::new::()` ? > +let sys = PyModule::import(py, "sys")?; > +let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; > +sys_modules.set_item(py, dotted_name, )?; We'll probably want to move this to the caller to avoid dups. The init function can have the same signature as the lambda of `py_module_initializer!()`. > +[lib] > +name='rusthg' Perhaps this is an old name. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5434 To: gracinet, #hg-reviewers Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5434: rust-cpython: started cpython crate bindings
I've queued the first 5 patches, thanks. Please send a follow up to fix nits and minor issues. > +py_module_initializer!(rustext, initrustext, PyInit_rustext, |py, m| { > +m.add( > +py, > +"__doc__", > +"Mercurial core concepts - Rust implementation", > +)?; > + > +let dotted_name: String = m.get(py, "__name__")?.extract(py)?; > +m.add(py, "__package__", "mercurial")?; IIUC, __package__ should be set by the importer, not by the module. And if set, I think it should be "mercurial.rustext" since this is a package module. https://stackoverflow.com/a/2124/10435339 > +impl GraphError { > +pub fn pynew(py: Python, inner: hg::GraphError) -> PyErr { > +match inner { > +hg::GraphError::ParentOutOfRange(r) => { > +GraphError::new(py, ("ParentOutOfRange", r)) > +} > +} > +} > +} Maybe this can be a trait impl or helper function that maps `hg::GraphError` to `PyErr::new::()` ? > +let sys = PyModule::import(py, "sys")?; > +let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; > +sys_modules.set_item(py, dotted_name, )?; We'll probably want to move this to the caller to avoid dups. The init function can have the same signature as the lambda of `py_module_initializer!()`. > +[lib] > +name='rusthg' Perhaps this is an old name. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5436: rust-cpython: build via HGWITHRUSTEXT=cpython
yuja added a comment. I expect we'll soon drop the support for the old ffi binding. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5436 To: gracinet, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5436: rust-cpython: build via HGWITHRUSTEXT=cpython
I expect we'll soon drop the support for the old ffi binding. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5435: rust: better treatment of cargo/rustc errors
yuja added a comment. > +cargocmd = ['cargo', 'build', '-vv', '--release'] > +try: > +subprocess.check_call(cargocmd, env=env, cwd=self.rustsrcdir) > +except OSError as exc: > +if exc.errno == os.errno.ENOENT: Added `import errno` to not abuse `os.errno`. I don't think it's a re-exported module. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5435 To: gracinet, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5435: rust: better treatment of cargo/rustc errors
> +cargocmd = ['cargo', 'build', '-vv', '--release'] > +try: > +subprocess.check_call(cargocmd, env=env, cwd=self.rustsrcdir) > +except OSError as exc: > +if exc.errno == os.errno.ENOENT: Added `import errno` to not abuse `os.errno`. I don't think it's a re-exported module. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 4] py3: ensure the proxied Windows fd doesn't escape by entering context manager
On Sat, 15 Dec 2018 20:02:08 -0500, Yuya Nishihara wrote: On Sat, 15 Dec 2018 15:04:24 -0500, Matt Harbison wrote: # HG changeset patch # User Matt Harbison # Date 1544855178 18000 # Sat Dec 15 01:26:18 2018 -0500 # Node ID 068910232e124a50a13fd7444844d9151db48d6b # Parent 5817c3b186a7799ecc3130493ba134b55cd4ba07 py3: ensure the proxied Windows fd doesn't escape by entering context manager Good catch. Queued these for stable because some of them will break things if "with" statement is involved, and I can't be sure we have no such invocation. There are a couple of other odd ones. fsmonitor.state_update[1], _AcquireFutures[2], and extensions.wrappedfunction[3] don't return anything from __enter__(), and bundle2.partiterator returns an iterator [4]. I thought not explicitly returning something means None is returned, and I *assume* something would be broke if partiterator is wrong. [1] https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/hgext/fsmonitor/__init__.py#l661 [2] https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/thirdparty/concurrent/futures/_base.py#l149 [3] https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/extensions.py#l530 [4] https://www.mercurial-scm.org/repo/hg/file/e06719b7544d/mercurial/bundle2.py#l369 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
yuja added a comment. I feel `--no-` and `--=off|false` are more natural, and `(default: %s)` is the latter form. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5444 To: av6, durin42, #hg-reviewers Cc: yuja, martinvonz, pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
I feel `--no-` and `--=off|false` are more natural, and `(default: %s)` is the latter form. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4 V2] sparse-revlog: protect C code against delta chain including nullrev
On Sat, 15 Dec 2018 15:10:56 +, Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1544804741 -3600 > # Fri Dec 14 17:25:41 2018 +0100 > # Node ID e7d33dc28696a1b2ee951cb82fe2cd611037afc8 > # Parent 36c68746763d6b93b00c19387e79bd9cb623da72 > # EXP-Topic sparse-followup > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > e7d33dc28696 > sparse-revlog: protect C code against delta chain including nullrev > > For unclear reasons, some repositories include nullrev (-1). Re-computing > delta for such repo remove nullrev from all chain, so some older versions have > been creating them. > > This currently raise an IndexError with the new C code doing chain slicing as > it expect all item to be positive. > > Both python and C code for reading delta chain preserve nullrev, and the > Python > code for chain slicing handle the case fine. So we take the safe route and > make > the new C code works fine in that case. > > diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c > --- a/mercurial/cext/revlog.c > +++ b/mercurial/cext/revlog.c > @@ -1195,7 +1195,7 @@ static PyObject *index_slicechunktodensi > if (revnum == -1 && PyErr_Occurred()) { > goto bail; > } > - if (revnum < 0 || revnum >= idxlen) { > + if (revnum < -1 || revnum >= idxlen) { s/-1/nullrev/ in flight. As a follow up, please add tests where nullrev is involved. We have bad history of buffer overflow in C. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 4 V2] sparse-revlog: handle nullrev in index_get_start
On Sat, 15 Dec 2018 15:10:54 +, Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1544804621 -3600 > # Fri Dec 14 17:23:41 2018 +0100 > # Node ID 216e6d5c773cd51c18e351b11a8105165d2ad2d7 > # Parent c1e47daaab82e7d9340b0bd179d022fdd21062fc > # EXP-Topic sparse-followup > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > 216e6d5c773c > sparse-revlog: handle nullrev in index_get_start > > The more generic index_get method handle nullrev fine, we apply the same logic > here. > > diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c > --- a/mercurial/cext/revlog.c > +++ b/mercurial/cext/revlog.c > @@ -190,6 +190,9 @@ static inline int index_get_parents(inde > > static inline int64_t index_get_start(indexObject *self, Py_ssize_t rev) > { > + if (rev == nullrev) { > + return 0; > + } > uint64_t offset; Moved the declaration to top. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 4] py3: ensure the proxied Windows fd doesn't escape by entering context manager
On Sat, 15 Dec 2018 15:04:24 -0500, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison > # Date 1544855178 18000 > # Sat Dec 15 01:26:18 2018 -0500 > # Node ID 068910232e124a50a13fd7444844d9151db48d6b > # Parent 5817c3b186a7799ecc3130493ba134b55cd4ba07 > py3: ensure the proxied Windows fd doesn't escape by entering context manager Good catch. Queued these for stable because some of them will break things if "with" statement is involved, and I can't be sure we have no such invocation. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 4] windows: ensure pure posixfile fd doesn't escape by entering context manager
# HG changeset patch # User Matt Harbison # Date 1544903706 18000 # Sat Dec 15 14:55:06 2018 -0500 # Node ID 46761e235265dc2cfb4e8f54c5f7c6f5b7d58906 # Parent 847613113d3ac24106fcd3ca727428b6e02022ae windows: ensure pure posixfile fd doesn't escape by entering context manager There are tests in test-revlog-mmapindex.t and test-rebase-mq-skip.t that are fixed by this, but we usually don't use --pure on Windows. For whatever reason, the remaining --pure failures are various errors like $ENOTDIR$ and "Access is denied" have a trailing '.'. diff --git a/mercurial/pure/osutil.py b/mercurial/pure/osutil.py --- a/mercurial/pure/osutil.py +++ b/mercurial/pure/osutil.py @@ -267,7 +267,8 @@ else: return self._file.__setattr__(name, value) def __enter__(self): -return self._file.__enter__() +self._file.__enter__() +return self def __exit__(self, exc_type, exc_value, exc_tb): return self._file.__exit__(exc_type, exc_value, exc_tb) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 4] vfs: ensure closewrapbase fh doesn't escape by entering context manager
# HG changeset patch # User Matt Harbison # Date 1544900077 18000 # Sat Dec 15 13:54:37 2018 -0500 # Node ID 847613113d3ac24106fcd3ca727428b6e02022ae # Parent d40ef8f0cd2cce5b41e48924fc2a3d486b4c534d vfs: ensure closewrapbase fh doesn't escape by entering context manager I'm not sure if there's a problem in practice here, as there's no test failure either way. The __exit__() and close() methods raise an exception, so maybe __exit__() and close() are being called directly on the underlying handle when delayclosedfile is used on a context manager? I doubt that was intended. diff --git a/mercurial/vfs.py b/mercurial/vfs.py --- a/mercurial/vfs.py +++ b/mercurial/vfs.py @@ -525,7 +525,8 @@ class closewrapbase(object): return delattr(self._origfh, attr) def __enter__(self): -return self._origfh.__enter__() +self._origfh.__enter__() +return self def __exit__(self, exc_type, exc_value, exc_tb): raise NotImplementedError('attempted instantiating ' + str(type(self))) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 4] windows: ensure mixedfilemodewrapper fd doesn't escape by entering context mgr
# HG changeset patch # User Matt Harbison # Date 1544899294 18000 # Sat Dec 15 13:41:34 2018 -0500 # Node ID d40ef8f0cd2cce5b41e48924fc2a3d486b4c534d # Parent 068910232e124a50a13fd7444844d9151db48d6b windows: ensure mixedfilemodewrapper fd doesn't escape by entering context mgr Otherwise it seems that the special read and write handling would be bypassed. diff --git a/mercurial/windows.py b/mercurial/windows.py --- a/mercurial/windows.py +++ b/mercurial/windows.py @@ -70,7 +70,8 @@ class mixedfilemodewrapper(object): object.__setattr__(self, r'_lastop', 0) def __enter__(self): -return self._fp.__enter__() +self._fp.__enter__() +return self def __exit__(self, exc_type, exc_val, exc_tb): self._fp.__exit__(exc_type, exc_val, exc_tb) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 4] py3: ensure the proxied Windows fd doesn't escape by entering context manager
# HG changeset patch # User Matt Harbison # Date 1544855178 18000 # Sat Dec 15 01:26:18 2018 -0500 # Node ID 068910232e124a50a13fd7444844d9151db48d6b # Parent 5817c3b186a7799ecc3130493ba134b55cd4ba07 py3: ensure the proxied Windows fd doesn't escape by entering context manager The purpose of the proxy class is to provide the `name` attribute which contains the file path. But in tests that used a context manager, it still blew up complaining that 'int' doesn't have a 'startswith' function. diff --git a/mercurial/windows.py b/mercurial/windows.py --- a/mercurial/windows.py +++ b/mercurial/windows.py @@ -132,7 +132,10 @@ class fdproxy(object): self._fp = fp def __enter__(self): -return self._fp.__enter__() +self._fp.__enter__() +# Return this wrapper for the context manager so that the name is +# still available. +return self def __exit__(self, exc_type, exc_value, traceback): self._fp.__exit__(exc_type, exc_value, traceback) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
martinvonz added a comment. Arguments against yes/no: 1. "default: yes/no" can easily be read as "is this a default", which is not quite the same, although it would have the same effect. 2. On/off fits well in sentences like "is --hidden on by default". I see your arguments too (mostly the last one of them), but I still slightly prefer on/off. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5444 To: av6, durin42, #hg-reviewers Cc: martinvonz, pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
av6 added a comment. The point of https://phab.mercurial-scm.org/D5430 was to make hg help output more human friendly by replacing `True` and `False` with something less Python-related. In other words, more human-friendly. "Yes" and "no" are way more human-friendly: - they are one of the most used words, so people naturally expect to see them, and can tell them apart from just "y" or "n" (they are also less ambiguous themselves — just look up how many meanings "on" or "off" has and how many parts of speech can they be), - they also work perfectly fine as answers to the flag descriptions (Consider hidden changesets? Yes, please. No, thanks.), - and "no" (more precisely, "no-") is the very same thing that users should add to the flags to negate them (not "--off-hidden"). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5444 To: av6, durin42, #hg-reviewers Cc: pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
pulkit added a comment. TBH, I think 'on' and 'off' are more appropriate than 'yes' and 'no'. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5444 To: av6, durin42, #hg-reviewers Cc: pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 4 V2] sparse-revlog: handle nullrev in index_get_length
# HG changeset patch # User Boris Feld # Date 1544804684 -3600 # Fri Dec 14 17:24:44 2018 +0100 # Node ID 36c68746763d6b93b00c19387e79bd9cb623da72 # Parent 216e6d5c773cd51c18e351b11a8105165d2ad2d7 # EXP-Topic sparse-followup # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 36c68746763d sparse-revlog: handle nullrev in index_get_length The more generic index_get method handle nullrev fine, we apply the same logic here. diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c --- a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -227,6 +227,9 @@ static inline int64_t index_get_start(in static inline int index_get_length(indexObject *self, Py_ssize_t rev) { + if (rev == nullrev) { + return 0; + } if (rev >= self->length) { PyObject *tuple; PyObject *pylong; ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 4 V2] sparse-revlog: protect C code against delta chain including nullrev
# HG changeset patch # User Boris Feld # Date 1544804741 -3600 # Fri Dec 14 17:25:41 2018 +0100 # Node ID e7d33dc28696a1b2ee951cb82fe2cd611037afc8 # Parent 36c68746763d6b93b00c19387e79bd9cb623da72 # EXP-Topic sparse-followup # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r e7d33dc28696 sparse-revlog: protect C code against delta chain including nullrev For unclear reasons, some repositories include nullrev (-1). Re-computing delta for such repo remove nullrev from all chain, so some older versions have been creating them. This currently raise an IndexError with the new C code doing chain slicing as it expect all item to be positive. Both python and C code for reading delta chain preserve nullrev, and the Python code for chain slicing handle the case fine. So we take the safe route and make the new C code works fine in that case. diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c --- a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -1195,7 +1195,7 @@ static PyObject *index_slicechunktodensi if (revnum == -1 && PyErr_Occurred()) { goto bail; } - if (revnum < 0 || revnum >= idxlen) { + if (revnum < -1 || revnum >= idxlen) { PyErr_Format(PyExc_IndexError, "index out of range: %zd", revnum); goto bail; ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 4 V2] revlog: introduce a constant for nullrev in `revlog.c`
# HG changeset patch # User Boris Feld # Date 1544804562 -3600 # Fri Dec 14 17:22:42 2018 +0100 # Node ID c1e47daaab82e7d9340b0bd179d022fdd21062fc # Parent 2f14d1bbc9a7a142b421285c0708320b46be7a56 # EXP-Topic sparse-followup # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r c1e47daaab82 revlog: introduce a constant for nullrev in `revlog.c` The value is important enough to be explicitly tracked. diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c --- a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -97,6 +97,7 @@ static Py_ssize_t index_length(const ind static PyObject *nullentry = NULL; static const char nullid[20] = {0}; +static const Py_ssize_t nullrev = -1; static Py_ssize_t inline_scan(indexObject *self, const char **offsets); @@ -274,7 +275,7 @@ static PyObject *index_get(indexObject * Py_ssize_t length = index_length(self); PyObject *entry; - if (pos == -1) { + if (pos == nullrev) { Py_INCREF(nullentry); return nullentry; } @@ -344,7 +345,7 @@ static const char *index_node(indexObjec Py_ssize_t length = index_length(self); const char *data; - if (pos == -1) + if (pos == nullrev) return nullid; if (pos >= length) @@ -667,7 +668,7 @@ static PyObject *reachableroots2(indexOb } /* Add its parents to the list of nodes to visit */ - if (revnum == -1) + if (revnum == nullrev) continue; r = index_get_parents(self, revnum, parents, (int)len - 1); if (r < 0) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 4 V2] sparse-revlog: handle nullrev in index_get_start
# HG changeset patch # User Boris Feld # Date 1544804621 -3600 # Fri Dec 14 17:23:41 2018 +0100 # Node ID 216e6d5c773cd51c18e351b11a8105165d2ad2d7 # Parent c1e47daaab82e7d9340b0bd179d022fdd21062fc # EXP-Topic sparse-followup # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 216e6d5c773c sparse-revlog: handle nullrev in index_get_start The more generic index_get method handle nullrev fine, we apply the same logic here. diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c --- a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -190,6 +190,9 @@ static inline int index_get_parents(inde static inline int64_t index_get_start(indexObject *self, Py_ssize_t rev) { + if (rev == nullrev) { + return 0; + } uint64_t offset; if (rev >= self->length) { PyObject *tuple; ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5441: rust-cpython: binding for LazyAncestors
kevincox accepted this revision. kevincox added inline comments. INLINE COMMENTS > ancestors.rs:98 > +// TODO borrow() can panic, replace with try_borrow() > +// or consider it's ok thanks to the GIL > +AncestorsIterator::from_inner(py, It should be fine with tbe GIL. RefCell is basically a single-thread lock. In this case it should be fine to have no renterance. > test-rust-ancestor.py:93 > +self.assertTrue(bool(lazy)) > +self.assertEqual([r for r in lazy], [3, 2, 1, 0]) > +# a second time to validate that we spawn new iterators `list(lazy)` REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5441 To: gracinet, #hg-reviewers, kevincox Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5440: rust: core implementation for lazyancestors
kevincox accepted this revision. kevincox added inline comments. INLINE COMMENTS > ancestors.rs:120 > +/// This is mostly meant for iterators backing a lazy ancestors set > +pub fn empty() -> bool { > +if self.visit.len() > 0 { Can you clarify this `is_empty` to match rust convention? > ancestors.rs:121 > +pub fn empty() -> bool { > +if self.visit.len() > 0 { > +return false; `if !self.visit.is_empty()` > ancestors.rs:124 > +} > +let seen = self.seen.len(); > +if seen > 1 { I think this variable makes the code harder to read. I would just repeat the calls to `.len()`. > ancestors.rs:196 > + > +#[inline] > +pub fn contains( self, rev: Revision) -> Result { In general I wouldn't add these without careful benchmarking. In this case the compiler can trivially notice that this method is a good inlining candidate. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5440 To: gracinet, #hg-reviewers, kevincox Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5439: rust-cpython: binding for AncestorsIterator
kevincox accepted this revision. kevincox added inline comments. INLINE COMMENTS > ancestors.rs:32 > +Err(_) => 0, // unknown > +}; > +let mut initvec: Vec = Vec::with_capacity(cap); revs.len(py).unwrap_or(0) > ancestors.rs:40 > +Ok(revpy) => revpy.extract(py)?, > +}; > +initvec.push(as_pyint.value(py) as Revision); result_revpy?.extract(py)? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5439 To: gracinet, #hg-reviewers, kevincox Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5443: ancestor: uniformity of calling lazyancestors classes
gracinet added a comment. obviously, this one could be adapted for application before the rust-cpython bindings, and extended for the `incrementalmissingancestors` as well REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5443 To: gracinet, indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5438: rust-cpython: implementing Graph using C parents function
gracinet added a comment. Here, I'd be tempted to submit a `py_capsule_fn` macro to rust-cpython, but I guess it can wait. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5438 To: gracinet, #hg-reviewers Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5370: rust: core implementation of missingancestors (no bindings)
gracinet abandoned this revision. gracinet marked an inline comment as done. gracinet added a comment. This Differential has been superseded by https://phab.mercurial-scm.org/D5414 through https://phab.mercurial-scm.org/D5417 REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5370 To: gracinet, #hg-reviewers Cc: yuja, durin42, kevincox, mjpieters, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5444: help: use "yes" and "no" for boolean defaults instead of "on" and "off"
av6 created this revision. Herald added a reviewer: durin42. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5444 AFFECTED FILES mercurial/help.py tests/test-extension.t tests/test-fix.t tests/test-help.t tests/test-narrow-trackedcmd.t tests/test-shelve.t tests/test-uncommit.t CHANGE DETAILS diff --git a/tests/test-uncommit.t b/tests/test-uncommit.t --- a/tests/test-uncommit.t +++ b/tests/test-uncommit.t @@ -35,7 +35,7 @@ options ([+] can be repeated): --[no-]keep allow an empty commit after uncommiting (default: -off) +no) -I --include PATTERN [+] include names matching the given patterns -X --exclude PATTERN [+] exclude names matching the given patterns diff --git a/tests/test-shelve.t b/tests/test-shelve.t --- a/tests/test-shelve.t +++ b/tests/test-shelve.t @@ -75,7 +75,7 @@ --cleanup delete all shelved changes --date DATE shelve with the specified commit date -d --delete delete the named shelved change(s) - -e --[no-]edit invoke editor on commit messages (default: off) + -e --[no-]edit invoke editor on commit messages (default: no) -l --listlist current shelves -m --message TEXTuse text as shelve message -n --name NAME use the given name for the shelved commit diff --git a/tests/test-narrow-trackedcmd.t b/tests/test-narrow-trackedcmd.t --- a/tests/test-narrow-trackedcmd.t +++ b/tests/test-narrow-trackedcmd.t @@ -105,9 +105,9 @@ --import-rules VALUE import narrowspecs from a file --removeexclude VALUE [+] old paths to no longer exclude --[no-]clear whether to replace the existing -narrowspec (default: off) +narrowspec (default: no) --[no-]force-delete-local-changes forces deletion of local changes when -narrowing (default: off) +narrowing (default: no) -e --ssh CMD specify ssh command to use --remotecmd CMD specify hg command to run on the remote side diff --git a/tests/test-help.t b/tests/test-help.t --- a/tests/test-help.t +++ b/tests/test-help.t @@ -440,7 +440,7 @@ --profile print command execution profile --version output version information and exit -h --help display help and exit - --[no-]hidden consider hidden changesets (default: off) + --[no-]hidden consider hidden changesets (default: no) --pager TYPEwhen to paginate (boolean, always, auto, or never) (default: auto) @@ -540,7 +540,7 @@ --profile print command execution profile --version output version information and exit -h --help display help and exit - --[no-]hidden consider hidden changesets (default: off) + --[no-]hidden consider hidden changesets (default: no) --pager TYPEwhen to paginate (boolean, always, auto, or never) (default: auto) @@ -809,8 +809,8 @@ > [(b'', b'longdesc', 3, b'x'*67), > (b'n', b'', None, b'normal desc'), > (b'', b'newline', b'', b'line1\nline2'), - > (b'', b'default-off', False, b'enable X'), - > (b'', b'default-on', True, b'enable Y'), + > (b'', b'default-no', False, b'enable X'), + > (b'', b'default-yes', True, b'enable Y'), > (b'', b'callableopt', func, b'adds foo'), > (b'', b'customopt', customopt(''), b'adds bar'), > (b'', b'customopt-withdefault', customopt('foo'), b'adds bar')], @@ -905,8 +905,8 @@ xxx (default: 3) -n --normal desc --newline VALUE line1 line2 - --[no-]default-offenable X (default: off) - --[no-]default-on enable Y (default: on) + --[no-]default-no enable X (default: no) + --[no-]default-yesenable Y (default: yes) --callableopt VALUE adds foo --customopt VALUE adds bar --customopt-withdefault VALUE adds bar (default: foo) @@ -2890,7 +2890,7 @@ display help and exit --[no-]hidden - consider hidden changesets (default: off) + consider hidden changesets (default: no) --pager TYPE when to paginate (boolean, always, auto, or never) (default: auto) @@ -3094,7 +3094,7 @@ display help and exit --[no-]hidden - consider hidden changesets (default: off) + consider hidden changesets (default: no)
D5443: ancestor: uniformity of calling lazyancestors classes
gracinet created this revision. Herald added a reviewer: indygreg. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Up to now, the pure Python lazyancestors had been taking the parents function in its constructor, whereas Rust-backed variants would have needed a revlog index. With this change, instantiation of all lazyancestors work uniformely with an index. This requires only minor wrapping in test-ancestor.py. It could be argued that this introduces a new duplication, and that therefore, parentsfunc(index) should be provided in some utility module and used both from revlog and ancestor modules. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5443 AFFECTED FILES mercurial/ancestor.py mercurial/revlog.py tests/test-ancestor.py CHANGE DETAILS diff --git a/tests/test-ancestor.py b/tests/test-ancestor.py --- a/tests/test-ancestor.py +++ b/tests/test-ancestor.py @@ -240,10 +240,26 @@ else: print("Ok") +class fakeindex(object): +'''A pseudo index backed by a dict or list graph + +It's only meant for parents lookup, and +all that matters is that self.graph[rev] is the pair of rev's parents. +''' +def __init__(self, graph): +self.graph = graph +def __getitem__(self, rev): +res = [None] * 5 +try: +res.extend(self.graph[rev]) +except KeyError: +raise IndexError(rev) +return res + def genlazyancestors(revs, stoprev=0, inclusive=False): print(("%% lazy ancestor set for %s, stoprev = %s, inclusive = %s" % (revs, stoprev, inclusive))) -return ancestor.lazyancestors(graph.get, revs, stoprev=stoprev, +return ancestor.lazyancestors(fakeindex(graph), revs, stoprev=stoprev, inclusive=inclusive) def printlazyancestors(s, l): diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -675,9 +675,6 @@ return entry[5], entry[6] -# fast parentrevs(rev) where rev isn't filtered -_uncheckedparentrevs = parentrevs - def node(self, rev): try: return self.index[rev][7] @@ -797,14 +794,12 @@ rustext = None if rustext is not None: lazyancestors = rustext.ancestor.lazyancestors -arg = self.index elif util.safehasattr(parsers, 'rustlazyancestors'): lazyancestors = ancestor.rustlazyancestors -arg = self.index else: lazyancestors = ancestor.lazyancestors -arg = self._uncheckedparentrevs -return lazyancestors(arg, revs, stoprev=stoprev, inclusive=inclusive) +return lazyancestors(self.index, revs, + stoprev=stoprev, inclusive=inclusive) def descendants(self, revs): return dagop.descendantrevs(revs, self.revs, self.parentrevs) diff --git a/mercurial/ancestor.py b/mercurial/ancestor.py --- a/mercurial/ancestor.py +++ b/mercurial/ancestor.py @@ -9,8 +9,12 @@ import heapq -from .node import nullrev +from .node import ( +nullrev, +wdirrev, +) from . import ( +error, policy, pycompat, ) @@ -306,8 +310,20 @@ heappush(visit, -p2) see(p2) +def parentsfunc(index): +def parentrevs(rev): +try: +entry = index[rev] +except IndexError: +if rev == wdirrev: +raise error.WdirUnsupported +raise + +return entry[5], entry[6] +return parentrevs + class lazyancestors(object): -def __init__(self, pfunc, revs, stoprev=0, inclusive=False): +def __init__(self, index, revs, stoprev=0, inclusive=False): """Create a new object generating ancestors for the given revs. Does not generate revs lower than stoprev. @@ -319,7 +335,7 @@ than stoprev will not be generated. Result does not include the null revision.""" -self._parentrevs = pfunc +self._parentrevs = parentsfunc(index) self._initrevs = revs = [r for r in revs if r >= stoprev] self._stoprev = stoprev self._inclusive = inclusive @@ -330,6 +346,7 @@ self._stoprev, self._inclusive) + def __nonzero__(self): """False if the set is empty, True otherwise.""" try: To: gracinet, indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5442: rust-cpython: using the new bindings from Python
gracinet created this revision. Herald added a reviewer: indygreg. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The Python callers detect if we have cpython or direct-ffi bindings and fallback to the Python implementation if none is present. This intermediate state allows to compare the three possibilities. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5442 AFFECTED FILES mercurial/revlog.py CHANGE DETAILS diff --git a/mercurial/revlog.py b/mercurial/revlog.py --- a/mercurial/revlog.py +++ b/mercurial/revlog.py @@ -97,6 +97,10 @@ REVIDX_RAWTEXT_CHANGING_FLAGS parsers = policy.importmod(r'parsers') +try: +from . import rustext +except ImportError: +rustext = None # Aliased for performance. _zlibdecompress = zlib.decompress @@ -779,12 +783,28 @@ for r in revs: checkrev(r) # and we're sure ancestors aren't filtered as well -if util.safehasattr(parsers, 'rustlazyancestors'): -return ancestor.rustlazyancestors( -self.index, revs, -stoprev=stoprev, inclusive=inclusive) -return ancestor.lazyancestors(self._uncheckedparentrevs, revs, - stoprev=stoprev, inclusive=inclusive) + +# rustext can be an unloaded module (see hgdemandimport) +# so we need to trigger an actual import to check for its presence, +# which is done by looking up any attribute. +# If in the future we reach the point where the ancestor module +# can be imported by policy, we will be able to get rid of this +global rustext +if rustext is not None: +try: +rustext.__name__ +except ImportError: +rustext = None +if rustext is not None: +lazyancestors = rustext.ancestor.lazyancestors +arg = self.index +elif util.safehasattr(parsers, 'rustlazyancestors'): +lazyancestors = ancestor.rustlazyancestors +arg = self.index +else: +lazyancestors = ancestor.lazyancestors +arg = self._uncheckedparentrevs +return lazyancestors(arg, revs, stoprev=stoprev, inclusive=inclusive) def descendants(self, revs): return dagop.descendantrevs(revs, self.revs, self.parentrevs) To: gracinet, indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5440: rust: core implementation for lazyancestors
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Once exposed through appropriate bindings, this should be able to replace ancestor.lazyancestors entirely. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5440 AFFECTED FILES rust/hg-core/src/ancestors.rs rust/hg-core/src/lib.rs rust/hg-cpython/src/cindex.rs rust/hg-direct-ffi/src/ancestors.rs CHANGE DETAILS diff --git a/rust/hg-direct-ffi/src/ancestors.rs b/rust/hg-direct-ffi/src/ancestors.rs --- a/rust/hg-direct-ffi/src/ancestors.rs +++ b/rust/hg-direct-ffi/src/ancestors.rs @@ -30,6 +30,12 @@ /// This implementation of the Graph trait, relies on (pointers to) /// - the C index object (`index` member) /// - the `index_get_parents()` function (`parents` member) +/// +/// In case of `.clone()` (would be from a full binding for `LazyAncestors` +/// that doesn't exist at the time of this writing), reference increasing +/// and decreasing would be the responsibility of the caller, same as it +/// is for `AncestorsIterator` +#[derive(Clone, Debug)] pub struct Index { index: IndexPtr, } diff --git a/rust/hg-cpython/src/cindex.rs b/rust/hg-cpython/src/cindex.rs --- a/rust/hg-cpython/src/cindex.rs +++ b/rust/hg-cpython/src/cindex.rs @@ -15,7 +15,7 @@ extern crate python3_sys as python_sys; use self::python_sys::PyCapsule_Import; -use cpython::{PyErr, PyObject, PyResult, Python}; +use cpython::{PyClone, PyErr, PyObject, PyResult, Python}; use hg::{Graph, GraphError, Revision}; use libc::{c_int, ssize_t}; use std::ffi::CStr; @@ -47,6 +47,16 @@ } } +impl Clone for Index { +fn clone() -> Self { +let guard = Python::acquire_gil(); +Index { +index: self.index.clone_ref(guard.python()), +parents: self.parents.clone(), +} +} +} + impl Graph for Index { /// wrap a call to the C extern parents function fn parents(, rev: Revision) -> Result<[Revision; 2], GraphError> { diff --git a/rust/hg-core/src/lib.rs b/rust/hg-core/src/lib.rs --- a/rust/hg-core/src/lib.rs +++ b/rust/hg-core/src/lib.rs @@ -2,8 +2,10 @@ // // This software may be used and distributed according to the terms of the // GNU General Public License version 2 or any later version. +use std::clone::Clone; + mod ancestors; -pub use ancestors::{AncestorsIterator, MissingAncestors}; +pub use ancestors::{AncestorsIterator, LazyAncestors, MissingAncestors}; /// Mercurial revision numbers /// @@ -14,7 +16,7 @@ pub const NULL_REVISION: Revision = -1; /// The simplest expression of what we need of Mercurial DAGs. -pub trait Graph { +pub trait Graph: Clone { /// Return the two parents of the given `Revision`. /// /// Each of the parents can be independently `NULL_REVISION` diff --git a/rust/hg-core/src/ancestors.rs b/rust/hg-core/src/ancestors.rs --- a/rust/hg-core/src/ancestors.rs +++ b/rust/hg-core/src/ancestors.rs @@ -25,6 +25,15 @@ stoprev: Revision, } +/// Lazy ancestors set, backed by AncestorsIterator +pub struct LazyAncestors { +graph: G, +containsiter: AncestorsIterator, +initrevs: Vec, +stoprev: Revision, +inclusive: bool, +} + pub struct MissingAncestors { graph: G, bases: HashSet, @@ -98,9 +107,34 @@ } Ok(false) } + +pub fn peek() -> Option { +self.visit.peek().map(|| r) +} + +/// Tell if the iterator is about an empty set +/// +/// The result does not depend whether the iterator has been consumed +/// or not. +/// This is mostly meant for iterators backing a lazy ancestors set +pub fn empty() -> bool { +if self.visit.len() > 0 { +return false; +} +let seen = self.seen.len(); +if seen > 1 { +return false; +} +// at this point, the seen set is a singleton. If not self.inclusive, +// then its only element can be the null revision +if seen == 0 || self.seen.contains(_REVISION) { +return true; +} +false +} } -/// Main implementation. +/// Main implementation for the iterator /// /// The algorithm is the same as in `_lazyancestorsiter()` from `ancestors.py` /// with a few non crucial differences: @@ -137,6 +171,51 @@ } } +impl LazyAncestors { +pub fn new( +graph: G, +initrevs: impl IntoIterator, +stoprev: Revision, +inclusive: bool, +) -> Result { +let v: Vec = initrevs.into_iter().collect(); +Ok(LazyAncestors { +graph: graph.clone(), +containsiter: AncestorsIterator::new( +graph, +v.iter().cloned(), +stoprev, +inclusive, +)?, +initrevs: v, +stoprev: stoprev, +inclusive: inclusive, +}) +} + +#[inline] +pub fn
D5441: rust-cpython: binding for LazyAncestors
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY For consistency with Python implementation, we're exposing it with the lower case spelling, so that if one day the whole ancestor module has a Rust implementation, it would be readily compatible, ready for HGMODULEPOLICY. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5441 AFFECTED FILES rust/hg-cpython/src/ancestors.rs tests/test-rust-ancestor.py CHANGE DETAILS diff --git a/tests/test-rust-ancestor.py b/tests/test-rust-ancestor.py --- a/tests/test-rust-ancestor.py +++ b/tests/test-rust-ancestor.py @@ -9,7 +9,10 @@ rustext = None else: # this would fail already without appropriate ancestor.__package__ -from mercurial.rustext.ancestor import AncestorsIterator +from mercurial.rustext.ancestor import ( +AncestorsIterator, +lazyancestors +) try: from mercurial.cext import parsers as cparsers @@ -71,6 +74,37 @@ ait = AncestorsIterator(idx, [3], 0, False) self.assertEqual([r for r in ait], [2, 1, 0]) +def testlazyancestors(self): +idx = self.parseindex() +start_count = sys.getrefcount(idx) # should be 2 (see Python doc) +self.assertEqual({i: (r[5], r[6]) for i, r in enumerate(idx)}, + {0: (-1, -1), + 1: (0, -1), + 2: (1, -1), + 3: (2, -1)}) +lazy = lazyancestors(idx, [3], 0, True) +# we have two more references to the index: +# - in its inner iterator for __contains__ and __bool__ +# - in the lazyancestors instance itself (to spawn new iterators) +self.assertEqual(sys.getrefcount(idx), start_count + 2) + +self.assertTrue(2 in lazy) +self.assertTrue(bool(lazy)) +self.assertEqual([r for r in lazy], [3, 2, 1, 0]) +# a second time to validate that we spawn new iterators +self.assertEqual([r for r in lazy], [3, 2, 1, 0]) + +# now let's watch the refcounts closer +ait = iter(lazy) +self.assertEqual(sys.getrefcount(idx), start_count + 3) +del ait +self.assertEqual(sys.getrefcount(idx), start_count + 2) +del lazy +self.assertEqual(sys.getrefcount(idx), start_count) + +# let's check bool for an empty one +self.assertFalse(lazyancestors(idx, [0], 0, False)) + def testrefcount(self): idx = self.parseindex() start_count = sys.getrefcount(idx) diff --git a/rust/hg-cpython/src/ancestors.rs b/rust/hg-cpython/src/ancestors.rs --- a/rust/hg-cpython/src/ancestors.rs +++ b/rust/hg-cpython/src/ancestors.rs @@ -13,8 +13,8 @@ }; use exceptions::GraphError; use hg; -use hg::AncestorsIterator as CoreIterator; use hg::Revision; +use hg::{AncestorsIterator as CoreIterator, LazyAncestors as CoreLazy}; use std::cell::RefCell; /// Utility function to convert a Python iterable into a Vec @@ -85,6 +85,43 @@ } } +py_class!(class lazyancestors |py| { +// TODO RW lock ? +data inner: RefCell>>; + +def __contains__(, rev: Revision) -> PyResult { +self.inner(py).borrow_mut().contains(rev).map_err(|e| GraphError::pynew(py, e)) +} + +def __iter__() -> PyResult { +// TODO borrow() can panic, replace with try_borrow() +// or consider it's ok thanks to the GIL +AncestorsIterator::from_inner(py, + self.inner(py).borrow().iter()) +} + +def __bool__() -> PyResult { +// TODO borrow() panic, see __iter__() +Ok(!self.inner(py).borrow().is_empty()) +} + +def __new__(_cls, index: PyObject, initrevs: PyObject, stoprev: Revision, +inclusive: bool) -> PyResult { +let initvec = reviter_to_revvec(py, initrevs)?; + +let lazy = match CoreLazy::new( +Index::new(py, index)?, initvec, stoprev, inclusive) { +Ok(lazy) => lazy, +Err(e) => { +return Err(GraphError::new(py, format!("{:?}", e))); +} +}; + +Self::create_instance(py, RefCell::new(Box::new(lazy))) +} + +}); + /// Create the module, with __package__ given from parent pub fn init_module(py: Python, package: ) -> PyResult { let dotted_name = !("{}.ancestor", package); @@ -96,6 +133,7 @@ "Generic DAG ancestor algorithms - Rust implementation", )?; m.add_class::(py)?; +m.add_class::(py)?; let sys = PyModule::import(py, "sys")?; let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; To: gracinet, #hg-reviewers Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5438: rust-cpython: implementing Graph using C parents function
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY A pointer to the parents function is stored on the parsers C extension module as a capsule object. This is the recommended way to export a C API for consumption from other extensions. See also: https://docs.python.org/2.7/c-api/capsule.html In our case, we use it in cindex.rs, retrieving function pointer from the capsule and storing it within the CIndex struct, alongside with a pointer to the index. From there, the implementation is very close to the one from hg-direct-ffi. The naming convention for the capsule is inspired from the one in datetime: >>> import datetime >>> datetime.datetime_CAPI REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5438 AFFECTED FILES mercurial/cext/revlog.c rust/hg-cpython/src/cindex.rs rust/hg-cpython/src/lib.rs CHANGE DETAILS diff --git a/rust/hg-cpython/src/lib.rs b/rust/hg-cpython/src/lib.rs --- a/rust/hg-cpython/src/lib.rs +++ b/rust/hg-cpython/src/lib.rs @@ -21,6 +21,7 @@ #[macro_use] extern crate cpython; extern crate hg; +extern crate libc; mod ancestors; mod exceptions; diff --git a/rust/hg-cpython/src/cindex.rs b/rust/hg-cpython/src/cindex.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/cindex.rs @@ -0,0 +1,95 @@ +// cindex.rs +// +// Copyright 2018 Georges Racinet +// +// This software may be used and distributed according to the terms of the +// GNU General Public License version 2 or any later version. + +//! Bindings to use the Index defined by the parsers C extension +//! +//! Ideally, we should use an Index entirely implemented in Rust, +//! but this will take some time to get there. +#[cfg(feature = "python27")] +extern crate python27_sys as python_sys; +#[cfg(feature = "python3")] +extern crate python3_sys as python_sys; + +use self::python_sys::PyCapsule_Import; +use cpython::{PyErr, PyObject, PyResult, Python}; +use hg::{Graph, GraphError, Revision}; +use libc::{c_int, ssize_t}; +use std::ffi::CStr; +use std::mem::transmute; + +type IndexParentsFn = unsafe extern "C" fn( +index: *mut python_sys::PyObject, +rev: ssize_t, +ps: *mut [c_int; 2], +max_rev: c_int, +) -> c_int; + +/// A Graph backed up by objects and functions from revlog.c +/// +/// This implementation of the Graph trait, relies on (pointers to) +/// - the C index object (`index` member) +/// - the `index_get_parents()` function (`parents` member) +pub struct Index { +index: PyObject, +parents: IndexParentsFn, +} + +impl Index { +pub fn new(py: Python, index: PyObject) -> PyResult { +Ok(Index { +index: index, +parents: decapsule_parents_fn(py)?, +}) +} +} + +impl Graph for Index { +/// wrap a call to the C extern parents function +fn parents(, rev: Revision) -> Result<[Revision; 2], GraphError> { +let mut res: [c_int; 2] = [0; 2]; +let code = unsafe { +(self.parents)( +self.index.as_ptr(), +rev as ssize_t, + res as *mut [c_int; 2], +rev, +) +}; +match code { +0 => Ok(res), +_ => Err(GraphError::ParentOutOfRange(rev)), +} +} +} + +/// Return the `index_get_parents` function of the parsers C Extension module. +/// +/// A pointer to the function is stored in the `parsers` module as a +/// standard [Python capsule](https://docs.python.org/2/c-api/capsule.html). +/// +/// This function retrieves the capsule and casts the function pointer +/// +/// Casting function pointers is one of the rare cases of +/// legitimate use cases of `mem::transmute()` (see +/// https://doc.rust-lang.org/std/mem/fn.transmute.html of +/// `mem::transmute()`. +/// It is inappropriate for architectures where +/// function and data pointer sizes differ (so-called "Harvard +/// architectures"), but these are nowadays mostly DSPs +/// and microcontrollers, hence out of our scope. +fn decapsule_parents_fn(py: Python) -> PyResult { +unsafe { +let caps_name = CStr::from_bytes_with_nul_unchecked( +b"mercurial.cext.parsers.index_get_parents_CAPI\0", +); +let from_caps = PyCapsule_Import(caps_name.as_ptr(), 0); +if from_caps.is_null() { +return Err(PyErr::fetch(py)); +} +Ok(transmute(from_caps)) +} +} diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c --- a/mercurial/cext/revlog.c +++ b/mercurial/cext/revlog.c @@ -2878,6 +2878,12 @@ if (nullentry) PyObject_GC_UnTrack(nullentry); + void *caps = PyCapsule_New( + HgRevlogIndex_GetParents, + "mercurial.cext.parsers.index_get_parents_CAPI", NULL); + if (caps != NULL) + PyModule_AddObject(mod, "index_get_parents_CAPI", caps); + #ifdef
D5434: rust-cpython: started cpython crate bindings
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This changeset introduces the hg-cpython crate, that compiles as a shared library holding a whole Python package (mercurial.rustext), with only the empty 'ancestor' submodule for now. Such bindings will be easier and safer to develop and maintain that those of `hg-direct-ffi`. They don't involve C code, only unsafe Rust that's mostly isolated within the cpython crate. The long-term goal would be to import the provided modules, such as rustext.ancestor with mercurial.policy.importmod, same as we already do with cext modules. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5434 AFFECTED FILES rust/Cargo.lock rust/Cargo.toml rust/hg-cpython/Cargo.toml rust/hg-cpython/rustfmt.toml rust/hg-cpython/src/ancestors.rs rust/hg-cpython/src/exceptions.rs rust/hg-cpython/src/lib.rs CHANGE DETAILS diff --git a/rust/hg-cpython/src/lib.rs b/rust/hg-cpython/src/lib.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/lib.rs @@ -0,0 +1,40 @@ +// lib.rs +// +// Copyright 2018 Georges Racinet +// +// This software may be used and distributed according to the terms of the +// GNU General Public License version 2 or any later version. + +//! Python bindings of `hg-core` objects using the `cpython` crate. +//! Once compiled, the resulting single shared library object can be placed in +//! the `mercurial` package directly as `rustext.so` or `rustext.dll`. +//! It holds several modules, so that from the point of view of Python, +//! it behaves as the `cext` package. +//! +//! Example: +//! ``` +//! >>> from mercurial.rustext import ancestor +//! >>> ancestor.__doc__ +//! 'Generic DAG ancestor algorithms - Rust implementation' +//! ``` + +#[macro_use] +extern crate cpython; +extern crate hg; + +mod ancestors; +mod exceptions; + +py_module_initializer!(rustext, initrustext, PyInit_rustext, |py, m| { +m.add( +py, +"__doc__", +"Mercurial core concepts - Rust implementation", +)?; + +let dotted_name: String = m.get(py, "__name__")?.extract(py)?; +m.add(py, "__package__", "mercurial")?; +m.add(py, "ancestor", ancestors::init_module(py, _name)?)?; +m.add(py, "GraphError", py.get_type::())?; +Ok(()) +}); diff --git a/rust/hg-cpython/src/exceptions.rs b/rust/hg-cpython/src/exceptions.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/exceptions.rs @@ -0,0 +1,15 @@ +use cpython::exc::ValueError; +use cpython::{PyErr, Python}; +use hg; + +py_exception!(rustext, GraphError, ValueError); + +impl GraphError { +pub fn pynew(py: Python, inner: hg::GraphError) -> PyErr { +match inner { +hg::GraphError::ParentOutOfRange(r) => { +GraphError::new(py, ("ParentOutOfRange", r)) +} +} +} +} diff --git a/rust/hg-cpython/src/ancestors.rs b/rust/hg-cpython/src/ancestors.rs new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/src/ancestors.rs @@ -0,0 +1,30 @@ +// ancestors.rs +// +// Copyright 2018 Georges Racinet +// +// This software may be used and distributed according to the terms of the +// GNU General Public License version 2 or any later version. + +//! Bindings for the hg::ancestors module provided by the +//! `hg-core` crate. From Python, this will be seen as `rustext.ancestor` +use cpython::{PyDict, PyModule, PyResult, Python}; + +/// Create the module, with __package__ given from parent +pub fn init_module(py: Python, package: ) -> PyResult { +let dotted_name = !("{}.ancestor", package); +let m = PyModule::new(py, dotted_name)?; +m.add(py, "__package__", package)?; +m.add( +py, +"__doc__", +"Generic DAG ancestor algorithms - Rust implementation", +)?; + +let sys = PyModule::import(py, "sys")?; +let sys_modules: PyDict = sys.get(py, "modules")?.extract(py)?; +sys_modules.set_item(py, dotted_name, )?; +// Example C code (see pyexpat.c and import.c) will "give away the +// reference", but we won't because it will be consumed once the +// Rust PyObject is dropped. +Ok(m) +} diff --git a/rust/hg-cpython/rustfmt.toml b/rust/hg-cpython/rustfmt.toml new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/rustfmt.toml @@ -0,0 +1,3 @@ +max_width = 79 +wrap_comments = true +error_on_line_overflow = true diff --git a/rust/hg-cpython/Cargo.toml b/rust/hg-cpython/Cargo.toml new file mode 100644 --- /dev/null +++ b/rust/hg-cpython/Cargo.toml @@ -0,0 +1,31 @@ +[package] +name = "hg-cpython" +version = "0.1.0" +authors = ["Georges Racinet "] + +[lib] +name='rusthg' +crate-type = ["cdylib"] + +[features] +default = ["python27", "python27-sys"] + +python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"] + +[dependencies] +hg-core = { path = "../hg-core" } +libc = '*' + +[dependencies.cpython] +version = "*"
D5437: rust-cpython: testing the bindings from Python
gracinet created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This is easier and more convincing than doing the same tests from a Rust tests module. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5437 AFFECTED FILES tests/test-rust-ancestor.py CHANGE DETAILS diff --git a/tests/test-rust-ancestor.py b/tests/test-rust-ancestor.py new file mode 100644 --- /dev/null +++ b/tests/test-rust-ancestor.py @@ -0,0 +1,39 @@ +from __future__ import absolute_import +import unittest + +try: +from mercurial import rustext +except ImportError: +rustext = None + +try: +from mercurial.cext import parsers as cparsers +except ImportError: +cparsers = None + +@unittest.skipIf(rustext is None or cparsers is None, + "rustext.ancestor or the C Extension parsers module " + "it relies on is not available") +class rustancestorstest(unittest.TestCase): +"""Test the correctness of binding to Rust code. + +This test is merely for the binding to Rust itself: extraction of +Python variable, giving back the results etc. + +It is not meant to test the algorithmic correctness of the operations +on ancestors it provides. Hence the very simple embedded index data is +good enough. + +Algorithmic correctness is asserted by the Rust unit tests. +""" + +def testmodule(self): +self.assertTrue('DAG' in rustext.ancestor.__doc__) + +def testgrapherror(self): +self.assertTrue('GraphError' in dir(rustext)) + + +if __name__ == '__main__': +import silenttestrunner +silenttestrunner.main(__name__) To: gracinet, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5439: rust-cpython: binding for AncestorsIterator
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY It's now reachable from Python as rustext.ancestor.AncestorsIterator Tests are provided in the previously introduced Python testcase: this is much more convenient that writing lengthy Rust code to call into Python. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5439 AFFECTED FILES rust/hg-cpython/src/ancestors.rs rust/hg-cpython/src/lib.rs tests/test-rust-ancestor.py CHANGE DETAILS diff --git a/tests/test-rust-ancestor.py b/tests/test-rust-ancestor.py --- a/tests/test-rust-ancestor.py +++ b/tests/test-rust-ancestor.py @@ -1,19 +1,46 @@ from __future__ import absolute_import +import sys import unittest try: from mercurial import rustext +rustext.__name__ # trigger immediate actual import except ImportError: rustext = None +else: +# this would fail already without appropriate ancestor.__package__ +from mercurial.rustext.ancestor import AncestorsIterator try: from mercurial.cext import parsers as cparsers except ImportError: cparsers = None +# picked from test-parse-index2, copied rather than imported +# so that it stays stable even if test-parse-index2 changes or disappears. +data_non_inlined = ( +b'\x00\x00\x00\x01\x00\x00\x00\x00\x00\x01D\x19' +b'\x00\x07e\x12\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff' +b'\xff\xff\xff\xff\xd1\xf4\xbb\xb0\xbe\xfc\x13\xbd\x8c\xd3\x9d' +b'\x0f\xcd\xd9;\x8c\x07\x8cJ/\x00\x00\x00\x00\x00\x00\x00\x00\x00' +b'\x00\x00\x00\x00\x00\x00\x01D\x19\x00\x00\x00\x00\x00\xdf\x00' +b'\x00\x01q\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff' +b'\xff\xff\xff\xc1\x12\xb9\x04\x96\xa4Z1t\x91\xdfsJ\x90\xf0\x9bh' +b'\x07l&\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' +b'\x00\x01D\xf8\x00\x00\x00\x00\x01\x1b\x00\x00\x01\xb8\x00\x00' +b'\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x02\n' +b'\x0e\xc6&\xa1\x92\xae6\x0b\x02i\xfe-\xe5\xbao\x05\xd1\xe7\x00' +b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01F' +b'\x13\x00\x00\x00\x00\x01\xec\x00\x00\x03\x06\x00\x00\x00\x01' +b'\x00\x00\x00\x03\x00\x00\x00\x02\xff\xff\xff\xff\x12\xcb\xeby1' +b'\xb6\r\x98B\xcb\x07\xbd`\x8f\x92\xd9\xc4\x84\xbdK\x00\x00\x00' +b'\x00\x00\x00\x00\x00\x00\x00\x00\x00' +) + + @unittest.skipIf(rustext is None or cparsers is None, - "rustext.ancestor or the C Extension parsers module " - "it relies on is not available") + "rustext or the C Extension parsers module " + "ancestor relies on is not available") class rustancestorstest(unittest.TestCase): """Test the correctness of binding to Rust code. @@ -27,12 +54,51 @@ Algorithmic correctness is asserted by the Rust unit tests. """ -def testmodule(self): -self.assertTrue('DAG' in rustext.ancestor.__doc__) +def parseindex(self): +return cparsers.parse_index2(data_non_inlined, False)[0] + +def testiteratorrevlist(self): +idx = self.parseindex() +# checking test assumption about the index binary data: +self.assertEqual({i: (r[5], r[6]) for i, r in enumerate(idx)}, + {0: (-1, -1), + 1: (0, -1), + 2: (1, -1), + 3: (2, -1)}) +ait = AncestorsIterator(idx, [3], 0, True) +self.assertEqual([r for r in ait], [3, 2, 1, 0]) + +ait = AncestorsIterator(idx, [3], 0, False) +self.assertEqual([r for r in ait], [2, 1, 0]) + +def testrefcount(self): +idx = self.parseindex() +start_count = sys.getrefcount(idx) + +# refcount increases upon iterator init... +ait = AncestorsIterator(idx, [3], 0, True) +self.assertEqual(sys.getrefcount(idx), start_count + 1) +self.assertEqual(next(ait), 3) + +# and decreases once the iterator is removed +del ait +self.assertEqual(sys.getrefcount(idx), start_count) + +# and removing ref to the index after iterator init is no issue +ait = AncestorsIterator(idx, [3], 0, True) +del idx +self.assertEqual([r for r in ait], [3, 2, 1, 0]) def testgrapherror(self): -self.assertTrue('GraphError' in dir(rustext)) - +data = (data_non_inlined[:64 + 27] + +b'\xf2' + +data_non_inlined[64 + 28:]) +idx = cparsers.parse_index2(data, False)[0] +with self.assertRaises(rustext.GraphError) as arc: +AncestorsIterator(idx, [1], -1, False) +exc = arc.exception +self.assertIsInstance(exc, ValueError) +self.assertEqual(exc.args, (b'ParentOutOfRange', 1)) if __name__ == '__main__': import silenttestrunner diff --git a/rust/hg-cpython/src/lib.rs
D5436: rust-cpython: build via HGWITHRUSTEXT=cpython
gracinet created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The existing behaviour, building the direct ffi bindings if HGIWTHRUSTEXT is just set is unchanged, but if HGWITHRUSTEXT is cpython, then the cpython bindings (aka mercurial/rustext.so) are built. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5436 AFFECTED FILES Makefile setup.py CHANGE DETAILS diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -132,7 +132,11 @@ ispypy = "PyPy" in sys.version -iswithrustextensions = 'HGWITHRUSTEXT' in os.environ +hgrustext = os.environ.get('HGWITHRUSTEXT') +# TODO record it for proper rebuild upon changes +# (see mercurial/__modulepolicy__.py) +if hgrustext != 'cpython' and hgrustext is not None: +hgrustext = 'direct-ffi' import ctypes import stat, subprocess, time @@ -457,11 +461,18 @@ return build_ext.initialize_options(self) def build_extensions(self): +ruststandalones = [e for e in self.extensions + if isinstance(e, RustStandaloneExtension)] +self.extensions = [e for e in self.extensions + if e not in ruststandalones] # Filter out zstd if disabled via argument. if not self.zstd: self.extensions = [e for e in self.extensions if e.name != 'mercurial.zstd'] +for rustext in ruststandalones: +rustext.build('' if self.inplace else self.build_lib) + return build_ext.build_extensions(self) def build_extension(self, ext): @@ -902,20 +913,16 @@ """Exception class for Rust compilation errors.""" class RustExtension(Extension): -"""A C Extension, conditionnally enhanced with Rust code. - -if iswithrustextensions is False, does nothing else than plain Extension +"""Base classes for concrete Rust Extension classes. """ rusttargetdir = os.path.join('rust', 'target', 'release') def __init__(self, mpath, sources, rustlibname, subcrate, **kw): Extension.__init__(self, mpath, sources, **kw) -if not iswithrustextensions: +if hgrustext is None: return srcdir = self.rustsrcdir = os.path.join('rust', subcrate) -self.libraries.append(rustlibname) -self.extra_compile_args.append('-DWITH_RUST') # adding Rust source and control files to depends so that the extension # gets rebuilt if they've changed @@ -929,7 +936,7 @@ if os.path.splitext(fname)[1] == '.rs') def rustbuild(self): -if not iswithrustextensions: +if hgrustext is None: return env = os.environ.copy() if 'HGTEST_RESTOREENV' in env: @@ -961,6 +968,40 @@ "Cargo failed. Working directory: %r, " "command: %r, environment: %r" % (self.rustsrcdir, cmd, env)) +class RustEnhancedExtension(RustExtension): +"""A C Extension, conditionally enhanced with Rust code. + +If the HGRUSTEXT environment variable is set to something else +than 'cpython', the Rust sources get compiled and linked within the +C target shared library object. +""" + +def __init__(self, mpath, sources, rustlibname, subcrate, **kw): +RustExtension.__init__(self, mpath, sources, rustlibname, subcrate, + **kw) +if hgrustext != 'direct-ffi': +return +self.extra_compile_args.append('-DWITH_RUST') +self.libraries.append(rustlibname) +self.library_dirs.append(self.rusttargetdir) + +class RustStandaloneExtension(RustExtension): + +def __init__(self, pydottedname, rustcrate, dylibname, **kw): +RustExtension.__init__(self, pydottedname, [], dylibname, rustcrate, + **kw) +self.dylibname = dylibname + +def build(self, target_dir): +self.rustbuild() +target = [target_dir] +target.extend(self.name.split('.')) +ext = '.so' # TODO Unix only +target[-1] += ext +shutil.copy2(os.path.join(self.rusttargetdir, self.dylibname + ext), + os.path.join(*target)) + + extmodules = [ Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'], include_dirs=common_include_dirs, @@ -973,19 +1014,20 @@ 'mercurial/cext/mpatch.c'], include_dirs=common_include_dirs, depends=common_depends), -RustExtension('mercurial.cext.parsers', ['mercurial/cext/charencode.c', - 'mercurial/cext/dirs.c', - 'mercurial/cext/manifest.c', - 'mercurial/cext/parsers.c', - 'mercurial/cext/pathencode.c', -
D5435: rust: better treatment of cargo/rustc errors
gracinet created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5435 AFFECTED FILES setup.py CHANGE DETAILS diff --git a/setup.py b/setup.py --- a/setup.py +++ b/setup.py @@ -898,6 +898,9 @@ 'mercurial/thirdparty/xdiff/xutils.h', ] +class RustCompilationError(CCompilerError): +"""Exception class for Rust compilation errors.""" + class RustExtension(Extension): """A C Extension, conditionnally enhanced with Rust code. @@ -942,9 +945,21 @@ import pwd env['HOME'] = pwd.getpwuid(os.getuid()).pw_dir -subprocess.check_call(['cargo', 'build', '-vv', '--release'], - env=env, cwd=self.rustsrcdir) -self.library_dirs.append(self.rusttargetdir) +cargocmd = ['cargo', 'build', '-vv', '--release'] +try: +subprocess.check_call(cargocmd, env=env, cwd=self.rustsrcdir) +except OSError as exc: +if exc.errno == os.errno.ENOENT: +raise RustCompilationError("Cargo not found") +elif exc.errno == os.errno.EACCES: +raise RustCompilationError( +"Cargo found, but permisssion to execute it is denied") +else: +raise +except subprocess.CalledProcessError: +raise RustCompilationError( +"Cargo failed. Working directory: %r, " +"command: %r, environment: %r" % (self.rustsrcdir, cmd, env)) extmodules = [ Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'], To: gracinet, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5433: rust-cpython: excluded hgcli from workspace
gracinet created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY hgcli uses a specific rust-cpython commit by indygreg, of which a PR has been derived which is not merged nor released yet. But we can't use several versions of the sys-python2.7 crate in a single workspace: it makes for a build error. Since hgcli does not at the time being need anything from hg-core, whereas the upcoming hg-cpython will. So for now we're moving hgcli aside, hoping we could base all of them on the same version of rust-cpython again in the future. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5433 AFFECTED FILES rust/Cargo.lock rust/Cargo.toml rust/hgcli/Cargo.lock CHANGE DETAILS diff --git a/rust/hgcli/Cargo.lock b/rust/hgcli/Cargo.lock new file mode 100644 --- /dev/null +++ b/rust/hgcli/Cargo.lock @@ -0,0 +1,136 @@ +[[package]] +name = "aho-corasick" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "cpython" +version = "0.1.0" +source = "git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52#c90d65cf84abfffce7ef54476bbfed56017a2f52; +dependencies = [ + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", + "num-traits 0.1.43 (registry+https://github.com/rust-lang/crates.io-index)", + "python27-sys 0.1.2 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", +] + +[[package]] +name = "hgcli" +version = "0.1.0" +dependencies = [ + "cpython 0.1.0 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", + "python27-sys 0.1.2 (git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52)", +] + +[[package]] +name = "kernel32-sys" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "winapi 0.2.8 (registry+https://github.com/rust-lang/crates.io-index)", + "winapi-build 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "libc" +version = "0.2.45" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[[package]] +name = "memchr" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "num-traits" +version = "0.1.43" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "num-traits" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[[package]] +name = "python27-sys" +version = "0.1.2" +source = "git+https://github.com/indygreg/rust-cpython.git?rev=c90d65cf84abfffce7ef54476bbfed56017a2f52#c90d65cf84abfffce7ef54476bbfed56017a2f52; +dependencies = [ + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", + "regex 0.1.80 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "regex" +version = "0.1.80" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "aho-corasick 0.5.3 (registry+https://github.com/rust-lang/crates.io-index)", + "memchr 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)", + "regex-syntax 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)", + "thread_local 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", + "utf8-ranges 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "regex-syntax" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[[package]] +name = "thread-id" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)", + "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "thread_local" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index; +dependencies = [ + "thread-id 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)", +] + +[[package]] +name = "utf8-ranges" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[[package]] +name = "winapi" +version = "0.2.8" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[[package]] +name = "winapi-build" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index; + +[metadata] +"checksum aho-corasick 0.5.3
D5417: rust: translated random test of missingancestors
gracinet added a comment. @yuja: I'll look into it, maybe that's a case for benches (I've not played with them yet). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5417 To: gracinet, #hg-reviewers Cc: yuja, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel