[PATCH] match: adding support for repository-root-based globs

[Rodrigo Damazio Bovendorp via Mercurial-devel]

# HG changeset patch
# User Rodrigo Damazio Bovendorp 
# Date 1475944120 25200
#  Sat Oct 08 09:28:40 2016 -0700
# Node ID 5a24706699632b6b91f1079549e7ddd0ea952267
# Parent  b032a7b676c6637b2ac6f3ef29431013b15a08f9
match: adding support for repository-root-based globs

The broader plan is to add explicit base directories for all patterns:
  === ===
pattern type root-ed  cwd-ed  any-of-path
  === ===
wildcard rootglob cwdglob anyglob
regexp   rootre   cwdre   anyre
raw string   rootpath cwdpath anypath
  === ===
(table by foozy)

I'm starting by adding rootglob.
One important characteristic and difference from the older glob types is
that rootglob does a *full* match, meaning that a * at the end will never
match recursively, even when the glob is used as an include pattern.

diff -r b032a7b676c6 -r 5a2470669963 mercurial/help/patterns.txt
--- a/mercurial/help/patterns.txt   Tue Nov 01 18:54:03 2016 -0700
+++ b/mercurial/help/patterns.txt   Sat Oct 08 09:28:40 2016 -0700
@@ -40,6 +40,11 @@
 ``-I`` or ``-X`` options), can match also against directories: files
 under matched directories are treated as matched.
 
+For ``-I`` and ``-X`` options, ``glob:`` will match directories recursively.
+``rootglob:``, on the other end, does a full match, meaning that all files, in
+directories or subdirectories, will only match if the entire expression 
matches.
+In that case, ``**`` can be used to obtain recursiveness.
+
 Plain examples::
 
   path:foo/bar   a name bar in a directory named foo in the root
@@ -48,13 +53,18 @@
 
 Glob examples::
 
-  glob:*.c   any name ending in ".c" in the current directory
-  *.cany name ending in ".c" in the current directory
-  **.c   any name ending in ".c" in any subdirectory of the
- current directory including itself.
-  foo/*.cany name ending in ".c" in the directory foo
-  foo/**.c   any name ending in ".c" in any subdirectory of foo
- including itself.
+  glob:*.cany name ending in ".c" in the current directory
+  *.c any name ending in ".c" in the current directory
+  **.cany name ending in ".c" in any subdirectory of the
+  current directory including itself.
+  foo/*   any file in directory foo plus all its subdirectories,
+  recursively
+  foo/*.c any name ending in ".c" in the directory foo
+  foo/**.cany name ending in ".c" in any subdirectory of foo
+  including itself.
+  rootglob:*.cany name ending in ".c" in the repository root
+  rootglob:foo/*  all files inside foo but not its subdirectories
+  rootglob:foo/** all files inside foo and its subdirectories
 
 Regexp examples::
 
diff -r b032a7b676c6 -r 5a2470669963 mercurial/match.py
--- a/mercurial/match.pyTue Nov 01 18:54:03 2016 -0700
+++ b/mercurial/match.pySat Oct 08 09:28:40 2016 -0700
@@ -105,6 +105,8 @@
 'glob:' - a glob relative to cwd
 're:' - a regular expression
 'path:' - a path relative to repository root
+'rootglob:' - a glob relative to repository root. Unlike glob, *
+will never match subdirectories.
 'relglob:' - an unrooted glob (*.c matches C files in all dirs)
 'relpath:' - a path relative to cwd
 'relre:' - a regexp that needn't match the start of a name
@@ -286,7 +288,7 @@
 for kind, pat in [_patsplit(p, default) for p in patterns]:
 if kind in ('glob', 'relpath'):
 pat = pathutil.canonpath(root, cwd, pat, auditor)
-elif kind in ('relglob', 'path'):
+elif kind in ('relglob', 'path', 'rootglob'):
 pat = util.normpath(pat)
 elif kind in ('listfile', 'listfile0'):
 try:
@@ -447,7 +449,8 @@
 if ':' in pattern:
 kind, pat = pattern.split(':', 1)
 if kind in ('re', 'glob', 'path', 'relglob', 'relpath', 'relre',
-'listfile', 'listfile0', 'set', 'include', 'subinclude'):
+'listfile', 'listfile0', 'set', 'include', 'subinclude',
+'rootglob'):
 return kind, pat
 return default, pattern
 
@@ -540,6 +543,8 @@
 if pat == '.':
 return ''
 return '^' + util.re.escape(pat) + '(?:/|$)'
+if kind == 'rootglob':
+return '^' + _globre(pat) + '$'
 if kind == 'relglob':
 return '(?:|.*/)' + _globre(pat) + globsuffix
 if kind == 'relpath':
@@ -614,6 +619,8 @@
 
 >>> _roots([('glob', 'g/*', ''), ('glob', 'g', ''), ('glob', 'g*', '')])
 ['g', 'g', '.']
+>>> _roots([('rootglob', 'g/*', ''), ('rootglob', 'g'), ('glob', 'g*', 
'')])
+['g', 'g', '.']
 >>> _roots([('relpath', 'r', ''), ('path', 'p/p', ''), ('path', '', '')])
 ['r', 'p/p', '.']
 

Re: [PATCH] match: adding support for repository-root-based globs

[Rodrigo Damazio via Mercurial-devel]

On Wed, Nov 2, 2016 at 7:58 AM, Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

>
>
> On 11/01/2016 01:50 PM, Yuya Nishihara wrote:
>
>> On Mon, 31 Oct 2016 21:47:35 -0400, Augie Fackler wrote:
>>
>>> On Oct 28, 2016, at 4:40 AM, Pierre-Yves David <
 pierre-yves.da...@ens-lyon.org> wrote:
 On 10/25/2016 09:41 AM, Rodrigo Damazio Bovendorp via Mercurial-devel
 wrote:

> # HG changeset patch
> # User Rodrigo Damazio Bovendorp 
> # Date 1475944120 25200
> #  Sat Oct 08 09:28:40 2016 -0700
> # Node ID e8454de81600e092f05aa22ecbac32925b70d074
> # Parent  260af19891f2bed679a662be07d1379bb8207592
> match: adding support for repository-root-based globs
>
> The broader plan is to add explicit base directories for all patterns:
>   === ===
> pattern type root-ed  cwd-ed  any-of-path
>   === ===
> wildcard rootglob cwdglob anyglob
> regexp   rootre   cwdre   anyre
> raw string   rootpath cwdpath anypath
>   === ===
> (table by foozy)
>

 The subject seems complicated enough that creating a Plan page seems
 relevant. This would help other people to follow what the problem space.

 https://www.mercurial-scm.org/wiki/WriteANewFeaturePlan

>>>
>>> I’m not sure if it needs a plan page. It strikes me as believable
>>> (perhaps even likely?) that rootglob will be the only part of this
>>> implemented for a long time, perhaps ever. (Having the whole table makes
>>> good sense to me though, because now we’ve though through the future in
>>> case it ever comes.)
>>>
>>
> If I'm counting correctly, this is at least the third time matcher
> specification is touched in the history of the project. This instance have
> already created multiple email threads with interesting data "hidden" into
> them.
>
> I think a plan page is important to give clear picture of the situation
> and the proposed solution in a single place. This would help increased the
> number of eyes on this topic that history have proved complex.


> Especially give how often this topic came up I think it important to make
> sure we actually map the problem space and nail the issue once and for all.
> I'm a bit concerned we could be taking the minimal step to snipe a specific
> requirement here without actually moving toward a real closure providing a
> simplified solution that fit all all users.
>
> Regarding how far we'll go in that plan, I hope that at least
> "rootre:/rootpath:" will be implement for feature parity with glob.
>
> I'm starting by adding rootglob.
> One important characteristic and difference from the older glob types
> is
> that * will never match recursively, even when the glob is used as an
> include
> pattern.
>

 This seems a bit strange to me. Given that the current glob matcher is
 already not recursive, why don't we work on an alternative non recursive -I
 flag instead?

>>>
>>> That means we’ll have a new flag that alters the behavior of existing
>>> matchers in subtle ways. It also requires cooperation from every command
>>> that we want to add new globbing features to, including extensions, whereas
>>> if we can add a couple of new atoms (as outlined in the proposal from
>>> foozy) we get consistent behavior across all matchers.
>>>
>>
> Command using -I/-X option usually just add the 'walkopts' list to their
> option and then pass their 'opts' dict to 'scmutil.match(…)'. So improving
> the command flag situation in a unified way does not seems too scary.
>
> My main concern here is that the vast majority of user will still use the
> basic glob we expose without any prefix. If the most common matcher behave
> "differently" than all the others, that will get confusing.
> That said, the exact state of all existing matchers behavior is getting
> fuzzier in my head as the discussion progress. This is one of the reason I
> would like to see the current situation and the Foozy plan summarized into
> a plan page.
>
> A good way to test¹ design is "how embarrassing is it to explain to new
> user" (that is correlated to the simplicity of the design). The recent
> discussion and confusion around matcher show that we are not great for the
> moment.
>
> It’s a mistake that existing matchers have a recursive * behavior with
>>> --include, but it’s too late to change that. As such, I’d much rather have
>>> this proposal as currently stated.
>>>
>>
> Given the current globsuffix thing is just for backward compatibility, new
>> rootglob prefix makes sense to me. Fileset, which is relatively new,
>> behaves
>> in that way.
>>
>
> On one hand, this extra bits from Yuya increase my wish for a nice summary
> of the current situation + objective. On the other hand there it start to
> have enough core people who seems to know what they are doing on this topic
> to 

[PATCH 1 of 6 V5] manifest: throw LookupError if node not in revlog

[Durham Goode]

# HG changeset patch
# User Durham Goode 
# Date 1478133211 25200
#  Wed Nov 02 17:33:31 2016 -0700
# Branch stable
# Node ID ac8875e9c49e6d66393aa412e52262640bdcf134
# Parent  b9f7b0c10027764cee77f9c6d61877fcffea837f
manifest: throw LookupError if node not in revlog

When accessing a manifest via manifestlog[node], let's verify that the node
actually exists and throw a LookupError if it doesn't. This matches the old read
behavior, so we don't accidentally return invalid manifestctxs.

We do this in manifestlog instead of in the manifestctx/treemanifestctx
constructors because the treemanifest code currently relies on the fact that
certain code paths can produce treemanifest's without touching the revlogs (and
it has tests that verify things work if certain revlogs are missing entirely, so
they break if we add validation that tries to read them).

diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -1262,8 +1262,8 @@ class manifestlog(object):
 self._mancache = self._oldmanifest._mancache
 
 def __getitem__(self, node):
-"""Retrieves the manifest instance for the given node. Throws a 
KeyError
-if not found.
+"""Retrieves the manifest instance for the given node. Throws a
+LookupError if not found.
 """
 if node in self._mancache:
 cachemf = self._mancache[node]
@@ -1273,6 +1273,9 @@ class manifestlog(object):
 isinstance(cachemf, treemanifestctx)):
 return cachemf
 
+if node not in self._revlog.nodemap:
+raise LookupError(node, self._revlog.indexfile,
+  _('no node'))
 if self._treeinmem:
 m = treemanifestctx(self._repo, '', node)
 else:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 6 V5] manifest: remove manifest.readshallowdelta

[Durham Goode]

# HG changeset patch
# User Durham Goode 
# Date 1478131847 25200
#  Wed Nov 02 17:10:47 2016 -0700
# Branch stable
# Node ID 5a8fa7811314ee6cc08c43e06b4ff0a9c38f5041
# Parent  42cbc5a35d816cf9e5c0ab31791ab90df8004dab
manifest: remove manifest.readshallowdelta

This removes manifest.readshallowdelta and converts its one consumer to use
manifestlog instead.

diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -1365,6 +1365,12 @@ class manifestctx(object):
 return self.read()
 
 def readdelta(self, shallow=False):
+'''Returns a manifest containing just the entries that are present
+in this manifest, but not in its p1 manifest. This is efficient to read
+if the revlog delta is already p1.
+
+Changing the value of `shallow` has no effect on flat manifests.
+'''
 revlog = self._repo.manifestlog._revlog
 if revlog._usemanifestv2:
 # Need to perform a slow delta
@@ -1427,6 +1433,16 @@ class treemanifestctx(object):
 return self._node
 
 def readdelta(self, shallow=False):
+'''Returns a manifest containing just the entries that are present
+in this manifest, but not in its p1 manifest. This is efficient to read
+if the revlog delta is already p1.
+
+If `shallow` is True, this will read the delta for this directory,
+without recursively reading subdirectory manifests. Instead, any
+subdirectory entry will be reported as it appears in the manifest, i.e.
+the subdirectory will be reported among files and distinguished only by
+its 't' flag.
+'''
 revlog = self._revlog()
 if shallow and revlog._treeondisk and not revlog._usemanifestv2:
 r = revlog.rev(self._node)
@@ -1500,41 +1516,6 @@ class manifest(manifestrevlog):
   self._dirlogcache)
 return self._dirlogcache[dir]
 
-def _slowreaddelta(self, node):
-r0 = self.deltaparent(self.rev(node))
-m0 = self.read(self.node(r0))
-m1 = self.read(node)
-md = self._newmanifest()
-for f, ((n0, fl0), (n1, fl1)) in m0.diff(m1).iteritems():
-if n1:
-md[f] = n1
-if fl1:
-md.setflag(f, fl1)
-return md
-
-def readdelta(self, node):
-if self._usemanifestv2 or self._treeondisk:
-return self._slowreaddelta(node)
-r = self.rev(node)
-d = mdiff.patchtext(self.revdiff(self.deltaparent(r), r))
-return self._newmanifest(d)
-
-def readshallowdelta(self, node):
-'''For flat manifests, this is the same as readdelta(). For
-treemanifests, this will read the delta for this revlog's directory,
-without recursively reading subdirectory manifests. Instead, any
-subdirectory entry will be reported as it appears in the manifests, 
i.e.
-the subdirectory will be reported among files and distinguished only by
-its 't' flag.'''
-if not self._treeondisk:
-return self.readdelta(node)
-if self._usemanifestv2:
-raise error.Abort(
-_("readshallowdelta() not implemented for manifestv2"))
-r = self.rev(node)
-d = mdiff.patchtext(self.revdiff(self.deltaparent(r), r))
-return manifestdict(d)
-
 def read(self, node):
 if node == revlog.nullid:
 return self._newmanifest() # don't upset local cache
diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -201,7 +201,8 @@ class verifier(object):
 progress=None):
 repo = self.repo
 ui = self.ui
-mf = self.repo.manifest.dirlog(dir)
+mfl = self.repo.manifestlog
+mf = mfl._revlog.dirlog(dir)
 
 if not dir:
 self.ui.status(_("checking manifests\n"))
@@ -235,7 +236,8 @@ class verifier(object):
 self.err(lr, _("%s not in changesets") % short(n), label)
 
 try:
-for f, fn, fl in mf.readshallowdelta(n).iterentries():
+mfdelta = mfl.get(dir, n).readdelta(shallow=True)
+for f, fn, fl in mfdelta.iterentries():
 if not f:
 self.err(lr, _("entry without name in manifest"))
 elif f == "/dev/null":  # ignore this in very old repos
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 6 V5] manifest: change manifestlog mancache to be directory based

[Durham Goode]

# HG changeset patch
# User Durham Goode 
# Date 1478131847 25200
#  Wed Nov 02 17:10:47 2016 -0700
# Branch stable
# Node ID 9034cbde98311be9a93da0554a3ca9d399d64089
# Parent  03f320268065807ad4fd90dbfb9b05d5493c250c
manifest: change manifestlog mancache to be directory based

In the last patch we added a get() function that allows fetching directory level
treemanifestctxs. It didn't handle caching at directory level though, so we 
need to
change our mancache to support multiple directories.

diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -1257,9 +1257,17 @@ class manifestlog(object):
 self._oldmanifest = repo._constructmanifest()
 self._revlog = self._oldmanifest
 
+# A cache of the manifestctx or treemanifestctx for each directory
+self._dirmancache = {}
+
 # We'll separate this into it's own cache once oldmanifest is no longer
 # used
 self._mancache = self._oldmanifest._mancache
+self._dirmancache[''] = self._mancache
+
+# A future patch makes this use the same config value as the existing
+# mancache
+self.cachesize = 4
 
 def __getitem__(self, node):
 """Retrieves the manifest instance for the given node. Throws a
@@ -1271,6 +1279,14 @@ class manifestlog(object):
 """Retrieves the manifest instance for the given node. Throws a
 LookupError if not found.
 """
+if node in self._dirmancache.get(dir, ()):
+cachemf = self._dirmancache[dir][node]
+# The old manifest may put non-ctx manifests in the cache, so
+# skip those since they don't implement the full api.
+if (isinstance(cachemf, manifestctx) or
+isinstance(cachemf, treemanifestctx)):
+return cachemf
+
 if dir:
 if self._revlog._treeondisk:
 dirlog = self._revlog.dirlog(dir)
@@ -1283,14 +1299,6 @@ class manifestlog(object):
 _("cannot ask for manifest directory '%s' in a flat "
   "manifest") % dir)
 else:
-if node in self._mancache:
-cachemf = self._mancache[node]
-# The old manifest may put non-ctx manifests in the cache, so
-# skip those since they don't implement the full api.
-if (isinstance(cachemf, manifestctx) or
-isinstance(cachemf, treemanifestctx)):
-return cachemf
-
 if node not in self._revlog.nodemap:
 raise LookupError(node, self._revlog.indexfile,
   _('no node'))
@@ -1298,8 +1306,13 @@ class manifestlog(object):
 m = treemanifestctx(self._repo, '', node)
 else:
 m = manifestctx(self._repo, node)
-if node != revlog.nullid:
-self._mancache[node] = m
+
+if node != revlog.nullid:
+mancache = self._dirmancache.get(dir)
+if not mancache:
+mancache = util.lrucachedict(self.cachesize)
+self._dirmancache[dir] = mancache
+mancache[node] = m
 return m
 
 def add(self, m, transaction, link, p1, p2, added, removed):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Fixing .hg file open ordering

[Bryan O'Sullivan]

On Wed, Nov 2, 2016 at 3:30 PM, Durham Goode  wrote:

> Bookmark writes are now within the wlock, and any inmemory read you did
> prior to taking the lock will be invalidated (during the next
> repo._bookmarks access) and thrown away when you take the lock. So once
> you're in the lock you are guaranteed to get the latest version off disk,
> and guaranteed that no one else will modify it until you release the lock.
>

OK, great, that was the step I was missing when reading the code.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Fixing .hg file open ordering

[Durham Goode]

On 11/2/16 3:20 PM, Bryan O'Sullivan wrote:
There's long been a well-defined order for accessing historical data: 
changelog first, then manifest, then revlog, and the reverse for writes.
Yea, I guess I'm proposing formalizing these dependencies into a single 
location in code so we cannot mess it up on accident.


I think that what has happened with bookmarks is that we literally 
forgot about the next necessary ordering constraint: you must read 
bookmarks before the changelog, otherwise you run into the race we see 
here. This same constraint should apply to any other file that 
contains commit hashes.


It's unfortunate that the bmstore API is "backwards" in this way, but 
what's the sequence of operations by which bookmarks seem to 
disappear? There's something missing from your description to help 
piece the sequence of events together, because a bookmark that points 
to a commit that is not yet known will be explicitly ignored in the 
bmstore constructor.


Here's what I *think* must be the sequence:

I read the changelog.
I'm asked to look up the foo bookmark.
Someone else writes out some commits and a new bookmarks file.
I open the bookmarks file, which was written *after* the last 
changelog write.
The newly written bookmarks file contains a version of foo that points 
at a commit that I haven't yet seen, and because the bmstore 
constructor ignores it, it appears to have vanished.
Yes, I tried to say this in my first paragraph, though not as clearly as 
I could have.


I think this will be a little harder to fix than you may be anticipating.

You can fix the read path by reading the bookmarks before the 
changelog, but the write path is subtle. Here's why.


Bookmarks are stored in memory. If I'm being asked to write out a new 
bookmarks file as part of a transaction, what happens in this 
interleaving?


I read bookmarks
You write bookmarks
I write bookmarks, based off my stale understanding

In other words, I'll potentially cause your write to go missing unless 
I validate during the transaction that the bookmarks haven't changed 
since I read them before I write them.
Bookmark writes are now within the wlock, and any inmemory read you did 
prior to taking the lock will be invalidated (during the next 
repo._bookmarks access) and thrown away when you take the lock. So once 
you're in the lock you are guaranteed to get the latest version off 
disk, and guaranteed that no one else will modify it until you release 
the lock.


So your sequence becomes:

I read bookmarks
You write bookmarks
You release the lock
I take the lock
I read bookmarks again
I write bookmarks
I release the lock
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 7] py3: use encoding.environ in ui.py

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478122977 -19800
#  Thu Nov 03 03:12:57 2016 +0530
# Node ID 512f0781517b6a209e94ef8eb14409048f0f4ae8
# Parent  aef03902a5c1a13e9775059a5efdeb2466399ada
py3: use encoding.environ in ui.py

Using source transformer we add b'' everywhere. So there are no chances
that those bytes string will work with os.environ on Py3 as that returns a
dict of unicodes. We are relying on the errors, even though no error is raised
even in future, these pieces of codes will tend to do wrong things.
if statements can result in wrong boolean and certain errors can be raised
while using this piece of code. Let's not wait for them to happen, fix what is
wrong. If this patch goes in, I will try to do it for all the cases.
Leaving it as it is buggy.

diff -r aef03902a5c1 -r 512f0781517b mercurial/ui.py
--- a/mercurial/ui.py   Thu Nov 03 02:53:45 2016 +0530
+++ b/mercurial/ui.py   Thu Nov 03 03:12:57 2016 +0530
@@ -22,6 +22,7 @@
 
 from . import (
 config,
+encoding,
 error,
 formatter,
 progress,
@@ -574,9 +575,11 @@
 - False if HGPLAIN is not set, or feature is in HGPLAINEXCEPT
 - True otherwise
 '''
-if 'HGPLAIN' not in os.environ and 'HGPLAINEXCEPT' not in os.environ:
+if ('HGPLAIN' not in encoding.environ and
+'HGPLAINEXCEPT' not in encoding.environ):
 return False
-exceptions = os.environ.get('HGPLAINEXCEPT', '').strip().split(',')
+exceptions = encoding.environ.get('HGPLAINEXCEPT',
+'').strip().split(',')
 if feature and exceptions:
 return feature not in exceptions
 return True
@@ -589,13 +592,13 @@
 If not found and ui.askusername is True, ask the user, else use
 ($LOGNAME or $USER or $LNAME or $USERNAME) + "@full.hostname".
 """
-user = os.environ.get("HGUSER")
+user = encoding.environ.get("HGUSER")
 if user is None:
 user = self.config("ui", ["username", "user"])
 if user is not None:
 user = os.path.expandvars(user)
 if user is None:
-user = os.environ.get("EMAIL")
+user = encoding.environ.get("EMAIL")
 if user is None and self.configbool("ui", "askusername"):
 user = self.prompt(_("enter a commit username:"), default=None)
 if user is None and not self.interactive():
@@ -819,9 +822,9 @@
 def termwidth(self):
 '''how wide is the terminal in columns?
 '''
-if 'COLUMNS' in os.environ:
+if 'COLUMNS' in encoding.environ:
 try:
-return int(os.environ['COLUMNS'])
+return int(encoding.environ['COLUMNS'])
 except ValueError:
 pass
 return util.termwidth()
@@ -1080,10 +1083,10 @@
 editor = 'E'
 else:
 editor = 'vi'
-return (os.environ.get("HGEDITOR") or
+return (encoding.environ.get("HGEDITOR") or
 self.config("ui", "editor") or
-os.environ.get("VISUAL") or
-os.environ.get("EDITOR", editor))
+encoding.environ.get("VISUAL") or
+encoding.environ.get("EDITOR", editor))
 
 @util.propertycache
 def _progbar(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 7] py3: make scmposix.userrcpath() return bytes

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478119621 -19800
#  Thu Nov 03 02:17:01 2016 +0530
# Node ID e0e794c3b580b1f64d37ccdd6d8bd606eb87880e
# Parent  9d54c24d17daddf2ede7fe7ce58751ab9a1780a4
py3: make scmposix.userrcpath() return bytes

We are making sure that we deal with bytes as much we can. This is a part
of fixing functions so that they return bytes if they have to.
Used encoding.environ to return bytes.

After this patch, scmposix.userrcpath() returns bytes and scmutil.osrcpath()
will also return bytes if the platform is posix. Functions is scmposix returns
bytes on Python 3 now.

diff -r 9d54c24d17da -r e0e794c3b580 mercurial/scmposix.py
--- a/mercurial/scmposix.py Thu Nov 03 02:09:38 2016 +0530
+++ b/mercurial/scmposix.py Thu Nov 03 02:17:01 2016 +0530
@@ -4,6 +4,7 @@
 import sys
 
 from . import (
+encoding,
 osutil,
 )
 
@@ -36,6 +37,6 @@
 
 def userrcpath():
 if sys.platform == 'plan9':
-return [os.environ['home'] + '/lib/hgrc']
+return [encoding.environ['home'] + '/lib/hgrc']
 else:
 return [os.path.expanduser('~/.hgrc')]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 7] py3: make scmutil.rcpath() return bytes

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478120266 -19800
#  Thu Nov 03 02:27:46 2016 +0530
# Node ID 9e259e7b59b6358eb842eabbc99f4c18a4cc5009
# Parent  e0e794c3b580b1f64d37ccdd6d8bd606eb87880e
py3: make scmutil.rcpath() return bytes

In the whole series we are dealing with path varaibles which must be bytes
on UNIX. This patch make sure scmutil.rcpath() returns bytes independent of
which platform is used on Python 3. If we want to change type for windows we
can just conditionalize the return variable.

diff -r e0e794c3b580 -r 9e259e7b59b6 mercurial/scmutil.py
--- a/mercurial/scmutil.py  Thu Nov 03 02:17:01 2016 +0530
+++ b/mercurial/scmutil.py  Thu Nov 03 02:27:46 2016 +0530
@@ -755,7 +755,8 @@
 if _rcpath is None:
 if 'HGRCPATH' in encoding.environ:
 _rcpath = []
-for p in os.environ['HGRCPATH'].split(os.pathsep):
+pathsep = os.pathsep.encode('ascii')
+for p in encoding.environ['HGRCPATH'].split(pathsep):
 if not p:
 continue
 p = util.expandpath(p)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 7] py3: make sure osutil.listdir() returns what it gets

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478118344 -19800
#  Thu Nov 03 01:55:44 2016 +0530
# Node ID 6585e9c1d915818d5138f6cb4134707002d5d749
# Parent  e541b0e5839988f63446c88509db68772a55775b
py3: make sure osutil.listdir() returns what it gets

osutil.listdir() on py3 was having problems with bytes. Since
os.sep is a str in Py3, we need to check if what passed is bytes and then
convert os.sep to bytes. On python 2, doing os.sep.encode() is okay.
After this patch, osutil.listdir() argument will return what is gets as an
argument on Python 3.5.

diff -r e541b0e58399 -r 6585e9c1d915 mercurial/pure/osutil.py
--- a/mercurial/pure/osutil.py  Thu Nov 03 00:28:33 2016 +0530
+++ b/mercurial/pure/osutil.py  Thu Nov 03 01:55:44 2016 +0530
@@ -51,8 +51,11 @@
 '''
 result = []
 prefix = path
-if not prefix.endswith(os.sep):
-prefix += os.sep
+sep = os.sep
+if isinstance(path, bytes):
+sep = sep.encode('ascii')
+if not prefix.endswith(sep):
+prefix += sep
 names = os.listdir(path)
 names.sort()
 for fn in names:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 7] py3: make util.datapath a bytes variable

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478113113 -19800
#  Thu Nov 03 00:28:33 2016 +0530
# Node ID e541b0e5839988f63446c88509db68772a55775b
# Parent  bb586966818986131068280bfd95fc96fbdaaa0d
py3: make util.datapath a bytes variable

Fixing things when a warning or error comes up was a good approach, but that
won't work in long time, because we will be having all the errors fixed
but no idea where we set which variable to bytes and which to unicodes.
Which function is returning bytes and which is returning unicodes.
We have to make sure if some variable is changed then its effects throughout
the repository are taken care.

In this patch we make util.datapath a bytes variables.

The line containing i18n.setdatapath is skipped for a reason.
i18n.setdatapath looks something like this.

def setdatapath(datapath):
localedir = os.path.join(datapath, pycompat.sysstr('locale'))
t = gettextmod.translation('hg', localedir, _languages, fallback=True)


Here we can't pass gettextmod.translation() bytes when we have _languages as
None in Python 3.5. But yeah we can pass 'hg' as bytes because the code which
returns TypeError deals with localedir variable only. So we need localedir to
be unicode to make gettextmod.translation() happy. If we pass the bytes
version of datapath we will have to convert localedir back to unicode.
So skipped that line of code before converting util.datapath to bytes to
use in rest of the code.

diff -r bb5869668189 -r e541b0e58399 mercurial/util.py
--- a/mercurial/util.py Tue Nov 01 15:40:21 2016 -0400
+++ b/mercurial/util.py Thu Nov 03 00:28:33 2016 +0530
@@ -940,6 +940,9 @@
 
 i18n.setdatapath(datapath)
 
+if not isinstance(datapath, bytes):
+datapath = datapath.encode('utf-8')
+
 _hgexecutable = None
 
 def hgexecutable():
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Fixing .hg file open ordering

[Bryan O'Sullivan]

There's long been a well-defined order for accessing historical data:
changelog first, then manifest, then revlog, and the reverse for writes.

I think that what has happened with bookmarks is that we literally forgot
about the next necessary ordering constraint: you must read bookmarks
before the changelog, otherwise you run into the race we see here. This
same constraint should apply to any other file that contains commit hashes.

It's unfortunate that the bmstore API is "backwards" in this way, but
what's the sequence of operations by which bookmarks seem to disappear?
There's something missing from your description to help piece the sequence
of events together, because a bookmark that points to a commit that is not
yet known will be explicitly ignored in the bmstore constructor.

Here's what I *think* must be the sequence:

I read the changelog.
I'm asked to look up the foo bookmark.
Someone else writes out some commits and a new bookmarks file.
I open the bookmarks file, which was written *after* the last changelog
write.
The newly written bookmarks file contains a version of foo that points at a
commit that I haven't yet seen, and because the bmstore constructor ignores
it, it appears to have vanished.

I think this will be a little harder to fix than you may be anticipating.

You can fix the read path by reading the bookmarks before the changelog,
but the write path is subtle. Here's why.

Bookmarks are stored in memory. If I'm being asked to write out a new
bookmarks file as part of a transaction, what happens in this interleaving?

I read bookmarks
You write bookmarks
I write bookmarks, based off my stale understanding

In other words, I'll potentially cause your write to go missing unless I
validate during the transaction that the bookmarks haven't changed since I
read them before I write them.


On Wed, Nov 2, 2016 at 2:38 PM, Durham Goode  wrote:

> On 11/2/16 2:09 PM, Durham Goode wrote:
>
>> There's currently no defined order in which Mercurial should open files
>> in the .hg directory.  For instance, it's possible to read the changelog
>> first, then several seconds later read the bookmark file. If during those
>> several seconds the repo receives new commits and a bookmark moves, then it
>> will read a bookmark that points at a node that doesn't exist in the
>> inmemory changelog.
>>
>> We're seeing this on our server, which causes certain bookmarks to
>> disappear for users in some situations (until they pull again).
>>
> To clarify (as Jun has brought up outside of email), we could also solve
> this by special casing our various dependencies.  Like make
> localrepo.changelog manually load the bookmarks first (we can't do it right
> now because the bookmarkstore constructor requires the changelog so it can
> check if each bookmark exists).  So we wouldn't necessarily need a new
> layer for it, but it would require more diligence to make sure all reads
> happen in the right order across the code base.
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 7] py3: make sure osutil.listdir() returns what it gets

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478118344 -19800
#  Thu Nov 03 01:55:44 2016 +0530
# Node ID f87503dd04ee704ed0c13d6ec9673fe7d946fd6c
# Parent  e23132d2c665e788646000318a9880bf9c5a177f
py3: make sure osutil.listdir() returns what it gets

osutil.listdir() on py3 was having problems with bytes. Since
os.sep is a str in Py3, we need to check if what passed is bytes and then
convert os.sep to bytes. On python 2, doing os.sep.encode() is okay.
After this patch, osutil.listdir() argument will return what is gets as an
argument on Python 3.5.

diff -r e23132d2c665 -r f87503dd04ee mercurial/pure/osutil.py
--- a/mercurial/pure/osutil.py  Thu Nov 03 00:28:33 2016 +0530
+++ b/mercurial/pure/osutil.py  Thu Nov 03 01:55:44 2016 +0530
@@ -51,8 +51,11 @@
 '''
 result = []
 prefix = path
-if not prefix.endswith(os.sep):
-prefix += os.sep
+sep = os.sep
+if isinstance(path, bytes):
+sep = sep.encode('ascii')
+if not prefix.endswith(sep):
+prefix += sep
 names = os.listdir(path)
 names.sort()
 for fn in names:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 7] py3: use encoding.environ in ui.py

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478122977 -19800
#  Thu Nov 03 03:12:57 2016 +0530
# Node ID 9bdf6ee77b17dc2a1ce29fb347e90bd8dd17eaed
# Parent  8cd0e855ebb448a4ec1e835b346a920a1cb835cb
py3: use encoding.environ in ui.py

Using source transformer we add b'' everywhere. So there are no chances
that those bytes string will work with os.environ on Py3 as that returns a
dict of unicodes. We are relying on the errors, even though no error is raised
even in future, these pieces of codes will tend to do wrong things.
if statements can result in wrong boolean and certain errors can be raised
while using this piece of code. Let's not wait for them to happen, fix what is
wrong. If this patch goes in, I will try to do it for all the cases.
Leaving it as it is buggy.

diff -r 8cd0e855ebb4 -r 9bdf6ee77b17 mercurial/ui.py
--- a/mercurial/ui.py   Thu Nov 03 02:53:45 2016 +0530
+++ b/mercurial/ui.py   Thu Nov 03 03:12:57 2016 +0530
@@ -22,6 +22,7 @@
 
 from . import (
 config,
+encoding,
 error,
 formatter,
 progress,
@@ -574,9 +575,11 @@
 - False if HGPLAIN is not set, or feature is in HGPLAINEXCEPT
 - True otherwise
 '''
-if 'HGPLAIN' not in os.environ and 'HGPLAINEXCEPT' not in os.environ:
+if ('HGPLAIN' not in encoding.environ and
+'HGPLAINEXCEPT' not in encoding.environ):
 return False
-exceptions = os.environ.get('HGPLAINEXCEPT', '').strip().split(',')
+exceptions = encoding.environ.get('HGPLAINEXCEPT',
+'').strip().split(',')
 if feature and exceptions:
 return feature not in exceptions
 return True
@@ -589,13 +592,13 @@
 If not found and ui.askusername is True, ask the user, else use
 ($LOGNAME or $USER or $LNAME or $USERNAME) + "@full.hostname".
 """
-user = os.environ.get("HGUSER")
+user = encoding.environ.get("HGUSER")
 if user is None:
 user = self.config("ui", ["username", "user"])
 if user is not None:
 user = os.path.expandvars(user)
 if user is None:
-user = os.environ.get("EMAIL")
+user = encoding.environ.get("EMAIL")
 if user is None and self.configbool("ui", "askusername"):
 user = self.prompt(_("enter a commit username:"), default=None)
 if user is None and not self.interactive():
@@ -819,9 +822,9 @@
 def termwidth(self):
 '''how wide is the terminal in columns?
 '''
-if 'COLUMNS' in os.environ:
+if 'COLUMNS' in encoding.environ:
 try:
-return int(os.environ['COLUMNS'])
+return int(encoding.environ['COLUMNS'])
 except ValueError:
 pass
 return util.termwidth()
@@ -1080,10 +1083,10 @@
 editor = 'E'
 else:
 editor = 'vi'
-return (os.environ.get("HGEDITOR") or
+return (encoding.environ.get("HGEDITOR") or
 self.config("ui", "editor") or
-os.environ.get("VISUAL") or
-os.environ.get("EDITOR", editor))
+encoding.environ.get("VISUAL") or
+encoding.environ.get("EDITOR", editor))
 
 @util.propertycache
 def _progbar(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 7] py3: make scmpoxis.systemrcpath() return bytes

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478119178 -19800
#  Thu Nov 03 02:09:38 2016 +0530
# Node ID 0b45aa3edc2c725eaa82dde9df39a17eb8585ff0
# Parent  f87503dd04ee704ed0c13d6ec9673fe7d946fd6c
py3: make scmpoxis.systemrcpath() return bytes

The variable `p` is a str on Python 3 which is not bytes. We should convert
it to bytes because
1. root is bytes and we want the final output in bytes
2. to make the if condition works fine because in py3 its p != b'/'

So even if p is '/' but left as a str on py3, will make the condition false.
This patch ensures that scmposix.systemrcpath() return bytes and also
scmposix._rcfiles() returns and accepts bytes. The later is used in
scmposix.py only.

diff -r f87503dd04ee -r 0b45aa3edc2c mercurial/scmposix.py
--- a/mercurial/scmposix.py Thu Nov 03 01:55:44 2016 +0530
+++ b/mercurial/scmposix.py Thu Nov 03 02:09:38 2016 +0530
@@ -27,6 +27,8 @@
 # old mod_python does not set sys.argv
 if len(getattr(sys, 'argv', [])) > 0:
 p = os.path.dirname(os.path.dirname(sys.argv[0]))
+if not isinstance(p, bytes):
+p = p.encode('utf-8')
 if p != '/':
 path.extend(_rcfiles(os.path.join(p, root)))
 path.extend(_rcfiles('/' + root))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 7] py3: make util.datapath a bytes variable

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478113113 -19800
#  Thu Nov 03 00:28:33 2016 +0530
# Node ID e23132d2c665e788646000318a9880bf9c5a177f
# Parent  bb586966818986131068280bfd95fc96fbdaaa0d
py3: make util.datapath a bytes variable

Fixing things when a warning or error comes up was a good approach, but that
won't work in long time, because we will be having all the errors fixed
but no idea where we set which variable to bytes and which to unicodes.
Which function is returning bytes and which is returning unicodes.
We have to make sure if some variable is changed then its effects throughout
the repository are taken care.

In this patch we make util.datapath a bytes variables.

The line containing i18n.setdatapath is skipped for a reason.
i18n.setdatapath looks something like this.

def setdatapath(datapath):
localedir = os.path.join(datapath, pycompat.sysstr('locale'))
t = gettextmod.translation('hg', localedir, _languages, fallback=True)


Here we can't pass gettextmod.translation() bytes when we have _languages as
None in Python 3.5. But yeah we can pass 'hg' as bytes because the code which
returns TypeError deals with localedir variable only. So we need localedir to
be unicode to make gettextmod.translation() happy. If we pass the bytes
version of datapath we will have to convert localedir back to unicode as
util.datapath will always be a str. So skipped that line of code before
converting util.datapath to bytes to use in rest of the code.

diff -r bb5869668189 -r e23132d2c665 mercurial/util.py
--- a/mercurial/util.py Tue Nov 01 15:40:21 2016 -0400
+++ b/mercurial/util.py Thu Nov 03 00:28:33 2016 +0530
@@ -940,6 +940,9 @@
 
 i18n.setdatapath(datapath)
 
+if not isinstance(datapath, bytes):
+datapath = datapath.encode('utf-8')
+
 _hgexecutable = None
 
 def hgexecutable():
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 7] py3: make scmutil.rcpath() return bytes

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478120266 -19800
#  Thu Nov 03 02:27:46 2016 +0530
# Node ID 495481800cc65bddf8a944ce0a091fef5972c6b9
# Parent  b0e4b8169c8f8080cee7648ddd8824025814e905
py3: make scmutil.rcpath() return bytes

In the whole series we are dealing with path varaibles which must be bytes
on UNIX. This patch make sure scmutil.rcpath() returns bytes independent of
which platform is used on Python 3. If we want to change type for windows we
can just conditionalize the return variable.

diff -r b0e4b8169c8f -r 495481800cc6 mercurial/scmutil.py
--- a/mercurial/scmutil.py  Thu Nov 03 02:17:01 2016 +0530
+++ b/mercurial/scmutil.py  Thu Nov 03 02:27:46 2016 +0530
@@ -755,7 +755,8 @@
 if _rcpath is None:
 if 'HGRCPATH' in encoding.environ:
 _rcpath = []
-for p in os.environ['HGRCPATH'].split(os.pathsep):
+pathsep = os.pathsep.encode('ascii')
+for p in encoding.environ['HGRCPATH'].split(pathsep):
 if not p:
 continue
 p = util.expandpath(p)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 7] py3: make scmposix.userrcpath() return bytes

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478119621 -19800
#  Thu Nov 03 02:17:01 2016 +0530
# Node ID b0e4b8169c8f8080cee7648ddd8824025814e905
# Parent  0b45aa3edc2c725eaa82dde9df39a17eb8585ff0
py3: make scmposix.userrcpath() return bytes

We are making sure that we deal with bytes as much we can. This is a part
of fixing functions so that they return bytes if they have to.
Used encoding.environ to return bytes.

After this patch, scmposix.userrcpath() returns bytes and scmutil.osrcpath()
will also return bytes if the platform is posix. Functions is scmposix returns
bytes on Python 3 now.

diff -r 0b45aa3edc2c -r b0e4b8169c8f mercurial/scmposix.py
--- a/mercurial/scmposix.py Thu Nov 03 02:09:38 2016 +0530
+++ b/mercurial/scmposix.py Thu Nov 03 02:17:01 2016 +0530
@@ -4,6 +4,7 @@
 import sys
 
 from . import (
+encoding,
 osutil,
 )
 
@@ -36,6 +37,6 @@
 
 def userrcpath():
 if sys.platform == 'plan9':
-return [os.environ['home'] + '/lib/hgrc']
+return [encoding.environ['home'] + '/lib/hgrc']
 else:
 return [os.path.expanduser('~/.hgrc')]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 7] py3: use try/except to check for basestring

[Pulkit Goyal]

# HG changeset patch
# User Pulkit Goyal <7895pul...@gmail.com>
# Date 1478121825 -19800
#  Thu Nov 03 02:53:45 2016 +0530
# Node ID 8cd0e855ebb448a4ec1e835b346a920a1cb835cb
# Parent  495481800cc65bddf8a944ce0a091fef5972c6b9
py3: use try/except to check for basestring

The term basestring don't exist in Python 3.5 and throws a NameError.
It used to refer to unicodes in Python 2. Used try/expect to handle this.

diff -r 495481800cc6 -r 8cd0e855ebb4 mercurial/ui.py
--- a/mercurial/ui.py   Thu Nov 03 02:27:46 2016 +0530
+++ b/mercurial/ui.py   Thu Nov 03 02:53:45 2016 +0530
@@ -520,7 +520,12 @@
 result = self.config(section, name, untrusted=untrusted)
 if result is None:
 result = default or []
-if isinstance(result, basestring):
+checkunicode = False
+try:
+checkunicode = isinstance(result, basestring)
+except NameError:
+checkunicode = isinstance(result, str)
+if checkunicode:
 result = _configlist(result.lstrip(' ,\n'))
 if result is None:
 result = default or []
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 01 of 11] util: put compression code next to each other

[Gregory Szorc]

On Wed, Nov 2, 2016 at 7:26 AM, Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

>
>
> On 11/02/2016 01:08 AM, Gregory Szorc wrote:
>
>> # HG changeset patch
>> # User Gregory Szorc 
>> # Date 1476577441 25200
>> #  Sat Oct 15 17:24:01 2016 -0700
>> # Node ID 60f180c9a030ebcee6c6f4f8584fdb94c73ac337
>> # Parent  e5cc44ea12de681d971fcbebb65a7fb71fd1c3c7
>> util: put compression code next to each other
>>
>> ctxmanager was injecting itself between the compression and
>> decompression code. Let's restore some order.
>>
>
> patch 1 is pushed, patch 2 is yet to be reviewed.
>
> Some feedback on your submition scheme, given the size of patches 2 (and
> therefor likeness of comment), it would have been better to keep the series
> small in the form of:
>
> 1) preparatory patches
> 2) large/new/complex patch
> 3) minimal usage of the new code
>
> Then you can send a larger series of more trivial usage of the new code
> once it is accepted.


>From my experience, submitting code to introduce a new API without clear
examples of how that API is used makes things harder because it isn't clear
how the new code will be used and/or why it is necessary. A reviewer could
justifiably drag their feet until these questions are answered. Code
answers those questions.

I agree that part 2 could have been split. However, I thought it best to
introduce a semi-complete snapshot of the API in a single go so reviewers
didn't have to piece it together from several diffs. Now that you've seen
the general scope of what I'm doing, if you want me to split it up, I can.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] repair: make strip() return backup file path

[Martin von Zweigbergk via Mercurial-devel]

# HG changeset patch
# User Martin von Zweigbergk 
# Date 1477953630 25200
#  Mon Oct 31 15:40:30 2016 -0700
# Node ID 2e435c3beb8c31822347661a898a7e94a5cee08e
# Parent  6a8aff737a17ada068b8ce4501184eacc66e827f
repair: make strip() return backup file path

narrowhg wants to strip some commits and then re-apply them after
applying another bundle. Having repair.strip() return the bundle path
will be helpful for it.

diff -r 6a8aff737a17 -r 2e435c3beb8c mercurial/repair.py
--- a/mercurial/repair.py   Sat Oct 15 17:24:01 2016 -0700
+++ b/mercurial/repair.py   Mon Oct 31 15:40:30 2016 -0700
@@ -244,6 +244,9 @@
 vfs.unlink(tmpbundlefile)
 
 repo.destroyed()
+# return the backup file path (or None if 'backup' was False) so
+# extensions can use it
+return backupfile
 
 def rebuildfncache(ui, repo):
 """Rebuilds the fncache file from repo history.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Fixing .hg file open ordering

[Durham Goode]

On 11/2/16 2:09 PM, Durham Goode wrote:
There's currently no defined order in which Mercurial should open 
files in the .hg directory.  For instance, it's possible to read the 
changelog first, then several seconds later read the bookmark file. If 
during those several seconds the repo receives new commits and a 
bookmark moves, then it will read a bookmark that points at a node 
that doesn't exist in the inmemory changelog.


We're seeing this on our server, which causes certain bookmarks to 
disappear for users in some situations (until they pull again).
To clarify (as Jun has brought up outside of email), we could also solve 
this by special casing our various dependencies.  Like make 
localrepo.changelog manually load the bookmarks first (we can't do it 
right now because the bookmarkstore constructor requires the changelog 
so it can check if each bookmark exists).  So we wouldn't necessarily 
need a new layer for it, but it would require more diligence to make 
sure all reads happen in the right order across the code base.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Fixing .hg file open ordering

[Durham Goode]

There's currently no defined order in which Mercurial should open files 
in the .hg directory.  For instance, it's possible to read the changelog 
first, then several seconds later read the bookmark file. If during 
those several seconds the repo receives new commits and a bookmark 
moves, then it will read a bookmark that points at a node that doesn't 
exist in the inmemory changelog.


We're seeing this on our server, which causes certain bookmarks to 
disappear for users in some situations (until they pull again).


Has anyone thought about how to solve this?  I know we talked about a 
better storage model that allowed for more snapshot views of the store, 
but that seems farther off.


I've been thinking about two possible ideas:

1. Introduce a store accessor class that all reads go through. An 
instance of it would live on the localrepo and it is responsible for 
providing a snapshotted view of the store. It would contain all the 
inmemory instances of store structures (the revlogs, the bookmark store, 
etc). All reads of those structures would go through that class, and it 
would be responsible for invalidation (like @storecache) and loading 
them in the right order (so reading the changelog would force the 
bookmarks to be read first).


This might also be useful for letting other classes hold a reference to 
the store without holding a reference to the repo. For instance, 
changectx and manifestctx could hold a reference to the store accessor 
(since they need access to the changelog and manifest revlogs for 
reads), instead of the repo (which is a source of circular dependencies).


Having the store be separate from the repo instance could also be useful 
for having multiple store snapshots inmemory at once.  I'm not sure why 
we would want to, but we could diff the originally loaded store state 
with a future store state by instantiating two.


2. The option above would be a pretty significant change. Alternatively, 
I was think about teaching the opener class about the concept of 
dependencies.  We could tell it ".hg/bookmark" should always be opened 
before ".hg/store/00changelog.i". Then, if anyone tries to read 
.hg/store/00changelog.i, we will automatically open .hg/bookmark first, 
if it hasn't already been opened.  Then when the bookmark reader comes 
along, we hand it the already opened file handle.


It's somewhat weird to do it at the opener level, but it would mean we 
could avoid refactoring the rest of the code base.



Thoughts? Better ideas?

Durham

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3] rebase: rename merge to mergemod

[timeless]

# HG changeset patch
# User timeless 
# Date 1478112967 0
#  Wed Nov 02 18:56:07 2016 +
# Node ID 78cf30254354eafb993ed7f226eb70d7b7abb6dc
# Parent  b032a7b676c6637b2ac6f3ef29431013b15a08f9
# Available At https://bitbucket.org/timeless/mercurial-crew
#  hg pull https://bitbucket.org/timeless/mercurial-crew -r 
78cf30254354
rebase: rename merge to mergemod

diff -r b032a7b676c6 -r 78cf30254354 hgext/rebase.py
--- a/hgext/rebase.py   Tue Nov 01 18:54:03 2016 -0700
+++ b/hgext/rebase.py   Wed Nov 02 18:56:07 2016 +
@@ -36,7 +36,7 @@
 extensions,
 hg,
 lock,
-merge,
+merge as mergemod,
 obsolete,
 patch,
 phases,
@@ -823,7 +823,7 @@
 # Update to target and merge it with local
 if repo['.'].rev() != p1:
 repo.ui.debug(" update to %d:%s\n" % (p1, repo[p1]))
-merge.update(repo, p1, False, True)
+mergemod.update(repo, p1, False, True)
 else:
 repo.ui.debug(" already in target\n")
 repo.dirstate.write(repo.currenttransaction())
@@ -832,8 +832,8 @@
 repo.ui.debug("   detach base %d:%s\n" % (base, repo[base]))
 # When collapsing in-place, the parent is the common ancestor, we
 # have to allow merging with it.
-stats = merge.update(repo, rev, True, True, base, collapse,
-labels=['dest', 'source'])
+stats = mergemod.update(repo, rev, True, True, base, collapse,
+labels=['dest', 'source'])
 if collapse:
 copies.duplicatecopies(repo, rev, target)
 else:
@@ -1150,7 +1150,7 @@
 
 # Update away from the rebase if necessary
 if shouldupdate or needupdate(repo, state):
-merge.update(repo, originalwd, False, True)
+mergemod.update(repo, originalwd, False, True)
 
 # Strip from the first rebased revision
 if rebased:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 3] cmdutil: refactor checkunresolved

[timeless]

# HG changeset patch
# User timeless 
# Date 1478112353 0
#  Wed Nov 02 18:45:53 2016 +
# Node ID 8dae4ad6767cf3456dd2fc4b2a7b478d855091f7
# Parent  78cf30254354eafb993ed7f226eb70d7b7abb6dc
# Available At https://bitbucket.org/timeless/mercurial-crew
#  hg pull https://bitbucket.org/timeless/mercurial-crew -r 
8dae4ad6767c
cmdutil: refactor checkunresolved

localrepo.commit had code to check for unresolved merge conflicts,
it would be helpful for at least rebase to be able to use that
code without calling commit().

diff -r 78cf30254354 -r 8dae4ad6767c mercurial/cmdutil.py
--- a/mercurial/cmdutil.py  Wed Nov 02 18:56:07 2016 +
+++ b/mercurial/cmdutil.py  Wed Nov 02 18:45:53 2016 +
@@ -3403,6 +3403,14 @@
 
 return cmd
 
+def checkunresolved(ms):
+if list(ms.unresolved()):
+raise error.Abort(_("unresolved merge conflicts "
+"(see 'hg help resolve')"))
+if ms.mdstate() != 's' or list(ms.driverresolved()):
+raise error.Abort(_('driver-resolved merge conflicts'),
+  hint=_('run "hg resolve --all" to resolve'))
+
 # a list of (ui, repo, otherpeer, opts, missing) functions called by
 # commands.outgoing.  "missing" is "missing" of the result of
 # "findcommonoutgoing()"
diff -r 78cf30254354 -r 8dae4ad6767c mercurial/localrepo.py
--- a/mercurial/localrepo.pyWed Nov 02 18:56:07 2016 +
+++ b/mercurial/localrepo.pyWed Nov 02 18:45:53 2016 +
@@ -1633,13 +1633,7 @@
 raise error.Abort(_("cannot commit merge with missing files"))
 
 ms = mergemod.mergestate.read(self)
-
-if list(ms.unresolved()):
-raise error.Abort(_("unresolved merge conflicts "
-"(see 'hg help resolve')"))
-if ms.mdstate() != 's' or list(ms.driverresolved()):
-raise error.Abort(_('driver-resolved merge conflicts'),
-  hint=_('run "hg resolve --all" to resolve'))
+cmdutil.checkunresolved(ms)
 
 if editor:
 cctx._text = editor(self, cctx, subs)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 3] rebase: check for conflicts before continuing

[timeless]

# HG changeset patch
# User timeless 
# Date 1478113169 0
#  Wed Nov 02 18:59:29 2016 +
# Node ID e9528bc734ce94456dd2389b4ea318278139c206
# Parent  8dae4ad6767cf3456dd2fc4b2a7b478d855091f7
# Available At https://bitbucket.org/timeless/mercurial-crew
#  hg pull https://bitbucket.org/timeless/mercurial-crew -r 
e9528bc734ce
rebase: check for conflicts before continuing

When there are unresolved merge conflicts, there is no reason
to make the user wait for rebase to process all of the
already rebased commits just to complain that it cannot do
anything. Abort early.

diff -r 8dae4ad6767c -r e9528bc734ce hgext/rebase.py
--- a/hgext/rebase.py   Wed Nov 02 18:45:53 2016 +
+++ b/hgext/rebase.py   Wed Nov 02 18:59:29 2016 +
@@ -661,6 +661,9 @@
 _('abort and continue do not allow specifying revisions'))
 if abortf and opts.get('tool', False):
 ui.warn(_('tool option will be ignored\n'))
+if contf:
+ms = mergemod.mergestate.read(repo)
+cmdutil.checkunresolved(ms)
 
 retcode = rbsrt._prepareabortorcontinue(abortf)
 if retcode is not None:
diff -r 8dae4ad6767c -r e9528bc734ce tests/test-rebase-conflicts.t
--- a/tests/test-rebase-conflicts.t Wed Nov 02 18:45:53 2016 +
+++ b/tests/test-rebase-conflicts.t Wed Nov 02 18:59:29 2016 +
@@ -73,8 +73,6 @@
 Try to continue without solving the conflict:
 
   $ hg rebase --continue
-  already rebased 3:3163e20567cc "L1" as 3e046f2ecedb
-  rebasing 4:46f0b057b5c0 "L2"
   abort: unresolved merge conflicts (see 'hg help resolve')
   [255]
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5410] New: Add patch-editing support to `hg record`

[bugzilla]

https://bz.mercurial-scm.org/show_bug.cgi?id=5410

Bug ID: 5410
   Summary: Add patch-editing support to `hg record`
   Product: Mercurial
   Version: unspecified
  Hardware: PC
OS: Windows
Status: UNCONFIRMED
  Severity: feature
  Priority: wish
 Component: record
  Assignee: bugzi...@selenic.com
  Reporter: scov...@gmail.com
CC: mercurial-de...@selenic.com

Created attachment 1939
  --> https://bz.mercurial-scm.org/attachment.cgi?id=1939=edit
Augmented record.py targeting mercurial-2.8

One very nice feature of git is the ability to edit patches in place (with a
suitable patch-enlightened editor) and commit the result rather than the code
that was actually in the working directory. This makes it vastly easier to
disentangle multiple code changes into separate commits.

I am attaching two versions of the record extension that implement this
feature: the first targets mercurial-2.8 that I have been using for several
years now, and the second is a lightly tested update for mercurial-3.7 that
works around the disappearance of the `mercurial.hg.revert()` function.

The augmented extension is reasonably well documented, using `-p` to request
patch editing. 

I know of two weaknesses that cause the commit to fail with an exception throw:

- file rename or copy operations
- some input diff contains a complaint about the lack of a newline

I have not attempted to address those weaknesses because they have not been
sufficiently annoying to be worth the trouble of figuring out.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] largefiles: clarify variable name holding file mode

[Mads Kiilerich]

# HG changeset patch
# User Mads Kiilerich 
# Date 1476801939 -7200
#  Tue Oct 18 16:45:39 2016 +0200
# Node ID 90300200bc1fcaedcc6ab109574d08b01ece2853
# Parent  bb586966818986131068280bfd95fc96fbdaaa0d
largefiles: clarify variable name holding file mode

A follow-up to c01acee367ec.

'st' sounds like the whole stat result while 'mode' is a better name for the
actual file mode.

diff --git a/hgext/largefiles/lfcommands.py b/hgext/largefiles/lfcommands.py
--- a/hgext/largefiles/lfcommands.py
+++ b/hgext/largefiles/lfcommands.py
@@ -510,18 +510,21 @@ def updatelfiles(ui, repo, filelist=None
 lfdirstate.normal(lfile)
 update1 = 1
 
-# copy the state of largefile standin from the repository's
+# copy the exec mode of largefile standin from the repository's
 # dirstate to its state in the lfdirstate.
 rellfile = lfile
 relstandin = lfutil.standin(lfile)
 if wvfs.exists(relstandin):
+# exec is decided by the users permissions using mask 0o100
 standinexec = wvfs.stat(relstandin).st_mode & 0o100
-st = wvfs.stat(rellfile).st_mode
-if standinexec != st & 0o100:
-st &= ~0o111
+st = wvfs.stat(rellfile)
+mode = st.st_mode
+if standinexec != mode & 0o100:
+# first remove all X bits, then shift all R bits to X
+mode &= ~0o111
 if standinexec:
-st |= (st >> 2) & 0o111 & ~util.umask
-wvfs.chmod(rellfile, st)
+mode |= (mode >> 2) & 0o111 & ~util.umask
+wvfs.chmod(rellfile, mode)
 update1 = 1
 
 updated += update1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@30253: new changeset

[Mercurial Commits]

New changeset in mercurial:

http://selenic.com/repo/hg//rev/b032a7b676c6
changeset:   30253:b032a7b676c6
bookmark:@
tag: tip
user:Gregory Szorc 
date:Tue Nov 01 18:54:03 2016 -0700
summary: statprof: vendor statprof.py

-- 
Repository URL: http://selenic.com/repo/hg/
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] spelling: fixes of non-dictionary words

[Augie Fackler]

> On Nov 2, 2016, at 11:13, Mads Kiilerich  wrote:
> 
> On 11/02/2016 03:41 PM, Augie Fackler wrote:
>>> On Nov 2, 2016, at 10:32, Mads Kiilerich  wrote:
>>> 
>>> On 10/18/2016 02:20 AM, Augie Fackler wrote:
 On Mon, Oct 17, 2016 at 11:38:02PM +0200, Mads Kiilerich wrote:
> # HG changeset patch
> # User Mads Kiilerich 
> # Date 1476739015 -7200
> #  Mon Oct 17 23:16:55 2016 +0200
> # Node ID efd5397e9da5b1d7e2c3353b0b06fc904651b150
> # Parent  8a864844d5a0c34bdb24d2e098a0cd339e32e020
> spelling: fixes of non-dictionary words
 Queued, with some hunks discarded. I've tried to note the interesting ones.
>>> Can you push your queued version now?
>> I thought I already had. If it's not local, then it's gone. Do you want to 
>> just send a new one, or should I try and reconstruct what I had?
> 
> Changes don't "go away" with Mercurial so I would assume that you must have 
> it locally somewhere. Digging it out is probably less work for you than the 
> alternatives.

I work across a wide variety of machines, and I *do not have* the change 
anymore.

> Alternatively, you are in a power position and can decide which alternative 
> you want ;-)

I'll regenerate the change.

But I'd rather you published your scripts for this, rather than sending us a 
code bomb once a year. ;)

> 
> /Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] spelling: fixes of non-dictionary words

[Mads Kiilerich]

On 11/02/2016 03:41 PM, Augie Fackler wrote:

On Nov 2, 2016, at 10:32, Mads Kiilerich  wrote:

On 10/18/2016 02:20 AM, Augie Fackler wrote:

On Mon, Oct 17, 2016 at 11:38:02PM +0200, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1476739015 -7200
#  Mon Oct 17 23:16:55 2016 +0200
# Node ID efd5397e9da5b1d7e2c3353b0b06fc904651b150
# Parent  8a864844d5a0c34bdb24d2e098a0cd339e32e020
spelling: fixes of non-dictionary words

Queued, with some hunks discarded. I've tried to note the interesting ones.

Can you push your queued version now?

I thought I already had. If it's not local, then it's gone. Do you want to just 
send a new one, or should I try and reconstruct what I had?


Changes don't "go away" with Mercurial so I would assume that you must 
have it locally somewhere. Digging it out is probably less work for you 
than the alternatives.


Alternatively, you are in a power position and can decide which 
alternative you want ;-)


/Mads
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] spelling: fixes of non-dictionary words

[Augie Fackler]

> On Nov 2, 2016, at 10:32, Mads Kiilerich  wrote:
> 
> On 10/18/2016 02:20 AM, Augie Fackler wrote:
>> On Mon, Oct 17, 2016 at 11:38:02PM +0200, Mads Kiilerich wrote:
>>> # HG changeset patch
>>> # User Mads Kiilerich 
>>> # Date 1476739015 -7200
>>> #  Mon Oct 17 23:16:55 2016 +0200
>>> # Node ID efd5397e9da5b1d7e2c3353b0b06fc904651b150
>>> # Parent  8a864844d5a0c34bdb24d2e098a0cd339e32e020
>>> spelling: fixes of non-dictionary words
>> Queued, with some hunks discarded. I've tried to note the interesting ones.
> 
> Can you push your queued version now?

I thought I already had. If it's not local, then it's gone. Do you want to just 
send a new one, or should I try and reconstruct what I had?

> 
> /Mads

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] spelling: fixes of non-dictionary words

[Mads Kiilerich]

On 10/18/2016 02:20 AM, Augie Fackler wrote:

On Mon, Oct 17, 2016 at 11:38:02PM +0200, Mads Kiilerich wrote:

# HG changeset patch
# User Mads Kiilerich 
# Date 1476739015 -7200
#  Mon Oct 17 23:16:55 2016 +0200
# Node ID efd5397e9da5b1d7e2c3353b0b06fc904651b150
# Parent  8a864844d5a0c34bdb24d2e098a0cd339e32e020
spelling: fixes of non-dictionary words

Queued, with some hunks discarded. I've tried to note the interesting ones.


Can you push your queued version now?

/Mads
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH evolve v2] tests: use curl instead of wget

[Pierre-Yves David]

On 10/28/2016 01:51 PM, Matt Harbison wrote:



On Oct 28, 2016, at 4:53 AM, Pierre-Yves David  
wrote:




On 10/25/2016 02:23 PM, Simon Farnsworth wrote:
# HG changeset patch
# User Simon Farnsworth 
# Date 1477397752 25200
#  Tue Oct 25 05:15:52 2016 -0700
# Branch stable
# Node ID f65f9acac6c69e6f2eb90b2ed9b51d818a046f67
# Parent  970a4c13ebc320a034bc0aff21e0ef0a84157a92
tests: use curl instead of wget


Matt (Harbison) can you confirm you are okay with this change before I take it?


If it's possible to dynamically switch between tools, that's obviously better.  
But I don't have a problem accepting this for short term convenience.


Gah, I forgot to push that one before 5.5. I've pushed it now

Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] build: include a dummy $PATH in the custom environment used by build.py

[Pierre-Yves David]

On 11/02/2016 02:40 PM, Gábor STEFANIK wrote:





--
This message, including its attachments, is confidential. For more information 
please read NNG's email policy here:
http://www.nng.com/emailpolicy/
By responding to this email you accept the email policy.


-Original Message-

From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
On Behalf Of Pierre-Yves David
Sent: Tuesday, November 1, 2016 4:56 PM
To: Jun Wu 
Cc: mercurial-devel@mercurial-scm.org
Subject: Re: [PATCH STABLE] build: include a dummy $PATH in the custom
environment used by build.py



On 11/01/2016 04:39 PM, Jun Wu wrote:

Excerpts from Pierre-Yves David's message of 2016-10-29 02:18:56 +0200:

My question is more "What is the meaning and effect of adding "." here?
I'm fine with trying to fix pypiwin32 but we have to be aware of the
actual consequence. A related quest is "What is happening is 'PATH'
is already in the envirement? are we overwriting it?


For Windows, PATH='.' is equivalent to PATH='' since Windows searches
for PWD by design.

For *nix, PATH='' is better. PATH='.' has undesired side-effects.

I checked the code, if PATH is already set, it will be dropped by our
code because we created a new "env" dict and pass it to "runhg" ->
"runcmd" -> "subprocess.Popen(..., env=env)" so the new process does
not inherit the old PATH.

We rely on "sys.executable" to be correct to find the correct Python.

I think having an empty value, or inherit it from os.environ are both
acceptable solutions. But it's pypiwin32 to blame anyway.


It seem slike we should at least inherit a value if one exists. This patch does
not seems ready for inclusion. I'm dropping it from patchwork and we'll
revisit post release.


I believe the point of using a custom environment here is precisely to avoid 
passing the real $PATH.
Inheriting from the system environment would defeat this.

The only reason for including a PATH entry here is so site.py can do 
os.environ['PATH'] instead of os.environ.get('PATH').
Unfortunately some python modules (like pypiwin32) assume that a $PATH 
environment variable must always exist.


Ha true, when set, 'env' does not inherit any variable. So setting the 
path to '' should be fine. That said, the comment this env dictionary is 
quite specific about its goal.


# Execute hg out of this directory with a custom environment which
# takes care to not use any hgrc files and do no localization.

We should add a note about the why windows required PATH in some case.

I think we are clear enough now. (Jun do you agree?)

Can you send a V2 with an empty PATH and an updated comment?

Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 8 V7] exchange: add `_getbookmarks()` function

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478016027 25200
#  Tue Nov 01 09:00:27 2016 -0700
# Branch stable
# Node ID a56a624a8a42b93ec980d2c284756a38719dffe6
# Parent  b9f7b0c10027764cee77f9c6d61877fcffea837f
exchange: add `_getbookmarks()` function

This function will be used to generate bookmarks bundle2 part.
It is a separate function in order to make it easy to overwrite it
in extensions. Passing `kwargs` to the function makes it easy to
add new parameters in extensions.

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1666,6 +1666,17 @@
 if chunks:
 bundler.newpart('hgtagsfnodes', data=''.join(chunks))
 
+def _getbookmarks(repo, **kwargs):
+"""Returns list of bookmarks.
+
+This function is primarily used to generate `bookmarks` bundle2 part.
+It is a separate function in order to make it easy to wrap it
+in extensions. Passing `kwargs` to the function makes it easy to
+add new parameters in extensions.
+"""
+
+return bookmod.listbookmarks(repo)
+
 def check_heads(repo, their_heads, context):
 """check if the heads of a repo have been modified
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8 V7] bundle2: advertise bookmark capability

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478088362 25200
#  Wed Nov 02 05:06:02 2016 -0700
# Branch stable
# Node ID c2a9f675fac55522bb954ea81c29fa0158106214
# Parent  e2122d93aeb4da4a39c0c257e74414ac348a89f1
bundle2: advertise bookmark capability

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1272,6 +1272,7 @@
 'digests': tuple(sorted(util.DIGESTS.keys())),
 'remote-changegroup': ('http', 'https'),
 'hgtagsfnodes': (),
+'bookmarks': (),
}
 
 def getrepocaps(repo, allowpushback=False):
diff --git a/tests/test-acl.t b/tests/test-acl.t
--- a/tests/test-acl.t
+++ b/tests/test-acl.t
@@ -92,13 +92,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 165 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 165
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -155,13 +155,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 165 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 165
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -221,13 +221,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 165 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 165
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -297,13 +297,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 165 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 165
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -362,13 +362,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 165 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 165
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -432,13 +432,13 @@
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 

[PATCH 6 of 8 V7] pull: use `bookmarks` bundle2 part

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478086459 25200
#  Wed Nov 02 04:34:19 2016 -0700
# Branch stable
# Node ID e2122d93aeb4da4a39c0c257e74414ac348a89f1
# Parent  12466c729bb6783fbef11a6e148c648809b4f592
pull: use `bookmarks` bundle2 part

Apply changes from `bookmarks` part.

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1322,9 +1322,13 @@
 kwargs['cg'] = pullop.fetch
 if 'listkeys' in pullop.remotebundle2caps:
 kwargs['listkeys'] = ['phases']
-if pullop.remotebookmarks is None:
-# make sure to always includes bookmark data when migrating
-# `hg incoming --bundle` to using this function.
+
+if pullop.remotebookmarks is None:
+# make sure to always includes bookmark data when migrating
+# `hg incoming --bundle` to using this function.
+if 'bookmarks' in pullop.remotebundle2caps:
+kwargs['bookmarks'] = True
+elif 'listkeys' in pullop.remotebundle2caps:
 kwargs['listkeys'].append('bookmarks')
 
 # If this is a full pull / clone and the server supports the clone bundles
@@ -1352,10 +1356,23 @@
 _pullbundle2extraprepare(pullop, kwargs)
 bundle = pullop.remote.getbundle('pull', **kwargs)
 try:
-op = bundle2.processbundle(pullop.repo, bundle, pullop.gettransaction)
+bundleopinput = {
+'processbookmarksmode': 'diverge',
+'explicitbookmarks': pullop.explicitbookmarks,
+'remotepath': pullop.remote.url(),
+}
+op = bundle2.bundleoperation(pullop.repo, pullop.gettransaction,
+ input=bundleopinput)
+op = bundle2.processbundle(pullop.repo, bundle,
+   pullop.gettransaction, op=op)
 except error.BundleValueError as exc:
 raise error.Abort(_('missing support for %s') % exc)
 
+# `bookmarks` part was in bundle and it was applied to the repo. No need to
+# apply bookmarks one more time
+if 'bookmarks' in kwargs and kwargs['bookmarks']:
+pullop.stepsdone.add('bookmarks')
+
 if pullop.fetch:
 results = [cg['return'] for cg in op.records['changegroup']]
 pullop.cgresult = changegroup.combineresults(results)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 5 of 5] hgweb: make log streams compatible with command server

[Pierre-Yves David]

On 11/02/2016 01:02 PM, Yuya Nishihara wrote:

# HG changeset patch
# User Yuya Nishihara 
# Date 1403928812 -32400
#  Sat Jun 28 13:13:32 2014 +0900
# Node ID 827132690102f124cf5ee9d1888fdddba1987b9d
# Parent  e1a050ebbf75c6f512fe041864a1e53b647c3967
# EXP-Topic stdio
hgweb: make log streams compatible with command server

Even though it would be useless to start a web server by a command server,
it should be doable in principle. Also, we can't use sys.stdout/err directly
on Python 3 because they are unicode streams.


This series is pushed, thanks.

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8 V7] exchange: getbundle `bookmarks` part generator

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478016027 25200
#  Tue Nov 01 09:00:27 2016 -0700
# Branch stable
# Node ID 12466c729bb6783fbef11a6e148c648809b4f592
# Parent  2f89680108366512c1a6345abec5cebdb85170ac
exchange: getbundle `bookmarks` part generator

This generator will be used during pull operation.

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1666,6 +1666,21 @@
 if chunks:
 bundler.newpart('hgtagsfnodes', data=''.join(chunks))
 
+@getbundle2partsgenerator('bookmarks')
+def _getbundlebookmarkspart(bundler, repo, source, bundlecaps=None,
+b2caps=None, heads=None, common=None,
+**kwargs):
+if not kwargs.get('bookmarks'):
+return
+if 'bookmarks' not in b2caps:
+raise ValueError(
+_('bookmarks are requested but client is not capable '
+  'of receiving it'))
+
+bookmarks = _getbookmarks(repo, **kwargs)
+encodedbookmarks = bookmod.encodebookmarks(bookmarks)
+bundler.newpart('bookmarks', data=encodedbookmarks)
+
 def _getbookmarks(repo, **kwargs):
 """Returns list of bookmarks.
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -228,7 +228,9 @@
  'bundlecaps': 'scsv',
  'listkeys': 'csv',
  'cg': 'boolean',
- 'cbattempted': 'boolean'}
+ 'cbattempted': 'boolean',
+ 'bookmarks': 'boolean',
+}
 
 # client side
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8 V7] bookmarks: introduce binary encoding

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478016027 25200
#  Tue Nov 01 09:00:27 2016 -0700
# Branch stable
# Node ID f3da841e3d47ccbb0be3892b521607c09fdeab13
# Parent  a56a624a8a42b93ec980d2c284756a38719dffe6
bookmarks: introduce binary encoding

Bookmarks binary encoding will be used for `bookmarks` bundle2 part.
The format is: <4 bytes - bookmark size, big-endian>
   <1 byte - 0 if node is empty 1 otherwise><20 bytes node>
BookmarksEncodeError and BookmarksDecodeError maybe thrown if input is
incorrect.

diff --git a/mercurial/bookmarks.py b/mercurial/bookmarks.py
--- a/mercurial/bookmarks.py
+++ b/mercurial/bookmarks.py
@@ -7,8 +7,10 @@
 
 from __future__ import absolute_import
 
+import StringIO
 import errno
 import os
+import struct
 
 from .i18n import _
 from .node import (
@@ -23,6 +25,70 @@
 util,
 )
 
+_NONEMPTYNODE = struct.pack('?', False)
+_EMPTYNODE = struct.pack('?', True)
+
+def _unpackbookmarksize(stream):
+"""Returns 0 if stream is empty.
+"""
+
+expectedsize = struct.calcsize('>i')
+encodedbookmarksize = stream.read(expectedsize)
+if len(encodedbookmarksize) == 0:
+return 0
+if len(encodedbookmarksize) != expectedsize:
+raise ValueError(
+_('cannot decode bookmark size: '
+  'expected size: %d, actual size: %d') %
+(expectedsize, len(encodedbookmarksize)))
+return struct.unpack('>i', encodedbookmarksize)[0]
+
+def encodebookmarks(bookmarks):
+"""Encodes bookmark to node mapping to the byte string.
+
+Format: <4 bytes - bookmark size>
+<1 byte - 0 if node is empty 1 otherwise><20 bytes node>
+Node may be 20 byte binary string, 40 byte hex string or empty
+"""
+
+for bookmark, node in bookmarks.iteritems():
+yield struct.pack('>i', (len(bookmark)))
+yield encoding.fromlocal(bookmark)
+if node:
+if len(node) != 20 and len(node) != 40:
+raise ValueError(_('node must be 20 or bytes long'))
+if len(node) == 40:
+node = bin(node)
+yield _NONEMPTYNODE
+yield node
+else:
+yield _EMPTYNODE
+
+def decodebookmarks(buf):
+"""Decodes buffer into bookmark to node mapping.
+
+Node is either 20 bytes or empty.
+"""
+
+stream = StringIO.StringIO(buf)
+bookmarks = {}
+bookmarksize = _unpackbookmarksize(stream)
+while bookmarksize:
+bookmark = stream.read(bookmarksize)
+if len(bookmark) != bookmarksize:
+raise ValueError(
+_('cannot decode bookmark: expected size: %d, '
+'actual size: %d') % (bookmarksize, len(bookmark)))
+bookmark = encoding.tolocal(bookmark)
+packedemptynodeflag = stream.read(struct.calcsize('?'))
+emptynode = struct.unpack('?', packedemptynodeflag)[0]
+node = ''
+if not emptynode:
+node = stream.read(20)
+bookmarks[bookmark] = node
+bookmarksize = _unpackbookmarksize(stream)
+return bookmarks
+
 def _getbkfile(repo):
 """Hook so that extensions that mess with the store can hook bm storage.
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8 V7] bookmarks: add srchex param to updatefromremote

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478016027 25200
#  Tue Nov 01 09:00:27 2016 -0700
# Branch stable
# Node ID d752daa2b736f0336a18760940104c59d6bd1a5c
# Parent  f3da841e3d47ccbb0be3892b521607c09fdeab13
bookmarks: add srchex param to updatefromremote

diff --git a/mercurial/bookmarks.py b/mercurial/bookmarks.py
--- a/mercurial/bookmarks.py
+++ b/mercurial/bookmarks.py
@@ -501,11 +501,12 @@
 
 return None
 
-def updatefromremote(ui, repo, remotemarks, path, trfunc, explicit=()):
+def updatefromremote(ui, repo, remotemarks, path, trfunc, explicit=(),
+ srchex=None):
 ui.debug("checking for updated bookmarks\n")
 localmarks = repo._bookmarks
 (addsrc, adddst, advsrc, advdst, diverge, differ, invalid, same
- ) = compare(repo, remotemarks, localmarks, dsthex=hex)
+ ) = compare(repo, remotemarks, localmarks, srchex=srchex, dsthex=hex)
 
 status = ui.status
 warn = ui.warn
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 8 V7] bundle2: add `bookmarks` part handler

[Stanislau Hlebik]

# HG changeset patch
# User Stanislau Hlebik 
# Date 1478016027 25200
#  Tue Nov 01 09:00:27 2016 -0700
# Branch stable
# Node ID 2f89680108366512c1a6345abec5cebdb85170ac
# Parent  d752daa2b736f0336a18760940104c59d6bd1a5c
bundle2: add `bookmarks` part handler

Applies bookmarks part to the local repo. `processbookmarksmode` determines
how remote bookmarks are handled. They are either ignored ('ignore' mode),
diverged ('diverge' mode) or applied ('apply' mode).

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -154,7 +154,9 @@
 import sys
 
 from .i18n import _
+from .node import hex
 from . import (
+bookmarks as bookmod,
 changegroup,
 error,
 obsolete,
@@ -287,13 +289,21 @@
 * a way to construct a bundle response when applicable.
 """
 
-def __init__(self, repo, transactiongetter, captureoutput=True):
+def __init__(self, repo, transactiongetter, captureoutput=True,
+ input=None):
+"""
+`input` is a dictionary that is passed to part handlers to tweak
+their behaviour
+"""
 self.repo = repo
 self.ui = repo.ui
 self.records = unbundlerecords()
 self.gettransaction = transactiongetter
 self.reply = None
 self.captureoutput = captureoutput
+if input is None:
+input = {}
+self.input = input
 
 class TransactionUnavailable(RuntimeError):
 pass
@@ -1624,3 +1634,33 @@
 
 cache.write()
 op.ui.debug('applied %i hgtags fnodes cache entries\n' % count)
+
+@parthandler('bookmarks')
+def handlebookmarks(op, inpart):
+"""Processes bookmarks part.
+
+`processbookmarksmode` determines how remote bookmarks are handled. They 
are
+either ignored ('ignore' mode), diverged ('diverge' mode) or applied
+('apply' mode). 'ignore' mode is used to get bookmarks and process them
+later, 'diverge' mode is used to process bookmarks during pull, 'apply'
+mode is used during push.
+"""
+
+bookmarks = {}
+bookmarks = bookmod.decodebookmarks(inpart.read())
+processbookmarksmode = op.input.get('processbookmarksmode', 'ignore')
+if processbookmarksmode == 'apply':
+for bookmark, node in bookmarks.items():
+if node:
+op.repo._bookmarks[bookmark] = node
+else:
+del op.repo._bookmarks[bookmark]
+op.repo._bookmarks.recordchange(op.gettransaction())
+elif processbookmarksmode == 'diverge':
+remotepath = op.input.get('remotepath', '')
+explicitbookmarks = op.input.get('explicitbookmarks', ())
+bookmod.updatefromremote(op.ui, op.repo, bookmarks,
+ remotepath, op.gettransaction,
+ explicit=explicitbookmarks,
+ srchex=hex)
+op.records.add('bookmarks', bookmarks)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

RE: [PATCH STABLE] build: include a dummy $PATH in the custom environment used by build.py

[Gábor STEFANIK]

>


--
This message, including its attachments, is confidential. For more information 
please read NNG's email policy here:
http://www.nng.com/emailpolicy/
By responding to this email you accept the email policy.


-Original Message-
> From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
> On Behalf Of Pierre-Yves David
> Sent: Tuesday, November 1, 2016 4:56 PM
> To: Jun Wu 
> Cc: mercurial-devel@mercurial-scm.org
> Subject: Re: [PATCH STABLE] build: include a dummy $PATH in the custom
> environment used by build.py
>
>
>
> On 11/01/2016 04:39 PM, Jun Wu wrote:
> > Excerpts from Pierre-Yves David's message of 2016-10-29 02:18:56 +0200:
> >> My question is more "What is the meaning and effect of adding "." here?
> >> I'm fine with trying to fix pypiwin32 but we have to be aware of the
> >> actual consequence. A related quest is "What is happening is 'PATH'
> >> is already in the envirement? are we overwriting it?
> >
> > For Windows, PATH='.' is equivalent to PATH='' since Windows searches
> > for PWD by design.
> >
> > For *nix, PATH='' is better. PATH='.' has undesired side-effects.
> >
> > I checked the code, if PATH is already set, it will be dropped by our
> > code because we created a new "env" dict and pass it to "runhg" ->
> > "runcmd" -> "subprocess.Popen(..., env=env)" so the new process does
> > not inherit the old PATH.
> >
> > We rely on "sys.executable" to be correct to find the correct Python.
> >
> > I think having an empty value, or inherit it from os.environ are both
> > acceptable solutions. But it's pypiwin32 to blame anyway.
>
> It seem slike we should at least inherit a value if one exists. This patch 
> does
> not seems ready for inclusion. I'm dropping it from patchwork and we'll
> revisit post release.

I believe the point of using a custom environment here is precisely to avoid 
passing the real $PATH.
Inheriting from the system environment would defeat this.

The only reason for including a PATH entry here is so site.py can do 
os.environ['PATH'] instead of os.environ.get('PATH').
Unfortunately some python modules (like pypiwin32) assume that a $PATH 
environment variable must always exist.

>
> --
> Pierre-Yves David
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 7 V3] statprof: require paths to save or load profile data

[Yuya Nishihara]

On Tue, 01 Nov 2016 19:11:07 -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc 
> # Date 1471227245 25200
> #  Sun Aug 14 19:14:05 2016 -0700
> # Node ID 4cd53260f947639e140a61b0698d9da1032d3794
> # Parent  4f740964f8843bdcb08a668ae1cf37ef153eba25
> statprof: require paths to save or load profile data

> -def load_data(path=None):
> -path = path or (os.environ['HOME'] + '/statprof.data')
> +def load_data(path):
>  lines = open(path, 'r').read().splitlines()

Maybe we need to fix main() to not pass None, as a follow-up.

% ./mercurial/statprof.py hotpath
Traceback (most recent call last):
  File "./mercurial/statprof.py", line 803, in 
sys.exit(main())
  File "./mercurial/statprof.py", line 796, in main
load_data(path=path)
  File "./mercurial/statprof.py", line 338, in load_data
lines = open(path, 'r').read().splitlines()
TypeError: coercing to Unicode: need string or buffer, NoneType found
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 7 V3] statprof: vendor statprof.py

[Yuya Nishihara]

On Tue, 01 Nov 2016 19:11:05 -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc 
> # Date 1478051643 25200
> #  Tue Nov 01 18:54:03 2016 -0700
> # Node ID 60b2d821b58fb1209f7a5a3aac95e66a61e503ea
> # Parent  e5cc44ea12de681d971fcbebb65a7fb71fd1c3c7
> statprof: vendor statprof.py

This series looks good to me, and there was no concern about vendoring
statprof.py in the previous discussion. Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@30252: new changeset

[Mercurial Commits]

New changeset in mercurial:

http://selenic.com/repo/hg//rev/bb5869668189
changeset:   30252:bb5869668189
bookmark:@
tag: tip
parent:  30212:260af19891f2
parent:  30251:e5cc44ea12de
user:Augie Fackler 
date:Tue Nov 01 15:40:21 2016 -0400
summary: merge with stable

-- 
Repository URL: http://selenic.com/repo/hg/
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5] convert: have debugsvnlog obtain standard streams from ui

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1443850496 -32400
#  Sat Oct 03 14:34:56 2015 +0900
# Node ID ed456439699b5af90a3a5ae36a8ac82431448c68
# Parent  5a7ca4f3983a0cd1573c3da450516b6c6a6277e0
# EXP-Topic stdio
convert: have debugsvnlog obtain standard streams from ui

This will help porting to Python 3, where sys.stdin/out/err are unfortunately
unicode streams so we can't use them directly.

diff --git a/hgext/convert/subversion.py b/hgext/convert/subversion.py
--- a/hgext/convert/subversion.py
+++ b/hgext/convert/subversion.py
@@ -5,7 +5,6 @@ from __future__ import absolute_import
 
 import os
 import re
-import sys
 import tempfile
 import xml.dom.minidom
 
@@ -164,8 +163,8 @@ def debugsvnlog(ui, **opts):
 raise error.Abort(_('debugsvnlog could not load Subversion python '
'bindings'))
 
-args = decodeargs(sys.stdin.read())
-get_log_child(sys.stdout, *args)
+args = decodeargs(ui.fin.read())
+get_log_child(ui.fout, *args)
 
 class logstream(object):
 """Interruptible revision log iterator."""
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 5] histedit: use ui.fin to read commands from stdin

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1458346506 25200
#  Fri Mar 18 17:15:06 2016 -0700
# Node ID 6f7e69bc46fc158f2df39f94f6e314fc7a66eca1
# Parent  ed456439699b5af90a3a5ae36a8ac82431448c68
# EXP-Topic stdio
histedit: use ui.fin to read commands from stdin

stdin is wrapped by channeledinput in command-server session.

diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -173,7 +173,6 @@ from __future__ import absolute_import
 
 import errno
 import os
-import sys
 
 from mercurial.i18n import _
 from mercurial import (
@@ -991,9 +990,9 @@ def _getgoal(opts):
 return goaleditplan
 return goalnew
 
-def _readfile(path):
+def _readfile(ui, path):
 if path == '-':
-return sys.stdin.read()
+return ui.fin.read()
 else:
 with open(path, 'rb') as f:
 return f.read()
@@ -1191,7 +1190,7 @@ def _edithisteditplan(ui, repo, state, r
  node.short(state.topmost))
 rules = ruleeditor(repo, ui, state.actions, comment)
 else:
-rules = _readfile(rules)
+rules = _readfile(ui, rules)
 actions = parserules(rules, state)
 ctxs = [repo[act.node] \
 for act in state.actions if act.node]
@@ -1232,7 +1231,7 @@ def _newhistedit(ui, repo, state, revs, 
 actions = [pick(state, r) for r in revs]
 rules = ruleeditor(repo, ui, actions, comment)
 else:
-rules = _readfile(rules)
+rules = _readfile(ui, rules)
 actions = parserules(rules, state)
 warnverifyactions(ui, repo, actions, state, ctxs)
 
diff --git a/tests/test-commandserver.t b/tests/test-commandserver.t
--- a/tests/test-commandserver.t
+++ b/tests/test-commandserver.t
@@ -135,6 +135,19 @@ typical client does not want echo-back m
   summary: 1
   
 
+check that "histedit --commands=-" can read rules from the input channel:
+
+  >>> import cStringIO
+  >>> from hgclient import readchannel, runcommand, check
+  >>> @check
+  ... def serverinput(server):
+  ... readchannel(server)
+  ... rules = 'pick eff892de26ec\n'
+  ... runcommand(server, ['histedit', '0', '--commands=-',
+  ... '--config', 'extensions.histedit='],
+  ...input=cStringIO.StringIO(rules))
+  *** runcommand histedit 0 --commands=- --config extensions.histedit=
+
 check that --cwd doesn't persist between requests:
 
   $ mkdir foo
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5] hgweb: make log streams compatible with command server

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1403928812 -32400
#  Sat Jun 28 13:13:32 2014 +0900
# Node ID 827132690102f124cf5ee9d1888fdddba1987b9d
# Parent  e1a050ebbf75c6f512fe041864a1e53b647c3967
# EXP-Topic stdio
hgweb: make log streams compatible with command server

Even though it would be useless to start a web server by a command server,
it should be doable in principle. Also, we can't use sys.stdout/err directly
on Python 3 because they are unicode streams.

diff --git a/mercurial/hgweb/server.py b/mercurial/hgweb/server.py
--- a/mercurial/hgweb/server.py
+++ b/mercurial/hgweb/server.py
@@ -281,8 +281,8 @@ class MercurialHTTPServer(_mixin, httpse
 prefix = '/' + prefix.strip('/')
 self.prefix = prefix
 
-alog = openlog(ui.config('web', 'accesslog', '-'), sys.stdout)
-elog = openlog(ui.config('web', 'errorlog', '-'), sys.stderr)
+alog = openlog(ui.config('web', 'accesslog', '-'), ui.fout)
+elog = openlog(ui.config('web', 'errorlog', '-'), ui.ferr)
 self.accesslog = alog
 self.errorlog = elog
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5] convert: remove superfluous setbinary() calls from debugsvnlog

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1443850153 -32400
#  Sat Oct 03 14:29:13 2015 +0900
# Node ID 5a7ca4f3983a0cd1573c3da450516b6c6a6277e0
# Parent  bb586966818986131068280bfd95fc96fbdaaa0d
# EXP-Topic stdio
convert: remove superfluous setbinary() calls from debugsvnlog

a3fe91b4f6eb made standard streams set to binary mode globally.

diff --git a/hgext/convert/subversion.py b/hgext/convert/subversion.py
--- a/hgext/convert/subversion.py
+++ b/hgext/convert/subversion.py
@@ -164,8 +164,6 @@ def debugsvnlog(ui, **opts):
 raise error.Abort(_('debugsvnlog could not load Subversion python '
'bindings'))
 
-util.setbinary(sys.stdin)
-util.setbinary(sys.stdout)
 args = decodeargs(sys.stdin.read())
 get_log_child(sys.stdout, *args)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3 STABLE V2] changectx: do not include hidden revisions on short node lookup (issue4964)

[Yuya Nishihara]

On Tue, 1 Nov 2016 16:17:57 +, Jun Wu wrote:
> Excerpts from Pierre-Yves David's message of 2016-11-01 16:59:36 +0100:
> > So, just to confirm, the performance impact will only show up in case 
> > where we would have raised and ambiguity error anyway ? So in the only 
> > behavior/performance impact we'll see is a move from an abort to a 
> > slightly slower lookup. Am I right ?
> 
> Yes. But the slower (much much slower) lookup is likely to end up with the
> same abort error. For example, if the user types a very short hash, which is
> definitely causing a ambiguity, the old code would return instantly, where
> the new one will be slowed down very much.
> 
> Some places even use unfiltered repo as a workaround for the performance
> issue:
> 
>   
> https://www.mercurial-scm.org/repo/hg/file/cac4ca036dff/mercurial/templater.py#l840

Just to make sure, shortest() must avoid the O(N) lookup since it exists in
the common code path, which is different from changectx.__init__(). But I'm
okay to drop this for now, and fix both shortest() and changectx() properly.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 5 of 9 V6] bookmarks: add srchex param to updatefromremote

[Stanislau Hlebik]

I’m using srchex in bookmarks part handler to convert from bin nodes to hex 
nodes, because updatefromremote expect hex nodes.

On 10/14/16, 2:28 AM, "Pierre-Yves David"  
wrote:



On 10/11/2016 06:25 PM, Stanislau Hlebik wrote:
> # HG changeset patch
> # User Stanislau Hlebik 
> # Date 1476197429 25200
> #  Tue Oct 11 07:50:29 2016 -0700
> # Node ID f781756b8de11a6f3e7dd5fd6354e9778defd8c3
> # Parent  718ed86a3698631077a087efaf668d70513056f5
> bookmarks: add srchex param to updatefromremote

I do not understand what the purpose and effect of this changeset is. 
Can you elaborate a little ?

>
> diff --git a/mercurial/bookmarks.py b/mercurial/bookmarks.py
> --- a/mercurial/bookmarks.py
> +++ b/mercurial/bookmarks.py
> @@ -508,11 +508,12 @@
>
>  return None
>
> -def updatefromremote(ui, repo, remotemarks, path, trfunc, explicit=()):
> +def updatefromremote(ui, repo, remotemarks, path, trfunc, explicit=(),
> + srchex=None):
>  ui.debug("checking for updated bookmarks\n")
>  localmarks = repo._bookmarks
>  (addsrc, adddst, advsrc, advdst, diverge, differ, invalid, same
> - ) = compare(repo, remotemarks, localmarks, dsthex=hex)
> + ) = compare(repo, remotemarks, localmarks, srchex=srchex, 
dsthex=hex)
>
>  status = ui.status
>  warn = ui.warn
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel=DQICaQ=5VD0RTtNlTh3ycd41b3MUw=1EQ58Dmb5uX1qHujcsT1Mg=8-TOccCZw4oFT0da5cQ-wA28AAvE6qKjZzPGBL8t-1k=V5DeajSgRhOwuad7CFu_QgJzUJxWLtqkhpa1BOyyA-k=
 
>

-- 
Pierre-Yves David


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel