[PATCH STABLE] record: update help message to use operation instead of "record" (issue5432)

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1493046804 -7200
#  Mon Apr 24 17:13:24 2017 +0200
# Branch stable
# Node ID 208a60229f9bdf04351442c7a2fd66f41b83ea3e
# Parent  0ea1d9a750da142e67690d88644ce4121ed268a1
record: update help message to use operation instead of "record" (issue5432)

Update the hunk selector help message to use the operation name instead
of using "record" for all operations. Extract the help message in the same way
as other single and multiple message line.
Update tests to make sure that both "revert" and "discard" variants are tested.

diff -r 0ea1d9a750da -r 208a60229f9b mercurial/patch.py
--- a/mercurial/patch.pyTue Apr 18 11:10:08 2017 -0700
+++ b/mercurial/patch.pyMon Apr 24 17:13:24 2017 +0200
@@ -992,6 +992,38 @@
 'record': _("record this change to '%s'?"),
 'revert': _("revert this change to '%s'?"),
 }[operation],
+'help': {
+'discard': _('[Ynesfdaq?]'
+  '$$ , discard this change'
+  '$$ , skip this change'
+  '$$  this change manually'
+  '$$  remaining changes to this file'
+  '$$ Discard remaining changes to this '
+  '$$ , skip remaining changes and files'
+  '$$ Discard  changes to all remaining files'
+  '$$ , discarding no changes'
+  '$$ &? (display help)'),
+'record': _('[Ynesfdaq?]'
+  '$$ , record this change'
+  '$$ , skip this change'
+  '$$  this change manually'
+  '$$  remaining changes to this file'
+  '$$ Record remaining changes to this '
+  '$$ , skip remaining changes and files'
+  '$$ Record  changes to all remaining files'
+  '$$ , recording no changes'
+  '$$ &? (display help)'),
+'revert': _('[Ynesfdaq?]'
+  '$$ , revert this change'
+  '$$ , skip this change'
+  '$$  this change manually'
+  '$$  remaining changes to this file'
+  '$$ Revert remaining changes to this '
+  '$$ , skip remaining changes and files'
+  '$$ Revert  changes to all remaining files'
+  '$$ , reverting no changes'
+  '$$ &? (display help)')
+}[operation]
 }
 
 def prompt(skipfile, skipall, query, chunk):
@@ -1010,16 +1042,7 @@
 if skipfile is not None:
 return skipfile, skipfile, skipall, newpatches
 while True:
-resps = _('[Ynesfdaq?]'
-  '$$ , record this change'
-  '$$ , skip this change'
-  '$$  this change manually'
-  '$$  remaining changes to this file'
-  '$$ Record remaining changes to this '
-  '$$ , skip remaining changes and files'
-  '$$ Record  changes to all remaining files'
-  '$$ , recording no changes'
-  '$$ &? (display help)')
+resps = messages['help']
 r = ui.promptchoice("%s %s" % (query, resps))
 ui.write("\n")
 if r == 8: # ?
diff -r 0ea1d9a750da -r 208a60229f9b tests/test-revert-interactive.t
--- a/tests/test-revert-interactive.t   Tue Apr 18 11:10:08 2017 -0700
+++ b/tests/test-revert-interactive.t   Mon Apr 24 17:13:24 2017 +0200
@@ -46,6 +46,7 @@
   > y
   > y
   > y
+  > ?
   > y
   > n
   > n
@@ -88,6 +89,17 @@
3
4
5
+  revert change 3/6 to 'folder1/g'? [Ynesfdaq?] ?
+  
+  y - yes, revert this change
+  n - no, skip this change
+  e - edit this change manually
+  s - skip remaining changes to this file
+  f - revert remaining changes to this file
+  d - done, skip remaining changes and files
+  a - revert all changes to all remaining files
+  q - quit, reverting no changes
+  ? - ? (display help)
   revert change 3/6 to 'folder1/g'? [Ynesfdaq?] y
   
   @@ -1,5 +2,6 @@
@@ -264,6 +276,7 @@
   M folder1/g
   $ hg revert --interactive f << EOF
   > y
+  > ?
   > y
   > n
   > n
@@ -279,6 +292,17 @@
3
4
5
+  discard change 1/2 to 'f'? [Ynesfdaq?] ?
+  
+  y - yes, discard this change
+  n - no, skip this change
+  e - edit this change manually
+  s - skip remaining changes to this file
+  f - discard remaining changes to this file
+  d - done, skip remaining changes and files
+  a - discard all changes to all remaining files
+  q - quit, discarding no changes
+  ? - ? (display help)
   discard change 1/2 to 'f'? [Ynesfdaq?] y
   
   @@ -2,6 +1,5 @@

[PATCH 2 of 2 pypy] Fix failing test-devel-warnings.t with Pypy5.6.0

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1501145576 -7200
#  Thu Jul 27 10:52:56 2017 +0200
# Branch stable
# Node ID f0e35277398ceab8b569651acab44de8e15eef18
# Parent  9dfca91aab1f705652beeb9c29d213d8ff8b25aa
# EXP-Topic pypy5.6.0-compat
Fix failing test-devel-warnings.t with Pypy5.6.0

In Pypy 5.6.0, traceback exception classes are not displayed with their full
qualified name.

Instead of displaying mercurial.error.ProgrammingError, Pypy displays
ProgrammingError.

Update the test to support both version.

diff -r 9dfca91aab1f -r f0e35277398c tests/test-devel-warnings.t
--- a/tests/test-devel-warnings.t   Mon Jul 31 17:43:45 2017 +0200
+++ b/tests/test-devel-warnings.t   Thu Jul 27 10:52:56 2017 +0200
@@ -129,7 +129,7 @@
   $ hg commit -m a
   $ hg stripintr 2>&1 | egrep -v '^(\*\*|  )'
   Traceback (most recent call last):
-  mercurial.error.ProgrammingError: cannot strip from inside a transaction
+  *ProgrammingError: cannot strip from inside a transaction (glob)
 
   $ hg oldanddeprecated
   devel-warn: foorbar is deprecated, go shopping
@@ -187,7 +187,7 @@
   ** Extensions loaded: * (glob)
   ** ProgrammingError: transaction requires locking
   Traceback (most recent call last):
-  mercurial.error.ProgrammingError: transaction requires locking
+  *ProgrammingError: transaction requires locking (glob)
 
   $ hg programmingerror 2>&1 | egrep -v '^  '
   ** Unknown exception encountered with possibly-broken third-party extension 
buggylocking
@@ -200,7 +200,7 @@
   ** ProgrammingError: something went wrong
   ** (try again)
   Traceback (most recent call last):
-  mercurial.error.ProgrammingError: something went wrong
+  *ProgrammingError: something went wrong (glob)
 
 Old style deprecation warning
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 pypy] Fix failing test files with Pypy5.6.0

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1501515825 -7200
#  Mon Jul 31 17:43:45 2017 +0200
# Branch stable
# Node ID 9dfca91aab1f705652beeb9c29d213d8ff8b25aa
# Parent  025017423e532037e470e3e5e9b845cba5c4e710
# EXP-Topic pypy5.6.0-compat
Fix failing test files with Pypy5.6.0

Pypy 5.6.0 saves cached bytecode files in __pycache__ directory, clean them in
tests to fix loading old test extensions code.

Doing so should also helps for Python3.x migration.

diff -r 025017423e53 -r 9dfca91aab1f tests/test-commandserver.t
--- a/tests/test-commandserver.tFri Jul 21 10:46:31 2017 -0400
+++ b/tests/test-commandserver.tMon Jul 31 17:43:45 2017 +0200
@@ -234,7 +234,9 @@
   *** runcommand --config hooks.pre-identify=python:hook.hook id
   eff892de26ec tip
 
+Clean hook cached version
   $ rm hook.py*
+  $ rm -Rf __pycache__
 
   $ echo a >> a
   >>> import os
diff -r 025017423e53 -r 9dfca91aab1f tests/test-fncache.t
--- a/tests/test-fncache.t  Fri Jul 21 10:46:31 2017 -0400
+++ b/tests/test-fncache.t  Mon Jul 31 17:43:45 2017 +0200
@@ -270,7 +270,11 @@
   > cmdtable = {}
   > 
   > EOF
+
+Clean cached version
   $ rm -f "${extpath}c"
+  $ rm -Rf "$(dirname $extpath)/__pycache__"
+
   $ touch z
   $ hg ci -qAm z
   transaction abort!
@@ -305,7 +309,11 @@
   > cmdtable = {}
   > 
   > EOF
+
+Clean cached versions
   $ rm -f "${extpath}c"
+  $ rm -Rf "$(dirname $extpath)/__pycache__"
+
   $ hg up -q 1
   $ touch z
   $ hg ci -qAm z 2>/dev/null
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: 4.3 freeze over, release delayed

[Boris Feld]

Will the security fix be backported in a 4.2.3 release for people that
are still on Python 2.6?

Cheers,
Boris

On Tue, 2017-08-01 at 10:32 -0400, Augie Fackler wrote:
> Howdy folks,
> 
> We've got an upcoming security fix that needs to be coordinated with
> other packages, so we're not releasing 4.3 today. We *are*, however,
> unfreezing and you're welcome to send patches for default (which will
> become 4.4) even as we're waiting for the all-clear to release 4.3.
> 
> Thanks,
> Augie
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D243: obsmarker: rename precnode into prednode

[Boris Feld]

On Mon, 2017-08-14 at 11:50 +, Gábor STEFANIK wrote:
> > -Original Message-
> > From: Mercurial-devel [mailto:mercurial-devel-bounces@mercurial-scm
> > .org]
> > On Behalf Of lothiraldan (Boris Feld)
> > Sent: Wednesday, August 9, 2017 7:20 PM
> > To: mercurial-devel@mercurial-scm.org
> > Subject: D243: obsmarker: rename precnode into prednode
> > 
> > lothiraldan updated this revision to Diff 705.
> > 
> > REPOSITORY
> >   rHG Mercurial
> > 
> > CHANGES SINCE LAST UPDATE
> >   https://phab.mercurial-scm.org/D243?vs=699=705
> > 
> > REVISION DETAIL
> >   https://phab.mercurial-scm.org/D243
> > 
> > AFFECTED FILES
> >   mercurial/cmdutil.py
> >   mercurial/obsutil.py
> > 
> > CHANGE DETAILS
> > 
> > diff --git a/mercurial/obsutil.py b/mercurial/obsutil.py
> > --- a/mercurial/obsutil.py
> > +++ b/mercurial/obsutil.py
> > @@ -9,6 +9,7 @@
> > 
> >  from . import (
> >  phases,
> > +util
> >  )
> > 
> >  class marker(object):
> > @@ -29,15 +30,21 @@
> >  return self._data == other._data
> > 
> >  def precnode(self):
> > -"""Precursor changeset node identifier"""
> > +msg = ("'marker.precnode' is deprecated, "
> > +   "use 'marker.precnode'")
> 
> precnode is deprecated, use precnode instead? is that a typo?

Good catch! I will send a follow-up.

> 
> > +util.nouideprecwarn(msg, '4.4')
> > +return self.prednode()
> > +
> > +def prednode(self):
> > +"""Predecessor changeset node identifier"""
> >  return self._data[0]
> > 
> >  def succnodes(self):
> >  """List of successor changesets node identifiers"""
> >  return self._data[1]
> > 
> >  def parentnodes(self):
> > -"""Parents of the precursors (None if not recorded)"""
> > +"""Parents of the predecessors (None if not recorded)"""
> >  return self._data[5]
> > 
> >  def metadata(self):
> > diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
> > --- a/mercurial/cmdutil.py
> > +++ b/mercurial/cmdutil.py
> > @@ -1913,7 +1913,7 @@
> >  To be used by debug function."""
> >  if index is not None:
> >  fm.write('index', '%i ', index)
> > -fm.write('precnode', '%s ', hex(marker.precnode()))
> > +fm.write('precnode', '%s ', hex(marker.prednode()))
> >  succs = marker.succnodes()
> >  fm.condwrite(succs, 'succnodes', '%s ',
> >   fm.formatlist(map(hex, succs), name='node'))
> > 
> > 
> > 
> > To: lothiraldan, #hg-reviewers, dsp
> > Cc: mercurial-devel
> > ___
> > Mercurial-devel mailing list
> > Mercurial-devel@mercurial-scm.org
> > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> 
> 
>  This message, including its attachments, is confidential and the
> property of NNG Llc. For more information please read NNG's email
> policy here:
> http://www.nng.com/emailpolicy/
> By responding to this email you accept the email policy.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] templatekw: specify plural form of instability

[Boris Feld]

LGTM

On Wed, 2017-08-16 at 14:22 +0900, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1502601148 -32400
> #  Sun Aug 13 14:12:28 2017 +0900
> # Node ID 50723ff5912ecde3f8e43bea4b84db03bae7f1f0
> # Parent  6cd0eafbd52b4e0efcad037876dc9bdf6c9bbfbb
> templatekw: specify plural form of instability
> 
> Follows up 40739aef97f7.
> 
> diff --git a/mercurial/templatekw.py b/mercurial/templatekw.py
> --- a/mercurial/templatekw.py
> +++ b/mercurial/templatekw.py
> @@ -783,7 +783,8 @@ def showinstabilities(**args):
>  (EXPERIMENTAL)
>  """
>  args = pycompat.byteskwargs(args)
> -return showlist('instability', args['ctx'].instabilities(),
> args)
> +return showlist('instability', args['ctx'].instabilities(),
> args,
> +plural='instabilities')
>  
>  # tell hggettext to extract docstrings from these functions:
>  i18nfunctions = keywords.values()
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D242: context: rename troubled into isunstable

[Boris Feld]

On Wed, 2017-08-09 at 12:32 -0400, Brandon McCaig wrote:
> On Wed, Aug 09, 2017 at 03:37:26PM +, lothiraldan (Boris Feld)
> wrote:
> > diff --git a/mercurial/context.py b/mercurial/context.py ---
> > a/mercurial/context.py +++ b/mercurial/context.py @@ -240,6
> > +240,12 @@ return self.rev() in obsmod.getrevs(self._repo,
> > 'divergent')
> >  
> >  def troubled(self):
> > +msg = ("'context.troubled' is deprecated, "
> > +   "use 'context.isunstable'")
> > +self._repo.ui.deprecwarn(msg, '4.4')
> > +return self.unstable()
> 
> Maybe I'm missing something, but shouldn't this be:
> 
> return self.isunstable()
> 
> It sounded like context.unstable() has a different purpose and so
> troubled was renamed to isunstable and so troubled() itself
> should also be calling isunstable().

Yes, you are right. I am sending a fix right now on phabricator. Thank
you for the catch.

> 
> > +
> > +def isunstable(self):
> >  """True if the changeset is either unstable, bumped or
> > divergent"""
> >  return self.orphan() or self.phasedivergent() or
> > self.contentdivergent()
> 
> Regards,
> 
> 
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 13 config] configitems: register the 'notify.diffstat' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498786999 -7200
#  Fri Jun 30 03:43:19 2017 +0200
# Node ID 0980f220e3d3e8d6737d2a8c7b468491b0e1cd52
# Parent  f60ab18ff3fd401c460755b77001a9fa1bbf6977
# EXP-Topic config.register.notify
configitems: register the 'notify.diffstat' config

diff -r f60ab18ff3fd -r 0980f220e3d3 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:18 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:19 2017 +0200
@@ -161,6 +161,9 @@
 configitem('notify', 'config',
 default=None,
 )
+configitem('notify', 'diffstat',
+default=True,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -368,7 +371,7 @@
 opts=patch.diffallopts(self.ui))
 difflines = ''.join(chunks).splitlines()
 
-if self.ui.configbool('notify', 'diffstat', True):
+if self.ui.configbool('notify', 'diffstat'):
 s = patch.diffstat(difflines)
 # s may be nil, don't include the header if it is
 if s:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 13 config] configitems: register the 'notify.merge' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787006 -7200
#  Fri Jun 30 03:43:26 2017 +0200
# Node ID aafac577cb93e119e24855779e31f82d7879bc6f
# Parent  bdd8d57b64f9aa5cc154a8ab5fe771d2caa09ee8
# EXP-Topic config.register.notify
configitems: register the 'notify.merge' config

diff -r bdd8d57b64f9 -r aafac577cb93 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:25 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:26 2017 +0200
@@ -179,6 +179,9 @@
 configitem('notify', 'mbox',
 default=None,
 )
+configitem('notify', 'merge',
+default=True,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -220,7 +223,7 @@
 self.test = self.ui.configbool('notify', 'test', True)
 self.charsets = mail._charsets(self.ui)
 self.subs = self.subscribers()
-self.merge = self.ui.configbool('notify', 'merge', True)
+self.merge = self.ui.configbool('notify', 'merge')
 
 mapfile = None
 template = (self.ui.config('notify', hooktype) or
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 13 config] configitems: register the 'notify.strip' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787008 -7200
#  Fri Jun 30 03:43:28 2017 +0200
# Node ID 7d5c8af4fd4a2ffdf41588a61509f73cfc8f3ea5
# Parent  99b3015b7ca492d785a3ac4351fc6812e80a3bc4
# EXP-Topic config.register.notify
configitems: register the 'notify.strip' config

diff -r 99b3015b7ca4 -r 7d5c8af4fd4a hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:27 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:28 2017 +0200
@@ -185,6 +185,9 @@
 configitem('notify', 'sources',
 default='serve',
 )
+configitem('notify', 'strip',
+default=0,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -219,7 +222,7 @@
 if cfg:
 self.ui.readconfig(cfg, sections=['usersubs', 'reposubs'])
 self.repo = repo
-self.stripcount = int(self.ui.config('notify', 'strip', 0))
+self.stripcount = int(self.ui.config('notify', 'strip'))
 self.root = self.strip(self.repo.root)
 self.domain = self.ui.config('notify', 'domain')
 self.mbox = self.ui.config('notify', 'mbox')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 13 of 13 config] configitems: register the 'notify.test' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787011 -7200
#  Fri Jun 30 03:43:31 2017 +0200
# Node ID a382bad84d9d3fe4d72025c755c084b3f6704ca1
# Parent  58446c552d9ec37c4c55d776f4681aa5906d35cb
# EXP-Topic config.register.notify
configitems: register the 'notify.test' config

diff -r 58446c552d9e -r a382bad84d9d hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:30 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:31 2017 +0200
@@ -194,6 +194,9 @@
 configitem('notify', 'template',
 default=None,
 )
+configitem('notify', 'test',
+default=True,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -232,7 +235,7 @@
 self.root = self.strip(self.repo.root)
 self.domain = self.ui.config('notify', 'domain')
 self.mbox = self.ui.config('notify', 'mbox')
-self.test = self.ui.configbool('notify', 'test', True)
+self.test = self.ui.configbool('notify', 'test')
 self.charsets = mail._charsets(self.ui)
 self.subs = self.subscribers()
 self.merge = self.ui.configbool('notify', 'merge')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 13 config] configitems: register the 'notify.domain' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787000 -7200
#  Fri Jun 30 03:43:20 2017 +0200
# Node ID 4621f29dbbbc655355efd0fa3ecaf5c86340791e
# Parent  0980f220e3d3e8d6737d2a8c7b468491b0e1cd52
# EXP-Topic config.register.notify
configitems: register the 'notify.domain' config

diff -r 0980f220e3d3 -r 4621f29dbbbc hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:19 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:20 2017 +0200
@@ -164,6 +164,9 @@
 configitem('notify', 'diffstat',
 default=True,
 )
+configitem('notify', 'domain',
+default=None,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 12 of 13 config] configitems: register the 'notify.template' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787010 -7200
#  Fri Jun 30 03:43:30 2017 +0200
# Node ID 58446c552d9ec37c4c55d776f4681aa5906d35cb
# Parent  83ee4bc7827a739c8dc3f38d41c2866ba0f12ff4
# EXP-Topic config.register.notify
configitems: register the 'notify.template' config

diff -r 83ee4bc7827a -r 58446c552d9e hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:29 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:30 2017 +0200
@@ -191,6 +191,9 @@
 configitem('notify', 'style',
 default=None,
 )
+configitem('notify', 'template',
+default=None,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 13 config] configitems: register the 'notify.maxdiff' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787003 -7200
#  Fri Jun 30 03:43:23 2017 +0200
# Node ID 54e8106d6945392e46360bd285e4504f66b968be
# Parent  21153b5c2b932b7b481281664e087a9c1f6f7bbd
# EXP-Topic config.register.notify
configitems: register the 'notify.maxdiff' config

diff -r 21153b5c2b93 -r 54e8106d6945 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:22 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:23 2017 +0200
@@ -170,6 +170,9 @@
 configitem('notify', 'fromauthor',
 default=None,
 )
+configitem('notify', 'maxdiff',
+default=300,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -367,7 +370,7 @@
 
 def diff(self, ctx, ref=None):
 
-maxdiff = int(self.ui.config('notify', 'maxdiff', 300))
+maxdiff = int(self.ui.config('notify', 'maxdiff'))
 prev = ctx.p1().node()
 if ref:
 ref = ref.node()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 13 config] configitems: register the 'notify.maxsubject' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787004 -7200
#  Fri Jun 30 03:43:24 2017 +0200
# Node ID b1fdeb6a53df2ef2a999b44858c13aa55b2a01eb
# Parent  54e8106d6945392e46360bd285e4504f66b968be
# EXP-Topic config.register.notify
configitems: register the 'notify.maxsubject' config

diff -r 54e8106d6945 -r b1fdeb6a53df hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:23 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:24 2017 +0200
@@ -173,6 +173,9 @@
 configitem('notify', 'maxdiff',
 default=300,
 )
+configitem('notify', 'maxsubject',
+default=67,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -336,7 +339,7 @@
 else:
 s = ctx.description().lstrip().split('\n', 1)[0].rstrip()
 subject = '%s: %s' % (self.root, s)
-maxsubject = int(self.ui.config('notify', 'maxsubject', 67))
+maxsubject = int(self.ui.config('notify', 'maxsubject'))
 if maxsubject:
 subject = util.ellipsis(subject, maxsubject)
 msg['Subject'] = mail.headencode(self.ui, subject,
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 13 config] configitems: register the 'notify.sources' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787007 -7200
#  Fri Jun 30 03:43:27 2017 +0200
# Node ID 99b3015b7ca492d785a3ac4351fc6812e80a3bc4
# Parent  aafac577cb93e119e24855779e31f82d7879bc6f
# EXP-Topic config.register.notify
configitems: register the 'notify.sources' config

diff -r aafac577cb93 -r 99b3015b7ca4 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:26 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:27 2017 +0200
@@ -182,6 +182,9 @@
 configitem('notify', 'merge',
 default=True,
 )
+configitem('notify', 'sources',
+default='serve',
+)
 
 # template for single changeset can include email headers.
 single_template = '''
@@ -294,7 +297,7 @@
 
 def skipsource(self, source):
 '''true if incoming changes from this source should be skipped.'''
-ok_sources = self.ui.config('notify', 'sources', 'serve').split()
+ok_sources = self.ui.config('notify', 'sources').split()
 return source not in ok_sources
 
 def send(self, ctx, count, data):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 13 config] configitems: register the 'notify.mbox' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787005 -7200
#  Fri Jun 30 03:43:25 2017 +0200
# Node ID bdd8d57b64f9aa5cc154a8ab5fe771d2caa09ee8
# Parent  b1fdeb6a53df2ef2a999b44858c13aa55b2a01eb
# EXP-Topic config.register.notify
configitems: register the 'notify.mbox' config

diff -r b1fdeb6a53df -r bdd8d57b64f9 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:24 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:25 2017 +0200
@@ -176,6 +176,9 @@
 configitem('notify', 'maxsubject',
 default=67,
 )
+configitem('notify', 'mbox',
+default=None,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 13 config] configitems: register the 'notify.fromauthor' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787002 -7200
#  Fri Jun 30 03:43:22 2017 +0200
# Node ID 21153b5c2b932b7b481281664e087a9c1f6f7bbd
# Parent  4621f29dbbbc655355efd0fa3ecaf5c86340791e
# EXP-Topic config.register.notify
configitems: register the 'notify.fromauthor' config

diff -r 4621f29dbbbc -r 21153b5c2b93 hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:20 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:22 2017 +0200
@@ -167,6 +167,9 @@
 configitem('notify', 'domain',
 default=None,
 )
+configitem('notify', 'fromauthor',
+default=None,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 01 of 13 config] configitems: register the 'notify.config' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498786998 -7200
#  Fri Jun 30 03:43:18 2017 +0200
# Node ID f60ab18ff3fd401c460755b77001a9fa1bbf6977
# Parent  02a745c20121f9add702f0200dc5522e912c2d20
# EXP-Topic config.register.notify
configitems: register the 'notify.config' config

diff -r 02a745c20121 -r f60ab18ff3fd hgext/notify.py
--- a/hgext/notify.py   Thu Aug 10 18:55:33 2017 -0400
+++ b/hgext/notify.py   Fri Jun 30 03:43:18 2017 +0200
@@ -145,6 +145,7 @@
 error,
 mail,
 patch,
+registrar,
 util,
 )
 
@@ -154,6 +155,13 @@
 # leave the attribute unspecified.
 testedwith = 'ships-with-hg-core'
 
+configtable = {}
+configitem = registrar.configitem(configtable)
+
+configitem('notify', 'config',
+default=None,
+)
+
 # template for single changeset can include email headers.
 single_template = '''
 Subject: changeset in {webroot}: {desc|firstline|strip}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 11 of 13 config] configitems: register the 'notify.style' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787009 -7200
#  Fri Jun 30 03:43:29 2017 +0200
# Node ID 83ee4bc7827a739c8dc3f38d41c2866ba0f12ff4
# Parent  7d5c8af4fd4a2ffdf41588a61509f73cfc8f3ea5
# EXP-Topic config.register.notify
configitems: register the 'notify.style' config

diff -r 7d5c8af4fd4a -r 83ee4bc7827a hgext/notify.py
--- a/hgext/notify.py   Fri Jun 30 03:43:28 2017 +0200
+++ b/hgext/notify.py   Fri Jun 30 03:43:29 2017 +0200
@@ -188,6 +188,9 @@
 configitem('notify', 'strip',
 default=0,
 )
+configitem('notify', 'style',
+default=None,
+)
 
 # template for single changeset can include email headers.
 single_template = '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Mercurial 4.3 and 4.2.3 released

[Boris Feld]

On Thu, 2017-08-10 at 14:09 -0400, Augie Fackler wrote:
> Moments ago, I released Mercurial 4.3 and 4.2.3. Please patch
> *immedately*:
> 
> CVE-2017-1000115:
> 
> Mercurial's symlink auditing was incomplete prior to 4.3, and could
> be abused to write to files outside the repository.
> 
> CVE-2017-1000116:
> 
> Mercurial was not sanitizing hostnames passed to ssh, allowing shell
> injection attacks by specifying a hostname starting with
> -oProxyCommand. This is also present in Git (CVE-2017-1000117) and
> Subversion (CVE-2017-9800), so please patch those tools as well if
> you have them installed. All three tools are doing their security
> release today.
> 
> Please update your packaged builds as soon as practical.
> 
> Note that since we dropped Python 2.6 and these issues are pretty
> bad, we did the back port to 4.2.3. We may not do further 4.2
> releases, so please plan around Python 2.7 in the near future if you
> haven't already.
> 
> Thanks!
> Augie

Thank you Augie for the release and thank you Yuja, Sean and Jun for
the security fixes!

We had to backport the patches for Mercurial 4.1.3 for some customers.

We made them available in case someone else needs them:

https://bitbucket.org/octobus/mercurial-backport/branch/backport-4.
1.

Sincerely,
Boris Feld

> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D242: context: rename troubled into isunstable

[Boris Feld]

On Sun, 2017-08-13 at 05:01 +, yuja (Yuya Nishihara) wrote:
> yuja added inline comments.
> 
> INLINE COMMENTS
> 
> > context.py:246
> > +self._repo.ui.deprecwarn(msg, '4.4')
> > +return self.isunstable()
> > +
> 
> Please send a follow-up. This is too far from the tip to amend.

I will do send a follow-up, thank you for catching that, I thought it
had been pushed into commited.

> 
> REPOSITORY
>   rHG Mercurial
> 
> REVISION DETAIL
>   https://phab.mercurial-scm.org/D242
> 
> To: lothiraldan, #hg-reviewers
> Cc: yuja, mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH website] downloads: update latest python 2.6 compatible release

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1502712731 -7200
#  Mon Aug 14 14:12:11 2017 +0200
# Node ID ddff2dfaa572d7b5da7c412da521c793fe57518a
# Parent  a3901266377812a15c68295882030bb6f77bb338
downloads: update latest python 2.6 compatible release

Replace link from 4.2.2 to 4.2.3 after the security exceptional release.

diff -r a39012663778 -r ddff2dfaa572 templates/downloads/index.html
--- a/templates/downloads/index.htmlFri Jul 28 22:28:24 2017 -0700
+++ b/templates/downloads/index.htmlMon Aug 14 14:12:11 2017 +0200
@@ -79,7 +79,7 @@
   https://www.mercurial-scm.org/wiki/SupportedPythonVersions;>Supported
   Python Versions on the wiki.
   Python 2.6
-  Mercurial 4.2.2
+  Mercurial 4.2.3
   is the last release to support Python 2.6. Use this if you need
   to run Mercurial on old platforms and you cannot update your
   Python installation.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] template: rename successorssets template into successorgroup

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499454363 -7200
#  Fri Jul 07 21:06:03 2017 +0200
# Node ID 508801f7cd1304488b0d9ebf02a76be030608b21
# Parent  e714159860fd0872ae0555bb07546aa7e9f700e0
# EXP-Topic renamesuccessosrssetstemplate
template: rename successorssets template into successorgroup

The new name seems better and easier to remember for users.

diff -r e714159860fd -r 508801f7cd13 mercurial/templatekw.py
--- a/mercurial/templatekw.py   Fri Jul 07 08:33:10 2017 +0200
+++ b/mercurial/templatekw.py   Fri Jul 07 21:06:03 2017 +0200
@@ -602,8 +602,8 @@
lambda x: {'ctx': repo[x], 'revcache': {}},
lambda d: _formatrevnode(d['ctx']))
 
-@templatekeyword("successorssets")
-def showsuccessorssets(repo, ctx, **args):
+@templatekeyword("successorgroup")
+def showsuccessorgroup(repo, ctx, **args):
 """Returns a string of sets of successors for a changectx
 
 Format used is: [ctx1, ctx2], [ctx3] if ctx has been splitted into ctx1 and
diff -r e714159860fd -r 508801f7cd13 tests/test-obsmarker-template.t
--- a/tests/test-obsmarker-template.t   Fri Jul 07 08:33:10 2017 +0200
+++ b/tests/test-obsmarker-template.t   Fri Jul 07 21:06:03 2017 +0200
@@ -17,9 +17,9 @@
   > {if(predecessors, "\n  semi-colon: {join(predecessors, "; ")}")}\
   > {if(predecessors, "\n  json: {predecessors|json}")}\
   > {if(predecessors, "\n  map: {join(predecessors % "{rev}:{node}", " 
")}")}\
-  > {if(successorssets, "\n  Successors: {successorssets}")}\
-  > {if(successorssets, "\n  multi-line: {join(successorssets, "\n  
multi-line: ")}")}\
-  > {if(successorssets, "\n  json: {successorssets|json}")}\n'
+  > {if(successorgroup, "\n  Successors: {successorgroup}")}\
+  > {if(successorgroup, "\n  multi-line: {join(successorgroup, "\n  
multi-line: ")}")}\
+  > {if(successorgroup, "\n  json: {successorgroup|json}")}\n'
   > EOF
 
 Test templates on amended commit
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 10] configitems: register the 'bugzilla.bzdir' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414604 -7200
#  Fri Jul 07 10:03:24 2017 +0200
# Node ID 4802d0b9e2ef4906f7114f01eaf87711dcdf3c40
# Parent  ff78bedcf36af5517172a74a19f2a223c0128c4f
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzdir' config

diff -r ff78bedcf36a -r 4802d0b9e2ef hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:22 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:24 2017 +0200
@@ -322,6 +322,9 @@
 configitem('bugzilla', 'apikey',
 default='',
 )
+configitem('bugzilla', 'bzdir',
+default='/var/www/html/bugzilla',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -457,8 +460,7 @@
 for id in bugs.keys():
 self.ui.status(_('  bug %s\n') % id)
 cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)
-bzdir = self.ui.config('bugzilla', 'bzdir',
-   '/var/www/html/bugzilla')
+bzdir = self.ui.config('bugzilla', 'bzdir')
 try:
 # Backwards-compatible with old notify string, which
 # took one string. This will throw with a new format
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 10] configitems: register the 'bugzilla.bzuser' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414611 -7200
#  Fri Jul 07 10:03:31 2017 +0200
# Node ID 2a7fec2040efcbb15afa77caf58e2023755be38a
# Parent  3246453035686bafc15f262bf35f7cb18f755f59
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzuser' config

diff -r 324645303568 -r 2a7fec2040ef hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:28 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:31 2017 +0200
@@ -331,6 +331,9 @@
 configitem('bugzilla', 'bzurl',
 default='http://localhost/bugzilla/',
 )
+configitem('bugzilla', 'bzuser',
+default=None,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 10] configitems: register the 'bugzilla.fixresolution' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414740 -7200
#  Fri Jul 07 10:05:40 2017 +0200
# Node ID c272cde1e7f4353121b69cbda1ab254e89db81a2
# Parent  19f7670da89197d4770c474f65bcc8427e1e07ad
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixresolution' config

diff -r 19f7670da891 -r c272cde1e7f4 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:36 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:05:40 2017 +0200
@@ -340,6 +340,9 @@
 configitem('bugzilla', 'fixregexp',
 default=lambda: bugzilla._default_fix_re,
 )
+configitem('bugzilla', 'fixresolution',
+default='FIXED',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -668,8 +671,7 @@
 passwd = self.ui.config('bugzilla', 'password')
 
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
-self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
-'FIXED')
+self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
 ver = self.bzproxy.Bugzilla.version()['version'].split('.')
@@ -827,8 +829,7 @@
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
-self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
-'FIXED')
+self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 def apiurl(self, targets, include_fields=None):
 url = '/'.join([self.bzroot] + [str(t) for t in targets])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 10] configitems: register the 'bugzilla.fixstatus' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414637 -7200
#  Fri Jul 07 10:03:57 2017 +0200
# Node ID f1eb434550a89f9705be8835409e1e0e6ff7a035
# Parent  c272cde1e7f4353121b69cbda1ab254e89db81a2
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixstatus' config

diff -r c272cde1e7f4 -r f1eb434550a8 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:05:40 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:57 2017 +0200
@@ -343,6 +343,9 @@
 configitem('bugzilla', 'fixresolution',
 default='FIXED',
 )
+configitem('bugzilla', 'fixstatus',
+default='RESOLVED',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -670,7 +673,7 @@
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
 
-self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
+self.fixstatus = self.ui.config('bugzilla', 'fixstatus')
 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
@@ -828,7 +831,7 @@
 self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
-self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
+self.fixstatus = self.ui.config('bugzilla', 'fixstatus')
 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 def apiurl(self, targets, include_fields=None):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 12 of 14] vfs: add the possibility to have a "ward" to check vfs usage

[Boris Feld]

On Fri, 2017-07-07 at 21:48 +0900, Yuya Nishihara wrote:
> On Thu, 06 Jul 2017 20:53:50 +0200, Boris Feld wrote:
> > On Wed, 2017-07-05 at 23:55 +0900, Yuya Nishihara wrote:
> > > On Sun, 02 Jul 2017 04:56:37 +0200, Pierre-Yves David wrote:
> > > > # HG changeset patch
> > > > # User Pierre-Yves David <pierre-yves.da...@ens-lyon.org>
> > > > # Date 1470323266 -7200
> > > > #  Thu Aug 04 17:07:46 2016 +0200
> > > > # Node ID ebf81efdd6d7ff15c64683933635616c00475688
> > > > # Parent  34b8be7f0420b07d0f7b71379c6055e5b26223d5
> > > > # EXP-Topic vfs.ward
> > > > # Available At https://www.mercurial-scm.org/repo/users/marmout
> > > > e/me
> > > > rcurial/
> > > > #  hg pull https://www.mercurial-scm.org/repo/users
> > > > /mar
> > > > moute/mercurial/ -r ebf81efdd6d7
> > > > vfs: add the possibility to have a "ward" to check vfs usage
> > > > The feature has some similarity with the 'pathauditor', but
> > > > further
> > > > digging
> > > > show it make more sense to keep them separated. The path
> > > > auditor is
> > > > fairly
> > > > autonomous (create from vfs.__init__ without any extra
> > > > information), and check
> > > > very generic details. On the other hand, the wards will have
> > > > deeper
> > > > knownledge
> > > > of the Mercurial logic and is created at the local repository
> > > > level. Mixing the
> > > > two would add much complexity for no real gains.
> > > 
> > > My gut feeling is that vfs isn't the right place if they require
> > > deeper
> > > knowledge of repository/dirstate logic. And the whole weakref
> > > business
> > > floating around wards, hooks and transaction seems like just
> > > getting
> > > around
> > > layering violation.
> > 
> > We pondered on this choice a long time and couldn't find any other
> > layer that is used by both Mercurial core and the extensions to
> > access
> > the file system. It seems to be the highest abstraction layer we
> > could
> > hook into without introducing a new one and updating all callers.
> > 
> > Our reflexion is, as the vfs classes comes from the scmutil file,
> > it
> > seems okay to have scm related logic in that layer.
> 
> IIRC, one possible alternative was to move lock itself to a vfs as
> the
> whole vfs is theoretically covered by the lock.

The practice divert a bit from the theory. For example, the
".hg/bookmark" file is in theory covered by the 'wlock'. In practice it
is now covered by the transaction and therefor the 'lock'. On the other
hand there are multiple files handled by the vfs ('hgrc', 'dirstate',
'store/', etc) that are not covered by the wlock.

Right now, all these exceptions are stored on the repository, and
extension can update them. To us, the repository layers seems the right
layer for logic regarding individual file semantic. We also want to add
some logic regarding open transaction to the ward, something unrelated
to the vfs.

The initial motivation for this series (beside catching bug) is to make
it safer to introduce new vfs. For example, the share extensions
suffers from lack of cache sharing, each share has its own '.hg/cache/'
directory. Introducing a 'repo.cvfs' dedicated to cache could fix that,
but we wants to warn people still accessing 'cache/' through
'repo.vfs'.

To push that logic further, "share" actually needs a better distinction
between the working copy specific piece and the more generic pieces. An
extra split of "repo.vfs" could take care of that. However, some of the
file, like bookmark, can be shared or not depending on some config and
share option. So the ward needs access to that logic (already living on
the repository) too.

We need to eventually rework the repository format to a more race free
data structure. This will likely involve more vfs to access data
dispatched in various directory (eg: append-only vs full update). The
ward should help a lot here but it need to be aware of many high level
logic regarding file semantic.

We plan to add more vfs-s and the current locking design seems better
than having one lock per vfs, as multiple vfs will be covered by the
same lock. If we move the locks on the vfs layer, we either would need
to update it again later or introduce much more complex code to manage
the interaction between vfs-s locks.

> 
> BTW, the ward itself doesn't require deeper knowledge of repo object
> right
> now. Neither does the auditor. How will the ward API be evolved to
> depend
> on

Re: [PATCH 5 of 5] template: add successors template

[Boris Feld]

On Fri, 2017-07-07 at 10:29 -0700, Jun Wu wrote:
> Excerpts from Sean Farley's message of 2017-07-06 11:05:22 -0700:
> > Augie Fackler <r...@durin42.com> writes:
> > > On Wed, Jul 05, 2017 at 10:53:58PM +0200, Boris Feld wrote:
> > > I'm not...super in love with the name successorssets, but I have
> > > no
> > > better ideas. I'd welcome some bikeshedding in the next ten days.
> > 
> > Off the top of my head:
> > 
> > successorgroup
> > successorlike
> > similarsuccessors
> > successorkin
> 
> "group" looks better to me than "similar". "similar" sounds like
> there is
> some filtering logic that removes non-similar ones.

I agree with Jun that similar might send indicates that the successors
are selected or processed in a specific way.

> 
> "successorsset" looks okay to me, since "set" and "group" are not
> that much
> different.

Successorgroup sound good to me, it is less implementation specific and
might be easier to understand for users.

> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 10] configitems: register the 'bugzilla.bzurl' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414608 -7200
#  Fri Jul 07 10:03:28 2017 +0200
# Node ID 3246453035686bafc15f262bf35f7cb18f755f59
# Parent  afee98344d67275a6bf631fc89629ca3e46e7166
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzurl' config

diff -r afee98344d67 -r 324645303568 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:26 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:28 2017 +0200
@@ -328,6 +328,9 @@
 configitem('bugzilla', 'bzemail',
 default=None,
 )
+configitem('bugzilla', 'bzurl',
+default='http://localhost/bugzilla/',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -649,8 +652,7 @@
 def __init__(self, ui):
 bzaccess.__init__(self, ui)
 
-bzweb = self.ui.config('bugzilla', 'bzurl',
-   'http://localhost/bugzilla/')
+bzweb = self.ui.config('bugzilla', 'bzurl')
 bzweb = bzweb.rstrip("/") + "/xmlrpc.cgi"
 
 user = self.ui.config('bugzilla', 'user', 'bugs')
@@ -810,8 +812,7 @@
 """
 def __init__(self, ui):
 bzaccess.__init__(self, ui)
-bz = self.ui.config('bugzilla', 'bzurl',
-'http://localhost/bugzilla/')
+bz = self.ui.config('bugzilla', 'bzurl')
 self.bzroot = '/'.join([bz, 'rest'])
 self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10] configitems: register the 'bugzilla.db' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414614 -7200
#  Fri Jul 07 10:03:34 2017 +0200
# Node ID dcec97037a786b460a7705b00cb2821568bb5842
# Parent  2a7fec2040efcbb15afa77caf58e2023755be38a
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.db' config

diff -r 2a7fec2040ef -r dcec97037a78 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:31 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:34 2017 +0200
@@ -334,6 +334,9 @@
 configitem('bugzilla', 'bzuser',
 default=None,
 )
+configitem('bugzilla', 'db',
+default='bugs',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -412,7 +415,7 @@
 host = self.ui.config('bugzilla', 'host', 'localhost')
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
-db = self.ui.config('bugzilla', 'db', 'bugs')
+db = self.ui.config('bugzilla', 'db')
 timeout = int(self.ui.config('bugzilla', 'timeout', 5))
 self.ui.note(_('connecting to %s:%s as %s, password %s\n') %
  (host, db, user, '*' * len(passwd)))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 10] configitems: register the 'bugzilla.bzemail' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414606 -7200
#  Fri Jul 07 10:03:26 2017 +0200
# Node ID afee98344d67275a6bf631fc89629ca3e46e7166
# Parent  4802d0b9e2ef4906f7114f01eaf87711dcdf3c40
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzemail' config

diff -r 4802d0b9e2ef -r afee98344d67 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:24 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:26 2017 +0200
@@ -325,6 +325,9 @@
 configitem('bugzilla', 'bzdir',
 default='/var/www/html/bugzilla',
 )
+configitem('bugzilla', 'bzemail',
+default=None,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10] configitems: register the 'bugzilla.fixregexp' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414616 -7200
#  Fri Jul 07 10:03:36 2017 +0200
# Node ID 19f7670da89197d4770c474f65bcc8427e1e07ad
# Parent  dcec97037a786b460a7705b00cb2821568bb5842
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixregexp' config

diff -r dcec97037a78 -r 19f7670da891 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:34 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:36 2017 +0200
@@ -337,6 +337,9 @@
 configitem('bugzilla', 'db',
 default='bugs',
 )
+configitem('bugzilla', 'fixregexp',
+default=lambda: bugzilla._default_fix_re,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -975,8 +978,7 @@
 self.ui.config('bugzilla', 'regexp',
bugzilla._default_bug_re), re.IGNORECASE)
 self.fix_re = re.compile(
-self.ui.config('bugzilla', 'fixregexp',
-   bugzilla._default_fix_re), re.IGNORECASE)
+self.ui.config('bugzilla', 'fixregexp'), re.IGNORECASE)
 self.split_re = re.compile(r'\D+')
 
 def find_bugs(self, ctx):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 01 of 10] configitems: register the 'bugzilla.apikey' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414602 -7200
#  Fri Jul 07 10:03:22 2017 +0200
# Node ID ff78bedcf36af5517172a74a19f2a223c0128c4f
# Parent  89796a25d4bb91fb418ad3e70faad2c586902ffb
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.apikey' config

diff -r 89796a25d4bb -r ff78bedcf36a hgext/bugzilla.py
--- a/hgext/bugzilla.py Mon Jul 03 11:22:00 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:22 2017 +0200
@@ -303,6 +303,7 @@
 cmdutil,
 error,
 mail,
+registrar,
 url,
 util,
 )
@@ -315,6 +316,13 @@
 # leave the attribute unspecified.
 testedwith = 'ships-with-hg-core'
 
+configtable = {}
+configitem = registrar.configitem(configtable)
+
+configitem('bugzilla', 'apikey',
+default='',
+)
+
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
 
@@ -800,7 +808,7 @@
 bz = self.ui.config('bugzilla', 'bzurl',
 'http://localhost/bugzilla/')
 self.bzroot = '/'.join([bz, 'rest'])
-self.apikey = self.ui.config('bugzilla', 'apikey', '')
+self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 10] configitems: register the 'bugzilla.host' config

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499414641 -7200
#  Fri Jul 07 10:04:01 2017 +0200
# Node ID 686908f9b45d95c674edccd189f3c6f4814dbfb4
# Parent  f1eb434550a89f9705be8835409e1e0e6ff7a035
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.host' config

diff -r f1eb434550a8 -r 686908f9b45d hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:57 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:04:01 2017 +0200
@@ -346,6 +346,9 @@
 configitem('bugzilla', 'fixstatus',
 default='RESOLVED',
 )
+configitem('bugzilla', 'host',
+default='localhost',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -421,7 +424,7 @@
 
 bzaccess.__init__(self, ui)
 
-host = self.ui.config('bugzilla', 'host', 'localhost')
+host = self.ui.config('bugzilla', 'host')
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
 db = self.ui.config('bugzilla', 'db')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 5 of 5] template: add successors template

[Boris Feld]

On Fri, 2017-07-07 at 14:52 -0400, Augie Fackler wrote:
> I think successorgroup is a bit better as well. Can someone mail a
> followup?

I will do
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4] configitems: register the 'color.pagermode' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499895388 -7200
#  Wed Jul 12 23:36:28 2017 +0200
# Node ID ad57894ea839e08f6b4f2dd3eb5c779c4746202e
# Parent  afad2f9ae04d35cef5165b07a2987b0e5604924b
# EXP-Topic config.register.special-case
configitems: register the 'color.pagermode' config

diff -r afad2f9ae04d -r ad57894ea839 mercurial/configitems.py
--- a/mercurial/configitems.py  Wed Jul 12 23:36:10 2017 +0200
+++ b/mercurial/configitems.py  Wed Jul 12 23:36:28 2017 +0200
@@ -79,6 +79,9 @@
 coreconfigitem('color', 'mode',
 default='auto',
 )
+coreconfigitem('color', 'pagermode',
+default=dynamicdefault,
+)
 coreconfigitem('devel', 'all-warnings',
 default=False,
 )
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 4] configitems: register the 'worker.backgroundclose' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787157 -7200
#  Fri Jun 30 03:45:57 2017 +0200
# Node ID f62a1b71aa6959c3be5a88afe574b056b9ebec5a
# Parent  5f2e71c738b41b8a864ae8c7b38eff41985ebd99
# EXP-Topic config.register.special-case
configitems: register the 'worker.backgroundclose' config

diff -r 5f2e71c738b4 -r f62a1b71aa69 mercurial/configitems.py
--- a/mercurial/configitems.py  Fri Jun 30 03:44:05 2017 +0200
+++ b/mercurial/configitems.py  Fri Jun 30 03:45:57 2017 +0200
@@ -208,6 +208,9 @@
 coreconfigitem('ui', 'username',
 alias=[('ui', 'user')]
 )
+coreconfigitem('worker', 'backgroundclose',
+default=dynamicdefault,
+)
 # Windows defaults to a limit of 512 open files. A buffer of 128
 # should give us enough headway.
 coreconfigitem('worker', 'backgroundclosemaxqueue',
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 4] configitems: handle case were the default value is not static

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499895370 -7200
#  Wed Jul 12 23:36:10 2017 +0200
# Node ID afad2f9ae04d35cef5165b07a2987b0e5604924b
# Parent  68e9762a357b19ec7751683e2fb80266c0d260ce
# EXP-Topic config.register.special-case
configitems: handle case were the default value is not static

In some case, the default of one value is derived from other value. We add a
way to register them anyway and an associated devel-warning.

The registration is very naive for the moment. We might be able to have a
better way for registering each of these cases but it could be done later.

diff -r 68e9762a357b -r afad2f9ae04d mercurial/configitems.py
--- a/mercurial/configitems.py  Mon Jul 10 23:09:52 2017 +0900
+++ b/mercurial/configitems.py  Wed Jul 12 23:36:10 2017 +0200
@@ -51,6 +51,9 @@
 raise error.ProgrammingError(msg % (item.section, item.name))
 section[item.name] = item
 
+# special value for case where the default is derived from other values
+dynamicdefault = object()
+
 # Registering actual config items
 
 def getitemregister(configtable):
diff -r 68e9762a357b -r afad2f9ae04d mercurial/ui.py
--- a/mercurial/ui.py   Mon Jul 10 23:09:52 2017 +0900
+++ b/mercurial/ui.py   Wed Jul 12 23:36:10 2017 +0200
@@ -457,11 +457,17 @@
 if default is _unset:
 if item is None:
 value = default
+elif item.default is configitems.dynamicdefault:
+value = None
+msg = "config item requires an explicit default value: '%s.%s'"
+msg %= (section, name)
+self.develwarn(msg, 2, 'warn-config-default')
 elif callable(item.default):
 value = item.default()
 else:
 value = item.default
-elif item is not None:
+elif (item is not None
+  and item.default is not configitems.dynamicdefault):
 msg = ("specifying a default value for a registered "
"config item: '%s.%s' '%s'")
 msg %= (section, name, default)
diff -r 68e9762a357b -r afad2f9ae04d tests/test-devel-warnings.t
--- a/tests/test-devel-warnings.t   Mon Jul 10 23:09:52 2017 +0900
+++ b/tests/test-devel-warnings.t   Wed Jul 12 23:36:10 2017 +0200
@@ -200,7 +200,7 @@
   $ cat << EOF > ${TESTTMP}/buggyconfig.py
   > """A small extension that tests our developer warnings for config"""
   > 
-  > from mercurial import registrar
+  > from mercurial import registrar, configitems
   > 
   > cmdtable = {}
   > command = registrar.command(cmdtable)
@@ -209,6 +209,7 @@
   > configitem = registrar.configitem(configtable)
   > 
   > configitem('test', 'some', default='foo')
+  > configitem('test', 'dynamic', default=configitems.dynamicdefault)
   > # overwrite a core config
   > configitem('ui', 'quiet', default=False)
   > configitem('ui', 'interactive', default=None)
@@ -218,6 +219,8 @@
   > repo.ui.config('ui', 'quiet', False)
   > repo.ui.config('ui', 'interactive', None)
   > repo.ui.config('test', 'some', 'foo')
+  > repo.ui.config('test', 'dynamic', 'some-required-default')
+  > repo.ui.config('test', 'dynamic')
   > EOF
 
   $ hg --config "extensions.buggyconfig=${TESTTMP}/buggyconfig.py" buggyconfig
@@ -226,5 +229,6 @@
   devel-warn: specifying a default value for a registered config item: 
'ui.quiet' 'False' at: $TESTTMP/buggyconfig.py:* (cmdbuggyconfig) (glob)
   devel-warn: specifying a default value for a registered config item: 
'ui.interactive' 'None' at: $TESTTMP/buggyconfig.py:* (cmdbuggyconfig) (glob)
   devel-warn: specifying a default value for a registered config item: 
'test.some' 'foo' at: $TESTTMP/buggyconfig.py:* (cmdbuggyconfig) (glob)
+  devel-warn: config item requires an explicit default value: 'test.dynamic' 
at: $TESTTMP/buggyconfig.py:* (cmdbuggyconfig) (glob)
 
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 4] configitems: register the 'progress.width' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1498787045 -7200
#  Fri Jun 30 03:44:05 2017 +0200
# Node ID 5f2e71c738b41b8a864ae8c7b38eff41985ebd99
# Parent  ad57894ea839e08f6b4f2dd3eb5c779c4746202e
# EXP-Topic config.register.special-case
configitems: register the 'progress.width' config

diff -r ad57894ea839 -r 5f2e71c738b4 mercurial/configitems.py
--- a/mercurial/configitems.py  Wed Jul 12 23:36:28 2017 +0200
+++ b/mercurial/configitems.py  Fri Jun 30 03:44:05 2017 +0200
@@ -163,6 +163,9 @@
 coreconfigitem('progress', 'estimate',
 default=2,
 )
+coreconfigitem('progress', 'width',
+default=dynamicdefault,
+)
 coreconfigitem('server', 'bundle1',
 default=True,
 )
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3 V3] vfs: allow to pass more argument to audit

[Boris Feld]

On Fri, 2017-07-14 at 00:28 +0900, Yuya Nishihara wrote:
> On Tue, 11 Jul 2017 17:55:27 +0200, Boris Feld wrote:
> > # HG changeset patch
> > # User Boris Feld <boris.f...@octobus.net>
> > # Date 1499768878 -7200
> > #  Tue Jul 11 12:27:58 2017 +0200
> > # Node ID 9addf65400ec8e6052a399c1586320988d73a321
> > # Parent  4672db164c986da4442bd864cd044512d975c3f2
> > # EXP-Topic vfs.ward
> > vfs: allow to pass more argument to audit
> 
> Looks generally good.
> 
> > We want to be able to do more precise check when auditing a path
> > depending of
> > the intend of the file access (eg read versus write). So we now
> > pass the 'mode'
> > and 'atomictemp' value to 'audit' and update the audit function to
> > accept them.
> > 
> > This will be put to use in the next changeset.
> > 
> > diff -r 4672db164c98 -r 9addf65400ec mercurial/pathutil.py
> > --- a/mercurial/pathutil.py Sat Jun 24 15:29:42 2017 -0700
> > +++ b/mercurial/pathutil.py Tue Jul 11 12:27:58 2017 +0200
> > @@ -46,7 +46,7 @@
> >  else:
> >  self.normcase = lambda x: x
> >  
> > -def __call__(self, path):
> > +def __call__(self, path, mode=None, atomictemp=None):
> >  '''Check the relative path.
> >  path may contain a pattern (e.g. foodir/**.txt)'''
> >  
> > diff -r 4672db164c98 -r 9addf65400ec mercurial/vfs.py
> > --- a/mercurial/vfs.py  Sat Jun 24 15:29:42 2017 -0700
> > +++ b/mercurial/vfs.py  Tue Jul 11 12:27:58 2017 +0200
> > @@ -306,7 +306,7 @@
> >  if audit:
> >  self.audit = pathutil.pathauditor(self.base)
> >  else:
> > -self.audit = util.always
> > +self.audit =  (lambda path, mode=None,
> > atomictemp=None: True)
> >  self.createmode = None
> >  self._trustnlink = None
> >  
> > @@ -360,7 +360,7 @@
> >  r = util.checkosfilename(path)
> >  if r:
> >  raise error.Abort("%s: %r" % (r, path))
> > -self.audit(path)
> > +self.audit(path, mode=mode, atomictemp=atomictemp)
> 
> Is 'atomictemp' needed? I don't think 'atomictemp' can be generalized
> well to
> the other vfs operations. And atomictemp=True doesn't mean
> repo.lock/wlock
> is unnecessary.

atomictemp wasn't technically necessary for this series, I included it
for completeness. If people starts wrapping pathauditor, adding
atomictemp later would results in breaking their wraps.

Do you want me to send a V4 to remove the atomictemp?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 11] localrepo: use the 'registernew' function to set the phase of new commit

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499727927 -7200
#  Tue Jul 11 01:05:27 2017 +0200
# Node ID 210688a668f0ff9f8395b2789b29fdb2a1de2c64
# Parent  f700a612e24139e8ee411d208791fc6d4ac9bab3
# EXP-Topic tr.changes.phases
localrepo: use the 'registernew' function to set the phase of new commit

diff -r f700a612e241 -r 210688a668f0 mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 03:47:25 2017 +0200
+++ b/mercurial/localrepo.pyTue Jul 11 01:05:27 2017 +0200
@@ -1872,7 +1872,7 @@
 # be compliant anyway
 #
 # if minimal phase was 0 we don't need to retract anything
-phases.retractboundary(self, tr, targetphase, [n])
+phases.registernew(self, tr, targetphase, [n])
 tr.close()
 return n
 finally:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 11] phases: extract the core of boundary retraction in '_retractboundary'

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499723416 -7200
#  Mon Jul 10 23:50:16 2017 +0200
# Node ID 2a23f55b1b3487c7d43c0709efa900c01235b3cf
# Parent  d578e17d880990375cf64d4c026e927987211fe7
# EXP-Topic tr.changes.phases
phases: extract the core of boundary retraction in '_retractboundary'

At the moment the 'retractboundary' function is called for multiple reasons:

First, actually retracting boundaries. There are only two cases for theses:
'hg phase --force' and 'hg qimport'. This will need extra graph computation to
retrieve the phase changes.

Second, setting the phases of newly added changesets. In this case we already
know all the affected nodes and we just needs to register different
information (old phase is None).

Third, when reducing the set of roots when advancing phase. The phase are
already properly tracked so we do not needs anything else in this case.

To deal with this difference in phase tracking, we extract the core logic into
a private method that all three cases can use.

diff -r d578e17d8809 -r 2a23f55b1b34 mercurial/phases.py
--- a/mercurial/phases.py   Tue Jul 11 02:39:52 2017 +0200
+++ b/mercurial/phases.py   Mon Jul 10 23:50:16 2017 +0200
@@ -331,10 +331,14 @@
 delroots.extend(olds - roots)
 # declare deleted root in the target phase
 if targetphase != 0:
-self.retractboundary(repo, tr, targetphase, delroots)
+self._retractboundary(repo, tr, targetphase, delroots)
 repo.invalidatevolatilesets()
 
 def retractboundary(self, repo, tr, targetphase, nodes):
+self._retractboundary(repo, tr, targetphase, nodes)
+repo.invalidatevolatilesets()
+
+def _retractboundary(self, repo, tr, targetphase, nodes):
 # Be careful to preserve shallow-copied values: do not update
 # phaseroots values, replace them.
 
@@ -343,6 +347,7 @@
 newroots = [n for n in nodes
 if self.phase(repo, repo[n].rev()) < targetphase]
 if newroots:
+
 if nullid in newroots:
 raise error.Abort(_('cannot change null revision phase'))
 currentroots = currentroots.copy()
@@ -360,7 +365,6 @@
 finalroots.update(ctx.node() for ctx in updatedroots)
 
 self._updateroots(targetphase, finalroots, tr)
-repo.invalidatevolatilesets()
 
 def filterunknown(self, repo):
 """remove unknown nodes from the phase boundary
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 11] phases: track phase changes from 'retractboundary'

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499883060 -7200
#  Wed Jul 12 20:11:00 2017 +0200
# Node ID 14007d8eba4dd21240bb243e45394fa0dc70545b
# Parent  d90b8e445f0f48ebcac362fb1df6f5498f814f90
# EXP-Topic tr.changes.phases
phases: track phase changes from 'retractboundary'

We adds new computation to find and record the revision affected by the
boundary retraction. This add more complication to the function but this seems
fine since it is only used in a couple of rare and explicit cases (`hg phase
--force` and `hg qimport`).

Having strong tracking of phase changes is worth the effort.

diff -r d90b8e445f0f -r 14007d8eba4d mercurial/phases.py
--- a/mercurial/phases.py   Wed Jul 12 23:15:09 2017 +0200
+++ b/mercurial/phases.py   Wed Jul 12 20:11:00 2017 +0200
@@ -348,7 +348,30 @@
 repo.invalidatevolatilesets()
 
 def retractboundary(self, repo, tr, targetphase, nodes):
-self._retractboundary(repo, tr, targetphase, nodes)
+oldroots = self.phaseroots[:targetphase + 1]
+if tr is None:
+phasetracking = None
+else:
+phasetracking = tr.changes.get('phases')
+repo = repo.unfiltered()
+if (self._retractboundary(repo, tr, targetphase, nodes)
+and phasetracking is not None):
+
+# find the affected revisions
+new = self.phaseroots[targetphase]
+old = oldroots[targetphase]
+affected = set(repo.revs('(%ln::) - (%ln::)', new, old))
+
+# find the phase of the affected revision
+for phase in xrange(targetphase, -1, -1):
+if phase:
+roots = oldroots[phase]
+revs = set(repo.revs('%ln::%ld', roots, affected))
+affected -= revs
+else: # public phase
+revs = affected
+for r in revs:
+_trackphasechange(phasetracking, r, phase, targetphase)
 repo.invalidatevolatilesets()
 
 def _retractboundary(self, repo, tr, targetphase, nodes):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 11] phases: add a 'registernew' method to set new phases

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499737645 -7200
#  Tue Jul 11 03:47:25 2017 +0200
# Node ID f700a612e24139e8ee411d208791fc6d4ac9bab3
# Parent  2a23f55b1b3487c7d43c0709efa900c01235b3cf
# EXP-Topic tr.changes.phases
phases: add a 'registernew' method to set new phases

This new function will be used by code that adds new changesets. It ajusts the
phase boundary to make sure added changesets are at least in their target
phase (they end up in an higher phase if their parents are in a higher phase).

Having a dedicated function also simplify the phases tracking. All the new
nodes are passed as argument, so we know that all of them needs to have their
new phase registered. We also know that no other nodes will be affected, so no
extra computation are needed.

This function differ from 'retractboundary' where some nodes might change
phase while some other might not. It can also affect nodes not passed as
parameters.

These simplification also apply to the computation itself. For now we use
'_retractboundary' there by convenience, but we may introduces simpler code
later.

While registering new revisions, we still need to check the actual phases of
the added node because it might be higher than the target phase (eg: target is
draft but parent is secret).

We will migrate users over the next changesets.

diff -r 2a23f55b1b34 -r f700a612e241 mercurial/phases.py
--- a/mercurial/phases.py   Mon Jul 10 23:50:16 2017 +0200
+++ b/mercurial/phases.py   Tue Jul 11 03:47:25 2017 +0200
@@ -294,6 +294,19 @@
 tr.addfilegenerator('phase', ('phaseroots',), self._write)
 tr.hookargs['phases_moved'] = '1'
 
+def registernew(self, repo, tr, targetphase, nodes):
+repo = repo.unfiltered()
+self._retractboundary(repo, tr, targetphase, nodes)
+if tr is not None and 'phases' in tr.changes:
+phasetracking = tr.changes['phases']
+torev = repo.changelog.rev
+phase = self.phase
+for n in nodes:
+rev = torev(n)
+revphase = phase(repo, rev)
+_trackphasechange(phasetracking, rev, None, revphase)
+repo.invalidatevolatilesets()
+
 def advanceboundary(self, repo, tr, targetphase, nodes):
 """Set all 'nodes' to phase 'targetphase'
 
@@ -417,6 +430,16 @@
 phcache.retractboundary(repo, tr, targetphase, nodes)
 repo._phasecache.replace(phcache)
 
+def registernew(repo, tr, targetphase, nodes):
+"""register a new revision and its phase
+
+Code adding revisions to the repository should use this function to
+set new changeset in their target phase (or higher).
+"""
+phcache = repo._phasecache.copy()
+phcache.registernew(repo, tr, targetphase, nodes)
+repo._phasecache.replace(phcache)
+
 def listphases(repo):
 """List phases root for serialization over pushkey"""
 # Use ordered dictionary so behavior is deterministic.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 11] phases: rework phase movement code in 'cg.apply' to use 'registernew'

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499728656 -7200
#  Tue Jul 11 01:17:36 2017 +0200
# Node ID c60ee8233517c9e84d5907ea2822f16667d31ca8
# Parent  d331c624636b7c88f54170c7d600e6d4480aa23b
# EXP-Topic tr.changes.phases
phases: rework phase movement code in 'cg.apply' to use 'registernew'

We rework the code to call 'registernew' before any other phase advancement.
This make 'changegroup.apply' register correct phase movement for the added
and bundled nodes.

diff -r d331c624636b -r c60ee8233517 mercurial/changegroup.py
--- a/mercurial/changegroup.py  Tue Jul 11 00:59:23 2017 +0200
+++ b/mercurial/changegroup.py  Tue Jul 11 01:17:36 2017 +0200
@@ -356,6 +356,7 @@
 repo.hook('pretxnchangegroup', throw=True, **hookargs)
 
 added = [cl.node(r) for r in xrange(clstart, clend)]
+phaseall = None
 if srctype in ('push', 'serve'):
 # Old servers can not push the boundary themselves.
 # New servers won't push the boundary if changeset already
@@ -364,16 +365,19 @@
 # We should not use added here but the list of all change in
 # the bundle
 if repo.publishing():
-phases.advanceboundary(repo, tr, phases.public, cgnodes)
+targetphase = phaseall = phases.public
 else:
+# closer target phase computation
+
 # Those changesets have been pushed from the
 # outside, their phases are going to be pushed
 # alongside. Therefor `targetphase` is
 # ignored.
-phases.advanceboundary(repo, tr, phases.draft, cgnodes)
-phases.retractboundary(repo, tr, phases.draft, added)
-else:
-phases.retractboundary(repo, tr, targetphase, added)
+targetphase = phaseall = phases.draft
+if added:
+phases.registernew(repo, tr, targetphase, added)
+if phaseall is not None:
+phases.advanceboundary(repo, tr, phaseall, cgnodes)
 
 if changesets > 0:
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 11 of 11] phases: test phases tracking at the transaction level

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499891988 -7200
#  Wed Jul 12 22:39:48 2017 +0200
# Node ID 0ccb45f63c27d373c02a87e6544e684db27659fe
# Parent  14007d8eba4dd21240bb243e45394fa0dc70545b
# EXP-Topic tr.changes.phases
phases: test phases tracking at the transaction level

Now that we have all tracking in place, the data in `tr.changes['phases']`
dictionary should be correct and we should test it.

It is a bit late in the cycle to discuss to add any public API (eg: hooks)
that expose the data to the user, so we just add a small test extension
displaying the data. It is enabled for the phases tests.

New output have been manually checked for consistency.

diff -r 14007d8eba4d -r 0ccb45f63c27 tests/test-phases-exchange.t
--- a/tests/test-phases-exchange.t  Wed Jul 12 20:11:00 2017 +0200
+++ b/tests/test-phases-exchange.t  Wed Jul 12 22:39:48 2017 +0200
@@ -1,5 +1,10 @@
 #require killdaemons
 
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > phasereport=$TESTDIR/testlib/ext-phase-report.py
+  > EOF
+
   $ hgph() { hg log -G --template "{rev} {phase} {desc} - {node|short}\n" $*; }
 
   $ mkcommit() {
@@ -13,9 +18,13 @@
   $ hg init alpha
   $ cd alpha
   $ mkcommit a-A
+  test-debug-phase: new rev 0:  x -> 1
   $ mkcommit a-B
+  test-debug-phase: new rev 1:  x -> 1
   $ mkcommit a-C
+  test-debug-phase: new rev 2:  x -> 1
   $ mkcommit a-D
+  test-debug-phase: new rev 3:  x -> 1
   $ hgph
   @  3 draft a-D - b555f63b6063
   |
@@ -34,6 +43,10 @@
   adding manifests
   adding file changes
   added 2 changesets with 2 changes to 2 files
+  test-debug-phase: new rev 0:  x -> 0
+  test-debug-phase: new rev 1:  x -> 0
+  test-debug-phase: move rev 0: 1 -> 0
+  test-debug-phase: move rev 1: 1 -> 0
   $ hgph
   @  3 draft a-D - b555f63b6063
   |
@@ -52,6 +65,7 @@
   
   $ hg up -q
   $ mkcommit b-A
+  test-debug-phase: new rev 2:  x -> 1
   $ hgph
   @  2 draft b-A - f54f1bb90ff3
   |
@@ -66,6 +80,8 @@
   adding manifests
   adding file changes
   added 2 changesets with 2 changes to 2 files (+1 heads)
+  test-debug-phase: new rev 3:  x -> 0
+  test-debug-phase: new rev 4:  x -> 0
   (run 'hg heads' to see heads, 'hg merge' to merge)
   $ hgph
   o  4 public a-D - b555f63b6063
@@ -96,6 +112,7 @@
   pushing to ../beta
   searching for changes
   no changes found
+  test-debug-phase: move rev 2: 1 -> 0
   [1]
   $ hgph
   @  3 draft a-D - b555f63b6063
@@ -110,6 +127,7 @@
   pushing to ../beta
   searching for changes
   no changes found
+  test-debug-phase: move rev 3: 1 -> 0
   [1]
   $ hgph
   @  3 public a-D - b555f63b6063
@@ -130,6 +148,7 @@
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files (+1 heads)
+  test-debug-phase: new rev 4:  x -> 0
   (run 'hg heads' to see heads, 'hg merge' to merge)
 
   $ cd ../beta
@@ -148,6 +167,7 @@
   pulling from ../alpha
   searching for changes
   no changes found
+  test-debug-phase: move rev 2: 1 -> 0
   $ hgph
   o  4 public a-D - b555f63b6063
   |
@@ -182,6 +202,11 @@
   adding manifests
   adding file changes
   added 5 changesets with 5 changes to 5 files (+1 heads)
+  test-debug-phase: new rev 0:  x -> 1
+  test-debug-phase: new rev 1:  x -> 1
+  test-debug-phase: new rev 2:  x -> 1
+  test-debug-phase: new rev 3:  x -> 1
+  test-debug-phase: new rev 4:  x -> 1
   (run 'hg heads' to see heads, 'hg merge' to merge)
   $ hgph
   o  4 draft a-D - b555f63b6063
@@ -210,6 +235,9 @@
   adding manifests
   adding file changes
   added 3 changesets with 3 changes to 3 files
+  test-debug-phase: new rev 0:  x -> 1
+  test-debug-phase: new rev 1:  x -> 1
+  test-debug-phase: new rev 2:  x -> 1
   (run 'hg update' to get a working copy)
   $ hgph
   o  2 draft a-C - 54acac6f23ab
@@ -228,6 +256,7 @@
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files (+1 heads)
+  test-debug-phase: new rev 3:  x -> 1
   (run 'hg heads' to see heads, 'hg merge' to merge)
   $ hgph
   o  3 draft b-A - f54f1bb90ff3
@@ -250,6 +279,10 @@
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
+  test-debug-phase: move rev 0: 1 -> 0
+  test-debug-phase: move rev 1: 1 -> 0
+  test-debug-phase: move rev 2: 1 -> 0
+  test-debug-phase: new rev 4:  x -> 0
   (run 'hg update' to get a working copy)
   $ hgph # f54f1bb90ff3 stay draft, not ancestor of -r
   o  4 public a-D - b555f63b6063
@@ -267,7 +300,9 @@
 
   $ hg up -q f54f1bb90ff3
   $ mkcommit n-A
+  test-debug-phase: new rev 5:  x -> 1
   $ mkcommit n-B
+  test-debug-phase: new rev 6:  x -> 1
   $ hgph
   @  6 draft n-B - 145e75495359
   |
@@ -291,6 +326,12 @@
   adding manifests
   adding file changes
   added 2 changesets with 2 changes to 2 files
+  test-debug-phase: move rev 0: 1 -> 0
+  test-debug-phase: move rev 1: 1 -> 0
+  test-debug-phase: move rev 3: 1 -&g

[PATCH 09 of 11] phases: detect when boundaries has been actually retracted

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499894109 -7200
#  Wed Jul 12 23:15:09 2017 +0200
# Node ID d90b8e445f0f48ebcac362fb1df6f5498f814f90
# Parent  c60ee8233517c9e84d5907ea2822f16667d31ca8
# EXP-Topic tr.changes.phases
phases: detect when boundaries has been actually retracted

It is useful to detect noop and avoid expensive operations in this case.

We return the information to inform the caller of a possible update. Top level
function might need to react to the phase update (eg: invalidating some
caches, tracking phase change).

diff -r c60ee8233517 -r d90b8e445f0f mercurial/phases.py
--- a/mercurial/phases.py   Tue Jul 11 01:17:36 2017 +0200
+++ b/mercurial/phases.py   Wed Jul 12 23:15:09 2017 +0200
@@ -357,6 +357,7 @@
 
 repo = repo.unfiltered()
 currentroots = self.phaseroots[targetphase]
+finalroots = oldroots = set(currentroots)
 newroots = [n for n in nodes
 if self.phase(repo, repo[n].rev()) < targetphase]
 if newroots:
@@ -376,8 +377,10 @@
 finalroots = set(n for n in currentroots if repo[n].rev() <
  minnewroot)
 finalroots.update(ctx.node() for ctx in updatedroots)
-
+if finalroots != oldroots:
 self._updateroots(targetphase, finalroots, tr)
+return True
+return False
 
 def filterunknown(self, repo):
 """remove unknown nodes from the phase boundary
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 01 of 11] phase: put retractboundary out of the loop in advanceboundary

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499718162 -7200
#  Mon Jul 10 22:22:42 2017 +0200
# Node ID 8c771f0d6e8d70b088abc2c31dc59be0d58ab9fa
# Parent  50243c975fc2ee605ebac493f4ab18d37117e46a
# EXP-Topic tr.changes.phases
phase: put retractboundary out of the loop in advanceboundary

It seems that we were calling retractboundary for each phases to process.
Putting the retractboundary out of the loop reduce the number of calls,
helping tracking the phases changes.

diff -r 50243c975fc2 -r 8c771f0d6e8d mercurial/phases.py
--- a/mercurial/phases.py   Tue Jul 11 05:06:01 2017 +0200
+++ b/mercurial/phases.py   Mon Jul 10 22:22:42 2017 +0200
@@ -301,9 +301,9 @@
 self._updateroots(phase, roots, tr)
 # some roots may need to be declared for lower phases
 delroots.extend(olds - roots)
-# declare deleted root in the target phase
-if targetphase != 0:
-self.retractboundary(repo, tr, targetphase, delroots)
+# declare deleted root in the target phase
+if targetphase != 0:
+self.retractboundary(repo, tr, targetphase, delroots)
 repo.invalidatevolatilesets()
 
 def retractboundary(self, repo, tr, targetphase, nodes):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 11] phases: track phase movements in 'advanceboundary'

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499733592 -7200
#  Tue Jul 11 02:39:52 2017 +0200
# Node ID d578e17d880990375cf64d4c026e927987211fe7
# Parent  936da074fe26e45133672419d670febbfdb447a8
# EXP-Topic tr.changes.phases
phases: track phase movements in 'advanceboundary'

Makes advanceboundary record the phase movement of affected revisions in
tr.changes['phases'].

The tracking is not usable yet because the 'retractboundary' function can also
affect phases.

We'll improve that in the coming changesets.

diff -r 936da074fe26 -r d578e17d8809 mercurial/localrepo.py
--- a/mercurial/localrepo.pyMon Jul 10 22:18:41 2017 +0200
+++ b/mercurial/localrepo.pyTue Jul 11 02:39:52 2017 +0200
@@ -1136,6 +1136,7 @@
  checkambigfiles=_cachedfiles)
 tr.changes['revs'] = set()
 tr.changes['obsmarkers'] = set()
+tr.changes['phases'] = {}
 
 tr.hookargs['txnid'] = txnid
 # note: writing the fncache only during finalize mean that the file is
diff -r 936da074fe26 -r d578e17d8809 mercurial/phases.py
--- a/mercurial/phases.py   Mon Jul 10 22:18:41 2017 +0200
+++ b/mercurial/phases.py   Tue Jul 11 02:39:52 2017 +0200
@@ -154,6 +154,18 @@
 dirty = True
 return roots, dirty
 
+def _trackphasechange(data, rev, old, new):
+"""add a phase move the  dictionnary
+
+If data is None, nothing happens.
+"""
+if data is None:
+return
+existing = data.get(rev)
+if existing is not None:
+old = existing[0]
+data[rev] = (old, new)
+
 class phasecache(object):
 def __init__(self, repo, phasedefaults, _load=True):
 if _load:
@@ -289,8 +301,13 @@
 """
 # Be careful to preserve shallow-copied values: do not update
 # phaseroots values, replace them.
+if tr is None:
+phasetracking = None
+else:
+phasetracking = tr.changes.get('phases')
 
 repo = repo.unfiltered()
+
 delroots = [] # set of root deleted by this path
 for phase in xrange(targetphase + 1, len(allphases)):
 # filter nodes that are not in a compatible phase already
@@ -300,7 +317,11 @@
 break # no roots to move anymore
 
 olds = self.phaseroots[phase]
+
 affected = repo.revs('%ln::%ln', olds, nodes)
+for r in affected:
+_trackphasechange(phasetracking, r, self.phase(repo, r),
+  targetphase)
 
 roots = set(ctx.node() for ctx in repo.set(
 'roots((%ln::) - %ld)', olds, affected))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 11] phases: extract the intermediate set of affected revs

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499717921 -7200
#  Mon Jul 10 22:18:41 2017 +0200
# Node ID 936da074fe26e45133672419d670febbfdb447a8
# Parent  8c771f0d6e8d70b088abc2c31dc59be0d58ab9fa
# EXP-Topic tr.changes.phases
phases: extract the intermediate set of affected revs

When advancing phases, we compute the new roots for the phases above. During
this process, we need to compute all the revisions that change phases (to the
new target phases). Extract these revisions into a separate variable. This
will be useful to record the phase changes in the transaction.

diff -r 8c771f0d6e8d -r 936da074fe26 mercurial/phases.py
--- a/mercurial/phases.py   Mon Jul 10 22:22:42 2017 +0200
+++ b/mercurial/phases.py   Mon Jul 10 22:18:41 2017 +0200
@@ -283,6 +283,10 @@
 tr.hookargs['phases_moved'] = '1'
 
 def advanceboundary(self, repo, tr, targetphase, nodes):
+"""Set all 'nodes' to phase 'targetphase'
+
+Nodes with a phase lower than 'targetphase' are not affected.
+"""
 # Be careful to preserve shallow-copied values: do not update
 # phaseroots values, replace them.
 
@@ -294,9 +298,12 @@
  if self.phase(repo, repo[n].rev()) >= phase]
 if not nodes:
 break # no roots to move anymore
+
 olds = self.phaseroots[phase]
+affected = repo.revs('%ln::%ln', olds, nodes)
+
 roots = set(ctx.node() for ctx in repo.set(
-'roots((%ln::) - (%ln::%ln))', olds, olds, nodes))
+'roots((%ln::) - %ld)', olds, affected))
 if olds != roots:
 self._updateroots(phase, roots, tr)
 # some roots may need to be declared for lower phases
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 3 V4] repovfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499769497 -7200
#  Tue Jul 11 12:38:17 2017 +0200
# Node ID cea0a88164c6d003b43fe542ec12def2662c1273
# Parent  201d9cbd64a276db278552a8924206afb67b1a4c
# EXP-Topic vfs.ward
repovfs: add a ward to check if locks are properly taken


When the appropriate developer warning are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).

This check system have caught a handful of locking issues that  has been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.

We introduces two real exceptions from extensions 'blackbox.log' (because a lot 
of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).

In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.

diff -r 201d9cbd64a2 -r cea0a88164c6 hgext/blackbox.py
--- a/hgext/blackbox.py Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/blackbox.py Tue Jul 11 12:38:17 2017 +0200
@@ -236,6 +236,7 @@
 
 if util.safehasattr(ui, 'setrepo'):
 ui.setrepo(repo)
+repo._wlockfreeprefix.add('blackbox.log')
 
 @command('^blackbox',
 [('l', 'limit', 10, _('the number of events to show')),
diff -r 201d9cbd64a2 -r cea0a88164c6 hgext/journal.py
--- a/hgext/journal.py  Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/journal.py  Tue Jul 11 12:38:17 2017 +0200
@@ -69,6 +69,7 @@
 def reposetup(ui, repo):
 if repo.local():
 repo.journal = journalstorage(repo)
+repo._wlockfreeprefix.add('namejournal')
 
 def runcommand(orig, lui, repo, cmd, fullargs, *args):
 """Track the command line options for recording in the journal"""
diff -r 201d9cbd64a2 -r cea0a88164c6 hgext/patchbomb.py
--- a/hgext/patchbomb.pyTue Jul 11 12:27:58 2017 +0200
+++ b/hgext/patchbomb.pyTue Jul 11 12:38:17 2017 +0200
@@ -122,6 +122,10 @@
 cmdutil.extraexport.append('pullurl')
 cmdutil.extraexportmap['pullurl'] = _addpullheader
 
+def reposetup(ui, repo):
+if not repo.local():
+return
+repo._wlockfreeprefix.add('last-email.txt')
 
 def prompt(ui, prompt, default=None, rest=':'):
 if default:
diff -r 201d9cbd64a2 -r cea0a88164c6 mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:27:58 2017 +0200
+++ b/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
@@ -289,6 +289,25 @@
 # only functions defined in module of enabled extensions are invoked
 featuresetupfuncs = set()
 
+# list of prefix for file which can be written without 'wlock'
+# Extensions should extend this list when needed
+_wlockfreeprefix = set([# We migh consider requiring 'wlock' for the next
+# two, but pretty much all the existing code assume
+# wlock is not needed so we keep them excluded for
+# now.
+'hgrc',
+'requires',
+# XXX cache is a complicatged business someone
+# should investigate this in depth at some point
+'cache/',
+# XXX shouldn't be dirstate covered by the wlock?
+'dirstate',
+# XXX bisect was still a bit too messy at the time
+# this changeset was introduced. Someone should fix
+# the remainig bit and drop this line
+'bisect.state',
+])
+
 def __init__(self, baseui, path, create=False):
 self.requirements = set()
 self.filtername = None
@@ -308,10 +327,13 @@
 self.auditor = pathutil.pathauditor(self.root, self._checknested)
 self.nofsauditor = pathutil.pathauditor(self.root, self._checknested,
 realfs=False)
-self.vfs = vfsmod.vfs(self.path)
 self.baseui = baseui
 self.ui = baseui.copy()
 self.ui.copy = baseui.copy # prevent copying repo configuration
+self.vfs = vfsmod.vfs(self.path)
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+self.vfs.audit = self._getvfsward(self.vfs.audit)
 # A list of callback to shape the phase if no data were found.
 # Callback are in the form: func(repo, roots) --> processed root.
 # This list it to be filled by extension during repo setup
@@ -427,6 +449,38 @@
 # Signature to

[PATCH 3 of 3 V4] reposvfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1470672882 -7200
#  Mon Aug 08 18:14:42 2016 +0200
# Node ID e132c6e95c50c6f23bd0084e5247032bb8f155ea
# Parent  cea0a88164c6d003b43fe542ec12def2662c1273
# EXP-Topic vfs.ward
reposvfs: add a ward to check if locks are properly taken

we wrap 'repo.svfs.audit' to check for the store lock when accessing file in
'.hg/store' for writing. This caught a couple of instance where the transaction
was released after the lock, we should probably have a dedicated checker for
that case.

diff -r cea0a88164c6 -r e132c6e95c50 mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
+++ b/mercurial/localrepo.pyMon Aug 08 18:14:42 2016 +0200
@@ -411,6 +411,12 @@
 self.svfs = self.store.vfs
 self.sjoin = self.store.join
 self.vfs.createmode = self.store.createmode
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+if util.safehasattr(self.svfs, 'vfs'): # this is filtervfs
+self.svfs.vfs.audit = self._getsvfsward(self.svfs.vfs.audit)
+else: # standard vfs
+self.svfs.audit = self._getsvfsward(self.svfs.audit)
 self._applyopenerreqs()
 if create:
 self._writerequirements()
@@ -481,6 +487,25 @@
 return ret
 return checkvfs
 
+def _getsvfsward(self, origfunc):
+"""build a ward for self.svfs"""
+rref = weakref.ref(self)
+def checksvfs(path, mode=None):
+ret = origfunc(path, mode=mode)
+repo = rref()
+if repo is None or not util.safehasattr(repo, '_lockref'):
+return
+if mode in (None, 'r', 'rb'):
+return
+if path.startswith(repo.sharedpath):
+# truncate name relative to the repository (.hg)
+path = path[len(repo.sharedpath) + 1:]
+if repo._currentlock(repo._lockref) is None:
+repo.ui.develwarn('write with no lock: "%s"' % path,
+  stacklevel=3)
+return ret
+return checksvfs
+
 def close(self):
 self._writecaches()
 
diff -r cea0a88164c6 -r e132c6e95c50 tests/test-devel-warnings.t
--- a/tests/test-devel-warnings.t   Tue Jul 11 12:38:17 2017 +0200
+++ b/tests/test-devel-warnings.t   Mon Aug 08 18:14:42 2016 +0200
@@ -49,6 +49,11 @@
   > with repo.vfs(b'branch', 'a'):
   > pass
   > 
+  > @command(b'no-lock-write', [], '')
+  > def nolockwrite(ui, repo):
+  > with repo.svfs(b'fncache', 'a'):
+  > pass
+  > 
   > @command(b'stripintr', [], '')
   > def stripintr(ui, repo):
   > lo = repo.lock()
@@ -114,6 +119,9 @@
   $ hg no-wlock-write
   devel-warn: write with no wlock: "branch" at: $TESTTMP/buggylocking.py:* 
(nowlockwrite) (glob)
 
+  $ hg no-lock-write
+  devel-warn: write with no lock: "fncache" at: $TESTTMP/buggylocking.py:* 
(nolockwrite) (glob)
+
 Stripping from a transaction
 
   $ echo a > a
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3 V4] vfs: allow to pass more argument to audit

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499768878 -7200
#  Tue Jul 11 12:27:58 2017 +0200
# Node ID 201d9cbd64a276db278552a8924206afb67b1a4c
# Parent  4672db164c986da4442bd864cd044512d975c3f2
# EXP-Topic vfs.ward
vfs: allow to pass more argument to audit

We want to be able to do more precise check when auditing a path depending of
the intend of the file access (eg read versus write). So we now pass the 'mode'
 value to 'audit' and update the audit function to accept them.

This will be put to use in the next changeset.

diff -r 4672db164c98 -r 201d9cbd64a2 mercurial/pathutil.py
--- a/mercurial/pathutil.py Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/pathutil.py Tue Jul 11 12:27:58 2017 +0200
@@ -46,7 +46,7 @@
 else:
 self.normcase = lambda x: x
 
-def __call__(self, path):
+def __call__(self, path, mode=None):
 '''Check the relative path.
 path may contain a pattern (e.g. foodir/**.txt)'''
 
diff -r 4672db164c98 -r 201d9cbd64a2 mercurial/vfs.py
--- a/mercurial/vfs.py  Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/vfs.py  Tue Jul 11 12:27:58 2017 +0200
@@ -306,7 +306,7 @@
 if audit:
 self.audit = pathutil.pathauditor(self.base)
 else:
-self.audit = util.always
+self.audit =  (lambda path, mode=None: True)
 self.createmode = None
 self._trustnlink = None
 
@@ -360,7 +360,7 @@
 r = util.checkosfilename(path)
 if r:
 raise error.Abort("%s: %r" % (r, path))
-self.audit(path)
+self.audit(path, mode=mode)
 f = self.join(path)
 
 if not text and "b" not in mode:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] changegroup: stop returning and recording added nodes in 'cg.apply'

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499972886 -7200
#  Thu Jul 13 21:08:06 2017 +0200
# Node ID ddf2ab3e3add92ed5cddbeb9a33679891d74094d
# Parent  23a2483541d50b8dce47a061a92a2b2089a42202
# EXP-Topic cleanup.changegroup
changegroup: stop returning and recording added nodes in 'cg.apply'

cg.apply used to returns the added nodes. Callers doesn't have a use for it
anymore, remove the added node and stops recording it in the current
operation.

This information was added in the current release cycle so no extensions
breakage should happens.

diff -r 23a2483541d5 -r ddf2ab3e3add mercurial/bundle2.py
--- a/mercurial/bundle2.py  Thu Jul 13 21:10:55 2017 +0200
+++ b/mercurial/bundle2.py  Thu Jul 13 21:08:06 2017 +0200
@@ -403,10 +403,9 @@
 return op
 
 def _processchangegroup(op, cg, tr, source, url, **kwargs):
-ret, addednodes = cg.apply(op.repo, tr, source, url, **kwargs)
+ret = cg.apply(op.repo, tr, source, url, **kwargs)
 op.records.add('changegroup', {
 'return': ret,
-'addednodes': addednodes,
 })
 return ret
 
diff -r 23a2483541d5 -r ddf2ab3e3add mercurial/changegroup.py
--- a/mercurial/changegroup.py  Thu Jul 13 21:10:55 2017 +0200
+++ b/mercurial/changegroup.py  Thu Jul 13 21:08:06 2017 +0200
@@ -408,7 +408,7 @@
 ret = deltaheads - 1
 else:
 ret = deltaheads + 1
-return ret, added
+return ret
 
 class cg2unpacker(cg1unpacker):
 """Unpacker for cg2 streams.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] phases: remove trace of addednodes in the 'phase-heads' handling

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499973055 -7200
#  Thu Jul 13 21:10:55 2017 +0200
# Node ID 23a2483541d50b8dce47a061a92a2b2089a42202
# Parent  50243c975fc2ee605ebac493f4ab18d37117e46a
# EXP-Topic cleanup.changegroup
phases: remove trace of addednodes in the 'phase-heads' handling

updatephases have no use of the 'addednodes' parameter since 50243c975fc2.
However caller are still passing it for nothing, remove the parameter and
remove computing of the added nodes in caller.

diff -r 50243c975fc2 -r 23a2483541d5 mercurial/bundle2.py
--- a/mercurial/bundle2.py  Tue Jul 11 05:06:01 2017 +0200
+++ b/mercurial/bundle2.py  Thu Jul 13 21:10:55 2017 +0200
@@ -1788,11 +1788,7 @@
 def handlephases(op, inpart):
 """apply phases from bundle part to repo"""
 headsbyphase = _readphaseheads(inpart)
-addednodes = []
-for entry in op.records['changegroup']:
-addednodes.extend(entry['addednodes'])
-phases.updatephases(op.repo.unfiltered(), op.gettransaction(), 
headsbyphase,
-addednodes)
+phases.updatephases(op.repo.unfiltered(), op.gettransaction(), 
headsbyphase)
 
 @parthandler('reply:pushkey', ('return', 'in-reply-to'))
 def handlepushkeyreply(op, inpart):
diff -r 50243c975fc2 -r 23a2483541d5 mercurial/phases.py
--- a/mercurial/phases.py   Tue Jul 11 05:06:01 2017 +0200
+++ b/mercurial/phases.py   Thu Jul 13 21:10:55 2017 +0200
@@ -446,7 +446,7 @@
 headsbyphase[phase] = [cl.node(r) for r in repo.revs(revset, subset)]
 return headsbyphase
 
-def updatephases(repo, tr, headsbyphase, addednodes):
+def updatephases(repo, tr, headsbyphase):
 """Updates the repo with the given phase heads"""
 # Now advance phase boundaries of all but secret phase
 for phase in allphases[:-1]:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 01 of 14] cache: introduce an abstract class for cache we can upgrade incrementally

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458441 -7200
#  Fri Jul 07 22:14:01 2017 +0200
# Node ID 6edb62505c697329de034c2fdc47befd5896f31f
# Parent  89796a25d4bb91fb418ad3e70faad2c586902ffb
# EXP-Topic obs-cache
cache: introduce an abstract class for cache we can upgrade incrementally

Right now each class implements their own mechanism for validation, and
update. We start introducing abstract class to ultimately allow more
unification of the cache code.

The end goal of this series is to introduce a cache for some obsolescence
property, not to actually implement the cache. However, taking advantage of
adding a new cache to introduce the abstract class seems like a win.

diff -r 89796a25d4bb -r 6edb62505c69 mercurial/cache.py
--- /dev/null   Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/cache.pyFri Jul 07 22:14:01 2017 +0200
@@ -0,0 +1,127 @@
+# cache.py - utilities for caching
+#
+# Copyright 2017 Octobus SAS <cont...@octobus.net>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+from __future__ import absolute_import
+
+import abc
+import struct
+
+from . import (
+util,
+)
+
+class incrementalcachebase(object):
+"""base class for incremental cache from append only source
+
+There are multiple append only data source we might want to cache
+computation from. One of the common pattern is to track the state of the
+file and update the cache with the extra data (eg: branchmap-cache tracking
+changelog). This pattern also needs to detect when a the source is striped
+
+The overall pattern is similar whatever the actual source is. This class
+introduces the basic patterns.
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = ()
+
+_cachekeyspec = '' # used for serialization
+_cachename = None # used for debug message
+
+@abc.abstractmethod
+def __init__(self):
+super(incrementalcachebase, self).__init__()
+self._cachekey = None
+
+@util.propertycache
+def _cachekeystruct(self):
+# dynamic property to help subclass to change it
+ return struct.Struct('>' + self._cachekeyspec)
+
+@util.propertycache
+def _cachekeysize(self):
+# dynamic property to help subclass to change it
+return self._cachekeystruct.size
+
+@abc.abstractmethod
+def _updatefrom(self, repo, data):
+"""override this method to update you date from incrementally read 
data.
+
+Content of  will depends of the sources.
+"""
+raise NotImplementedError
+
+@abc.abstractmethod
+def clear(self, reset=False):
+"""invalidate the cache content
+
+if 'reset' is passed, we detected a strip and the cache will have to be
+recomputed.
+
+Subclasses MUST overide this method to actually affect the cache data.
+"""
+if reset:
+self._cachekey = self.emptykey if reset else None
+else:
+self._cachekey = None
+
+@abc.abstractmethod
+def load(self, repo):
+"""Load data from disk
+
+Subclasses MUST restore the "cachekey" attribute while doing so.
+"""
+raise NotImplementedError
+
+@abc.abstractmethod
+def _fetchupdatedata(self, repo):
+"""Check the source for possible changes and return necessary data
+
+The return is a tree elements tuple: reset, data, cachekey
+
+* reset: `True` when a strip is detected and cache need to be reset
+* data: new data to take in account, actual type depends of the source
+* cachekey: the cache key covering  and precious covered data
+"""
+raise NotImplementedError
+
+# Useful "public" function (no need to override them)
+
+def update(self, repo):
+"""update the cache with new repository data
+
+The update will be incremental when possible"""
+repo = repo.unfiltered()
+
+# If we do not have any data, try loading from disk
+if self._cachekey is None:
+self.load(repo)
+
+reset, data, newkey = self._fetchupdatedata(repo)
+if newkey == self._cachekey:
+return
+if reset or self._cachekey is None:
+repo.ui.log('cache', 'strip detected, %s cache reset\n'
+% self._cachename)
+self.clear(reset=True)
+
+starttime = util.timer()
+self._updatefrom(repo, data)
+duration = util.timer() - starttime
+repo.ui.log('cache', 'updated %s in %.4f seconds\n',
+self._cachename, duration)
+
+self._cachekey = newkey
+
+def

[PATCH 11 of 14] obscache: instantiate the cache and keep it warm

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1495198006 -7200
#  Fri May 19 14:46:46 2017 +0200
# Node ID f8953ed43f8d1b146dcff688030133f0d6123a49
# Parent  985d753d4f5799f2a332140adedb06efd465d62b
# EXP-Topic obs-cache
obscache: instantiate the cache and keep it warm

We are not using it yet, but we make sure we have a cache and that we keep it up
to date after each transaction.

The two "warning: ignoring unknown working parent" output in the tests are
caused by 'blackbox' accessing the dirstate during the strip. I've not found an
easy way to work around this so I kept them as they are harmless.

diff -r 985d753d4f57 -r f8953ed43f8d mercurial/localrepo.py
--- a/mercurial/localrepo.pyFri May 19 14:44:22 2017 +0200
+++ b/mercurial/localrepo.pyFri May 19 14:46:46 2017 +0200
@@ -1274,6 +1274,10 @@
 self.ui.debug('updating the branch cache\n')
 branchmap.updatecache(self.filtered('served'))
 
+if self.obsstore:
+self.obsstore.obscache.update(self)
+self.obsstore.obscache.save(self)
+
 def invalidatecaches(self):
 
 if '_tagscache' in vars(self):
diff -r 985d753d4f57 -r f8953ed43f8d mercurial/obsolete.py
--- a/mercurial/obsolete.py Fri May 19 14:44:22 2017 +0200
+++ b/mercurial/obsolete.py Fri May 19 14:46:46 2017 +0200
@@ -525,6 +525,7 @@
 self.svfs = repo.svfs
 self._defaultformat = defaultformat
 self._readonly = readonly
+self.obscache = obscache(repo)
 
 def __iter__(self):
 return iter(self._all)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 14] cache: add a obsstoresourcebase class

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458924 -7200
#  Fri Jul 07 22:22:04 2017 +0200
# Node ID 5b49f653a4a50607127e37e7511c8a8e343cc8d9
# Parent  7c41d1484f53c53d516153d19a90c331c3f87c16
# EXP-Topic obs-cache
cache: add a obsstoresourcebase class

This class provides an abstract base for cache based on the obsstore. Having
this as a separared class is useful as it could be used for the obsindexes
cache that have been discussed in June this year.

diff -r 7c41d1484f53 -r 5b49f653a4a5 mercurial/cache.py
--- a/mercurial/cache.pySat Jun 17 06:39:43 2017 +0200
+++ b/mercurial/cache.pyFri Jul 07 22:22:04 2017 +0200
@@ -164,3 +164,20 @@
 
 def _fetchupdatedata(self, repo):
 return self._fetchchangelogdata(self._cachekey, repo.changelog)
+
+class obsstoresourcebase(incrementalcachebase):
+"""an abstract class for cache that source data from the obsstore
+
+For this purpose it use a cache key covering obsstore
+content provided by the obsstore itself
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = (0, node.nullid)
+_cachekeyspec = 'I20s'
+_cachename = None # used for debug message
+
+def _fetchupdatedata(self, repo):
+return repo.obsstore.getmarkerssince(self._cachekey)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 14] obsstore: add a method to incrementally retrieve obsmarkers

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1497674383 -7200
#  Sat Jun 17 06:39:43 2017 +0200
# Node ID 7c41d1484f53c53d516153d19a90c331c3f87c16
# Parent  e73d330f6fdb0cf1635920f4302b61b321ec58e2
# EXP-Topic obs-cache
obsstore: add a method to incrementally retrieve obsmarkers

Parsing the full obsstore is slow, so cache that depends on obsstore content
wants a way to know if the obsstore changed, and it this change was append
only.

For this purpose we introduce an official cache-key for the obsstore. This
cache-key work in a way similar to the '(tiprev, tipnode)' pair used for the
changelog. We use the size of the obsstore file and the hash of its tail. That
way, we can check if the obsstore grew and if the content we knew is still
present in the obsstore.

The cachekey comes with a method that only returns the obsmarkers that changed
between the last seen "cache-key" value and the current state of the
repository. That method is race free and should be used by caches. See
documentation for details.

note: we are reading the full obsstore file from disk as part of the key
validation process. This could be avoided but we keep the implementation
simple for now. Once it is in place and running we can iterate to make it
better.

diff -r e73d330f6fdb -r 7c41d1484f53 mercurial/obsolete.py
--- a/mercurial/obsolete.py Sun Jun 04 10:02:09 2017 -0700
+++ b/mercurial/obsolete.py Sat Jun 17 06:39:43 2017 +0200
@@ -70,6 +70,7 @@
 from __future__ import absolute_import
 
 import errno
+import hashlib
 import struct
 
 from .i18n import _
@@ -513,6 +514,10 @@
 # parents: (tuple of nodeid) or None, parents of precursors
 #  None is used when no data has been recorded
 
+# how much data to read at the end of file,
+# 1024 byte should be about 10 markers in average
+_obskeyspan = 1024
+
 def __init__(self, svfs, defaultformat=_fm1version, readonly=False):
 # caches for various obsolescence related cache
 self.caches = {}
@@ -540,6 +545,72 @@
 
 __bool__ = __nonzero__
 
+def getmarkerssince(self, previouscontentkey):
+"""retrieve all new markers (since "contentkey") + updated content key
+
+This function is to be used by cache depending on obsstore content. It
+make sure cache can incrementally update themselves without fear
+obsstore stripping or race condition.  If the content key is invalid
+(some obsstore content got stripped), all markers in the obsstore will
+be returned.
+
+return: (reset, obsmarkers, contentkey)
+
+:reset: boolean, True if previouscontentkey was invalided. Previously
+stored data are invalid and should be discarded. Full markers
+content is return in this case.
+
+:obsmarkers: the list of new obsmarkers.
+
+:contentkey: a key matching the content of 'previouscontentkey' +
+ 'obsmarkers'.
+
+The content key is a pair of:
+
+(obsstore size, hash of last N bytes of the obsstore)
+
+It must be kept around by cache and provided again for the next
+incremental read of new obsmarkers.
+"""
+# XXX This function could avoid loading the whole data from disk (and
+# only read new markers). It currently use 'self._data' to keep the
+# code simpler.
+keysize, keyhash = previouscontentkey
+fulldata = self._data
+
+# check the existing cache key
+reset = False
+if len(fulldata) < keysize: # less data than expected, this is a strip
+reset = True
+else:
+if keysize == 0: # no obsstore
+actualhash = node.nullid
+else:
+first = max(0, keysize - self._obskeyspan)
+keydata = fulldata[first:keysize]
+actualhash = hashlib.sha1(keydata).digest()
+reset = actualhash != keyhash
+newsize = len(fulldata)
+
+# read the new data
+if reset:
+keysize = None # read all data
+elif keysize == newsize:
+# no update since last change, exist early
+return False, [], previouscontentkey
+if newsize:
+start = max(0, newsize - self._obskeyspan)
+newhash = hashlib.sha1(fulldata[start:newsize]).digest()
+__, obsmarkers = _readmarkers(fulldata, keysize, newsize)
+else:
+# obsstore is empty, use a generic hash and skip reading markers.
+newhash == node.nullid
+obsmarkers = []
+
+# for now and for the sake of simplicity make sure obsmarkers is a list
+obsmarkers = list(obsmarkers)
+return reset, obsmarkers, (newsize, newhash)
+
 @property
 def readonly(self):
 """True if marker creation is disabled

[PATCH 12 of 14] obscache: use the obscache to compute the obsolete set

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1495198021 -7200
#  Fri May 19 14:47:01 2017 +0200
# Node ID 3a93e99b0e718befd57a32615a14fd0d3c194456
# Parent  f8953ed43f8d1b146dcff688030133f0d6123a49
# EXP-Topic obs-cache
obscache: use the obscache to compute the obsolete set

Now that we have a cache and that the cache is kept up to date, we can use it to
speeds up the obsolete set computation. This way, we no longer need to load the
obsstore for most operation.

On the mercurial-core repository, this provides a significant speed up for
operation that do not need further obsolescence information:

| command  | before | after |
| id -r @  |  0.670 | 0.093 |
| log -r @ |  0.951 | 0.916 |
| diff |  0.070 | 0.069 |
| diff -r .~1  |  0.705 | 0.120 |
| export @ |  0.672 | 0.103 |
| log -G -r draft()|  1.117 | 1.084 |
| log -G -r draft()|  1.063 | 1.064 |
|   -T "{node} {troubles}" ||   |
| log -G -r tip|  0.658 | 0.086 |
|   -T "{node} {desc}" ||   |

The obsstore loading operation usually disappear from execution profile.

The speeds up rely on a couple of different mechanism:

* First, not having to parse the obsstore provides a massive speedup:

  Time spent computing 'obsolete', no obsstore pre-loaded.

before: wall 0.449502 comb 0.46 user 0.42 sys 0.04 (best of 17)
after:  wall 0.005752 comb 0.01 user 0.01 sys 0.00 (best of 340)

* Second keeping the computation fully in the revision space (no usage of node),
  raise and extra 4x speedup.

  Time spent computing 'obsolete', obsstore preloaded:

before: wall 0.007684 comb 0.00 user 0.00 sys 0.00 (best of 305)
after:  wall 0.001928 comb 0.00 user 0.00 sys 0.00 (best of 
1250)

To keep the changeset simple, we assume the cache is up to date (from the last
transaction). This won't be true when both old and new clients access the
repository. (with the special case of no new transactions since last upgrade).
We'll address this issue in the next couple of changesets.

This changesets is a first step to install the basic. There are a couple of easy
improvement that can further improve this cache:

 * Improving handling of outdated cache on read only operation (see above),

 * Avoid reaading the full obsstore data from disk to check the cache key
   (about -4ms, 3x speedup)

 * Optimise the python code to reduce attribute lookup
   (about 25% of the remaining of the time spent there).

diff -r f8953ed43f8d -r 3a93e99b0e71 mercurial/obsolete.py
--- a/mercurial/obsolete.py Fri May 19 14:46:46 2017 +0200
+++ b/mercurial/obsolete.py Fri May 19 14:47:01 2017 +0200
@@ -1096,11 +1096,24 @@
 @cachefor('obsolete')
 def _computeobsoleteset(repo):
 """the set of obsolete revisions"""
-getnode = repo.changelog.node
 notpublic = _mutablerevs(repo)
-isobs = repo.obsstore.successors.__contains__
-obs = set(r for r in notpublic if isobs(getnode(r)))
-return obs
+if not notpublic or not repo.obsstore:
+# all changeset are public, none are obsolete
+return set()
+
+# XXX There are a couple of case where the cache could not be up to date:
+#
+# 1) no transaction happened in the repository since the upgrade,
+# 2) both old and new client touches that repository
+#
+# recomputing the whole cache in these case is a bit slower that using the
+# good old version (parsing markers and checking them). We could add some
+# logic to fall back to the old way in these cases.
+obscache = repo.obsstore.obscache
+obscache.update(repo) # ensure it is up to date:
+isobs = obscache.get
+
+return set(r for r in notpublic if isobs(r))
 
 @cachefor('unstable')
 def _computeunstableset(repo):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 14] obscache: add a cache for 1/2 of the "obsolete" property

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1495197986 -7200
#  Fri May 19 14:46:26 2017 +0200
# Node ID 63214f4d9a766761259b650539eede424413e6a2
# Parent  774ff18cc36b72822f598b4fa5a51628513926e3
# EXP-Topic obs-cache
obscache: add a cache for 1/2 of the "obsolete" property

Knowing if a changeset is obsolete requires two data:

1) is the change non-public,
2) is the changeset affected by any obsolescence markers.

The phase related data has some fast implementation already. However, the
obsmarkers based property currently requires to parse the whole obsstore, a
slow operation.

That information is monotonic (new changeset are not affected, and once they
are affected, they will be for ever), making it is easy to cache. We introduce
a new class dedicated to caching of this information. That first
implementation still needs to parse the full obsstore when updating for the
sake of simplicity. It will be improved later to allow lighter upgrade.

The next changesets will put this new cache to use.

That code is coming from the evolve extension, were it matured. To keep this
changeset simple, there are a couple of improvement in the extension that will
be ported later.

diff -r 774ff18cc36b -r 63214f4d9a76 mercurial/obsolete.py
--- a/mercurial/obsolete.py Sat Jul 08 16:26:16 2017 +0200
+++ b/mercurial/obsolete.py Fri May 19 14:46:26 2017 +0200
@@ -75,6 +75,7 @@
 
 from .i18n import _
 from . import (
+cache,
 error,
 node,
 obsutil,
@@ -907,6 +908,145 @@
 repo.ui.deprecwarn(movemsg, '4.3')
 return obsutil.successorssets(repo, initialnode, cache=cache)
 
+class obscache(cache.dualsourcecache):
+"""cache "does a rev is the precursors of some obsmarkers" property
+
+This is not directly holding the "is this revision obsolete" information,
+because phases data gets into play here. However, it allow to compute the
+"obsolescence" set without reading the obsstore content.
+
+The cache use a bytearray to store that data and simply write it on disk
+for storage.
+
+Implementation note #1:
+
+  The obsstore is implementing only half of the transaction logic it
+  should. It properly record the starting point of the obsstore to allow
+  clean rollback. However it still write to the obsstore file directly
+  during the transaction. Instead it should be keeping data in memory and
+  write to a '.pending' file to make the data vailable for hooks.
+
+  This cache is not going further than what the obsstore is doing, so it
+  does not has any '.pending' logic. When the obsstore gains proper
+  '.pending' support, adding it to this cache should not be too hard. As
+  the flag always move from 0 to 1, we could have a second '.pending' cache
+  file to be read. If flag is set in any of them, the value is 1. For the
+  same reason, updating the file in place should be possible.
+
+Implementation note #2:
+
+Storage-wise, we could have a "start rev" to avoid storing useless
+zero. That would be especially useful for the '.pending' overlay.
+"""
+
+_filepath = 'cache/obscache-v01'
+_cachename = 'obscache' # used for error message
+
+def __init__(self, repo):
+super(obscache, self).__init__()
+self._ondiskkey = None
+self._vfs = repo.vfs
+self._data = bytearray()
+
+def get(self, rev):
+"""return True if "rev" is used as "precursors for any obsmarkers
+
+IMPORTANT: make sure the cache has been updated to match the repository
+content before using it"""
+return self._data[rev]
+
+def clear(self, reset=False):
+"""invalidate the in memory cache content"""
+super(obscache, self).clear(reset=reset)
+self._data = bytearray()
+
+def _updatefrom(self, repo, data):
+if data[0]:
+self._updaterevs(repo,  data[0])
+if data[1]:
+self._updatemarkers(repo,  data[1])
+
+def _updaterevs(self, repo, revs):
+"""update the cache with new revisions
+
+Newly added changesets might be affected by obsolescence markers we
+already have locally. So we needs to have some global knowledge about
+the markers to handle that question.
+
+XXX performance note:
+
+Right now this requires parsing all markers in the obsstore. We could
+imagine using various optimisation (eg: another cache, network
+exchange, etc).
+
+A possible approach to this is to build a set of all nodes used as
+precursors in `obsstore._obscandidate`. If markers are not loaded yet,
+we could initialize it by doing a quick scan through the obsstore data
+and filling a (pre-sized) set. Doing so would 

[PATCH 01 of 14] cache: introduce an abstract class for cache we can upgrade incrementally

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458441 -7200
#  Fri Jul 07 22:14:01 2017 +0200
# Node ID 6edb62505c697329de034c2fdc47befd5896f31f
# Parent  89796a25d4bb91fb418ad3e70faad2c586902ffb
# EXP-Topic obs-cache
cache: introduce an abstract class for cache we can upgrade incrementally

Right now each class implements their own mechanism for validation, and
update. We start introducing abstract class to ultimately allow more
unification of the cache code.

The end goal of this series is to introduce a cache for some obsolescence
property, not to actually implement the cache. However, taking advantage of
adding a new cache to introduce the abstract class seems like a win.

diff -r 89796a25d4bb -r 6edb62505c69 mercurial/cache.py
--- /dev/null   Thu Jan 01 00:00:00 1970 +
+++ b/mercurial/cache.pyFri Jul 07 22:14:01 2017 +0200
@@ -0,0 +1,127 @@
+# cache.py - utilities for caching
+#
+# Copyright 2017 Octobus SAS <cont...@octobus.net>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+from __future__ import absolute_import
+
+import abc
+import struct
+
+from . import (
+util,
+)
+
+class incrementalcachebase(object):
+"""base class for incremental cache from append only source
+
+There are multiple append only data source we might want to cache
+computation from. One of the common pattern is to track the state of the
+file and update the cache with the extra data (eg: branchmap-cache tracking
+changelog). This pattern also needs to detect when a the source is striped
+
+The overall pattern is similar whatever the actual source is. This class
+introduces the basic patterns.
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = ()
+
+_cachekeyspec = '' # used for serialization
+_cachename = None # used for debug message
+
+@abc.abstractmethod
+def __init__(self):
+super(incrementalcachebase, self).__init__()
+self._cachekey = None
+
+@util.propertycache
+def _cachekeystruct(self):
+# dynamic property to help subclass to change it
+ return struct.Struct('>' + self._cachekeyspec)
+
+@util.propertycache
+def _cachekeysize(self):
+# dynamic property to help subclass to change it
+return self._cachekeystruct.size
+
+@abc.abstractmethod
+def _updatefrom(self, repo, data):
+"""override this method to update you date from incrementally read 
data.
+
+Content of  will depends of the sources.
+"""
+raise NotImplementedError
+
+@abc.abstractmethod
+def clear(self, reset=False):
+"""invalidate the cache content
+
+if 'reset' is passed, we detected a strip and the cache will have to be
+recomputed.
+
+Subclasses MUST overide this method to actually affect the cache data.
+"""
+if reset:
+self._cachekey = self.emptykey if reset else None
+else:
+self._cachekey = None
+
+@abc.abstractmethod
+def load(self, repo):
+"""Load data from disk
+
+Subclasses MUST restore the "cachekey" attribute while doing so.
+"""
+raise NotImplementedError
+
+@abc.abstractmethod
+def _fetchupdatedata(self, repo):
+"""Check the source for possible changes and return necessary data
+
+The return is a tree elements tuple: reset, data, cachekey
+
+* reset: `True` when a strip is detected and cache need to be reset
+* data: new data to take in account, actual type depends of the source
+* cachekey: the cache key covering  and precious covered data
+"""
+raise NotImplementedError
+
+# Useful "public" function (no need to override them)
+
+def update(self, repo):
+"""update the cache with new repository data
+
+The update will be incremental when possible"""
+repo = repo.unfiltered()
+
+# If we do not have any data, try loading from disk
+if self._cachekey is None:
+self.load(repo)
+
+reset, data, newkey = self._fetchupdatedata(repo)
+if newkey == self._cachekey:
+return
+if reset or self._cachekey is None:
+repo.ui.log('cache', 'strip detected, %s cache reset\n'
+% self._cachename)
+self.clear(reset=True)
+
+starttime = util.timer()
+self._updatefrom(repo, data)
+duration = util.timer() - starttime
+repo.ui.log('cache', 'updated %s in %.4f seconds\n',
+self._cachename, duration)
+
+self._cachekey = newkey
+
+def

[PATCH 03 of 14] obsstore: keep self._data updated with _addmarkers

[Boris Feld]

# HG changeset patch
# User Jun Wu 
# Date 1496552183 25200
#  Sat Jun 03 21:56:23 2017 -0700
# Node ID ff2ebc11f12a26a4e0bda3ccf5fde63f5f690813
# Parent  8b71290526ddb77f157e075191dd748793d85601
# EXP-Topic obs-cache
obsstore: keep self._data updated with _addmarkers

This makes sure obsstore._data is still correct with added markers.

The '_data' propertycache was added in 17ce57b7873f.

diff -r 8b71290526dd -r ff2ebc11f12a mercurial/obsolete.py
--- a/mercurial/obsolete.py Fri Jul 07 22:15:52 2017 +0200
+++ b/mercurial/obsolete.py Sat Jun 03 21:56:23 2017 -0700
@@ -607,8 +607,8 @@
 offset = f.tell()
 transaction.add('obsstore', offset)
 # offset == 0: new file - add the version header
-for bytes in encodemarkers(new, offset == 0, self._version):
-f.write(bytes)
+data = b''.join(encodemarkers(new, offset == 0, self._version))
+f.write(data)
 finally:
 # XXX: f.close() == filecache invalidation == obsstore rebuilt.
 # call 'filecacheentry.refresh()'  here
@@ -616,7 +616,7 @@
 addedmarkers = transaction.changes.get('obsmarkers')
 if addedmarkers is not None:
 addedmarkers.update(new)
-self._addmarkers(new)
+self._addmarkers(new, data)
 # new marker *may* have changed several set. invalidate the cache.
 self.caches.clear()
 # records the number of new markers for the transaction hooks
@@ -673,8 +673,9 @@
 def _cached(self, attr):
 return attr in self.__dict__
 
-def _addmarkers(self, markers):
+def _addmarkers(self, markers, rawdata):
 markers = list(markers) # to allow repeated iteration
+self._data = self._data + rawdata
 self._all.extend(markers)
 if self._cached('successors'):
 _addsuccessors(self.successors, markers)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 14] obsstore: let read marker API take a range of offsets

[Boris Feld]

# HG changeset patch
# User Jun Wu 
# Date 1496595729 25200
#  Sun Jun 04 10:02:09 2017 -0700
# Node ID e73d330f6fdb0cf1635920f4302b61b321ec58e2
# Parent  ff2ebc11f12a26a4e0bda3ccf5fde63f5f690813
# EXP-Topic obs-cache
obsstore: let read marker API take a range of offsets

This allows us to read a customized range of markers, instead of loading all
of them.

The condition of stop is made consistent across C and Python implementation
so we will still read marker when offset=a, stop=a+1.

diff -r ff2ebc11f12a -r e73d330f6fdb mercurial/obsolete.py
--- a/mercurial/obsolete.py Sat Jun 03 21:56:23 2017 -0700
+++ b/mercurial/obsolete.py Sun Jun 04 10:02:09 2017 -0700
@@ -178,10 +178,9 @@
 _fm0fsize = _calcsize(_fm0fixed)
 _fm0fnodesize = _calcsize(_fm0node)
 
-def _fm0readmarkers(data, off):
+def _fm0readmarkers(data, off, stop):
 # Loop on markers
-l = len(data)
-while off + _fm0fsize <= l:
+while off < stop:
 # read fixed part
 cur = data[off:off + _fm0fsize]
 off += _fm0fsize
@@ -317,7 +316,7 @@
 _fm1metapair = 'BB'
 _fm1metapairsize = _calcsize('BB')
 
-def _fm1purereadmarkers(data, off):
+def _fm1purereadmarkers(data, off, stop):
 # make some global constants local for performance
 noneflag = _fm1parentnone
 sha2flag = usingsha256
@@ -331,10 +330,9 @@
 unpack = _unpack
 
 # Loop on markers
-stop = len(data) - _fm1fsize
 ufixed = struct.Struct(_fm1fixed).unpack
 
-while off <= stop:
+while off < stop:
 # read fixed part
 o1 = off + fsize
 t, secs, tz, flags, numsuc, numpar, nummeta, prec = 
ufixed(data[off:o1])
@@ -428,11 +426,10 @@
 data.append(value)
 return ''.join(data)
 
-def _fm1readmarkers(data, off):
+def _fm1readmarkers(data, off, stop):
 native = getattr(parsers, 'fm1readmarkers', None)
 if not native:
-return _fm1purereadmarkers(data, off)
-stop = len(data) - _fm1fsize
+return _fm1purereadmarkers(data, off, stop)
 return native(data, off, stop)
 
 # mapping to read/write various marker formats
@@ -444,14 +441,17 @@
 return _unpack('>B', data[0:1])[0]
 
 @util.nogc
-def _readmarkers(data):
+def _readmarkers(data, off=None, stop=None):
 """Read and enumerate markers from raw data"""
 diskversion = _readmarkerversion(data)
-off = 1
+if not off:
+off = 1  # skip 1 byte version number
+if stop is None:
+stop = len(data)
 if diskversion not in formats:
 msg = _('parsing obsolete marker: unknown version %r') % diskversion
 raise error.UnknownVersion(msg, version=diskversion)
-return diskversion, formats[diskversion][0](data, off)
+return diskversion, formats[diskversion][0](data, off, stop)
 
 def encodeheader(version=_fm0version):
 return _pack('>B', version)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 14] cache: adds debug details about what the content of the update

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499523976 -7200
#  Sat Jul 08 16:26:16 2017 +0200
# Node ID 774ff18cc36b72822f598b4fa5a51628513926e3
# Parent  5d3e659c979aa428ba44b138cfac30b7cca28fb3
# EXP-Topic obs-cache
cache: adds debug details about what the content of the update

Having more data in debug is good, we add a small method to help providing them.

diff -r 5d3e659c979a -r 774ff18cc36b mercurial/cache.py
--- a/mercurial/cache.pyFri Jul 07 22:22:39 2017 +0200
+++ b/mercurial/cache.pySat Jul 08 16:26:16 2017 +0200
@@ -91,6 +91,11 @@
 """
 raise NotImplementedError
 
+@abc.abstractmethod
+def _updatesummary(self, data):
+"""return a small string to be included in debug output"""
+raise NotImplementedError
+
 # Useful "public" function (no need to override them)
 
 def update(self, repo):
@@ -114,8 +119,9 @@
 starttime = util.timer()
 self._updatefrom(repo, data)
 duration = util.timer() - starttime
-repo.ui.log('cache', 'updated %s in %.4f seconds\n',
-self._cachename, duration)
+summary = self._updatesummary(data)
+repo.ui.log('cache', 'updated %s in %.4f seconds (%s)\n',
+self._cachename, duration, summary)
 
 self._cachekey = newkey
 
@@ -165,6 +171,9 @@
 def _fetchupdatedata(self, repo):
 return self._fetchchangelogdata(self._cachekey, repo.changelog)
 
+def _updatesummary(self, data):
+return '%ir' % len(data)
+
 class obsstoresourcebase(incrementalcachebase):
 """an abstract class for cache that source data from the obsstore
 
@@ -182,6 +191,9 @@
 def _fetchupdatedata(self, repo):
 return repo.obsstore.getmarkerssince(self._cachekey)
 
+def _updatesummary(self, data):
+return '%io' % len(data)
+
 class dualsourcecache(obsstoresourcebase, changelogsourcebase):
 """An abstract class for cache that needs both changelog and obsstore
 
@@ -216,3 +228,6 @@
 newkey = newclkey + newobskey
 data = (revs, obsmarkers)
 return reset, data, newkey
+
+def _updatesummary(self, data):
+return '%ir, %io' % (len(data[0]), len(data[1]))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 14] cache: introduce a dualsourcebase class

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458959 -7200
#  Fri Jul 07 22:22:39 2017 +0200
# Node ID 5d3e659c979aa428ba44b138cfac30b7cca28fb3
# Parent  5b49f653a4a50607127e37e7511c8a8e343cc8d9
# EXP-Topic obs-cache
cache: introduce a dualsourcebase class

This abstract class cover the case where cache are computed from data
contained in the changelog -and- the obsstore.

diff -r 5b49f653a4a5 -r 5d3e659c979a mercurial/cache.py
--- a/mercurial/cache.pyFri Jul 07 22:22:04 2017 +0200
+++ b/mercurial/cache.pyFri Jul 07 22:22:39 2017 +0200
@@ -181,3 +181,38 @@
 
 def _fetchupdatedata(self, repo):
 return repo.obsstore.getmarkerssince(self._cachekey)
+
+class dualsourcecache(obsstoresourcebase, changelogsourcebase):
+"""An abstract class for cache that needs both changelog and obsstore
+
+The cache key used is a combinaison of the one used for the changelog and
+the one used for the obsstore. See inherited class for details.
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = (changelogsourcebase.emptykey
++ obsstoresourcebase.emptykey)
+_cachekeyspec = (changelogsourcebase._cachekeyspec
+ + obsstoresourcebase._cachekeyspec)
+_cachename = None # used for debug message
+
+def _fetchupdatedata(self, repo):
+clkey = self._cachekey[0:2]
+obskey = self._cachekey[2:4]
+
+reset, revs, newclkey = self._fetchchangelogdata(clkey, repo.changelog)
+if reset:
+obskey = obsstoresourcebase.emptykey
+obsreturn = repo.obsstore.getmarkerssince(obskey)
+obsreset, obsmarkers, newobskey = obsreturn
+if obsreset:
+reset = True
+clkey = changelogsourcebase.emptykey
+clreturn = self._fetchchangelogdata(clkey, repo.changelog)
+__, revs, newclkey = clreturn
+
+newkey = newclkey + newobskey
+data = (revs, obsmarkers)
+return reset, data, newkey
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 14 of 14] obscache: skip updating outdated obscache obscache for readonly operation

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1496358960 -7200
#  Fri Jun 02 01:16:00 2017 +0200
# Node ID 59528ec2969e10de1c3eec16149b483083b89218
# Parent  7774ff78ef2e7433368184e7de743342b771b91c
# EXP-Topic obs-cache
obscache: skip updating outdated obscache obscache for readonly operation

During read only operation, we fallback to the old way of computing obsolescence
markers when we detect that the obscache is behind/invalid. This might happen
when both old and new clients touch the same repository.

This is a simple way to avoiding paying extra cache warming overhead without
making the cache write pattern more complex.

diff -r 7774ff78ef2e -r 59528ec2969e mercurial/obsolete.py
--- a/mercurial/obsolete.py Sun Jul 09 03:56:12 2017 +0200
+++ b/mercurial/obsolete.py Fri Jun 02 01:16:00 2017 +0200
@@ -1125,9 +1125,14 @@
 # good old version (parsing markers and checking them). We could add some
 # logic to fall back to the old way in these cases.
 obscache = repo.obsstore.obscache
-obscache.update(repo) # ensure it is up to date:
-isobs = obscache.get
-
+if obscache.uptodate(repo) and repo.currenttransaction() is None:
+hasnode = repo.obsstore.successors.__contains__
+node = repo.changelog.node
+def isobs(rev):
+return hasnode(node(rev))
+else:
+obscache.update(repo) # ensure it is up to date:
+isobs = obscache.get
 return set(r for r in notpublic if isobs(r))
 
 @cachefor('unstable')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 13 of 14] cache: add a way to check if a cache is up to date

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499565372 -7200
#  Sun Jul 09 03:56:12 2017 +0200
# Node ID 7774ff78ef2e7433368184e7de743342b771b91c
# Parent  3a93e99b0e718befd57a32615a14fd0d3c194456
# EXP-Topic obs-cache
cache: add a way to check if a cache is up to date

This will be useful to decide if a cache can be used as-is or not.

diff -r 3a93e99b0e71 -r 7774ff78ef2e mercurial/cache.py
--- a/mercurial/cache.pyFri May 19 14:47:01 2017 +0200
+++ b/mercurial/cache.pySun Jul 09 03:56:12 2017 +0200
@@ -80,6 +80,15 @@
 raise NotImplementedError
 
 @abc.abstractmethod
+def _uptodate(self, repo):
+"""True if the cache is up to date with the repository
+
+This function is for overriding only. Use the public 'uptodate(repo)'
+for actual usage.
+"""
+raise NotImplementedError
+
+@abc.abstractmethod
 def _fetchupdatedata(self, repo):
 """Check the source for possible changes and return necessary data
 
@@ -98,6 +107,12 @@
 
 # Useful "public" function (no need to override them)
 
+def uptodate(self, repo):
+"""True if the cache is up to date with the repository"""
+if self._cachekey is None:
+self.load(repo)
+return self._uptodate(repo)
+
 def update(self, repo):
 """update the cache with new repository data
 
@@ -168,6 +183,17 @@
 else:
 return False, list(cl.revs(start=keyrev + 1, stop=tiprev)), newkey
 
+def _checkchangelogkey(self, cachekey, cl):
+# Exists as its own method to help subclass to reuse it.
+tiprev = len(cl) - 1
+tipnode = cl.node(tiprev)
+newkey = (tiprev, tipnode)
+tiprev = len(cl) - 1
+return newkey == cachekey
+
+def _uptodate(self, repo):
+return self._checkchangelogkey(self._cachekey, repo.changelog)
+
 def _fetchupdatedata(self, repo):
 return self._fetchchangelogdata(self._cachekey, repo.changelog)
 
@@ -188,6 +214,9 @@
 _cachekeyspec = 'I20s'
 _cachename = None # used for debug message
 
+def _uptodate(self, repo):
+return repo.obsstore.matchcontentkey(self._cachekey)
+
 def _fetchupdatedata(self, repo):
 return repo.obsstore.getmarkerssince(self._cachekey)
 
@@ -210,6 +239,10 @@
  + obsstoresourcebase._cachekeyspec)
 _cachename = None # used for debug message
 
+def _uptodate(self, repo):
+return (self._checkchangelogkey(self._cachekey, repo.changelog)
+and repo.obsstore.matchcontentkey(self._cachekey))
+
 def _fetchupdatedata(self, repo):
 clkey = self._cachekey[0:2]
 obskey = self._cachekey[2:4]
diff -r 3a93e99b0e71 -r 7774ff78ef2e mercurial/obsolete.py
--- a/mercurial/obsolete.py Fri May 19 14:47:01 2017 +0200
+++ b/mercurial/obsolete.py Sun Jul 09 03:56:12 2017 +0200
@@ -547,6 +547,21 @@
 
 __bool__ = __nonzero__
 
+def matchcontentkey(self, key):
+"""provide the cachekey (as usable by """
+keysize, keyhash = key
+# XXX This function could avoid loading the whole data from disk (and
+# only read new markers). It currently use 'self._data' to keep the
+# code simpler.
+fulldata = self._data
+currentsize = len(fulldata)
+if currentsize != keysize:
+return False
+first = max(0, keysize - self._obskeyspan)
+keydata = fulldata[first:keysize]
+actualhash = hashlib.sha1(keydata).digest()
+return actualhash == keyhash
+
 def getmarkerssince(self, previouscontentkey):
 """retrieve all new markers (since "contentkey") + updated content key
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 14] obsstore: pass a repository object for initialisation

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1495197862 -7200
#  Fri May 19 14:44:22 2017 +0200
# Node ID 985d753d4f5799f2a332140adedb06efd465d62b
# Parent  63214f4d9a766761259b650539eede424413e6a2
# EXP-Topic obs-cache
obsstore: pass a repository object for initialisation

The cache will needs a repository object (to grab a 'vfs'), so we pass a repo 
object instead of
just the 'svfs' and we grab the 'svfs' from there.

diff -r 63214f4d9a76 -r 985d753d4f57 contrib/perf.py
--- a/contrib/perf.py   Fri May 19 14:46:26 2017 +0200
+++ b/contrib/perf.py   Fri May 19 14:44:22 2017 +0200
@@ -1391,8 +1391,7 @@
 
 Result is the number of markers in the repo."""
 timer, fm = gettimer(ui)
-svfs = getsvfs(repo)
-timer(lambda: len(obsolete.obsstore(svfs)))
+timer(lambda: len(obsolete.obsstore(repo)))
 fm.end()
 
 @command('perflrucachedict', formatteropts +
diff -r 63214f4d9a76 -r 985d753d4f57 mercurial/obsolete.py
--- a/mercurial/obsolete.py Fri May 19 14:46:26 2017 +0200
+++ b/mercurial/obsolete.py Fri May 19 14:44:22 2017 +0200
@@ -519,10 +519,10 @@
 # 1024 byte should be about 10 markers in average
 _obskeyspan = 1024
 
-def __init__(self, svfs, defaultformat=_fm1version, readonly=False):
+def __init__(self, repo, defaultformat=_fm1version, readonly=False):
 # caches for various obsolescence related cache
 self.caches = {}
-self.svfs = svfs
+self.svfs = repo.svfs
 self._defaultformat = defaultformat
 self._readonly = readonly
 
@@ -799,7 +799,7 @@
 if defaultformat is not None:
 kwargs['defaultformat'] = defaultformat
 readonly = not isenabled(repo, createmarkersopt)
-store = obsstore(repo.svfs, readonly=readonly, **kwargs)
+store = obsstore(repo, readonly=readonly, **kwargs)
 if store and readonly:
 ui.warn(_('obsolete feature not enabled but %i markers found!\n')
 % len(list(store)))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 14] cache: introduce a changelogsourcebase class

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458552 -7200
#  Fri Jul 07 22:15:52 2017 +0200
# Node ID 8b71290526ddb77f157e075191dd748793d85601
# Parent  6edb62505c697329de034c2fdc47befd5896f31f
# EXP-Topic obs-cache
cache: introduce a changelogsourcebase class

This abstract class will help code that need a cache tracking the changelog
content (eg: the branchmap cache). The cache key used is the same as what the
branchmap uses.

diff -r 6edb62505c69 -r 8b71290526dd mercurial/cache.py
--- a/mercurial/cache.pyFri Jul 07 22:14:01 2017 +0200
+++ b/mercurial/cache.pyFri Jul 07 22:15:52 2017 +0200
@@ -10,6 +10,7 @@
 import struct
 
 from . import (
+node,
 util,
 )
 
@@ -125,3 +126,41 @@
 def _deserializecachekey(self, data):
 """read the cachekey from bytes"""
 return self._cachekeystruct.unpack(data)
+
+class changelogsourcebase(incrementalcachebase):
+"""an abstract class for cache sourcing data from the changelog
+
+For this purpose it use a cache key covering changelog content.
+The cache key parts are: (tiprev, tipnode)
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = (0, node.nullid)
+_cachekeyspec = 'i20s'
+_cachename = None # used for debug message
+
+# Useful "public" function (no need to override them)
+
+def _fetchchangelogdata(self, cachekey, cl):
+"""use a cachekey to fetch incremental data
+
+Exists as its own method to help subclass to reuse it."""
+tiprev = len(cl) - 1
+tipnode = cl.node(tiprev)
+newkey = (tiprev, tipnode)
+tiprev = len(cl) - 1
+if newkey == cachekey:
+return False, [], newkey
+keyrev, keynode = cachekey
+if tiprev < keyrev or cl.node(keyrev) != keynode:
+revs = ()
+if len(cl):
+revs = list(cl.revs(stop=tiprev))
+return True, revs, newkey
+else:
+return False, list(cl.revs(start=keyrev + 1, stop=tiprev)), newkey
+
+def _fetchupdatedata(self, repo):
+return self._fetchchangelogdata(self._cachekey, repo.changelog)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 14] cache: introduce a changelogsourcebase class

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499458552 -7200
#  Fri Jul 07 22:15:52 2017 +0200
# Node ID 8b71290526ddb77f157e075191dd748793d85601
# Parent  6edb62505c697329de034c2fdc47befd5896f31f
# EXP-Topic obs-cache
cache: introduce a changelogsourcebase class

This abstract class will help code that need a cache tracking the changelog
content (eg: the branchmap cache). The cache key used is the same as what the
branchmap uses.

diff -r 6edb62505c69 -r 8b71290526dd mercurial/cache.py
--- a/mercurial/cache.pyFri Jul 07 22:14:01 2017 +0200
+++ b/mercurial/cache.pyFri Jul 07 22:15:52 2017 +0200
@@ -10,6 +10,7 @@
 import struct
 
 from . import (
+node,
 util,
 )
 
@@ -125,3 +126,41 @@
 def _deserializecachekey(self, data):
 """read the cachekey from bytes"""
 return self._cachekeystruct.unpack(data)
+
+class changelogsourcebase(incrementalcachebase):
+"""an abstract class for cache sourcing data from the changelog
+
+For this purpose it use a cache key covering changelog content.
+The cache key parts are: (tiprev, tipnode)
+"""
+
+__metaclass__ = abc.ABCMeta
+
+# default key used for an empty cache
+emptykey = (0, node.nullid)
+_cachekeyspec = 'i20s'
+_cachename = None # used for debug message
+
+# Useful "public" function (no need to override them)
+
+def _fetchchangelogdata(self, cachekey, cl):
+"""use a cachekey to fetch incremental data
+
+Exists as its own method to help subclass to reuse it."""
+tiprev = len(cl) - 1
+tipnode = cl.node(tiprev)
+newkey = (tiprev, tipnode)
+tiprev = len(cl) - 1
+if newkey == cachekey:
+return False, [], newkey
+keyrev, keynode = cachekey
+if tiprev < keyrev or cl.node(keyrev) != keynode:
+revs = ()
+if len(cl):
+revs = list(cl.revs(stop=tiprev))
+return True, revs, newkey
+else:
+return False, list(cl.revs(start=keyrev + 1, stop=tiprev)), newkey
+
+def _fetchupdatedata(self, repo):
+return self._fetchchangelogdata(self._cachekey, repo.changelog)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] cleanupnode: do not use generator for node mapping

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499605879 -7200
#  Sun Jul 09 15:11:19 2017 +0200
# Node ID a6ed7f0670010e5a6551d736b00dc0bf7367bba8
# Parent  4672db164c986da4442bd864cd044512d975c3f2
# EXP-Topic fix-cleanup
cleanupnode: do not use generator for node mapping

The 'successors' part of the mappings used of be a tuple. This avoid issue from
code consuming the generator "by mistake". For example, an extension inspecting 
the
mapping content used to be able to iterate over the successors mapping without
consequence.

Since the mapping are small we do not expect any performance impact we use tuple
again for this.

diff -r 4672db164c98 -r a6ed7f067001 mercurial/scmutil.py
--- a/mercurial/scmutil.py  Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/scmutil.py  Sun Jul 09 15:11:19 2017 +0200
@@ -638,7 +638,7 @@
 isobs = unfi.obsstore.successors.__contains__
 torev = unfi.changelog.rev
 sortfunc = lambda ns: torev(ns[0])
-rels = [(unfi[n], (unfi[m] for m in s))
+rels = [(unfi[n], tuple(unfi[m] for m in s))
 for n, s in sorted(mapping.items(), key=sortfunc)
 if s or not isobs(n)]
 obsolete.createmarkers(repo, rels, operation=operation)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 02 of 14] cache: introduce a changelogsourcebase class

[Boris Feld]

On Sun, 2017-07-09 at 19:52 +0200, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.f...@octobus.net>
> # Date 1499458552 -7200
> #  Fri Jul 07 22:15:52 2017 +0200
> # Node ID 8b71290526ddb77f157e075191dd748793d85601
> # Parent  6edb62505c697329de034c2fdc47befd5896f31f
> # EXP-Topic obs-cache
> cache: introduce a changelogsourcebase class
> 
Sorry, my battery has discharged while sending the series.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 12 of 14] vfs: add the possibility to have a "ward" to check vfs usage

[Boris Feld]

On Sat, 2017-07-08 at 16:36 +0900, Yuya Nishihara wrote:
> On Fri, 07 Jul 2017 18:48:44 +0200, Boris Feld wrote:
> > On Fri, 2017-07-07 at 21:48 +0900, Yuya Nishihara wrote:
> > > On Thu, 06 Jul 2017 20:53:50 +0200, Boris Feld wrote:
> > > > On Wed, 2017-07-05 at 23:55 +0900, Yuya Nishihara wrote:
> > > > > On Sun, 02 Jul 2017 04:56:37 +0200, Pierre-Yves David wrote:
> > > > > > # HG changeset patch
> > > > > > # User Pierre-Yves David <pierre-yves.da...@ens-lyon.org>
> > > > > > # Date 1470323266 -7200
> > > > > > #  Thu Aug 04 17:07:46 2016 +0200
> > > > > > # Node ID ebf81efdd6d7ff15c64683933635616c00475688
> > > > > > # Parent  34b8be7f0420b07d0f7b71379c6055e5b26223d5
> > > > > > # EXP-Topic vfs.ward
> > > > > > # Available At https://www.mercurial-scm.org/repo/users/mar
> > > > > > mout
> > > > > > e/me
> > > > > > rcurial/
> > > > > > #  hg pull https://www.mercurial-scm.org/repo/u
> > > > > > sers
> > > > > > /mar
> > > > > > moute/mercurial/ -r ebf81efdd6d7
> > > > > > vfs: add the possibility to have a "ward" to check vfs
> > > > > > usage
> > > > > > The feature has some similarity with the 'pathauditor', but
> > > > > > further
> > > > > > digging
> > > > > > show it make more sense to keep them separated. The path
> > > > > > auditor is
> > > > > > fairly
> > > > > > autonomous (create from vfs.__init__ without any extra
> > > > > > information), and check
> > > > > > very generic details. On the other hand, the wards will
> > > > > > have
> > > > > > deeper
> > > > > > knownledge
> > > > > > of the Mercurial logic and is created at the local
> > > > > > repository
> > > > > > level. Mixing the
> > > > > > two would add much complexity for no real gains.
> > > > > 
> > > > > My gut feeling is that vfs isn't the right place if they
> > > > > require
> > > > > deeper
> > > > > knowledge of repository/dirstate logic. And the whole weakref
> > > > > business
> > > > > floating around wards, hooks and transaction seems like just
> > > > > getting
> > > > > around
> > > > > layering violation.
> > > > 
> > > > We pondered on this choice a long time and couldn't find any
> > > > other
> > > > layer that is used by both Mercurial core and the extensions to
> > > > access
> > > > the file system. It seems to be the highest abstraction layer
> > > > we
> > > > could
> > > > hook into without introducing a new one and updating all
> > > > callers.
> > > > 
> > > > Our reflexion is, as the vfs classes comes from the scmutil
> > > > file,
> > > > it
> > > > seems okay to have scm related logic in that layer.
> > > 
> > > IIRC, one possible alternative was to move lock itself to a vfs
> > > as
> > > the
> > > whole vfs is theoretically covered by the lock.
> > 
> > The practice divert a bit from the theory. For example, the
> > ".hg/bookmark" file is in theory covered by the 'wlock'. In
> > practice it
> > is now covered by the transaction and therefor the 'lock'. On the
> > other
> > hand there are multiple files handled by the vfs ('hgrc',
> > 'dirstate',
> > 'store/', etc) that are not covered by the wlock.
> > 
> > Right now, all these exceptions are stored on the repository, and
> > extension can update them. To us, the repository layers seems the
> > right
> > layer for logic regarding individual file semantic. We also want to
> > add
> > some logic regarding open transaction to the ward, something
> > unrelated
> > to the vfs.
> > 
> > The initial motivation for this series (beside catching bug) is to
> > make
> > it safer to introduce new vfs. For example, the share extensions
> > suffers from lack of cache sharing, each share has its own
> > '.hg/cache/'
> > directory. Introducing a 'repo.cvfs' dedicated to cache could fix
> > that,
> > but we wants to warn people still accessing 'cache/' through
> &

[PATCH 1 of 3 V2] vfs: allow to pass more argument to audit

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499768878 -7200
#  Tue Jul 11 12:27:58 2017 +0200
# Node ID b780af8186e280e459365533133ea6bbad5c6ae2
# Parent  4672db164c986da4442bd864cd044512d975c3f2
# EXP-Topic vfs.ward
vfs: allow to pass more argument to audit

We want to be able to do more precise check when auditing a path depending of
the intend of the file access (eg read versus write). So we now pass the 'mode'
and 'atomictemp' value to 'audit' and update the audit function to accept them.

This will be put to use in the next changeset.

diff -r 4672db164c98 -r b780af8186e2 mercurial/pathutil.py
--- a/mercurial/pathutil.py Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/pathutil.py Tue Jul 11 12:27:58 2017 +0200
@@ -46,7 +46,7 @@
 else:
 self.normcase = lambda x: x
 
-def __call__(self, path):
+def __call__(self, path, mode=None, atomictemp=None):
 '''Check the relative path.
 path may contain a pattern (e.g. foodir/**.txt)'''
 
diff -r 4672db164c98 -r b780af8186e2 mercurial/vfs.py
--- a/mercurial/vfs.py  Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/vfs.py  Tue Jul 11 12:27:58 2017 +0200
@@ -306,7 +306,7 @@
 if audit:
 self.audit = pathutil.pathauditor(self.base)
 else:
-self.audit = util.always
+self.audit =  (lambda path, mode=None, atomictemp=None: True)
 self.createmode = None
 self._trustnlink = None
 
@@ -360,7 +360,7 @@
 r = util.checkosfilename(path)
 if r:
 raise error.Abort("%s: %r" % (r, path))
-self.audit(path)
+self.audit(path, mode=mode, atomictemp=atomictemp)
 f = self.join(path)
 
 if not text and "b" not in mode:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 3 V2] repovfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1499769497 -7200
#  Tue Jul 11 12:38:17 2017 +0200
# Node ID a630c1dfccedff6a47d16bda8c47082caf24b24c
# Parent  b780af8186e280e459365533133ea6bbad5c6ae2
# EXP-Topic vfs.ward
repovfs: add a ward to check if locks are properly taken


When the appropriate developer warning are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).

This check system have caught a handful of locking issues that  has been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.

We introduces two real exceptions from extensions 'blackbox.log' (because a lot 
of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).

In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.

diff -r b780af8186e2 -r a630c1dfcced hgext/blackbox.py
--- a/hgext/blackbox.py Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/blackbox.py Tue Jul 11 12:38:17 2017 +0200
@@ -236,6 +236,7 @@
 
 if util.safehasattr(ui, 'setrepo'):
 ui.setrepo(repo)
+repo._wlockfreeprefix.add('blackbox.log')
 
 @command('^blackbox',
 [('l', 'limit', 10, _('the number of events to show')),
diff -r b780af8186e2 -r a630c1dfcced hgext/journal.py
--- a/hgext/journal.py  Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/journal.py  Tue Jul 11 12:38:17 2017 +0200
@@ -69,6 +69,7 @@
 def reposetup(ui, repo):
 if repo.local():
 repo.journal = journalstorage(repo)
+repo._wlockfreeprefix.add('namejournal')
 
 def runcommand(orig, lui, repo, cmd, fullargs, *args):
 """Track the command line options for recording in the journal"""
diff -r b780af8186e2 -r a630c1dfcced hgext/patchbomb.py
--- a/hgext/patchbomb.pyTue Jul 11 12:27:58 2017 +0200
+++ b/hgext/patchbomb.pyTue Jul 11 12:38:17 2017 +0200
@@ -122,6 +122,10 @@
 cmdutil.extraexport.append('pullurl')
 cmdutil.extraexportmap['pullurl'] = _addpullheader
 
+def reposetup(ui, repo):
+if not repo.local():
+return
+repo._wlockfreeprefix.add('last-email.txt')
 
 def prompt(ui, prompt, default=None, rest=':'):
 if default:
diff -r b780af8186e2 -r a630c1dfcced mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:27:58 2017 +0200
+++ b/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
@@ -289,6 +289,25 @@
 # only functions defined in module of enabled extensions are invoked
 featuresetupfuncs = set()
 
+# list of prefix for file which can be written without 'wlock'
+# Extensions should extend this list when needed
+_wlockfreeprefix = set([# We migh consider requiring 'wlock' for the next
+# two, but pretty much all the existing code assume
+# wlock is not needed so we keep them excluded for
+# now.
+'hgrc',
+'requires',
+# XXX cache is a complicatged business someone
+# should investigate this in depth at some point
+'cache/',
+# XXX shouldn't be dirstate covered by the wlock?
+'dirstate',
+# XXX bisect was still a bit too messy at the time
+# this changeset was introduced. Someone should fix
+# the remainig bit and drop this line
+'bisect.state',
+])
+
 def __init__(self, baseui, path, create=False):
 self.requirements = set()
 self.filtername = None
@@ -308,10 +327,13 @@
 self.auditor = pathutil.pathauditor(self.root, self._checknested)
 self.nofsauditor = pathutil.pathauditor(self.root, self._checknested,
 realfs=False)
-self.vfs = vfsmod.vfs(self.path)
 self.baseui = baseui
 self.ui = baseui.copy()
 self.ui.copy = baseui.copy # prevent copying repo configuration
+self.vfs = vfsmod.vfs(self.path)
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+self.vfs.audit = self._getvfsward(self.vfs.audit)
 # A list of callback to shape the phase if no data were found.
 # Callback are in the form: func(repo, roots) --> processed root.
 # This list it to be filled by extension during repo setup
@@ -427,6 +449,38 @@
 # Signature to cached matcher instance.
 self._sparsematchercache = 

[PATCH 3 of 3 V2] reposvfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Octobus 
# Date 1470672882 -7200
#  Mon Aug 08 18:14:42 2016 +0200
# Node ID 443e188172a87eb4700a84cdfc16bd60ca3d4989
# Parent  a630c1dfccedff6a47d16bda8c47082caf24b24c
# EXP-Topic vfs.ward
reposvfs: add a ward to check if locks are properly taken

we wrap 'repo.svfs.audit' to check for the store lock when accessing file in
'.hg/store' for writing. This caught a couple of instance where the transaction
was released after the lock, we should probably have a dedicated checker for
that case.

diff -r a630c1dfcced -r 443e188172a8 mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
+++ b/mercurial/localrepo.pyMon Aug 08 18:14:42 2016 +0200
@@ -411,6 +411,12 @@
 self.svfs = self.store.vfs
 self.sjoin = self.store.join
 self.vfs.createmode = self.store.createmode
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+if util.safehasattr(self.svfs, 'vfs'): # this is filtervfs
+self.svfs.vfs.audit = self._getsvfsward(self.svfs.vfs.audit)
+else: # standard vfs
+self.svfs.audit = self._getsvfsward(self.svfs.audit)
 self._applyopenerreqs()
 if create:
 self._writerequirements()
@@ -481,6 +487,25 @@
 return ret
 return checkvfs
 
+def _getsvfsward(self, origfunc):
+"""build a ward for self.svfs"""
+rref = weakref.ref(self)
+def checksvfs(path, mode=None, atomictemp=None):
+ret = origfunc(path, mode=mode, atomictemp=atomictemp)
+repo = rref()
+if repo is None or not util.safehasattr(repo, '_lockref'):
+return
+if mode in (None, 'r', 'rb'):
+return
+if path.startswith(repo.sharedpath):
+# truncate name relative to the repository (.hg)
+path = path[len(repo.sharedpath) + 1:]
+if repo._currentlock(repo._lockref) is None:
+repo.ui.develwarn('write with no lock: "%s"' % path,
+  stacklevel=3)
+return ret
+return checksvfs
+
 def close(self):
 self._writecaches()
 
diff -r a630c1dfcced -r 443e188172a8 tests/test-devel-warnings.t
--- a/tests/test-devel-warnings.t   Tue Jul 11 12:38:17 2017 +0200
+++ b/tests/test-devel-warnings.t   Mon Aug 08 18:14:42 2016 +0200
@@ -49,6 +49,11 @@
   > with repo.vfs(b'branch', 'a'):
   > pass
   > 
+  > @command(b'no-lock-write', [], '')
+  > def nolockwrite(ui, repo):
+  > with repo.svfs(b'fncache', 'a'):
+  > pass
+  > 
   > @command(b'stripintr', [], '')
   > def stripintr(ui, repo):
   > lo = repo.lock()
@@ -114,6 +119,9 @@
   $ hg no-wlock-write
   devel-warn: write with no wlock: "branch" at: $TESTTMP/buggylocking.py:* 
(nowlockwrite) (glob)
 
+  $ hg no-lock-write
+  devel-warn: write with no lock: "fncache" at: $TESTTMP/buggylocking.py:* 
(nolockwrite) (glob)
+
 Stripping from a transaction
 
   $ echo a > a
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 01 of 10 V2] configitems: register the 'bugzilla.apikey' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414602 -7200
#  Fri Jul 07 10:03:22 2017 +0200
# Node ID 63cf0aae1472ea685e1543472ea722b0ea89784e
# Parent  89796a25d4bb91fb418ad3e70faad2c586902ffb
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.apikey' config

diff -r 89796a25d4bb -r 63cf0aae1472 hgext/bugzilla.py
--- a/hgext/bugzilla.py Mon Jul 03 11:22:00 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:22 2017 +0200
@@ -303,6 +303,7 @@
 cmdutil,
 error,
 mail,
+registrar,
 url,
 util,
 )
@@ -315,6 +316,13 @@
 # leave the attribute unspecified.
 testedwith = 'ships-with-hg-core'
 
+configtable = {}
+configitem = registrar.configitem(configtable)
+
+configitem('bugzilla', 'apikey',
+default='',
+)
+
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
 
@@ -800,7 +808,7 @@
 bz = self.ui.config('bugzilla', 'bzurl',
 'http://localhost/bugzilla/')
 self.bzroot = '/'.join([bz, 'rest'])
-self.apikey = self.ui.config('bugzilla', 'apikey', '')
+self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 10 V2] configitems: register the 'bugzilla.bzdir' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414604 -7200
#  Fri Jul 07 10:03:24 2017 +0200
# Node ID 3d5a8cdda2a96f147a2542c65aeb8d8addec06f0
# Parent  63cf0aae1472ea685e1543472ea722b0ea89784e
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzdir' config

diff -r 63cf0aae1472 -r 3d5a8cdda2a9 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:22 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:24 2017 +0200
@@ -322,6 +322,9 @@
 configitem('bugzilla', 'apikey',
 default='',
 )
+configitem('bugzilla', 'bzdir',
+default='/var/www/html/bugzilla',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -457,8 +460,7 @@
 for id in bugs.keys():
 self.ui.status(_('  bug %s\n') % id)
 cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)
-bzdir = self.ui.config('bugzilla', 'bzdir',
-   '/var/www/html/bugzilla')
+bzdir = self.ui.config('bugzilla', 'bzdir')
 try:
 # Backwards-compatible with old notify string, which
 # took one string. This will throw with a new format
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 04 of 10 V2] configitems: register the 'bugzilla.bzurl' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414608 -7200
#  Fri Jul 07 10:03:28 2017 +0200
# Node ID b98505c496cf15809cd90e5bdab5165d4731dbd8
# Parent  97aa2edd521c6999219991836ba66abb0aeb2bd9
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzurl' config

diff -r 97aa2edd521c -r b98505c496cf hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:26 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:28 2017 +0200
@@ -328,6 +328,9 @@
 configitem('bugzilla', 'bzemail',
 default=None,
 )
+configitem('bugzilla', 'bzurl',
+default='http://localhost/bugzilla/',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -649,8 +652,7 @@
 def __init__(self, ui):
 bzaccess.__init__(self, ui)
 
-bzweb = self.ui.config('bugzilla', 'bzurl',
-   'http://localhost/bugzilla/')
+bzweb = self.ui.config('bugzilla', 'bzurl')
 bzweb = bzweb.rstrip("/") + "/xmlrpc.cgi"
 
 user = self.ui.config('bugzilla', 'user', 'bugs')
@@ -810,8 +812,7 @@
 """
 def __init__(self, ui):
 bzaccess.__init__(self, ui)
-bz = self.ui.config('bugzilla', 'bzurl',
-'http://localhost/bugzilla/')
+bz = self.ui.config('bugzilla', 'bzurl')
 self.bzroot = '/'.join([bz, 'rest'])
 self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 10 V2] configitems: register the 'bugzilla.bzemail' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414606 -7200
#  Fri Jul 07 10:03:26 2017 +0200
# Node ID 97aa2edd521c6999219991836ba66abb0aeb2bd9
# Parent  3d5a8cdda2a96f147a2542c65aeb8d8addec06f0
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzemail' config

diff -r 3d5a8cdda2a9 -r 97aa2edd521c hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:24 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:26 2017 +0200
@@ -325,6 +325,9 @@
 configitem('bugzilla', 'bzdir',
 default='/var/www/html/bugzilla',
 )
+configitem('bugzilla', 'bzemail',
+default=None,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 10 V2] configitems: register the 'bugzilla.bzuser' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414611 -7200
#  Fri Jul 07 10:03:31 2017 +0200
# Node ID 3d09ecfeedc7f19db069bd2cac8e269c7e561b05
# Parent  b98505c496cf15809cd90e5bdab5165d4731dbd8
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.bzuser' config

diff -r b98505c496cf -r 3d09ecfeedc7 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:28 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:31 2017 +0200
@@ -331,6 +331,9 @@
 configitem('bugzilla', 'bzurl',
 default='http://localhost/bugzilla/',
 )
+configitem('bugzilla', 'bzuser',
+default=None,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10 V2] configitems: register the 'bugzilla.db' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414614 -7200
#  Fri Jul 07 10:03:34 2017 +0200
# Node ID cdef7e9e6b3cd6785755edb3cb1be3cb9cec2799
# Parent  3d09ecfeedc7f19db069bd2cac8e269c7e561b05
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.db' config

diff -r 3d09ecfeedc7 -r cdef7e9e6b3c hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:31 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:34 2017 +0200
@@ -334,6 +334,9 @@
 configitem('bugzilla', 'bzuser',
 default=None,
 )
+configitem('bugzilla', 'db',
+default='bugs',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -412,7 +415,7 @@
 host = self.ui.config('bugzilla', 'host', 'localhost')
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
-db = self.ui.config('bugzilla', 'db', 'bugs')
+db = self.ui.config('bugzilla', 'db')
 timeout = int(self.ui.config('bugzilla', 'timeout', 5))
 self.ui.note(_('connecting to %s:%s as %s, password %s\n') %
  (host, db, user, '*' * len(passwd)))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 10 V2] configitems: register the 'bugzilla.fixstatus' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414637 -7200
#  Fri Jul 07 10:03:57 2017 +0200
# Node ID b065c6b0905ff4304d5f049e4b56ed55247339c0
# Parent  7899b38ce3e21d04d578f101bc6fc503600ca07b
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixstatus' config

diff -r 7899b38ce3e2 -r b065c6b0905f hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:05:40 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:57 2017 +0200
@@ -343,6 +343,9 @@
 configitem('bugzilla', 'fixresolution',
 default='FIXED',
 )
+configitem('bugzilla', 'fixstatus',
+default='RESOLVED',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -670,7 +673,7 @@
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
 
-self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
+self.fixstatus = self.ui.config('bugzilla', 'fixstatus')
 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
@@ -828,7 +831,7 @@
 self.apikey = self.ui.config('bugzilla', 'apikey')
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
-self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
+self.fixstatus = self.ui.config('bugzilla', 'fixstatus')
 self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 def apiurl(self, targets, include_fields=None):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 10 V2] configitems: register the 'bugzilla.fixresolution' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414740 -7200
#  Fri Jul 07 10:05:40 2017 +0200
# Node ID 7899b38ce3e21d04d578f101bc6fc503600ca07b
# Parent  0bb83a78698c7762b4c0a897797f5340cc9d0684
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixresolution' config

diff -r 0bb83a78698c -r 7899b38ce3e2 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:36 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:05:40 2017 +0200
@@ -340,6 +340,9 @@
 configitem('bugzilla', 'fixregexp',
 default=lambda: bugzilla._default_fix_re,
 )
+configitem('bugzilla', 'fixresolution',
+default='FIXED',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -668,8 +671,7 @@
 passwd = self.ui.config('bugzilla', 'password')
 
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
-self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
-'FIXED')
+self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
 ver = self.bzproxy.Bugzilla.version()['version'].split('.')
@@ -827,8 +829,7 @@
 self.user = self.ui.config('bugzilla', 'user', 'bugs')
 self.passwd = self.ui.config('bugzilla', 'password')
 self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
-self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
-'FIXED')
+self.fixresolution = self.ui.config('bugzilla', 'fixresolution')
 
 def apiurl(self, targets, include_fields=None):
 url = '/'.join([self.bzroot] + [str(t) for t in targets])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10 V2] configitems: register the 'bugzilla.fixregexp' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414616 -7200
#  Fri Jul 07 10:03:36 2017 +0200
# Node ID 0bb83a78698c7762b4c0a897797f5340cc9d0684
# Parent  cdef7e9e6b3cd6785755edb3cb1be3cb9cec2799
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.fixregexp' config

diff -r cdef7e9e6b3c -r 0bb83a78698c hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:34 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:03:36 2017 +0200
@@ -337,6 +337,9 @@
 configitem('bugzilla', 'db',
 default='bugs',
 )
+configitem('bugzilla', 'fixregexp',
+default=lambda: bugzilla._default_fix_re,
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -975,8 +978,7 @@
 self.ui.config('bugzilla', 'regexp',
bugzilla._default_bug_re), re.IGNORECASE)
 self.fix_re = re.compile(
-self.ui.config('bugzilla', 'fixregexp',
-   bugzilla._default_fix_re), re.IGNORECASE)
+self.ui.config('bugzilla', 'fixregexp'), re.IGNORECASE)
 self.split_re = re.compile(r'\D+')
 
 def find_bugs(self, ctx):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 10 V2] configitems: register the 'bugzilla.host' config

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499414641 -7200
#  Fri Jul 07 10:04:01 2017 +0200
# Node ID 18c26fe8b1863306f3a9ef9b585e69517ba957e3
# Parent  b065c6b0905ff4304d5f049e4b56ed55247339c0
# EXP-Topic config.register.bugzilla
configitems: register the 'bugzilla.host' config

diff -r b065c6b0905f -r 18c26fe8b186 hgext/bugzilla.py
--- a/hgext/bugzilla.py Fri Jul 07 10:03:57 2017 +0200
+++ b/hgext/bugzilla.py Fri Jul 07 10:04:01 2017 +0200
@@ -346,6 +346,9 @@
 configitem('bugzilla', 'fixstatus',
 default='RESOLVED',
 )
+configitem('bugzilla', 'host',
+default='localhost',
+)
 
 class bzaccess(object):
 '''Base class for access to Bugzilla.'''
@@ -421,7 +424,7 @@
 
 bzaccess.__init__(self, ui)
 
-host = self.ui.config('bugzilla', 'host', 'localhost')
+host = self.ui.config('bugzilla', 'host')
 user = self.ui.config('bugzilla', 'user', 'bugs')
 passwd = self.ui.config('bugzilla', 'password')
 db = self.ui.config('bugzilla', 'db')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 3 V3] repovfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499769497 -7200
#  Tue Jul 11 12:38:17 2017 +0200
# Node ID aacbe5f5ce3b41c0cf2332d05bf732e265c834a2
# Parent  9addf65400ec8e6052a399c1586320988d73a321
# EXP-Topic vfs.ward
repovfs: add a ward to check if locks are properly taken


When the appropriate developer warning are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).

This check system have caught a handful of locking issues that  has been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.

We introduces two real exceptions from extensions 'blackbox.log' (because a lot 
of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).

In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.

diff -r 9addf65400ec -r aacbe5f5ce3b hgext/blackbox.py
--- a/hgext/blackbox.py Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/blackbox.py Tue Jul 11 12:38:17 2017 +0200
@@ -236,6 +236,7 @@
 
 if util.safehasattr(ui, 'setrepo'):
 ui.setrepo(repo)
+repo._wlockfreeprefix.add('blackbox.log')
 
 @command('^blackbox',
 [('l', 'limit', 10, _('the number of events to show')),
diff -r 9addf65400ec -r aacbe5f5ce3b hgext/journal.py
--- a/hgext/journal.py  Tue Jul 11 12:27:58 2017 +0200
+++ b/hgext/journal.py  Tue Jul 11 12:38:17 2017 +0200
@@ -69,6 +69,7 @@
 def reposetup(ui, repo):
 if repo.local():
 repo.journal = journalstorage(repo)
+repo._wlockfreeprefix.add('namejournal')
 
 def runcommand(orig, lui, repo, cmd, fullargs, *args):
 """Track the command line options for recording in the journal"""
diff -r 9addf65400ec -r aacbe5f5ce3b hgext/patchbomb.py
--- a/hgext/patchbomb.pyTue Jul 11 12:27:58 2017 +0200
+++ b/hgext/patchbomb.pyTue Jul 11 12:38:17 2017 +0200
@@ -122,6 +122,10 @@
 cmdutil.extraexport.append('pullurl')
 cmdutil.extraexportmap['pullurl'] = _addpullheader
 
+def reposetup(ui, repo):
+if not repo.local():
+return
+repo._wlockfreeprefix.add('last-email.txt')
 
 def prompt(ui, prompt, default=None, rest=':'):
 if default:
diff -r 9addf65400ec -r aacbe5f5ce3b mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:27:58 2017 +0200
+++ b/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
@@ -289,6 +289,25 @@
 # only functions defined in module of enabled extensions are invoked
 featuresetupfuncs = set()
 
+# list of prefix for file which can be written without 'wlock'
+# Extensions should extend this list when needed
+_wlockfreeprefix = set([# We migh consider requiring 'wlock' for the next
+# two, but pretty much all the existing code assume
+# wlock is not needed so we keep them excluded for
+# now.
+'hgrc',
+'requires',
+# XXX cache is a complicatged business someone
+# should investigate this in depth at some point
+'cache/',
+# XXX shouldn't be dirstate covered by the wlock?
+'dirstate',
+# XXX bisect was still a bit too messy at the time
+# this changeset was introduced. Someone should fix
+# the remainig bit and drop this line
+'bisect.state',
+])
+
 def __init__(self, baseui, path, create=False):
 self.requirements = set()
 self.filtername = None
@@ -308,10 +327,13 @@
 self.auditor = pathutil.pathauditor(self.root, self._checknested)
 self.nofsauditor = pathutil.pathauditor(self.root, self._checknested,
 realfs=False)
-self.vfs = vfsmod.vfs(self.path)
 self.baseui = baseui
 self.ui = baseui.copy()
 self.ui.copy = baseui.copy # prevent copying repo configuration
+self.vfs = vfsmod.vfs(self.path)
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+self.vfs.audit = self._getvfsward(self.vfs.audit)
 # A list of callback to shape the phase if no data were found.
 # Callback are in the form: func(repo, roots) --> processed root.
 # This list it to be filled by extension during repo setup
@@ -427,6 +449,38 @@
 # Signature to

[PATCH 3 of 3 V3] reposvfs: add a ward to check if locks are properly taken

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1470672882 -7200
#  Mon Aug 08 18:14:42 2016 +0200
# Node ID 9856ed393ddd28ee570486e495bb9fd9762b3c9f
# Parent  aacbe5f5ce3b41c0cf2332d05bf732e265c834a2
# EXP-Topic vfs.ward
reposvfs: add a ward to check if locks are properly taken

we wrap 'repo.svfs.audit' to check for the store lock when accessing file in
'.hg/store' for writing. This caught a couple of instance where the transaction
was released after the lock, we should probably have a dedicated checker for
that case.

diff -r aacbe5f5ce3b -r 9856ed393ddd mercurial/localrepo.py
--- a/mercurial/localrepo.pyTue Jul 11 12:38:17 2017 +0200
+++ b/mercurial/localrepo.pyMon Aug 08 18:14:42 2016 +0200
@@ -411,6 +411,12 @@
 self.svfs = self.store.vfs
 self.sjoin = self.store.join
 self.vfs.createmode = self.store.createmode
+if (self.ui.configbool('devel', 'all-warnings') or
+self.ui.configbool('devel', 'check-locks')):
+if util.safehasattr(self.svfs, 'vfs'): # this is filtervfs
+self.svfs.vfs.audit = self._getsvfsward(self.svfs.vfs.audit)
+else: # standard vfs
+self.svfs.audit = self._getsvfsward(self.svfs.audit)
 self._applyopenerreqs()
 if create:
 self._writerequirements()
@@ -481,6 +487,25 @@
 return ret
 return checkvfs
 
+def _getsvfsward(self, origfunc):
+"""build a ward for self.svfs"""
+rref = weakref.ref(self)
+def checksvfs(path, mode=None, atomictemp=None):
+ret = origfunc(path, mode=mode, atomictemp=atomictemp)
+repo = rref()
+if repo is None or not util.safehasattr(repo, '_lockref'):
+return
+if mode in (None, 'r', 'rb'):
+return
+if path.startswith(repo.sharedpath):
+# truncate name relative to the repository (.hg)
+path = path[len(repo.sharedpath) + 1:]
+if repo._currentlock(repo._lockref) is None:
+repo.ui.develwarn('write with no lock: "%s"' % path,
+  stacklevel=3)
+return ret
+return checksvfs
+
 def close(self):
 self._writecaches()
 
diff -r aacbe5f5ce3b -r 9856ed393ddd tests/test-devel-warnings.t
--- a/tests/test-devel-warnings.t   Tue Jul 11 12:38:17 2017 +0200
+++ b/tests/test-devel-warnings.t   Mon Aug 08 18:14:42 2016 +0200
@@ -49,6 +49,11 @@
   > with repo.vfs(b'branch', 'a'):
   > pass
   > 
+  > @command(b'no-lock-write', [], '')
+  > def nolockwrite(ui, repo):
+  > with repo.svfs(b'fncache', 'a'):
+  > pass
+  > 
   > @command(b'stripintr', [], '')
   > def stripintr(ui, repo):
   > lo = repo.lock()
@@ -114,6 +119,9 @@
   $ hg no-wlock-write
   devel-warn: write with no wlock: "branch" at: $TESTTMP/buggylocking.py:* 
(nowlockwrite) (glob)
 
+  $ hg no-lock-write
+  devel-warn: write with no lock: "fncache" at: $TESTTMP/buggylocking.py:* 
(nolockwrite) (glob)
+
 Stripping from a transaction
 
   $ echo a > a
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3 V3] vfs: allow to pass more argument to audit

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499768878 -7200
#  Tue Jul 11 12:27:58 2017 +0200
# Node ID 9addf65400ec8e6052a399c1586320988d73a321
# Parent  4672db164c986da4442bd864cd044512d975c3f2
# EXP-Topic vfs.ward
vfs: allow to pass more argument to audit

We want to be able to do more precise check when auditing a path depending of
the intend of the file access (eg read versus write). So we now pass the 'mode'
and 'atomictemp' value to 'audit' and update the audit function to accept them.

This will be put to use in the next changeset.

diff -r 4672db164c98 -r 9addf65400ec mercurial/pathutil.py
--- a/mercurial/pathutil.py Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/pathutil.py Tue Jul 11 12:27:58 2017 +0200
@@ -46,7 +46,7 @@
 else:
 self.normcase = lambda x: x
 
-def __call__(self, path):
+def __call__(self, path, mode=None, atomictemp=None):
 '''Check the relative path.
 path may contain a pattern (e.g. foodir/**.txt)'''
 
diff -r 4672db164c98 -r 9addf65400ec mercurial/vfs.py
--- a/mercurial/vfs.py  Sat Jun 24 15:29:42 2017 -0700
+++ b/mercurial/vfs.py  Tue Jul 11 12:27:58 2017 +0200
@@ -306,7 +306,7 @@
 if audit:
 self.audit = pathutil.pathauditor(self.base)
 else:
-self.audit = util.always
+self.audit =  (lambda path, mode=None, atomictemp=None: True)
 self.createmode = None
 self._trustnlink = None
 
@@ -360,7 +360,7 @@
 r = util.checkosfilename(path)
 if r:
 raise error.Abort("%s: %r" % (r, path))
-self.audit(path)
+self.audit(path, mode=mode, atomictemp=atomictemp)
 f = self.join(path)
 
 if not text and "b" not in mode:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 9 of 9] template: use template-engine for obsfate

[Boris Feld]

On Fri, 2017-07-07 at 21:09 +0900, Yuya Nishihara wrote:
> On Thu, 06 Jul 2017 23:50:17 +0200, Boris Feld wrote:
> > # HG changeset patch
> > # User Boris Feld <boris.f...@octobus.net>
> > # Date 1499085172 -7200
> > #  Mon Jul 03 14:32:52 2017 +0200
> > # Node ID e18d8e61b7260e246a82727c8cde01f936692cff
> > # Parent  098585d4fbc88dc54513e12fa306d0e52ea2b323
> > # EXP-Topic obsfatetemplate
> > template: use template-engine for obsfate
> > 
> > Try to replace the obsfateprinter part by "sub-templates".
> > 
> > The code is hacky, I've tried to use at maximum the template engine
> > but the
> > raw data-structure doesn't seems well supported:
> > 
> > [{'markers': [{}, {}, ...], 'successors': [A, ...], 'verb': '',
> > ...}, ...]
> > 
> > I've put this changeset at the end so the beginning of the serie
> > might be
> > accepted without it. Even without this changeset, we already have
> > good tests
> > and the right structure for the computation of obsfate data.
> > 
> > diff -r 098585d4fbc8 -r e18d8e61b726 mercurial/templatekw.py
> > --- a/mercurial/templatekw.py   Mon Jul 03 17:38:56 2017 +0200
> > +++ b/mercurial/templatekw.py   Mon Jul 03 14:32:52 2017 +0200
> > @@ -699,6 +699,32 @@
> >  
> >  return "; ".join(lines)
> >  
> > +def obsfatedefaulttempl(ui):
> > +""" Returns a dict with the default templates for obs fate
> > +"""
> > +# Prepare templates
> > +verbtempl = '{verb}'
> > +usertempl = '{if(users, " by {join(users, ", ")}")}'
> > +succtempl = '{if(successors, " as ")}{successors}' # Bypass if
> > limitation
> > +datetempleq = ' (at {min_date|isodate})'
> > +datetemplnoteq = ' (between {min_date|isodate} and
> > {max_date|isodate})'
> > +
> > +datetempl = '{if(max_date, "{ifeq(min_date, max_date, "%s",
> > "%s")}")}'
> > +datetempl = datetempl % (datetempleq, datetemplnoteq)
> > +
> > +optionalusertempl = usertempl
> > +username = _getusername(ui)
> > +if username is not None:
> > +optionalusertempl = ('{ifeq(join(users, "\0"), "%s", "",
> > "%s")}'
> > + % (username, usertempl))
> > +
> > +# Assemble them
> > +return {
> > +'obsfate_quiet': verbtempl + succtempl,
> > +'obsfate': verbtempl + optionalusertempl + succtempl,
> > +'obsfate_verbose': verbtempl + usertempl + succtempl +
> > datetempl,
> > +}
> 
> This makes me feel you're doing things in wrong layer. In principle,
> template
> keywords provide primitive data, which are pretty-printed by using
> user/stock
> templates (e.g. templates/map-cmdline.default.)

Definitely, I should have made it more explicit that this one patch is
in RFC state.

Obsfate has a difficult task: summarize the obsolescence history
(potentially spanning multiple obs-markers), aggregating the different
values and all of this in multiple dimensions when there's a
divergence.

This template seems quite complex, it felt complex during
implementation using templates. I tried finding an existing template
that was close to this complexity, successorssets was close but obsfate
adds one more layer of nesting, so I didn't find a good example to
mimic.

I'm pretty sure that implementing obsfate cleanly with the template
engine can be done, but after spending several days, I'm afraid I won't
be able to do it on my own. For example, I wasn't able to successfully
format the users list using templates, I tried doing this:

hg log -G -T '{obsfate % "obsolete: {get(obsfate, "verb")} by
{join(get(obsfate, "users"), ', ')}\n"}'

Lately, I was thinking about sending a V2 that, instead of returning
the formatted string, would returns an _hybrid object:

-return _obsfateprinter(values, repo, repo.ui)
+gen = _obsfateprinter(values, repo, repo.ui)
+return _hybrid(gen, values, None, None)

This way people would be able to start customizing it (with template
function "get") and we would be able to improve the implementation with
potential syntactic sugar addition in the template engine.

What do you think? Could you provide me with some direction to move
forward?

Cheers,
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 12 of 14] vfs: add the possibility to have a "ward" to check vfs usage

[Boris Feld]

On Wed, 2017-07-05 at 23:55 +0900, Yuya Nishihara wrote:
> On Sun, 02 Jul 2017 04:56:37 +0200, Pierre-Yves David wrote:
> > # HG changeset patch
> > # User Pierre-Yves David 
> > # Date 1470323266 -7200
> > #  Thu Aug 04 17:07:46 2016 +0200
> > # Node ID ebf81efdd6d7ff15c64683933635616c00475688
> > # Parent  34b8be7f0420b07d0f7b71379c6055e5b26223d5
> > # EXP-Topic vfs.ward
> > # Available At https://www.mercurial-scm.org/repo/users/marmoute/me
> > rcurial/
> > #  hg pull https://www.mercurial-scm.org/repo/users/mar
> > moute/mercurial/ -r ebf81efdd6d7
> > vfs: add the possibility to have a "ward" to check vfs usage
> > The feature has some similarity with the 'pathauditor', but further
> > digging
> > show it make more sense to keep them separated. The path auditor is
> > fairly
> > autonomous (create from vfs.__init__ without any extra
> > information), and check
> > very generic details. On the other hand, the wards will have deeper
> > knownledge
> > of the Mercurial logic and is created at the local repository
> > level. Mixing the
> > two would add much complexity for no real gains.
> 
> My gut feeling is that vfs isn't the right place if they require
> deeper
> knowledge of repository/dirstate logic. And the whole weakref
> business
> floating around wards, hooks and transaction seems like just getting
> around
> layering violation.

We pondered on this choice a long time and couldn't find any other
layer that is used by both Mercurial core and the extensions to access
the file system. It seems to be the highest abstraction layer we could
hook into without introducing a new one and updating all callers.

Our reflexion is, as the vfs classes comes from the scmutil file, it
seems okay to have scm related logic in that layer.

What do you think? Do you have another layer in mind where the callback
could be hooked to be able to catch any file-related operation?
> 
> > We currently only apply the ward on 'open' operation. We will
> > extend this to
> > other operations like copy, creation and removal later. The current
> > readonlyvfs
> > seems to have the same shortcoming.
> > 
> > diff --git a/mercurial/vfs.py b/mercurial/vfs.py
> > --- a/mercurial/vfs.py
> > +++ b/mercurial/vfs.py
> > @@ -284,7 +284,7 @@ class vfs(abstractvfs):
> >  This class is used to hide the details of COW semantics and
> >  remote file access from higher level code.
> >  '''
> > -def __init__(self, base, expandpath=False, realpath=False):
> > +def __init__(self, base, expandpath=False, realpath=False,
> > ward=None):
> >  if expandpath:
> >  base = util.expandpath(base)
> >  if realpath:
> > @@ -293,6 +293,11 @@ class vfs(abstractvfs):
> >  self.audit = pathutil.pathauditor(self.base)
> >  self.createmode = None
> >  self._trustnlink = None
> > +# optional function to validate operation on file
> > +# intended to be user for developer checks.
> > +#
> > +# XXX should be call for other things than 'open'
> > +self._ward = ward
> >  
> >  @util.propertycache
> >  def _cansymlink(self):
> > @@ -343,6 +348,9 @@ class vfs(abstractvfs):
> >  if not text and "b" not in mode:
> >  mode += "b" # for that other OS
> >  
> > +if self._ward is not None:
> > +self._ward(f, mode, atomictemp)
> 
> Do you have any idea how to expand this ward() callback to the other
> vfs
> operations, which have different set of parameters?

All other operations (move, delete, copy, etc) could be "translated" to
'read' or 'write' mode. An alternative would be to abstract the mode
with a couple of symbolic constant "create, update, append, delete, …".
What do you think?

Cheers,
Boris
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 9] template: better prune support in obsfate

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499096336 -7200
#  Mon Jul 03 17:38:56 2017 +0200
# Node ID 098585d4fbc88dc54513e12fa306d0e52ea2b323
# Parent  817d5a7df52dff4584ac3d9e5eef17230d2b1ae8
# EXP-Topic obsfatetemplate
template: better prune support in obsfate

successorssets don't returns good results for pruned commit, add a workaround
for simple cases.

A proper fix would require a large rework of successorssets algorithm, I will
send a separate series for this refactoring.

diff -r 817d5a7df52d -r 098585d4fbc8 mercurial/obsutil.py
--- a/mercurial/obsutil.py  Mon Jul 03 15:34:10 2017 +0200
+++ b/mercurial/obsutil.py  Mon Jul 03 17:38:56 2017 +0200
@@ -636,8 +636,32 @@
 
 ssets = successorssets(repo, ctx.node(), closest=True)
 
+# closestsuccessors returns an empty list for pruned revisions, remap it
+# into a list containing en empty list for future processing
+if ssets == []:
+ssets = [[]]
+
+# Try to recover pruned markers
+succsmap = repo.obsstore.successors
+fullsuccessorsets = [] # successor set + markers
+for sset in ssets:
+if sset:
+fullsuccessorsets.append(sset)
+else:
+# XXX we do not catch all prune markers (eg rewritten then pruned)
+# (fix me later)
+foundany = False
+for mark in succsmap.get(ctx.node(), ()):
+if not mark[1]:
+foundany = True
+sset = _succs()
+sset.markers.add(mark)
+fullsuccessorsets.append(sset)
+if not foundany:
+fullsuccessorsets.append(_succs())
+
 values = []
-for sset in ssets:
+for sset in fullsuccessorsets:
 raw = preparesuccessorset(sset, sset.markers)
 values.append(raw)
 
diff -r 817d5a7df52d -r 098585d4fbc8 tests/test-obsmarker-template.t
--- a/tests/test-obsmarker-template.t   Mon Jul 03 15:34:10 2017 +0200
+++ b/tests/test-obsmarker-template.t   Mon Jul 03 17:38:56 2017 +0200
@@ -158,7 +158,7 @@
   | @  a468dc9b3633
   |/ Obsfate: rewritten as 4:d004c8f274b9
   | x  f137d23bb3e1
-  | |
+  | |Obsfate: pruned
   | x  471f378eab4c
   |/ Obsfate: rewritten as 3:a468dc9b3633
   o  ea207398892e
@@ -207,7 +207,7 @@
   | x  a468dc9b3633
   |/ Obsfate: rewritten by test2 as 4:d004c8f274b9 (at 2001-04-19 04:25 
+)
   | x  f137d23bb3e1
-  | |
+  | |Obsfate: pruned by test1 (at 2009-02-13 23:31 +)
   | x  471f378eab4c
   |/ Obsfate: rewritten by test1 as 3:a468dc9b3633 (at 2009-02-13 23:31 
+)
   o  ea207398892e
@@ -1147,7 +1147,7 @@
   o  f897c6137566
   |
   | @  471f378eab4c
-  |/
+  |/ Obsfate: pruned
   o  ea207398892e
   
 
@@ -1541,11 +1541,11 @@
   
   $ hg fatelog
   @  471f378eab4c
-  |
+  |Obsfate: pruned
   o  ea207398892e
   
   $ hg fatelog -v
   @  471f378eab4c
-  |
+  |Obsfate: pruned by test (at 1970-01-01 00:00 +)
   o  ea207398892e
   
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 9] template: show user in obsfateprinter

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499088840 -7200
#  Mon Jul 03 15:34:00 2017 +0200
# Node ID ae148d33f0cff729874e141d9638960da19b0232
# Parent  955b9374e3bee67ac49bf2924f44be67a6528747
# EXP-Topic obsfatetemplate
template: show user in obsfateprinter

Extract, deduplicate users informations from obs markers in order to display
them.

Obsfate display them differently in verbose mode or not:
- In verbose mode, they are all displayed
- In other cases, only users different from default user are displayed. The
  logic here is that an user should remember what changes he made so in most
  cases that would be extraneous information.

diff -r 955b9374e3be -r ae148d33f0cf mercurial/obsutil.py
--- a/mercurial/obsutil.py  Mon Jul 03 15:33:27 2017 +0200
+++ b/mercurial/obsutil.py  Mon Jul 03 15:34:00 2017 +0200
@@ -564,8 +564,21 @@
 verb = 'split'
 return {'verb': verb}
 
+def _successorsetusers(successorset, markers):
+""" Returns a sorted list of markers users without duplicates
+"""
+if not markers:
+return {}
+
+# Check that user is present in meta
+markersmeta = [dict(m[3]) for m in markers]
+users = set(meta.get('user') for meta in markersmeta if meta.get('user'))
+
+return {'users': sorted(users)}
+
 FORMATSSETSFUNCTIONS = [
 _successorsetverb,
+_successorsetusers,
 ]
 
 def preparesuccessorset(successorset, rawmarkers):
diff -r 955b9374e3be -r ae148d33f0cf mercurial/templatekw.py
--- a/mercurial/templatekw.py   Mon Jul 03 15:33:27 2017 +0200
+++ b/mercurial/templatekw.py   Mon Jul 03 15:34:00 2017 +0200
@@ -634,6 +634,13 @@
 return _hybrid(gen(data), data, lambda x: {'successorset': x},
lambda d: d["successorset"])
 
+def _getusername(ui):
+"""the default username in the config or None"""
+try:
+return ui.username()
+except error.Abort: # no easy way to avoid ui raising Abort here :-/
+return None
+
 def _obsfatelineprinter(obsfateline, repo, ui):
 """Format and display obsfate related data
 """
@@ -647,6 +654,17 @@
 # Verb
 line.append(obsfateline['verb'])
 
+# Users
+if (verbose or normal) and 'users' in obsfateline:
+users = obsfateline['users']
+
+if normal:
+username = _getusername(ui)
+users = [user for user in users if user != username]
+
+if users:
+line.append(" by %s" % ", ".join(users))
+
 # Successors
 successors = obsfateline["successors"]
 
@@ -675,6 +693,7 @@
 
 This line will contains these information:
 - The list of closest successors in the log output
+- The users which have evolve the changeset
 """
 
 # Get the needed obsfate data
diff -r 955b9374e3be -r ae148d33f0cf tests/test-obsmarker-template.t
--- a/tests/test-obsmarker-template.t   Mon Jul 03 15:33:27 2017 +0200
+++ b/tests/test-obsmarker-template.t   Mon Jul 03 15:34:00 2017 +0200
@@ -95,14 +95,14 @@
   o  d004c8f274b9
   |
   | @  471f378eab4c
-  |/ Obsfate: rewritten as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test1, test2 as 4:d004c8f274b9
   o  ea207398892e
   
   $ hg fatelog -v
   o  d004c8f274b9
   |
   | @  471f378eab4c
-  |/ Obsfate: rewritten as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test1, test2 as 4:d004c8f274b9
   o  ea207398892e
   
   $ hg up 'desc(A1)' --hidden
@@ -125,7 +125,7 @@
   o  d004c8f274b9
   |
   | @  a468dc9b3633
-  |/ Obsfate: rewritten as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9
   o  ea207398892e
   
 Predecessors template should show all the predecessors as we force their 
display
@@ -205,14 +205,13 @@
   @  d004c8f274b9
   |
   | x  a468dc9b3633
-  |/ Obsfate: rewritten as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9
   | x  f137d23bb3e1
   | |
   | x  471f378eab4c
-  |/ Obsfate: rewritten as 3:a468dc9b3633
+  |/ Obsfate: rewritten by test1 as 3:a468dc9b3633
   o  ea207398892e
   
-
 Test templates with splitted commit
 ===
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 9] template: show dates in obsfateprinter

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499088850 -7200
#  Mon Jul 03 15:34:10 2017 +0200
# Node ID 817d5a7df52dff4584ac3d9e5eef17230d2b1ae8
# Parent  ae148d33f0cff729874e141d9638960da19b0232
# EXP-Topic obsfatetemplate
template: show dates in obsfateprinter

Extract the dates from obsmarkers. Display them only in verbose mode and group
them if we are grouping several obsmarkers.

The output will be this one in case of a single obsmarker:

(at 2001-04-19 04:25 +)

And this one in case of several obsmarkers:

(between 2001-04-19 04:25 + and 2009-02-13 23:31 +)

diff -r ae148d33f0cf -r 817d5a7df52d mercurial/obsutil.py
--- a/mercurial/obsutil.py  Mon Jul 03 15:34:00 2017 +0200
+++ b/mercurial/obsutil.py  Mon Jul 03 15:34:10 2017 +0200
@@ -576,9 +576,24 @@
 
 return {'users': sorted(users)}
 
+def _successorsetdates(successorset, markers):
+"""returns the max date and the min date of the markers list
+"""
+
+if not markers:
+return {}
+
+dates = [m[4] for m in markers]
+
+return {
+'min_date': min(dates),
+'max_date': max(dates)
+}
+
 FORMATSSETSFUNCTIONS = [
 _successorsetverb,
 _successorsetusers,
+_successorsetdates,
 ]
 
 def preparesuccessorset(successorset, rawmarkers):
diff -r ae148d33f0cf -r 817d5a7df52d mercurial/templatekw.py
--- a/mercurial/templatekw.py   Mon Jul 03 15:34:00 2017 +0200
+++ b/mercurial/templatekw.py   Mon Jul 03 15:34:10 2017 +0200
@@ -672,6 +672,19 @@
 fmtsuccessors = map(lambda s: _formatrevnode(repo[s]), successors)
 line.append(" as %s" % ", ".join(fmtsuccessors))
 
+# Date
+if verbose:
+min_date = obsfateline['min_date']
+max_date = obsfateline['max_date']
+
+if min_date == max_date:
+fmtmin_date = util.datestr(min_date, '%Y-%m-%d %H:%M %1%2')
+line.append(" (at %s)" % fmtmin_date)
+else:
+fmtmin_date = util.datestr(min_date, '%Y-%m-%d %H:%M %1%2')
+fmtmax_date = util.datestr(max_date, '%Y-%m-%d %H:%M %1%2')
+line.append(" (between %s and %s)" % (fmtmin_date, fmtmax_date))
+
 return "".join(line)
 
 def _obsfateprinter(obsfate, repo, ui, prefix=""):
@@ -694,6 +707,8 @@
 This line will contains these information:
 - The list of closest successors in the log output
 - The users which have evolve the changeset
+- The date or date range of the evolution betwwen the changeset and its
+  successors
 """
 
 # Get the needed obsfate data
diff -r ae148d33f0cf -r 817d5a7df52d tests/test-obsmarker-template.t
--- a/tests/test-obsmarker-template.t   Mon Jul 03 15:34:00 2017 +0200
+++ b/tests/test-obsmarker-template.t   Mon Jul 03 15:34:10 2017 +0200
@@ -102,7 +102,7 @@
   o  d004c8f274b9
   |
   | @  471f378eab4c
-  |/ Obsfate: rewritten by test1, test2 as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test1, test2 as 4:d004c8f274b9 (between 
2001-04-19 04:25 + and 2009-02-13 23:31 +)
   o  ea207398892e
   
   $ hg up 'desc(A1)' --hidden
@@ -125,7 +125,7 @@
   o  d004c8f274b9
   |
   | @  a468dc9b3633
-  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9 (at 2001-04-19 04:25 
+)
   o  ea207398892e
   
 Predecessors template should show all the predecessors as we force their 
display
@@ -205,11 +205,11 @@
   @  d004c8f274b9
   |
   | x  a468dc9b3633
-  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9
+  |/ Obsfate: rewritten by test2 as 4:d004c8f274b9 (at 2001-04-19 04:25 
+)
   | x  f137d23bb3e1
   | |
   | x  471f378eab4c
-  |/ Obsfate: rewritten by test1 as 3:a468dc9b3633
+  |/ Obsfate: rewritten by test1 as 3:a468dc9b3633 (at 2009-02-13 23:31 
+)
   o  ea207398892e
   
 Test templates with splitted commit
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 9 of 9] template: use template-engine for obsfate

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499085172 -7200
#  Mon Jul 03 14:32:52 2017 +0200
# Node ID e18d8e61b7260e246a82727c8cde01f936692cff
# Parent  098585d4fbc88dc54513e12fa306d0e52ea2b323
# EXP-Topic obsfatetemplate
template: use template-engine for obsfate

Try to replace the obsfateprinter part by "sub-templates".

The code is hacky, I've tried to use at maximum the template engine but the
raw data-structure doesn't seems well supported:

[{'markers': [{}, {}, ...], 'successors': [A, ...], 'verb': '', ...}, ...]

I've put this changeset at the end so the beginning of the serie might be
accepted without it. Even without this changeset, we already have good tests
and the right structure for the computation of obsfate data.

diff -r 098585d4fbc8 -r e18d8e61b726 mercurial/templatekw.py
--- a/mercurial/templatekw.py   Mon Jul 03 17:38:56 2017 +0200
+++ b/mercurial/templatekw.py   Mon Jul 03 14:32:52 2017 +0200
@@ -699,6 +699,32 @@
 
 return "; ".join(lines)
 
+def obsfatedefaulttempl(ui):
+""" Returns a dict with the default templates for obs fate
+"""
+# Prepare templates
+verbtempl = '{verb}'
+usertempl = '{if(users, " by {join(users, ", ")}")}'
+succtempl = '{if(successors, " as ")}{successors}' # Bypass if limitation
+datetempleq = ' (at {min_date|isodate})'
+datetemplnoteq = ' (between {min_date|isodate} and {max_date|isodate})'
+
+datetempl = '{if(max_date, "{ifeq(min_date, max_date, "%s", "%s")}")}'
+datetempl = datetempl % (datetempleq, datetemplnoteq)
+
+optionalusertempl = usertempl
+username = _getusername(ui)
+if username is not None:
+optionalusertempl = ('{ifeq(join(users, "\0"), "%s", "", "%s")}'
+ % (username, usertempl))
+
+# Assemble them
+return {
+'obsfate_quiet': verbtempl + succtempl,
+'obsfate': verbtempl + optionalusertempl + succtempl,
+'obsfate_verbose': verbtempl + usertempl + succtempl + datetempl,
+}
+
 @templatekeyword("obsfate")
 def showobsfate(repo, ctx, **args):
 """Returns a string describing how an obsolete changeset has evolved in a
@@ -717,7 +743,38 @@
 if values is None:
 return ''
 
-return _obsfateprinter(values, repo, repo.ui)
+# Format each successorset successors list
+for raw in values:
+# As we can't do something like
+# "{join(map(nodeshort, successors), ', '}" in template, manually
+# create a correct textual representation
+gen = ', '.join(_formatrevnode(repo[n]) for n in raw['successors'])
+
+makemap = lambda x: {'successor': x}
+joinfmt = lambda d: "%s" % d['successor']
+raw['successors'] = _hybrid(gen, raw['successors'], makemap,
+joinfmt)
+
+# And then format them
+# Insert default obsfate templates
+args['templ'].cache.update(obsfatedefaulttempl(repo.ui))
+
+if repo.ui.quiet:
+name = "obsfate_quiet"
+elif repo.ui.verbose:
+name = "obsfate_verbose"
+elif repo.ui.debugflag:
+name = "obsfate_debug"
+else:
+name = "obsfate"
+
+# Format a single value using template
+def fmt(d):
+nargs = args.copy()
+nargs.update(d[name])
+return args['templ'](name, **nargs)
+
+return _hybrid(None, values, lambda x: {name: x}, fmt)
 
 @templatekeyword('p1rev')
 def showp1rev(repo, ctx, templ, **args):
diff -r 098585d4fbc8 -r e18d8e61b726 tests/test-obsmarker-template.t
--- a/tests/test-obsmarker-template.t   Mon Jul 03 17:38:56 2017 +0200
+++ b/tests/test-obsmarker-template.t   Mon Jul 03 14:32:52 2017 +0200
@@ -20,7 +20,8 @@
   > {if(successorssets, "\n  Successors: {successorssets}")}\
   > {if(successorssets, "\n  multi-line: {join(successorssets, "\n  
multi-line: ")}")}\
   > {if(successorssets, "\n  json: {successorssets|json}")}\n'
-  > fatelog = log -G -T '{node|short}\n{if(obsfate, "  Obsfate: {obsfate}\n")}'
+  > fatelog = log -G -T '{node|short}\n{if(obsfate, "  Obsfate: {join(obsfate, 
"; ")}\n")}'
+  > fatelogjson = log -G -T '{node|short} {obsfate|json}\n'
   > EOF
 
 Test templates on amended commit
@@ -212,6 +213,19 @@
   |/ Obsfate: rewritten by test1 as 3:a468dc9b3633 (at 2009-02-13 23:31 
+)
   o  ea207398892e
   
+
+  $ hg fatelogjson --hidden
+  @  d004c8f274b9 ""
+  |
+  | x  a468dc9b3633 [{"markers": [["a468dc9b36338b14fdb7825f55ce3df4e71517ad", 
["d004c8f274b9ec480a47a93c10dac5eee63adb78"], 0, [["user", "test2"]], 
[987654321.0, 0], null]], "max_date&

[PATCH 2 of 9] obsolete: add an explicit '_succs.copy()' method

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499044397 -7200
#  Mon Jul 03 03:13:17 2017 +0200
# Node ID f1e0955f8dda31eef2a65380ea97c35cb637810e
# Parent  105c150648b0c43c240b27047d0be89359932d10
# EXP-Topic obsfatetemplate
obsolete: add an explicit '_succs.copy()' method

Mimic the standard API for copying in the _succs class, it makes the code
slightly cleaner and will be needed later for copying markers at the same time
than copying the list content.

diff -r 105c150648b0 -r f1e0955f8dda mercurial/obsutil.py
--- a/mercurial/obsutil.py  Mon Jul 03 00:53:55 2017 +0200
+++ b/mercurial/obsutil.py  Mon Jul 03 03:13:17 2017 +0200
@@ -314,6 +314,9 @@
 class _succs(list):
 """small class to represent a successors with some metadata about it"""
 
+def copy(self):
+return _succs(self)
+
 def successorssets(repo, initialnode, closest=False, cache=None):
 """Return set of all latest successors of initial nodes
 
@@ -514,7 +517,7 @@
 productresult = []
 for prefix in markss:
 for suffix in cache[suc]:
-newss = _succs(prefix)
+newss = prefix.copy()
 for part in suffix:
 # do not duplicated entry in successors set
 # first entry wins.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 9] obsolete: introduce a _succs class

[Boris Feld]

# HG changeset patch
# User Boris Feld <boris.f...@octobus.net>
# Date 1499036035 -7200
#  Mon Jul 03 00:53:55 2017 +0200
# Node ID 105c150648b0c43c240b27047d0be89359932d10
# Parent  634b259079c55e5dc3d38c3c73f87611d0be
# EXP-Topic obsfatetemplate
obsolete: introduce a _succs class

It will be useful later when we will be adding markers to _succs in order to
represent a successorset with the list of markers from the root to each
successors sets. This information will be needed for the obsfate template I will
introduce.

Makes it a subclass of list so all callers will continue to work.

diff -r 634b259079c5 -r 105c150648b0 mercurial/obsutil.py
--- a/mercurial/obsutil.py  Tue Jul 04 22:35:52 2017 -0700
+++ b/mercurial/obsutil.py  Mon Jul 03 00:53:55 2017 +0200
@@ -311,6 +311,9 @@
 obsoleted.add(rev)
 return obsoleted
 
+class _succs(list):
+"""small class to represent a successors with some metadata about it"""
+
 def successorssets(repo, initialnode, closest=False, cache=None):
 """Return set of all latest successors of initial nodes
 
@@ -429,11 +432,11 @@
 # case (2): end of walk.
 if current in repo:
 # We have a valid successors.
-cache[current] = [(current,)]
+cache[current] = [_succs((current,))]
 else:
 # Final obsolete version is unknown locally.
 # Do not count that as a valid successors
-cache[current] = []
+cache[current] = _succs()
 else:
 # cases (3) and (4)
 #
@@ -471,7 +474,7 @@
 if suc not in cache:
 if suc in stackedset:
 # cycle breaking
-cache[suc] = []
+cache[suc] = _succs()
 else:
 # case (3) If we have not computed successors sets
 # of one of those successors we add it to the
@@ -505,13 +508,13 @@
 succssets = []
 for mark in sorted(succmarkers[current]):
 # successors sets contributed by this marker
-markss = [[]]
+markss = [_succs()]
 for suc in mark[1]:
 # cardinal product with previous successors
 productresult = []
 for prefix in markss:
 for suffix in cache[suc]:
-newss = list(prefix)
+newss = _succs(prefix)
 for part in suffix:
 # do not duplicated entry in successors set
 # first entry wins.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel