Re: [PATCH 2 of 3] upgrade: add '-' in optimisation name

[Yuya Nishihara]

On Sun, 02 Dec 2018 16:56:45 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1531443930 -7200
> #  Fri Jul 13 03:05:30 2018 +0200
> # Node ID 67ee61288306b5f74c2151a3e71a5321bc5989cb
> # Parent  8ca558d1dc064bef67b6fb4341feecddceb216ce
> # EXP-Topic upgrade-test
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 67ee61288306
> upgrade: add '-' in optimisation name

>  def upgraderepo(ui, repo, run=False, optimize=None):
>  """Upgrade a repository in place."""
> -optimize = set(optimize or [])
> +if optimize is None:
> +optimize = []
> +optimize = set(legacy_opts_map.get(o, o) for o in optimize)
>  repo = repo.unfiltered()
>  
>  # Ensure the repository can be upgraded.
> @@ -777,7 +792,7 @@ def upgraderepo(ui, repo, run=False, opt
>  # Apply and Validate arguments.
>  optimizations = []
>  for o in alloptimizations:
> -if o.name in optimize:
> +if o.name in optimize or o.name.replace('-', '') in optimize:

Perhaps this would be handled by the legacy_opts_map.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3] upgrade: test that fncache is valid after repository upgrade

[Yuya Nishihara]

On Sun, 02 Dec 2018 16:56:44 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1531443292 -7200
> #  Fri Jul 13 02:54:52 2018 +0200
> # Node ID 8ca558d1dc064bef67b6fb4341feecddceb216ce
> # Parent  7e6834ade51d98da575cfbeef651fc8a465c38b8
> # EXP-Topic upgrade-test
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 8ca558d1dc06
> upgrade: test that fncache is valid after repository upgrade

Queued 1 and 3, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D4312: New bookflow extension for bookmark-based branching

[smf (Sean Farley)]

smf added a comment.


  In https://phab.mercurial-scm.org/D4312#79581, @durin42 wrote:
  
  > In https://phab.mercurial-scm.org/D4312#79573, @smf wrote:
  >
  > > In https://phab.mercurial-scm.org/D4312#79509, @durin42 wrote:
  > >
  > > > There's been some good discussion on this. I'm sympathetic to both 
arguments here, namely: "we could improve bookmarks and make them less bad" and 
"bookmarks are a dead end and nobody should use them and we shouldn't improve 
them" (or thereabouts - I'm summarizing complicated positions to less than a 
sentence, so bear with me.) I think not following up on our plans to at least 
make plausible incremental improvements to bookmarks serves our users poorly, 
and this extension merits landing as an experimental extension. We can always 
spin it back out if we get unhappy with it.
  > >
  > >
  > > Obviously, I can't say I'm too happy with this. Allowing users to shoot 
themselves in the foot even more is pretty bad.
  >
  >
  > That ship has sailed: bookmarks exist, and are more visible than features 
like rebase.
  >
  > > 
  > > 
  > >> I'm going to land this patch largely as-is on default, with the 
following tweaks: I'm adding (EXPERIMENTAL) to the docstring of the extension 
so we can iterate its behavior, some misc. check-code fixes, tweak the commit 
message slightly to make check-commit happy.
  > > 
  > > When I bring up community issues (including at this sprint) I thought 
this project was more than just one person. Having to constantly put up the 
good fight is completely negated if no one is going to listen and just ship 
this anyways. Bookmark related features belong as a third-party extension just 
like hgsubversion, hg-git, etc.
  >
  > ...it is? You're the only voice of objection that I'm seeing. I see some 
mostly indifference from BitBucket and some pretty good enthusiasm from the 
contributor, rhodecode, Kevin, etc. And it's experimental, so we can dump it if 
I've made the wrong choice. If I've missed some other /objections/ (as 
contrasted with what I perceive to be indifference - maybe I'm misreading 
Erik?) please point out my error.
  
  
  I hung out with Erik yesterday and he said he was too frustrated and tired of 
explaining himself to reply to this thread. I can't say I really blame him. The 
previous discussion outlines why having (and encouraging) two branching models 
is bad for the ecosystem. This is based on Erik and mine’s years of experience 
working on Bitbucket.
  
  The more important and pressing issues are exchanging obs markers and 
improving named branching. A cash donation in those directions has gone unused 
and it is frustrating that the weight of Bitbucket’s experience and resources 
has gone ignored.
  
  This will not help the *average* user and sends a mixed (and dangerous) 
message that bookmarks should be used. For a team that truly wants this 
feature, hosting this extension as a package on pypi is the best solution.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4312

To: idlsoft, #hg-reviewers, pulkit, marcink
Cc: evzijst, krbullock, mharbison72, smf, markand, marcink, durin42, jwatt, 
pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5370: rust: core implementation of missingancestors (no bindings)

[yuja (Yuya Nishihara)]

yuja added a comment.


  >   @yuja: do you mean one of those Differential Revisions of this system for 
each commit, sure I can do.
  
  I don't know how Phabricator is working, but probably yes. I meant we want
  https://phab.mercurial-scm.org/D5370, https://phab.mercurial-scm.org/D5371, 
and https://phab.mercurial-scm.org/D5372 instead of a folded 
https://phab.mercurial-scm.org/D5370. I think phabsend will
  do that way.
  
  >   With respect to rust-cpython bindings, I'm currently waiting for feedback 
on https://github.com/dgrunwald/rust-cpython/issues/164
  >   Perhaps you'd have an idea ?
  
  No, but it looks interesting. I'll take a look this weekend.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5370

To: gracinet, #hg-reviewers
Cc: yuja, durin42, kevincox, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D5372: perfrevlogwrite: use progress helper

[Yuya Nishihara]

Queued the patches but for this, thanks. perf.py has to support old Mercurial
versions.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5374: fastannotate: use progress helper

[yuja (Yuya Nishihara)]

yuja added a comment.


  >   if ismainbranch: # need to write to linelog
  > 
  > - if not self.ui.quiet:
  > - progress += 1
  > - self.ui.progress(_('building cache'), progress,
  > - total=len(newmainbranch)) +progress.increment()
  
  This one is BC, but I think that's okay.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5374

To: martinvonz, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5380: remotefilelog: use progress helper in shallowrepo

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb6a6dc1a14bd: remotefilelog: use progress helper in 
shallowrepo (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5380?vs=12719=12741

REVISION DETAIL
  https://phab.mercurial-scm.org/D5380

AFFECTED FILES
  hgext/remotefilelog/shallowrepo.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowrepo.py 
b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -40,8 +40,6 @@
 from mercurial.utils import procutil
 _hgexecutable = procutil.hgexecutable
 
-_prefetching = _('prefetching')
-
 # These make*stores functions are global so that other extensions can replace
 # them.
 def makelocalstores(repo):
@@ -233,9 +231,9 @@
 serverfiles = skip.copy()
 visited = set()
 visited.add(nullrev)
-revnum = 0
 revcount = len(revs)
-self.ui.progress(_prefetching, revnum, total=revcount)
+progress = self.ui.makeprogress(_('prefetching'), total=revcount)
+progress.update(0)
 for rev in sorted(revs):
 ctx = repo[rev]
 if pats:
@@ -264,12 +262,11 @@
 files.update(diff)
 
 visited.add(mfrev)
-revnum += 1
-self.ui.progress(_prefetching, revnum, total=revcount)
+progress.increment()
 
 files.difference_update(skip)
 serverfiles.difference_update(skip)
-self.ui.progress(_prefetching, None)
+progress.complete()
 
 # Fetch files known to be on the server
 if serverfiles:



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5382: remotefilelog: avoid temporarily using "count" variable as synonym for "total"

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb34b1b865ef9: remotefilelog: avoid temporarily using 
count variable as synonym for total (authored by 
martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5382?vs=12721=12743

REVISION DETAIL
  https://phab.mercurial-scm.org/D5382

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -317,8 +317,8 @@
 writedata = self.writedata
 
 repo = self.repo
-count = len(fileids)
-request = "get\n%d\n" % count
+total = len(fileids)
+request = "get\n%d\n" % total
 idmap = {}
 reponame = repo.name
 for file, id in fileids:
@@ -330,8 +330,7 @@
 
 cache.request(request)
 
-total = count
-progress = self.ui.makeprogress(_('downloading'), total=count)
+progress = self.ui.makeprogress(_('downloading'), total=total)
 progress.update(0)
 
 missed = []



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5383: remotefilelog: remove an unnecessary update of "count" container

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG70de33b9d328: remotefilelog: remove an unnecessary update 
of count container (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5383?vs=12722=12744

REVISION DETAIL
  https://phab.mercurial-scm.org/D5383

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -422,8 +422,7 @@
 finally:
 self.ui.verbose = verbose
 # send to memcache
-count[0] = len(missed)
-request = "set\n%d\n%s\n" % (count[0], "\n".join(missed))
+request = "set\n%d\n%s\n" % (len(missed), "\n".join(missed))
 cache.request(request)
 
 progress.complete()



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5391: cleanupnodes: trust caller when "moves" is not None

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb7823bd59b07: cleanupnodes: trust caller when 
moves is not None (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5391?vs=12731=12751

REVISION DETAIL
  https://phab.mercurial-scm.org/D5391

AFFECTED FILES
  mercurial/scmutil.py

CHANGE DETAILS

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -901,32 +901,33 @@
 repls[key] = value
 replacements = repls
 
+# Unfiltered repo is needed since nodes in replacements might be hidden.
+unfi = repo.unfiltered()
+
 # Calculate bookmark movements
 if moves is None:
 moves = {}
-# Unfiltered repo is needed since nodes in replacements might be hidden.
-unfi = repo.unfiltered()
-for oldnodes, newnodes in replacements.items():
-for oldnode in oldnodes:
-if oldnode in moves:
-continue
-if len(newnodes) > 1:
-# usually a split, take the one with biggest rev number
-newnode = next(unfi.set('max(%ln)', newnodes)).node()
-elif len(newnodes) == 0:
-# move bookmark backwards
-allreplaced = []
-for rep in replacements:
-allreplaced.extend(rep)
-roots = list(unfi.set('max((::%n) - %ln)', oldnode,
-  allreplaced))
-if roots:
-newnode = roots[0].node()
+for oldnodes, newnodes in replacements.items():
+for oldnode in oldnodes:
+if oldnode in moves:
+continue
+if len(newnodes) > 1:
+# usually a split, take the one with biggest rev number
+newnode = next(unfi.set('max(%ln)', newnodes)).node()
+elif len(newnodes) == 0:
+# move bookmark backwards
+allreplaced = []
+for rep in replacements:
+allreplaced.extend(rep)
+roots = list(unfi.set('max((::%n) - %ln)', oldnode,
+  allreplaced))
+if roots:
+newnode = roots[0].node()
+else:
+newnode = nullid
 else:
-newnode = nullid
-else:
-newnode = newnodes[0]
-moves[oldnode] = newnode
+newnode = newnodes[0]
+moves[oldnode] = newnode
 
 allnewnodes = [n for ns in replacements.values() for n in ns]
 toretract = {}



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5393: rebase: remove now-unnecessary arguments to _abort()

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2041991f1ce2: rebase: remove now-unnecessary arguments to 
_abort() (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5393?vs=12733=12753

REVISION DETAIL
  https://phab.mercurial-scm.org/D5393

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -347,9 +347,7 @@
 
 if isabort:
 backup = backup and self.backupf
-return self._abort(self.repo, self.originalwd, self.destmap,
-   self.state, activebookmark=self.activebookmark,
-   backup=backup, suppwarns=suppwarns)
+return self._abort(backup=backup, suppwarns=suppwarns)
 
 def _preparenewrebase(self, destmap):
 if not destmap:
@@ -653,20 +651,17 @@
 repo['.'].node() == repo._bookmarks[self.activebookmark]):
 bookmarks.activate(repo, self.activebookmark)
 
-def _abort(self, repo, originalwd, destmap, state, activebookmark=None,
-backup=True, suppwarns=False):
-'''Restore the repository to its original state.  Additional args:
+def _abort(self, backup=True, suppwarns=False):
+'''Restore the repository to its original state.'''
 
-activebookmark: the name of the bookmark that should be active after 
the
-restore'''
-
+repo = self.repo
 try:
 # If the first commits in the rebased set get skipped during the
 # rebase, their values within the state mapping will be the dest
 # rev id. The rebased list must must not contain the dest rev
 # (issue4896)
-rebased = [s for r, s in state.items()
-   if s >= 0 and s != r and s != destmap[r]]
+rebased = [s for r, s in self.state.items()
+   if s >= 0 and s != r and s != self.destmap[r]]
 immutable = [d for d in rebased if not repo[d].mutable()]
 cleanup = True
 if immutable:
@@ -690,21 +685,21 @@
 c.node() for c in repo.set('roots(%ld)', rebased)]
 
 updateifonnodes = set(rebased)
-updateifonnodes.update(destmap.values())
-updateifonnodes.add(originalwd)
+updateifonnodes.update(self.destmap.values())
+updateifonnodes.add(self.originalwd)
 shouldupdate = repo['.'].rev() in updateifonnodes
 
 # Update away from the rebase if necessary
-if shouldupdate or needupdate(repo, state):
-mergemod.update(repo, originalwd, branchmerge=False,
+if shouldupdate or needupdate(repo, self.state):
+mergemod.update(repo, self.originalwd, branchmerge=False,
 force=True)
 
 # Strip from the first rebased revision
 if rebased:
 repair.strip(repo.ui, repo, strippoints, backup=backup)
 
-if activebookmark and activebookmark in repo._bookmarks:
-bookmarks.activate(repo, activebookmark)
+if self.activebookmark and self.activebookmark in repo._bookmarks:
+bookmarks.activate(repo, self.activebookmark)
 
 finally:
 clearstatus(repo)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8] commandserver: remove redundant "if True" block

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540990987 -32400
#  Wed Oct 31 22:03:07 2018 +0900
# Node ID 92023422fbe3cf05e6d25b8213bd1e5a749c8e98
# Parent  c9533007e0e09fe8129b46208dbb760ff8799a88
commandserver: remove redundant "if True" block

diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -570,35 +570,34 @@ class unixforkingservice(object):
 
 def _acceptnewconnection(self, sock, selector):
 h = self._servicehandler
-if True:
+try:
+conn, _addr = sock.accept()
+except socket.error as inst:
+if inst.args[0] == errno.EINTR:
+return
+raise
+
+pid = os.fork()
+if pid:
 try:
-conn, _addr = sock.accept()
-except socket.error as inst:
-if inst.args[0] == errno.EINTR:
-return
-raise
-
-pid = os.fork()
-if pid:
+self.ui.log(b'cmdserver', b'forked worker process (pid=%d)\n',
+pid)
+self._workerpids.add(pid)
+h.newconnection()
+finally:
+conn.close()  # release handle in parent process
+else:
+try:
+selector.close()
+sock.close()
+self._runworker(conn)
+conn.close()
+os._exit(0)
+except:  # never return, hence no re-raises
 try:
-self.ui.log(b'cmdserver',
-b'forked worker process (pid=%d)\n', pid)
-self._workerpids.add(pid)
-h.newconnection()
+self.ui.traceback(force=True)
 finally:
-conn.close()  # release handle in parent process
-else:
-try:
-selector.close()
-sock.close()
-self._runworker(conn)
-conn.close()
-os._exit(0)
-except:  # never return, hence no re-raises
-try:
-self.ui.traceback(force=True)
-finally:
-os._exit(255)
+os._exit(255)
 
 def _sigchldhandler(self, signal, frame):
 self._reapworkers(os.WNOHANG)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8] util: implement pop() on lrucachedict

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1541318225 -32400
#  Sun Nov 04 16:57:05 2018 +0900
# Node ID d5e7ded8150f66f89b5b6ed0a0c49ea38018982e
# Parent  2d23efc7274a923baffe9d0dc46709d651c468b4
util: implement pop() on lrucachedict

This moves __delitem__() to pop() as the requirement is pretty much the same,
and reimplement __delitem__() by using pop().

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -1320,7 +1320,16 @@ class lrucachedict(object):
 self.insert(k, v)
 
 def __delitem__(self, k):
-node = self._cache.pop(k)
+self.pop(k)
+
+def pop(self, k, default=_notset):
+try:
+node = self._cache.pop(k)
+except KeyError:
+if default is _notset:
+raise
+return default
+value = node.value
 self.totalcost -= node.cost
 node.markempty()
 
@@ -1329,6 +1338,8 @@ class lrucachedict(object):
 self._movetohead(node)
 self._head = node.next
 
+return value
+
 # Additional dict methods.
 
 def get(self, k, default=None):
diff --git a/tests/test-lrucachedict.py b/tests/test-lrucachedict.py
--- a/tests/test-lrucachedict.py
+++ b/tests/test-lrucachedict.py
@@ -94,6 +94,21 @@ class testlrucachedict(unittest.TestCase
 self.assertEqual(d.peek('a'), 'va')
 self.assertEqual(list(d), ['c', 'b', 'a'])
 
+def testpop(self):
+d = util.lrucachedict(4)
+d['a'] = 'va'
+d['b'] = 'vb'
+d['c'] = 'vc'
+
+with self.assertRaises(KeyError):
+d.pop('missing')
+self.assertEqual(list(d), ['c', 'b', 'a'])
+self.assertIsNone(d.pop('missing', None))
+self.assertEqual(list(d), ['c', 'b', 'a'])
+
+self.assertEqual(d.pop('b'), 'vb')
+self.assertEqual(list(d), ['c', 'a'])
+
 def testcopypartial(self):
 d = util.lrucachedict(4)
 d.insert('a', 'va', cost=4)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 8] util: add method to peek item in lrucachedict

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540992545 -32400
#  Wed Oct 31 22:29:05 2018 +0900
# Node ID 2d23efc7274a923baffe9d0dc46709d651c468b4
# Parent  5f4354d1aa370a028eee8bcc6de245b1212cf35c
util: add method to peek item in lrucachedict

I want a function that doesn't unnecessarily update the internal state of
the cache dict after fork().

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -1337,6 +1337,20 @@ class lrucachedict(object):
 except KeyError:
 return default
 
+def peek(self, k, default=_notset):
+"""Get the specified item without moving it to the head
+
+Unlike get(), this doesn't mutate the internal state. But be aware
+that it doesn't mean peek() is thread safe.
+"""
+try:
+node = self._cache[k]
+return node.value
+except KeyError:
+if default is _notset:
+raise
+return default
+
 def clear(self):
 n = self._head
 while n.key is not _notset:
diff --git a/tests/test-lrucachedict.py b/tests/test-lrucachedict.py
--- a/tests/test-lrucachedict.py
+++ b/tests/test-lrucachedict.py
@@ -79,6 +79,21 @@ class testlrucachedict(unittest.TestCase
 self.assertEqual(d.get('a'), 'va')
 self.assertEqual(list(d), ['a', 'c', 'b'])
 
+def testpeek(self):
+d = util.lrucachedict(4)
+d['a'] = 'va'
+d['b'] = 'vb'
+d['c'] = 'vc'
+
+with self.assertRaises(KeyError):
+d.peek('missing')
+self.assertEqual(list(d), ['c', 'b', 'a'])
+self.assertIsNone(d.peek('missing', None))
+self.assertEqual(list(d), ['c', 'b', 'a'])
+
+self.assertEqual(d.peek('a'), 'va')
+self.assertEqual(list(d), ['c', 'b', 'a'])
+
 def testcopypartial(self):
 d = util.lrucachedict(4)
 d.insert('a', 'va', cost=4)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5370: rust: core implementation of missingancestors (no bindings)

[yuja (Yuya Nishihara)]

yuja added a comment.


  Quickly scanned, and looks generally good to me.
  
  >   rust: iterator version of Graph.parents
  >   rust: translation of missingancestors
  >   rust: translated random test of missingancestors
  
  Can you send these as separate patches?
  
  >   An alternative would have been to expose to Python
  >   MissingAncestors but that would have meant
  >   
  >   - pollution of the release build used from Python,  whereas we do it in 
this changeset within the tests submodule
  >   - waiting on rust-cpython bindings to be ready or doing the cumbersome 
direct-ffi (more pollution with unsafe C code)
  
  Still I want some CPython interface to measure the perf win. Are there
  lots of work remaining to bring rust-cpython to us?
  
  >   although one could argue that actually parents() should return an
  >   array instead of a tuple, giving us a similar iterator for free (but on
  >   references instead of values, unless we also use the arrayvec crate
  >   could help). Notably, the current C-backed parents() internally uses an
  >   array for communication with C code, so that currently, we'd get less 
memory
  >   copy and less code using an array.
  
  I prefer changing parents() to return `[Revision; 2]`. Then, we can write a
  simple utility function that drops NULL_REVISION, `&[Revision; 2] -> 
&[Revision]`.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5370

To: gracinet, #hg-reviewers
Cc: yuja, durin42, kevincox, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D4312: New bookflow extension for bookmark-based branching

[idlsoft (Sandu Turcan)]

idlsoft added a comment.


  In https://phab.mercurial-scm.org/D4312#79938, @smf wrote:
  
  > This will not help the *average* user and sends a mixed (and dangerous) 
message that bookmarks should be used.
  
  
  Bookmarks have been in core for some time now, and there is not one mention 
anywhere that they are not to be used.
  They may not be trivial to use but it's certainly not officially discouraged 
anywhere.
  Bookmarks is the first thing that comes up when you search for "mercurial 
light branching", although it's from a blog post, not the official wiki.
  Meanwhile the official wiki plainly states 
 that light branching is 
a different abstraction, and branches should not be used for for that.
  
  So what IS the recommended way then?
  Is there one?
  
  I may have a horse in this race but I'd be more than happy to see it lose to 
a better one.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4312

To: idlsoft, #hg-reviewers, pulkit, marcink
Cc: evzijst, krbullock, mharbison72, smf, markand, marcink, durin42, jwatt, 
pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5373: debugbuildannotatecache: use progress helper

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG901f6ef670ca: debugbuildannotatecache: use progress helper 
(authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5373?vs=12712=12734

REVISION DETAIL
  https://phab.mercurial-scm.org/D5373

AFFECTED FILES
  hgext/fastannotate/commands.py

CHANGE DETAILS

diff --git a/hgext/fastannotate/commands.py b/hgext/fastannotate/commands.py
--- a/hgext/fastannotate/commands.py
+++ b/hgext/fastannotate/commands.py
@@ -261,8 +261,9 @@
 repo.prefetchfastannotate(paths)
 else:
 # server, or full repo
+progress = ui.makeprogress(_('building'), total=len(paths))
 for i, path in enumerate(paths):
-ui.progress(_('building'), i, total=len(paths))
+progress.update(i)
 with facontext.annotatecontext(repo, path) as actx:
 try:
 if actx.isuptodate(rev):
@@ -281,5 +282,4 @@
 # cache for other files.
 ui.warn(_('fastannotate: %s: failed to '
   'build cache: %r\n') % (path, ex))
-# clear the progress bar
-ui.write()
+progress.complete()



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5374: fastannotate: use progress helper

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGdd1006874a5a: fastannotate: use progress helper (authored 
by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5374?vs=12713=12735

REVISION DETAIL
  https://phab.mercurial-scm.org/D5374

AFFECTED FILES
  hgext/fastannotate/context.py

CHANGE DETAILS

diff --git a/hgext/fastannotate/context.py b/hgext/fastannotate/context.py
--- a/hgext/fastannotate/context.py
+++ b/hgext/fastannotate/context.py
@@ -398,7 +398,8 @@
 
 # 3rd DFS does the actual annotate
 visit = initvisit[:]
-progress = 0
+progress = self.ui.makeprogress(('building cache'),
+total=len(newmainbranch))
 while visit:
 f = visit[-1]
 if f in hist:
@@ -437,10 +438,7 @@
 del pcache[f]
 
 if ismainbranch: # need to write to linelog
-if not self.ui.quiet:
-progress += 1
-self.ui.progress(_('building cache'), progress,
- total=len(newmainbranch))
+progress.increment()
 bannotated = None
 if len(pl) == 2 and self.opts.followmerge: # merge
 bannotated = curr[0]
@@ -450,8 +448,7 @@
 elif showpath: # not append linelog, but we need to record path
 self._node2path[f.node()] = f.path()
 
-if progress: # clean progress bar
-self.ui.write()
+progress.complete()
 
 result = [
 ((self.revmap.rev2hsh(fr) if isinstance(fr, int) else fr.node()), 
l)



To: martinvonz, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5377: remotefilelog: use progress helper in remotefilelogserver

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGfbd053af2eda: remotefilelog: use progress helper in 
remotefilelogserver (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5377?vs=12716=12738

REVISION DETAIL
  https://phab.mercurial-scm.org/D5377

AFFECTED FILES
  hgext/remotefilelog/remotefilelogserver.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/remotefilelogserver.py 
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -388,19 +388,17 @@
 days = repo.ui.configint("remotefilelog", "serverexpiration")
 expiration = time.time() - (days * 24 * 60 * 60)
 
-_removing = _("removing old server cache")
-count = 0
-ui.progress(_removing, count, unit="files")
+progress = ui.makeprogress(_("removing old server cache"), unit="files")
+progress.update(0)
 for root, dirs, files in os.walk(cachepath):
 for file in files:
 filepath = os.path.join(root, file)
-count += 1
-ui.progress(_removing, count, unit="files")
+progress.increment()
 if filepath in neededfiles:
 continue
 
 stat = os.stat(filepath)
 if stat.st_mtime < expiration:
 os.remove(filepath)
 
-ui.progress(_removing, None)
+progress.complete()



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5379: remotefilelog: use progress helper in shallowbundle

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG4e08bbfc6d51: remotefilelog: use progress helper in 
shallowbundle (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5379?vs=12718=12739

REVISION DETAIL
  https://phab.mercurial-scm.org/D5379

AFFECTED FILES
  hgext/remotefilelog/shallowbundle.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowbundle.py 
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -171,7 +171,6 @@
 if not shallowutil.isenabled(repo):
 return orig(repo, source, revmap, trp, expectedfiles, *args)
 
-files = 0
 newfiles = 0
 visited = set()
 revisiondatas = {}
@@ -184,14 +183,14 @@
 # files in topological order.
 
 # read all the file chunks but don't add them
+progress = repo.ui.makeprogress(_('files'), total=expectedfiles)
 while True:
 chunkdata = source.filelogheader()
 if not chunkdata:
 break
-files += 1
 f = chunkdata["filename"]
 repo.ui.debug("adding %s revisions\n" % f)
-repo.ui.progress(_('files'), files, total=expectedfiles)
+progress.increment()
 
 if not repo.shallowmatch(f):
 fl = repo.file(f)
@@ -289,6 +288,6 @@
 processed.add((f, node))
 skipcount = 0
 
-repo.ui.progress(_('files'), None)
+progress.complete()
 
 return len(revisiondatas), newfiles



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5384: remotefilelog: replace a "a=[expr]; b=a[0]" by "b=expr; a = [b]"

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGfe0b65ff00e7: remotefilelog: replace a a=[expr]; 
b=a[0] by b=expr; a = [b] (authored by martinvonz, committed 
by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5384?vs=12723=12745

REVISION DETAIL
  https://phab.mercurial-scm.org/D5384

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -359,8 +359,8 @@
 global fetchmisses
 fetchmisses += len(missed)
 
-count = [total - len(missed)]
-fromcache = count[0]
+fromcache = total - len(missed)
+count = [fromcache]
 progress.update(count[0], total=total)
 self.ui.log("remotefilelog", "remote cache hit rate is %r of %r\n",
 count[0], total, hit=count[0], total=total)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 8] commandserver: pass around option to hook repo instance creation

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540990631 -32400
#  Wed Oct 31 21:57:11 2018 +0900
# Node ID 958a93f4f9210c4b5669e4424f8d73b048ef72ed
# Parent  53bdf0ab321c90189c61beb021d8eac54cf40918
commandserver: pass around option to hook repo instance creation

This is necessary to wrap a repo instance so the master process will be
notified on repo.close().

diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py
--- a/mercurial/chgserver.py
+++ b/mercurial/chgserver.py
@@ -313,10 +313,11 @@ class channeledsystem(object):
 ]
 
 class chgcmdserver(commandserver.server):
-def __init__(self, ui, repo, fin, fout, sock, hashstate, baseaddress):
+def __init__(self, ui, repo, fin, fout, sock, prereposetups,
+ hashstate, baseaddress):
 super(chgcmdserver, self).__init__(
 _newchgui(ui, channeledsystem(fin, fout, 'S'), self.attachio),
-repo, fin, fout)
+repo, fin, fout, prereposetups)
 self.clientsock = sock
 self._ioattached = False
 self._oldios = []  # original (self.ch, ui.fp, fd) before "attachio"
@@ -617,8 +618,8 @@ class chgunixservicehandler(object):
 def newconnection(self):
 self._lastactive = time.time()
 
-def createcmdserver(self, repo, conn, fin, fout):
-return chgcmdserver(self.ui, repo, fin, fout, conn,
+def createcmdserver(self, repo, conn, fin, fout, prereposetups):
+return chgcmdserver(self.ui, repo, fin, fout, conn, prereposetups,
 self._hashstate, self._baseaddress)
 
 def chgunixservice(ui, repo, opts):
diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -196,7 +196,7 @@ class server(object):
 Listens for commands on fin, runs them and writes the output on a channel
 based stream to fout.
 """
-def __init__(self, ui, repo, fin, fout):
+def __init__(self, ui, repo, fin, fout, prereposetups=None):
 self.cwd = encoding.getcwd()
 
 if repo:
@@ -208,6 +208,7 @@ class server(object):
 else:
 self.ui = ui
 self.repo = self.repoui = None
+self._prereposetups = prereposetups
 
 self.cdebug = channeledoutput(fout, 'd')
 self.cerr = channeledoutput(fout, 'e')
@@ -294,7 +295,8 @@ class server(object):
 ui.setconfig('ui', 'nontty', 'true', 'commandserver')
 
 req = dispatch.request(args[:], copiedui, self.repo, self.cin,
-   self.cout, self.cerr, self.cmsg)
+   self.cout, self.cerr, self.cmsg,
+   prereposetups=self._prereposetups)
 
 try:
 ret = dispatch.dispatch(req) & 255
@@ -420,12 +422,12 @@ def _initworkerprocess():
 # same state inherited from parent.
 random.seed()
 
-def _serverequest(ui, repo, conn, createcmdserver):
+def _serverequest(ui, repo, conn, createcmdserver, prereposetups):
 fin = conn.makefile(r'rb')
 fout = conn.makefile(r'wb')
 sv = None
 try:
-sv = createcmdserver(repo, conn, fin, fout)
+sv = createcmdserver(repo, conn, fin, fout, prereposetups)
 try:
 sv.serve()
 # handle exceptions that may be raised by command server. most of
@@ -484,10 +486,10 @@ class unixservicehandler(object):
 def newconnection(self):
 """Called when main process notices new connection"""
 
-def createcmdserver(self, repo, conn, fin, fout):
+def createcmdserver(self, repo, conn, fin, fout, prereposetups):
 """Create new command server instance; called in the process that
 serves for the current connection"""
-return server(self.ui, repo, fin, fout)
+return server(self.ui, repo, fin, fout, prereposetups)
 
 class unixforkingservice(object):
 """
@@ -619,6 +621,7 @@ class unixforkingservice(object):
 _initworkerprocess()
 h = self._servicehandler
 try:
-_serverequest(self.ui, self.repo, conn, h.createcmdserver)
+_serverequest(self.ui, self.repo, conn, h.createcmdserver,
+  prereposetups=None)  # TODO: pass in hook functions
 finally:
 gc.collect()  # trigger __del__ since worker process uses os._exit
diff --git a/tests/test-commandserver.t b/tests/test-commandserver.t
--- a/tests/test-commandserver.t
+++ b/tests/test-commandserver.t
@@ -848,10 +848,10 @@ unix domain socket:
 
   $ cat <<'EOF' > ../earlycrasher.py
   > from mercurial import commandserver, extensions
-  > def _serverequest(orig, ui, repo, conn, createcmdserver):
+  > def _serverequest(orig, ui, repo, conn, createcmdserver, prereposetups):
   > def createcmdserver(*args, **kwargs):
   > raise Exception('crash')
-  > return orig(ui, repo, conn, createcmdserver)
+  > return orig(ui, repo, conn, createcmdserver, prereposetups)
   > def 

[PATCH 4 of 8] commandserver: loop over selector events

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540991145 -32400
#  Wed Oct 31 22:05:45 2018 +0900
# Node ID af9746ae6d62457788853eda54374b192b7a7134
# Parent  92023422fbe3cf05e6d25b8213bd1e5a749c8e98
commandserver: loop over selector events

An IPC socket will be waited by the same selector.

diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -541,7 +541,8 @@ class unixforkingservice(object):
 exiting = False
 h = self._servicehandler
 selector = selectors.DefaultSelector()
-selector.register(self._sock, selectors.EVENT_READ)
+selector.register(self._sock, selectors.EVENT_READ,
+  self._acceptnewconnection)
 while True:
 if not exiting and h.shouldexit():
 # clients can no longer connect() to the domain socket, so
@@ -552,20 +553,21 @@ class unixforkingservice(object):
 self._unlinksocket()
 exiting = True
 try:
-ready = selector.select(timeout=h.pollinterval)
+events = selector.select(timeout=h.pollinterval)
 except OSError as inst:
 # selectors2 raises ETIMEDOUT if timeout exceeded while
 # handling signal interrupt. That's probably wrong, but
 # we can easily get around it.
 if inst.errno != errno.ETIMEDOUT:
 raise
-ready = []
-if not ready:
+events = []
+if not events:
 # only exit if we completed all queued requests
 if exiting:
 break
 continue
-self._acceptnewconnection(self._sock, selector)
+for key, _mask in events:
+key.data(key.fileobj, selector)
 selector.close()
 
 def _acceptnewconnection(self, sock, selector):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5375: remotefilelog: use progress helper when analyzing repos

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG05afb32e9380: remotefilelog: use progress helper when 
analyzing repos (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5375?vs=12714=12737

REVISION DETAIL
  https://phab.mercurial-scm.org/D5375

AFFECTED FILES
  hgext/remotefilelog/__init__.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -811,14 +811,14 @@
 validrepos = []
 keepkeys = set()
 
-_analyzing = _("analyzing repositories")
-
 sharedcache = None
 filesrepacked = False
 
 count = 0
+progress = ui.makeprogress(_("analyzing repositories"), unit="repos",
+   total=len(repos))
 for path in repos:
-ui.progress(_analyzing, count, unit="repos", total=len(repos))
+progress.update(count)
 count += 1
 try:
 path = ui.expandpath(os.path.normpath(path))
@@ -868,7 +868,7 @@
 return fileserverclient.getcachekey(reponame, fname, hex(fnode))
 keepkeys = repackmod.keepset(repo, keyfn=keyfn, lastkeepkeys=keepkeys)
 
-ui.progress(_analyzing, None)
+progress.complete()
 
 # write list of valid repos back
 oldumask = os.umask(0o002)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5385: remotefilelog: reduce use of "count" container

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGfcee112fb7c4: remotefilelog: reduce use of 
count container (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5385?vs=12724=12746

REVISION DETAIL
  https://phab.mercurial-scm.org/D5385

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -361,9 +361,9 @@
 
 fromcache = total - len(missed)
 count = [fromcache]
-progress.update(count[0], total=total)
+progress.update(fromcache, total=total)
 self.ui.log("remotefilelog", "remote cache hit rate is %r of %r\n",
-count[0], total, hit=count[0], total=total)
+fromcache, total, hit=fromcache, total=total)
 
 oldumask = os.umask(0o002)
 try:



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5388: tests: split test-shelve.t in two

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1b836cee2d91: tests: split test-shelve.t in two (authored 
by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5388?vs=12728=12748

REVISION DETAIL
  https://phab.mercurial-scm.org/D5388

AFFECTED FILES
  tests/test-shelve.t
  tests/test-shelve2.t

CHANGE DETAILS

diff --git a/tests/test-shelve.t b/tests/test-shelve2.t
copy from tests/test-shelve.t
copy to tests/test-shelve2.t
--- a/tests/test-shelve.t
+++ b/tests/test-shelve2.t
@@ -20,657 +20,8 @@
 
 #endif
 
-  $ hg init repo
-  $ cd repo
-  $ mkdir a b
-  $ echo a > a/a
-  $ echo b > b/b
-  $ echo c > c
-  $ echo d > d
-  $ echo x > x
-  $ hg addremove -q
-
-shelve has a help message
-  $ hg shelve -h
-  hg shelve [OPTION]... [FILE]...
-  
-  save and set aside changes from the working directory
-  
-  Shelving takes files that "hg status" reports as not clean, saves the
-  modifications to a bundle (a shelved change), and reverts the files so
-  that their state in the working directory becomes clean.
-  
-  To restore these changes to the working directory, using "hg unshelve";
-  this will work even if you switch to a different commit.
-  
-  When no files are specified, "hg shelve" saves all not-clean files. If
-  specific files or directories are named, only changes to those files are
-  shelved.
-  
-  In bare shelve (when no files are specified, without interactive, include
-  and exclude option), shelving remembers information if the working
-  directory was on newly created branch, in other words working directory
-  was on different branch than its first parent. In this situation
-  unshelving restores branch information to the working directory.
-  
-  Each shelved change has a name that makes it easier to find later. The
-  name of a shelved change defaults to being based on the active bookmark,
-  or if there is no active bookmark, the current named branch.  To specify 
a
-  different name, use "--name".
-  
-  To see a list of existing shelved changes, use the "--list" option. For
-  each shelved change, this will print its name, age, and description; use 
"
-  --patch" or "--stat" for more details.
-  
-  To delete specific shelved changes, use "--delete". To delete all shelved
-  changes, use "--cleanup".
-  
-  (use 'hg help -e shelve' to show help for the shelve extension)
-  
-  options ([+] can be repeated):
-  
-   -A --addremove   mark new/missing files as added/removed before
-shelving
-   -u --unknown store unknown files in the shelve
-  --cleanup delete all shelved changes
-  --date DATE   shelve with the specified commit date
-   -d --delete  delete the named shelved change(s)
-   -e --editinvoke editor on commit messages
-   -l --listlist current shelves
-   -m --message TEXTuse text as shelve message
-   -n --name NAME   use the given name for the shelved commit
-   -p --patch   output patches for changes (provide the names of 
the
-shelved changes as positional arguments)
-   -i --interactive interactive mode, only works while creating a 
shelve
-  --statoutput diffstat-style summary of changes (provide
-the names of the shelved changes as positional
-arguments)
-   -I --include PATTERN [+] include names matching the given patterns
-   -X --exclude PATTERN [+] exclude names matching the given patterns
-  --mq  operate on patch repository
-  
-  (some details hidden, use --verbose to show complete help)
-
-shelving in an empty repo should be possible
-(this tests also that editor is not invoked, if '--edit' is not
-specified)
-
-  $ HGEDITOR=cat hg shelve
-  shelved as default
-  0 files updated, 0 files merged, 5 files removed, 0 files unresolved
-
-  $ hg unshelve
-  unshelving change 'default'
-
-  $ hg commit -q -m 'initial commit'
-
-  $ hg shelve
-  nothing changed
-  [1]
-
-make sure shelve files were backed up
-
-  $ ls .hg/shelve-backup
-  default.hg
-  default.patch
-  default.shelve
-
-checks to make sure we dont create a directory or
-hidden file while choosing a new shelve name
-
-when we are given a name
-
-  $ hg shelve -n foo/bar
-  abort: shelved change names can not contain slashes
-  [255]
-  $ hg shelve -n .baz
-  abort: shelved change names can not start with '.'
-  [255]
-  $ hg shelve -n foo\\bar
-  abort: shelved change names can not contain slashes
-  [255]
-
-when shelve has to choose itself
-
-  $ hg branch x/y -q
-  $ hg commit -q -m "Branch commit 0"
-  $ hg shelve
-  nothing changed
-  [1]
-  $ hg branch .x -q
-  $ hg commit -q -m "Branch commit 1"
-  $ hg shelve
-  nothing 

D5389: shelve: drop unnecessary backup of dirstate for phase-based case

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd7e44da4dc70: shelve: drop unnecessary backup of dirstate 
for phase-based case (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5389?vs=12729=12749

REVISION DETAIL
  https://phab.mercurial-scm.org/D5389

AFFECTED FILES
  hgext/shelve.py

CHANGE DETAILS

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -446,11 +446,8 @@
 
 def _finishshelve(repo):
 if phases.supportinternal(repo):
-backupname = 'dirstate.shelve'
 tr = repo.currenttransaction()
-repo.dirstate.savebackup(tr, backupname)
 tr.close()
-repo.dirstate.restorebackup(None, backupname)
 else:
 _aborttransaction(repo)
 



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 8] commandserver: preload repository in master server and reuse its file cache

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540993388 -32400
#  Wed Oct 31 22:43:08 2018 +0900
# Node ID 91fd85965d35c7eab04c8601442953639a65bf7c
# Parent  d5e7ded8150f66f89b5b6ed0a0c49ea38018982e
commandserver: preload repository in master server and reuse its file cache

This greatly speeds up repository operation with lots of obsolete markers:

  $ ls -lh .hg/store/obsstore
  -rw-r--r-- 1 yuya yuya 21M Dec  2 17:55 .hg/store/obsstore
  $ time hg log -G -l10 --pager no
  (hg)   1.79s user 0.13s system 99% cpu 1.919 total
  (chg uncached) 0.00s user 0.01s system 0% cpu 1.328 total
  (chg cached)   0.00s user 0.00s system 3% cpu 0.180 total

As you can see, the implementation of the preloader function is highly
experimental. It works, but I'm yet to be sure how things can be organized.
So I don't want to formalize the API at this point.

diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -28,6 +28,7 @@ from . import (
 error,
 loggingutil,
 pycompat,
+repocache,
 util,
 vfs as vfsmod,
 )
@@ -511,6 +512,11 @@ class unixforkingservice(object):
 self._oldsigchldhandler = None
 self._workerpids = set()  # updated by signal handler; do not iterate
 self._socketunlinked = None
+# experimental config: cmdserver.max-repo-cache
+maxlen = ui.configint(b'cmdserver', b'max-repo-cache')
+if maxlen < 0:
+raise error.Abort(_('negative max-repo-cache size not allowed'))
+self._repoloader = repocache.repoloader(ui, maxlen)
 
 def init(self):
 self._sock = socket.socket(socket.AF_UNIX)
@@ -525,6 +531,7 @@ class unixforkingservice(object):
 o = signal.signal(signal.SIGCHLD, self._sigchldhandler)
 self._oldsigchldhandler = o
 self._socketunlinked = False
+self._repoloader.start()
 
 def _unlinksocket(self):
 if not self._socketunlinked:
@@ -537,6 +544,7 @@ class unixforkingservice(object):
 self._mainipc.close()
 self._workeripc.close()
 self._unlinksocket()
+self._repoloader.stop()
 # don't kill child processes as they have active clients, just wait
 self._reapworkers(0)
 
@@ -590,6 +598,10 @@ class unixforkingservice(object):
 return
 raise
 
+# Future improvement: On Python 3.7, maybe gc.freeze() can be used
+# to prevent COW memory from being touched by GC.
+# https://instagram-engineering.com/
+#   copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf
 pid = os.fork()
 if pid:
 try:
@@ -622,8 +634,7 @@ class unixforkingservice(object):
 if inst.args[0] == errno.EINTR:
 return
 raise
-
-self.ui.log(b'cmdserver', b'repository: %s\n', path)
+self._repoloader.load(path)
 
 def _sigchldhandler(self, signal, frame):
 self._reapworkers(os.WNOHANG)
@@ -671,3 +682,9 @@ class unixforkingservice(object):
 
 repo.__class__ = unixcmdserverrepo
 repo._cmdserveripc = self._workeripc
+
+cachedrepo = self._repoloader.get(repo.root)
+if cachedrepo is None:
+return
+repo.ui.log(b'repocache', b'repo from cache: %s\n', repo.root)
+repocache.copycache(cachedrepo, repo)
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -179,11 +179,14 @@ coreconfigitem('cmdserver', 'max-log-fil
 coreconfigitem('cmdserver', 'max-log-size',
 default='1 MB',
 )
+coreconfigitem('cmdserver', 'max-repo-cache',
+default=0,
+)
 coreconfigitem('cmdserver', 'message-encodings',
 default=list,
 )
 coreconfigitem('cmdserver', 'track-log',
-default=lambda: ['chgserver', 'cmdserver'],
+default=lambda: ['chgserver', 'cmdserver', 'repocache'],
 )
 coreconfigitem('color', '.*',
 default=None,
diff --git a/mercurial/repocache.py b/mercurial/repocache.py
new file mode 100644
--- /dev/null
+++ b/mercurial/repocache.py
@@ -0,0 +1,131 @@
+# repocache.py - in-memory repository cache for long-running services
+#
+# Copyright 2018 Yuya Nishihara 
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+import collections
+import gc
+import threading
+
+from . import (
+error,
+hg,
+obsolete,
+scmutil,
+util,
+)
+
+class repoloader(object):
+"""Load repositories in background thread
+
+This is designed for a forking server. A cached repo cannot be obtained
+until the server fork()s a worker and the loader thread stops.
+"""
+
+def __init__(self, ui, maxlen):
+self._ui = ui.copy()
+self._cache = util.lrucachedict(max=maxlen)
+# use deque and Event instead of Queue since deque 

Re: D5374: fastannotate: use progress helper

[Yuya Nishihara]

>  if ismainbranch: # need to write to linelog
> -if not self.ui.quiet:
> -progress += 1
> -self.ui.progress(_('building cache'), progress,
> - total=len(newmainbranch))
> +progress.increment()

This one is BC, but I think that's okay.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5376: remotefilelog: use progress helper in basestore

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGad21fbcb1ba5: remotefilelog: use progress helper in 
basestore (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5376?vs=12715=12736

REVISION DETAIL
  https://phab.mercurial-scm.org/D5376

AFFECTED FILES
  hgext/remotefilelog/basestore.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/basestore.py b/hgext/remotefilelog/basestore.py
--- a/hgext/remotefilelog/basestore.py
+++ b/hgext/remotefilelog/basestore.py
@@ -77,14 +77,15 @@
 ui = self.ui
 entries = ledger.sources.get(self, [])
 count = 0
+progress = ui.makeprogress(_("cleaning up"), unit="files",
+   total=len(entries))
 for entry in entries:
 if entry.gced or (entry.datarepacked and entry.historyrepacked):
-ui.progress(_("cleaning up"), count, unit="files",
-total=len(entries))
+progress.update(count)
 path = self._getfilepath(entry.filename, entry.node)
 util.tryunlink(path)
 count += 1
-ui.progress(_("cleaning up"), None)
+progress.complete()
 
 # Clean up the repo cache directory.
 self._cleanupdirectory(self._getrepocachepath())
@@ -302,8 +303,6 @@
 def gc(self, keepkeys):
 ui = self.ui
 cachepath = self._path
-_removing = _("removing unnecessary files")
-_truncating = _("enforcing cache limit")
 
 # prune cache
 import Queue
@@ -316,7 +315,9 @@
 # keep files newer than a day even if they aren't needed
 limit = time.time() - (60 * 60 * 24)
 
-ui.progress(_removing, count, unit="files")
+progress = ui.makeprogress(_("removing unnecessary files"),
+   unit="files")
+progress.update(0)
 for root, dirs, files in os.walk(cachepath):
 for file in files:
 if file == 'repos':
@@ -326,7 +327,7 @@
 if '/packs/' in root:
 continue
 
-ui.progress(_removing, count, unit="files")
+progress.update(count)
 path = os.path.join(root, file)
 key = os.path.relpath(path, cachepath)
 count += 1
@@ -357,16 +358,17 @@
 ui.warn(msg % path)
 continue
 removed += 1
-ui.progress(_removing, None)
+progress.complete()
 
 # remove oldest files until under limit
 limit = ui.configbytes("remotefilelog", "cachelimit")
 if size > limit:
 excess = size - limit
+progress = ui.makeprogress(_("enforcing cache limit"), 
unit="bytes",
+   total=excess)
 removedexcess = 0
 while queue and size > limit and size > 0:
-ui.progress(_truncating, removedexcess, unit="bytes",
-total=excess)
+progress.update(removedexcess)
 atime, oldpath, oldpathstat = queue.get()
 try:
 shallowutil.unlinkfile(oldpath)
@@ -379,7 +381,7 @@
 size -= oldpathstat.st_size
 removed += 1
 removedexcess += oldpathstat.st_size
-ui.progress(_truncating, None)
+progress.complete()
 
 ui.status(_("finished: removed %s of %s files (%0.2f GB to %0.2f 
GB)\n")
   % (removed, count,



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5381: remotefilelog: use progress helper in fileserverclient

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe58cd7ede1c3: remotefilelog: use progress helper in 
fileserverclient (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5381?vs=12720=12742

REVISION DETAIL
  https://phab.mercurial-scm.org/D5381

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -42,7 +42,6 @@
 fetchmisses = 0
 
 _lfsmod = None
-_downloading = _('downloading')
 
 def getcachekey(reponame, file, id):
 pathhash = node.hex(hashlib.sha1(file).digest())
@@ -332,7 +331,8 @@
 cache.request(request)
 
 total = count
-self.ui.progress(_downloading, 0, total=count)
+progress = self.ui.makeprogress(_('downloading'), total=count)
+progress.update(0)
 
 missed = []
 count = 0
@@ -352,7 +352,7 @@
 # receive progress reports
 parts = missingid.split("_")
 count += int(parts[2])
-self.ui.progress(_downloading, count, total=total)
+progress.update(count)
 continue
 
 missed.append(missingid)
@@ -362,7 +362,7 @@
 
 count = [total - len(missed)]
 fromcache = count[0]
-self.ui.progress(_downloading, count[0], total=total)
+progress.update(count[0], total=total)
 self.ui.log("remotefilelog", "remote cache hit rate is %r of %r\n",
 count[0], total, hit=count[0], total=total)
 
@@ -372,7 +372,7 @@
 if missed:
 def progresstick():
 count[0] += 1
-self.ui.progress(_downloading, count[0], total=total)
+progress.update(count[0])
 # When verbose is true, sshpeer prints 'running ssh...'
 # to stdout, which can interfere with some command
 # outputs
@@ -427,7 +427,7 @@
 request = "set\n%d\n%s\n" % (count[0], "\n".join(missed))
 cache.request(request)
 
-self.ui.progress(_downloading, None)
+progress.complete()
 
 # mark ourselves as a user of this cache
 writedata.markrepo(self.repo.path)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5386: remotefilelog: rely on progress helper for keeping track of position

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG74e3df766052: remotefilelog: rely on progress helper for 
keeping track of position (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5386?vs=12725=12747

REVISION DETAIL
  https://phab.mercurial-scm.org/D5386

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -334,7 +334,6 @@
 progress.update(0)
 
 missed = []
-count = 0
 while True:
 missingid = cache.receiveline()
 if not missingid:
@@ -350,28 +349,23 @@
 if missingid.startswith("_hits_"):
 # receive progress reports
 parts = missingid.split("_")
-count += int(parts[2])
-progress.update(count)
+progress.increment(int(parts[2]))
 continue
 
 missed.append(missingid)
 
 global fetchmisses
 fetchmisses += len(missed)
 
 fromcache = total - len(missed)
-count = [fromcache]
 progress.update(fromcache, total=total)
 self.ui.log("remotefilelog", "remote cache hit rate is %r of %r\n",
 fromcache, total, hit=fromcache, total=total)
 
 oldumask = os.umask(0o002)
 try:
 # receive cache misses from master
 if missed:
-def progresstick():
-count[0] += 1
-progress.update(count[0])
 # When verbose is true, sshpeer prints 'running ssh...'
 # to stdout, which can interfere with some command
 # outputs
@@ -393,30 +387,30 @@
 _getfiles = _getfiles_threaded
 else:
 _getfiles = _getfiles_optimistic
-_getfiles(remote, self.receivemissing, 
progresstick,
-  missed, idmap, step)
+_getfiles(remote, self.receivemissing,
+  progress.increment, missed, idmap, step)
 elif remote.capable("x_rfl_getfile"):
 if remote.capable('batch'):
 batchdefault = 100
 else:
 batchdefault = 10
 batchsize = self.ui.configint(
 'remotefilelog', 'batchsize', batchdefault)
 _getfilesbatch(
-remote, self.receivemissing, progresstick,
+remote, self.receivemissing, 
progress.increment,
 missed, idmap, batchsize)
 else:
 raise error.Abort("configured remotefilelog server"
  " does not support remotefilelog")
 
 self.ui.log("remotefilefetchlog",
 "Success\n",
-fetched_files = count[0] - fromcache,
+fetched_files = progress.pos - fromcache,
 total_to_fetch = total - fromcache)
 except Exception:
 self.ui.log("remotefilefetchlog",
 "Fail\n",
-fetched_files = count[0] - fromcache,
+fetched_files = progress.pos - fromcache,
 total_to_fetch = total - fromcache)
 raise
 finally:



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5392: rebase: move abort() onto rebaseruntime

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc8eb8d1fc6cf: rebase: move abort() onto rebaseruntime 
(authored by martinvonz, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5392?vs=12732=12752#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5392?vs=12732=12752

REVISION DETAIL
  https://phab.mercurial-scm.org/D5392

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -347,9 +347,9 @@
 
 if isabort:
 backup = backup and self.backupf
-return abort(self.repo, self.originalwd, self.destmap, self.state,
- activebookmark=self.activebookmark, backup=backup,
- suppwarns=suppwarns)
+return self._abort(self.repo, self.originalwd, self.destmap,
+   self.state, activebookmark=self.activebookmark,
+   backup=backup, suppwarns=suppwarns)
 
 def _preparenewrebase(self, destmap):
 if not destmap:
@@ -653,6 +653,66 @@
 repo['.'].node() == repo._bookmarks[self.activebookmark]):
 bookmarks.activate(repo, self.activebookmark)
 
+def _abort(self, repo, originalwd, destmap, state, activebookmark=None,
+backup=True, suppwarns=False):
+'''Restore the repository to its original state.  Additional args:
+
+activebookmark: the name of the bookmark that should be active after 
the
+restore'''
+
+try:
+# If the first commits in the rebased set get skipped during the
+# rebase, their values within the state mapping will be the dest
+# rev id. The rebased list must must not contain the dest rev
+# (issue4896)
+rebased = [s for r, s in state.items()
+   if s >= 0 and s != r and s != destmap[r]]
+immutable = [d for d in rebased if not repo[d].mutable()]
+cleanup = True
+if immutable:
+repo.ui.warn(_("warning: can't clean up public changesets 
%s\n")
+ % ', '.join(bytes(repo[r]) for r in immutable),
+ hint=_("see 'hg help phases' for details"))
+cleanup = False
+
+descendants = set()
+if rebased:
+descendants = set(repo.changelog.descendants(rebased))
+if descendants - set(rebased):
+repo.ui.warn(_("warning: new changesets detected on "
+   "destination branch, can't strip\n"))
+cleanup = False
+
+if cleanup:
+shouldupdate = False
+if rebased:
+strippoints = [
+c.node() for c in repo.set('roots(%ld)', rebased)]
+
+updateifonnodes = set(rebased)
+updateifonnodes.update(destmap.values())
+updateifonnodes.add(originalwd)
+shouldupdate = repo['.'].rev() in updateifonnodes
+
+# Update away from the rebase if necessary
+if shouldupdate or needupdate(repo, state):
+mergemod.update(repo, originalwd, branchmerge=False,
+force=True)
+
+# Strip from the first rebased revision
+if rebased:
+repair.strip(repo.ui, repo, strippoints, backup=backup)
+
+if activebookmark and activebookmark in repo._bookmarks:
+bookmarks.activate(repo, activebookmark)
+
+finally:
+clearstatus(repo)
+clearcollapsemsg(repo)
+if not suppwarns:
+repo.ui.warn(_('rebase aborted\n'))
+return 0
+
 @command('rebase',
 [('s', 'source', '',
  _('rebase the specified changeset and descendants'), _('REV')),
@@ -1609,64 +1669,6 @@
 
 return False
 
-def abort(repo, originalwd, destmap, state, activebookmark=None, backup=True,
-  suppwarns=False):
-'''Restore the repository to its original state.  Additional args:
-
-activebookmark: the name of the bookmark that should be active after the
-restore'''
-
-try:
-# If the first commits in the rebased set get skipped during the 
rebase,
-# their values within the state mapping will be the dest rev id. The
-# rebased list must must not contain the dest rev (issue4896)
-rebased = [s for r, s in state.items()
-   if s >= 0 and s != r and s != destmap[r]]
-immutable = [d for d in rebased if not repo[d].mutable()]
-cleanup = True
-if immutable:
-repo.ui.warn(_("warning: can't clean up public changesets %s\n")
-% ', '.join(bytes(repo[r]) for r in immutable),
-  

D5390: shelve: change transaction description from "commit" to "shelve"

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG236af7cfa4c3: shelve: change transaction description from 
commit to shelve (authored by martinvonz, committed by 
).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5390?vs=12730=12750

REVISION DETAIL
  https://phab.mercurial-scm.org/D5390

AFFECTED FILES
  hgext/shelve.py

CHANGE DETAILS

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -479,7 +479,7 @@
 
 # use an uncommitted transaction to generate the bundle to avoid
 # pull races. ensure we don't print the abort message to stderr.
-tr = repo.transaction('commit', report=lambda x: None)
+tr = repo.transaction('shelve', report=lambda x: None)
 
 interactive = opts.get('interactive', False)
 includeunknown = (opts.get('unknown', False) and



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5370: rust: core implementation of missingancestors (no bindings)

[kevincox (Kevin Cox)]

kevincox added a comment.


  I think sticking close to the python version makes sense for the initial 
version. Then improvements can be made in follow-ups.

INLINE COMMENTS

> gracinet wrote in lib.rs:26
> The reason I've preferred to implement it directly is that `into_iter()` 
> iterates on references, which I found an unnecessary overhead,.
> 
> But maybe that's a case of premature optimization ? In the same vein, I don't 
> think `HashSet` is the best choice for a set of `i32`, unless it becomes 
> really big, but I don't know where the  threshold would be, compared to, say, 
> a bit array.

You can add `.cloned()` to remove the references. I would be surprised if this 
has poor code generation.

As for HashSet vs bit array I would stick to HashSet for now. HashSet also has 
the advantage of being sparse. I guess after the initial implementation we 
could benchmark the two to see what is better.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5370

To: gracinet, #hg-reviewers
Cc: durin42, kevincox, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8] commandserver: extract handler of new socket connection

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540990958 -32400
#  Wed Oct 31 22:02:38 2018 +0900
# Node ID c9533007e0e09fe8129b46208dbb760ff8799a88
# Parent  958a93f4f9210c4b5669e4424f8d73b048ef72ed
commandserver: extract handler of new socket connection

This prepares for adding an IPC socket.

diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -565,11 +565,17 @@ class unixforkingservice(object):
 if exiting:
 break
 continue
+self._acceptnewconnection(self._sock, selector)
+selector.close()
+
+def _acceptnewconnection(self, sock, selector):
+h = self._servicehandler
+if True:
 try:
-conn, _addr = self._sock.accept()
+conn, _addr = sock.accept()
 except socket.error as inst:
 if inst.args[0] == errno.EINTR:
-continue
+return
 raise
 
 pid = os.fork()
@@ -584,7 +590,7 @@ class unixforkingservice(object):
 else:
 try:
 selector.close()
-self._sock.close()
+sock.close()
 self._runworker(conn)
 conn.close()
 os._exit(0)
@@ -593,7 +599,6 @@ class unixforkingservice(object):
 self.ui.traceback(force=True)
 finally:
 os._exit(255)
-selector.close()
 
 def _sigchldhandler(self, signal, frame):
 self._reapworkers(os.WNOHANG)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5372: perfrevlogwrite: use progress helper

[yuja (Yuya Nishihara)]

yuja added a comment.


  Queued the patches but for this, thanks. perf.py has to support old Mercurial
  versions.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5372

To: martinvonz, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5378: remotefilelog: use progress helper in repack

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb80915b52476: remotefilelog: use progress helper in repack 
(authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5378?vs=12717=12740

REVISION DETAIL
  https://phab.mercurial-scm.org/D5378

AFFECTED FILES
  hgext/remotefilelog/repack.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/repack.py b/hgext/remotefilelog/repack.py
--- a/hgext/remotefilelog/repack.py
+++ b/hgext/remotefilelog/repack.py
@@ -495,27 +495,29 @@
 byfile.setdefault(entry.filename, {})[entry.node] = entry
 
 count = 0
+repackprogress = ui.makeprogress(_("repacking data"), unit=self.unit,
+total=len(byfile))
 for filename, entries in sorted(byfile.iteritems()):
-ui.progress(_("repacking data"), count, unit=self.unit,
-total=len(byfile))
+repackprogress.update(count)
 
 ancestors = {}
 nodes = list(node for node in entries)
 nohistory = []
+buildprogress = ui.makeprogress(_("building history"), 
unit='nodes',
+total=len(nodes))
 for i, node in enumerate(nodes):
 if node in ancestors:
 continue
-ui.progress(_("building history"), i, unit='nodes',
-total=len(nodes))
+buildprogress.update(i)
 try:
 ancestors.update(self.fullhistory.getancestors(filename,
 node, known=ancestors))
 except KeyError:
 # Since we're packing data entries, we may not have the
 # corresponding history entries for them. It's not a big
 # deal, but the entries won't be delta'd perfectly.
 nohistory.append(node)
-ui.progress(_("building history"), None)
+buildprogress.complete()
 
 # Order the nodes children first, so we can produce reverse deltas
 orderednodes = list(reversed(self._toposort(ancestors)))
@@ -547,9 +549,11 @@
 nobase = set()
 referenced = set()
 nodes = set(nodes)
+processprogress = ui.makeprogress(_("processing nodes"),
+  unit='nodes',
+  total=len(orderednodes))
 for i, node in enumerate(orderednodes):
-ui.progress(_("processing nodes"), i, unit='nodes',
-total=len(orderednodes))
+processprogress.update(i)
 # Find delta base
 # TODO: allow delta'ing against most recent descendant instead
 # of immediate child
@@ -620,10 +624,10 @@
 
 entries[node].datarepacked = True
 
-ui.progress(_("processing nodes"), None)
+processprogress.complete()
 count += 1
 
-ui.progress(_("repacking data"), None)
+repackprogress.complete()
 target.close(ledger=ledger)
 
 def repackhistory(self, ledger, target):
@@ -634,7 +638,8 @@
 if entry.historysource:
 byfile.setdefault(entry.filename, {})[entry.node] = entry
 
-count = 0
+progress = ui.makeprogress(_("repacking history"), unit=self.unit,
+   total=len(byfile))
 for filename, entries in sorted(byfile.iteritems()):
 ancestors = {}
 nodes = list(node for node in entries)
@@ -678,11 +683,9 @@
 if node in entries:
 entries[node].historyrepacked = True
 
-count += 1
-ui.progress(_("repacking history"), count, unit=self.unit,
-total=len(byfile))
+progress.increment()
 
-ui.progress(_("repacking history"), None)
+progress.complete()
 target.close(ledger=ledger)
 
 def _toposort(self, ancestors):



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8] commandserver: add IPC channel to teach repository path on command finished

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1540991943 -32400
#  Wed Oct 31 22:19:03 2018 +0900
# Node ID 5f4354d1aa370a028eee8bcc6de245b1212cf35c
# Parent  af9746ae6d62457788853eda54374b192b7a7134
commandserver: add IPC channel to teach repository path on command finished

The idea is to load recently-used repositories first in the master process,
and fork(). The forked worker can reuse a warm repository if it's preloaded.

There are a couple of ways of in-memory repository caching. They have pros
and cons:

 a. "preload by master"
pros: can use a single cache dict, maximizing cache hit rate
cons: need to reload a repo in master process (because worker process
  dies per command)
 b. "prefork"
pros: can cache a repo without reloading (as worker processes persist)
cons: lower cache hit rate since each worker has to maintain its own cache
 c. "shared memory" (or separate key-value store server)
pros: no need to reload a repo in master process, ideally
cons: need to serialize objects to sharable form

Since my primary goal is to get rid of the cost of loading obsstore without
massive rewrites, (c) doesn't work. (b) isn't ideal since it would require
much more SDRAMs than (a). So I take (a).

The idea credits to Jun Wu.

diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -506,12 +506,19 @@ class unixforkingservice(object):
 raise error.Abort(_('no socket path specified with --address'))
 self._servicehandler = handler or unixservicehandler(ui)
 self._sock = None
+self._mainipc = None
+self._workeripc = None
 self._oldsigchldhandler = None
 self._workerpids = set()  # updated by signal handler; do not iterate
 self._socketunlinked = None
 
 def init(self):
 self._sock = socket.socket(socket.AF_UNIX)
+# IPC channel from many workers to one main process; this is actually
+# a uni-directional pipe, but is backed by a DGRAM socket so each
+# message can be easily separated.
+o = socket.socketpair(socket.AF_UNIX, socket.SOCK_DGRAM)
+self._mainipc, self._workeripc = o
 self._servicehandler.bindsocket(self._sock, self.address)
 if util.safehasattr(procutil, 'unblocksignal'):
 procutil.unblocksignal(signal.SIGCHLD)
@@ -527,6 +534,8 @@ class unixforkingservice(object):
 def _cleanup(self):
 signal.signal(signal.SIGCHLD, self._oldsigchldhandler)
 self._sock.close()
+self._mainipc.close()
+self._workeripc.close()
 self._unlinksocket()
 # don't kill child processes as they have active clients, just wait
 self._reapworkers(0)
@@ -543,6 +552,8 @@ class unixforkingservice(object):
 selector = selectors.DefaultSelector()
 selector.register(self._sock, selectors.EVENT_READ,
   self._acceptnewconnection)
+selector.register(self._mainipc, selectors.EVENT_READ,
+  self._handlemainipc)
 while True:
 if not exiting and h.shouldexit():
 # clients can no longer connect() to the domain socket, so
@@ -592,8 +603,10 @@ class unixforkingservice(object):
 try:
 selector.close()
 sock.close()
+self._mainipc.close()
 self._runworker(conn)
 conn.close()
+self._workeripc.close()
 os._exit(0)
 except:  # never return, hence no re-raises
 try:
@@ -601,6 +614,17 @@ class unixforkingservice(object):
 finally:
 os._exit(255)
 
+def _handlemainipc(self, sock, selector):
+"""Process messages sent from a worker"""
+try:
+path = sock.recv(32768)  # large enough to receive path
+except socket.error as inst:
+if inst.args[0] == errno.EINTR:
+return
+raise
+
+self.ui.log(b'cmdserver', b'repository: %s\n', path)
+
 def _sigchldhandler(self, signal, frame):
 self._reapworkers(os.WNOHANG)
 
@@ -628,6 +652,22 @@ class unixforkingservice(object):
 h = self._servicehandler
 try:
 _serverequest(self.ui, self.repo, conn, h.createcmdserver,
-  prereposetups=None)  # TODO: pass in hook functions
+  prereposetups=[self._reposetup])
 finally:
 gc.collect()  # trigger __del__ since worker process uses os._exit
+
+def _reposetup(self, ui, repo):
+if not repo.local():
+return
+
+class unixcmdserverrepo(repo.__class__):
+def close(self):
+super(unixcmdserverrepo, self).close()
+try:
+self._cmdserveripc.send(self.root)
+except socket.error:
+ 

Re: D5370: rust: core implementation of missingancestors (no bindings)

[Yuya Nishihara]

Quickly scanned, and looks generally good to me.

>   rust: iterator version of Graph.parents
>   rust: translation of missingancestors
>   rust: translated random test of missingancestors

Can you send these as separate patches?

>   An alternative would have been to expose to Python
>   MissingAncestors but that would have meant
>   
>   - pollution of the release build used from Python,  whereas we do it in 
> this changeset within the tests submodule
>   - waiting on rust-cpython bindings to be ready or doing the cumbersome 
> direct-ffi (more pollution with unsafe C code)

Still I want some CPython interface to measure the perf win. Are there
lots of work remaining to bring rust-cpython to us?

>   although one could argue that actually parents() should return an
>   array instead of a tuple, giving us a similar iterator for free (but on
>   references instead of values, unless we also use the arrayvec crate
>   could help). Notably, the current C-backed parents() internally uses an
>   array for communication with C code, so that currently, we'd get less memory
>   copy and less code using an array.

I prefer changing parents() to return `[Revision; 2]`. Then, we can write a
simple utility function that drops NULL_REVISION, `&[Revision; 2] -> 
&[Revision]`.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Auto-formatting code with black - object now if you have a strong opinion

[Matt Harbison]

On Wed, 05 Dec 2018 08:23:17 -0500, Yuya Nishihara  wrote:


On Tue, 4 Dec 2018 10:06:24 -0500, Augie Fackler wrote:


So, options to move forward:
1) blacken everything (controversial for good reasons)
2) try black only on a subset
3) explore yapf
4) Give up and keep manually formatting files (I'd rather not do this,  
but I understand if it's where we end up)


My vote: 3 > 4 > 2 > 1

I'm not super enthusiastic about 100%-machine-forced formatting. I like
consistency level provided by e.g. astyle command. clang-format is pretty
good IMHO, but the black seems to sacrifice the code readability.


+1.

That said, I got used to longnamesthataresmooshedtogether, so I can  
probably adjust to anything after awhile.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5370: rust: core implementation of missingancestors (no bindings)

[gracinet (Georges Racinet)]

gracinet added a comment.


  @yuja: do you mean one of those Differential Revisions of this system for 
each commit, sure I can do.
  
  With respect to rust-cpython bindings, I'm currently waiting for feedback on 
https://github.com/dgrunwald/rust-cpython/issues/164
  Perhaps you'd have an idea ?
  Short summary: sporadic segfaults that I can for now reproduce only by 
running the whole test suite.
  
  That being said, I do have rust-cpython bindings for AncestorsIterator and 
MissingAncestors (plus the full lazyancestors class). Unless I'm mistaken, that 
means that together with the existing C implementations, there's a potential 
for a full native version of `mercurial/ancestor.py`.
  I can send them to the mailing-list or here. What would be the correct flag 
for that ? RFC ?
  
  I'm also working on a perfmissingancestors (started a few hours ago). For now 
it confirms measurements I'd observed earlier: about x5 performance boost.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5370

To: gracinet, #hg-reviewers
Cc: yuja, durin42, kevincox, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D5370: rust: core implementation of missingancestors (no bindings)

[Yuya Nishihara]

>   @yuja: do you mean one of those Differential Revisions of this system for 
> each commit, sure I can do.

I don't know how Phabricator is working, but probably yes. I meant we want
D5370, D5371, and D5372 instead of a folded D5370. I think phabsend will
do that way.

>   With respect to rust-cpython bindings, I'm currently waiting for feedback 
> on https://github.com/dgrunwald/rust-cpython/issues/164
>   Perhaps you'd have an idea ?

No, but it looks interesting. I'll take a look this weekend.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel