D6844: remotefilelog: replace repack lock to solve race condition

[lothiraldan (Boris Feld)]

Closed by commit rHG5fadf6103790: remotefilelog: replace repack lock to solve 
race condition (authored by lothiraldan).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6844?vs=16522=16565

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6844/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6844

AFFECTED FILES
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/repack.py
  tests/test-remotefilelog-bgprefetch.t
  tests/test-remotefilelog-gc.t
  tests/test-remotefilelog-repack-fast.t
  tests/test-remotefilelog-repack.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-repack.t 
b/tests/test-remotefilelog-repack.t
--- a/tests/test-remotefilelog-repack.t
+++ b/tests/test-remotefilelog-repack.t
@@ -54,7 +54,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Test that the packs are readonly
@@ -63,7 +62,6 @@
   -r--r--r-- 172 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   -r--r--r--1074 b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   -r--r--r--  72 b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  -rw-r--r--   0 repacklock
 
 # Test that the data in the new packs is accessible
   $ hg cat -r . x
@@ -87,7 +85,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # First assert that with --packsonly, the loose object will be ignored:
@@ -100,7 +97,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
   $ hg repack --traceback
@@ -110,7 +106,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Verify all the file data is still available
@@ -130,7 +125,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Run two repacks at once
@@ -154,7 +148,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
   $ hg repack --background
@@ -166,7 +159,6 @@
   
$TESTTMP/hgcache/master/packs/39443fa1064182e93d968b5cba292eb5283260d0.datapack
   
$TESTTMP/hgcache/master/packs/604552d403a1381749faf656feca0ca265a6d52c.histidx
   
$TESTTMP/hgcache/master/packs/604552d403a1381749faf656feca0ca265a6d52c.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Test debug commands
@@ -243,7 +235,6 @@
   bfd60adb76018bb952e27cd23fc151bf94865d7d.histpack
   fb3aa57b22789ebcc45706c352e2d6af099c5816.dataidx
   fb3aa57b22789ebcc45706c352e2d6af099c5816.datapack
-  repacklock
   $ hg debughistorypack $TESTTMP/hgcache/master/packs/*.histidx
   
   x
diff --git a/tests/test-remotefilelog-repack-fast.t 
b/tests/test-remotefilelog-repack-fast.t
--- a/tests/test-remotefilelog-repack-fast.t
+++ b/tests/test-remotefilelog-repack-fast.t
@@ -56,7 +56,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Test that the packs are readonly
@@ -65,7 +64,6 @@
   -r--r--r-- 172 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   -r--r--r--1074 b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   -r--r--r--  72 b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  

D6844: remotefilelog: replace repack lock to solve race condition

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  2c74337e6483 
 
reduced the probability of race-conditions when starting
  background repack and prefetch and we saw the difference in our CI instance
  with all failures disappearing except one where one call to waitonrepack seems
  to returns too early.
  
  I'm not sure what exactly goes wrong but I realized that while the prefetch
  operation uses a standard Mercurial lock, the repack operation is using a
  custom lock based on `fcntl.flock` on available platforms. As `extutil.flock`
  fallback on traditional Mercurial locks on other platforms and the tests are
  stable on my laptop, our CI environment and GCC112, I'm sending this patch to
  standardize the behavior across environments.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6844

AFFECTED FILES
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/repack.py
  tests/test-remotefilelog-bgprefetch.t
  tests/test-remotefilelog-gc.t
  tests/test-remotefilelog-repack-fast.t
  tests/test-remotefilelog-repack.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-repack.t 
b/tests/test-remotefilelog-repack.t
--- a/tests/test-remotefilelog-repack.t
+++ b/tests/test-remotefilelog-repack.t
@@ -54,7 +54,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Test that the packs are readonly
@@ -63,7 +62,6 @@
   -r--r--r-- 172 1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   -r--r--r--1074 b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   -r--r--r--  72 b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  -rw-r--r--   0 repacklock
 
 # Test that the data in the new packs is accessible
   $ hg cat -r . x
@@ -87,7 +85,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # First assert that with --packsonly, the loose object will be ignored:
@@ -100,7 +97,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.dataidx
   
$TESTTMP/hgcache/master/packs/b1e0cfc7f345e408a7825e3081501959488d59ce.datapack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
   $ hg repack --traceback
@@ -110,7 +106,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Verify all the file data is still available
@@ -130,7 +125,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Run two repacks at once
@@ -154,7 +148,6 @@
   
$TESTTMP/hgcache/master/packs/78840d69389c7404327f7477e3931c89945c37d1.datapack
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histidx
   
$TESTTMP/hgcache/master/packs/8abe7889aae389337d12ebe6085d4ee13854c7c9.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
   $ hg repack --background
@@ -166,7 +159,6 @@
   
$TESTTMP/hgcache/master/packs/39443fa1064182e93d968b5cba292eb5283260d0.datapack
   
$TESTTMP/hgcache/master/packs/604552d403a1381749faf656feca0ca265a6d52c.histidx
   
$TESTTMP/hgcache/master/packs/604552d403a1381749faf656feca0ca265a6d52c.histpack
-  $TESTTMP/hgcache/master/packs/repacklock
   $TESTTMP/hgcache/repos
 
 # Test debug commands
@@ -243,7 +235,6 @@
   bfd60adb76018bb952e27cd23fc151bf94865d7d.histpack
   fb3aa57b22789ebcc45706c352e2d6af099c5816.dataidx
   fb3aa57b22789ebcc45706c352e2d6af099c5816.datapack
-  repacklock
   $ hg debughistorypack $TESTTMP/hgcache/master/packs/*.histidx
   
   x
diff --git a/tests/test-remotefilelog-repack-fast.t 
b/tests/test-remotefilelog-repack-fast.t
--- a/tests/test-remotefilelog-repack-fast.t
+++ b/tests/test-remotefilelog-repack-fast.t
@@ -56,7 +56,6 @@
   
$TESTTMP/hgcache/master/packs/1e91b207daf5d7b48f1be9c587d6b5ae654ce78c.histpack
   

D6789: check-code: allow command substitution with $(command)

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  FYI, Fish also doesn't support the `$(expr)` syntax but it's not a POSIX 
compatible shell

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6789/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6789

To: martinvonz, #hg-reviewers, indygreg
Cc: lothiraldan, durin42, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6772: remotefilelog: reduce probability of race-condition in remotefilelog tests

[lothiraldan (Boris Feld)]

Closed by commit rHG2c74337e6483: remotefilelog: reduce probability of 
race-condition in remotefilelog tests (authored by lothiraldan).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs 
Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6772?vs=16334=16368

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6772/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6772

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/repack.py
  hgext/remotefilelog/shallowrepo.py
  tests/test-remotefilelog-bgprefetch.t
  tests/test-remotefilelog-repack-fast.t
  tests/test-remotefilelog-repack.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-repack.t 
b/tests/test-remotefilelog-repack.t
--- a/tests/test-remotefilelog-repack.t
+++ b/tests/test-remotefilelog-repack.t
@@ -1,6 +1,12 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
+  $ cat >> $HGRCPATH < [devel]
+  > remotefilelog.ensurestart=True
+  > EOF
 
   $ hg init master
   $ cd master
diff --git a/tests/test-remotefilelog-repack-fast.t 
b/tests/test-remotefilelog-repack-fast.t
--- a/tests/test-remotefilelog-repack-fast.t
+++ b/tests/test-remotefilelog-repack-fast.t
@@ -1,10 +1,13 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
-
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
   $ cat >> $HGRCPATH < [remotefilelog]
   > fastdatapack=True
+  > [devel]
+  > remotefilelog.ensurestart=True
   > EOF
 
   $ hg init master
diff --git a/tests/test-remotefilelog-bgprefetch.t 
b/tests/test-remotefilelog-bgprefetch.t
--- a/tests/test-remotefilelog-bgprefetch.t
+++ b/tests/test-remotefilelog-bgprefetch.t
@@ -1,6 +1,12 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
+  $ cat >> $HGRCPATH < [devel]
+  > remotefilelog.ensurestart=True
+  > EOF
 
   $ hg init master
   $ cd master
diff --git a/hgext/remotefilelog/shallowrepo.py 
b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -183,7 +183,7 @@
 origctx=origctx)
 
 def backgroundprefetch(self, revs, base=None, repack=False, pats=None,
-   opts=None):
+   opts=None, ensurestart=False):
 """Runs prefetch in background with optional repack
 """
 cmd = [procutil.hgexecutable(), '-R', repo.origroot, 'prefetch']
@@ -193,7 +193,8 @@
 cmd += ['-r', revs]
 # We know this command will find a binary, so don't block
 # on it starting.
-procutil.runbgcommand(cmd, encoding.environ, ensurestart=False)
+procutil.runbgcommand(cmd, encoding.environ,
+  ensurestart=ensurestart)
 
 def prefetch(self, revs, base=None, pats=None, opts=None):
 """Prefetches all the necessary file revisions for the given revs
diff --git a/hgext/remotefilelog/repack.py b/hgext/remotefilelog/repack.py
--- a/hgext/remotefilelog/repack.py
+++ b/hgext/remotefilelog/repack.py
@@ -34,7 +34,8 @@
 class RepackAlreadyRunning(error.Abort):
 pass
 
-def backgroundrepack(repo, incremental=True, packsonly=False):
+def backgroundrepack(repo, incremental=True, packsonly=False,
+ ensurestart=False):
 cmd = [procutil.hgexecutable(), '-R', repo.origroot, 'repack']
 msg = _("(running background repack)\n")
 if incremental:
@@ -44,7 +45,7 @@
 cmd.append('--packsonly')
 repo.ui.warn(msg)
 # We know this command will find a binary, so don't block on it starting.
-procutil.runbgcommand(cmd, encoding.environ, ensurestart=False)
+procutil.runbgcommand(cmd, encoding.environ, ensurestart=ensurestart)
 
 def fullrepack(repo, options=None):
 """If ``packsonly`` is True, stores creating only loose objects are 
skipped.
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -224,6 +224,8 @@
 configitem('packs', 'maxpacksize', default=0)
 configitem('packs', 'maxchainlen', default=1000)
 
+configitem('devel', 'remotefilelog.ensurestart', default=False)
+
 #  default TTL limit is 30 days
 _defaultlimit = 60 * 60 * 24 * 30
 configitem('remotefilelog', 'nodettl', default=_defaultlimit)
@@ -949,19 +951,23 @@
 prefetchrevset = ui.config('remotefilelog', 'pullprefetch')
 bgrepack = repo.ui.configbool('remotefilelog', 'backgroundrepack')
 bgprefetch = repo.ui.configbool('remotefilelog', 'backgroundprefetch')
+

D6772: remotefilelog: reduce probability of race-condition in remotefilelog tests

[lothiraldan (Boris Feld)]

lothiraldan added a comment.
lothiraldan added a subscriber: durin42.


  @durin42 Could you take a look? I think you are in the best condition to 
review this patch.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6772/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6772

To: lothiraldan, #hg-reviewers
Cc: durin42, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6772: remotefilelog: reduce probability of race-condition in remotefilelog tests

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  ca1014ad3de4 
 
introduced a new parameter `ensurestart` to speed up
  remotefilelog background processes start. Unfortunately it seems to have
  increased the possibility of race-conditions in remotefilelog tests testing
  those background processes.
  
  With `ensurestart=False`, it seems that it's more probable to enter in a race
  condition with `debugwaitonprefetch` and `debugwaitonrepack` in remotefilelog
  background tests. Our CI seems to have a high probability of triggering this
  race condition so make it configurable to ensure tests are stable.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6772

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/repack.py
  hgext/remotefilelog/shallowrepo.py
  tests/test-remotefilelog-bgprefetch.t
  tests/test-remotefilelog-repack-fast.t
  tests/test-remotefilelog-repack.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-repack.t 
b/tests/test-remotefilelog-repack.t
--- a/tests/test-remotefilelog-repack.t
+++ b/tests/test-remotefilelog-repack.t
@@ -1,6 +1,12 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
+  $ cat >> $HGRCPATH < [devel]
+  > remotefilelog.ensurestart=True
+  > EOF
 
   $ hg init master
   $ cd master
diff --git a/tests/test-remotefilelog-repack-fast.t 
b/tests/test-remotefilelog-repack-fast.t
--- a/tests/test-remotefilelog-repack-fast.t
+++ b/tests/test-remotefilelog-repack-fast.t
@@ -1,10 +1,13 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
-
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
   $ cat >> $HGRCPATH < [remotefilelog]
   > fastdatapack=True
+  > [devel]
+  > remotefilelog.ensurestart=True
   > EOF
 
   $ hg init master
diff --git a/tests/test-remotefilelog-bgprefetch.t 
b/tests/test-remotefilelog-bgprefetch.t
--- a/tests/test-remotefilelog-bgprefetch.t
+++ b/tests/test-remotefilelog-bgprefetch.t
@@ -1,6 +1,12 @@
 #require no-windows
 
   $ . "$TESTDIR/remotefilelog-library.sh"
+# devel.remotefilelog.ensurestart: reduce race condition with
+# waiton{repack/prefetch}
+  $ cat >> $HGRCPATH < [devel]
+  > remotefilelog.ensurestart=True
+  > EOF
 
   $ hg init master
   $ cd master
diff --git a/hgext/remotefilelog/shallowrepo.py 
b/hgext/remotefilelog/shallowrepo.py
--- a/hgext/remotefilelog/shallowrepo.py
+++ b/hgext/remotefilelog/shallowrepo.py
@@ -183,7 +183,7 @@
 origctx=origctx)
 
 def backgroundprefetch(self, revs, base=None, repack=False, pats=None,
-   opts=None):
+   opts=None, ensurestart=False):
 """Runs prefetch in background with optional repack
 """
 cmd = [procutil.hgexecutable(), '-R', repo.origroot, 'prefetch']
@@ -193,7 +193,8 @@
 cmd += ['-r', revs]
 # We know this command will find a binary, so don't block
 # on it starting.
-procutil.runbgcommand(cmd, encoding.environ, ensurestart=False)
+procutil.runbgcommand(cmd, encoding.environ,
+  ensurestart=ensurestart)
 
 def prefetch(self, revs, base=None, pats=None, opts=None):
 """Prefetches all the necessary file revisions for the given revs
diff --git a/hgext/remotefilelog/repack.py b/hgext/remotefilelog/repack.py
--- a/hgext/remotefilelog/repack.py
+++ b/hgext/remotefilelog/repack.py
@@ -34,7 +34,8 @@
 class RepackAlreadyRunning(error.Abort):
 pass
 
-def backgroundrepack(repo, incremental=True, packsonly=False):
+def backgroundrepack(repo, incremental=True, packsonly=False,
+ ensurestart=False):
 cmd = [procutil.hgexecutable(), '-R', repo.origroot, 'repack']
 msg = _("(running background repack)\n")
 if incremental:
@@ -44,7 +45,7 @@
 cmd.append('--packsonly')
 repo.ui.warn(msg)
 # We know this command will find a binary, so don't block on it starting.
-procutil.runbgcommand(cmd, encoding.environ, ensurestart=False)
+procutil.runbgcommand(cmd, encoding.environ, ensurestart=ensurestart)
 
 def fullrepack(repo, options=None):
 """If ``packsonly`` is True, stores creating only loose objects are 
skipped.
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -224,6 +224,8 @@
 configitem('packs', 'maxpacksize', default=0)
 configitem('packs', 'maxchainlen', default=1000)
 
+configitem('devel', 'remotefilelog.ensurestart', default=False)
+
 #  default TTL limit is 30 days
 _defaultlimit = 

D6122: tests: add a file listing all the flaky tests

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  In D6122#89861 , @lothiraldan 
wrote:
  
  > In D6122#89819 , @av6 wrote:
  >
  >> In D6122#89818 , @lothiraldan 
wrote:
  >>
  >>> @av6 is a `--no-flaky-tests` flag what you had in mind?
  >>
  >> Something like that, yes. I suggested that flag seeing how this change 
introduces a quite specific file with just one line. And I thought "if it's 
important, let's integrate it deeper into the test runner".
  >> But a flag for run-tests.py is not the only alternative. Now that I think 
about `#require`, can ​test-remotefilelog-bgprefetch.t be marked with `#require 
not-an-exotic-fs-inside-docker` (or whatever the actual thing that causes 
flakiness if it's known)? hghave.py already checks various filesystem features, 
why not make it check for the cause of issue6083?
  >
  > For this specific test, we are not sure that it's the exact issue, it's our 
best lead. I'm afraid that identifying the underlying cause will be 90% of the 
work fixing it.
  > My goal is to allow people running tests outside our buildbot and gcc build 
farm to being able to quickly identify which test error are expected or not. 
And for CI setups, we want the flaky test to not impact the return code as it 
would makes the build fails in most of CI setups.
  > Identifying the root cause of the issue is a problem that takes time and 
expertise, the docker exotic fs lead is actually from @durin42, I would have 
not guessed it. I would like to find a way for people to report that a test is 
flaky on their setup without requiring them to dig too deep in the issue.
  > I would be ok with the `--no-flaky-tests` flag, even if I found the 
`#require flaky` a bit confusing.
  
  @av6 What is your final thoughts on this patch?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D6122/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D6122

To: lothiraldan, #hg-reviewers
Cc: durin42, av6, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] root: add template variables pointing to repository directories

[Boris FELD]

Looks good to me, queue thanks

On 05/06/2019 01:30, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1559650415 -32400
> #  Tue Jun 04 21:13:35 2019 +0900
> # Node ID 671a49973d8b12fe223b6f81022f02061c4ddb01
> # Parent  7182320ee57475303eb677a41dde142b7f87fc9b
> root: add template variables pointing to repository directories
>
> These paths are useful for GUI applications to detect changes. A GUI process
> typically monitors .hg and .hg/store directories so that it will be notified
> on lock/wlock deletion.
>
> Alternatively, maybe we can add debugpaths command if we don't want to extend
> the root command. I'm not sure which will be nicer.
>
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -5232,12 +5232,23 @@ def root(ui, repo, **opts):
>  
>  Print the root directory of the current repository.
>  
> +.. container:: verbose
> +
> +  Template:
> +
> +  The following keywords are supported in addition to the common template
> +  keywords and functions. See also :hg:`help templates`.
> +
> +  :hgpath:String. Path to the .hg directory.
> +  :storepath: String. Path to the directory holding versioned data.
> +
>  Returns 0 on success.
>  """
>  opts = pycompat.byteskwargs(opts)
>  with ui.formatter('root', opts) as fm:
>  fm.startitem()
>  fm.write('reporoot', '%s\n', repo.root)
> +fm.data(hgpath=repo.path, storepath=repo.spath)
>  
>  @command('serve',
>  [('A', 'accesslog', '', _('name of access log file to write to'),
> diff --git a/tests/test-basic.t b/tests/test-basic.t
> --- a/tests/test-basic.t
> +++ b/tests/test-basic.t
> @@ -101,7 +101,9 @@ Repository root:
>$ hg root -Tjson | sed 's||\\|g'
>[
> {
> -"reporoot": "$TESTTMP/t"
> +"hgpath": "$TESTTMP/t/.hg",
> +"reporoot": "$TESTTMP/t",
> +"storepath": "$TESTTMP/t/.hg/store"
> }
>]
>  
> diff --git a/tests/test-clone.t b/tests/test-clone.t
> --- a/tests/test-clone.t
> +++ b/tests/test-clone.t
> @@ -719,6 +719,14 @@ Test clone from the repository in (emula
>$ hg -R src debugrevlog -c | egrep 'format|flags'
>format : 0
>flags  : (none)
> +  $ hg root -R src -T json | sed 's||\\|g'
> +  [
> +   {
> +"hgpath": "$TESTTMP/src/.hg",
> +"reporoot": "$TESTTMP/src",
> +"storepath": "$TESTTMP/src/.hg"
> +   }
> +  ]
>$ hg clone -U -q src dst
>$ hg -R dst log -q
>0:e1bab28bca43
> diff --git a/tests/test-share.t b/tests/test-share.t
> --- a/tests/test-share.t
> +++ b/tests/test-share.t
> @@ -21,6 +21,14 @@ share shouldn't have a store dir
>$ cd repo2
>$ test -d .hg/store
>[1]
> +  $ hg root -Tjson | sed 's||\\|g'
> +  [
> +   {
> +"hgpath": "$TESTTMP/repo2/.hg",
> +"reporoot": "$TESTTMP/repo2",
> +"storepath": "$TESTTMP/repo1/.hg/store"
> +   }
> +  ]
>  
>  share shouldn't have a full cache dir, original repo should
>  
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6122: tests: add a file listing all the flaky tests

[lothiraldan (Boris Feld)]

lothiraldan added a subscriber: durin42.
lothiraldan added a comment.


  In https://phab.mercurial-scm.org/D6122#89819, @av6 wrote:
  
  > In https://phab.mercurial-scm.org/D6122#89818, @lothiraldan wrote:
  >
  > > @av6 is a `--no-flaky-tests` flag what you had in mind?
  >
  >
  > Something like that, yes. I suggested that flag seeing how this change 
introduces a quite specific file with just one line. And I thought "if it's 
important, let's integrate it deeper into the test runner".
  >
  > But a flag for run-tests.py is not the only alternative. Now that I think 
about `#require`, can ​test-remotefilelog-bgprefetch.t be marked with `#require 
not-an-exotic-fs-inside-docker` (or whatever the actual thing that causes 
flakiness if it's known)? hghave.py already checks various filesystem features, 
why not make it check for the cause of issue6083?
  
  
  For this specific test, we are not sure that it's the exact issue, it's our 
best lead. I'm afraid that identifying the underlying cause will be 90% of the 
work fixing it.
  
  My goal is to allow people running tests outside our buildbot and gcc build 
farm to being able to quickly identify which test error are expected or not. 
And for CI setups, we want the flaky test to not impact the return code as it 
would makes the build fails in most of CI setups.
  
  Identifying the root cause of the issue is a problem that takes time and 
expertise, the docker exotic fs lead is actually from @durin42, I would have 
not guessed it. I would like to find a way for people to report that a test is 
flaky on their setup without requiring them to dig too deep in the issue.
  
  I would be ok with the `--no-flaky-tests` flag, even if I found the `#require 
flaky` a bit confusing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6122

To: lothiraldan, #hg-reviewers
Cc: durin42, av6, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6122: tests: add a file listing all the flaky tests

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  The big difference between `#require slow` and `#require flaky` is that we 
don't want to run the slow tests by default but we want to run the flaky test 
by default if not specified otherwise. I don't know enough about the test 
runner to evaluate the amount of work need to support a `--no-flaky-tests` flag,
  
  @av6 is a `--no-flaky-tests` flag what you had in mind?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6122

To: lothiraldan, #hg-reviewers
Cc: av6, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

European mini-sprint in April

[Boris FELD]

Hello everybody,

Early April, we will have a small Mercurial gathering in Paris. The
initial motivation is to work on tooling, workflow, and documentation
needed for modern Mercurial hosting and collaboration.

We have been trying to schedule it a couple of months and we ended
selecting from Thursday April 4th to Sunday April 7th. Logilab will
nicely host us for 4-5th, the Saturday hosting is being finalized,
Sunday will be at Mozilla Paris.

Logilab made an announcement about the mini-sprint:
https://www.logilab.org/blogentry/10131360. Please register in the
survey (https://framadate.org/Gmwmc5mnAXBaJNQx) in order to anticipate
the number of people.

Since the 5.0 sprint has been canceled, we will probably cover a bit
more topics than initially planned. We have a dedicated pad for
discussing the sprint topics and taking some notes:
https://mensuel.framapad.org/p/mini-sprint-hg.

Feel welcome to join us!

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Missed 4.9.1 release - should we do it now? (was: Re: Missing march update?)

[Boris FELD]

On 18/03/2019 16:01, Boris FELD wrote:
> There is an open issue with subrepository support in 4.9 on Windows:
> 87a6e3c953e045d92147925fc71aad7c327fdbfd

I wanted to paste this link instead:
https://stackoverflow.com/questions/54868056/can-a-mercurial-subrepostiory-reside-in-a-subfolder/54940807#54940807

Sorry for the noise

>
> So I would vote for a 4.9.1 release
>
> On 18/03/2019 14:35, Augie Fackler wrote:
>> (mercurial@ to bcc, mercurial-devel@ to cc)
>>
>>> On Mar 16, 2019, at 22:10, Jesus Cea  wrote:
>>>
>>> Mercurial 4.9.1 is late. Any issue?
>> Got overlooked because I was on vacation and the other people that can roll 
>> releases were similarly absent.
>>
>> I guess we'll make a judgement call today and tomorrow if we should bother 
>> with .1 now, or just hold it to April.
>>
>> Denizens of -devel, what do you think?
>>
>>> Thanks for your work!.
>>>
>>> -- 
>>> Jesús Cea Avión _/_/  _/_/_/_/_/_/
>>> j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
>>> Twitter: @jcea_/_/_/_/  _/_/_/_/_/
>>> jabber / xmpp:j...@jabber.org  _/_/  _/_/_/_/  _/_/  _/_/
>>> "Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
>>> "My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
>>> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
>>>
>>> ___
>>> Mercurial mailing list
>>> mercur...@mercurial-scm.org
>>> https://www.mercurial-scm.org/mailman/listinfo/mercurial
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Missed 4.9.1 release - should we do it now? (was: Re: Missing march update?)

[Boris FELD]

There is an open issue with subrepository support in 4.9 on Windows:
87a6e3c953e045d92147925fc71aad7c327fdbfd

So I would vote for a 4.9.1 release

On 18/03/2019 14:35, Augie Fackler wrote:
> (mercurial@ to bcc, mercurial-devel@ to cc)
>
>> On Mar 16, 2019, at 22:10, Jesus Cea  wrote:
>>
>> Mercurial 4.9.1 is late. Any issue?
> Got overlooked because I was on vacation and the other people that can roll 
> releases were similarly absent.
>
> I guess we'll make a judgement call today and tomorrow if we should bother 
> with .1 now, or just hold it to April.
>
> Denizens of -devel, what do you think?
>
>> Thanks for your work!.
>>
>> -- 
>> Jesús Cea Avión _/_/  _/_/_/_/_/_/
>> j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
>> Twitter: @jcea_/_/_/_/  _/_/_/_/_/
>> jabber / xmpp:j...@jabber.org  _/_/  _/_/_/_/  _/_/  _/_/
>> "Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
>> "My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
>> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
>>
>> ___
>> Mercurial mailing list
>> mercur...@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6147: discovery: move cl.hasnode outside of the for-loop

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  The series looks good to me after a very quick review. I think a more in-deep 
review will be necessary.
  
  @pulkit Do you have measured some performance improvement with this series?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6147

To: pulkit, #hg-reviewers
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5955: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG76361969662a: watchman: ignore some of watchman errors 
(authored by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5955?vs=14077=14528

REVISION DETAIL
  https://phab.mercurial-scm.org/D5955

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -180,7 +180,8 @@
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: fsmonitor.verbose
 if ex.warn and ui.configbool('fsmonitor', 'verbose'):
-ui.warn(str(ex) + '\n')
+if 'illegal_fstypes' not in str(ex):
+ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
 # experimental config: fsmonitor.verbose



To: lothiraldan, #hg-reviewers, pulkit
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5954: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG57264906a996: watchman: add the possibility to set the 
exact watchman binary location (authored by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5954?vs=14076=14527

REVISION DETAIL
  https://phab.mercurial-scm.org/D5954

AFFECTED FILES
  hgext/fsmonitor/__init__.py
  hgext/fsmonitor/pywatchman/__init__.py
  hgext/fsmonitor/watchmanclient.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/watchmanclient.py 
b/hgext/fsmonitor/watchmanclient.py
--- a/hgext/fsmonitor/watchmanclient.py
+++ b/hgext/fsmonitor/watchmanclient.py
@@ -82,9 +82,11 @@
 try:
 if self._watchmanclient is None:
 self._firsttime = False
+watchman_exe = self._ui.configpath('fsmonitor', 'watchman_exe')
 self._watchmanclient = pywatchman.client(
 timeout=self._timeout,
-useImmutableBser=True)
+useImmutableBser=True,
+watchman_exe=watchman_exe)
 return self._watchmanclient.query(*watchmanargs)
 except pywatchman.CommandError as ex:
 if 'unable to resolve root' in ex.msg:
diff --git a/hgext/fsmonitor/pywatchman/__init__.py 
b/hgext/fsmonitor/pywatchman/__init__.py
--- a/hgext/fsmonitor/pywatchman/__init__.py
+++ b/hgext/fsmonitor/pywatchman/__init__.py
@@ -317,7 +317,7 @@
 """ local unix domain socket transport """
 sock = None
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
 
@@ -397,7 +397,7 @@
 class WindowsNamedPipeTransport(Transport):
 """ connect to a named pipe """
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = int(math.ceil(timeout * 1000))
 self._iobuf = None
@@ -563,9 +563,10 @@
 proc = None
 closed = True
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
+self.watchman_exe = watchman_exe
 
 def close(self):
 if self.proc:
@@ -579,7 +580,7 @@
 if self.proc:
 return self.proc
 args = [
-'watchman',
+self.watchman_exe,
 '--sockname={0}'.format(self.sockpath),
 '--logfile=/BOGUS',
 '--statefile=/BOGUS',
@@ -756,17 +757,20 @@
 unilateral = ['log', 'subscription']
 tport = None
 useImmutableBser = None
+watchman_exe = None
 
 def __init__(self,
  sockpath=None,
  timeout=1.0,
  transport=None,
  sendEncoding=None,
  recvEncoding=None,
- useImmutableBser=False):
+ useImmutableBser=False,
+ watchman_exe=None):
 self.sockpath = sockpath
 self.timeout = timeout
 self.useImmutableBser = useImmutableBser
+self.watchman_exe = watchman_exe
 
 if inspect.isclass(transport) and issubclass(transport, Transport):
 self.transport = transport
@@ -817,7 +821,7 @@
 if path:
 return path
 
-cmd = ['watchman', '--output-encoding=bser', 'get-sockname']
+cmd = [self.watchman_exe, '--output-encoding=bser', 'get-sockname']
 try:
 args = dict(stdout=subprocess.PIPE,
 stderr=subprocess.PIPE,
@@ -858,7 +862,7 @@
 if self.sockpath is None:
 self.sockpath = self._resolvesockname()
 
-self.tport = self.transport(self.sockpath, self.timeout)
+self.tport = self.transport(self.sockpath, self.timeout, 
self.watchman_exe)
 self.sendConn = self.sendCodec(self.tport)
 self.recvConn = self.recvCodec(self.tport)
 
diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('fsmonitor', 'watchman_exe',
+default='watchman',
+)
 configitem('fsmonitor', 'verbose',
 default=True,
 )



To: lothiraldan, #hg-reviewers, pulkit
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6122: tests: add a file listing all the flaky tests

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  In https://phab.mercurial-scm.org/D6122#89169, @av6 wrote:
  
  > Another way to do this could be to introduce another special keyword, 
similar to "slow", so that tests that are known flaky can `#require flaky` or 
even mark a specific block with `#if flaky`. Obviously fixing 
​test-remotefilelog-bgprefetch.t is a better idea altogether, but I'm not going 
to claim that it's easy.
  
  
  For this specific test case, it's not flaky on our Buildbot instance 
(https://buildbot.mercurial-scm.org/builders/hg%20tests). It only happens on 
exotic filesystems (like inside a Docker container) and when running the tests 
in parallel.
  
  Flaky tests on our buildbot instances are usually fixed quite quickly, this 
contrib file is more for tests that are flaky outside our buildbot setup.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6122

To: lothiraldan, #hg-reviewers
Cc: av6, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6122: tests: add a file listing all the flaky tests

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This file can be used by CI instances to skip tests known to be flaky and
  focus on tests with no false negatives.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6122

AFFECTED FILES
  contrib/flaky-tests

CHANGE DETAILS

diff --git a/contrib/flaky-tests b/contrib/flaky-tests
new file mode 100644
--- /dev/null
+++ b/contrib/flaky-tests
@@ -0,0 +1 @@
+test-remotefilelog-bgprefetch.t # 
https://bz.mercurial-scm.org/show_bug.cgi?id=6083
\ No newline at end of file



To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6028: branchmap: prevent reading the file twice through different iterators

[lothiraldan (Boris Feld)]

lothiraldan added subscribers: yuja, lothiraldan.
lothiraldan accepted this revision.
lothiraldan added a comment.


  Queued thanks, the test now pass both locally and on gcc112. @yuja nice catch!

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6028

To: pulkit, #hg-reviewers, lothiraldan
Cc: lothiraldan, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5955: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  Gentle ping on this series

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5955

To: lothiraldan, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6023: branchcache: move loading of branch names and nodes into it's own function

[lothiraldan (Boris Feld)]

lothiraldan accepted this revision.
lothiraldan added a comment.


  LGTM, queued thanks

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6023

To: pulkit, #hg-reviewers, lothiraldan
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH FOLLOW-UP] test: follow-up on 1c4d6ab2ecb8, stabilize test-remotefilelog-bgprefetch.t

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1551081749 -3600
#  lun. févr. 25 09:02:29 2019 +0100
# Node ID f611bf911c863d35aced20dfb2546725c911a722
# Parent  1c1c4ef8b72e39f5ed1c62dc4e31e02e9e08b652
# EXP-Topic stabilize-test-remotefilelog-bgprefetch.t
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
f611bf911c86
test: follow-up on 1c4d6ab2ecb8, stabilize test-remotefilelog-bgprefetch.t

In 1c4d6ab2ecb8, I've added some sleeps after about half of the calls to `hg
debugwaitonrepack` but it seems the other calls are also flaky as catched by
our CI: https://ci.octobus.net/job/MercurialPy2/299/console. Add sleeps after
all calls in order to be safe.

diff --git a/tests/test-remotefilelog-bgprefetch.t 
b/tests/test-remotefilelog-bgprefetch.t
--- a/tests/test-remotefilelog-bgprefetch.t
+++ b/tests/test-remotefilelog-bgprefetch.t
@@ -142,6 +142,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 1
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 1
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
@@ -287,6 +288,7 @@
 
   $ sleep 0.5
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 0.5
 
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
@@ -331,6 +333,7 @@
   * files fetched over 1 fetches - (* misses, 0.00% hit ratio) over *s (glob) 
(?)
   $ sleep 0.5
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 0.5
 
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2 RESENT] rewriting: add an option for rewrite commands to use the archived phase

[Boris FELD]

What can we do to make this series go forward?

On 13/02/2019 15:38, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1546394872 -3600
> #  Wed Jan 02 03:07:52 2019 +0100
> # Node ID 9939d8e412e3e440f3b564fb96c187745d7a008c
> # Parent  61ec4a834e2c88056ff47c0d3a7ff3bcb0f0d912
> # EXP-Topic archived-phase-UX
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 9939d8e412e3
> rewriting: add an option for rewrite commands to use the archived phase
>
> Using the archived phase for cleanup provide the same effect than stripping,
> but in a faster, append-only way.
>
> We keep the feature experimental for now until it gets a bit more testing.
>
> diff --git a/mercurial/configitems.py b/mercurial/configitems.py
> --- a/mercurial/configitems.py
> +++ b/mercurial/configitems.py
> @@ -470,6 +470,9 @@ coreconfigitem('experimental', 'bundleco
>  coreconfigitem('experimental', 'changegroup3',
>  default=False,
>  )
> +coreconfigitem('experimental', 'cleanup-as-archived',
> +default=False,
> +)
>  coreconfigitem('experimental', 'clientcompressionengines',
>  default=list,
>  )
> diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
> --- a/mercurial/scmutil.py
> +++ b/mercurial/scmutil.py
> @@ -1014,6 +1014,7 @@ def cleanupnodes(repo, replacements, ope
>  for phase, nodes in toadvance.items():
>  phases.advanceboundary(repo, tr, phase, nodes)
>  
> +mayusearchived = repo.ui.config('experimental', 
> 'cleanup-as-archived')
>  # Obsolete or strip nodes
>  if obsolete.isenabled(repo, obsolete.createmarkersopt):
>  # If a node is already obsoleted, and we want to obsolete it
> @@ -1031,6 +1032,17 @@ def cleanupnodes(repo, replacements, ope
>  if rels:
>  obsolete.createmarkers(repo, rels, operation=operation,
> metadata=metadata)
> +elif phases.supportinternal(repo) and mayusearchived:
> +# this assume we do not have "unstable" nodes above the cleaned 
> ones
> +allreplaced = set()
> +for ns in replacements.keys():
> +allreplaced.update(ns)
> +if backup:
> +from . import repair # avoid import cycle
> +node = min(allreplaced, key=repo.changelog.rev)
> +repair.backupbundle(repo, allreplaced, allreplaced, node,
> +operation)
> +phases.retractboundary(repo, tr, phases.archived, allreplaced)
>  else:
>  from . import repair # avoid import cycle
>  tostrip = list(n for ns in replacements for n in ns)
> diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
> --- a/tests/test-phase-archived.t
> +++ b/tests/test-phase-archived.t
> @@ -75,3 +75,69 @@ Test that bundle can unarchive a changes
>   date:Thu Jan 01 00:00:00 1970 +
>   summary: root
>
> +
> +Test that history rewriting command can use the archived phase when allowed 
> to
> +--
> +
> +  $ hg up 'desc(unbundletesting)'
> +  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
> +  $ echo bar >> a
> +  $ hg commit --amend --config experimental.cleanup-as-archived=yes
> +  $ hg log -G
> +  @  changeset:   2:d1e73e428f29
> +  |  tag: tip
> +  |  parent:  0:c1863a3840c6
> +  |  user:test
> +  |  date:Thu Jan 01 00:00:00 1970 +
> +  |  summary: unbundletesting
> +  |
> +  o  changeset:   0:c1863a3840c6
> + user:test
> + date:Thu Jan 01 00:00:00 1970 +
> + summary: root
> +  
> +  $ hg log -G --hidden
> +  @  changeset:   2:d1e73e428f29
> +  |  tag: tip
> +  |  parent:  0:c1863a3840c6
> +  |  user:test
> +  |  date:Thu Jan 01 00:00:00 1970 +
> +  |  summary: unbundletesting
> +  |
> +  | o  changeset:   1:883aadbbf309
> +  |/   user:test
> +  |date:Thu Jan 01 00:00:00 1970 +
> +  |summary: unbundletesting
> +  |
> +  o  changeset:   0:c1863a3840c6
> + user:test
> + date:Thu Jan 01 00:00:00 1970 +
> + summary: root
> +  
> +  $ ls -1 .hg/strip-backup/
> +  883aadbbf309-efc55adc-amend.hg
> +  883aadbbf309-efc55adc-backup.hg
> +  $ hg unbundle .hg/strip-backup/883aadbbf309*amend.hg
> +  adding changesets
> +  adding manifests
> +  adding file changes
> +  added 0 change

D5966: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG4618bdf75d8a: test: stabilize test-run-tests.t output 
(authored by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5966?vs=14095=14146

REVISION DETAIL
  https://phab.mercurial-scm.org/D5966

AFFECTED FILES
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -1174,31 +1174,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }
   } (no-eol)
 --json with --outputdir
@@ -1231,31 +1231,31 @@
   $ cat output/report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }
   } (no-eol)
   $ ls -a output
@@ -1287,31 +1287,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+

D5960: test: stabilize test-remotefilelog-bgprefetch.t flaky output

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5b6ee1b16f9a: test: stabilize 
test-remotefilelog-bgprefetch.t flaky output (authored by lothiraldan, 
committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5960?vs=14089=14145

REVISION DETAIL
  https://phab.mercurial-scm.org/D5960

AFFECTED FILES
  tests/test-remotefilelog-bgprefetch.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-bgprefetch.t 
b/tests/test-remotefilelog-bgprefetch.t
--- a/tests/test-remotefilelog-bgprefetch.t
+++ b/tests/test-remotefilelog-bgprefetch.t
@@ -105,6 +105,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 0.5
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 0.5
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histidx
   
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histpack
@@ -193,6 +194,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 1
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 1
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
@@ -243,6 +245,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 1
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 1
 
 # Ensure that file 'y' was prefetched - it was not part of the rebase 
operation and therefore
 # could only be downloaded by the background prefetch



To: lothiraldan, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5955: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan added a subscriber: indygreg.
lothiraldan added a comment.


  @indygreg I've update the patchs with your reviews.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5955

To: lothiraldan, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Auto-formatting code with black - object now if you have a strong opinion

[Boris FELD]

On 19/02/2019 16:28, Augie Fackler wrote:
> On Wed, Feb 13, 2019 at 10:33:15AM +0100, Boris FELD wrote:
>> On 01/12/2018 02:35, Matt Harbison wrote:
>>> On Fri, 30 Nov 2018 07:25:04 -0500, Boris FELD
>>>  wrote:
>>>
>>>> I think using automatic formatting is a great idea and we should move
>>>> forward with this plan. Black seems a good option. I share other's
>>>> concerns about the formatting of import. I also wonder if this also
>>>> applies to list and dict formatting that we tend to express with one
>>>> value per line for clarity.
>>> It looks like yes, unfortunately, if it fits on one line:
>>>
>>> diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
>>> --- a/hgext/lfs/blobstore.py
>>> +++ b/hgext/lfs/blobstore.py
>>> @@ -289,50 +289,47 @@ class _gitlfsremote(object):
>>>  Return decoded JSON object like {'objects': [{'oid': '',
>>> 'size': 1}]}
>>>  See
>>> https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
>>>  """
>>> -    objects = [{'oid': p.oid(), 'size': p.size()} for p in pointers]
>>> -    requestdata = json.dumps({
>>> -    'objects': objects,
>>> -    'operation': action,
>>> -    })
>>> -    url = '%s/objects/batch' % self.baseurl
>>> +    objects = [{"oid": p.oid(), "size": p.size()} for p in pointers]
>>> +    requestdata = json.dumps({"objects": objects, "operation":
>>> action})
>>> +    url = "%s/objects/batch" % self.baseurl
>>>  batchreq = util.urlreq.request(url, data=requestdata)
>>> ...
>> We have been discussing with the Black author about how we could handle
>> those cases and we found a `hack` which is adding an empty comment on
>> the first line of a list, dict, multi-line construction:
> If you're talking to Łukasz, could you float the way clang-format
> works by him? That is, if a trailing comma is present in the literal
> it's formatted one element per line, but if there's no trailing comma
> it's formatted compactly. I've found that to be a good tradeoff. It'd
> also make our import blocks format correctly without modification.
He is quite aware of that option and seems to have decided against it
for good. He pointed us toward the # hack instead saying it works but
finding it inelegant. (So we seems to be in a case of "Ĺukasz doesn't
want people to use this hack, but he seems to be even less willing to
offer a cleaner alternative")
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5966: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

lothiraldan added inline comments.

INLINE COMMENTS

> durin42 wrote in test-run-tests.t:1177
> I don't feel strongly, let's just make it permissive so the test passes on 
> both fast and slow systems.

I think it's the case, the first part of the regex `\d+` will match any amount 
of seconds and the remaining part `\.\d{3,4}` should match as long the 
precision doesn't change.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5966

To: lothiraldan, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5960: test: stabilize test-remotefilelog-bgprefetch.t flaky output

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  In https://phab.mercurial-scm.org/D5960#87357, @durin42 wrote:
  
  > In https://phab.mercurial-scm.org/D5960#87330, @lothiraldan wrote:
  >
  > > In https://phab.mercurial-scm.org/D5960#87251, @durin42 wrote:
  > >
  > > > Huh. That's strictly a bug, as `debugwaitonrepack` should actually be 
waiting.
  > >
  > >
  > > I've created a ticket to track it: https://phab.mercurial-scm.org/D5960
  > >
  > > Do you have any idea what could be happening?
  >
  >
  > I've never seen this failure, so I'm suspicious of your test environment. 
Is there anything exotic, perhaps at the filesystem level?
  
  
  I'm using BTRFS localy, I reproduced inside a mounted Docker container 
locally. On the CI side, the slave seems to run with an OverlayFS disk.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5960

To: lothiraldan, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC] cmdtest: add a new command that checks revset matching

[Boris FELD]

On 19/02/2019 13:52, Pulkit Goyal wrote:
>
>
> On Fri, Feb 15, 2019 at 6:11 PM Raphaël Gomès
> mailto:raphael.go...@octobus.net>> wrote:
>
> # HG changeset patch
> # User Raphaël Gomès mailto:rgo...@octobus.net>>
> # Date 1550068454 -3600
> #      Wed Feb 13 15:34:14 2019 +0100
> # Node ID f850e5c8547df95cd4bcb38b4ade5b5af955
> # Parent  91701785c2c56a3ee395038488758ad2e1865265
> # EXP-Topic cmd-test
> cmdtest: add a new command that checks revset matching
>
> This command is meant to ease scripting that need to check revsets
> validity
> on a repository. Right now, doing such revset testing requires
> abuse of `hg log`
> output parsing, and is less than ideal.
>
> It mirrors the `test` shell builtin which is used in almost every
> shell script
> for branching.
>
> This is a first draft of this command, there is of room for
> adjustement and evolution,
> like fileset matching or other arguments.
>
>
> You can use the `expectsize()` revset which is recently implemented
> and pass the size as 0. So if a revset is empty, the operation will
> succeed and you will get 0 as return value. If there are revs in the
> revset, it will abort.
The expectsize revset is less flexible, and provide an ambiguous return
code when it does not match (same return as an invalid revset). The
revset seems useful in itself, but having a clear and extensible command
will make scripting simpler, clearer and less error-prone.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: caching pull - stable partitioning of bundle requests

[Boris FELD]

On 15/02/2019 21:16, Pulkit Goyal wrote:
>
>
> On Thu, Oct 4, 2018 at 6:16 PM Boris FELD  <mailto:lothiral...@gmail.com>> wrote:
>
> The road for moving this in Core is clear, but not short. So far
> we have not been able to free the necessary time to do it. Between
> the paying-client work, we have to do to pay salaries (and keep
> users happy) and all the time we are already investing in the
> community, we are already fairly busy.
>
>
> In early 2017, Bitbucket gave $13, 500 to the Mercurial project to
> be spent to help evolution to move forward. As far as we know,
> this money is still unspent. Since stable range is a critical part
> of obsmarkers discovery, unlocking this money to be spent on
> upstreaming stable range would be a good idea (and fits its
> initial purposes). Paying for this kind of work will reduce the
> contention with client work and help us, or others, to dedicate
> time for it sooner than later.
>
>
> I definitely agree that obsmarker discovery is a critical part.
> Pulling from `hg-committed` is slower sometimes as compared to pulling
> on a repo (5-7x size of hg-committed) with server having thousands of
> heads.
>
> Do you have any updates on the stable-range cache? In the current
> state the cache is pretty big and a lot of people faced problem with
> the cache size.  Also in case of strip or some other commands, it
> rebuilts the cache which takes more than 10 minutes on large repo
> which is definitely a bummer. Are you working on making that better
> and take less size? How is experimentation of evolve+stablerange going on?

# Regarding the cache-size:

We know that the current version caches many entries that are trivial to
compute and does not need to be cached. In addition, the current storage
(SQLite) does not seem very efficient.

So the next iteration of the cache should be significantly smaller.


# Regarding cache invalidation:

A lot of the data in the caches are an inherent property of the
changeset and therefore immutable. It's easy to preserve then during
strip to avoid having to recompute things from scratch. In addition,
these immutable data should be exchange during pull alongside the
associated changesets to avoid client recomputing the same data over and
over.

The current implementation is an experimental/research implementation,
all this should get smoothed directly in Core during the upstreaming.


# Regarding  cache-computation speed:

The current implementation is a "research" version written in Python, it
is not geared toward efficiency and contains a lot of indirections that
were helpful to reach the current solution but are now getting in the
way of performance.

The initial implementation (in Evolve), focused on finding a solution
with good scaling property (good computational and space complexity).
However, we did not spend too much time improving the "constant" factor.
Now that we know where we are headed we can have a much better
implementation.

Once we have better on disk storage, native code and client/server
exchange of most of the data, the impact of stable-range should get to a
negligible level.


# Regarding what's next:

The experimental implementation cleared the unknown around stable-range
computation and caching. However, even if the road is clear, a sizable
amount of work remains, especially to move away from the unsuitable
SQLite storage. We think that putting the Bitbucket donation to use is
the best way to make sure this work gets done soon.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] test-sqlitestore: run sqlite3 with no init file

[Boris FELD]

Queued, thanks

On 19/02/2019 02:15, Jordi Gutiérrez Hermoso wrote:
> # HG changeset patch
> # User Jordi Gutiérrez Hermoso 
> # Date 1550533461 18000
> #  Mon Feb 18 18:44:21 2019 -0500
> # Node ID 3cac2f5ea9314a7d45079e629e0570bebcb3b6bf
> # Parent  37b33c34bf4f890857b5e8728febbc82a99368a5
> test-sqlitestore: run sqlite3 with no init file
>
> These tests were failing for me because I have a ~/.sqliterc that
> alters the default output. Tests should ignore any init file.
>
> diff --git a/tests/test-sqlitestore.t b/tests/test-sqlitestore.t
> --- a/tests/test-sqlitestore.t
> +++ b/tests/test-sqlitestore.t
> @@ -71,17 +71,17 @@ Can make a local commit
>  
>  That results in a row being inserted into various tables
>  
> -  $ sqlite3 .hg/store/db.sqlite << EOF
> +  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF
>> SELECT * FROM filepath;
>> EOF
>1|foo
>  
> -  $ sqlite3 .hg/store/db.sqlite << EOF
> +  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF
>> SELECT * FROM fileindex;
>> EOF
>
> 1|1|0|-1|-1|0|0|1||6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe
>  (esc)
>  
> -  $ sqlite3 .hg/store/db.sqlite << EOF
> +  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF
>> SELECT * FROM delta;
>> EOF
>1|1|   
> \xd2\xaf\x8d\xd2"\x01\xdd\x8dH\xe5\xdc\xfc\xae\xd2\x81\xff\x94"\xc7|0 (esc)
> @@ -93,7 +93,7 @@ Tracking multiple files works
>$ hg commit -A -m 'add bar'
>adding bar
>  
> -  $ sqlite3 .hg/store/db.sqlite << EOF
> +  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF
>> SELECT * FROM filedata ORDER BY id ASC;
>> EOF
>
> 1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1|
>  (esc)
> @@ -104,7 +104,7 @@ Multiple revisions of a file works
>$ echo a >> foo
>$ hg commit -m 'modify foo'
>  
> -  $ sqlite3 .hg/store/db.sqlite << EOF
> +  $ sqlite3 .hg/store/db.sqlite -init /dev/null << EOF
>> SELECT * FROM filedata ORDER BY id ASC;
>> EOF
>
> 1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1|
>  (esc)
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5960: test: stabilize test-remotefilelog-bgprefetch.t flaky output

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  In https://phab.mercurial-scm.org/D5960#87251, @durin42 wrote:
  
  > Huh. That's strictly a bug, as `debugwaitonrepack` should actually be 
waiting.
  
  
  I've created a ticket to track it: https://phab.mercurial-scm.org/D5960
  
  Do you have any idea what could be happening?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5960

To: lothiraldan, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5966: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

lothiraldan added inline comments.

INLINE COMMENTS

> durin42 wrote in test-run-tests.t:1177
> Maybe we could make it 3,5 so it's be more permissive?

Yes we could.

I think the regex was previously `{4,5}` for two reasons:

- We expected the duration to be less than 10 seconds.

- The precision of the number was 3 digits or 4 digits after the dot depending 
if we used json or simplejson for dumping the data.

Right now, it seems that `run-tests.py` itself is limitation the precision of 
the digits.

So we can either drop the {3,4} as we should have a stable number of digits 
after the dot, and anyone changing the precision of times in `run-tests.py` 
will have a diff in this test.

Or we make it more permissive so we don't need to bother about updating it 
later.

I don't have a strong opinion about either, what do you think we should do?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5966

To: lothiraldan, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 10 of 10] transaction: include txnname in the hookargs dictionary

[Boris FELD]

On 09/02/2019 23:34, Pulkit Goyal wrote:
>
>
> On Sun, Feb 10, 2019 at 3:54 AM Boris Feld  <mailto:boris.f...@octobus.net>> wrote:
>
> # HG changeset patch
> # User Boris Feld  <mailto:boris.f...@octobus.net>>
> # Date 1549749690 -3600
> #      Sat Feb 09 23:01:30 2019 +0100
> # Node ID f4936120d74a81199068bed14a04ac65f24f6109
> # Parent  62d13ee8373ad4a7c97192680a4781fe5818a1df
> # EXP-Topic hook.txname
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull
> https://bitbucket.org/octobus/mercurial-devel/ -r f4936120d74a
> transaction: include txnname in the hookargs dictionary
>
> There is no reason to not include the txnname alongside the txnid
> in all case.
> So let's do that. If the transaction name is useful to internal
> code, it can
> be useful for hooks.
>
>
> What are the cases when we have transaction name and source different?

I don't remember anything specific about the rationale for this patch.
It has been laying around for a long while (first version date back from
May 2015). At that point, I think the point is mostly to make the shell
hook consistent with the python one. (Because the Python one take this
argument already).
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5968: obsutil: don't assume leftctx and rightctx repo as same

[lothiraldan (Boris Feld)]

lothiraldan accepted this revision.
lothiraldan added a comment.


  LGTM, nice catch! Could you add a comment to clarify it?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5968

To: pulkit, #hg-reviewers, lothiraldan
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5965: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

lothiraldan abandoned this revision.
lothiraldan added a comment.


  Replaced by https://phab.mercurial-scm.org/D5966

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5965

To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5966: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We have reached a point where the duration in JSON reports of
  `test-run-tests.t` were greater or equal than 10 seconds, which doesn't match
  anymore the regex. For example here:
  
https://ci.octobus.net/blue/organizations/jenkins/MercurialPy2/detail/MercurialPy2/276/pipeline
  
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "10.040",
   "result": "skip", ? (re)
  
  Instead of accepting more characters, I changed the regex to accept any number
  of digits before the `.` then 3 or 4 digits after. This way the regex is more
  precise (only one `.` is authorized and we can ensure that the precision
  doesn't change).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5966

AFFECTED FILES
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -1174,31 +1174,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }
   } (no-eol)
 --json with --outputdir
@@ -1231,31 +1231,31 @@
   $ cat output/report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\s*\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\s*\d+\.\d{3,4}", ? (re)
+  "time": "\s*\d+\.\d{3,4}" (re)
   }
   } (no-eol)
   $ ls -a output
@@ -1287,31 +1287,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\s*\d+\.\d{3,4}", ? (re)
+  "cuser": "\s*\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)

D5965: test: stabilize test-run-tests.t output

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We have reached a point where the duration in `test-run-tests.t` were greater
  or equal than 10s, which doesn't match anymore the regex. For example here:
  
https://ci.octobus.net/blue/organizations/jenkins/MercurialPy2/detail/MercurialPy2/276/pipeline
  
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "10.040",
   "result": "skip", ? (re)
  
  Instead of accepting more characters, I changed the regex to accept any number
  of digits before the `.` than 3 or 4 digits after. This way the regex is more
  precise (only one `.` is authorized and we can ensure that the precision
  doesn't change).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5965

AFFECTED FILES
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -1174,31 +1174,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }
   } (no-eol)
 --json with --outputdir
@@ -1231,31 +1231,31 @@
   $ cat output/report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "---.+\+\+\+.+", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "failure", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-skip.t": {
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "skip", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }, ? (re)
   "test-success.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? (re)
-  "time": "\s*[\d\.]{4,5}" (re)
+  "start": "\d+\.\d{3,4}", ? (re)
+  "time": "\d+\.\d{3,4}" (re)
   }
   } (no-eol)
   $ ls -a output
@@ -1287,31 +1287,31 @@
   $ cat report.json
   testreport ={
   "test-failure.t": [\{] (re)
-  "csys": "\s*[\d\.]{4,5}", ? (re)
-  "cuser": "\s*[\d\.]{4,5}", ? (re)
+  "csys": "\d+\.\d{3,4}", ? (re)
+  "cuser": "\d+\.\d{3,4}", ? (re)
   "diff": "", ? (re)
-  "end": "\s*[\d\.]{4,5}", ? (re)
+  "end": "\d+\.\d{3,4}", ? (re)
   "result": "success", ? (re)
-  "start": "\s*[\d\.]{4,5}", ? 

D5960: test: stabilize test-remotefilelog-bgprefetch.t flaky output

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When running the test suite with multiple processes, we often get flaky
  outputs, like here: https://ci.octobus.net/job/MercurialPy2/274/console
  
-  
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
-  
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
-  
$TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.dataidx
-  
$TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.datapack
+  
$TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0
+  
$TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/ef95c5376f34698742fe34f315fd82136f8f68c0
+  
$TESTTMP/hgcache/master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a
+  
$TESTTMP/hgcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
+  
$TESTTMP/hgcache/master/af/f024fe4ab0fece4091de044c58c9ae4233383a/bb6ccd5dceaa5e9dc220e0dad65e051b94f69a2c
  
  Add some sleeps after `debugwaitonrepack` calls as it seems it needs some
  extra time to cleanup.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5960

AFFECTED FILES
  tests/test-remotefilelog-bgprefetch.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-bgprefetch.t 
b/tests/test-remotefilelog-bgprefetch.t
--- a/tests/test-remotefilelog-bgprefetch.t
+++ b/tests/test-remotefilelog-bgprefetch.t
@@ -105,6 +105,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 0.5
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 0.5
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histidx
   
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histpack
@@ -193,6 +194,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 1
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 1
   $ find $CACHEDIR -type f | sort
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
   
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
@@ -243,6 +245,7 @@
   $ hg debugwaitonprefetch >/dev/null 2>%1
   $ sleep 1
   $ hg debugwaitonrepack >/dev/null 2>%1
+  $ sleep 1
 
 # Ensure that file 'y' was prefetched - it was not part of the rebase 
operation and therefore
 # could only be downloaded by the background prefetch



To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5590: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan abandoned this revision.
lothiraldan added a comment.


  Resent as https://phab.mercurial-scm.org/D5955

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5590

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5589: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

lothiraldan abandoned this revision.
lothiraldan added a comment.


  Resent as https://phab.mercurial-scm.org/D5954

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5589

To: lothiraldan, #hg-reviewers
Cc: indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5955: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Don't display 'illegal_fstypes' errors. In environments with network
  filesystems, the error messages are quickly pilling up and polluting outputs.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5955

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -180,7 +180,8 @@
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: fsmonitor.verbose
 if ex.warn and ui.configbool('fsmonitor', 'verbose'):
-ui.warn(str(ex) + '\n')
+if 'illegal_fstypes' not in str(ex):
+ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
 # experimental config: fsmonitor.verbose



To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5954: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is necessary to make rolling releases of new watchman versions across
  users.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5954

AFFECTED FILES
  hgext/fsmonitor/__init__.py
  hgext/fsmonitor/pywatchman/__init__.py
  hgext/fsmonitor/watchmanclient.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/watchmanclient.py 
b/hgext/fsmonitor/watchmanclient.py
--- a/hgext/fsmonitor/watchmanclient.py
+++ b/hgext/fsmonitor/watchmanclient.py
@@ -82,9 +82,11 @@
 try:
 if self._watchmanclient is None:
 self._firsttime = False
+watchman_exe = self._ui.configpath('fsmonitor', 'watchman_exe')
 self._watchmanclient = pywatchman.client(
 timeout=self._timeout,
-useImmutableBser=True)
+useImmutableBser=True,
+watchman_exe=watchman_exe)
 return self._watchmanclient.query(*watchmanargs)
 except pywatchman.CommandError as ex:
 if 'unable to resolve root' in ex.msg:
diff --git a/hgext/fsmonitor/pywatchman/__init__.py 
b/hgext/fsmonitor/pywatchman/__init__.py
--- a/hgext/fsmonitor/pywatchman/__init__.py
+++ b/hgext/fsmonitor/pywatchman/__init__.py
@@ -317,7 +317,7 @@
 """ local unix domain socket transport """
 sock = None
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
 
@@ -397,7 +397,7 @@
 class WindowsNamedPipeTransport(Transport):
 """ connect to a named pipe """
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = int(math.ceil(timeout * 1000))
 self._iobuf = None
@@ -563,9 +563,10 @@
 proc = None
 closed = True
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
+self.watchman_exe = watchman_exe
 
 def close(self):
 if self.proc:
@@ -579,7 +580,7 @@
 if self.proc:
 return self.proc
 args = [
-'watchman',
+self.watchman_exe,
 '--sockname={0}'.format(self.sockpath),
 '--logfile=/BOGUS',
 '--statefile=/BOGUS',
@@ -756,17 +757,20 @@
 unilateral = ['log', 'subscription']
 tport = None
 useImmutableBser = None
+watchman_exe = None
 
 def __init__(self,
  sockpath=None,
  timeout=1.0,
  transport=None,
  sendEncoding=None,
  recvEncoding=None,
- useImmutableBser=False):
+ useImmutableBser=False,
+ watchman_exe=None):
 self.sockpath = sockpath
 self.timeout = timeout
 self.useImmutableBser = useImmutableBser
+self.watchman_exe = watchman_exe
 
 if inspect.isclass(transport) and issubclass(transport, Transport):
 self.transport = transport
@@ -817,7 +821,7 @@
 if path:
 return path
 
-cmd = ['watchman', '--output-encoding=bser', 'get-sockname']
+cmd = [self.watchman_exe, '--output-encoding=bser', 'get-sockname']
 try:
 args = dict(stdout=subprocess.PIPE,
 stderr=subprocess.PIPE,
@@ -858,7 +862,7 @@
 if self.sockpath is None:
 self.sockpath = self._resolvesockname()
 
-self.tport = self.transport(self.sockpath, self.timeout)
+self.tport = self.transport(self.sockpath, self.timeout, 
self.watchman_exe)
 self.sendConn = self.sendCodec(self.tport)
 self.recvConn = self.recvCodec(self.tport)
 
diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('fsmonitor', 'watchman_exe',
+default='watchman',
+)
 configitem('fsmonitor', 'verbose',
 default=True,
 )



To: lothiraldan, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5587: watchman: detect nested mercurial repositories and abort

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14075.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5587?vs=13645=14075

REVISION DETAIL
  https://phab.mercurial-scm.org/D5587

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -367,6 +367,9 @@
 fexists = entry['exists']
 kind = getkind(fmode)
 
+if '/.hg/' in fname or fname.endswith('/.hg'):
+return bail('nested-repo-detected')
+
 if not fexists:
 # if marked as deleted and we don't already have a change
 # record, mark it as deleted.  If we already have an entry
@@ -740,6 +743,14 @@
 repo, node, branchmerge, force, ancestor, mergeancestor,
 labels, matcher, **kwargs)
 
+def repo_has_depth_one_nested_repo(repo):
+for f in repo.wvfs.listdir():
+if os.path.isdir(os.path.join(repo.root, f, '.hg')):
+msg = 'fsmonitor: sub-repository %r detected, fsmonitor disabled\n'
+repo.ui.debug(msg % f)
+return True
+return False
+
 def reposetup(ui, repo):
 # We don't work with largefiles or inotify
 exts = extensions.enabled()
@@ -757,6 +768,9 @@
 if repo.wvfs.exists('.hgsubstate') or repo.wvfs.exists('.hgsub'):
 return
 
+if repo_has_depth_one_nested_repo(repo):
+return
+
 fsmonitorstate = state.state(repo)
 if fsmonitorstate.mode == 'off':
 return



To: lothiraldan, #hg-reviewers, indygreg
Cc: mjpieters, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5590: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14074.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5590?vs=14071=14074

REVISION DETAIL
  https://phab.mercurial-scm.org/D5590

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -180,7 +180,8 @@
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: fsmonitor.verbose
 if ex.warn and ui.configbool('fsmonitor', 'verbose'):
-ui.warn(str(ex) + '\n')
+if 'illegal_fstypes' not in str(ex):
+ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
 # experimental config: fsmonitor.verbose



To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5589: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14073.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5589?vs=14070=14073

REVISION DETAIL
  https://phab.mercurial-scm.org/D5589

AFFECTED FILES
  hgext/fsmonitor/__init__.py
  hgext/fsmonitor/pywatchman/__init__.py
  hgext/fsmonitor/watchmanclient.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/watchmanclient.py 
b/hgext/fsmonitor/watchmanclient.py
--- a/hgext/fsmonitor/watchmanclient.py
+++ b/hgext/fsmonitor/watchmanclient.py
@@ -82,9 +82,11 @@
 try:
 if self._watchmanclient is None:
 self._firsttime = False
+watchman_exe = self._ui.configpath('fsmonitor', 'watchman_exe')
 self._watchmanclient = pywatchman.client(
 timeout=self._timeout,
-useImmutableBser=True)
+useImmutableBser=True,
+watchman_exe=watchman_exe)
 return self._watchmanclient.query(*watchmanargs)
 except pywatchman.CommandError as ex:
 if 'unable to resolve root' in ex.msg:
diff --git a/hgext/fsmonitor/pywatchman/__init__.py 
b/hgext/fsmonitor/pywatchman/__init__.py
--- a/hgext/fsmonitor/pywatchman/__init__.py
+++ b/hgext/fsmonitor/pywatchman/__init__.py
@@ -317,7 +317,7 @@
 """ local unix domain socket transport """
 sock = None
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
 
@@ -397,7 +397,7 @@
 class WindowsNamedPipeTransport(Transport):
 """ connect to a named pipe """
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = int(math.ceil(timeout * 1000))
 self._iobuf = None
@@ -563,9 +563,10 @@
 proc = None
 closed = True
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
+self.watchman_exe = watchman_exe
 
 def close(self):
 if self.proc:
@@ -579,7 +580,7 @@
 if self.proc:
 return self.proc
 args = [
-'watchman',
+self.watchman_exe,
 '--sockname={0}'.format(self.sockpath),
 '--logfile=/BOGUS',
 '--statefile=/BOGUS',
@@ -756,17 +757,20 @@
 unilateral = ['log', 'subscription']
 tport = None
 useImmutableBser = None
+watchman_exe = None
 
 def __init__(self,
  sockpath=None,
  timeout=1.0,
  transport=None,
  sendEncoding=None,
  recvEncoding=None,
- useImmutableBser=False):
+ useImmutableBser=False,
+ watchman_exe=None):
 self.sockpath = sockpath
 self.timeout = timeout
 self.useImmutableBser = useImmutableBser
+self.watchman_exe = watchman_exe
 
 if inspect.isclass(transport) and issubclass(transport, Transport):
 self.transport = transport
@@ -817,7 +821,7 @@
 if path:
 return path
 
-cmd = ['watchman', '--output-encoding=bser', 'get-sockname']
+cmd = [self.watchman_exe, '--output-encoding=bser', 'get-sockname']
 try:
 args = dict(stdout=subprocess.PIPE,
 stderr=subprocess.PIPE,
@@ -858,7 +862,7 @@
 if self.sockpath is None:
 self.sockpath = self._resolvesockname()
 
-self.tport = self.transport(self.sockpath, self.timeout)
+self.tport = self.transport(self.sockpath, self.timeout, 
self.watchman_exe)
 self.sendConn = self.sendCodec(self.tport)
 self.recvConn = self.recvCodec(self.tport)
 
diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('fsmonitor', 'watchman_exe',
+default='watchman',
+)
 configitem('fsmonitor', 'verbose',
 default=True,
 )



To: lothiraldan, #hg-reviewers
Cc: indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5589: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14070.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5589?vs=13414=14070

REVISION DETAIL
  https://phab.mercurial-scm.org/D5589

AFFECTED FILES
  hgext/fsmonitor/__init__.py
  hgext/fsmonitor/pywatchman/__init__.py
  hgext/fsmonitor/watchmanclient.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/watchmanclient.py 
b/hgext/fsmonitor/watchmanclient.py
--- a/hgext/fsmonitor/watchmanclient.py
+++ b/hgext/fsmonitor/watchmanclient.py
@@ -82,9 +82,11 @@
 try:
 if self._watchmanclient is None:
 self._firsttime = False
+watchman_exe = self._ui.configpath('fsmonitor', 'watchman_exe')
 self._watchmanclient = pywatchman.client(
 timeout=self._timeout,
-useImmutableBser=True)
+useImmutableBser=True,
+watchman_exe=watchman_exe)
 return self._watchmanclient.query(*watchmanargs)
 except pywatchman.CommandError as ex:
 if 'unable to resolve root' in ex.msg:
diff --git a/hgext/fsmonitor/pywatchman/__init__.py 
b/hgext/fsmonitor/pywatchman/__init__.py
--- a/hgext/fsmonitor/pywatchman/__init__.py
+++ b/hgext/fsmonitor/pywatchman/__init__.py
@@ -317,7 +317,7 @@
 """ local unix domain socket transport """
 sock = None
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
 
@@ -397,7 +397,7 @@
 class WindowsNamedPipeTransport(Transport):
 """ connect to a named pipe """
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = int(math.ceil(timeout * 1000))
 self._iobuf = None
@@ -563,9 +563,10 @@
 proc = None
 closed = True
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
+self.watchman_exe = watchman_exe
 
 def close(self):
 if self.proc:
@@ -579,7 +580,7 @@
 if self.proc:
 return self.proc
 args = [
-'watchman',
+self.watchman_exe,
 '--sockname={0}'.format(self.sockpath),
 '--logfile=/BOGUS',
 '--statefile=/BOGUS',
@@ -756,17 +757,20 @@
 unilateral = ['log', 'subscription']
 tport = None
 useImmutableBser = None
+watchman_exe = None
 
 def __init__(self,
  sockpath=None,
  timeout=1.0,
  transport=None,
  sendEncoding=None,
  recvEncoding=None,
- useImmutableBser=False):
+ useImmutableBser=False,
+ watchman_exe=None):
 self.sockpath = sockpath
 self.timeout = timeout
 self.useImmutableBser = useImmutableBser
+self.watchman_exe = watchman_exe
 
 if inspect.isclass(transport) and issubclass(transport, Transport):
 self.transport = transport
@@ -817,7 +821,7 @@
 if path:
 return path
 
-cmd = ['watchman', '--output-encoding=bser', 'get-sockname']
+cmd = [self.watchman_exe, '--output-encoding=bser', 'get-sockname']
 try:
 args = dict(stdout=subprocess.PIPE,
 stderr=subprocess.PIPE,
@@ -858,7 +862,7 @@
 if self.sockpath is None:
 self.sockpath = self._resolvesockname()
 
-self.tport = self.transport(self.sockpath, self.timeout)
+self.tport = self.transport(self.sockpath, self.timeout, 
self.watchman_exe)
 self.sendConn = self.sendCodec(self.tport)
 self.recvConn = self.recvCodec(self.tport)
 
diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -164,6 +164,9 @@
 configitem('fsmonitor', 'enable_on_non_interactive',
 default=True,
 )
+configitem('fsmonitor', 'watchman_exe',
+default='watchman',
+)
 configitem('fsmonitor', 'verbose',
 default=True,
 )



To: lothiraldan, #hg-reviewers
Cc: indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5590: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14071.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5590?vs=13415=14071

REVISION DETAIL
  https://phab.mercurial-scm.org/D5590

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -183,7 +183,8 @@
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: fsmonitor.verbose
 if ex.warn and ui.configbool('fsmonitor', 'verbose'):
-ui.warn(str(ex) + '\n')
+if 'illegal_fstypes' not in str(ex):
+ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
 # experimental config: fsmonitor.verbose



To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5588: watchman: disable fsmonitor when running non-interactively

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 14069.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5588?vs=13413=14069

REVISION DETAIL
  https://phab.mercurial-scm.org/D5588

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('fsmonitor', 'enable_on_non_interactive',
+default=True,
+)
 configitem('fsmonitor', 'verbose',
 default=True,
 )
@@ -760,6 +763,24 @@
   'extension and has been disabled.\n') % ext)
 return
 
+enabled = None
+if 'HGWATCHMAN_ENABLE' in encoding.environ:
+enabled = encoding.environ['HGWATCHMAN_ENABLE'].lower()
+
+if enabled == 'true' or enabled == '':
+pass
+elif enabled == 'false':
+ui.debug('fsmonitor: disabled through HGWATCHMAN_ENABLE env variable')
+return
+else:
+interactive = ui.interactive()
+enable_on_non_interactive = ui.configbool(
+'fsmonitor', 'enable_on_non_interactive'
+)
+if not interactive and not enable_on_non_interactive:
+ui.debug('fsmonitor: disabled for non-interactive commands')
+return
+
 if repo.local():
 # We don't work with subrepos either.
 #



To: lothiraldan, #hg-reviewers, indygreg
Cc: mjpieters, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 RESENT] rewriting: add an option for rewrite commands to use the archived phase

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1546394872 -3600
#  Wed Jan 02 03:07:52 2019 +0100
# Node ID 9939d8e412e3e440f3b564fb96c187745d7a008c
# Parent  61ec4a834e2c88056ff47c0d3a7ff3bcb0f0d912
# EXP-Topic archived-phase-UX
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
9939d8e412e3
rewriting: add an option for rewrite commands to use the archived phase

Using the archived phase for cleanup provide the same effect than stripping,
but in a faster, append-only way.

We keep the feature experimental for now until it gets a bit more testing.

diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -470,6 +470,9 @@ coreconfigitem('experimental', 'bundleco
 coreconfigitem('experimental', 'changegroup3',
 default=False,
 )
+coreconfigitem('experimental', 'cleanup-as-archived',
+default=False,
+)
 coreconfigitem('experimental', 'clientcompressionengines',
 default=list,
 )
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1014,6 +1014,7 @@ def cleanupnodes(repo, replacements, ope
 for phase, nodes in toadvance.items():
 phases.advanceboundary(repo, tr, phase, nodes)
 
+mayusearchived = repo.ui.config('experimental', 'cleanup-as-archived')
 # Obsolete or strip nodes
 if obsolete.isenabled(repo, obsolete.createmarkersopt):
 # If a node is already obsoleted, and we want to obsolete it
@@ -1031,6 +1032,17 @@ def cleanupnodes(repo, replacements, ope
 if rels:
 obsolete.createmarkers(repo, rels, operation=operation,
metadata=metadata)
+elif phases.supportinternal(repo) and mayusearchived:
+# this assume we do not have "unstable" nodes above the cleaned 
ones
+allreplaced = set()
+for ns in replacements.keys():
+allreplaced.update(ns)
+if backup:
+from . import repair # avoid import cycle
+node = min(allreplaced, key=repo.changelog.rev)
+repair.backupbundle(repo, allreplaced, allreplaced, node,
+operation)
+phases.retractboundary(repo, tr, phases.archived, allreplaced)
 else:
 from . import repair # avoid import cycle
 tostrip = list(n for ns in replacements for n in ns)
diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
--- a/tests/test-phase-archived.t
+++ b/tests/test-phase-archived.t
@@ -75,3 +75,69 @@ Test that bundle can unarchive a changes
  date:Thu Jan 01 00:00:00 1970 +
  summary: root
   
+
+Test that history rewriting command can use the archived phase when allowed to
+--
+
+  $ hg up 'desc(unbundletesting)'
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo bar >> a
+  $ hg commit --amend --config experimental.cleanup-as-archived=yes
+  $ hg log -G
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
+  $ hg log -G --hidden
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  | o  changeset:   1:883aadbbf309
+  |/   user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
+  $ ls -1 .hg/strip-backup/
+  883aadbbf309-efc55adc-amend.hg
+  883aadbbf309-efc55adc-backup.hg
+  $ hg unbundle .hg/strip-backup/883aadbbf309*amend.hg
+  adding changesets
+  adding manifests
+  adding file changes
+  added 0 changesets with 0 changes to 1 files
+  (run 'hg update' to get a working copy)
+  $ hg log -G
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  | o  changeset:   1:883aadbbf309
+  |/   user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
___
Mercurial-devel mailing list

[PATCH 1 of 2 RESENT] strip: introduce a soft strip option

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1539697680 -7200
#  Tue Oct 16 15:48:00 2018 +0200
# Node ID 61ec4a834e2c88056ff47c0d3a7ff3bcb0f0d912
# Parent  61415361e90684a8c7a031413e9182f51937c2e7
# EXP-Topic archived-phase-UX
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
61ec4a834e2c
strip: introduce a soft strip option

This is the first user-accessible way to use the archived phase introduced in
4.8. This implements a feature discussed during the Stockholm sprint, using
the archived phase for hiding changesets.

The archived phase behaves exactly as stripping: changesets are no longer
visible, but pulling/unbundling them will make then reappear. The only notable
difference is that unlike hard stripping, soft stripping does not affect
obsmarkers.

The next changeset will make use of the archived phase for history rewriting
command. However, having a way to manually trigger the feature first seems a
necessary step before exposing users to this phase; there is a way to
un-archived changesets (unbundling), so there must be a way to archive them
again.

Adding a flag to strip is a good way to provide access to the feature without
taking a too big risk on the final UI we want. The flag is experimental so it
won't be exposed by default.

Using the archived phase is faster and less traumatic for the repository than
actually stripping changesets.

diff --git a/hgext/strip.py b/hgext/strip.py
--- a/hgext/strip.py
+++ b/hgext/strip.py
@@ -76,7 +76,8 @@ def _findupdatetarget(repo, nodes):
 
 return unode
 
-def strip(ui, repo, revs, update=True, backup=True, force=None, 
bookmarks=None):
+def strip(ui, repo, revs, update=True, backup=True, force=None, bookmarks=None,
+  soft=False):
 with repo.wlock(), repo.lock():
 
 if update:
@@ -85,7 +86,10 @@ def strip(ui, repo, revs, update=True, b
 hg.clean(repo, urev)
 repo.dirstate.write(repo.currenttransaction())
 
-repair.strip(ui, repo, revs, backup)
+if soft:
+repair.softstrip(ui, repo, revs, backup)
+else:
+repair.strip(ui, repo, revs, backup)
 
 repomarks = repo._bookmarks
 if bookmarks:
@@ -110,7 +114,10 @@ def strip(ui, repo, revs, update=True, b
   ('k', 'keep', None, _("do not modify working directory during "
 "strip")),
   ('B', 'bookmark', [], _("remove revs only reachable from given"
-  " bookmark"), _('BOOKMARK'))],
+  " bookmark"), _('BOOKMARK')),
+  ('', 'soft', None,
+  _("simply drop changesets from visible history (EXPERIMENTAL)")),
+ ],
   _('hg strip [-k] [-f] [-B bookmark] [-r] REV...'),
   helpcategory=command.CATEGORY_MAINTENANCE)
 def stripcmd(ui, repo, *revs, **opts):
@@ -235,6 +242,7 @@ def stripcmd(ui, repo, *revs, **opts):
 
 
 strip(ui, repo, revs, backup=backup, update=update,
-  force=opts.get('force'), bookmarks=bookmarks)
+  force=opts.get('force'), bookmarks=bookmarks,
+  soft=opts['soft'])
 
 return 0
diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -252,6 +252,24 @@ def strip(ui, repo, nodelist, backup=Tru
 # extensions can use it
 return backupfile
 
+def softstrip(ui, repo, nodelist, backup=True, topic='backup'):
+"""perform a "soft" strip using the archived phase"""
+tostrip = [c.node() for c in repo.set('sort(%ln::)', nodelist)]
+if not tostrip:
+return None
+
+newbmtarget, updatebm = _bookmarkmovements(repo, tostrip)
+if backup:
+node = tostrip[0]
+backupfile = _createstripbackup(repo, tostrip, node, topic)
+
+with repo.transaction('strip') as tr:
+phases.retractboundary(repo, tr, phases.archived, tostrip)
+bmchanges = [(m, repo[newbmtarget].node()) for m in updatebm]
+repo._bookmarks.applychanges(repo, tr, bmchanges)
+return backupfile
+
+
 def _bookmarkmovements(repo, tostrip):
 # compute necessary bookmark movement
 bm = repo._bookmarks
diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
new file mode 100644
--- /dev/null
+++ b/tests/test-phase-archived.t
@@ -0,0 +1,77 @@
+=
+Test features and behaviors related to the archived phase
+=
+
+  $ cat << EOF >> $HGRCPATH
+  > [format]
+  > internal-phase=yes
+  > [extensions]
+  > strip=
+  > [experimental]
+  > EOF
+
+  $ hg init repo
+  $ cd repo
+  $ echo  root > a
+  $ hg add a
+  $ hg ci -m 'root'
+
+Test that bundle can unarchive a changeset
+

Re: Auto-formatting code with black - object now if you have a strong opinion

[Boris FELD]

On 01/12/2018 02:35, Matt Harbison wrote:
> On Fri, 30 Nov 2018 07:25:04 -0500, Boris FELD
>  wrote:
>
>> I think using automatic formatting is a great idea and we should move
>> forward with this plan. Black seems a good option. I share other's
>> concerns about the formatting of import. I also wonder if this also
>> applies to list and dict formatting that we tend to express with one
>> value per line for clarity.
>
> It looks like yes, unfortunately, if it fits on one line:
>
> diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
> --- a/hgext/lfs/blobstore.py
> +++ b/hgext/lfs/blobstore.py
> @@ -289,50 +289,47 @@ class _gitlfsremote(object):
>  Return decoded JSON object like {'objects': [{'oid': '',
> 'size': 1}]}
>  See
> https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
>  """
> -    objects = [{'oid': p.oid(), 'size': p.size()} for p in pointers]
> -    requestdata = json.dumps({
> -    'objects': objects,
> -    'operation': action,
> -    })
> -    url = '%s/objects/batch' % self.baseurl
> +    objects = [{"oid": p.oid(), "size": p.size()} for p in pointers]
> +    requestdata = json.dumps({"objects": objects, "operation":
> action})
> +    url = "%s/objects/batch" % self.baseurl
>  batchreq = util.urlreq.request(url, data=requestdata)
> ...

We have been discussing with the Black author about how we could handle
those cases and we found a `hack` which is adding an empty comment on
the first line of a list, dict, multi-line construction:

For example:

requestdata = json.dumps({
    #
    'objects': objects,
    'operation': action,

})

Would get transformed into:

requestdata = json.dumps(
    {
    #
    "objects": objects,
    "operation": action,
    }
)

which is then stable.

> I'm also concerned about stuff like this, which seems far less
> readable than the original (especially the conditional):
>
> diff --git a/hgext/relink.py b/hgext/relink.py
> --- a/hgext/relink.py
> +++ b/hgext/relink.py
> @@ -56,29 +50,32 @@ def relink(ui, repo, origin=None, **opts
>  command is running. (Both repositories will be locked against
>  writes.)
>  """
> -    if (not util.safehasattr(util, 'samefile') or
> -    not util.safehasattr(util, 'samedevice')):
> -    raise error.Abort(_('hardlinks are not supported on this
> system'))
> -    src = hg.repository(repo.baseui, ui.expandpath(origin or
> 'default-relink',
> -  origin or 'default'))
> -    ui.status(_('relinking %s to %s\n') % (src.store.path,
> repo.store.path))
> +    if not util.safehasattr(util, "samefile") or not util.safehasattr(
> +    util, "samedevice"
> +    ):
> +    raise error.Abort(_("hardlinks are not supported on this
> system"))
> +    src = hg.repository(
> +    repo.baseui, ui.expandpath(origin or "default-relink", origin
> or "default")
> +    )
> +    ui.status(_("relinking %s to %s\n") % (src.store.path,
> repo.store.path))
>  if repo.root == src.root:
Black output is not final yet, Black author wants to have the
possibility to make bugfixes. This particular example might be a bug
that could be solved. It could also be solved by extracting parameters
into variables.
>
> This was actually in the first file that I randomly sampled.  I think
> there were a few more instances like this, but don't feel motivated to
> find them now.  There were a bunch of lines (in lfs anyway) that were
> flattened out, and were more readable.  But that was before I saw that
> the default formatting is 88 columns.  So maybe allowing longer lines
> would help?  (At the cost of possibly rolling up more lists and dicts
> into a single line.)
>
> I'm not adamantly opposed, and the idea of combining version control
> and tooling to enforce a format is intriguing.  But FWIW, I'm not
> thrilled with the result of this.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5897: test: stabilize test-wireproto-exchangev2.t flaky output

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1ab6f5df263e: test: stabilize test-wireproto-exchangev2.t 
flaky output (authored by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5897?vs=13917=14048

REVISION DETAIL
  https://phab.mercurial-scm.org/D5897

AFFECTED FILES
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -36,22 +36,18 @@
 
 Test basic clone
 
-  $ hg --debug clone -U http://localhost:$HGPORT client-simple
+Output is flaky, save it in a file and check part independently
+  $ hg --debug clone -U http://localhost:$HGPORT client-simple > clone-output
+
+  $ cat clone-output | grep -v "received frame"
   using http://localhost:$HGPORT/
   sending capabilities command
   query 1; heads
   sending 2 commands
   sending command heads: {}
   sending command known: {
 'nodes': []
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=43; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
-  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
   sending 1 commands
   sending command changesetdata: {
 'fields': set([
@@ -71,10 +67,6 @@
   }
 ]
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=941; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   add changeset 3390ef850073
   add changeset 4432d83626e8
   add changeset cd2534766bec
@@ -97,10 +89,6 @@
 ],
 'tree': ''
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=992; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   sending 1 commands
   sending command filesdata: {
 'fields': set([
@@ -121,13 +109,32 @@
   }
 ]
   }
+  updating the branch cache
+  new changesets 3390ef850073:caa2a465451d (3 drafts)
+  (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob)
+
+  $ cat clone-output | grep "received frame"
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=43; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=941; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=992; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; 

[PATCH 10 of 10] transaction: include txnname in the hookargs dictionary

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549749690 -3600
#  Sat Feb 09 23:01:30 2019 +0100
# Node ID f4936120d74a81199068bed14a04ac65f24f6109
# Parent  62d13ee8373ad4a7c97192680a4781fe5818a1df
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
f4936120d74a
transaction: include txnname in the hookargs dictionary

There is no reason to not include the txnname alongside the txnid in all case.
So let's do that. If the transaction name is useful to internal code, it can
be useful for hooks.

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1811,7 +1811,6 @@ class localrepository(object):
 args = tr.hookargs.copy()
 args.update(bookmarks.preparehookargs(name, old, new))
 repo.hook('pretxnclose-bookmark', throw=True,
-  txnname=desc,
   **pycompat.strkwargs(args))
 if hook.hashook(repo.ui, 'pretxnclose-phase'):
 cl = repo.unfiltered().changelog
@@ -1819,11 +1818,11 @@ class localrepository(object):
 args = tr.hookargs.copy()
 node = hex(cl.node(rev))
 args.update(phases.preparehookargs(node, old, new))
-repo.hook('pretxnclose-phase', throw=True, txnname=desc,
+repo.hook('pretxnclose-phase', throw=True,
   **pycompat.strkwargs(args))
 
 repo.hook('pretxnclose', throw=True,
-  txnname=desc, **pycompat.strkwargs(tr.hookargs))
+  **pycompat.strkwargs(tr.hookargs))
 def releasefn(tr, success):
 repo = reporef()
 if success:
@@ -1857,6 +1856,7 @@ class localrepository(object):
 tr.changes['bookmarks'] = {}
 
 tr.hookargs['txnid'] = txnid
+tr.hookargs['txnname'] = desc
 # note: writing the fncache only during finalize mean that the file is
 # outdated when running hooks. As fncache is used for streaming clone,
 # this is not expected to break anything that happen during the hooks.
@@ -1878,7 +1878,7 @@ class localrepository(object):
 args = tr.hookargs.copy()
 args.update(bookmarks.preparehookargs(name, old, new))
 repo.hook('txnclose-bookmark', throw=False,
-  txnname=desc, **pycompat.strkwargs(args))
+  **pycompat.strkwargs(args))
 
 if hook.hashook(repo.ui, 'txnclose-phase'):
 cl = repo.unfiltered().changelog
@@ -1887,10 +1887,10 @@ class localrepository(object):
 args = tr.hookargs.copy()
 node = hex(cl.node(rev))
 args.update(phases.preparehookargs(node, old, new))
-repo.hook('txnclose-phase', throw=False, txnname=desc,
+repo.hook('txnclose-phase', throw=False,
   **pycompat.strkwargs(args))
 
-repo.hook('txnclose', throw=False, txnname=desc,
+repo.hook('txnclose', throw=False,
   **pycompat.strkwargs(hookargs))
 reporef()._afterlock(hookfunc)
 tr.addfinalize('txnclose-hook', txnclosehook)
@@ -1902,7 +1902,7 @@ class localrepository(object):
 def txnaborthook(tr2):
 """To be run if transaction is aborted
 """
-reporef().hook('txnabort', throw=False, txnname=desc,
+reporef().hook('txnabort', throw=False,
**pycompat.strkwargs(tr2.hookargs))
 tr.addabort('txnabort-hook', txnaborthook)
 # avoid eager cache invalidation. in-memory data should be identical
diff --git a/tests/test-bundle.t b/tests/test-bundle.t
--- a/tests/test-bundle.t
+++ b/tests/test-bundle.t
@@ -239,6 +239,8 @@ hg -R bundle://../full.hg verify
   HG_NODE_LAST=aa35859c02ea8bd48da5da68cd2740ac71afcbaf
   HG_SOURCE=pull
   HG_TXNID=TXN:$ID$
+  HG_TXNNAME=pull
+  bundle:../full.hg
   HG_URL=bundle:../full.hg
   
   (run 'hg heads' to see heads, 'hg merge' to merge)
@@ -270,6 +272,8 @@ Pull full.hg into empty again (using -R;
   HG_NODE_LAST=aa35859c02ea8bd48da5da68cd2740ac71afcbaf
   HG_SOURCE=pull
   HG_TXNID=TXN:$ID$
+  HG_TXNNAME=pull
+  bundle:empty+full.hg
   HG_URL=bundle:empty+full.hg
   
   (run 'hg heads' to see heads, 'hg merge' to merge)
diff --git a/tests/test-bundle2-multiple-changegroups.t 
b/tests/test-bundle2-multiple-changegroups.t
--- a/tests/test-bundle2-multiple-changegroups.t
+++ b/tests/test-bundle2-multiple-changegroups.t
@@ -88,6 +88,8 @@ Pull the new commits in the clone
   HG_PENDING=$TESTTMP/clone
   HG_SOURCE=pull
   H

[PATCH 08 of 10] test: use `printenv.py --line` in `test-ssh.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671859 -3600
#  Sat Feb 09 01:24:19 2019 +0100
# Node ID a625670bc72dc8ad2b950d77fb5a72e004b911c7
# Parent  c9b41ca8817953610befc9a8e89eed6e98b5a8f0
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
a625670bc72d
test: use `printenv.py --line` in `test-ssh.t`

This makes the output easier to update and read.

diff --git a/tests/test-ssh.t b/tests/test-ssh.t
--- a/tests/test-ssh.t
+++ b/tests/test-ssh.t
@@ -36,7 +36,7 @@ configure for serving
   > uncompressed = True
   > 
   > [hooks]
-  > changegroup = sh -c "printenv.py changegroup-in-remote 0 ../dummylog"
+  > changegroup = sh -c "printenv.py --line changegroup-in-remote 0 
../dummylog"
   > EOF
   $ cd $TESTTMP
 
@@ -563,7 +563,15 @@ debug output
   Got arguments 1:user@dummy 2:hg -R local serve --stdio
   Got arguments 1:user@dummy 2:hg -R $TESTTMP/local serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_BUNDLE2=1 HG_HOOKNAME=changegroup 
HG_HOOKTYPE=changegroup HG_NODE=a28a9d1a809cab7d4e2fde4bee738a9ede948b60 
HG_NODE_LAST=a28a9d1a809cab7d4e2fde4bee738a9ede948b60 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_BUNDLE2=1
+  HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=a28a9d1a809cab7d4e2fde4bee738a9ede948b60
+  HG_NODE_LAST=a28a9d1a809cab7d4e2fde4bee738a9ede948b60
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
@@ -573,9 +581,25 @@ debug output
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_BUNDLE2=1 HG_HOOKNAME=changegroup 
HG_HOOKTYPE=changegroup HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6 
HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_BUNDLE2=1
+  HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6
+  HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:chg -R remote serve --stdio (chg !)
-  changegroup-in-remote hook: HG_BUNDLE2=1 HG_HOOKNAME=changegroup 
HG_HOOKTYPE=changegroup HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6 
HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP (chg !)
+  changegroup-in-remote hook: HG_BUNDLE2=1 (chg !)
+  HG_HOOKNAME=changegroup (chg !)
+  HG_HOOKTYPE=changegroup (chg !)
+  HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6 (chg !)
+  HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6 (chg !)
+  HG_SOURCE=serve (chg !)
+  HG_TXNID=TXN:$ID$ (chg !)
+  HG_URL=remote:ssh:$LOCALIP (chg !)
+   (chg !)
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg init 'a repo'
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
@@ -583,9 +607,18 @@ debug output
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_BUNDLE2=1 HG_HOOKNAME=changegroup 
HG_HOOKTYPE=changegroup HG_NODE=65c38f4125f9602c8db4af56530cc221d93b8ef8 
HG_NODE_LAST=65c38f4125f9602c8db4af56530cc221d93b8ef8 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_BUNDLE2=1
+  HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=65c38f4125f9602c8db4af56530cc221d93b8ef8
+  HG_NODE_LAST=65c38f4125f9602c8db4af56530cc221d93b8ef8
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
 
+
 remote hook failure is attributed to remote
 
   $ cat > $TESTTMP/failhook << EOF
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 10] test: use `printenv.py --line` in `test-static-http.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671872 -3600
#  Sat Feb 09 01:24:32 2019 +0100
# Node ID 62d13ee8373ad4a7c97192680a4781fe5818a1df
# Parent  a625670bc72dc8ad2b950d77fb5a72e004b911c7
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
62d13ee8373a
test: use `printenv.py --line` in `test-static-http.t`

This makes the output easier to update and read.

diff --git a/tests/test-static-http.t b/tests/test-static-http.t
--- a/tests/test-static-http.t
+++ b/tests/test-static-http.t
@@ -57,7 +57,7 @@ check for HTTP opener failures when cach
   $ cd ../local
   $ cat >> .hg/hgrc < [hooks]
-  > changegroup = sh -c "printenv.py changegroup"
+  > changegroup = sh -c "printenv.py --line changegroup"
   > EOF
   $ hg pull
   pulling from static-http://localhost:$HGPORT/remote
@@ -67,7 +67,14 @@ check for HTTP opener failures when cach
   adding file changes
   added 1 changesets with 1 changes to 1 files
   new changesets 4ac2e3648604
-  changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=4ac2e3648604439c580c69b09ec9d93a88d93432 
HG_NODE_LAST=4ac2e3648604439c580c69b09ec9d93a88d93432 HG_SOURCE=pull 
HG_TXNID=TXN:$ID$ HG_URL=http://localhost:$HGPORT/remote
+  changegroup hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=4ac2e3648604439c580c69b09ec9d93a88d93432
+  HG_NODE_LAST=4ac2e3648604439c580c69b09ec9d93a88d93432
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=http://localhost:$HGPORT/remote
+  
   (run 'hg update' to get a working copy)
 
 trying to push
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10] test: use `printenv.py --line` in `test-ssh-bundle1.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671829 -3600
#  Sat Feb 09 01:23:49 2019 +0100
# Node ID c9b41ca8817953610befc9a8e89eed6e98b5a8f0
# Parent  57b13716b11564118f9ade8808c4ccff2e0eeaab
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
c9b41ca88179
test: use `printenv.py --line` in `test-ssh-bundle1.t`

This makes the output easier to update and read.

diff --git a/tests/test-ssh-bundle1.t b/tests/test-ssh-bundle1.t
--- a/tests/test-ssh-bundle1.t
+++ b/tests/test-ssh-bundle1.t
@@ -46,7 +46,7 @@ configure for serving
   > uncompressed = True
   > 
   > [hooks]
-  > changegroup = sh -c "printenv.py changegroup-in-remote 0 ../dummylog"
+  > changegroup = sh -c "printenv.py --line changegroup-in-remote 0 
../dummylog"
   > EOF
   $ cd $TESTTMP
 
@@ -131,7 +131,7 @@ verify
   checked 3 changesets with 2 changes to 2 files
   $ cat >> .hg/hgrc < [hooks]
-  > changegroup = sh -c "printenv.py changegroup-in-local 0 ../dummylog"
+  > changegroup = sh -c "printenv.py --line changegroup-in-local 0 ../dummylog"
   > EOF
 
 empty default pull
@@ -514,7 +514,14 @@ debug output
   Got arguments 1:user@dummy 2:hg -R local serve --stdio
   Got arguments 1:user@dummy 2:hg -R $TESTTMP/local serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=a28a9d1a809cab7d4e2fde4bee738a9ede948b60 
HG_NODE_LAST=a28a9d1a809cab7d4e2fde4bee738a9ede948b60 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=a28a9d1a809cab7d4e2fde4bee738a9ede948b60
+  HG_NODE_LAST=a28a9d1a809cab7d4e2fde4bee738a9ede948b60
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
@@ -524,7 +531,14 @@ debug output
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6 
HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=1383141674ec756a6056f6a9097618482fe0f4a6
+  HG_NODE_LAST=1383141674ec756a6056f6a9097618482fe0f4a6
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
   Got arguments 1:user@dummy 2:hg init 'a repo'
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
@@ -532,7 +546,14 @@ debug output
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
   Got arguments 1:user@dummy 2:hg -R 'a repo' serve --stdio
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
-  changegroup-in-remote hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=65c38f4125f9602c8db4af56530cc221d93b8ef8 
HG_NODE_LAST=65c38f4125f9602c8db4af56530cc221d93b8ef8 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:ssh:$LOCALIP
+  changegroup-in-remote hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=65c38f4125f9602c8db4af56530cc221d93b8ef8
+  HG_NODE_LAST=65c38f4125f9602c8db4af56530cc221d93b8ef8
+  HG_SOURCE=serve
+  HG_TXNID=TXN:$ID$
+  HG_URL=remote:ssh:$LOCALIP
+  
   Got arguments 1:user@dummy 2:hg -R remote serve --stdio
 
 remote hook failure is attributed to remote
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10] test: use `printenv.py --line` in `test-push-http.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671809 -3600
#  Sat Feb 09 01:23:29 2019 +0100
# Node ID 57b13716b11564118f9ade8808c4ccff2e0eeaab
# Parent  a49388cc4f239e28d47a9f61992a58957e8e2bca
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
57b13716b115
test: use `printenv.py --line` in `test-push-http.t`

This makes the output easier to update and read.

diff --git a/tests/test-push-http.t b/tests/test-push-http.t
--- a/tests/test-push-http.t
+++ b/tests/test-push-http.t
@@ -74,8 +74,8 @@ expect success
   $ cat >> .hg/hgrc < allow_push = *
   > [hooks]
-  > changegroup = sh -c "printenv.py changegroup 0"
-  > pushkey = sh -c "printenv.py pushkey 0"
+  > changegroup = sh -c "printenv.py --line changegroup 0"
+  > pushkey = sh -c "printenv.py --line pushkey 0"
   > txnclose-phase.test = sh $TESTTMP/hook.sh 
   > EOF
   $ req "--debug --config extensions.blackbox="
@@ -94,8 +94,15 @@ expect success
   remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b:  draft -> 
public
   remote: running hook txnclose-phase.test: sh $TESTTMP/hook.sh
   remote: phase-move: ba677d0156c1196c1a699fa53f390dcfc3ce3872:   -> public
-  remote: running hook changegroup: sh -c "printenv.py changegroup 0"
-  remote: changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872 
HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:http:$LOCALIP: (glob)
+  remote: running hook changegroup: sh -c "printenv.py --line changegroup 0"
+  remote: changegroup hook: HG_HOOKNAME=changegroup
+  remote: HG_HOOKTYPE=changegroup
+  remote: HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_SOURCE=serve
+  remote: HG_TXNID=TXN:$ID$
+  remote: HG_URL=remote:http:$LOCALIP: (glob)
+  remote: 
   % serve errors
   $ hg rollback
   repository tip rolled back to revision 0 (undo serve)
@@ -114,8 +121,15 @@ expect success
   remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b:  draft -> 
public
   remote: running hook txnclose-phase.test: sh $TESTTMP/hook.sh
   remote: phase-move: ba677d0156c1196c1a699fa53f390dcfc3ce3872:   -> public
-  remote: running hook changegroup: sh -c "printenv.py changegroup 0"
-  remote: changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872 
HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:http:$LOCALIP: (glob)
+  remote: running hook changegroup: sh -c "printenv.py --line changegroup 0"
+  remote: changegroup hook: HG_HOOKNAME=changegroup
+  remote: HG_HOOKTYPE=changegroup
+  remote: HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_SOURCE=serve
+  remote: HG_TXNID=TXN:$ID$
+  remote: HG_URL=remote:http:$LOCALIP: (glob)
+  remote: 
   % serve errors
   $ hg rollback
   repository tip rolled back to revision 0 (undo serve)
@@ -125,8 +139,8 @@ expect success
   $ cat >> .hg/hgrc < allow_push = *
   > [hooks]
-  > changegroup = sh -c "printenv.py changegroup 0"
-  > pushkey = sh -c "printenv.py pushkey 0"
+  > changegroup = sh -c "printenv.py --line changegroup 0"
+  > pushkey = sh -c "printenv.py --line pushkey 0"
   > txnclose-phase.test = sh $TESTTMP/hook.sh 
   > EOF
   $ req
@@ -138,7 +152,15 @@ expect success
   remote: added 1 changesets with 1 changes to 1 files
   remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b:  draft -> 
public
   remote: phase-move: ba677d0156c1196c1a699fa53f390dcfc3ce3872:   -> public
-  remote: changegroup hook: HG_BUNDLE2=1 HG_HOOKNAME=changegroup 
HG_HOOKTYPE=changegroup HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872 
HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872 HG_SOURCE=serve 
HG_TXNID=TXN:$ID$ HG_URL=remote:http:$LOCALIP: (glob)
+  remote: changegroup hook: HG_BUNDLE2=1
+  remote: HG_HOOKNAME=changegroup
+  remote: HG_HOOKTYPE=changegroup
+  remote: HG_NODE=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_NODE_LAST=ba677d0156c1196c1a699fa53f390dcfc3ce3872
+  remote: HG_SOURCE=serve
+  remote: HG_TXNID=TXN:$ID$
+  remote: HG_URL=remote:http:$LOCALIP: (glob)
+  remote: 
   % serve errors
   $ hg rollback
   repository tip rolled back to revision 0 (undo serve)
@@ -157,8 +179,16 @@ expect success, server lacks the httphea
   remote: added 1 changesets with 1 changes to 1 files
   remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b:  draft -> 
public
   remote: phase-move: ba677d0156c1196c1a699fa53f390dcfc3ce3872:   -> public
-  remote

[PATCH 05 of 10] test: use `printenv.py --line` in `test-https.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671740 -3600
#  Sat Feb 09 01:22:20 2019 +0100
# Node ID a49388cc4f239e28d47a9f61992a58957e8e2bca
# Parent  558d98d70bd179e79d8e24510933425062118ace
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
a49388cc4f23
test: use `printenv.py --line` in `test-https.t`

This makes the output easier to update and read.

diff --git a/tests/test-https.t b/tests/test-https.t
--- a/tests/test-https.t
+++ b/tests/test-https.t
@@ -207,7 +207,7 @@ pull without cacert
   $ cd copy-pull
   $ cat >> .hg/hgrc < [hooks]
-  > changegroup = sh -c "printenv.py changegroup"
+  > changegroup = sh -c "printenv.py --line changegroup"
   > EOF
   $ hg pull $DISABLECACERTS
   pulling from https://localhost:$HGPORT/
@@ -226,7 +226,14 @@ pull without cacert
   adding file changes
   added 1 changesets with 1 changes to 1 files
   new changesets 5fed3813f7f5
-  changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d 
HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_SOURCE=pull 
HG_TXNID=TXN:$ID$ HG_URL=https://localhost:$HGPORT/
+  changegroup hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d
+  HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=https://localhost:$HGPORT/
+  
   (run 'hg update' to get a working copy)
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 10] test: use `printenv.py --line` in `test-hook.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671603 -3600
#  Sat Feb 09 01:20:03 2019 +0100
# Node ID 6179e9a9970f23c881187c71705cfe7347ed5c08
# Parent  c0b16be6b9b2d9535be8f1f2c91c5a8ebffa0b03
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
6179e9a9970f
test: use `printenv.py --line` in `test-hook.t`

This makes the output easier to update and read.

diff --git a/tests/test-hook.t b/tests/test-hook.t
--- a/tests/test-hook.t
+++ b/tests/test-hook.t
@@ -14,32 +14,63 @@ commit hooks can see env vars
   $ cd a
   $ cat > .hg/hgrc < [hooks]
-  > commit = sh -c "HG_LOCAL= HG_TAG= printenv.py commit"
-  > commit.b = sh -c "HG_LOCAL= HG_TAG= printenv.py commit.b"
-  > precommit = sh -c  "HG_LOCAL= HG_NODE= HG_TAG= printenv.py precommit"
-  > pretxncommit = sh -c "HG_LOCAL= HG_TAG= printenv.py pretxncommit"
+  > commit = sh -c "HG_LOCAL= HG_TAG= printenv.py --line commit"
+  > commit.b = sh -c "HG_LOCAL= HG_TAG= printenv.py --line commit.b"
+  > precommit = sh -c  "HG_LOCAL= HG_NODE= HG_TAG= printenv.py --line 
precommit"
+  > pretxncommit = sh -c "HG_LOCAL= HG_TAG= printenv.py --line pretxncommit"
   > pretxncommit.tip = hg -q tip
-  > pre-identify = sh -c "printenv.py pre-identify 1"
-  > pre-cat = sh -c "printenv.py pre-cat"
-  > post-cat = sh -c "printenv.py post-cat"
-  > pretxnopen = sh -c "HG_LOCAL= HG_TAG= printenv.py pretxnopen"
-  > pretxnclose = sh -c "HG_LOCAL= HG_TAG= printenv.py pretxnclose"
-  > txnclose = sh -c "HG_LOCAL= HG_TAG= printenv.py txnclose"
+  > pre-identify = sh -c "printenv.py --line pre-identify 1"
+  > pre-cat = sh -c "printenv.py --line pre-cat"
+  > post-cat = sh -c "printenv.py --line post-cat"
+  > pretxnopen = sh -c "HG_LOCAL= HG_TAG= printenv.py --line pretxnopen"
+  > pretxnclose = sh -c "HG_LOCAL= HG_TAG= printenv.py --line pretxnclose"
+  > txnclose = sh -c "HG_LOCAL= HG_TAG= printenv.py --line txnclose"
   > txnabort.0 = python:$TESTTMP/txnabort.checkargs.py:showargs
-  > txnabort.1 = sh -c "HG_LOCAL= HG_TAG= printenv.py txnabort"
+  > txnabort.1 = sh -c "HG_LOCAL= HG_TAG= printenv.py --line txnabort"
   > txnclose.checklock = sh -c "hg debuglock > /dev/null"
   > EOF
   $ echo a > a
   $ hg add a
   $ hg commit -m a
-  precommit hook: HG_HOOKNAME=precommit HG_HOOKTYPE=precommit 
HG_PARENT1=
-  pretxnopen hook: HG_HOOKNAME=pretxnopen HG_HOOKTYPE=pretxnopen 
HG_TXNID=TXN:$ID$ HG_TXNNAME=commit
-  pretxncommit hook: HG_HOOKNAME=pretxncommit HG_HOOKTYPE=pretxncommit 
HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b 
HG_PARENT1= HG_PENDING=$TESTTMP/a
+  precommit hook: HG_HOOKNAME=precommit
+  HG_HOOKTYPE=precommit
+  HG_PARENT1=
+  
+  pretxnopen hook: HG_HOOKNAME=pretxnopen
+  HG_HOOKTYPE=pretxnopen
+  HG_TXNID=TXN:$ID$
+  HG_TXNNAME=commit
+  
+  pretxncommit hook: HG_HOOKNAME=pretxncommit
+  HG_HOOKTYPE=pretxncommit
+  HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b
+  HG_PARENT1=
+  HG_PENDING=$TESTTMP/a
+  
   0:cb9a9f314b8b
-  pretxnclose hook: HG_HOOKNAME=pretxnclose HG_HOOKTYPE=pretxnclose 
HG_PENDING=$TESTTMP/a HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=commit
-  txnclose hook: HG_HOOKNAME=txnclose HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 
HG_TXNID=TXN:$ID$ HG_TXNNAME=commit
-  commit hook: HG_HOOKNAME=commit HG_HOOKTYPE=commit 
HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b 
HG_PARENT1=
-  commit.b hook: HG_HOOKNAME=commit.b HG_HOOKTYPE=commit 
HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b 
HG_PARENT1=
+  pretxnclose hook: HG_HOOKNAME=pretxnclose
+  HG_HOOKTYPE=pretxnclose
+  HG_PENDING=$TESTTMP/a
+  HG_PHASES_MOVED=1
+  HG_TXNID=TXN:$ID$
+  HG_TXNNAME=commit
+  
+  txnclose hook: HG_HOOKNAME=txnclose
+  HG_HOOKTYPE=txnclose
+  HG_PHASES_MOVED=1
+  HG_TXNID=TXN:$ID$
+  HG_TXNNAME=commit
+  
+  commit hook: HG_HOOKNAME=commit
+  HG_HOOKTYPE=commit
+  HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b
+  HG_PARENT1=
+  
+  commit.b hook: HG_HOOKNAME=commit.b
+  HG_HOOKTYPE=commit
+  HG_NODE=cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b
+  HG_PARENT1=
+  
 
   $ hg clone . ../b
   updating to branch default
@@ -50,9 +81,9 @@ changegroup hooks can see env vars
 
   $ cat > .hg/hgrc < [hooks]
-  > prechangegroup = sh -c "printenv.py prechangegroup"
-  > changegroup = sh -c "printenv.py changegr

[PATCH 02 of 10] test: use `printenv.py --line` in `test-bundle2-multiple-changegroups.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671583 -3600
#  Sat Feb 09 01:19:43 2019 +0100
# Node ID c0b16be6b9b2d9535be8f1f2c91c5a8ebffa0b03
# Parent  1736fe7d07d8a96349dc678f7a7bcba29886d1a7
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
c0b16be6b9b2
test: use `printenv.py --line` in `test-bundle2-multiple-changegroups.t`

This makes the output easier to update and read.

diff --git a/tests/test-bundle2-multiple-changegroups.t 
b/tests/test-bundle2-multiple-changegroups.t
--- a/tests/test-bundle2-multiple-changegroups.t
+++ b/tests/test-bundle2-multiple-changegroups.t
@@ -66,9 +66,9 @@ Add two linear commits
   $ cd ../clone
   $ cat >> .hg/hgrc < [hooks]
-  > pretxnchangegroup = sh -c "printenv.py pretxnchangegroup"
-  > changegroup = sh -c "printenv.py changegroup"
-  > incoming = sh -c "printenv.py incoming"
+  > pretxnchangegroup = sh -c "printenv.py --line pretxnchangegroup"
+  > changegroup = sh -c "printenv.py --line changegroup"
+  > incoming = sh -c "printenv.py --line incoming"
   > EOF
 
 Pull the new commits in the clone
@@ -81,18 +81,63 @@ Pull the new commits in the clone
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
-  pretxnchangegroup hook: HG_HOOKNAME=pretxnchangegroup 
HG_HOOKTYPE=pretxnchangegroup HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56 
HG_NODE_LAST=27547f69f25460a52fff66ad004e58da7ad3fb56 HG_PENDING=$TESTTMP/clone 
HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
+  pretxnchangegroup hook: HG_HOOKNAME=pretxnchangegroup
+  HG_HOOKTYPE=pretxnchangegroup
+  HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56
+  HG_NODE_LAST=27547f69f25460a52fff66ad004e58da7ad3fb56
+  HG_PENDING=$TESTTMP/clone
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
   remote: changegroup2
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
-  pretxnchangegroup hook: HG_HOOKNAME=pretxnchangegroup 
HG_HOOKTYPE=pretxnchangegroup HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757 
HG_NODE_LAST=f838bfaca5c7226600ebcfd84f3c3c13a28d3757 HG_PENDING=$TESTTMP/clone 
HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
+  pretxnchangegroup hook: HG_HOOKNAME=pretxnchangegroup
+  HG_HOOKTYPE=pretxnchangegroup
+  HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757
+  HG_NODE_LAST=f838bfaca5c7226600ebcfd84f3c3c13a28d3757
+  HG_PENDING=$TESTTMP/clone
+  HG_PHASES_MOVED=1
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
   new changesets 27547f69f254:f838bfaca5c7
-  changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56 
HG_NODE_LAST=27547f69f25460a52fff66ad004e58da7ad3fb56 HG_SOURCE=pull 
HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
-  incoming hook: HG_HOOKNAME=incoming HG_HOOKTYPE=incoming 
HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56 HG_SOURCE=pull 
HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
-  changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757 
HG_NODE_LAST=f838bfaca5c7226600ebcfd84f3c3c13a28d3757 HG_PHASES_MOVED=1 
HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
-  incoming hook: HG_HOOKNAME=incoming HG_HOOKTYPE=incoming 
HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757 HG_PHASES_MOVED=1 
HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_URL=file:$TESTTMP/repo
+  changegroup hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56
+  HG_NODE_LAST=27547f69f25460a52fff66ad004e58da7ad3fb56
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
+  incoming hook: HG_HOOKNAME=incoming
+  HG_HOOKTYPE=incoming
+  HG_NODE=27547f69f25460a52fff66ad004e58da7ad3fb56
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
+  changegroup hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757
+  HG_NODE_LAST=f838bfaca5c7226600ebcfd84f3c3c13a28d3757
+  HG_PHASES_MOVED=1
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
+  incoming hook: HG_HOOKNAME=incoming
+  HG_HOOKTYPE=incoming
+  HG_NODE=f838bfaca5c7226600ebcfd84f3c3c13a28d3757
+  HG_PHASES_MOVED=1
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=file:$TESTTMP/repo
+  
   pullop.cgresult is 1
   (run 'hg update' to get a working copy)
   $ hg update
@@ -152,21 +197,86 @@ pullop.cgresult
   adding manifests
   adding file changes
   added 2 changesets with 2 changes to 2 files (+1 heads)
-  pretxnchangegroup hook: HG_HOOKNAME=pretxnchangegroup 
HG_HOOKTYPE=pretxnchangegroup HG_NODE=b3325c91a4d916bcc4cdc83ea3fe4ece46a42f6e 
HG_NODE_LAST=8a5212ebc8527f9fb821601504794e3eb11a1ed3 HG_PENDING=$TESTTMP/c

[PATCH 04 of 10] test: use `printenv.py --line` in `test-http-bundle1.t`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549671635 -3600
#  Sat Feb 09 01:20:35 2019 +0100
# Node ID 558d98d70bd179e79d8e24510933425062118ace
# Parent  6179e9a9970f23c881187c71705cfe7347ed5c08
# EXP-Topic hook.txname
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
558d98d70bd1
test: use `printenv.py --line` in `test-http-bundle1.t`

This makes the output easier to update and read.

diff --git a/tests/test-http-bundle1.t b/tests/test-http-bundle1.t
--- a/tests/test-http-bundle1.t
+++ b/tests/test-http-bundle1.t
@@ -151,7 +151,7 @@ pull
   $ cd copy-pull
   $ cat >> .hg/hgrc < [hooks]
-  > changegroup = sh -c "printenv.py changegroup"
+  > changegroup = sh -c "printenv.py --line changegroup"
   > EOF
   $ hg pull
   pulling from http://localhost:$HGPORT1/
@@ -161,7 +161,14 @@ pull
   adding file changes
   added 1 changesets with 1 changes to 1 files
   new changesets 5fed3813f7f5
-  changegroup hook: HG_HOOKNAME=changegroup HG_HOOKTYPE=changegroup 
HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d 
HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_SOURCE=pull 
HG_TXNID=TXN:$ID$ HG_URL=http://localhost:$HGPORT1/
+  changegroup hook: HG_HOOKNAME=changegroup
+  HG_HOOKTYPE=changegroup
+  HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d
+  HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d
+  HG_SOURCE=pull
+  HG_TXNID=TXN:$ID$
+  HG_URL=http://localhost:$HGPORT1/
+  
   (run 'hg update' to get a working copy)
   $ cd ..
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5897: test: stabilize test-wireproto-exchangev2.t flaky output

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When running the test suite with multiple processes, we often get flaky
  outputs, like here: https://ci.octobus.net/job/MercurialPy2/267/console
  
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   add changeset cd2534766bec
   add changeset e96ae20f4188
   add changeset caa2a465451d
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
  
  Instead of simply sorting the clone and pull output, I saved the output in a
  separate file and checked the `received frame` messages on one side and then
  the rest of the output on the other side. This way we conserve the order of
  messages as it seems important.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5897

AFFECTED FILES
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -36,22 +36,18 @@
 
 Test basic clone
 
-  $ hg --debug clone -U http://localhost:$HGPORT client-simple
+Output is flaky, save it in a file and check part independently
+  $ hg --debug clone -U http://localhost:$HGPORT client-simple > clone-output
+
+  $ cat clone-output | grep -v "received frame"
   using http://localhost:$HGPORT/
   sending capabilities command
   query 1; heads
   sending 2 commands
   sending command heads: {}
   sending command known: {
 'nodes': []
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=43; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
-  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
   sending 1 commands
   sending command changesetdata: {
 'fields': set([
@@ -71,10 +67,6 @@
   }
 ]
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=941; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   add changeset 3390ef850073
   add changeset 4432d83626e8
   add changeset cd2534766bec
@@ -97,10 +89,6 @@
 ],
 'tree': ''
   }
-  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
-  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=992; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
-  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
   sending 1 commands
   sending command filesdata: {
 'fields': set([
@@ -121,13 +109,32 @@
   }
 ]
   }
+  updating the branch cache
+  new changesets 3390ef850073:caa2a465451d (3 drafts)
+  (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob)
+
+  $ cat clone-output | grep "received frame"
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=43; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=941; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; 

D5880: fsmonitor: rename new verbose config knob

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG286eeed14893: fsmonitor: rename new verbose config knob 
(authored by lothiraldan, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5880?vs=13890=13891#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5880?vs=13890=13891

REVISION DETAIL
  https://phab.mercurial-scm.org/D5880

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,7 +161,7 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
-configitem('hgwatchman', 'verbose',
+configitem('fsmonitor', 'verbose',
 default=True,
 )
 configitem('experimental', 'fsmonitor.transaction_notify',
@@ -175,13 +175,13 @@
 def _handleunavailable(ui, state, ex):
 """Exception handler for Watchman interaction exceptions"""
 if isinstance(ex, watchmanclient.Unavailable):
-# experimental config: hgwatchman.verbose
-if ex.warn and ui.configbool('hgwatchman', 'verbose'):
+# experimental config: fsmonitor.verbose
+if ex.warn and ui.configbool('fsmonitor', 'verbose'):
 ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
-# experimental config: hgwatchman.verbose
-if ui.configbool('hgwatchman','verbose'):
+# experimental config: fsmonitor.verbose
+if ui.configbool('fsmonitor', 'verbose'):
 ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
 else:
 ui.log('fsmonitor', 'Watchman exception: %s\n', ex)



To: lothiraldan, #hg-reviewers, pulkit
Cc: pulkit, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5880: fsmonitor: rename new verbose config knob

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  Nice catch, sorry for the hasty phabsend

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5880

To: lothiraldan, #hg-reviewers
Cc: pulkit, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5880: fsmonitor: rename new verbose config knob

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13890.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5880?vs=13889=13890

REVISION DETAIL
  https://phab.mercurial-scm.org/D5880

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,7 +161,7 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
-configitem('hgwatchman', 'verbose',
+configitem('fsmonitor', 'verbose',
 default=True,
 )
 configitem('experimental', 'fsmonitor.transaction_notify',
@@ -176,12 +176,12 @@
 """Exception handler for Watchman interaction exceptions"""
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: hgwatchman.verbose
-if ex.warn and ui.configbool('hgwatchman', 'verbose'):
+if ex.warn and ui.configbool('fsmonitor', 'verbose'):
 ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
-# experimental config: hgwatchman.verbose
-if ui.configbool('hgwatchman','verbose'):
+# experimental config: fsmonitor.verbose
+if ui.configbool('fsmonitor', 'verbose'):
 ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
 else:
 ui.log('fsmonitor', 'Watchman exception: %s\n', ex)



To: lothiraldan, #hg-reviewers
Cc: pulkit, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5880: fsmonitor: rename new verbose config knob

[lothiraldan (Boris Feld)]

lothiraldan created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The config knob was introduced in this release cycle under the old extension
  name, rename it before it is part of a release.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5880

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,7 +161,7 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
-configitem('hgwatchman', 'verbose',
+configitem('fsmonitor', 'verbose',
 default=True,
 )
 configitem('experimental', 'fsmonitor.transaction_notify',



To: lothiraldan, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] py3: use integer division instead of `int(...)` call

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1549184503 -3600
#  Sun Feb 03 10:01:43 2019 +0100
# Node ID 23e2af1929c66ab34e9f09bb2caec298e9a8e71c
# Parent  9955776e2adf36c5baabad8f73840a0cdfa19589
# EXP-Topic div-cleanup
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
23e2af1929c6
py3: use integer division instead of `int(...)` call

Changeset 38a82e0333c9 and 7f853549823b introduced explicit conversion to
integer to work around the division behavior change from python2 to python3.
Using the integer division operator is a simpler and clearer way to achieve
this.

diff --git a/hgext/remotefilelog/datapack.py b/hgext/remotefilelog/datapack.py
--- a/hgext/remotefilelog/datapack.py
+++ b/hgext/remotefilelog/datapack.py
@@ -242,8 +242,8 @@ class datapack(basepack.basepack):
 entry = index[end:end + entrylen]
 else:
 while start < end - entrylen:
-mid = start  + (end - start) / 2
-mid = int(mid - ((mid - params.indexstart) % entrylen))
+mid = start  + (end - start) // 2
+mid = mid - ((mid - params.indexstart) % entrylen)
 midnode = index[mid:mid + NODELENGTH]
 if midnode == node:
 entry = index[mid:mid + entrylen]
diff --git a/tests/test-remotefilelog-datapack.py 
b/tests/test-remotefilelog-datapack.py
--- a/tests/test-remotefilelog-datapack.py
+++ b/tests/test-remotefilelog-datapack.py
@@ -292,7 +292,7 @@ class datapacktestsbase(object):
 
 class testdatapackstore(datapack.datapackstore):
 # Ensures that we are not keeping everything in the cache.
-DEFAULTCACHESIZE = int(numpacks / 2)
+DEFAULTCACHESIZE = numpacks // 2
 
 store = testdatapackstore(uimod.ui(), packdir)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5587: watchman: detect nested mercurial repositories and abort

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6a7b2d2de124: watchman: detect nested mercurial 
repositories and abort (authored by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5587?vs=13412=13645

REVISION DETAIL
  https://phab.mercurial-scm.org/D5587

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -367,6 +367,9 @@
 fexists = entry['exists']
 kind = getkind(fmode)
 
+if '/.hg/' in fname or fname.endswith('/.hg'):
+return bail('nested-repo-detected')
+
 if not fexists:
 # if marked as deleted and we don't already have a change
 # record, mark it as deleted.  If we already have an entry
@@ -740,6 +743,14 @@
 repo, node, branchmerge, force, ancestor, mergeancestor,
 labels, matcher, **kwargs)
 
+def repo_has_depth_one_nested_repo(repo):
+for f in repo.wvfs.listdir():
+if os.path.isdir(os.path.join(repo.root, f, '.hg')):
+msg = 'fsmonitor: sub-repository %r detected, fsmonitor disabled\n'
+repo.ui.debug(msg % f)
+return True
+return False
+
 def reposetup(ui, repo):
 # We don't work with largefiles or inotify
 exts = extensions.enabled()
@@ -757,6 +768,9 @@
 if repo.wvfs.exists('.hgsubstate') or repo.wvfs.exists('.hgsub'):
 return
 
+if repo_has_depth_one_nested_repo(repo):
+return
+
 fsmonitorstate = state.state(repo)
 if fsmonitorstate.mode == 'off':
 return



To: lothiraldan, #hg-reviewers, indygreg
Cc: mjpieters, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5586: watchman: add verbose config knob

[lothiraldan (Boris Feld)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG158962bf4a52: watchman: add verbose config knob (authored 
by lothiraldan, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5586?vs=13411=13644

REVISION DETAIL
  https://phab.mercurial-scm.org/D5586

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('hgwatchman', 'verbose',
+default=True,
+)
 configitem('experimental', 'fsmonitor.transaction_notify',
 default=False,
 )
@@ -172,11 +175,14 @@
 def _handleunavailable(ui, state, ex):
 """Exception handler for Watchman interaction exceptions"""
 if isinstance(ex, watchmanclient.Unavailable):
-if ex.warn:
+# experimental config: hgwatchman.verbose
+if ex.warn and ui.configbool('hgwatchman', 'verbose'):
 ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
-ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
+# experimental config: hgwatchman.verbose
+if ui.configbool('hgwatchman','verbose'):
+ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
 else:
 ui.log('fsmonitor', 'Watchman exception: %s\n', ex)
 



To: lothiraldan, #hg-reviewers
Cc: pulkit, indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] revlog: make sure we never use sparserevlog without general delta (issue6056)

[Boris FELD]

On 30/01/2019 17:43, Yuya Nishihara wrote:
> On Wed, 30 Jan 2019 14:18:29 -0500, Boris FELD wrote:
>> On 29/01/2019 07:11, Yuya Nishihara wrote:
>>> On Tue, 29 Jan 2019 09:25:34 +0100, Mathias De Maré wrote:
>>>> On Mon, Jan 28, 2019 at 1:21 PM Yuya Nishihara  wrote:
>>>>> On Fri, 25 Jan 2019 15:00:52 -0500, Boris Feld wrote:
>>>>>> # HG changeset patch
>>>>>> # User Boris Feld 
>>>>>> # Date 1548372167 18000
>>>>>> #  Thu Jan 24 18:22:47 2019 -0500
>>>>>> # Branch stable
>>>>>> # Node ID c7661fdcaeac457aa5b8ae6b6a10b259713934b9
>>>>>> # Parent  26ee61c33dee366576a0f7266fea9834985e78ec
>>>>>> # EXP-Topic issue6056
>>>>>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>>>>>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/
>>>>> -r c7661fdcaeac
>>>>>> revlog: make sure we never use sparserevlog without general delta
>>>>> (issue6056)
>>>>>> We are getting user report where the delta code tries to use
>>>>> `sparse-revlog`
>>>>>> logic on repository where `generaldelta` is disabled. This can't work so
>>>>> we
>>>>>> ensure the two booleans have a consistent value.
>>>>>>
>>>>>> Creating this kind of repository is not expected to be possible and we
>>>>> weren't
>>>>>> able to create one for testing purpose. But if users run into this, we
>>>>> better
>>>>>> put stronger protection.
>>>>> Mathias, do you have any idea how the broken repository could be created?
>>>>>
>>>> I think the issue might be related to our use of streaming clonebundles and
>>>> a mismatch between some repositories on our server.
>>>> For one specific repository, I see on the server that .hg/requires contains
>>>> "revlogv1 store fncache dotencode".
>>>> On the client that cloned (and cannot commit), the requires contains:
>>>> "dotencode fncache generaldelta revlogv1 sparserevlog store"
>>> That makes sense, thanks.
>>>
>>> The problem could be reproduced by adding --config format.usegeneraldelta=0
>>> --config format.sparse-revlog=0 to test-clonebundles.t.
>> Where are you adding this? We could not reproduce on our side.
> Added it to "hg init server", and some repository mutation to clones.
Do you have an exact patch to share? Adding it to `init` only was not
enough.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 STABLE V2 STABLE] revlog: make sure we never use sparserevlog without general delta (issue6056)

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548372167 18000
#  Thu Jan 24 18:22:47 2019 -0500
# Branch stable
# Node ID 8e4137a7d6715c554f3689d48fd1c4b3101985ff
# Parent  cf5421ca2bea707ce52fc96f5ca68cf8f4894e8c
# EXP-Topic issue6056
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
8e4137a7d671
revlog: make sure we never use sparserevlog without general delta (issue6056)

We are getting user report where the delta code tries to use `sparse-revlog`
logic on repository where `generaldelta` is disabled. This can't work so we
ensure the two booleans have a consistent value.

Creating this kind of repository is not expected to be possible the current bug
report point at a clonebundle related bug that is still to be properly isolated
(Yuya Nishihara seems to a have done it).

Corrupting a repository to reproduce the issue is possible. A test using this
method is included in this fix.

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -497,6 +497,9 @@ class revlog(object):
 else:
 raise error.RevlogError(_('unknown version (%d) in revlog %s') %
 (fmt, self.indexfile))
+# sparse-revlog can't be on without general-delta (issue6056)
+if not self._generaldelta:
+self._sparserevlog = False
 
 self._storedeltachains = True
 
diff --git a/mercurial/revlogutils/deltas.py b/mercurial/revlogutils/deltas.py
--- a/mercurial/revlogutils/deltas.py
+++ b/mercurial/revlogutils/deltas.py
@@ -755,7 +755,8 @@ def _rawgroups(revlog, p1, p2, cachedelt
 The group order aims at providing fast or small candidates first.
 """
 gdelta = revlog._generaldelta
-sparse = revlog._sparserevlog
+# gate sparse behind general-delta because of issue-6056
+sparse = gdelta and revlog._sparserevlog
 curr = len(revlog)
 prev = curr - 1
 deltachain = lambda rev: revlog._deltachain(rev)[0]
diff --git a/tests/test-init.t b/tests/test-init.t
--- a/tests/test-init.t
+++ b/tests/test-init.t
@@ -268,3 +268,26 @@ clone bookmarks
   exporting bookmark test
   $ hg -R remote-bookmarks bookmarks
  test  0:08b9e9f63b32
+
+Check format constraint
+---
+
+
+  $ hg init issue6056 --config format.usegeneraldelta=0 --config 
format.sparse-revlog=0
+  $ cd issue6056
+  $ echo foo > 1
+  $ echo foo > 2
+  $ echo foo > 3
+  $ echo foo > 4
+  $ echo foo > 5
+  $ hg add *
+
+Build a bogus repository (sparserevlog without general delta)
+
+  $ hg commit -m 'initial changesets'
+  $ echo 'sparserevlog' >> .hg/requires
+  $ for x in `$TESTDIR/seq.py 100`; do
+  > echo $x >> `expr $x % 5 + 1`
+  > hg commit -m $x
+  > done
+  $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 STABLE V2 STABLE] sparserevlog: document the config option

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548868538 -3600
#  Wed Jan 30 18:15:38 2019 +0100
# Branch stable
# Node ID cf5421ca2bea707ce52fc96f5ca68cf8f4894e8c
# Parent  ab0d762d89ef6e2fdcf2a4cc31e9889ea04bc13b
# EXP-Topic issue6056
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
cf5421ca2bea
sparserevlog: document the config option

This was overlooked when this graduated from experimental.

diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -857,6 +857,16 @@ https://www.mercurial-scm.org/wiki/Missi
 
 Enabled by default.
 
+``sparse-revlog``
+Enable of disable the ``sparse-revlog`` delta strategy. This format 
improves
+delta re-use inside revlog. For very branchy repostiories, it result in a
+smaller store. For repositories with many revisions it also help 
performance
+(by using shorted delta chains)
+
+Repositories with this on-disk format require Mercurial version 4.7
+
+Enabled by default.
+
 ``graph``
 -
 
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -1502,6 +1502,8 @@ Separate sections from subsections
   
   "usestore"
   
+  "sparse-revlog"
+  
   "profiling"
   ---
   
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] revlog: make sure we never use sparserevlog without general delta (issue6056)

[Boris FELD]

On 29/01/2019 07:11, Yuya Nishihara wrote:
> On Tue, 29 Jan 2019 09:25:34 +0100, Mathias De Maré wrote:
>> On Mon, Jan 28, 2019 at 1:21 PM Yuya Nishihara  wrote:
>>> On Fri, 25 Jan 2019 15:00:52 -0500, Boris Feld wrote:
>>>> # HG changeset patch
>>>> # User Boris Feld 
>>>> # Date 1548372167 18000
>>>> #  Thu Jan 24 18:22:47 2019 -0500
>>>> # Branch stable
>>>> # Node ID c7661fdcaeac457aa5b8ae6b6a10b259713934b9
>>>> # Parent  26ee61c33dee366576a0f7266fea9834985e78ec
>>>> # EXP-Topic issue6056
>>>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>>>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/
>>> -r c7661fdcaeac
>>>> revlog: make sure we never use sparserevlog without general delta
>>> (issue6056)
>>>> We are getting user report where the delta code tries to use
>>> `sparse-revlog`
>>>> logic on repository where `generaldelta` is disabled. This can't work so
>>> we
>>>> ensure the two booleans have a consistent value.
>>>>
>>>> Creating this kind of repository is not expected to be possible and we
>>> weren't
>>>> able to create one for testing purpose. But if users run into this, we
>>> better
>>>> put stronger protection.
>>> Mathias, do you have any idea how the broken repository could be created?
>>>
>> I think the issue might be related to our use of streaming clonebundles and
>> a mismatch between some repositories on our server.
>> For one specific repository, I see on the server that .hg/requires contains
>> "revlogv1 store fncache dotencode".
>> On the client that cloned (and cannot commit), the requires contains:
>> "dotencode fncache generaldelta revlogv1 sparserevlog store"
> That makes sense, thanks.
>
> The problem could be reproduced by adding --config format.usegeneraldelta=0
> --config format.sparse-revlog=0 to test-clonebundles.t.
Where are you adding this? We could not reproduce on our side.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] revlog: make sure we never use sparserevlog without general delta (issue6056)

[Boris FELD]

On 29/01/2019 13:41, Gregory Szorc wrote:
> On Tue, Jan 29, 2019 at 3:44 AM Yuya Nishihara  <mailto:y...@tcha.org>> wrote:
>
> On Mon, 28 Jan 2019 21:20:07 +0900, Yuya Nishihara wrote:
> > On Fri, 25 Jan 2019 15:00:52 -0500, Boris Feld wrote:
> > > # HG changeset patch
> > > # User Boris Feld  <mailto:boris.f...@octobus.net>>
> > > # Date 1548372167 18000
> > > #      Thu Jan 24 18:22:47 2019 -0500
> > > # Branch stable
> > > # Node ID c7661fdcaeac457aa5b8ae6b6a10b259713934b9
> > > # Parent  26ee61c33dee366576a0f7266fea9834985e78ec
> > > # EXP-Topic issue6056
> > > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > > #              hg pull
> https://bitbucket.org/octobus/mercurial-devel/ -r c7661fdcaeac
> > > revlog: make sure we never use sparserevlog without general
> delta (issue6056)
> > >
> > > We are getting user report where the delta code tries to use
> `sparse-revlog`
> > > logic on repository where `generaldelta` is disabled. This
> can't work so we
> > > ensure the two booleans have a consistent value.
> > >
> > > Creating this kind of repository is not expected to be
> possible and we weren't
> > > able to create one for testing purpose. But if users run into
> this, we better
> > > put stronger protection.
>
> > > --- a/mercurial/revlog.py
> > > +++ b/mercurial/revlog.py
> > > @@ -497,6 +497,9 @@ class revlog(object):
> > >          else:
> > >              raise error.RevlogError(_('unknown version (%d)
> in revlog %s') %
> > >                                      (fmt, self.indexfile))
> > > +        # sparse-revlog can't be on without general-delta
> (issue6056)
> > > +        if not self._generaldelta:
> > > +            self._sparserevlog = False
> >
> > Maybe this can be handled at localrepo.ensurerequirements*() or
> > esolverevlogstorevfsoptions(), where ui is available so we can add
> > some debug/warning messages.
>
> Okay, it doesn't work since generaldelta is a revlog flag, whereas
> sparserevlog
> is a global requirement. I don't know if sparserevlog should be a
> revlog flag
> or not, but anyway only thing we can do here is to turn off the
> sparse, as your
> patch does.
>
>
> Sparse should have a revlog flag in addition to a repo requirement. My
> understanding is we already shipped sparse support in 4.8.
Support is shipped since 4.7.
> This means we can't introduce a revlog flag (and corresponding repo
> requirement) without a BC change. I started a series to introduce
> revlog version 2 (mainly to support zstd compression in revlogs). We
> can do things more properly in revlog version 2. But I fear the lack
> of a sparse revlog flag will just have to be an historical wart since
> we've already shipped it.
>  
>
>
> Can you add some tests with crafted repo?
>
> Also, I noticed format.sparse-revlog isn't documented. Since it's
> incompatible
> with older versions, it needs some doc.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> <mailto:Mercurial-devel@mercurial-scm.org>
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 V2] perf: add a perfnodemap command

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548460545 18000
#  Fri Jan 25 18:55:45 2019 -0500
# Node ID 3414a69b2e99e8fa605f63f9973894c80dd144b3
# Parent  7eb7637e34bf412374a165fbd234022a5a804683
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
3414a69b2e99
perf: add a perfnodemap command

The command focus on timing of the nodemap object itself.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1036,7 +1036,8 @@ def perfindex(ui, repo, **opts):
 * -1:
 * -1: + 0
 
-It is not currently possible to check for lookup of a missing node."""
+It is not currently possible to check for lookup of a missing node. For
+deeper lookup benchmarking, checkout the `perfnodemap` command."""
 import mercurial.revlog
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
@@ -1066,6 +1067,58 @@ def perfindex(ui, repo, **opts):
 timer(d, setup=setup)
 fm.end()
 
+@command(b'perfnodemap', [
+(b'', b'rev', [], b'revision to be looked up (default tip)'),
+] + formatteropts)
+def perfnodemap(ui, repo, **opts):
+"""benchmark the time necessary to look up revision from a cold nodemap
+
+Depending on the implementation, the amount and order of revision we look
+up can varies. Example of useful set to test:
+* tip
+* 0
+* -10:
+* :10
+* -10: + :10
+* :10: + -10:
+* -1:
+* -1: + 0
+
+The command currently focus on valid binary lookup. Benchmarking for
+hexlookup, prefix lookup and missing lookup would also be valuable.
+"""
+import mercurial.revlog
+opts = _byteskwargs(opts)
+timer, fm = gettimer(ui, opts)
+mercurial.revlog._prereadsize = 2**24 # disable lazy parser in old hg
+
+unfi = repo.unfiltered()
+# find the filecache func directly
+# This avoid polluting the benchmark with the filecache logic
+makecl = unfi.__class__.changelog.func
+if not opts[b'rev']:
+raise error.Abort('use --rev to specify revisions to look up')
+revs = scmutil.revrange(repo, opts[b'rev'])
+cl = repo.changelog
+nodes = [cl.node(r) for r in revs]
+
+# use a list to pass reference to a nodemap from one closure to the next
+nodeget = [None]
+def setnodeget():
+# probably not necessary, but for good measure
+clearchangelog(unfi)
+nodeget[0] = makecl(unfi).nodemap.get
+
+def setup():
+setnodeget()
+def d():
+get = nodeget[0]
+for n in nodes:
+get(n)
+
+timer(d, setup=setup)
+fm.end()
+
 @command(b'perfstartup', formatteropts)
 def perfstartup(ui, repo, **opts):
 opts = _byteskwargs(opts)
diff --git a/tests/test-contrib-perf.t b/tests/test-contrib-perf.t
--- a/tests/test-contrib-perf.t
+++ b/tests/test-contrib-perf.t
@@ -109,6 +109,8 @@ perfstatus
perfmoonwalk  benchmark walking the changelog backwards
perfnodelookup
  (no help text available)
+   perfnodemap   benchmark the time necessary to look up revision from a cold
+ nodemap
perfparents   (no help text available)
perfpathcopies
  benchmark the copy tracing logic
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 V2] perf: add a --[no-]clear-caches option to `perfnodemap`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548664893 18000
#  Mon Jan 28 03:41:33 2019 -0500
# Node ID 4ecd8c3fa0df874dcae358025c160b0bbbaf57d1
# Parent  3414a69b2e99e8fa605f63f9973894c80dd144b3
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
4ecd8c3fa0df
perf: add a --[no-]clear-caches option to `perfnodemap`

The option is useful to look at pure lookup performance on a warm data
structure.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1068,7 +1068,8 @@ def perfindex(ui, repo, **opts):
 fm.end()
 
 @command(b'perfnodemap', [
-(b'', b'rev', [], b'revision to be looked up (default tip)'),
+  (b'', b'rev', [], b'revision to be looked up (default tip)'),
+  (b'', b'clear-caches', True, b'clear revlog cache between calls'),
 ] + formatteropts)
 def perfnodemap(ui, repo, **opts):
 """benchmark the time necessary to look up revision from a cold nodemap
@@ -1093,6 +1094,7 @@ def perfnodemap(ui, repo, **opts):
 mercurial.revlog._prereadsize = 2**24 # disable lazy parser in old hg
 
 unfi = repo.unfiltered()
+clearcaches = opts['clear_caches']
 # find the filecache func directly
 # This avoid polluting the benchmark with the filecache logic
 makecl = unfi.__class__.changelog.func
@@ -1109,13 +,18 @@ def perfnodemap(ui, repo, **opts):
 clearchangelog(unfi)
 nodeget[0] = makecl(unfi).nodemap.get
 
-def setup():
-setnodeget()
 def d():
 get = nodeget[0]
 for n in nodes:
 get(n)
 
+setup = None
+if clearcaches:
+def setup():
+setnodeget()
+else:
+setnodeget()
+d() # prewarm the data structure
 timer(d, setup=setup)
 fm.end()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 7] perf: add a perfnodemap command

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548460545 18000
#  Fri Jan 25 18:55:45 2019 -0500
# Node ID 884df78c77a1c3b11ca2e5728ae6b5ebd7f6d2aa
# Parent  1e135ddcbdd182a6b7ae7b121d254fb58c7fd658
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
884df78c77a1
perf: add a perfnodemap command

The command focus on timing of the nodemap object itself.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1038,7 +1038,8 @@ def perfindex(ui, repo, **opts):
 * -1:
 * -1: + 0
 
-It is not currently possible to check for lookup of a missing node."""
+It is not currently possible to check for lookup of a missing node. For
+deeper lookup benchmarking, checkout the `perfnodemap` command."""
 import mercurial.revlog
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
@@ -1068,6 +1069,58 @@ def perfindex(ui, repo, **opts):
 timer(d, setup=setup)
 fm.end()
 
+@command(b'perfnodemap', [
+(b'', b'rev', [], b'revision to be looked up (default tip)'),
+] + formatteropts)
+def perfnodemap(ui, repo, **opts):
+"""benchmark the time necessary to look up revision from a cold nodemap
+
+Depending on the implementation, the amount and order of revision we look
+up can varies. Example of useful set to test:
+* tip
+* 0
+* -10:
+* :10
+* -10: + :10
+* :10: + -10:
+* -1:
+* -1: + 0
+
+The command currently focus on valid binary lookup. Benchmarking for
+hexlookup, prefix lookup and missing lookup would also be valuable.
+"""
+import mercurial.revlog
+opts = _byteskwargs(opts)
+timer, fm = gettimer(ui, opts)
+mercurial.revlog._prereadsize = 2**24 # disable lazy parser in old hg
+
+unfi = repo.unfiltered()
+# find the filecache func directly
+# This avoid polluting the benchmark with the filecache logic
+makecl = unfi.__class__.changelog.func
+if not opts[b'rev']:
+raise error.Abort('use --rev to specify revisions to look up')
+revs = scmutil.revrange(repo, opts[b'rev'])
+cl = repo.changelog
+nodes = [cl.node(r) for r in revs]
+
+# use a list to pass reference to a nodemap from one closure to the next
+nodeget = [None]
+def setnodeget():
+# probably not necessary, but for good measure
+clearchangelog(unfi)
+nodeget[0] = makecl(unfi).nodemap.get
+
+def setup():
+setnodeget()
+def d():
+get = nodeget[0]
+for n in nodes:
+get(n)
+
+timer(d, setup=setup)
+fm.end()
+
 @command(b'perfstartup', formatteropts)
 def perfstartup(ui, repo, **opts):
 opts = _byteskwargs(opts)
diff --git a/tests/test-contrib-perf.t b/tests/test-contrib-perf.t
--- a/tests/test-contrib-perf.t
+++ b/tests/test-contrib-perf.t
@@ -109,6 +109,8 @@ perfstatus
perfmoonwalk  benchmark walking the changelog backwards
perfnodelookup
  (no help text available)
+   perfnodemap   benchmark the time necessary to look up revision from a cold
+ nodemap
perfparents   (no help text available)
perfpathcopies
  benchmark the copy tracing logic
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 7] perf: support looking up multiple revisions

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548663631 18000
#  Mon Jan 28 03:20:31 2019 -0500
# Node ID 1e135ddcbdd182a6b7ae7b121d254fb58c7fd658
# Parent  2518d5acdc4e994193ca5b70f945983a61e22ff5
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
1e135ddcbdd1
perf: support looking up multiple revisions

The nodemap code has optimisations around the number of lookup we actually made.
As a result, being able to specify multiple revisions to look up is important
when measuring performances. One can now specify full revspecs with the --rev
arguments.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1016,7 +1016,7 @@ def perfignore(ui, repo, **opts):
 fm.end()
 
 @command(b'perfindex', [
-(b'', b'rev', b'', b'revision to be looked up (default tip)'),
+(b'', b'rev', [], b'revision to be looked up (default tip)'),
 (b'', b'no-lookup', None, b'do not revision lookup post creation'),
  ] + formatteropts)
 def perfindex(ui, repo, **opts):
@@ -1025,7 +1025,18 @@ def perfindex(ui, repo, **opts):
 The default is to look `tip` up. Depending on the index implementation,
 the revision looked up can matters. For example, an implementation
 scanning the index will have a faster lookup time for `--rev tip` than for
-`--rev 0`.
+`--rev 0`. The number of looked up revisions and their order can also
+matters.
+
+Example of useful set to test:
+* tip
+* 0
+* -10:
+* :10
+* -10: + :10
+* :10: + -10:
+* -1:
+* -1: + 0
 
 It is not currently possible to check for lookup of a missing node."""
 import mercurial.revlog
@@ -1033,12 +1044,15 @@ def perfindex(ui, repo, **opts):
 timer, fm = gettimer(ui, opts)
 mercurial.revlog._prereadsize = 2**24 # disable lazy parser in old hg
 if opts[b'no_lookup']:
-n = None
-elif opts[b'rev'] is None:
-n = repo[b"tip"].node()
+if opts['rev']:
+raise error.Abort('--no-lookup and --rev are mutually exclusive')
+nodes = []
+elif not opts[b'rev']:
+nodes = [repo[b"tip"].node()]
 else:
-rev = scmutil.revsingle(repo, opts[b'rev'])
-n = repo[rev].node()
+revs = scmutil.revrange(repo, opts[b'rev'])
+cl = repo.changelog
+nodes = [cl.node(r) for r in revs]
 
 unfi = repo.unfiltered()
 # find the filecache func directly
@@ -1049,7 +1063,7 @@ def perfindex(ui, repo, **opts):
 clearchangelog(unfi)
 def d():
 cl = makecl(unfi)
-if n is not None:
+for n in nodes:
 cl.rev(n)
 timer(d, setup=setup)
 fm.end()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 7] perf: add a no-lookup variant to perfindex

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548459828 18000
#  Fri Jan 25 18:43:48 2019 -0500
# Node ID 2518d5acdc4e994193ca5b70f945983a61e22ff5
# Parent  9a0d513d684855e3fbdd023c1fe84e7bad6579c4
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
2518d5acdc4e
perf: add a no-lookup variant to perfindex

It is useful to check how long it takes to create a index object without doing
anything with it. We add a new flag dedicated to that.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1017,6 +1017,7 @@ def perfignore(ui, repo, **opts):
 
 @command(b'perfindex', [
 (b'', b'rev', b'', b'revision to be looked up (default tip)'),
+(b'', b'no-lookup', None, b'do not revision lookup post creation'),
  ] + formatteropts)
 def perfindex(ui, repo, **opts):
 """benchmark index creation time followed by a lookup
@@ -1031,7 +1032,9 @@ def perfindex(ui, repo, **opts):
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
 mercurial.revlog._prereadsize = 2**24 # disable lazy parser in old hg
-if opts[b'rev'] is None:
+if opts[b'no_lookup']:
+n = None
+elif opts[b'rev'] is None:
 n = repo[b"tip"].node()
 else:
 rev = scmutil.revsingle(repo, opts[b'rev'])
@@ -1046,7 +1049,8 @@ def perfindex(ui, repo, **opts):
 clearchangelog(unfi)
 def d():
 cl = makecl(unfi)
-cl.rev(n)
+if n is not None:
+cl.rev(n)
 timer(d, setup=setup)
 fm.end()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 7] perf: add some documentation to perfindex

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548668860 18000
#  Mon Jan 28 04:47:40 2019 -0500
# Node ID 9a0d513d684855e3fbdd023c1fe84e7bad6579c4
# Parent  1e7a5bec6326c537b65ffb59b51c4bb34ea189cc
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
9a0d513d6848
perf: add some documentation to perfindex

It seems useful to document how the arguments can affect the benchmark.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1019,6 +1019,14 @@ def perfignore(ui, repo, **opts):
 (b'', b'rev', b'', b'revision to be looked up (default tip)'),
  ] + formatteropts)
 def perfindex(ui, repo, **opts):
+"""benchmark index creation time followed by a lookup
+
+The default is to look `tip` up. Depending on the index implementation,
+the revision looked up can matters. For example, an implementation
+scanning the index will have a faster lookup time for `--rev tip` than for
+`--rev 0`.
+
+It is not currently possible to check for lookup of a missing node."""
 import mercurial.revlog
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
diff --git a/tests/test-contrib-perf.t b/tests/test-contrib-perf.t
--- a/tests/test-contrib-perf.t
+++ b/tests/test-contrib-perf.t
@@ -93,7 +93,7 @@ perfstatus
  find statistic about potential parameters for the
  'perftracecopies'
perfignorebenchmark operation related to computing ignore
-   perfindex (no help text available)
+   perfindex benchmark index creation time followed by a lookup
perflinelogedits
  (no help text available)
perfloadmarkers
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 7] perf: move cache clearing in the `setup` step of `perfheads`

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548445999 18000
#  Fri Jan 25 14:53:19 2019 -0500
# Node ID 1e7a5bec6326c537b65ffb59b51c4bb34ea189cc
# Parent  dbc0d2061b0584c423f4ecf007cd0125e16b11d1
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
1e7a5bec6326
perf: move cache clearing in the `setup` step of `perfheads`

The cache clearing is pretty fast, but this seems more "correct".

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -539,10 +539,11 @@ def perfheads(ui, repo, **opts):
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
 cl = repo.changelog
+def s():
+clearcaches(cl)
 def d():
 len(cl.headrevs())
-clearcaches(cl)
-timer(d)
+timer(d, setup=s)
 fm.end()
 
 @command(b'perftags', formatteropts+
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 7] perf: document perfheads

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548445951 18000
#  Fri Jan 25 14:52:31 2019 -0500
# Node ID f39f3732ef42fe901b418e23433475076b599757
# Parent  431cf2c8c83969a68d132d6ba92af1998e147927
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
f39f3732ef42
perf: document perfheads

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -535,6 +535,7 @@ def clearcaches(cl):
 
 @command(b'perfheads', formatteropts)
 def perfheads(ui, repo, **opts):
+"""benchmark the computation of a changelog heads"""
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
 cl = repo.changelog
diff --git a/tests/test-contrib-perf.t b/tests/test-contrib-perf.t
--- a/tests/test-contrib-perf.t
+++ b/tests/test-contrib-perf.t
@@ -88,7 +88,7 @@ perfstatus
  (no help text available)
perffncachewrite
  (no help text available)
-   perfheads (no help text available)
+   perfheads benchmark the computation of a changelog heads
perfhelper-pathcopies
  find statistic about potential parameters for the
  'perftracecopies'
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 7] revlog: document cext oddities in terms of object/caches

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548458522 18000
#  Fri Jan 25 18:22:02 2019 -0500
# Node ID dbc0d2061b0584c423f4ecf007cd0125e16b11d1
# Parent  f39f3732ef42fe901b418e23433475076b599757
# EXP-Topic perf-ext
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
dbc0d2061b05
revlog: document cext oddities in terms of object/caches

This clarify why we just call clearcaches on a single object.

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -607,6 +607,9 @@ class revlog(object):
 self._pcache = {}
 
 try:
+# If we are using the native C version, you are in a fun case
+# where self.index, self.nodemap and self._nodecaches is the same
+# object.
 self._nodecache.clearcaches()
 except AttributeError:
 self._nodecache = {nullid: nullrev}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH STABLE] revlog: make sure we never use sparserevlog without general delta (issue6056)

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548372167 18000
#  Thu Jan 24 18:22:47 2019 -0500
# Branch stable
# Node ID c7661fdcaeac457aa5b8ae6b6a10b259713934b9
# Parent  26ee61c33dee366576a0f7266fea9834985e78ec
# EXP-Topic issue6056
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
c7661fdcaeac
revlog: make sure we never use sparserevlog without general delta (issue6056)

We are getting user report where the delta code tries to use `sparse-revlog`
logic on repository where `generaldelta` is disabled. This can't work so we
ensure the two booleans have a consistent value.

Creating this kind of repository is not expected to be possible and we weren't
able to create one for testing purpose. But if users run into this, we better
put stronger protection.

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -497,6 +497,9 @@ class revlog(object):
 else:
 raise error.RevlogError(_('unknown version (%d) in revlog %s') %
 (fmt, self.indexfile))
+# sparse-revlog can't be on without general-delta (issue6056)
+if not self._generaldelta:
+self._sparserevlog = False
 
 self._storedeltachains = True
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 STABLE V3 STABLE] strip: introduce a soft strip option

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1539697680 -7200
#  Tue Oct 16 15:48:00 2018 +0200
# Branch stable
# Node ID 4e7166569731150da6c5aca19f4b6c64a3d259a9
# Parent  13c23396c7fe1633a2336b29e3a32b9b76274f28
# EXP-Topic archived-phase-UX
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
4e7166569731
strip: introduce a soft strip option

This is the first user-accessible way to use the archived phase introduced in
4.8. This implements a feature discussed during the Stockholm sprint, using
the archived phase for hiding changesets.

The archived phase behaves exactly as stripping: changesets are no longer
visible, but pulling/unbundling them will make then reappear. The only notable
difference is that unlike hard stripping, soft stripping does not affect
obsmarkers.

The next changeset will make use of the archived phase for history rewriting
command. However, having a way to manually trigger the feature first seems a
necessary step before exposing users to this phase; there is a way to
un-archived changesets (unbundling), so there must be a way to archive them
again.

Adding a flag to strip is a good way to provide access to the feature without
taking a too big risk on the final UI we want. The flag is experimental so it
won't be exposed by default.

Using the archived phase is faster and less traumatic for the repository than
actually stripping changesets.

diff --git a/hgext/strip.py b/hgext/strip.py
--- a/hgext/strip.py
+++ b/hgext/strip.py
@@ -76,7 +76,8 @@ def _findupdatetarget(repo, nodes):
 
 return unode
 
-def strip(ui, repo, revs, update=True, backup=True, force=None, 
bookmarks=None):
+def strip(ui, repo, revs, update=True, backup=True, force=None, bookmarks=None,
+  soft=False):
 with repo.wlock(), repo.lock():
 
 if update:
@@ -85,7 +86,10 @@ def strip(ui, repo, revs, update=True, b
 hg.clean(repo, urev)
 repo.dirstate.write(repo.currenttransaction())
 
-repair.strip(ui, repo, revs, backup)
+if soft:
+repair.softstrip(ui, repo, revs, backup)
+else:
+repair.strip(ui, repo, revs, backup)
 
 repomarks = repo._bookmarks
 if bookmarks:
@@ -110,7 +114,10 @@ def strip(ui, repo, revs, update=True, b
   ('k', 'keep', None, _("do not modify working directory during "
 "strip")),
   ('B', 'bookmark', [], _("remove revs only reachable from given"
-  " bookmark"), _('BOOKMARK'))],
+  " bookmark"), _('BOOKMARK')),
+  ('', 'soft', None,
+  _("simply drop changesets from visible history (EXPERIMENTAL)")),
+ ],
   _('hg strip [-k] [-f] [-B bookmark] [-r] REV...'),
   helpcategory=command.CATEGORY_MAINTENANCE)
 def stripcmd(ui, repo, *revs, **opts):
@@ -235,6 +242,7 @@ def stripcmd(ui, repo, *revs, **opts):
 
 
 strip(ui, repo, revs, backup=backup, update=update,
-  force=opts.get('force'), bookmarks=bookmarks)
+  force=opts.get('force'), bookmarks=bookmarks,
+  soft=opts['soft'])
 
 return 0
diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -252,6 +252,24 @@ def strip(ui, repo, nodelist, backup=Tru
 # extensions can use it
 return backupfile
 
+def softstrip(ui, repo, nodelist, backup=True, topic='backup'):
+"""perform a "soft" strip using the archived phase"""
+tostrip = [c.node() for c in repo.set('sort(%ln::)', nodelist)]
+if not tostrip:
+return None
+
+newbmtarget, updatebm = _bookmarkmovements(repo, tostrip)
+if backup:
+node = tostrip[0]
+backupfile = _createstripbackup(repo, tostrip, node, topic)
+
+with repo.transaction('strip') as tr:
+phases.retractboundary(repo, tr, phases.archived, tostrip)
+bmchanges = [(m, repo[newbmtarget].node()) for m in updatebm]
+repo._bookmarks.applychanges(repo, tr, bmchanges)
+return backupfile
+
+
 def _bookmarkmovements(repo, tostrip):
 # compute necessary bookmark movement
 bm = repo._bookmarks
diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
new file mode 100644
--- /dev/null
+++ b/tests/test-phase-archived.t
@@ -0,0 +1,77 @@
+=
+Test features and behaviors related to the archived phase
+=
+
+  $ cat << EOF >> $HGRCPATH
+  > [format]
+  > internal-phase=yes
+  > [extensions]
+  > strip=
+  > [experimental]
+  > EOF
+
+  $ hg init repo
+  $ cd repo
+  $ echo  root > a
+  $ hg add a
+  $ hg ci -m 'root'
+
+Test that bundle can unarchive a change

[PATCH 2 of 2 STABLE V3 STABLE] rewriting: add an option for rewrite commands to use the archived phase

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1546394872 -3600
#  Wed Jan 02 03:07:52 2019 +0100
# Branch stable
# Node ID c7de05822f70b02d95b7171c81c9360f1986671b
# Parent  4e7166569731150da6c5aca19f4b6c64a3d259a9
# EXP-Topic archived-phase-UX
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
c7de05822f70
rewriting: add an option for rewrite commands to use the archived phase

Using the archived phase for cleanup provide the same effect than stripping,
but in a faster, append-only way.

We keep the feature experimental for now until it gets a bit more testing.

diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -497,6 +497,9 @@ coreconfigitem('experimental', 'bundleco
 coreconfigitem('experimental', 'changegroup3',
 default=False,
 )
+coreconfigitem('experimental', 'cleanup-as-archived',
+default=False,
+)
 coreconfigitem('experimental', 'clientcompressionengines',
 default=list,
 )
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -984,6 +984,7 @@ def cleanupnodes(repo, replacements, ope
 for phase, nodes in toadvance.items():
 phases.advanceboundary(repo, tr, phase, nodes)
 
+mayusearchived = repo.ui.config('experimental', 'cleanup-as-archived')
 # Obsolete or strip nodes
 if obsolete.isenabled(repo, obsolete.createmarkersopt):
 # If a node is already obsoleted, and we want to obsolete it
@@ -1001,6 +1002,17 @@ def cleanupnodes(repo, replacements, ope
 if rels:
 obsolete.createmarkers(repo, rels, operation=operation,
metadata=metadata)
+elif phases.supportinternal(repo) and mayusearchived:
+# this assume we do not have "unstable" nodes above the cleaned 
ones
+allreplaced = set()
+for ns in replacements.keys():
+allreplaced.update(ns)
+if backup:
+from . import repair # avoid import cycle
+node = min(allreplaced, key=repo.changelog.rev)
+repair.backupbundle(repo, allreplaced, allreplaced, node,
+operation)
+phases.retractboundary(repo, tr, phases.archived, allreplaced)
 else:
 from . import repair # avoid import cycle
 tostrip = list(n for ns in replacements for n in ns)
diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
--- a/tests/test-phase-archived.t
+++ b/tests/test-phase-archived.t
@@ -75,3 +75,69 @@ Test that bundle can unarchive a changes
  date:Thu Jan 01 00:00:00 1970 +
  summary: root
   
+
+Test that history rewriting command can use the archived phase when allowed to
+--
+
+  $ hg up 'desc(unbundletesting)'
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo bar >> a
+  $ hg commit --amend --config experimental.cleanup-as-archived=yes
+  $ hg log -G
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
+  $ hg log -G --hidden
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  | o  changeset:   1:883aadbbf309
+  |/   user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
+  $ ls -1 .hg/strip-backup/
+  883aadbbf309-efc55adc-amend.hg
+  883aadbbf309-efc55adc-backup.hg
+  $ hg unbundle .hg/strip-backup/883aadbbf309*amend.hg
+  adding changesets
+  adding manifests
+  adding file changes
+  added 0 changesets with 0 changes to 1 files
+  (run 'hg update' to get a working copy)
+  $ hg log -G
+  @  changeset:   2:d1e73e428f29
+  |  tag: tip
+  |  parent:  0:c1863a3840c6
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesting
+  |
+  | o  changeset:   1:883aadbbf309
+  |/   user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: unbundletesting
+  |
+  o  changeset:   0:c1863a3840c6
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: root
+  
___
Mercurial-de

D5590: watchman: ignore some of watchman errors

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13415.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5590?vs=13226=13415

REVISION DETAIL
  https://phab.mercurial-scm.org/D5590

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -183,7 +183,8 @@
 if isinstance(ex, watchmanclient.Unavailable):
 # experimental config: hgwatchman.verbose
 if ex.warn and ui.configbool('hgwatchman', 'verbose'):
-ui.warn(str(ex) + '\n')
+if 'illegal_fstypes' not in str(ex):
+ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
 # experimental config: hgwatchman.verbose



To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5589: watchman: add the possibility to set the exact watchman binary location

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13414.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5589?vs=13225=13414

REVISION DETAIL
  https://phab.mercurial-scm.org/D5589

AFFECTED FILES
  hgext/fsmonitor/__init__.py
  hgext/fsmonitor/pywatchman/__init__.py
  hgext/fsmonitor/watchmanclient.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/watchmanclient.py 
b/hgext/fsmonitor/watchmanclient.py
--- a/hgext/fsmonitor/watchmanclient.py
+++ b/hgext/fsmonitor/watchmanclient.py
@@ -82,9 +82,11 @@
 try:
 if self._watchmanclient is None:
 self._firsttime = False
+watchman_exe = self._ui.configpath('fsmonitor', 'watchman_exe')
 self._watchmanclient = pywatchman.client(
 timeout=self._timeout,
-useImmutableBser=True)
+useImmutableBser=True,
+watchman_exe=watchman_exe)
 return self._watchmanclient.query(*watchmanargs)
 except pywatchman.CommandError as ex:
 if 'unable to resolve root' in ex.msg:
diff --git a/hgext/fsmonitor/pywatchman/__init__.py 
b/hgext/fsmonitor/pywatchman/__init__.py
--- a/hgext/fsmonitor/pywatchman/__init__.py
+++ b/hgext/fsmonitor/pywatchman/__init__.py
@@ -317,7 +317,7 @@
 """ local unix domain socket transport """
 sock = None
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
 
@@ -397,7 +397,7 @@
 class WindowsNamedPipeTransport(Transport):
 """ connect to a named pipe """
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = int(math.ceil(timeout * 1000))
 self._iobuf = None
@@ -563,9 +563,10 @@
 proc = None
 closed = True
 
-def __init__(self, sockpath, timeout):
+def __init__(self, sockpath, timeout, watchman_exe):
 self.sockpath = sockpath
 self.timeout = timeout
+self.watchman_exe = watchman_exe
 
 def close(self):
 if self.proc:
@@ -579,7 +580,7 @@
 if self.proc:
 return self.proc
 args = [
-'watchman',
+self.watchman_exe,
 '--sockname={0}'.format(self.sockpath),
 '--logfile=/BOGUS',
 '--statefile=/BOGUS',
@@ -756,17 +757,20 @@
 unilateral = ['log', 'subscription']
 tport = None
 useImmutableBser = None
+watchman_exe = None
 
 def __init__(self,
  sockpath=None,
  timeout=1.0,
  transport=None,
  sendEncoding=None,
  recvEncoding=None,
- useImmutableBser=False):
+ useImmutableBser=False,
+ watchman_exe=None):
 self.sockpath = sockpath
 self.timeout = timeout
 self.useImmutableBser = useImmutableBser
+self.watchman_exe = watchman_exe
 
 if inspect.isclass(transport) and issubclass(transport, Transport):
 self.transport = transport
@@ -817,7 +821,7 @@
 if path:
 return path
 
-cmd = ['watchman', '--output-encoding=bser', 'get-sockname']
+cmd = [self.watchman_exe, '--output-encoding=bser', 'get-sockname']
 try:
 args = dict(stdout=subprocess.PIPE,
 stderr=subprocess.PIPE,
@@ -858,7 +862,7 @@
 if self.sockpath is None:
 self.sockpath = self._resolvesockname()
 
-self.tport = self.transport(self.sockpath, self.timeout)
+self.tport = self.transport(self.sockpath, self.timeout, 
self.watchman_exe)
 self.sendConn = self.sendCodec(self.tport)
 self.recvConn = self.recvCodec(self.tport)
 
diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -164,6 +164,9 @@
 configitem('fsmonitor', 'enable_on_non_interactive',
 default=True,
 )
+configitem('fsmonitor', 'watchman_exe',
+default='watchman',
+)
 configitem('hgwatchman', 'verbose',
 default=True,
 )



To: lothiraldan, #hg-reviewers
Cc: indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5588: watchman: disable fsmonitor when running non-interactively

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13413.
Herald added a subscriber: mjpieters.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5588?vs=13224=13413

REVISION DETAIL
  https://phab.mercurial-scm.org/D5588

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('fsmonitor', 'enable_on_non_interactive',
+default=True,
+)
 configitem('hgwatchman', 'verbose',
 default=True,
 )
@@ -778,6 +781,24 @@
   'extension and has been disabled.\n') % ext)
 return
 
+enabled = None
+if 'HGWATCHMAN_ENABLE' in encoding.environ:
+enabled = encoding.environ['HGWATCHMAN_ENABLE'].lower()
+
+if enabled == 'true' or enabled == '':
+pass
+elif enabled == 'false':
+ui.debug('fsmonitor: disabled through HGWATCHMAN_ENABLE env variable')
+return
+else:
+interactive = ui.interactive()
+enable_on_non_interactive = ui.configbool(
+'fsmonitor', 'enable_on_non_interactive'
+)
+if not interactive and not enable_on_non_interactive:
+ui.debug('fsmonitor: disabled for non-interactive commands')
+return
+
 if repo.local():
 # We don't work with subrepos either.
 #



To: lothiraldan, #hg-reviewers, indygreg
Cc: mjpieters, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5587: watchman: detect nested mercurial repositories and abort

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13412.
Herald added a subscriber: mjpieters.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5587?vs=13223=13412

REVISION DETAIL
  https://phab.mercurial-scm.org/D5587

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -385,6 +385,9 @@
 fexists = entry['exists']
 kind = getkind(fmode)
 
+if '/.hg/' in fname or fname.endswith('/.hg'):
+return bail('nested-repo-detected')
+
 if not fexists:
 # if marked as deleted and we don't already have a change
 # record, mark it as deleted.  If we already have an entry
@@ -758,6 +761,14 @@
 repo, node, branchmerge, force, ancestor, mergeancestor,
 labels, matcher, **kwargs)
 
+def repo_has_depth_one_nested_repo(repo):
+for f in repo.wvfs.listdir():
+if os.path.isdir(os.path.join(repo.root, f, '.hg')):
+msg = 'fsmonitor: sub-repository %r detected, fsmonitor disabled\n'
+repo.ui.debug(msg % f)
+return True
+return False
+
 def reposetup(ui, repo):
 # We don't work with largefiles or inotify
 exts = extensions.enabled()
@@ -775,6 +786,9 @@
 if repo.wvfs.exists('.hgsubstate') or repo.wvfs.exists('.hgsub'):
 return
 
+if repo_has_depth_one_nested_repo(repo):
+return
+
 fsmonitorstate = state.state(repo)
 if fsmonitorstate.mode == 'off':
 return



To: lothiraldan, #hg-reviewers, indygreg
Cc: mjpieters, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5586: watchman: add verbose config knob

[lothiraldan (Boris Feld)]

lothiraldan updated this revision to Diff 13411.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5586?vs=13222=13411

REVISION DETAIL
  https://phab.mercurial-scm.org/D5586

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -161,6 +161,9 @@
 configitem('fsmonitor', 'blacklistusers',
 default=list,
 )
+configitem('hgwatchman', 'verbose',
+default=True,
+)
 configitem('experimental', 'fsmonitor.transaction_notify',
 default=False,
 )
@@ -172,11 +175,14 @@
 def _handleunavailable(ui, state, ex):
 """Exception handler for Watchman interaction exceptions"""
 if isinstance(ex, watchmanclient.Unavailable):
-if ex.warn:
+# experimental config: hgwatchman.verbose
+if ex.warn and ui.configbool('hgwatchman', 'verbose'):
 ui.warn(str(ex) + '\n')
 if ex.invalidate:
 state.invalidate()
-ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
+# experimental config: hgwatchman.verbose
+if ui.configbool('hgwatchman','verbose'):
+ui.log('fsmonitor', 'Watchman unavailable: %s\n', ex.msg)
 else:
 ui.log('fsmonitor', 'Watchman exception: %s\n', ex)
 



To: lothiraldan, #hg-reviewers
Cc: pulkit, indygreg, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH STABLE] partialdiscovery: avoid `undecided` related computation sooner than necessary

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1548284862 18000
#  Wed Jan 23 18:07:42 2019 -0500
# Branch stable
# Node ID 2ac72f1b08d3394c3db21c1f576c189118b41eba
# Parent  c953c2a94d68b18c2f3c79262826fdfed743a839
# EXP-Topic discovery-regression
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
2ac72f1b08d3
partialdiscovery: avoid `undecided` related computation sooner than necessary

Changeset 1d30be90c move the update of the `undecided` set within the
`partialdiscovery` object in order to clarify the API.

The update to the `undecided` set was unconditional in 1d30be90c and the first
access to the `self.undecided` property triggered the initial computation of
the set of undecided revisions. As a result, the set was computed much
earlier, at a time where less information is available, immediately followed
by an update of this set to remove common revisions.

To fix this regression, we ignore the `undecided` related logic in
`addcommons` when that `undecided` set has not been computed yet. Code that
actually needs to know the `undecided` set will trigger its computation later.
The change has no effects on semantic because the initial computation
`undecided` set takes all knowns `common` into account.

Example performance running `hg debugdiscovery` from a pypy repo missing 10
changesets:

870a89c6909d: 52.3ms (regression parent)
1d30be90c9dc: 72.0ms (regression)
5a5f504a7175: 64.8ms (this fix parent)
this fix: 52.6ms

diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -183,7 +183,8 @@ class partialdiscovery(object):
 def addcommons(self, commons):
 """registrer nodes known as common"""
 self._common.addbases(commons)
-self._common.removeancestorsfrom(self.undecided)
+if self._undecided is not None:
+self._common.removeancestorsfrom(self._undecided)
 
 def addmissings(self, missings):
 """registrer some nodes as missing"""
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5633: grep: use set instead of dict with dummy value

[lothiraldan (Boris Feld)]

lothiraldan added a comment.


  I get a message stating that one of the hunk of patch failed to apply and 
couldn't understand what is going on, could you try rebase it to see if you 
have the issue locally?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5633

To: martinvonz, #hg-reviewers
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 STABLE V2 STABLE] strip: introduce a soft strip option

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1539697680 -7200
#  Tue Oct 16 15:48:00 2018 +0200
# Branch stable
# Node ID 4285aa379de0ffdfb68c6a2920a0d09f037dfa2e
# Parent  13c23396c7fe1633a2336b29e3a32b9b76274f28
# EXP-Topic archived-phase-UX
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
4285aa379de0
strip: introduce a soft strip option

This is the first user-accessible way to use the archived phase introduced in
4.8. This implements a feature implemented during the Stockholm sprint. The
archived phase behave as stripping, changesets are no longer accessible, but
pulling/unbundling them will make then reappear. The only notable difference
is that unlike hard stripping, soft stripping does not affect obsmarkers.

Adding flag to strip is a good way to provide access to the feature without
taking a too big risk on the final UI we want.

The next changeset will make use of the archived phase for history rewriting
command. However, having a way to manually trigger the feature first seemed
better.

Using the archived phase is faster and less traumatic for the repository.

diff --git a/hgext/strip.py b/hgext/strip.py
--- a/hgext/strip.py
+++ b/hgext/strip.py
@@ -76,7 +76,8 @@ def _findupdatetarget(repo, nodes):
 
 return unode
 
-def strip(ui, repo, revs, update=True, backup=True, force=None, 
bookmarks=None):
+def strip(ui, repo, revs, update=True, backup=True, force=None, bookmarks=None,
+  soft=False):
 with repo.wlock(), repo.lock():
 
 if update:
@@ -85,7 +86,10 @@ def strip(ui, repo, revs, update=True, b
 hg.clean(repo, urev)
 repo.dirstate.write(repo.currenttransaction())
 
-repair.strip(ui, repo, revs, backup)
+if soft:
+repair.softstrip(ui, repo, revs, backup)
+else:
+repair.strip(ui, repo, revs, backup)
 
 repomarks = repo._bookmarks
 if bookmarks:
@@ -110,7 +114,10 @@ def strip(ui, repo, revs, update=True, b
   ('k', 'keep', None, _("do not modify working directory during "
 "strip")),
   ('B', 'bookmark', [], _("remove revs only reachable from given"
-  " bookmark"), _('BOOKMARK'))],
+  " bookmark"), _('BOOKMARK')),
+  ('', 'soft', [],
+  _("simply drop changesets from visible history (EXPERIMENTAL)")),
+ ],
   _('hg strip [-k] [-f] [-B bookmark] [-r] REV...'),
   helpcategory=command.CATEGORY_MAINTENANCE)
 def stripcmd(ui, repo, *revs, **opts):
@@ -235,6 +242,7 @@ def stripcmd(ui, repo, *revs, **opts):
 
 
 strip(ui, repo, revs, backup=backup, update=update,
-  force=opts.get('force'), bookmarks=bookmarks)
+  force=opts.get('force'), bookmarks=bookmarks,
+  soft=opts['soft'])
 
 return 0
diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -252,6 +252,24 @@ def strip(ui, repo, nodelist, backup=Tru
 # extensions can use it
 return backupfile
 
+def softstrip(ui, repo, nodelist, backup=True, topic='backup'):
+"""perform a "soft" strip using the archived phase"""
+tostrip = [c.node() for c in repo.set('sort(%ln::)', nodelist)]
+if not tostrip:
+return None
+
+newbmtarget, updatebm = _bookmarkmovements(repo, tostrip)
+if backup:
+node = tostrip[0]
+backupfile = _createstripbackup(repo, tostrip, node, topic)
+
+with repo.transaction('strip') as tr:
+phases.retractboundary(repo, tr, phases.archived, tostrip)
+bmchanges = [(m, repo[newbmtarget].node()) for m in updatebm]
+repo._bookmarks.applychanges(repo, tr, bmchanges)
+return backupfile
+
+
 def _bookmarkmovements(repo, tostrip):
 # compute necessary bookmark movement
 bm = repo._bookmarks
diff --git a/tests/test-phase-archived.t b/tests/test-phase-archived.t
new file mode 100644
--- /dev/null
+++ b/tests/test-phase-archived.t
@@ -0,0 +1,77 @@
+=
+Test features and behaviors related to the archived phase
+=
+
+  $ cat << EOF >> $HGRCPATH
+  > [format]
+  > internal-phase=yes
+  > [extensions]
+  > strip=
+  > [experimental]
+  > EOF
+
+  $ hg init repo
+  $ cd repo
+  $ echo  root > a
+  $ hg add a
+  $ hg ci -m 'root'
+
+Test that bundle can unarchive a changeset
+--
+
+  $ echo foo >> a
+  $ hg st
+  M a
+  $ hg ci -m 'unbundletesting'
+  $ hg log -G
+  @  changeset:   1:883aadbbf309
+  |  tag: tip
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: unbundletesti

[PATCH 4 of 4] revset: no longer silently filter out invalid revision in _intlist (API) (BC)

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1547826690 -3600
#  Fri Jan 18 16:51:30 2019 +0100
# Node ID 43543a826aab4e6495694f84e9197b903a7cbf10
# Parent  aa032edd96aeb34f8d220b11fbdbd08a9497cbc9
# EXP-Topic intlist
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
43543a826aab
revset: no longer silently filter out invalid revision in _intlist (API) (BC)

This change makes the fastpath (fullrepo case) evaluated first, so invalid
revision in the _intlist() entries will be simply forwarded to the lower layer
in this case. This is similar to what 'rawsmartet' does in the case where
"%ld" did not get serialized.

Further processing of the resulting smartset is likely to raise an error
because of the filtering.

It would be possible to strictly check for any invalid entry in the input
revs, but we have not decided on doing this yet.

This series focuses on having a consistent behavior for %d and %ld in all
cases.

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -2179,7 +2179,7 @@ def _orderedlist(repo, subset, x):
 for r in revs:
 if r in seen:
 continue
-if (r in subset or full and (r == nullrev or r == wdirrev)):
+if (full and (r == nullrev or r == wdirrev)) or r in subset:
 ls.append(r)
 seen.add(r)
 return baseset(ls)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 4] revset: add wdir support to _intlist

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1547826678 -3600
#  Fri Jan 18 16:51:18 2019 +0100
# Node ID aa032edd96aeb34f8d220b11fbdbd08a9497cbc9
# Parent  35c9dec1849f6a41f87fdc130798387376bb8345
# EXP-Topic intlist
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
aa032edd96ae
revset: add wdir support to _intlist

The previous changeset made it clear that we did not supported the wdirrev in
_intlist. So we fix it.

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -2174,11 +2174,12 @@ def _orderedlist(repo, subset, x):
 
 full = isinstance(subset, fullreposet)
 nullrev = node.nullrev
+wdirrev = node.wdirrev
 
 for r in revs:
 if r in seen:
 continue
-if (r in subset or full and r == nullrev):
+if (r in subset or full and (r == nullrev or r == wdirrev)):
 ls.append(r)
 seen.add(r)
 return baseset(ls)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4] revset: minor preparatory refactor in _orderedlist

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1547818460 -3600
#  Fri Jan 18 14:34:20 2019 +0100
# Node ID 35c9dec1849f6a41f87fdc130798387376bb8345
# Parent  52d20e4fe2e3049eec58140fdcadb43528729b3d
# EXP-Topic intlist
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
35c9dec1849f
revset: minor preparatory refactor in _orderedlist

In order to enforce the new %ld behavior in the cases where _intlist
serialization is still used, we will have to update this function. We do first
some small changes that do not alter the semantic. They just introduce lookup
caching and some pre-computation.

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -2172,11 +2172,13 @@ def _orderedlist(repo, subset, x):
 except ValueError:
 revs = stringset(repo, subset, t, defineorder)
 
+full = isinstance(subset, fullreposet)
+nullrev = node.nullrev
+
 for r in revs:
 if r in seen:
 continue
-if (r in subset
-or r == node.nullrev and isinstance(subset, fullreposet)):
+if (r in subset or full and r == nullrev):
 ls.append(r)
 seen.add(r)
 return baseset(ls)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 4] revset: introduce an internal `_rev` predicate for '%d' usage

[Boris Feld]

# HG changeset patch
# User Boris Feld 
# Date 1547817707 -3600
#  Fri Jan 18 14:21:47 2019 +0100
# Node ID 52d20e4fe2e3049eec58140fdcadb43528729b3d
# Parent  4fab8a7d2d72b4d50d1d647a015d33f64c1e2e4d
# EXP-Topic intlist
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
52d20e4fe2e3
revset: introduce an internal `_rev` predicate for '%d' usage

In 24a1f67bb75a, we aligned "%d" behavior on "%ld" one, invalid revisions got
silently ignored. However, soon after in 8aca89a694d4 and 26b0a7514f01, a side
effect changed the behavior of "%ld" to no longer silently filter invalid
revisions.

After discussion on the mailing list, it was decided to align on the new %ld
behavior:

https://www.mercurial-scm.org/pipermail/mercurial-devel/2019-January/127291.html


This changeset introduce a '_rev()' predicated that keep the benefit from
24a1f67bb75a while enforcing a more strict checking on the inputs.

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -1765,6 +1765,20 @@ def rev(repo, subset, x):
 return baseset()
 return subset & baseset([l])
 
+@predicate('_rev(number)', safe=True)
+def _rev(repo, subset, x):
+# internal version of "rev(x)" that raise error if "x" is invalid
+# i18n: "rev" is a keyword
+l = getargs(x, 1, 1, _("_rev requires one argument"))
+try:
+# i18n: "rev" is a keyword
+l = int(getstring(l[0], _("rev requires a number")))
+except (TypeError, ValueError):
+# i18n: "rev" is a keyword
+raise error.ParseError(_("rev expects a number"))
+repo.changelog.node(l) # check that the rev exists
+return subset & baseset([l])
+
 @predicate('revset(set)', safe=True, takeorder=True)
 def revsetpredicate(repo, subset, x, order):
 """Strictly interpret the content as a revset.
diff --git a/mercurial/revsetlang.py b/mercurial/revsetlang.py
--- a/mercurial/revsetlang.py
+++ b/mercurial/revsetlang.py
@@ -585,7 +585,7 @@ def _quote(s):
 
 def _formatargtype(c, arg):
 if c == 'd':
-return 'rev(%d)' % int(arg)
+return '_rev(%d)' % int(arg)
 elif c == 's':
 return _quote(arg)
 elif c == 'r':
@@ -663,9 +663,9 @@ def formatspec(expr, *args):
 >>> formatspec(b'%r:: and %lr', b'10 or 11', (b"this()", b"that()"))
 '(10 or 11):: and ((this()) or (that()))'
 >>> formatspec(b'%d:: and not %d::', 10, 20)
-'rev(10):: and not rev(20)::'
+'_rev(10):: and not _rev(20)::'
 >>> formatspec(b'%ld or %ld', [], [1])
-"_list('') or rev(1)"
+"_list('') or _rev(1)"
 >>> formatspec(b'keyword(%s)', b'foo\\xe9')
 "keyword('fooxe9')"
 >>> b = lambda: b'default'
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel