D340: rebase: prefer choosing merge base with successor in destination

[quark (Jun Wu)]

quark updated this revision to Diff 768.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D340?vs=766=768

REVISION DETAIL
  https://phab.mercurial-scm.org/D340

AFFECTED FILES
  hgext/rebase.py
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -1069,9 +1069,8 @@
   rebasing 2:b18e25de2cf5 "D" (D)
   note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
   $ hg log -G
-  o  7:9ed45af61fa0 F
-  |
   o  6:8f47515dda15 D
   |
   | x5:66f1a38021c9 F
@@ -1105,12 +1104,11 @@
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 3:7fb047a69f22 "E" (E)
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
 
-Rebased F should have one parent, just like in the test case above
+F should disappear, just like in the test case above
 
   $ hg log -G
-  o  7:502540f44880 F
-  |
   o  6:533690786a86 E
   |
   | x5:66f1a38021c9 F
@@ -1127,6 +1125,77 @@
   
   $ cd ..
 
+Rebase merge where both parents have successors in destination
+
+  $ hg init p12-succ-in-dest
+  $ cd p12-succ-in-dest
+  $ hg debugdrawdag <<'EOS'
+  >   E   F
+  >  /|  /|  # replace: A -> C
+  > A B C D  # replace: B -> D
+  > EOS
+  $ hg rebase -r ::E -d F
+  note: not rebasing 0:426bada5c675 "A" (A), already in destination as 
2:96cc3511f894 "C"
+  note: not rebasing 1:fc2b737bb2e5 "B" (B), already in destination as 
3:058c1e1fb10a "D"
+  rebasing 4:d6e82823588a "E" (E)
+  warning: cannot decide a unique merge base for 4:d6e82823588a, merge result 
may be suboptimal
+  $ hg log -Gp -T '{rev}:{node|short} {desc|firstline}\n'
+  o  6:d0827eab33f0 E
+  |  diff -r e0c929a964ce -r d0827eab33f0 A
+  |  --- /dev/null Thu Jan 01 00:00:00 1970 +
+  |  +++ b/A   Thu Jan 01 00:00:00 1970 +
+  |  @@ -0,0 +1,1 @@
+  |  +A
+  |  \ No newline at end of file
+  |
+  o5:e0c929a964ce F
+  |\   diff -r 058c1e1fb10a -r e0c929a964ce C
+  | |  --- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | |  +++ b/C Thu Jan 01 00:00:00 1970 +
+  | |  @@ -0,0 +1,1 @@
+  | |  +C
+  | |  \ No newline at end of file
+  | |
+  | | x4:d6e82823588a E
+  | | |\   diff -r 426bada5c675 -r d6e82823588a B
+  | | | |  --- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | | | |  +++ b/B Thu Jan 01 00:00:00 1970 +
+  | | | |  @@ -0,0 +1,1 @@
+  | | | |  +B
+  | | | |  \ No newline at end of file
+  | | | |
+  | o | |  3:058c1e1fb10a D
+  |  / /   diff -r  -r 058c1e1fb10a D
+  | | |--- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | | |+++ b/D Thu Jan 01 00:00:00 1970 +
+  | | |@@ -0,0 +1,1 @@
+  | | |+D
+  | | |\ No newline at end of file
+  | | |
+  o | |  2:96cc3511f894 C
+   / /   diff -r  -r 96cc3511f894 C
+  | |--- /dev/null Thu Jan 01 00:00:00 1970 +
+  | |+++ b/C   Thu Jan 01 00:00:00 1970 +
+  | |@@ -0,0 +1,1 @@
+  | |+C
+  | |\ No newline at end of file
+  | |
+  | x  1:fc2b737bb2e5 B
+  |diff -r  -r fc2b737bb2e5 B
+  |--- /dev/null   Thu Jan 01 00:00:00 1970 +
+  |+++ b/B Thu Jan 01 00:00:00 1970 +
+  |@@ -0,0 +1,1 @@
+  |+B
+  |\ No newline at end of file
+  |
+  x  0:426bada5c675 A
+ diff -r  -r 426bada5c675 A
+ --- /dev/null Thu Jan 01 00:00:00 1970 +
+ +++ b/A   Thu Jan 01 00:00:00 1970 +
+ @@ -0,0 +1,1 @@
+ +A
+ \ No newline at end of file
+  
 Test that bookmark is moved and working dir is updated when all changesets have
 equivalents in destination
   $ hg init rbsrepo && cd rbsrepo
diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -1087,9 +1087,35 @@
 if len(bases) > 1:
 bases.difference_update(ancestor(rev, d) for d in set(dests))
 
-# Only pick the merge base if we have a unique candidate
-if len(bases) == 1:
-base = next(iter(bases))
+# If there are still multiple candidates, prefer obsoleted ones with
+# successor in destination. Otherwise we might re-introduce unwanted
+# obsoleted changes. For example,
+#
+# C  # rebase -r A+B+C -d D
+#/|  # Suppose A has content "+A", B has "+B", D has "+D".
+#   A B   D  # replace: A is replaced by D
+#
+# B gets moved on top of D, A gets skipped, C gets moved on top of B':
+#
+# C' # When choosing merge base for C, A and B are candidates.
+# |  # If we choose B, the difference between C and B are "+A",
+# C   B' # and C' will have the content "+A", which is suboptimal
+#/|   |  # because it re-introduces obsoleted content. If we choose
+#   

D341: test-rebase-obsolete: add "\n" in template

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This makes the test a bit cleaner since `log --stat` does not need to
  specify `-T` manually to keep its output readable.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D341

AFFECTED FILES
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -6,7 +6,7 @@
 
   $ cat >> $HGRCPATH << EOF
   > [ui]
-  > logtemplate= {rev}:{node|short} {desc|firstline}
+  > logtemplate= {rev}:{node|short} {desc|firstline}\n
   > [experimental]
   > evolution=createmarkers,allowunstable
   > [phases]
@@ -420,7 +420,7 @@
 -
 
   $ hg log -r 'children(8)'
-  9:cf44d2f5a9f4 D (no-eol)
+  9:cf44d2f5a9f4 D
   $ hg rebase -r 8
   rebasing 8:e273c5e7d2d2 "C"
   $ hg log -G
@@ -1139,7 +1139,7 @@
   note: not rebasing 1:fc2b737bb2e5 "B" (B), already in destination as 
3:058c1e1fb10a "D"
   rebasing 4:d6e82823588a "E" (E)
   warning: cannot decide a unique merge base for 4:d6e82823588a, merge result 
may be suboptimal
-  $ hg log -Gp -T '{rev}:{node|short} {desc|firstline}\n'
+  $ hg log -Gp
   o  6:d0827eab33f0 E
   |  diff -r e0c929a964ce -r d0827eab33f0 A
   |  --- /dev/null Thu Jan 01 00:00:00 1970 +
@@ -1214,17 +1214,17 @@
   $ hg rebase -r 2 -d 1
   rebasing 2:1e9a3c00cbe9 "b" (tip)
   $ hg log -r .  # working dir is at rev 3 (successor of 2)
-  3:be1832deae9a b (no-eol)
+  3:be1832deae9a b
   $ hg book -r 2 mybook --hidden  # rev 2 has a bookmark on it now
   $ hg up 2 && hg log -r .  # working dir is at rev 2 again
   0 files updated, 0 files merged, 1 files removed, 0 files unresolved
-  2:1e9a3c00cbe9 b (no-eol)
+  2:1e9a3c00cbe9 b
   $ hg rebase -r 2 -d 3 --config experimental.evolution.track-operation=1
   note: not rebasing 2:1e9a3c00cbe9 "b" (mybook), already in destination as 
3:be1832deae9a "b"
 Check that working directory was updated to rev 3 although rev 2 was skipped
 during the rebase operation
   $ hg log -r .
-  3:be1832deae9a b (no-eol)
+  3:be1832deae9a b
 
 Check that bookmark was not moved to rev 3 if rev 2 was skipped during the
 rebase operation. This makes sense because if rev 2 has a successor, the



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D340: rebase: prefer choosing merge base with successor in destination

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As demonstrated by the test case change, and the new comment. Choosing a
  merge base candidate with a successor in destination would allow us to avoid
  re-introducing obsoleted changes and is therefore considered better.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D340

AFFECTED FILES
  hgext/rebase.py
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -1069,9 +1069,8 @@
   rebasing 2:b18e25de2cf5 "D" (D)
   note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
   $ hg log -G
-  o  7:9ed45af61fa0 F
-  |
   o  6:8f47515dda15 D
   |
   | x5:66f1a38021c9 F
@@ -1105,12 +1104,11 @@
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 3:7fb047a69f22 "E" (E)
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
 
 Rebased F should have one parent, just like in the test case above
 
   $ hg log -G
-  o  7:502540f44880 F
-  |
   o  6:533690786a86 E
   |
   | x5:66f1a38021c9 F
@@ -1127,6 +1125,77 @@
   
   $ cd ..
 
+Rebase merge where both parents have successors in destination
+
+  $ hg init p12-succ-in-dest
+  $ cd p12-succ-in-dest
+  $ hg debugdrawdag <<'EOS'
+  >   E   F
+  >  /|  /|  # replace: A -> C
+  > A B C D  # replace: B -> D
+  > EOS
+  $ hg rebase -r ::E -d F
+  note: not rebasing 0:426bada5c675 "A" (A), already in destination as 
2:96cc3511f894 "C"
+  note: not rebasing 1:fc2b737bb2e5 "B" (B), already in destination as 
3:058c1e1fb10a "D"
+  rebasing 4:d6e82823588a "E" (E)
+  warning: cannot decide a unique merge base for 4:d6e82823588a, merge result 
may be suboptimal
+  $ hg log -Gp -T '{rev}:{node|short} {desc|firstline}\n'
+  o  6:d0827eab33f0 E
+  |  diff -r e0c929a964ce -r d0827eab33f0 A
+  |  --- /dev/null Thu Jan 01 00:00:00 1970 +
+  |  +++ b/A   Thu Jan 01 00:00:00 1970 +
+  |  @@ -0,0 +1,1 @@
+  |  +A
+  |  \ No newline at end of file
+  |
+  o5:e0c929a964ce F
+  |\   diff -r 058c1e1fb10a -r e0c929a964ce C
+  | |  --- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | |  +++ b/C Thu Jan 01 00:00:00 1970 +
+  | |  @@ -0,0 +1,1 @@
+  | |  +C
+  | |  \ No newline at end of file
+  | |
+  | | x4:d6e82823588a E
+  | | |\   diff -r 426bada5c675 -r d6e82823588a B
+  | | | |  --- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | | | |  +++ b/B Thu Jan 01 00:00:00 1970 +
+  | | | |  @@ -0,0 +1,1 @@
+  | | | |  +B
+  | | | |  \ No newline at end of file
+  | | | |
+  | o | |  3:058c1e1fb10a D
+  |  / /   diff -r  -r 058c1e1fb10a D
+  | | |--- /dev/null   Thu Jan 01 00:00:00 1970 +
+  | | |+++ b/D Thu Jan 01 00:00:00 1970 +
+  | | |@@ -0,0 +1,1 @@
+  | | |+D
+  | | |\ No newline at end of file
+  | | |
+  o | |  2:96cc3511f894 C
+   / /   diff -r  -r 96cc3511f894 C
+  | |--- /dev/null Thu Jan 01 00:00:00 1970 +
+  | |+++ b/C   Thu Jan 01 00:00:00 1970 +
+  | |@@ -0,0 +1,1 @@
+  | |+C
+  | |\ No newline at end of file
+  | |
+  | x  1:fc2b737bb2e5 B
+  |diff -r  -r fc2b737bb2e5 B
+  |--- /dev/null   Thu Jan 01 00:00:00 1970 +
+  |+++ b/B Thu Jan 01 00:00:00 1970 +
+  |@@ -0,0 +1,1 @@
+  |+B
+  |\ No newline at end of file
+  |
+  x  0:426bada5c675 A
+ diff -r  -r 426bada5c675 A
+ --- /dev/null Thu Jan 01 00:00:00 1970 +
+ +++ b/A   Thu Jan 01 00:00:00 1970 +
+ @@ -0,0 +1,1 @@
+ +A
+ \ No newline at end of file
+  
 Test that bookmark is moved and working dir is updated when all changesets have
 equivalents in destination
   $ hg init rbsrepo && cd rbsrepo
diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -1087,9 +1087,35 @@
 if len(bases) > 1:
 bases.difference_update(ancestor(rev, d) for d in set(dests))
 
-# Only pick the merge base if we have a unique candidate
-if len(bases) == 1:
-base = next(iter(bases))
+# If there are still multiple candidates, prefer obsoleted ones with
+# successor in destination. Otherwise we might re-introduce unwanted
+# obsoleted changes. For example,
+#
+# C  # rebase -r A+B+C -d D
+#/|  # Suppose A has content "+A", B has "+B", D has "+D".
+#   A B   D  # replace: A is replaced by D
+#
+# B gets moved on top of D, A gets skipped, C gets moved on top of B':
+#
+# C' # When choosing merge base for C, A and B are candidates.
+# |  # If we choose B, the difference 

D21: rebase: rewrite core algorithm (issue5578) (issue5630)

[quark (Jun Wu)]

quark updated this revision to Diff 765.
quark edited the summary of this revision.
This revision is now accepted and ready to land.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D21?vs=754=765

REVISION DETAIL
  https://phab.mercurial-scm.org/D21

AFFECTED FILES
  hgext/rebase.py
  tests/test-rebase-brute-force.t
  tests/test-rebase-newancestor.t
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -488,9 +488,14 @@
   >   A
   > EOF
 
-BROKEN: This raises an exception
-  $ hg rebase -d G -r 'B + D + F' 2>&1 | grep '^AssertionError'
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d G -r 'B + D + F'
+  rebasing 1:112478962961 "B" (B)
+  rebasing 2:b18e25de2cf5 "D" (D)
+  not rebasing ignored 4:26805aba1e60 "C" (C)
+  not rebasing ignored 5:4b61ff5c62e2 "E" (E)
+  rebasing 6:f15c3adaf214 "F" (F tip)
+  abort: cannot use revision 6 as base, result would have 3 parents
+  [255]
 
   $ cd ..
 
@@ -962,17 +967,19 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d B -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d B -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 4:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o4:66f1a38021c9 F
+  o5:aae1787dacee F
   |\
-  | x  3:7fb047a69f22 E
-  | |
-  o |  2:b18e25de2cf5 D
-  |/
-  | o  1:112478962961 B
+  | | x  4:66f1a38021c9 F
+  | |/|
+  | | x  3:7fb047a69f22 E
+  | | |
+  | o |  2:b18e25de2cf5 D
+  | |/
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -994,19 +1001,19 @@
   $ hg rebase -d C -s D
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: not rebased on top of requested destination (C)
+
   $ hg log -G
-  o6:50e9d60b99c6 F
+  o6:0913febf6439 F
   |\
-  | | x  5:66f1a38021c9 F
-  | |/|
-  +-o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
+  | | |
+  | o |  4:26805aba1e60 C
   | | |
-  | o |  3:7fb047a69f22 E
+  o | |  3:7fb047a69f22 E
   | | |
-  | | x  2:b18e25de2cf5 D
-  | |/
-  o |  1:112478962961 B
+  +---x  2:b18e25de2cf5 D
+  | |
+  | o  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1025,19 +1032,21 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d C -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d C -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 5:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o5:66f1a38021c9 F
+  o6:c6ab0cc6d220 F
   |\
-  | | o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
   | | |
-  | x |  3:7fb047a69f22 E
+  | o |  4:26805aba1e60 C
   | | |
-  o | |  2:b18e25de2cf5 D
-  |/ /
-  | o  1:112478962961 B
+  | | x  3:7fb047a69f22 E
+  | | |
+  o---+  2:b18e25de2cf5 D
+   / /
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1096,13 +1105,14 @@
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 3:7fb047a69f22 "E" (E)
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: This should have resulted in a rebased F with one parent, just like in
-the test case above
+
+Rebased F should have one parent, just like in the test case above
+
   $ hg log -G
-  o7:c1e6f26e339d F
-  |\
-  | o  6:533690786a86 E
-  |/
+  o  7:502540f44880 F
+  |
+  o  6:533690786a86 E
+  |
   | x5:66f1a38021c9 F
   | |\
   o | |  4:26805aba1e60 C
diff --git a/tests/test-rebase-newancestor.t b/tests/test-rebase-newancestor.t
--- a/tests/test-rebase-newancestor.t
+++ b/tests/test-rebase-newancestor.t
@@ -284,18 +284,7 @@
   rebasing 6:4c5f12f25ebe "merge rebase ancestors" (tip)
   resolving manifests
   removing other
-  note: merging f9daf77ffe76+ and 4c5f12f25ebe using bids from ancestors 
a60552eb93fb and f59da8fc0fcf
-  
-  calculating bids for ancestor a60552eb93fb
   resolving manifests
-  
-  calculating bids for ancestor f59da8fc0fcf
-  resolving manifests
-  
-  auction for merging merge bids
-   other: consensus for g
-  end of auction
-  
   getting other
   committing files:
   other
diff --git a/tests/test-rebase-brute-force.t b/tests/test-rebase-brute-force.t
--- a/tests/test-rebase-brute-force.t
+++ b/tests/test-rebase-brute-force.t
@@ -31,8 +31,8 @@
 AD: A':Z D':Z
 BD: B':Z D':B'
ABD: A':Z B':Z D':B'
-CD: CRASH: revlog index out of range
-   ACD: A':Z C':A'A' D':Z
+CD: ABORT: cannot use revision 3 as base, result would have 3 parents
+   ACD: A':Z C':A'B D':Z
BCD: B':Z C':B'A D':B'
   ABCD: A':Z B':Z C':A'B' D':B'
 
@@ -52,4 +52,4 @@
 C: ABORT: cannot use revision 3 as base, result would have 3 parents
BC: 

Formalizing Interfaces

[Gregory Szorc]

I've wanted to hack on overhauling revlogs and storage for several months
now. The more I think about how to go about it and the reasons why I want
to do it (mainly performance and scaling), the more I think a more drastic
departure from revlogs and the current storage "backend" is needed. There
are some great properties of revlogs and the direct-addressable model of
the current store, don't get me wrong. But we can only scale that model so
far (as I'm sure anyone from Facebook or Google will tell you).

Anyway, overhauling storage is a daunting proposition. It should be
terrifying (because it is).

I started thinking about how we could facilitate extreme experimentation on
things like say swapping out a new storage backend. Perhaps even with one
implemented in Rust :)

Durham did a lot of work on the manifest code a few months back.
Essentially, he successfully decoupled the API of the manifest from the
low-level implementation of a revlog. It even abstracts away flat versus
tree manifests. It's great stuff. I was reminded of his work when I started
trying to do something similar for revlogs.

Long story short, one thing led to another and I had this crazy idea that
it would be a good idea to declare formal interfaces for important
constructs, like the changelog, manifests, file history, peers, and quite
possibly the repo object itself. If we did this and somehow enforced the
interface, it would be possible to swap out implementations as needed. For
example, if the classic store with 1 file/revlog per tracked path doesn't
scale for you because you have >1M files, then you can swap in a store that
uses "pack files," remote storage, etc. If a classic filesystem-based
working directory doesn't fit the bill, perhaps you swap in one that is
virtual filesystem aware.

I wanted to experiment with this idea with something that is easier to
reason about than storage. That led me to the "peer" classes (peer.py,
httppeer.py, sshpeer.py, etc). I've just submitted a series where I
formalize the peer interface using abstract base classes. Read the commit
messages for https://phab.mercurial-scm.org/D332 and
https://phab.mercurial-scm.org/D339 and the commits in that stack for more
details.

Part of developing that series uncovered a number of minor bugs. And, I
think the end result is a peer API that is more easily understood and
easier to hack on. So, I think there is merit to the approach for code
maintainability reasons alone.

But I want to dream bigger and apply this to more significant primitives
(like storage).

If we adopt formal interfaces for important constructs, I'd like to see
support in the test harness for swapping in alternate implementations of
these things. Jun recently added #testcases syntax to .t tests so we could
run multiple variations of the same test. I'd like do something similar at
the entire test suite level. e.g. `run-tests.py --changelog=sqlite` or
`run-tests.py --store=leveldb`. Or more realistically, `run-tests.py
--peers http,ssh` would run the test suite using both the http and ssh peer
implementations. I could see that culminating with tests naturally dividing
themselves into low-level unit tests (interface implementation specific)
and higher-level, generic integration tests. If we "code to the interface,"
it should be possible to swap in a brand new implementation of something
like a peer protocol or changelog and it will pass the integration tests.
We could even have dummy, super fast implementations to facilitate hacking
on things like frontend features to help reduce the edit-test cycle.

Formal interfaces will facilitate extreme experimentation without the
traditional fragility that a dynamic language like Python introduces. They
will allow us to more clearly define boundaries between components. This
will make it vastly easier to refactor and do things like rewrite large
components in Rust. It would also make it *much* easier to implement "hgit"
(using the Mercurial CLI to interface with a Git repository, advanced
features like revlogs and all). (Call me crazy, but I'd love to ship this
feature as part of Mercurial to help entice new users.)

Regardless of whether we go all in on formal interfaces, there's an
interesting idea 2 paragraphs back: tests that are implementation agnostic.
Today, we end up duplicating test functionality for minor variations. e.g.
http vs ssh vs local peer interactions. bundle1 vs bundle2. I'd really like
to move many tests to Jun's #testcases feature because it will allow us to
achieve higher test coverage while writing fewer tests. I think it would be
worthwhile to figure out how to consolidate as many tests as possible so
they can test behavior, not a specific implementation. A side-benefit of
doing this is we'll uncover areas where implementations vary in behavior.
This will help squash bugs and produce a more consistent user interface and
experience, because every time we see e.g. different behavior between
things performing the same role, we'll 

D338: wireproto: use new peer interface

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The wirepeer class provides concrete implementations of peer interface
  methods for calling wire protocol commands. It makes sense for this
  class to inherit from the peer abstract base class. So we change
  that.
  
  Since httppeer and sshpeer have already been converted to the new
  interface, peerrepository is no longer adding any value. So it has
  been removed. httppeer and sshpeer have been updated to reflect the
  loss of peerrepository and the inheritance of the abstract base
  class in wirepeer.
  
  The code changes in wirepeer are reordering of methods to group
  by interface.
  
  Some Python code in tests was updated to reflect changed APIs.
  
  .. api::
  
peer.peerrepository has been removed. Use repository.peer abstract
base class to represent a peer repository.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D338

AFFECTED FILES
  mercurial/httppeer.py
  mercurial/peer.py
  mercurial/sshpeer.py
  mercurial/wireproto.py
  tests/notcapable
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -19,7 +19,26 @@
 def __init__(self, serverrepo):
 self.serverrepo = serverrepo
 
-def _capabilities(self):
+@property
+def ui(self):
+return self.serverrepo.ui
+
+def url(self):
+return 'test'
+
+def local(self):
+return None
+
+def peer(self):
+return self
+
+def canpush(self):
+return True
+
+def close(self):
+pass
+
+def capabilities(self):
 return ['batch']
 
 def _call(self, cmd, **args):
diff --git a/tests/notcapable b/tests/notcapable
--- a/tests/notcapable
+++ b/tests/notcapable
@@ -6,9 +6,9 @@
 fi
 
 cat > notcapable-$CAP.py << EOF
-from mercurial import extensions, peer, localrepo
+from mercurial import extensions, localrepo, repository
 def extsetup():
-extensions.wrapfunction(peer.peerrepository, 'capable', wrapcapable)
+extensions.wrapfunction(repository.peer, 'capable', wrapcapable)
 extensions.wrapfunction(localrepo.localrepository, 'peer', wrappeer)
 def wrapcapable(orig, self, name, *args, **kwargs):
 if name in '$CAP'.split(' '):
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -27,6 +27,7 @@
 peer,
 pushkey as pushkeymod,
 pycompat,
+repository,
 streamclone,
 util,
 )
@@ -212,36 +213,15 @@
 
 # client side
 
-class wirepeer(peer.peerrepository):
+class wirepeer(repository.legacypeer):
 """Client-side interface for communicating with a peer repository.
 
 Methods commonly call wire protocol commands of the same name.
 
 See also httppeer.py and sshpeer.py for protocol-specific
 implementations of this interface.
 """
-def _submitbatch(self, req):
-"""run batch request  on the server
-
-Returns an iterator of the raw responses from the server.
-"""
-rsp = self._callstream("batch", cmds=encodebatchcmds(req))
-chunk = rsp.read(1024)
-work = [chunk]
-while chunk:
-while ';' not in chunk and chunk:
-chunk = rsp.read(1024)
-work.append(chunk)
-merged = ''.join(work)
-while ';' in merged:
-one, merged = merged.split(';', 1)
-yield unescapearg(one)
-chunk = rsp.read(1024)
-work = [merged, chunk]
-yield unescapearg(''.join(work))
-
-def _submitone(self, op, args):
-return self._call(op, **args)
+# Begin of basewirepeer interface.
 
 def iterbatch(self):
 return remoteiterbatcher(self)
@@ -293,26 +273,17 @@
 except TypeError:
 self._abort(error.ResponseError(_("unexpected response:"), d))
 
-def branches(self, nodes):
-n = encodelist(nodes)
-d = self._call("branches", nodes=n)
-try:
-br = [tuple(decodelist(b)) for b in d.splitlines()]
-return br
-except ValueError:
-self._abort(error.ResponseError(_("unexpected response:"), d))
-
-def between(self, pairs):
-batch = 8 # avoid giant requests
-r = []
-for i in xrange(0, len(pairs), batch):
-n = " ".join([encodelist(p, '-') for p in pairs[i:i + batch]])
-d = self._call("between", pairs=n)
-try:
-r.extend(l and decodelist(l) or [] for l in d.splitlines())
-except ValueError:
-self._abort(error.ResponseError(_("unexpected response:"), d))
-return r
+@batchable
+def listkeys(self, namespace):
+if not self.capable('pushkey'):
+yield {}, None
+f = future()
+

D337: httppeer: use peer interface

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is similar to what we did to sshpeer. Quirks and all.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D337

AFFECTED FILES
  mercurial/httppeer.py

CHANGE DETAILS

diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -21,6 +21,7 @@
 error,
 httpconnection,
 pycompat,
+repository,
 statichttprepo,
 url,
 util,
@@ -86,7 +87,7 @@
 
 resp.__class__ = readerproxy
 
-class httppeer(wireproto.wirepeer):
+class httppeer(wireproto.wirepeer, repository.legacypeer):
 def __init__(self, ui, path):
 self._path = path
 self._caps = None
@@ -100,28 +101,48 @@
 # urllib cannot handle URLs with embedded user or passwd
 self._url, authinfo = u.authinfo()
 
-self.ui = ui
-self.ui.debug('using %s\n' % self._url)
+self._ui = ui
+ui.debug('using %s\n' % self._url)
 
 self._urlopener = url.opener(ui, authinfo)
 self._requestbuilder = urlreq.request
 
+# TODO remove once peerrepository isn't in inheritance.
+self._capabilities = self.capabilities
+
 def __del__(self):
 urlopener = getattr(self, '_urlopener', None)
 if urlopener:
 for h in urlopener.handlers:
 h.close()
 getattr(h, "close_all", lambda : None)()
 
+# Begin of _basepeer interface.
+
+@util.propertycache
+def ui(self):
+return self._ui
+
 def url(self):
 return self._path
 
-# look up capabilities only when needed
+def local(self):
+return None
+
+def peer(self):
+return self
+
+def canpush(self):
+return True
 
-def _fetchcaps(self):
-self._caps = set(self._call('capabilities').split())
+def close(self):
+pass
 
-def _capabilities(self):
+# End of _basepeer interface.
+
+# Begin of _basewirepeer interface.
+
+def capabilities(self):
 if self._caps is None:
 try:
 self._fetchcaps()
@@ -131,6 +152,13 @@
   (' '.join(self._caps or ['none'])))
 return self._caps
 
+# End of _basewirepeer interface.
+
+# look up capabilities only when needed
+
+def _fetchcaps(self):
+self._caps = set(self._call('capabilities').split())
+
 def _callstream(self, cmd, _compressible=False, **args):
 if cmd == 'pushkey':
 args['data'] = ''



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D335: localrepo: use peer interfaces

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We now have a formal abstract base class for peers. Let's
  transition the peer classes in localrepo to it.
  
  As part of the transition, we reorder methods so they are grouped
  by interface and match the order they are defined in the interface.
  We also had to change self.ui from an instance attribute to a
  property to satisfy the @abstractproperty requirement.
  
  As part of this change, we uncover the first "bug" as part of
  enforcing interfaces: stream_out() wasn't implemented on localpeer!
  This isn't technically a bug since the repo isn't advertising the
  stream capability, so clients shouldn't be attempting to call it.
  But I don't think there's a good reason why this is the case.
  We implement a dummy method to satisfy the interface requriements.
  We can make localpeer instances streamable as a future enhancement.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D335

AFFECTED FILES
  mercurial/localrepo.py

CHANGE DETAILS

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -49,6 +49,7 @@
 phases,
 pushkey,
 pycompat,
+repository,
 repoview,
 revset,
 revsetlang,
@@ -144,43 +145,52 @@
   'unbundle'}
 legacycaps = moderncaps.union({'changegroupsubset'})
 
-class localpeer(peer.peerrepository):
+class localpeer(repository.peer):
 '''peer for a local repo; reflects only the most recent API'''
 
 def __init__(self, repo, caps=None):
+super(localpeer, self).__init__()
+
 if caps is None:
 caps = moderncaps.copy()
-peer.peerrepository.__init__(self)
 self._repo = repo.filtered('served')
-self.ui = repo.ui
+self._ui = repo.ui
 self._caps = repo._restrictcapabilities(caps)
 
+# Begin of _basepeer interface.
+
+@util.propertycache
+def ui(self):
+return self._ui
+
+def url(self):
+return self._repo.url()
+
+def local(self):
+return self._repo
+
+def peer(self):
+return self
+
+def canpush(self):
+return True
+
 def close(self):
 self._repo.close()
 
-def _capabilities(self):
-return self._caps
-
-def local(self):
-return self._repo
+# End of _basepeer interface.
 
-def canpush(self):
-return True
-
-def url(self):
-return self._repo.url()
-
-def lookup(self, key):
-return self._repo.lookup(key)
+# Begin of _basewirecommands interface.
 
 def branchmap(self):
 return self._repo.branchmap()
 
-def heads(self):
-return self._repo.heads()
+def capabilities(self):
+return self._caps
 
-def known(self, nodes):
-return self._repo.known(nodes)
+def debugwireargs(self, one, two, three=None, four=None, five=None):
+"""Used to test argument passing over the wire"""
+return "%s %s %s %s %s" % (one, two, three, four, five)
 
 def getbundle(self, source, heads=None, common=None, bundlecaps=None,
   **kwargs):
@@ -197,8 +207,24 @@
 else:
 return changegroup.getunbundler('01', cb, None)
 
-# TODO We might want to move the next two calls into legacypeer and add
-# unbundle instead.
+def heads(self):
+return self._repo.heads()
+
+def known(self, nodes):
+return self._repo.known(nodes)
+
+def listkeys(self, namespace):
+return self._repo.listkeys(namespace)
+
+def lookup(self, key):
+return self._repo.lookup(key)
+
+def pushkey(self, namespace, key, old, new):
+return self._repo.pushkey(namespace, key, old, new)
+
+def stream_out(self):
+raise error.Abort(_('cannot perform stream clone against local '
+'peer'))
 
 def unbundle(self, cg, heads, url):
 """apply a bundle on a repo
@@ -235,35 +261,38 @@
 except error.PushRaced as exc:
 raise error.ResponseError(_('push failed:'), str(exc))
 
-def pushkey(self, namespace, key, old, new):
-return self._repo.pushkey(namespace, key, old, new)
+# End of _basewirecommands interface.
 
-def listkeys(self, namespace):
-return self._repo.listkeys(namespace)
+# Begin of peer interface.
 
-def debugwireargs(self, one, two, three=None, four=None, five=None):
-'''used to test argument passing over the wire'''
-return "%s %s %s %s %s" % (one, two, three, four, five)
+def iterbatch(self):
+return peer.localiterbatcher(self)
 
-class locallegacypeer(localpeer):
+# End of peer interface.
+
+class locallegacypeer(repository.legacypeer, localpeer):
 '''peer extension which implements legacy methods too; used for tests with
 restricted capabilities'''
 
 def __init__(self, 

D332: repository: formalize peer interface with abstract base class

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  There are various interfaces for interacting with repositories
  and peers. They form a contract for how one should interact with
  a repo or peer object.
  
  The contracts today aren't very well-defined or enforced. There
  have been several bugs over the years where peers or repo types
  have forgotten to implement certain methods. In addition, the
  inheritance of some classes is wonky. For example, localrepository
  doesn't inherit from an interface and the god-object nature of
  that class means the repository interface isn't well-defined. Other
  repository types inherit from localrepository then stub out
  methods that don't make sense (e.g. statichttprepository
  re-defining locking methods to fail fast).
  
  Not having well-defined interfaces makes implementing alternate
  storage backends, wire protocol transports, and repository types
  difficult because it isn't clear what exactly needs to be
  implemented.
  
  This patch starts the process of attempting to establish more
  order to the type system around repositories and peers.
  
  Our first patch starts with a problem space that already has a
  partial solution: peers. The peer.peerrepository class already
  somewhat defines a peer interface. But it is missing a few things
  and the total interface isn't well-defined because it is combined
  with wireproto.wirepeer.
  
  Our newly-established basepeer class uses the abc module to
  declare an abstract base class with the properties and methods that
  a generic peer must implement.
  
  We create a new class that inherits from it. This class will hold
  our other future abstract base classes / interfaces so we can expose
  a unified base class/interface.
  
  We don't yet use the new interface because subsequent additions
  will break existing code without some refactoring first.
  
  A new module (repository.py) was created to hold the interfaces.
  I could have put things in peer.py. However, I have plans to
  eventually add interfaces to define repository and storage types.
  These almost certainly require a new module. And I figured having
  all the interfaces live in one module makes sense. So I created
  repository.py to be that future home.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D332

AFFECTED FILES
  mercurial/repository.py

CHANGE DETAILS

diff --git a/mercurial/repository.py b/mercurial/repository.py
new file mode 100644
--- /dev/null
+++ b/mercurial/repository.py
@@ -0,0 +1,62 @@
+# repository.py - Interfaces and base classes for repositories and peers.
+#
+# Copyright 2017 Gregory Szorc 
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+import abc
+
+class _basepeer(object):
+"""Represents a "connection" to a repository.
+
+This is the base interface for representing a connection to a repository.
+It holds basic properties and methods applicable to all peer types.
+
+This is not a complete interface definition and should not be used
+outside of this module.
+"""
+__metaclass__ = abc.ABCMeta
+
+@abc.abstractproperty
+def ui(self):
+"""ui.ui instance."""
+
+@abc.abstractmethod
+def url(self):
+"""Returns a string representing this peer."""
+
+@abc.abstractmethod
+def local(self):
+"""Returns a local repository instance.
+
+If the peer represents a local repository, returns an object that
+can be used to interface with it. Otherwise returns ``None``.
+"""
+
+@abc.abstractmethod
+def peer(self):
+"""Returns an object conforming to this interface.
+
+Most implementations will ``return self``.
+"""
+
+@abc.abstractmethod
+def canpush(self):
+"""Returns a boolean indicating if this peer can be pushed to."""
+
+@abc.abstractmethod
+def close(self):
+"""Close the connection to this peer.
+
+This is called when the peer will no longer be used. Resources
+associated with the peer should be cleaned up.
+"""
+
+class peer(_basepeer):
+"""Unified interface and base class for peer repositories.
+
+All peer instances must inherit from this class.
+"""



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D339: tests: verify that peer instances only expose interface members

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Our abstract interfaces are more useful if we guarantee that
  implementations conform to certain rules. Namely, we want to ensure
  that objects implementing interfaces don't expose new public
  attributes that aren't part of the interface. That way, as long as
  consumers don't access "internal" attributes (those beginning with
  "_") then (in theory) objects implementing interfaces can be swapped
  out and everything will "just work."
  
  We add a test that enforces our "no public attributes not part
  of the abstract interface" rule.
  
  We /could/ implement "interface compliance detection" at run-time.
  However, that is littered with problems.
  
  The obvious solutions are custom __new__ and __init__ methods.
  These rely on derived types actually calling the parent's
  implementation, which is no sure bet. Furthermore, __new__ and
  __init__ will likely be called before instance-specific attributes
  are assigned. In other words, they won't detect public attributes
  set on self.__dict__. This means public attribute detection won't
  be robust.
  
  We could work around lack of robust self.__dict__ public attribute
  detection by having our interfaces implement a custom __getattribute__,
  __getattr__, and/or __setattr__. However, this incurs an undesirable
  run-time penalty. And, subclasses could override our custom
  method, bypassing the check.
  
  The most robust solution is a non-runtime test. So that's what this
  commit implements. We have a generic function for validating that an
  object only has public attributes defined by abstract classes. Then,
  we instantiate some peers and verify a newly constructed object
  plays by the rules.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D339

AFFECTED FILES
  tests/test-check-interfaces.py
  tests/test-check-interfaces.py.out

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py.out 
b/tests/test-check-interfaces.py.out
new file mode 100644
--- /dev/null
+++ b/tests/test-check-interfaces.py.out
@@ -0,0 +1,2 @@
+public attributes not in abstract interface: badpeer.badattribute
+public attributes not in abstract interface: badpeer.badmethod
diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
new file mode 100644
--- /dev/null
+++ b/tests/test-check-interfaces.py
@@ -0,0 +1,71 @@
+# Test that certain objects conform to well-defined interfaces.
+
+from __future__ import absolute_import, print_function
+
+from mercurial import (
+httppeer,
+localrepo,
+sshpeer,
+ui as uimod,
+)
+
+def checkobject(o):
+"""Verify a constructed object conforms to interface rules.
+
+An object must have __abstractmethods__ defined.
+
+All "public" attributes of the object (attributes not prefixed with
+an underscore) must be in __abstractmethods__ or appear on a base class
+with __abstractmethods__.
+"""
+name = o.__class__.__name__
+
+allowed = set()
+for cls in o.__class__.__mro__:
+if not getattr(cls, '__abstractmethods__', set()):
+continue
+
+allowed |= cls.__abstractmethods__
+allowed |= {a for a in dir(cls) if not a.startswith('_')}
+
+if not allowed:
+print('%s does not have abstract methods' % name)
+return
+
+public = {a for a in dir(o) if not a.startswith('_')}
+
+for attr in sorted(public - allowed):
+print('public attributes not in abstract interface: %s.%s' % (
+name, attr))
+
+# Facilitates testing localpeer.
+class dummyrepo(object):
+def __init__(self):
+self.ui = uimod.ui()
+def filtered(self, name):
+pass
+def _restrictcapabilities(self, caps):
+pass
+
+# Facilitates testing sshpeer without requiring an SSH server.
+class testingsshpeer(sshpeer.sshpeer):
+def _validaterepo(self, *args, **kwargs):
+pass
+
+class badpeer(httppeer.httppeer):
+def __init__(self):
+super(badpeer, self).__init__(uimod.ui(), 'http://localhost')
+self.badattribute = True
+
+def badmethod(self):
+pass
+
+def main():
+ui = uimod.ui()
+
+checkobject(badpeer())
+checkobject(httppeer.httppeer(ui, 'http://localhost'))
+checkobject(localrepo.localpeer(dummyrepo()))
+checkobject(testingsshpeer(ui, 'ssh://localhost/foo'))
+
+main()



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D336: sshpeer: use peer interface

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We need the same @property conversion of ui like we did for localpeer.
  We renamed _capabilities() to capabilities() to satisfy the new
  naming requirement.
  
  However, since we're inheriting from wireproto.wirepeer which inherits
  from peer.peerrepository and provides its own code accessing
  _capabilities(), we need to keep the old alias around. This wonkiness
  will disappear once wirepeer is cleaned up in subsequent commits.
  
  We also implement methods for basepeer that are identical to the
  defaults in peer.peerrepository in preparation for the removal of
  peerrepository.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D336

AFFECTED FILES
  mercurial/sshpeer.py

CHANGE DETAILS

diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -13,6 +13,7 @@
 from . import (
 error,
 pycompat,
+repository,
 util,
 wireproto,
 )
@@ -114,10 +115,10 @@
 def flush(self):
 return self._main.flush()
 
-class sshpeer(wireproto.wirepeer):
+class sshpeer(wireproto.wirepeer, repository.legacypeer):
 def __init__(self, ui, path, create=False):
 self._url = path
-self.ui = ui
+self._ui = ui
 self._pipeo = self._pipei = self._pipee = None
 
 u = util.url(path, parsequery=False, parsefragment=False)
@@ -150,9 +151,39 @@
 
 self._validaterepo(sshcmd, args, remotecmd)
 
+# TODO remove this alias once peerrepository inheritance is removed.
+self._capabilities = self.capabilities
+
+# Begin of _basepeer interface.
+
+@util.propertycache
+def ui(self):
+return self._ui
+
 def url(self):
 return self._url
 
+def local(self):
+return None
+
+def peer(self):
+return self
+
+def canpush(self):
+return True
+
+def close(self):
+pass
+
+# End of _basepeer interface.
+
+# Begin of _basewirecommands interface.
+
+def capabilities(self):
+return self._caps
+
+# End of _basewirecommands interface.
+
 def _validaterepo(self, sshcmd, args, remotecmd):
 # cleanup up previous run
 self._cleanup()
@@ -200,9 +231,6 @@
 self._caps.update(l[:-1].split(":")[1].split())
 break
 
-def _capabilities(self):
-return self._caps
-
 def _readerr(self):
 _forwardoutput(self.ui, self._pipee)
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D333: repository: formalize wire protocol interface

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  There are a well-defined set of commands constituting the wire
  protocol. Interaction with these and methods for calling them in
  batches are exposed via methods on peer instances.
  
  Let's formalize support for these features in abstract classes.
  
  The command parts come from the existing wireproto.wirepeer class.
  The batch methods come from peer.peerrepository.
  
  Ample documentation has been added as part of defining the interfaces.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D333

AFFECTED FILES
  mercurial/repository.py

CHANGE DETAILS

diff --git a/mercurial/repository.py b/mercurial/repository.py
--- a/mercurial/repository.py
+++ b/mercurial/repository.py
@@ -55,8 +55,166 @@
 associated with the peer should be cleaned up.
 """
 
-class peer(_basepeer):
+class _basewirecommands(object):
+"""Client-side interfac for communicating over the wire protocol.
+
+This interface is used as a gateway to the Mercurial wire protocol.
+methods commonly call wire protocol commands of the same name.
+"""
+__metaclass__ = abc.ABCMeta
+
+@abc.abstractmethod
+def branchmap(self):
+"""Obtain heads in named branches.
+
+Returns a dict mapping branch name to an iterable of nodes that are
+heads on that branch.
+"""
+
+@abc.abstractmethod
+def capabilities(self):
+"""Obtain capabilities of the peer.
+
+Returns a set of string capabilities.
+"""
+
+@abc.abstractmethod
+def debugwireargs(self, one, two, three=None, four=None, five=None):
+"""Used to facilitate debugging of arguments passed over the wire."""
+
+@abc.abstractmethod
+def getbundle(self, source, **kwargs):
+"""Obtain remote repository data as a bundle.
+
+This command is how the bulk of repository data is transferred from
+the peer to the local repository
+
+Returns a generator of bundle data.
+"""
+
+@abc.abstractmethod
+def heads(self):
+"""Determine all known head revisions in the peer.
+
+Returns an iterable of binary nodes.
+"""
+
+@abc.abstractmethod
+def known(self, nodes):
+"""Determine whether multiple nodes are known.
+
+Accepts an iterable of nodes whose presence to check for.
+
+Returns an iterable of booleans indicating of the corresponding node
+at that index is known to the peer.
+"""
+
+@abc.abstractmethod
+def listkeys(self, namespace):
+"""Obtain all keys in a pushkey namespace.
+
+Returns an iterable of key names.
+"""
+
+@abc.abstractmethod
+def lookup(self, key):
+"""Resolve a value to a known revision.
+
+Returns a binary node of the resolved revision on success.
+"""
+
+@abc.abstractmethod
+def pushkey(self, namespace, key, old, new):
+"""Set a value using the ``pushkey`` protocol.
+
+Arguments correspond to the pushkey namespace and key to operate on and
+the old and new values for that key.
+
+Returns a string with the peer result. The value inside varies by the
+namespace.
+"""
+
+@abc.abstractmethod
+def stream_out(self):
+"""Obtain streaming clone data.
+
+Successful result should be a generator of data chunks.
+"""
+
+@abc.abstractmethod
+def unbundle(self, bundle, heads, url):
+"""Transfer repository data to the peer.
+
+This is how the bulk of data during a push is transferred.
+
+Returns the integer number of heads added to the peer.
+"""
+
+class _baselegacywirecommands(object):
+"""Interface for implementing support for legacy wire protocol commands.
+
+Wire protocol commands transition to legacy status when they are no longer
+used by modern clients. To facilitate identifying which commands are
+legacy, the interfaces are split.
+"""
+__metaclass__ = abc.ABCMeta
+
+@abc.abstractmethod
+def between(self, pairs):
+"""Obtain nodes between pairs of nodes.
+
+``pairs`` is an iterable of node pairs.
+
+Returns an iterable of iterables of nodes corresponding to each
+requested pair.
+"""
+
+@abc.abstractmethod
+def branches(self, nodes):
+"""Obtain ancestor changesets of specific nodes back to a branch point.
+
+For each requested node, the peer finds the first ancestor node that is
+a DAG root or is a merge.
+
+Returns an iterable of iterables with the resolved values for each 
node.
+"""
+
+@abc.abstractmethod
+def changegroup(self, nodes, kind):
+"""Obtain a changegroup with data for descendants of specified 
nodes."""
+
+@abc.abstractmethod
+def changegroupsubset(self, bases, 

D334: repository: implement generic capability methods on peer class

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  These methods are part of the peer interface, are generic, and can
  be implemented in terms of other members of the peer interface. So we
  implement them on the peer base class as a convenience.
  
  The implementation is essentially copied from peer.py. The code
  in peer.py will eventually be deleted.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D334

AFFECTED FILES
  mercurial/repository.py

CHANGE DETAILS

diff --git a/mercurial/repository.py b/mercurial/repository.py
--- a/mercurial/repository.py
+++ b/mercurial/repository.py
@@ -9,6 +9,11 @@
 
 import abc
 
+from .i18n import _
+from . import (
+error,
+)
+
 class _basepeer(object):
 """Represents a "connection" to a repository.
 
@@ -216,5 +221,36 @@
 of results in the order they were called.
 """
 
+def capable(self, name):
+"""Determine support for a named capability.
+
+Returns ``False`` if capability not supported.
+
+Returns ``True`` if boolean capability is supported. Returns a string
+if capability support is non-boolean.
+"""
+caps = self.capabilities()
+if name in caps:
+return True
+
+name = '%s=' % name
+for cap in caps:
+if cap.startswith(name):
+return cap[len(name):]
+
+return False
+
+def requirecap(self, name, purpose):
+"""Require a capability to be present.
+
+Raises a ``CapabilityError`` if the capability isn't present.
+"""
+if self.capable(name):
+return
+
+raise error.CapabilityError(
+_('cannot %s; remote repository does not support the %r '
+  'capability') % (purpose, name))
+
 class legacypeer(peer, _baselegacywirecommands):
 """peer but with support for legacy wire protocol commands."""



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D330: Backed out changeset c34532365b38

[glandium (Mike Hommey)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1814ca418b30: branchmap: revert c34532365b38 for Python 2.7 
compatibility (authored by glandium).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D330?vs=752=755

REVISION DETAIL
  https://phab.mercurial-scm.org/D330

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -406,7 +406,8 @@
 
 # fast path: extract data from cache, use it if node is matching
 reponode = changelog.node(rev)[:_rbcnodelen]
-cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, 
rbcrevidx)
+cachenode, branchidx = unpack_from(
+_rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
 close = bool(branchidx & _rbccloseflag)
 if close:
 branchidx &= _rbcbranchidxmask



To: glandium, #hg-reviewers, quark, indygreg
Cc: indygreg, quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D21: rebase: rewrite core algorithm (issue5578) (issue5630)

[quark (Jun Wu)]

quark planned changes to this revision.
quark added inline comments.

INLINE COMMENTS

> test-rebase-obsolete.t:1064-1065
>$ hg log -G
> -  o  7:9ed45af61fa0 F
> -  |
>o  6:8f47515dda15 D

This case needs investigation.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D21

To: quark, durin42, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D21: rebase: rewrite core algorithm (issue5578) (issue5630)

[quark (Jun Wu)]

quark updated this revision to Diff 754.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D21?vs=753=754

REVISION DETAIL
  https://phab.mercurial-scm.org/D21

AFFECTED FILES
  hgext/rebase.py
  tests/test-rebase-brute-force.t
  tests/test-rebase-newancestor.t
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -488,9 +488,33 @@
   >   A
   > EOF
 
-BROKEN: This raises an exception
-  $ hg rebase -d G -r 'B + D + F' 2>&1 | grep '^AssertionError'
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d G -r 'B + D + F'
+  rebasing 1:112478962961 "B" (B)
+  rebasing 2:b18e25de2cf5 "D" (D)
+  not rebasing ignored 4:26805aba1e60 "C" (C)
+  not rebasing ignored 5:4b61ff5c62e2 "E" (E)
+  rebasing 6:f15c3adaf214 "F" (F tip)
+  abort: cannot use revision 6 as base, result would have 3 parents
+  [255]
+  $ hg log -G
+  @  8:1971b36cbe4e D
+  |
+  | o  7:f7e03ec6c148 B
+  |/
+  | o6:f15c3adaf214 F
+  | |\
+  | | o  5:4b61ff5c62e2 E
+  | | |
+  | o |  4:26805aba1e60 C
+  | | |
+  o | |  3:6fa3874a3b67 G
+  | | |
+  +---o  2:b18e25de2cf5 D
+  | |
+  | o  1:112478962961 B
+  |/
+  o  0:426bada5c675 A
+  
 
   $ cd ..
 
@@ -962,17 +986,19 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d B -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d B -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 4:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o4:66f1a38021c9 F
+  o5:aae1787dacee F
   |\
-  | x  3:7fb047a69f22 E
-  | |
-  o |  2:b18e25de2cf5 D
-  |/
-  | o  1:112478962961 B
+  | | x  4:66f1a38021c9 F
+  | |/|
+  | | x  3:7fb047a69f22 E
+  | | |
+  | o |  2:b18e25de2cf5 D
+  | |/
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -994,19 +1020,19 @@
   $ hg rebase -d C -s D
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: not rebased on top of requested destination (C)
+
   $ hg log -G
-  o6:50e9d60b99c6 F
+  o6:0913febf6439 F
   |\
-  | | x  5:66f1a38021c9 F
-  | |/|
-  +-o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
+  | | |
+  | o |  4:26805aba1e60 C
   | | |
-  | o |  3:7fb047a69f22 E
+  o | |  3:7fb047a69f22 E
   | | |
-  | | x  2:b18e25de2cf5 D
-  | |/
-  o |  1:112478962961 B
+  +---x  2:b18e25de2cf5 D
+  | |
+  | o  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1025,19 +1051,21 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d C -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d C -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 5:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o5:66f1a38021c9 F
+  o6:c6ab0cc6d220 F
   |\
-  | | o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
   | | |
-  | x |  3:7fb047a69f22 E
+  | o |  4:26805aba1e60 C
   | | |
-  o | |  2:b18e25de2cf5 D
-  |/ /
-  | o  1:112478962961 B
+  | | x  3:7fb047a69f22 E
+  | | |
+  o---+  2:b18e25de2cf5 D
+   / /
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1060,9 +1088,8 @@
   rebasing 2:b18e25de2cf5 "D" (D)
   note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
   $ hg log -G
-  o  7:9ed45af61fa0 F
-  |
   o  6:8f47515dda15 D
   |
   | x5:66f1a38021c9 F
@@ -1096,13 +1123,14 @@
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 3:7fb047a69f22 "E" (E)
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: This should have resulted in a rebased F with one parent, just like in
-the test case above
+
+Rebased F should have one parent, just like in the test case above
+
   $ hg log -G
-  o7:c1e6f26e339d F
-  |\
-  | o  6:533690786a86 E
-  |/
+  o  7:502540f44880 F
+  |
+  o  6:533690786a86 E
+  |
   | x5:66f1a38021c9 F
   | |\
   o | |  4:26805aba1e60 C
diff --git a/tests/test-rebase-newancestor.t b/tests/test-rebase-newancestor.t
--- a/tests/test-rebase-newancestor.t
+++ b/tests/test-rebase-newancestor.t
@@ -284,18 +284,7 @@
   rebasing 6:4c5f12f25ebe "merge rebase ancestors" (tip)
   resolving manifests
   removing other
-  note: merging f9daf77ffe76+ and 4c5f12f25ebe using bids from ancestors 
a60552eb93fb and f59da8fc0fcf
-  
-  calculating bids for ancestor a60552eb93fb
   resolving manifests
-  
-  calculating bids for ancestor f59da8fc0fcf
-  resolving manifests
-  
-  auction for merging merge bids
-   other: consensus for g
-  end of auction
-  
   getting other
   committing files:
   other

D21: rebase: rewrite core algorithm (issue5578) (issue5630)

[quark (Jun Wu)]

quark updated this revision to Diff 753.
quark marked an inline comment as done.
quark edited the summary of this revision.
quark retitled this revision from "rebase: rewrite defineparents" to "rebase: 
rewrite core algorithm (issue5578) (issue5630)".
This revision is now accepted and ready to land.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D21?vs=161=753

REVISION DETAIL
  https://phab.mercurial-scm.org/D21

AFFECTED FILES
  hgext/rebase.py
  tests/test-rebase-brute-force.t
  tests/test-rebase-newancestor.t
  tests/test-rebase-obsolete.t

CHANGE DETAILS

diff --git a/tests/test-rebase-obsolete.t b/tests/test-rebase-obsolete.t
--- a/tests/test-rebase-obsolete.t
+++ b/tests/test-rebase-obsolete.t
@@ -488,9 +488,33 @@
   >   A
   > EOF
 
-BROKEN: This raises an exception
-  $ hg rebase -d G -r 'B + D + F' 2>&1 | grep '^AssertionError'
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d G -r 'B + D + F'
+  rebasing 1:112478962961 "B" (B)
+  rebasing 2:b18e25de2cf5 "D" (D)
+  not rebasing ignored 4:26805aba1e60 "C" (C)
+  not rebasing ignored 5:4b61ff5c62e2 "E" (E)
+  rebasing 6:f15c3adaf214 "F" (F tip)
+  abort: cannot use revision 6 as base, result would have 3 parents
+  [255]
+  $ hg log -G
+  @  8:1971b36cbe4e D
+  |
+  | o  7:f7e03ec6c148 B
+  |/
+  | o6:f15c3adaf214 F
+  | |\
+  | | o  5:4b61ff5c62e2 E
+  | | |
+  | o |  4:26805aba1e60 C
+  | | |
+  o | |  3:6fa3874a3b67 G
+  | | |
+  +---o  2:b18e25de2cf5 D
+  | |
+  | o  1:112478962961 B
+  |/
+  o  0:426bada5c675 A
+  
 
   $ cd ..
 
@@ -962,17 +986,19 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d B -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d B -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 4:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o4:66f1a38021c9 F
+  o5:aae1787dacee F
   |\
-  | x  3:7fb047a69f22 E
-  | |
-  o |  2:b18e25de2cf5 D
-  |/
-  | o  1:112478962961 B
+  | | x  4:66f1a38021c9 F
+  | |/|
+  | | x  3:7fb047a69f22 E
+  | | |
+  | o |  2:b18e25de2cf5 D
+  | |/
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -994,19 +1020,19 @@
   $ hg rebase -d C -s D
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: not rebased on top of requested destination (C)
+
   $ hg log -G
-  o6:50e9d60b99c6 F
+  o6:0913febf6439 F
   |\
-  | | x  5:66f1a38021c9 F
-  | |/|
-  +-o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
+  | | |
+  | o |  4:26805aba1e60 C
   | | |
-  | o |  3:7fb047a69f22 E
+  o | |  3:7fb047a69f22 E
   | | |
-  | | x  2:b18e25de2cf5 D
-  | |/
-  o |  1:112478962961 B
+  +---x  2:b18e25de2cf5 D
+  | |
+  | o  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1025,19 +1051,21 @@
   >   A
   > EOF
 
-BROKEN: Raises an exception
-  $ hg rebase -d C -s E 2>&1 | grep AssertionError:
-  AssertionError: no base found to rebase on (defineparents called wrong)
+  $ hg rebase -d C -s E
+  note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
+  rebasing 5:66f1a38021c9 "F" (F tip)
   $ hg log -G
-  o5:66f1a38021c9 F
+  o6:c6ab0cc6d220 F
   |\
-  | | o  4:26805aba1e60 C
+  +---x  5:66f1a38021c9 F
   | | |
-  | x |  3:7fb047a69f22 E
+  | o |  4:26805aba1e60 C
   | | |
-  o | |  2:b18e25de2cf5 D
-  |/ /
-  | o  1:112478962961 B
+  | | x  3:7fb047a69f22 E
+  | | |
+  o---+  2:b18e25de2cf5 D
+   / /
+  o /  1:112478962961 B
   |/
   o  0:426bada5c675 A
   
@@ -1060,9 +1088,8 @@
   rebasing 2:b18e25de2cf5 "D" (D)
   note: not rebasing 3:7fb047a69f22 "E" (E), already in destination as 
1:112478962961 "B"
   rebasing 5:66f1a38021c9 "F" (F tip)
+  note: rebase of 5:66f1a38021c9 created no changes to commit
   $ hg log -G
-  o  7:9ed45af61fa0 F
-  |
   o  6:8f47515dda15 D
   |
   | x5:66f1a38021c9 F
@@ -1096,13 +1123,14 @@
   note: not rebasing 2:b18e25de2cf5 "D" (D), already in destination as 
1:112478962961 "B"
   rebasing 3:7fb047a69f22 "E" (E)
   rebasing 5:66f1a38021c9 "F" (F tip)
-BROKEN: This should have resulted in a rebased F with one parent, just like in
-the test case above
+
+Rebased F should have one parent, just like in the test case above
+
   $ hg log -G
-  o7:c1e6f26e339d F
-  |\
-  | o  6:533690786a86 E
-  |/
+  o  7:502540f44880 F
+  |
+  o  6:533690786a86 E
+  |
   | x5:66f1a38021c9 F
   | |\
   o | |  4:26805aba1e60 C
diff --git a/tests/test-rebase-newancestor.t b/tests/test-rebase-newancestor.t
--- a/tests/test-rebase-newancestor.t
+++ b/tests/test-rebase-newancestor.t
@@ -284,18 +284,7 @@
   rebasing 6:4c5f12f25ebe "merge rebase ancestors" (tip)
   resolving manifests
   removing other
-  note: merging f9daf77ffe76+ and 4c5f12f25ebe using bids from ancestors 
a60552eb93fb and f59da8fc0fcf
-  
-  calculating 

mercurial@33662: 14 new changesets (14 on stable)

[Mercurial Commits]

14 new changesets (14 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/377e8ddaebef
changeset:   33649:377e8ddaebef
branch:  stable
user:Yuya Nishihara 
date:Wed Jul 26 22:10:15 2017 +0900
summary: pathauditor: disable cache of audited paths by default (issue5628)

https://www.mercurial-scm.org/repo/hg/rev/0b3fe3910ef5
changeset:   33650:0b3fe3910ef5
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:32:25 2017 -0700
summary: util: add utility method to check for bad ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/739cc0f9cbb4
changeset:   33651:739cc0f9cbb4
branch:  stable
user:Augie Fackler 
date:Fri Aug 04 14:00:03 2017 -0400
summary: ssh: ban any username@host or host that starts with - (SEC)

https://www.mercurial-scm.org/repo/hg/rev/d7a1c4c1181d
changeset:   33652:d7a1c4c1181d
branch:  stable
user:Sean Farley 
date:Tue Aug 01 14:40:19 2017 -0700
summary: sshpeer: check for safe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/0bcceb58b036
changeset:   33653:0bcceb58b036
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:36:36 2017 -0700
summary: clone: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/0ee42a0df5be
changeset:   33654:0ee42a0df5be
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:47:32 2017 -0700
summary: pull: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/48d520fdf880
changeset:   33655:48d520fdf880
branch:  stable
user:Sean Farley 
date:Mon Jul 31 14:40:28 2017 -0700
summary: push: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/475af2f89636
changeset:   33656:475af2f89636
branch:  stable
user:Sean Farley 
date:Mon Jul 31 16:04:44 2017 -0700
summary: subrepo: add tests for hg rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/60ee7af2a2ba
changeset:   33657:60ee7af2a2ba
branch:  stable
user:Sean Farley 
date:Mon Jul 31 16:44:17 2017 -0700
summary: subrepo: add tests for svn rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/db83a1df03fe
changeset:   33658:db83a1df03fe
branch:  stable
user:Sean Farley 
date:Mon Jul 31 14:55:11 2017 -0700
summary: subrepo: add tests for git rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/8cb9e921ef8c
changeset:   33659:8cb9e921ef8c
branch:  stable
user:Jun Wu 
date:Fri Aug 04 23:54:12 2017 -0700
summary: ssh: quote parameters using shellquote (SEC)

https://www.mercurial-scm.org/repo/hg/rev/3fee7f7d2da0
changeset:   33660:3fee7f7d2da0
branch:  stable
tag: 4.3.1
user:Yuya Nishihara 
date:Mon Aug 07 22:22:28 2017 +0900
summary: ssh: unban the use of pipe character in user@host:port string

https://www.mercurial-scm.org/repo/hg/rev/a864659cfcbe
changeset:   33661:a864659cfcbe
branch:  stable
user:Augie Fackler 
date:Thu Aug 10 18:46:54 2017 -0400
summary: Added tag 4.3.1 for changeset 3fee7f7d2da0

https://www.mercurial-scm.org/repo/hg/rev/a3ce07e2dde5
changeset:   33662:a3ce07e2dde5
branch:  stable
tag: tip
user:Augie Fackler 
date:Thu Aug 10 18:46:55 2017 -0400
summary: Added signature for changeset 3fee7f7d2da0

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D330: Backed out changeset c34532365b38

[indygreg (Gregory Szorc)]

indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  This looks good. I'll queue this for stable when I get home.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D330

To: glandium, #hg-reviewers, quark, indygreg
Cc: indygreg, quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D328: setup: Fix installing in a mingw environment

[quark (Jun Wu)]

quark requested changes to this revision.
quark added a comment.
This revision now requires changes to proceed.


  https://phab.mercurial-scm.org/D329 is a replacement. Maybe abandon this diff 
then.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D328

To: glandium, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D330: Backed out changeset c34532365b38

[quark (Jun Wu)]

quark accepted this revision.
quark added a comment.


  LGTM.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D330

To: glandium, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D328: setup: Fix installing in a mingw environment

[quark (Jun Wu)]

quark added a comment.


  @glandium `phabsend` verifies the tags with old versions on Phabricator (so 
it requires obsstore to work properly). After 
https://phab.mercurial-scm.org/D122, it will be possible to `phabsend --amend` 
and `phabsend` will use the commit message to update corresponding diffs.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D328

To: glandium, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH STABLE] osx: delay version computation on macOS builds

[Kevin Bullock]

# HG changeset patch
# User Kevin Bullock 
# Date 1502414702 14400
#  Thu Aug 10 21:25:02 2017 -0400
# Branch stable
# Node ID 0d3a872cbb37b7845eacdcaa578c0048e3f4873f
# Parent  a3ce07e2dde5737644dd719806b1e52fb1cc1004
osx: delay version computation on macOS builds

Before this patch, HGVER would be evaluated at the beginning of the make
execution, and would be unset because build/mercurial/ doesn't exist yet
at that point. Now we compute the version after the `make install` run
has completed.

This is backported to stable from 8626b44516c1, but that revision had an
error in the shell invocation syntax.

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -186,7 +186,7 @@ osx:
  PREFIX=/usr/local \
  clean install
mkdir -p $${OUTPUTDIR:-dist}
-   HGVER=$(shell python contrib/genosxversion.py $(OSXVERSIONFLAGS) 
build/mercurial/Library/Python/2.7/site-packages/mercurial/__version__.py ) && \
+   HGVER=$$(python contrib/genosxversion.py $(OSXVERSIONFLAGS) 
build/mercurial/Library/Python/2.7/site-packages/mercurial/__version__.py) && \
OSXVER=$$(sw_vers -productVersion | cut -d. -f1,2) && \
pkgbuild --filter \\.DS_Store --root build/mercurial/ \
  --identifier org.mercurial-scm.mercurial \
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D330: Backed out changeset c34532365b38

[glandium (Mike Hommey)]

glandium created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Old versions of python 2.7 don't like that the second argument to
  struct.unpack_from is a bytearray, so the change removing the util.buffer
  around that argument in branchmap broke running on older versions of python
  2.7.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D330

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -406,7 +406,8 @@
 
 # fast path: extract data from cache, use it if node is matching
 reponode = changelog.node(rev)[:_rbcnodelen]
-cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, 
rbcrevidx)
+cachenode, branchidx = unpack_from(
+_rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
 close = bool(branchidx & _rbccloseflag)
 if close:
 branchidx &= _rbcbranchidxmask



To: glandium, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D328: setup: Fix installing in a mingw environment

[glandium (Mike Hommey)]

glandium added a comment.


  Sorry, I rebased to stable, and that created a new differential: 
https://phab.mercurial-scm.org/D329, even though I updated the local tag.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D328

To: glandium, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D329: setup: Fix installing in a mingw environment

[glandium (Mike Hommey)]

glandium created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The addition, in 
https://phab.mercurial-scm.org/rHG9a4adc76c88a1a217983f051766b3009c0bca3aa, of 
a hack for the MSVC compiler class was
  overwriting the original class for the Mingw32CCompiler class, leading to an
  error when the HackedMingw32CCompiler is instantiated.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D329

AFFECTED FILES
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -784,11 +784,11 @@
 from distutils import cygwinccompiler
 
 # the -mno-cygwin option has been deprecated for years
-compiler = cygwinccompiler.Mingw32CCompiler
+mingw32compilerclass = cygwinccompiler.Mingw32CCompiler
 
 class HackedMingw32CCompiler(cygwinccompiler.Mingw32CCompiler):
 def __init__(self, *args, **kwargs):
-compiler.__init__(self, *args, **kwargs)
+mingw32compilerclass.__init__(self, *args, **kwargs)
 for i in 'compiler compiler_so linker_exe linker_so'.split():
 try:
 getattr(self, i).remove('-mno-cygwin')
@@ -809,11 +809,11 @@
 # effect.
 from distutils import msvccompiler
 
-compiler = msvccompiler.MSVCCompiler
+msvccompilerclass = msvccompiler.MSVCCompiler
 
 class HackedMSVCCompiler(msvccompiler.MSVCCompiler):
 def initialize(self):
-compiler.initialize(self)
+msvccompilerclass.initialize(self)
 # "warning LNK4197: export 'func' specified multiple times"
 self.ldflags_shared.append('/ignore:4197')
 self.ldflags_shared_debug.append('/ignore:4197')



To: glandium, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D30: merge: Removed sorting in casefolding detection, for a slight performance win

[quark (Jun Wu)]

quark accepted this revision.
quark added a comment.


  Looks good to me. I'd prefer perf to error message stability. If the latter 
is a concern, we can re-run the loop if there is at least one case-folding 
collision detected.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D30

To: alex_gaynor, durin42, dsp, #hg-reviewers, quark
Cc: quark, krbullock, dsp, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D328: setup: Fix installing in a mingw environment

[quark (Jun Wu)]

quark accepted this revision.
quark added a comment.


  Looks good to me. Bonus point if you can remove `_` so the variable names 
match the rest of the code.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D328

To: glandium, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D328: setup: Fix installing in a mingw environment

[glandium (Mike Hommey)]

glandium created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The addition, in 
https://phab.mercurial-scm.org/rHG9a4adc76c88a1a217983f051766b3009c0bca3aa, of 
a hack for the MSVC compiler class was
  overwriting the original class for the Mingw32CCompiler class, leading to an
  error when the HackedMingw32CCompiler is instantiated.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D328

AFFECTED FILES
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -784,11 +784,11 @@
 from distutils import cygwinccompiler
 
 # the -mno-cygwin option has been deprecated for years
-compiler = cygwinccompiler.Mingw32CCompiler
+mingw32_compiler_class = cygwinccompiler.Mingw32CCompiler
 
 class HackedMingw32CCompiler(cygwinccompiler.Mingw32CCompiler):
 def __init__(self, *args, **kwargs):
-compiler.__init__(self, *args, **kwargs)
+mingw32_compiler_class.__init__(self, *args, **kwargs)
 for i in 'compiler compiler_so linker_exe linker_so'.split():
 try:
 getattr(self, i).remove('-mno-cygwin')
@@ -809,11 +809,11 @@
 # effect.
 from distutils import msvccompiler
 
-compiler = msvccompiler.MSVCCompiler
+msvc_compiler_class = msvccompiler.MSVCCompiler
 
 class HackedMSVCCompiler(msvccompiler.MSVCCompiler):
 def initialize(self):
-compiler.initialize(self)
+msvc_compiler_class.initialize(self)
 # "warning LNK4197: export 'func' specified multiple times"
 self.ldflags_shared.append('/ignore:4197')
 self.ldflags_shared_debug.append('/ignore:4197')



To: glandium, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@33648: new changeset (1 on stable)

[Mercurial Commits]

New changeset (1 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/47ea28293d30
changeset:   33648:47ea28293d30
branch:  stable
tag: tip
user:Yuya Nishihara 
date:Tue Aug 01 21:03:25 2017 +0900
summary: tests: show cache of audited paths is never invalidated

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Mercurial 4.3 and 4.2.3 released

[Augie Fackler]

> On Aug 10, 2017, at 14:25, Augie Fackler  wrote:
> 
> 
>> On Aug 10, 2017, at 14:11, Augie Fackler > > wrote:
>> 
>> 
>>> On Aug 10, 2017, at 14:09, Augie Fackler >> > wrote:
>>> 
>>> Moments ago, I released Mercurial 4.3 and 4.2.3. Please patch *immedately*:
>> 
>> Update: the release script misfired and 4.2.3 is wrong - I'll fix it shortly.
> 
> 4.2.3 is now correctly available from mercurial-scm.org 
>  and has a tag in 
> mercurial-scm.org/repo/hg-committed 
> .
> 
> I can't (sadly) upload it to pypi, please let me know if that's a major 
> concern for you.

The betrayal of the release scripts continues: 4.3 didn't include the security 
patches correctly.

So there's now a 4.3.1 with the patches.

(I'll do a mini-postmortem on this later, not to worry.)

> 
>> 
>>> 
>>> CVE-2017-1000115:
>>> 
>>> Mercurial's symlink auditing was incomplete prior to 4.3, and could be 
>>> abused to write to files outside the repository.
>>> 
>>> CVE-2017-1000116:
>>> 
>>> Mercurial was not sanitizing hostnames passed to ssh, allowing shell 
>>> injection attacks by specifying a hostname starting with -oProxyCommand. 
>>> This is also present in Git (CVE-2017-1000117) and Subversion 
>>> (CVE-2017-9800), so please patch those tools as well if you have them 
>>> installed. All three tools are doing their security release today.
>>> 
>>> Please update your packaged builds as soon as practical.
>>> 
>>> Note that since we dropped Python 2.6 and these issues are pretty bad, we 
>>> did the back port to 4.2.3. We may not do further 4.2 releases, so please 
>>> plan around Python 2.7 in the near future if you haven't already.
>>> 
>>> Thanks!
>>> Augie
>> 
> 



signature.asc
Description: Message signed with OpenPGP
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@33647: 20 new changesets (20 on stable)

[Mercurial Commits]

20 new changesets (20 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/aa7a3f6e3729
changeset:   33628:aa7a3f6e3729
branch:  stable
user:Mathias De Maré 
date:Mon Aug 07 13:38:25 2017 +0200
summary: centos5: use vault urls

https://www.mercurial-scm.org/repo/hg/rev/5544af862286
changeset:   33629:5544af862286
branch:  stable
tag: 4.3
user:Mathias De Maré 
date:Mon Aug 07 13:40:36 2017 +0200
summary: chg: define _GNU_SOURCE to allow CentOS 5 compilation

https://www.mercurial-scm.org/repo/hg/rev/d7ca01a6492a
changeset:   33630:d7ca01a6492a
branch:  stable
user:Augie Fackler 
date:Thu Aug 10 13:29:08 2017 -0400
summary: Added tag 4.3 for changeset 5544af862286

https://www.mercurial-scm.org/repo/hg/rev/460121715f4a
changeset:   33631:460121715f4a
branch:  stable
user:Augie Fackler 
date:Thu Aug 10 13:29:09 2017 -0400
summary: Added signature for changeset 5544af862286

https://www.mercurial-scm.org/repo/hg/rev/0134d839444b
changeset:   33632:0134d839444b
branch:  stable
parent:  33170:26c49ed51a69
user:Yuya Nishihara 
date:Tue Aug 01 21:03:25 2017 +0900
summary: tests: show cache of audited paths is never invalidated

https://www.mercurial-scm.org/repo/hg/rev/20bac46f7744
changeset:   33633:20bac46f7744
branch:  stable
user:Yuya Nishihara 
date:Wed Jul 26 22:10:15 2017 +0900
summary: pathauditor: disable cache of audited paths by default (issue5628)

https://www.mercurial-scm.org/repo/hg/rev/53224b1ffbc2
changeset:   33634:53224b1ffbc2
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:32:25 2017 -0700
summary: util: add utility method to check for bad ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/e10745311406
changeset:   33635:e10745311406
branch:  stable
user:Augie Fackler 
date:Fri Aug 04 14:00:03 2017 -0400
summary: ssh: ban any username@host or host that starts with - (SEC)

https://www.mercurial-scm.org/repo/hg/rev/f93975a5ebe8
changeset:   33636:f93975a5ebe8
branch:  stable
user:Sean Farley 
date:Tue Aug 01 14:40:19 2017 -0700
summary: sshpeer: check for safe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/f9134e96ed0f
changeset:   33637:f9134e96ed0f
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:36:36 2017 -0700
summary: clone: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/92b583e3e522
changeset:   33638:92b583e3e522
branch:  stable
user:Sean Farley 
date:Fri Jul 28 16:47:32 2017 -0700
summary: pull: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/08cfc4baf3ba
changeset:   33639:08cfc4baf3ba
branch:  stable
user:Sean Farley 
date:Mon Jul 31 14:40:28 2017 -0700
summary: push: add tests for unsafe ssh url (SEC)

https://www.mercurial-scm.org/repo/hg/rev/55681baf4cf9
changeset:   33640:55681baf4cf9
branch:  stable
user:Sean Farley 
date:Mon Jul 31 16:04:44 2017 -0700
summary: subrepo: add tests for hg rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/173ecccb9ee7
changeset:   33641:173ecccb9ee7
branch:  stable
user:Sean Farley 
date:Mon Jul 31 16:44:17 2017 -0700
summary: subrepo: add tests for svn rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/ca398a50ca00
changeset:   33642:ca398a50ca00
branch:  stable
user:Sean Farley 
date:Mon Jul 31 14:55:11 2017 -0700
summary: subrepo: add tests for git rogue ssh urls (SEC)

https://www.mercurial-scm.org/repo/hg/rev/00a75672a9cb
changeset:   33643:00a75672a9cb
branch:  stable
user:Jun Wu 
date:Fri Aug 04 23:54:12 2017 -0700
summary: ssh: quote parameters using shellquote (SEC)

https://www.mercurial-scm.org/repo/hg/rev/943c91326b23
changeset:   33644:943c91326b23
branch:  stable
tag: 4.2.3
user:Yuya Nishihara 
date:Mon Aug 07 22:22:28 2017 +0900
summary: ssh: unban the use of pipe character in user@host:port string

https://www.mercurial-scm.org/repo/hg/rev/fef451e3d9ca
changeset:   33645:fef451e3d9ca
branch:  stable
user:Augie Fackler 
date:Thu Aug 10 14:14:48 2017 -0400
summary: Added tag 4.2.3 for changeset 943c91326b23

https://www.mercurial-scm.org/repo/hg/rev/506d7e48fbe6
changeset:   33646:506d7e48fbe6
branch:  stable
user:Augie Fackler 
date:Thu Aug 10 14:14:49 2017 -0400
summary: Added signature for changeset 943c91326b23

D125: phabricator: add a small language to query Differential Revisions

[quark (Jun Wu)]

quark updated this revision to Diff 743.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D125?vs=570=743

REVISION DETAIL
  https://phab.mercurial-scm.org/D125

AFFECTED FILES
  contrib/phabricator.py

CHANGE DETAILS

diff --git a/contrib/phabricator.py b/contrib/phabricator.py
--- a/contrib/phabricator.py
+++ b/contrib/phabricator.py
@@ -32,7 +32,9 @@
 
 from __future__ import absolute_import
 
+import itertools
 import json
+import operator
 import re
 
 from mercurial.node import bin, nullid
@@ -44,9 +46,11 @@
 error,
 mdiff,
 obsutil,
+parser,
 patch,
 registrar,
 scmutil,
+smartset,
 tags,
 url as urlmod,
 util,
@@ -485,11 +489,77 @@
 
 return True
 
-def querydrev(repo, params, stack=False):
+# Small language to specify differential revisions. Support symbols: (), :X,
+# +, and -.
+
+_elements = {
+# token-type: binding-strength, primary, prefix, infix, suffix
+'(':  (12, None, ('group', 1, ')'), None, None),
+':':  (8, None, ('ancestors', 8), None, None),
+'&':  (5,  None, None, ('and_', 5), None),
+'+':  (4,  None, None, ('add', 4), None),
+'-':  (4,  None, None, ('sub', 4), None),
+')':  (0,  None, None, None, None),
+'symbol': (0, 'symbol', None, None, None),
+'end':(0, None, None, None, None),
+}
+
+def _tokenize(text):
+text = text.replace(' ', '') # remove space
+view = memoryview(text) # zero-copy slice
+special = '():+-&'
+pos = 0
+length = len(text)
+while pos < length:
+symbol = ''.join(itertools.takewhile(lambda ch: ch not in special,
+ view[pos:]))
+if symbol:
+yield ('symbol', symbol, pos)
+pos += len(symbol)
+else: # special char
+yield (text[pos], None, pos)
+pos += 1
+yield ('end', None, pos)
+
+def _parse(text):
+tree, pos = parser.parser(_elements).parse(_tokenize(text))
+if pos != len(text):
+raise error.ParseError('invalid token', pos)
+return tree
+
+def _parsedrev(symbol):
+"""str -> int or None, ex. 'D45' -> 45; '12' -> 12; 'x' -> None"""
+if symbol.startswith('D') and symbol[1:].isdigit():
+return int(symbol[1:])
+if symbol.isdigit():
+return int(symbol)
+
+def _prefetchdrevs(tree):
+"""return ({single-drev-id}, {ancestor-drev-id}) to prefetch"""
+drevs = set()
+ancestordrevs = set()
+op = tree[0]
+if op == 'symbol':
+r = _parsedrev(tree[1])
+if r:
+drevs.add(r)
+elif op == 'ancestors':
+r, a = _prefetchdrevs(tree[1])
+drevs.update(r)
+ancestordrevs.update(r)
+ancestordrevs.update(a)
+else:
+for t in tree[1:]:
+r, a = _prefetchdrevs(t)
+drevs.update(r)
+ancestordrevs.update(a)
+return drevs, ancestordrevs
+
+def querydrev(repo, spec):
 """return a list of "Differential Revision" dicts
 
-params is the input of "differential.query" API, and is expected to match
-just a single Differential Revision.
+spec is a string using a simple query language, see docstring in phabread
+for details.
 
 A "Differential Revision dict" looks like:
 
@@ -526,51 +596,77 @@
 "repositoryPHID": "PHID-REPO-hub2hx62ieuqeheznasv",
 "sourcePath": null
 }
-
-If stack is True, return a list of "Differential Revision dict"s in an
-order that the latter ones depend on the former ones. Otherwise, return a
-list of a unique "Differential Revision dict".
 """
-prefetched = {} # {id or phid: drev}
 def fetch(params):
 """params -> single drev or None"""
 key = (params.get(r'ids') or params.get(r'phids') or [None])[0]
 if key in prefetched:
 return prefetched[key]
-# Otherwise, send the request. If we're fetching a stack, be smarter
-# and fetch more ids in one batch, even if it could be unnecessary.
-batchparams = params
-if stack and len(params.get(r'ids', [])) == 1:
-i = int(params[r'ids'][0])
-# developer config: phabricator.batchsize
-batchsize = repo.ui.configint('phabricator', 'batchsize', 12)
-batchparams = {'ids': range(max(1, i - batchsize), i + 1)}
-drevs = callconduit(repo, 'differential.query', batchparams)
+drevs = callconduit(repo, 'differential.query', params)
 # Fill prefetched with the result
 for drev in drevs:
 prefetched[drev[r'phid']] = drev
 prefetched[int(drev[r'id'])] = drev
 if key not in prefetched:
 raise error.Abort(_('cannot get Differential Revision %r') % 
params)
 return prefetched[key]
 
-visited = set()
-result = []
-queue = [params]
-while queue:
-params = queue.pop()
-drev = fetch(params)
-

Re: Mercurial 4.3 and 4.2.3 released

[Augie Fackler]

> On Aug 10, 2017, at 14:09, Augie Fackler  wrote:
> 
> Moments ago, I released Mercurial 4.3 and 4.2.3. Please patch *immedately*:

Update: the release script misfired and 4.2.3 is wrong - I'll fix it shortly.

> 
> CVE-2017-1000115:
> 
> Mercurial's symlink auditing was incomplete prior to 4.3, and could be abused 
> to write to files outside the repository.
> 
> CVE-2017-1000116:
> 
> Mercurial was not sanitizing hostnames passed to ssh, allowing shell 
> injection attacks by specifying a hostname starting with -oProxyCommand. This 
> is also present in Git (CVE-2017-1000117) and Subversion (CVE-2017-9800), so 
> please patch those tools as well if you have them installed. All three tools 
> are doing their security release today.
> 
> Please update your packaged builds as soon as practical.
> 
> Note that since we dropped Python 2.6 and these issues are pretty bad, we did 
> the back port to 4.2.3. We may not do further 4.2 releases, so please plan 
> around Python 2.7 in the near future if you haven't already.
> 
> Thanks!
> Augie



signature.asc
Description: Message signed with OpenPGP
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Mercurial 4.3 and 4.2.3 released

[Augie Fackler]

Moments ago, I released Mercurial 4.3 and 4.2.3. Please patch *immedately*:

CVE-2017-1000115:

Mercurial's symlink auditing was incomplete prior to 4.3, and could be abused 
to write to files outside the repository.

CVE-2017-1000116:

Mercurial was not sanitizing hostnames passed to ssh, allowing shell injection 
attacks by specifying a hostname starting with -oProxyCommand. This is also 
present in Git (CVE-2017-1000117) and Subversion (CVE-2017-9800), so please 
patch those tools as well if you have them installed. All three tools are doing 
their security release today.

Please update your packaged builds as soon as practical.

Note that since we dropped Python 2.6 and these issues are pretty bad, we did 
the back port to 4.2.3. We may not do further 4.2 releases, so please plan 
around Python 2.7 in the near future if you haven't already.

Thanks!
Augie


signature.asc
Description: Message signed with OpenPGP
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D322: fsmonitor: correct an error message

[lothiraldan (Boris Feld)]

lothiraldan accepted this revision.
lothiraldan added a comment.


  LGTM, we don't have a test for it?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D322

To: quark, #hg-reviewers, lothiraldan
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5652] New: hg commit -i with empty file fails

[mercurial-bugs]

https://bz.mercurial-scm.org/show_bug.cgi?id=5652

Bug ID: 5652
   Summary: hg commit -i with empty file fails
   Product: Mercurial
   Version: 4.2.2
  Hardware: PC
OS: Windows
Status: UNCONFIRMED
  Severity: feature
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: c...@perspexis.com
CC: mercurial-devel@mercurial-scm.org

I had a file with contents. I removed all of the contents but kept the file.

When I try to commit with `hg commit -i` it fails with 

abort: bad hunk #1

`hg commit -m` worked fine.

I'm using evolve and in order to reproduce this I had to ammend a changeset.

Steps:

echo a > a
hg init
hg add a
hg commit -m "Add a"
hg commit --amend -m "Better message"
vi a
# Press dd to delete the line then 
# Press :x to save and exit
hg commit -i
# Leave the file checked and press c
# abort: bad hunk #1

It didn't happen if I replace `vi a` with `echo "" > a` or `echo "\n" > a`. The
file can't even have a newline in it.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5651] New: interactive revert of no-eol hunk fails

[mercurial-bugs]

https://bz.mercurial-scm.org/show_bug.cgi?id=5651

Bug ID: 5651
   Summary: interactive revert of no-eol hunk fails
   Product: Mercurial
   Version: 4.3-rc
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Keywords: regression
  Severity: bug
  Priority: urgent
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: y...@tcha.org
CC: arcppzju+hg...@gmail.com,
mercurial-devel@mercurial-scm.org

STR (works on 4.2):

  $ hg init repo
  $ cd repo
  $ echo 0 > a
  $ hg ci -qAm 0
  $ printf 1 >> a
  $ hg ci -qAm 1
  $ cat a
  0
  1 (no-eol)

  $ hg revert -ir.^ --config ui.interactive=1 < y
  > y
  > EOF
  reverting a
  diff --git a/a b/a
  1 hunks, 1 lines changed
  examine changes to 'a'? [Ynesfdaq?] y

  @@ -1,1 +1,2 @@
   0
  +1
  \ No newline at end of file
  revert this change to 'a'? [Ynesfdaq?] y

  $ cat a
  0

  $ cd ..


On 4.3-rc:

+  Traceback (most recent call last):
+File "hg", line 45, in 
+  mercurial.dispatch.run()
+File "mercurial/dispatch.py", line 86, in run
+  status = (dispatch(req) or 0) & 255
+File "mercurial/dispatch.py", line 167, in dispatch
+  ret = _runcatch(req)
+File "mercurial/dispatch.py", line 307, in _runcatch
+  return _callcatch(ui, _runcatchfunc)
+File "mercurial/dispatch.py", line 315, in _callcatch
+  return scmutil.callcatch(ui, func)
+File "mercurial/scmutil.py", line 145, in callcatch
+  return func()
+File "mercurial/dispatch.py", line 297, in _runcatchfunc
+  return _dispatch(req)
+File "mercurial/dispatch.py", line 927, in _dispatch
+  cmdpats, cmdoptions)
+File "mercurial/dispatch.py", line 663, in runcommand
+  ret = _runcommand(ui, options, cmd, d)
+File "mercurial/dispatch.py", line 935, in _runcommand
+  return cmdfunc()
+File "mercurial/dispatch.py", line 924, in 
+  d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
+File "mercurial/util.py", line 1056, in check
+  return func(*args, **kwargs)
+File "mercurial/commands.py", line 4520, in revert
+  return cmdutil.revert(ui, repo, ctx, (parent, p2), *pats, **opts)
+File "mercurial/cmdutil.py", line 3332, in revert
+  _performrevert(repo, parents, ctx, actions, interactive, tobackup)
+File "mercurial/cmdutil.py", line 3434, in _performrevert
+  chunks = patch.reversehunks(chunks)
+File "mercurial/patch.py", line 1526, in reversehunks
+  c = c.reversehunk()
+File "mercurial/patch.py", line 970, in reversehunk
+  hunk = ['%s%s' % (m[l[0]], l[1:]) for l in self.hunk]
+  KeyError: '\\'
+  [1]

The first bad revision is:

changeset:   43372:66117dae87f9
user:Jun Wu 
date:Tue Jun 20 23:22:38 2017 -0700
summary: patch: rewrite reversehunks (issue5337)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D30: merge: Removed sorting in casefolding detection, for a slight performance win

[alex_gaynor (Alex Gaynor)]

alex_gaynor updated this revision to Diff 737.
alex_gaynor added a comment.
Herald added a reviewer: hg-reviewers.


  Undo a change which was incorrect

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D30?vs=46=737

BRANCH
  performance-changes (bookmark) on default (branch)

REVISION DETAIL
  https://phab.mercurial-scm.org/D30

AFFECTED FILES
  mercurial/merge.py

CHANGE DETAILS

diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -753,7 +753,7 @@
 
 # check case-folding collision in provisional merged manifest
 foldmap = {}
-for f in sorted(pmmf):
+for f in pmmf:
 fold = util.normcase(f)
 if fold in foldmap:
 raise error.Abort(_("case-folding collision between %s and %s")



To: alex_gaynor, durin42, dsp, #hg-reviewers
Cc: quark, krbullock, dsp, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D321: merge: removed sorting in caseflow detection, for a slight performance win

[alex_gaynor (Alex Gaynor)]

alex_gaynor abandoned this revision.
alex_gaynor added a comment.


  This was created by accident, https://phab.mercurial-scm.org/D30 should have 
been updated.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D321

To: alex_gaynor, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D321: merge: removed sorting in caseflow detection, for a slight performance win

[alex_gaynor (Alex Gaynor)]

alex_gaynor created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

BRANCH
  performance-changes (bookmark) on default (branch)

REVISION DETAIL
  https://phab.mercurial-scm.org/D321

AFFECTED FILES
  mercurial/merge.py

CHANGE DETAILS

diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -753,7 +753,7 @@
 
 # check case-folding collision in provisional merged manifest
 foldmap = {}
-for f in sorted(pmmf):
+for f in pmmf:
 fold = util.normcase(f)
 if fold in foldmap:
 raise error.Abort(_("case-folding collision between %s and %s")



To: alex_gaynor, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D30: merge: Removed sorting in casefolding detection, for a slight performance win

[alex_gaynor (Alex Gaynor)]

alex_gaynor added inline comments.

INLINE COMMENTS

> quark wrote in merge.py:765
> I think this would be incorrect, since `foldprefix` is mutated and used in 
> the loop.

I think you're right, removing the first `sorted` should be safe though.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D30

To: alex_gaynor, durin42, dsp
Cc: quark, krbullock, dsp, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D98: revset: support reading aliases from a .hgrevsets file

[yuja (Yuya Nishihara)]

yuja added a comment.


  > Security-wise, the "shelling out revset" seems hard to solve cleanly.
  
  Yes, and IMHO the only reasonable solution is to not add such revset 
functions.
  revsets and filesets can be executed through the web UI.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D98

To: indygreg, #hg-reviewers
Cc: durin42, yuja, mharbison72, quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D318: wireproto: remove support for local results in @batchable (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  @peer.batchable decorated generator functions have two forms:
  
yield value, None
  
  and
  
yield args, future
yield value
  
  These forms have been present since the decorator was introduced.
  
  There are currently no in-repo consumers of the first form. So this
  commit removes support for it.
  
  Note that remoteiterbatcher.submit() asserts the 2nd form. And
  https://phab.mercurial-scm.org/rHGb6e71f8af5b8710139b64706a375b3f16eb1241e 
removed the last user of remotebatcher, forcing everyone
  to remoteiterbatcher. So anything relying on this in the wild would
  have been broken since 
https://phab.mercurial-scm.org/rHGb6e71f8af5b8710139b64706a375b3f16eb1241e.
  
  .. api::
  
@peer.batchable can no longer emit local values

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D318

AFFECTED FILES
  mercurial/peer.py
  mercurial/wireproto.py
  tests/test-batching.py
  tests/test-batching.py.out

CHANGE DETAILS

diff --git a/tests/test-batching.py.out b/tests/test-batching.py.out
--- a/tests/test-batching.py.out
+++ b/tests/test-batching.py.out
@@ -4,7 +4,6 @@
 Un and Deux
 Eins und Zwei
 One and Two
-Nope
 Eins und Zwei
 Hello, John Smith
 Ready.
@@ -25,7 +24,6 @@
 REQ: batch?cmds=bar:b=Vop,a=Evf
   -> Vop!voe!Evf
 One and Two
-Nope
 Eins und Zwei
 Hello, John Smith
 Ready.
diff --git a/tests/test-batching.py b/tests/test-batching.py
--- a/tests/test-batching.py
+++ b/tests/test-batching.py
@@ -45,7 +45,6 @@
 batch = it.batch()
 # The calls return futures to eventually hold results.
 foo = batch.foo(one="One", two="Two")
-foo2 = batch.foo(None)
 bar = batch.bar("Eins", "Zwei")
 # We can call non-batchable proxy methods, but the break the current batch
 # request and cause additional roundtrips.
@@ -58,7 +57,6 @@
 batch.submit()
 # After the call to submit, the futures actually contain values.
 print(foo.value)
-print(foo2.value)
 print(bar.value)
 print(greet.value)
 print(hello.value)
@@ -153,8 +151,6 @@
 
 @peer.batchable
 def foo(self, one, two=None):
-if not one:
-yield "Nope", None
 encargs = [('one', mangle(one),), ('two', mangle(two),)]
 encresref = peer.future()
 yield encargs, encresref
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -95,11 +95,9 @@
 if batchablefn is not None:
 batchable = batchablefn(mtd.im_self, *args, **opts)
 encargsorres, encresref = next(batchable)
-if encresref:
-req.append((name, encargsorres,))
-rsp.append((batchable, encresref, resref,))
-else:
-resref.set(encargsorres)
+assert encresref
+req.append((name, encargsorres,))
+rsp.append((batchable, encresref, resref,))
 else:
 if req:
 self._submitreq(req, rsp)
diff --git a/mercurial/peer.py b/mercurial/peer.py
--- a/mercurial/peer.py
+++ b/mercurial/peer.py
@@ -78,9 +78,6 @@
 
 @batchable
 def sample(self, one, two=None):
-# Handle locally computable results first:
-if not one:
-yield "a local result", None
 # Build list of encoded arguments suitable for your wire protocol:
 encargs = [('one', encode(one),), ('two', encode(two),)]
 # Create future for injection of encoded result:



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D320: peer: remove non iterating batcher (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The last use of this API was removed in 
https://phab.mercurial-scm.org/rHGb6e71f8af5b8710139b64706a375b3f16eb1241e in 
2016. While
  not formally deprecated, as of the last commit the code is no longer
  explicitly tested. I think the new API has existed long enough for
  people to transition to it.
  
  I also have plans to more formalize the peer API and removing batch()
  makes that work easier.
  
  I'm not convinced the current client-side API around batching is
  great. But it's the best we have at the moment.
  
  .. api:: remove peer.batch()
  
Replace with peer.iterbatch().

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D320

AFFECTED FILES
  mercurial/peer.py
  mercurial/wireproto.py
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -55,7 +55,7 @@
 clt = clientpeer(srv)
 
 print(clt.greet("Foobar"))
-b = clt.batch()
-fs = [b.greet(s) for s in ["Fo, =;:

D319: wireproto: overhaul iterating batcher code (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The remote batching code is difficult to read. Let's improve it.
  
  As part of the refactor, the future returned by method calls on
  batchiter() instances is now populated. However, you still need to
  consume the results() generator for the future to be set.  But at
  least now we can stuff the future somewhere and not have to worry
  about aligning method call order with result order since you can
  use a future to hold the result.
  
  Also as part of the change, we now verify that @batchable generators
  yield exactly 2 values. In other words, we enforce their API.
  
  The non-iter batcher has been unused since 
https://phab.mercurial-scm.org/rHGb6e71f8af5b8710139b64706a375b3f16eb1241e. And 
to my
  surprise we had no explicit unit test coverage of it! test-batching.py
  has been overhauled to use the iterating batcher.
  
  Since the iterating batcher doesn't allow non-batchable method
  calls nor local calls, tests have been updated to reflect reality.
  The iterating batcher has been used for multiple releases apparently
  without major issue. So this shouldn't cause alarm.
  
  .. api::
  
@peer.batchable functions must now yield exactly 2 values

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D319

AFFECTED FILES
  mercurial/peer.py
  mercurial/wireproto.py
  tests/test-batching.py
  tests/test-batching.py.out

CHANGE DETAILS

diff --git a/tests/test-batching.py.out b/tests/test-batching.py.out
--- a/tests/test-batching.py.out
+++ b/tests/test-batching.py.out
@@ -5,9 +5,8 @@
 Eins und Zwei
 One and Two
 Eins und Zwei
-Hello, John Smith
-Ready.
 Uno und Due
+proper end of results generator
 
 == Remote
 Ready.
@@ -17,14 +16,11 @@
 REQ: bar?b=Fjot=[xfj
   -> Fjot!voe![xfj
 Eins und Zwei
-REQ: batch?cmds=foo:one=Pof,two=Uxp;bar:b=Fjot,a=[xfj
-  -> Pof!boe!Uxp;Fjot!voe![xfj
-REQ: greet?name=Kpio!Tnjui
-  -> Ifmmp-!Kpio!Tnjui
-REQ: batch?cmds=bar:b=Vop,a=Evf
-  -> Vop!voe!Evf
+REQ: batch?cmds=foo:one=Pof,two=Uxp;bar:b=Fjot,a=[xfj;bar:b=Vop,a=Evf
+  -> Pof!boe!Uxp;Fjot!voe![xfj;Vop!voe!Evf
 One and Two
 Eins und Zwei
-Hello, John Smith
-Ready.
 Uno und Due
+proper end of results generator
+Attempted to batch a non-batchable call to 'greet'
+Attempted to batch a non-batchable call to 'hello'
diff --git a/tests/test-batching.py b/tests/test-batching.py
--- a/tests/test-batching.py
+++ b/tests/test-batching.py
@@ -8,7 +8,9 @@
 from __future__ import absolute_import, print_function
 
 from mercurial import (
+error,
 peer,
+util,
 wireproto,
 )
 
@@ -27,9 +29,9 @@
 return "%s und %s" % (b, a,)
 def greet(self, name=None):
 return "Hello, %s" % name
-def batch(self):
+def batchiter(self):
 '''Support for local batching.'''
-return peer.localbatch(self)
+return peer.localiterbatcher(self)
 
 # usage of "thing" interface
 def use(it):
@@ -41,27 +43,54 @@
 print(it.foo("Un", two="Deux"))
 print(it.bar("Eins", "Zwei"))
 
-# Batched call to a couple of (possibly proxied) methods.
-batch = it.batch()
+# Batched call to a couple of proxied methods.
+batch = it.batchiter()
 # The calls return futures to eventually hold results.
 foo = batch.foo(one="One", two="Two")
 bar = batch.bar("Eins", "Zwei")
-# We can call non-batchable proxy methods, but the break the current batch
-# request and cause additional roundtrips.
-greet = batch.greet(name="John Smith")
-# We can also add local methods into the mix, but they break the batch too.
-hello = batch.hello()
 bar2 = batch.bar(b="Uno", a="Due")
-# Only now are all the calls executed in sequence, with as few roundtrips
-# as possible.
+
+# Future shouldn't be set until we submit().
+assert isinstance(foo, peer.future)
+assert not util.safehasattr(foo, 'value')
+assert not util.safehasattr(bar, 'value')
 batch.submit()
-# After the call to submit, the futures actually contain values.
+# Call results() to obtain results as a generator.
+results = batch.results()
+
+# Future results shouldn't be set until we consume a value.
+assert not util.safehasattr(foo, 'value')
+foovalue = next(results)
+assert util.safehasattr(foo, 'value')
+assert foovalue == foo.value
 print(foo.value)
+next(results)
 print(bar.value)
-print(greet.value)
-print(hello.value)
+next(results)
 print(bar2.value)
 
+# We should be at the end of the results generator.
+try:
+next(results)
+except StopIteration:
+print('proper end of results generator')
+else:
+print('extra emitted element!')
+
+# Attempting to call a non-batchable method inside a batch fails.
+batch = it.batchiter()
+try:
+batch.greet(name='John Smith')
+except error.ProgrammingError as e:
+ 

D316: largefiles: remove remotestore.batch()

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This method was added in 
https://phab.mercurial-scm.org/rHG9e1616307c4cca80d177203afc39e5eecda60c84. 
AFAICT it didn't do anything at
  inception. If it did, there was no test coverage for it because
  changing it to raise doesn't fail any tests at that revision.
  
  https://phab.mercurial-scm.org/rHGb6e71f8af5b8710139b64706a375b3f16eb1241e 
later refactored all remote.batch() calls to
  remote.iterbatch(). So if this was somehow used, it isn't called
  any more because there are no calls to .batch() remaining in the
  repo.
  
  I suspect the original patch author got confused by the distinction
  between the peer/remote interface and the largefiles store. The lf
  store is a gateway to a peer instance. It exposes additional
  lf-specific methods to execute against a peer. However, it is not
  a peer and doesn't need to implement batch() because peer itself
  does that.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D316

AFFECTED FILES
  hgext/largefiles/remotestore.py

CHANGE DETAILS

diff --git a/hgext/largefiles/remotestore.py b/hgext/largefiles/remotestore.py
--- a/hgext/largefiles/remotestore.py
+++ b/hgext/largefiles/remotestore.py
@@ -109,10 +109,6 @@
'from statlfile (%r)' % stat)
 return failed
 
-def batch(self):
-'''Support for remote batching.'''
-return wireproto.remotebatch(self)
-
 def _put(self, hash, fd):
 '''Put file with the given hash in the remote store.'''
 raise NotImplementedError('abstract method')



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel