D2588: commit: adds multiline commit message support(issue5616)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 updated this revision to Diff 6537.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2588?vs=6536=6537

REVISION DETAIL
  https://phab.mercurial-scm.org/D2588

AFFECTED FILES
  mercurial/fancyopts.py
  tests/test-commit.t

CHANGE DETAILS

diff --git a/tests/test-commit.t b/tests/test-commit.t
--- a/tests/test-commit.t
+++ b/tests/test-commit.t
@@ -832,3 +832,21 @@
 
   $ cd ..
 
+Test that commit accpets multiple message (-m) flags
+
+  $ hg init test4
+  $ cd test4/
+  $ echo a>>a
+  $ hg commit -qAm "This is the first line" -m "This is the follwing line"
+  $ hg log -v
+  changeset:   0:5e33a0fb6989
+  tag: tip
+  user:test
+  date:Thu Jan 01 00:00:00 1970 +
+  files:   a
+  description:
+  This is the first line
+  
+  This is the follwing line
+
+  $ cd ..
diff --git a/mercurial/fancyopts.py b/mercurial/fancyopts.py
--- a/mercurial/fancyopts.py
+++ b/mercurial/fancyopts.py
@@ -362,7 +362,15 @@
 def abort(s):
 raise error.Abort(
 _('invalid value %r for option %s, %s') % (val, opt, s))
-state[name] = defmap[name].newstate(state[name], val, abort)
+if name == 'message':
+if state[name] != '':
+new_message = state[name] + "\n\n" + val
+state[name] = defmap[name].newstate(state[name], \
+new_message, abort)
+else:
+state[name] = defmap[name].newstate(state[name], val, 
abort)
+else :
+state[name] = defmap[name].newstate(state[name], val, abort)
 
 # return unparsed args
 return args



To: sangeet259, #hg-reviewers
Cc: jeffpc, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2588: commit: adds multiline commit message support(issue5616)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 added a comment.


  @jeffpc That means two new line. Shall I edit this then?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2588

To: sangeet259, #hg-reviewers
Cc: jeffpc, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2588: commit: adds multiline commit message support(issue5616)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 updated this revision to Diff 6536.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2588?vs=6438=6536

REVISION DETAIL
  https://phab.mercurial-scm.org/D2588

AFFECTED FILES
  mercurial/fancyopts.py
  tests/test-commit.t

CHANGE DETAILS

diff --git a/tests/test-commit.t b/tests/test-commit.t
--- a/tests/test-commit.t
+++ b/tests/test-commit.t
@@ -832,3 +832,20 @@
 
   $ cd ..
 
+Test that commit accpets multiple message (-m) flags
+
+  $ hg init test4
+  $ cd test4/
+  $ echo a>>a
+  $ hg commit -qAm "This is the first line" -m "This is the follwing line"
+  $ hg log -v
+  changeset:   0:5e33a0fb6989
+  tag: tip
+  user:test
+  date:Thu Jan 01 00:00:00 1970 +
+  files:   a
+  description:
+  This is the first line
+  This is the follwing line
+
+  $ cd ..
diff --git a/mercurial/fancyopts.py b/mercurial/fancyopts.py
--- a/mercurial/fancyopts.py
+++ b/mercurial/fancyopts.py
@@ -362,7 +362,15 @@
 def abort(s):
 raise error.Abort(
 _('invalid value %r for option %s, %s') % (val, opt, s))
-state[name] = defmap[name].newstate(state[name], val, abort)
+if name == 'message':
+if state[name] != '':
+new_message = state[name] + "\n" + val
+state[name] = defmap[name].newstate(state[name], \
+new_message, abort)
+else:
+state[name] = defmap[name].newstate(state[name], val, 
abort)
+else :
+state[name] = defmap[name].newstate(state[name], val, abort)
 
 # return unparsed args
 return args



To: sangeet259, #hg-reviewers
Cc: jeffpc, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] bdiff: avoid pointer arithmetic on void*

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1520137780 18000
#  Sat Mar 03 23:29:40 2018 -0500
# Node ID 8b9334b59784246d5a09bad537d6e15a0f190c97
# Parent  3f829cbdfe3a89c28c8807d3a933159fb2d93992
bdiff: avoid pointer arithmetic on void*

MSVC 2008 complains:

  mercurial/cext/bdiff.c(106) : error C2036: 'void *' : unknown size
  mercurial/cext/bdiff.c(107) : error C2036: 'void *' : unknown size

Maybe it's a gcc extension?

https://stackoverflow.com/questions/37460579/error-c2036-void-unknown-size

diff --git a/mercurial/cext/bdiff.c b/mercurial/cext/bdiff.c
--- a/mercurial/cext/bdiff.c
+++ b/mercurial/cext/bdiff.c
@@ -103,8 +103,8 @@
}
/* we can almost add: if (li == lmax) lcommon = li; */
 
-   an = bdiff_splitlines(ba.buf + lcommon, la - lcommon, );
-   bn = bdiff_splitlines(bb.buf + lcommon, lb - lcommon, );
+   an = bdiff_splitlines(((char *)ba.buf) + lcommon, la - lcommon, );
+   bn = bdiff_splitlines(((char *)ba.buf) + lcommon, lb - lcommon, );
if (!al || !bl) {
PyErr_NoMemory();
goto cleanup;
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2634: [DRAFT] xdiff: avoid hashing trimmed lines

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Things are a bit faster now:
  
hg perfbdiff --count 3000 --blocks --xdiff 
.hg/store/data/mercurial/commands.py.i 1
# before
! wall 1.510821 comb 1.50 user 1.50 sys 0.00 (best of 6)
# after
! wall 1.025628 comb 1.02 user 1.02 sys 0.00 (best of 8)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2634

AFFECTED FILES
  mercurial/thirdparty/xdiff/xprepare.c
  mercurial/thirdparty/xdiff/xutils.c
  mercurial/thirdparty/xdiff/xutils.h

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xutils.h 
b/mercurial/thirdparty/xdiff/xutils.h
--- a/mercurial/thirdparty/xdiff/xutils.h
+++ b/mercurial/thirdparty/xdiff/xutils.h
@@ -34,6 +34,7 @@
 long xdl_guess_lines(mmfile_t *mf, long sample);
 int xdl_blankline(const char *line, long size, long flags);
 int xdl_recmatch(const char *l1, long s1, const char *l2, long s2, long flags);
+void xdl_next_record(char const **data, char const *top);
 unsigned long xdl_hash_record(char const **data, char const *top, long flags);
 unsigned int xdl_hashbits(unsigned int size);
 int xdl_num_out(char *out, long val);
diff --git a/mercurial/thirdparty/xdiff/xutils.c 
b/mercurial/thirdparty/xdiff/xutils.c
--- a/mercurial/thirdparty/xdiff/xutils.c
+++ b/mercurial/thirdparty/xdiff/xutils.c
@@ -298,6 +298,15 @@
return ha;
 }
 
+inline void xdl_next_record(char const **data, char const *top) {
+   char const *next = memchr(*data, '\n', top - *data);
+   if (next) {
+   *data = next + 1;
+   } else {
+   *data = top;
+   }
+}
+
 unsigned long xdl_hash_record(char const **data, char const *top, long flags) {
unsigned long ha = 5381;
char const *ptr = *data;
diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -181,7 +181,7 @@
if ((cur = blk = xdl_mmfile_first(mf, )) != NULL) {
for (top = blk + bsize; cur < top; ) {
prev = cur;
-   hav = xdl_hash_record(, top, xpp->flags);
+   xdl_next_record(, top);
if (nrec >= narec) {
narec *= 2;
if (!(rrecs = (xrecord_t **) xdl_realloc(recs, 
narec * sizeof(xrecord_t *
@@ -192,7 +192,7 @@
goto abort;
crec->ptr = prev;
crec->size = (long) (cur - prev);
-   crec->ha = hav;
+   crec->ha = 0;
recs[nrec++] = crec;
}
}
@@ -243,7 +243,7 @@
 
unsigned int hbits;
long hsize;
-   long i;
+   long i, j;
long start = xdf->dstart - reserved;
long end = xdf->dend + reserved;
 
@@ -261,6 +261,16 @@
memset(rhash, 0, hsize * sizeof(xrecord_t *));
 
for (i = start; i <= end; i++) {
+   {
+   unsigned long ha = 5381;
+   char const *ptr = xdf->recs[i]->ptr;
+
+   for (j = 0; j < xdf->recs[i]->size; ++j) {
+   ha += (ha << 5);
+   ha ^= (unsigned long)ptr[j];
+   }
+   xdf->recs[i]->ha = ha;
+   }
if (xdl_classify_record(pass, cf, rhash, hbits, xdf->recs[i]) < 
0)
goto abort;
}



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 4 STABLE] subrepo: activate clone pooling to enable sharing with remote URLs

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 151990 18000
#  Thu Mar 01 11:37:00 2018 -0500
# Branch stable
# Node ID f8882513decb64848575894d21b2829c5d1c8f7c
# Parent  e070bffa43b8dcfad366a4941a69f0d34805d548
subrepo: activate clone pooling to enable sharing with remote URLs

This is the easiest way to ensure that repositories with remote subrepo
references can share the subrepos, consistent with how local subrepos can be
shared.

diff --git a/mercurial/subrepo.py b/mercurial/subrepo.py
--- a/mercurial/subrepo.py
+++ b/mercurial/subrepo.py
@@ -996,11 +996,25 @@
   update=False, bookmarks=False)
 self._repo = shared.local()
 else:
+# TODO: find a common place for this and this code in the
+# share.py wrap of the clone command.
+if parentrepo.shared():
+pool = self.ui.config('share', 'pool')
+if pool:
+pool = util.expandpath(pool)
+
+shareopts = {
+'pool': pool,
+'mode': self.ui.config('share', 'poolnaming'),
+}
+else:
+shareopts = {}
+
 self.ui.status(_('cloning subrepo %s from %s\n')
% (subrelpath(self), srcurl))
 other, cloned = hg.clone(self._repo._subparent.baseui, {},
  other, self._repo.root,
- update=False)
+ update=False, shareopts=shareopts)
 self._repo = cloned.local()
 self._initrepo(parentrepo, source, create=True)
 self._cachestorehash(srcurl)
diff --git a/tests/test-subrepo-recursion.t b/tests/test-subrepo-recursion.t
--- a/tests/test-subrepo-recursion.t
+++ b/tests/test-subrepo-recursion.t
@@ -292,8 +292,8 @@
   z2
   z3
 
-BUG: The remote subrepo should be cloned to the local pool, and then shared
-from there.
+Clone pooling from a remote URL will share the top level repo and the subrepos,
+even if they are referenced by remote URL.
 
   $ hg --config extensions.share= --config share.pool=$TESTTMP/pool \
   >clone http://localhost:$HGPORT shared
@@ -308,19 +308,25 @@
   no changes found
   updating working directory
   cloning subrepo foo from http://localhost:$HGPORT/foo
+  (sharing from new pooled repository af048e97ade2e236f754f05d07013e586af0f8bf)
   requesting all changes
   adding changesets
   adding manifests
   adding file changes
   added 4 changesets with 7 changes to 3 files
   new changesets af048e97ade2:65903cebad86
+  searching for changes
+  no changes found
   cloning subrepo foo/bar from http://localhost:$HGPORT/foo/bar
+  (sharing from new pooled repository 4904098473f96c900fec436dad267edd4da59fad)
   requesting all changes
   adding changesets
   adding manifests
   adding file changes
   added 3 changesets with 3 changes to 1 files
   new changesets 4904098473f9:31ecbdafd357
+  searching for changes
+  no changes found
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ cat access.log
@@ -341,11 +347,19 @@
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D1326fa26d0c00d2146c63b56bb6a45149d7325ac 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=1326fa26d0c00d2146c63b56bb6a45149d7325ac=1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=lookup HTTP/1.1" 200 - 
x-hgarg-1:key=0 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob)
   $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 
comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D65903cebad86f1a84bd4f1134f62fa7dcb7a1c98 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=capabilities HTTP/1.1" 200 - 
(glob)
+  $LOCALIP - 

[PATCH 1 of 4 STABLE] test-subrepo: demonstrate problems with subrepo sharing and absolute paths

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1519795767 18000
#  Wed Feb 28 00:29:27 2018 -0500
# Branch stable
# Node ID eca5075fdd5ec15e1a4917c0cadeb23e00feb2fe
# Parent  0a7c59a4c8352b9277a9676708cf6a30d3e2e306
test-subrepo: demonstrate problems with subrepo sharing and absolute paths

This affects remote paths in .hgsub, as well as clone pooling from a remote
source.

For reasons unknown, there are stability issues with the relative-path.t tests.
If run as a single test, it is stable.  If run with --loop, or with -jX for X>1,
the hash of the parent repo changes.  I'm seeing this on both Windows and Fedora
26.  I added an `hg log --debug`, and the manifest hash changes, but I have no
idea why.

diff --git a/tests/test-subrepo-recursion.t b/tests/test-subrepo-recursion.t
--- a/tests/test-subrepo-recursion.t
+++ b/tests/test-subrepo-recursion.t
@@ -292,6 +292,25 @@
   z2
   z3
 
+BUG: The remote subrepo should be cloned to the local pool, and then shared
+from there.
+
+  $ hg --config extensions.share= --config share.pool=$TESTTMP/pool \
+  >clone http://localhost:$HGPORT shared
+  (sharing from new pooled repository 23376cbba0d87c15906bb3652584927c140907bf)
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 3 changesets with 5 changes to 3 files
+  new changesets 23376cbba0d8:1326fa26d0c0
+  searching for changes
+  no changes found
+  updating working directory
+  sharing subrepo foo from http://localhost:$HGPORT/foo
+  abort: can only share local repositories (in subrepository "foo")
+  [255]
+
   $ cat access.log
   * "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
   * "GET /?cmd=batch HTTP/1.1" 200 - * (glob)
@@ -302,6 +321,14 @@
   * "GET /foo/bar?cmd=capabilities HTTP/1.1" 200 - (glob)
   * "GET /foo/bar?cmd=batch HTTP/1.1" 200 - * (glob)
   * "GET /foo/bar?cmd=getbundle HTTP/1.1" 200 - * (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=0 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 
comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D1326fa26d0c00d2146c63b56bb6a45149d7325ac 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=1326fa26d0c00d2146c63b56bb6a45149d7325ac=1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob)
 
   $ killdaemons.py
   $ rm hg1.pid error.log access.log
@@ -485,6 +512,22 @@
   commit: (clean)
   update: 4 new changesets (update)
 
+Sharing a local repo without the locally referenced subrepo (i.e. it was never
+updated from null), fails the same as a clone operation.
+
+  $ hg --config progress.disable=True clone -U ../empty ../empty2
+
+  $ hg --config extensions.share= --config progress.disable=True \
+  >share ../empty2 ../empty_share
+  updating working directory
+  abort: repository $TESTTMP/empty2/foo not found!
+  [255]
+
+  $ hg --config progress.disable=True clone ../empty2 ../empty_clone
+  updating to branch default
+  abort: repository $TESTTMP/empty2/foo not found!
+  [255]
+
 Disable progress extension and cleanup:
 
   $ mv $HGRCPATH.no-progress $HGRCPATH
diff --git a/tests/test-subrepo-relative-path.t 
b/tests/test-subrepo-relative-path.t
--- a/tests/test-subrepo-relative-path.t
+++ b/tests/test-subrepo-relative-path.t
@@ -72,6 +72,59 @@
source   ../sub
revision 863c1745b441bd97a8c4a096e87793073f4fb215
 
+Test sharing with a remote URL reference
+
+  $ hg init absolute_subrepo
+  $ cd absolute_subrepo
+  $ echo foo > foo.txt
+  $ hg ci -Am 'initial commit'
+  adding foo.txt
+  $ echo "sub = http://localhost:$HGPORT/sub; > .hgsub
+  $ hg ci -Am 'add absolute subrepo'
+  adding .hgsub
+  $ cd ..
+
+BUG: Remote subrepos cannot be shared, and pooled repos don't have their
+relative subrepos in the relative location stated in .hgsub.
+
+  $ hg --config extensions.share= --config share.pool=$TESTTMP/pool \
+  >clone absolute_subrepo cloned_from_abs
+  (sharing from new pooled repository 8d6a2f1e993b34b6557de0042cfe825ae12a8dae)
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 2 changesets with 3 changes to 3 files
+  new changesets 

[PATCH 4 of 4 STABLE] test-subrepo: glob away an unstable hash

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1520134164 18000
#  Sat Mar 03 22:29:24 2018 -0500
# Branch stable
# Node ID 564d66c2f7babd694a7c1da296574910763cd8b0
# Parent  f8882513decb64848575894d21b2829c5d1c8f7c
test-subrepo: glob away an unstable hash

This is the instability mentioned at the beginning of the series.  I don't like
hiding it, but I don't want to sit on a fix for a user reported problem while
trying to figure this out.

The instability seems related to the cset with a .hgsub with a remote URL.
(There's very little existing remote URL subrepo testing.)

diff --git a/tests/test-subrepo-relative-path.t 
b/tests/test-subrepo-relative-path.t
--- a/tests/test-subrepo-relative-path.t
+++ b/tests/test-subrepo-relative-path.t
@@ -96,7 +96,7 @@
   adding manifests
   adding file changes
   added 2 changesets with 3 changes to 3 files
-  new changesets 8d6a2f1e993b:c6d0e6ebd1c9
+  new changesets 8d6a2f1e993b:* (glob)
   searching for changes
   no changes found
   updating working directory
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4 STABLE] subrepo: don't attempt to share remote sources (issue5793)

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1519920780 18000
#  Thu Mar 01 11:13:00 2018 -0500
# Branch stable
# Node ID e070bffa43b8dcfad366a4941a69f0d34805d548
# Parent  eca5075fdd5ec15e1a4917c0cadeb23e00feb2fe
subrepo: don't attempt to share remote sources (issue5793)

Untangling _abssource() to resolve the new subrepo relative to the shared
parent's share path, and then either sharing from there (if it exists), or
cloning to that location and then sharing, is probably more than should be
attempted on stable.  Absolute subrepo references are discouraged, so for now,
this resumes the behavior prior to 68e0bcb90357 of cloning the absolute subrepo
locally.

diff --git a/mercurial/subrepo.py b/mercurial/subrepo.py
--- a/mercurial/subrepo.py
+++ b/mercurial/subrepo.py
@@ -982,7 +982,13 @@
 if len(self._repo) == 0:
 # use self._repo.vfs instead of self.wvfs to remove .hg only
 self._repo.vfs.rmtree()
-if parentrepo.shared():
+
+# A remote subrepo could be shared if there is a local copy
+# relative to the parent's share source.  But clone pooling doesn't
+# assemble the repos in a tree, so that can't be consistently done.
+# A simpler option is for the user to configure clone pooling, and
+# work with that.
+if parentrepo.shared() and hg.islocal(srcurl):
 self.ui.status(_('sharing subrepo %s from %s\n')
% (subrelpath(self), srcurl))
 shared = hg.share(self._repo._subparent.baseui,
diff --git a/tests/test-subrepo-recursion.t b/tests/test-subrepo-recursion.t
--- a/tests/test-subrepo-recursion.t
+++ b/tests/test-subrepo-recursion.t
@@ -307,9 +307,21 @@
   searching for changes
   no changes found
   updating working directory
-  sharing subrepo foo from http://localhost:$HGPORT/foo
-  abort: can only share local repositories (in subrepository "foo")
-  [255]
+  cloning subrepo foo from http://localhost:$HGPORT/foo
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 4 changesets with 7 changes to 3 files
+  new changesets af048e97ade2:65903cebad86
+  cloning subrepo foo/bar from http://localhost:$HGPORT/foo/bar
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 3 changesets with 3 changes to 1 files
+  new changesets 4904098473f9:31ecbdafd357
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
   $ cat access.log
   * "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
@@ -329,6 +341,11 @@
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D1326fa26d0c00d2146c63b56bb6a45149d7325ac 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=0=1326fa26d0c00d2146c63b56bb6a45149d7325ac=1326fa26d0c00d2146c63b56bb6a45149d7325ac=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=capabilities HTTP/1.1" 200 - (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 
comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==65903cebad86f1a84bd4f1134f62fa7dcb7a1c98=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=capabilities HTTP/1.1" 200 - 
(glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 
comp=$USUAL_COMPRESSIONS$ (glob)
+  $LOCALIP - - [$LOGDATE$] "GET /foo/bar?cmd=getbundle HTTP/1.1" 200 - 
x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$=1==31ecbdafd357f54b281c9bd1d681bb90de219e22=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
 
   $ killdaemons.py
   $ rm hg1.pid error.log access.log
diff --git a/tests/test-subrepo-relative-path.t 
b/tests/test-subrepo-relative-path.t
--- a/tests/test-subrepo-relative-path.t
+++ b/tests/test-subrepo-relative-path.t
@@ -84,8 +84,9 @@
   adding .hgsub
   $ cd ..
 
-BUG: Remote subrepos cannot be shared, and pooled repos don't have their
-relative subrepos in the relative location stated in .hgsub.
+Clone pooling works for local clones with a remote subrepo reference.
+
+BUG: subrepos should be shared out of the pool.
 
   $ hg --config extensions.share= --config share.pool=$TESTTMP/pool \
   >clone absolute_subrepo cloned_from_abs
@@ -99,31 +100,55 @@
   searching for changes
   no changes found
   updating working directory
-  sharing subrepo sub from http://localhost:$HGPORT/sub
-  abort: can only share local repositories (in subrepository "sub")
-  [255]
+  

Re: [PATCH STABLE] annotate: do not poorly split lines at CR (issue5798)

[Matt Harbison]

On Wed, 21 Feb 2018 07:49:41 -0500, Yuya Nishihara  wrote:


# HG changeset patch
# User Yuya Nishihara 
# Date 1519215245 -32400
#  Wed Feb 21 21:14:05 2018 +0900
# Branch stable
# Node ID 24b17a714a92c8fe860db5f6d0d49c23293deec6
# Parent  c19e66dacaa184feba31136c18a369ba995ddfe4
annotate: do not poorly split lines at CR (issue5798)


Should this difference be conditionalized away, or is it  
important/unexpected?


$ ./run-tests.py --local -t9000 -j9 test-annotate.t

--- c:/Users/Matt/projects/hg/tests/test-annotate.t
+++ c:/Users/Matt/projects/hg/tests/test-annotate.t.err
@@ -913,14 +913,14 @@
   $ hg ci -m1

   $ hg annotate -r0 a | substcr
-  0: 0a[CR]0b[CR]
-  0: 0c[CR]0d[CR]
+  0: 0a[CR]0b
+  0: 0c[CR]0d
   0: 0e
   0: 0f
   0: 0g
   $ hg annotate -r1 a | substcr
-  0: 0a[CR]0b[CR]
-  1: 1c[CR]1d[CR]
+  0: 0a[CR]0b
+  1: 1c[CR]1d
   0: 0e
   1: 1f
   0: 0g
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2632: fuzz: add a fuzzer for xdiff

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG38a81b88688b: fuzz: add a fuzzer for xdiff (authored by 
durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2632?vs=6531=6533

REVISION DETAIL
  https://phab.mercurial-scm.org/D2632

AFFECTED FILES
  contrib/fuzz/Makefile
  contrib/fuzz/xdiff.cc

CHANGE DETAILS

diff --git a/contrib/fuzz/xdiff.cc b/contrib/fuzz/xdiff.cc
new file mode 100644
--- /dev/null
+++ b/contrib/fuzz/xdiff.cc
@@ -0,0 +1,67 @@
+/*
+ * xdiff.cc - fuzzer harness for thirdparty/xdiff
+ *
+ * Copyright 2018, Google Inc.
+ *
+ * This software may be used and distributed according to the terms of
+ * the GNU General Public License, incorporated herein by reference.
+ */
+#include "thirdparty/xdiff/xdiff.h"
+#include 
+#include 
+
+extern "C" {
+
+int hunk_consumer(long a1, long a2, long b1, long b2, void *priv)
+{
+   // TODO: probably also test returning -1 from this when things break?
+   return 0;
+}
+
+int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)
+{
+   if (!Size) {
+   return 0;
+   }
+   // figure out a random point in [0, Size] to split our input.
+   size_t split = Data[0] / 255.0 * Size;
+
+   mmfile_t a, b;
+
+   // `a` input to diff is data[1:split]
+   a.ptr = (char *)Data + 1;
+   // which has len split-1
+   a.size = split - 1;
+   // `b` starts at the next byte after `a` ends
+   b.ptr = a.ptr + a.size;
+   b.size = Size - split;
+   xpparam_t xpp = {
+   XDF_INDENT_HEURISTIC, /* flags */
+   NULL, /* anchors */
+   0,/* anchors_nr */
+   };
+   xdemitconf_t xecfg = {
+   0,  /* ctxlen */
+   0,  /* interhunkctxlen */
+   XDL_EMIT_BDIFFHUNK, /* flags */
+   NULL,   /* find_func */
+   NULL,   /* find_func_priv */
+   hunk_consumer,  /* hunk_consume_func */
+   };
+   xdemitcb_t ecb = {
+   NULL, /* priv */
+   NULL, /* outf */
+   };
+   xdl_diff(, , , , );
+   return 0; // Non-zero return values are reserved for future use.
+}
+
+#ifdef HG_FUZZER_INCLUDE_MAIN
+int main(int argc, char **argv)
+{
+   const char data[] = "asdf";
+   return LLVMFuzzerTestOneInput((const uint8_t *)data, 4);
+}
+#endif
+
+} // extern "C"
diff --git a/contrib/fuzz/Makefile b/contrib/fuzz/Makefile
--- a/contrib/fuzz/Makefile
+++ b/contrib/fuzz/Makefile
@@ -13,8 +13,28 @@
$$CXX $$CXXFLAGS -std=c++11 -I../../mercurial bdiff.cc \
  bdiff-oss-fuzz.o -lFuzzingEngine -o $$OUT/bdiff_fuzzer
 
-all: bdiff
+x%.o: ../../mercurial/thirdparty/xdiff/x%.c 
../../mercurial/thirdparty/xdiff/*.h
+   clang -g -O1 -fsanitize=fuzzer-no-link,address -c \
+ -o $@ \
+ $<
+
+xdiff: xdiff.cc xdiffi.o  xemit.o  xmerge.o  xprepare.o  xutils.o
+   clang -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 
-fsanitize=fuzzer-no-link,address \
+ -I../../mercurial xdiff.cc \
+ xdiffi.o xemit.o xmerge.o xprepare.o xutils.o -o xdiff
 
-oss-fuzz: bdiff_fuzzer
+fuzz-x%.o: ../../mercurial/thirdparty/xdiff/x%.c 
../../mercurial/thirdparty/xdiff/*.h
+   $$CC $$CFLAGS -c \
+ -o $@ \
+ $<
+
+xdiff_fuzzer: xdiff.cc fuzz-xdiffi.o  fuzz-xemit.o  fuzz-xmerge.o  
fuzz-xprepare.o  fuzz-xutils.o
+   $$CXX $$CXXFLAGS -std=c++11 -I../../mercurial xdiff.cc \
+ fuzz-xdiffi.o fuzz-xemit.o fuzz-xmerge.o fuzz-xprepare.o 
fuzz-xutils.o \
+ -lFuzzingEngine -o $$OUT/xdiff_fuzzer
+
+all: bdiff xdiff
+
+oss-fuzz: bdiff_fuzzer xdiff_fuzzer
 
 .PHONY: all oss-fuzz



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2633: fuzz: add a quick README to try and document how to test new fuzzers

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3f829cbdfe3a: fuzz: add a quick README to try and document 
how to test new fuzzers (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2633?vs=6532=6534

REVISION DETAIL
  https://phab.mercurial-scm.org/D2633

AFFECTED FILES
  contrib/fuzz/README.rst

CHANGE DETAILS

diff --git a/contrib/fuzz/README.rst b/contrib/fuzz/README.rst
new file mode 100644
--- /dev/null
+++ b/contrib/fuzz/README.rst
@@ -0,0 +1,14 @@
+How to add fuzzers (partially cribbed from oss-fuzz[0]):
+
+  1) git clone https://github.com/google/oss-fuzz
+  2) cd oss-fuzz
+  3) python infra/helper.py build_image mercurial
+  4) docker run --cap-add=SYS_PTRACE -it -v $HG_REPO_PATH:/hg-new \
+ gcr.io/oss-fuzz/mercurial bash
+  5) cd /src
+  6) rm -r mercurial
+  7) ln -s /hg-new mercurial
+  8) cd mercurial
+  9) compile
+
+0: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2632: fuzz: add a fuzzer for xdiff

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Based entirely on the fuzzer for bdiff.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2632

AFFECTED FILES
  contrib/fuzz/Makefile
  contrib/fuzz/xdiff.cc

CHANGE DETAILS

diff --git a/contrib/fuzz/xdiff.cc b/contrib/fuzz/xdiff.cc
new file mode 100644
--- /dev/null
+++ b/contrib/fuzz/xdiff.cc
@@ -0,0 +1,67 @@
+/*
+ * xdiff.cc - fuzzer harness for thirdparty/xdiff
+ *
+ * Copyright 2018, Google Inc.
+ *
+ * This software may be used and distributed according to the terms of
+ * the GNU General Public License, incorporated herein by reference.
+ */
+#include "thirdparty/xdiff/xdiff.h"
+#include 
+#include 
+
+extern "C" {
+
+int hunk_consumer(long a1, long a2, long b1, long b2, void *priv)
+{
+   // TODO: probably also test returning -1 from this when things break?
+   return 0;
+}
+
+int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)
+{
+   if (!Size) {
+   return 0;
+   }
+   // figure out a random point in [0, Size] to split our input.
+   size_t split = Data[0] / 255.0 * Size;
+
+   mmfile_t a, b;
+
+   // `a` input to diff is data[1:split]
+   a.ptr = (char *)Data + 1;
+   // which has len split-1
+   a.size = split - 1;
+   // `b` starts at the next byte after `a` ends
+   b.ptr = a.ptr + a.size;
+   b.size = Size - split;
+   xpparam_t xpp = {
+   XDF_INDENT_HEURISTIC, /* flags */
+   NULL, /* anchors */
+   0,/* anchors_nr */
+   };
+   xdemitconf_t xecfg = {
+   0,  /* ctxlen */
+   0,  /* interhunkctxlen */
+   XDL_EMIT_BDIFFHUNK, /* flags */
+   NULL,   /* find_func */
+   NULL,   /* find_func_priv */
+   hunk_consumer,  /* hunk_consume_func */
+   };
+   xdemitcb_t ecb = {
+   NULL, /* priv */
+   NULL, /* outf */
+   };
+   xdl_diff(, , , , );
+   return 0; // Non-zero return values are reserved for future use.
+}
+
+#ifdef HG_FUZZER_INCLUDE_MAIN
+int main(int argc, char **argv)
+{
+   const char data[] = "asdf";
+   return LLVMFuzzerTestOneInput((const uint8_t *)data, 4);
+}
+#endif
+
+} // extern "C"
diff --git a/contrib/fuzz/Makefile b/contrib/fuzz/Makefile
--- a/contrib/fuzz/Makefile
+++ b/contrib/fuzz/Makefile
@@ -13,8 +13,28 @@
$$CXX $$CXXFLAGS -std=c++11 -I../../mercurial bdiff.cc \
  bdiff-oss-fuzz.o -lFuzzingEngine -o $$OUT/bdiff_fuzzer
 
-all: bdiff
+x%.o: ../../mercurial/thirdparty/xdiff/x%.c 
../../mercurial/thirdparty/xdiff/*.h
+   clang -g -O1 -fsanitize=fuzzer-no-link,address -c \
+ -o $@ \
+ $<
 
-oss-fuzz: bdiff_fuzzer
+xdiff: xdiff.cc xdiffi.o  xemit.o  xmerge.o  xprepare.o  xutils.o
+   clang -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 
-fsanitize=fuzzer-no-link,address \
+ -I../../mercurial xdiff.cc \
+ xdiffi.o xemit.o xmerge.o xprepare.o xutils.o -o xdiff
+
+fuzz-x%.o: ../../mercurial/thirdparty/xdiff/x%.c 
../../mercurial/thirdparty/xdiff/*.h
+   $$CC $$CFLAGS -c \
+ -o $@ \
+ $<
+
+xdiff_fuzzer: xdiff.cc fuzz-xdiffi.o  fuzz-xemit.o  fuzz-xmerge.o  
fuzz-xprepare.o  fuzz-xutils.o
+   $$CXX $$CXXFLAGS -std=c++11 -I../../mercurial xdiff.cc \
+ fuzz-xdiffi.o fuzz-xemit.o fuzz-xmerge.o fuzz-xprepare.o 
fuzz-xutils.o \
+ -lFuzzingEngine -o $$OUT/xdiff_fuzzer
+
+all: bdiff xdiff
+
+oss-fuzz: bdiff_fuzzer xdiff_fuzzer
 
 .PHONY: all oss-fuzz



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2633: fuzz: add a quick README to try and document how to test new fuzzers

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2633

AFFECTED FILES
  contrib/fuzz/README.rst

CHANGE DETAILS

diff --git a/contrib/fuzz/README.rst b/contrib/fuzz/README.rst
new file mode 100644
--- /dev/null
+++ b/contrib/fuzz/README.rst
@@ -0,0 +1,14 @@
+How to add fuzzers (partially cribbed from oss-fuzz[0]):
+
+  1) git clone https://github.com/google/oss-fuzz
+  2) cd oss-fuzz
+  3) python infra/helper.py build_image mercurial
+  4) docker run --cap-add=SYS_PTRACE -it -v $HG_REPO_PATH:/hg-new \
+ gcr.io/oss-fuzz/mercurial bash
+  5) cd /src
+  6) rm -r mercurial
+  7) ln -s /hg-new mercurial
+  8) cd mercurial
+  9) compile
+
+0: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2631: [RFC] xdiff: skip trimmed lines when preparing the hashtable

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  NOTE: I'm still reading xdiffi.c to understand whether this is a safe
  optimization or not.
  
  xdiff has a "xdl_trim_ends" function that removes common prefix and suffix.
  
  Previously, xdiff will build a hashtable for all lines. That is a waste of
  time for trimmed lines. This diff changes the logic so trimmed lines will be
  ignored when building the hashtable. Note: the hashtable is still needed for
  shifting purpose, so it does not blindly take whatever `xdl_trim_ends` says,
  but also looks around.
  
  For the following test case:
  
#!python
open('a','w').write(''.join('%s\n' % (i % 10) for i in 
xrange(1000)))
open('b','w').write(''.join('%s\n' % (i % 10) for i in 
xrange(1001)))
  
  This series reduces xdiff's time for the above case from 1.1 seconds 
(https://phab.mercurial-scm.org/D2604)
  to 0.6 seconds.
  
  However, GNU diffutils can perform even better (<0.1 seconds), there are
  still things to catch up.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2631

AFFECTED FILES
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -62,6 +62,7 @@
 static int xdl_clean_mmatch(char const *dis, long i, long s, long e);
 static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t 
*xdf2);
 static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);
+static long xdl_trim_reserved_lines(xdfile_t *xdf1, xdfile_t *xdf2);
 
 
 
@@ -234,21 +235,32 @@
  * unique. This makes future calculation faster - they can just compare "ha"
  * instead of comparing line content.
  */
-static int xdl_prepare_hashtable(unsigned int pass, mmfile_t *mf,
-   xpparam_t const *xpp, xdlclassifier_t *cf, xdfile_t *xdf) {
+static int xdl_prepare_hashtable(unsigned int pass, long reserved, mmfile_t
+   *mf, xpparam_t const *xpp, xdlclassifier_t *cf, xdfile_t *xdf)
+{
xrecord_t **rhash = NULL;
-   long nrec = xdf->nrec;
+   long nrec;
+
unsigned int hbits;
long hsize;
long i;
+   long start = xdf->dstart - reserved;
+   long end = xdf->dend + reserved;
+
+   if (start < 0)
+   start = 0;
+   if (end >= xdf->nrec)
+   end = xdf->nrec - 1;
+
+   nrec = end - start;
 
hbits = xdl_hashbits((unsigned int) nrec);
hsize = 1 << hbits;
if (!(rhash = (xrecord_t **) xdl_malloc(hsize * sizeof(xrecord_t *
goto abort;
memset(rhash, 0, hsize * sizeof(xrecord_t *));
 
-   for (i = 0; i < nrec; ++i) {
+   for (i = start; i <= end; i++) {
if (xdl_classify_record(pass, cf, rhash, hbits, xdf->recs[i]) < 
0)
goto abort;
}
@@ -275,7 +287,7 @@
 
 int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,
xdfenv_t *xe) {
-   long enl1, enl2, sample;
+   long enl1, enl2, sample, reserved;
xdlclassifier_t cf;
 
memset(, 0, sizeof(cf));
@@ -307,13 +319,14 @@
return -1;
}
 
-   if (xdl_prepare_hashtable(1, mf1, xpp, , >xdf1) < 0) {
+   reserved = xdl_trim_reserved_lines(>xdf1, >xdf2);
+   if (xdl_prepare_hashtable(1, reserved, mf1, xpp, , >xdf1) < 0) {
xdl_free_ctx(>xdf1);
xdl_free_ctx(>xdf2);
xdl_free_classifier();
return -1;
}
-   if (xdl_prepare_hashtable(2, mf2, xpp, , >xdf2) < 0) {
+   if (xdl_prepare_hashtable(2, reserved, mf2, xpp, , >xdf2) < 0) {
xdl_free_ctx(>xdf1);
xdl_free_ctx(>xdf2);
xdl_free_classifier();
@@ -462,7 +475,6 @@
return 0;
 }
 
-
 /*
  * Early trim initial and terminal matching records.
  */
@@ -500,3 +512,22 @@
 
return 0;
 }
+
+
+/*
+ * Return "reserved lines" for possible hunk shifting. Normally, only look at
+ * lines in dstart..dend range. But hunk shifting also needs accurate line
+ * hashes. Estimated hunk size and reserve lines for shifting purpose.
+ *
+ * This would be used by xdl_prepare_hashtable, to build accurate hash values.
+ */
+static long xdl_trim_reserved_lines(xdfile_t *xdf1, xdfile_t *xdf2) {
+   long lines = 0;
+   if (xdf1->dend > xdf1->dstart)
+   lines += xdf1->dend - xdf1->dstart;
+   if (xdf2->dend > xdf2->dstart)
+   lines += xdf2->dend - xdf2->dstart;
+   return lines;
+}
+
+



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2630: xdiff: move hashtable calculation to a separate function

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Before running the main diff algorithm, xdiff will "prepare" the contexts
  for both files. That includes splitting, hashing all the lines, and building
  hash tables for those lines. The hash table building process could be
  expensive. Moving it out so it can be optimized separately.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2630

AFFECTED FILES
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -157,36 +157,25 @@
 
 static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, 
xpparam_t const *xpp,
   xdlclassifier_t *cf, xdfile_t *xdf) {
-   unsigned int hbits;
-   long nrec, hsize, bsize;
+   long nrec, bsize;
unsigned long hav;
char const *blk, *cur, *top, *prev;
xrecord_t *crec;
xrecord_t **recs, **rrecs;
-   xrecord_t **rhash;
unsigned long *ha;
char *rchg;
long *rindex;
 
ha = NULL;
rindex = NULL;
rchg = NULL;
-   rhash = NULL;
recs = NULL;
 
if (xdl_cha_init(>rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)
goto abort;
if (!(recs = (xrecord_t **) xdl_malloc(narec * sizeof(xrecord_t *
goto abort;
 
-   {
-   hbits = xdl_hashbits((unsigned int) narec);
-   hsize = 1 << hbits;
-   if (!(rhash = (xrecord_t **) xdl_malloc(hsize * 
sizeof(xrecord_t *
-   goto abort;
-   memset(rhash, 0, hsize * sizeof(xrecord_t *));
-   }
-
nrec = 0;
if ((cur = blk = xdl_mmfile_first(mf, )) != NULL) {
for (top = blk + bsize; cur < top; ) {
@@ -204,9 +193,6 @@
crec->size = (long) (cur - prev);
crec->ha = hav;
recs[nrec++] = crec;
-
-   if (xdl_classify_record(pass, cf, rhash, hbits, crec) < 
0)
-   goto abort;
}
}
 
@@ -221,27 +207,60 @@
 
xdf->nrec = nrec;
xdf->recs = recs;
-   xdf->hbits = hbits;
-   xdf->rhash = rhash;
xdf->rchg = rchg + 1;
xdf->rindex = rindex;
xdf->nreff = 0;
xdf->ha = ha;
xdf->dstart = 0;
xdf->dend = nrec - 1;
 
+   /* use xdl_prepare_hashtable to set them */
+   xdf->hbits = 0;
+   xdf->rhash = NULL;
+
return 0;
 
 abort:
xdl_free(ha);
xdl_free(rindex);
xdl_free(rchg);
-   xdl_free(rhash);
xdl_free(recs);
xdl_cha_free(>rcha);
return -1;
 }
 
+/*
+ * Adjust hash values for records (lines) in a file so the hash values become
+ * unique. This makes future calculation faster - they can just compare "ha"
+ * instead of comparing line content.
+ */
+static int xdl_prepare_hashtable(unsigned int pass, mmfile_t *mf,
+   xpparam_t const *xpp, xdlclassifier_t *cf, xdfile_t *xdf) {
+   xrecord_t **rhash = NULL;
+   long nrec = xdf->nrec;
+   unsigned int hbits;
+   long hsize;
+   long i;
+
+   hbits = xdl_hashbits((unsigned int) nrec);
+   hsize = 1 << hbits;
+   if (!(rhash = (xrecord_t **) xdl_malloc(hsize * sizeof(xrecord_t *
+   goto abort;
+   memset(rhash, 0, hsize * sizeof(xrecord_t *));
+
+   for (i = 0; i < nrec; ++i) {
+   if (xdl_classify_record(pass, cf, rhash, hbits, xdf->recs[i]) < 
0)
+   goto abort;
+   }
+
+   xdf->hbits = hbits;
+   xdf->rhash = rhash;
+
+   return 0;
+abort:
+   xdl_free(rhash);
+   return -1;
+}
 
 static void xdl_free_ctx(xdfile_t *xdf) {
 
@@ -288,6 +307,19 @@
return -1;
}
 
+   if (xdl_prepare_hashtable(1, mf1, xpp, , >xdf1) < 0) {
+   xdl_free_ctx(>xdf1);
+   xdl_free_ctx(>xdf2);
+   xdl_free_classifier();
+   return -1;
+   }
+   if (xdl_prepare_hashtable(2, mf2, xpp, , >xdf2) < 0) {
+   xdl_free_ctx(>xdf1);
+   xdl_free_ctx(>xdf2);
+   xdl_free_classifier();
+   return -1;
+   }
+
if (xdl_cleanup_records(, >xdf1, >xdf2) < 0) {
xdl_free_ctx(>xdf2);
xdl_free_ctx(>xdf1);



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2628: xdiff: expand xdl_optimize_ctxs

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The function has 2 steps: trim_ends, and cleanup_records. We'll be inserting
  something in-between that is not "optimize", but "initialize". So expand the
  function to make the latter change easier.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2628

AFFECTED FILES
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -62,7 +62,6 @@
 static int xdl_clean_mmatch(char const *dis, long i, long s, long e);
 static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t 
*xdf2);
 static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);
-static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t 
*xdf2);
 
 
 
@@ -282,7 +281,14 @@
return -1;
}
 
-   if (xdl_optimize_ctxs(, >xdf1, >xdf2) < 0) {
+   if (xdl_trim_ends(>xdf1, >xdf2) < 0) {
+   xdl_free_ctx(>xdf2);
+   xdl_free_ctx(>xdf1);
+   xdl_free_classifier();
+   return -1;
+   }
+
+   if (xdl_cleanup_records(, >xdf1, >xdf2) < 0) {
xdl_free_ctx(>xdf2);
xdl_free_ctx(>xdf1);
xdl_free_classifier();
@@ -452,15 +458,3 @@
 
return 0;
 }
-
-
-static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t 
*xdf2) {
-
-   if (xdl_trim_ends(xdf1, xdf2) < 0 ||
-   xdl_cleanup_records(cf, xdf1, xdf2) < 0) {
-
-   return -1;
-   }
-
-   return 0;
-}



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2629: xdiff: do not rely on hashtable in xdl_trim_ends

[quark (Jun Wu)]

quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Upcoming changes will narrow down the hashtable content to not include
  trimmed lines. So let's just compare line contents in `xdl_trim_ends`.
  It's still cheaper than calculating unique hash values for all lines.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2629

AFFECTED FILES
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -441,17 +441,27 @@
recs1 = xdf1->recs;
recs2 = xdf2->recs;
for (i = 0, lim = XDL_MIN(xdf1->nrec, xdf2->nrec); i < lim;
-i++, recs1++, recs2++)
-   if ((*recs1)->ha != (*recs2)->ha)
+i++, recs1++, recs2++) {
+   /* rec->ha is not processed by xdl_prepare_hashtable, so
+* they are not unique - do a content match here */
+   if ((*recs1)->size != (*recs2)->size)
break;
+   if (memcmp((*recs1)->ptr, (*recs2)->ptr, (*recs1)->size) != 0)
+   break;
+   }
 
xdf1->dstart = xdf2->dstart = i;
 
recs1 = xdf1->recs + xdf1->nrec - 1;
recs2 = xdf2->recs + xdf2->nrec - 1;
-   for (lim -= i, i = 0; i < lim; i++, recs1--, recs2--)
-   if ((*recs1)->ha != (*recs2)->ha)
+   for (lim -= i, i = 0; i < lim; i++, recs1--, recs2--) {
+   /* rec->ha is not processed by xdl_prepare_hashtable, so
+* they are not unique - do a content match here */
+   if ((*recs1)->size != (*recs2)->size)
break;
+   if (memcmp((*recs1)->ptr, (*recs2)->ptr, (*recs1)->size) != 0)
+   break;
+   }
 
xdf1->dend = xdf1->nrec - i - 1;
xdf2->dend = xdf2->nrec - i - 1;



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2626: xdiff: use xxhash in xdiff

[indygreg (Gregory Szorc)]

indygreg added a comment.


  I can reproduce the results. xxhash needs more than 4-8 bytes to outperform 
the old hasher. I also suspect more hash collisions are present with xxhash. 
Although I haven't verified.
  
  The input showing the regression effectively turns the code into a hash table 
benchmark.
  
  But my results on actual repo data show an improvement.
  
  FWIW, memchr+djb hash perform about the same as the original code on the 
perfbdiff -m case.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2626

To: indygreg, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2626: xdiff: use xxhash in xdiff

[quark (Jun Wu)]

quark requested changes to this revision.
quark added a comment.
This revision now requires changes to proceed.


  It seems xxhash can make things slower if the line is short.
  
## test case
open('a','w').write(''.join('%s\n' % (i % 10) for i in 
xrange(1000)))
open('b','w').write(''.join('%s\n' % (i % 10) for i in 
xrange(1001)))
  
  
  
## before xxhash
./xdiff /tmp/[ab] &> /dev/null  0.67s user 0.21s system 99% cpu 0.875 total
## after xxhash
./xdiff /tmp/[ab] &> /dev/null  2.86s user 0.43s system 98% cpu 3.315 total
  
  The memchr change does not change much - it's still about 3 seconds on my 
machine.
  
  Code I use for testing:
  
// contrib/xdiff/xdiff.c
#include 
#include 
#include 
#include 

#include "mercurial/thirdparty/xdiff/xdiff.h"

#define abort(...)  
   \
{  \
fprintf(stderr, __VA_ARGS__);  \
exit(-1);  \
}

char buf[4096000];
void readfile(const char *path, mmfile_t *file)
{
memset(file, 0, sizeof(*file));
FILE *fp = fopen(path, "r");
if (!fp) {
abort("cannot open %s\n", path);
}
while (!feof(fp)) {
size_t size = fread(buf, 1, sizeof buf, fp);
if (size > 0) {
size_t new_size = file->size + size;
file->ptr = realloc(file->ptr, new_size);
if (!file->ptr) {
abort("cannot allocate\n");
}
memcpy(file->ptr + file->size, buf, size);
file->size = new_size;
}
}
fclose(fp);
}

static int xdiff_outf(void *priv_, mmbuffer_t *mb, int nbuf)
{
int i;
for (i = 0; i < nbuf; i++) {
write(STDOUT_FILENO, mb[i].ptr, mb[i].size);
}
return 0;
}

int main(int argc, char const *argv[])
{
if (argc < 3) {
abort("usage: %s FILE1 FILE2\n", argv[0]);
}

mmfile_t a, b;

readfile(argv[1], );
readfile(argv[2], );

xpparam_t xpp = {
0,/* flags */
NULL, /* anchors */
0,/* anchors_nr */
};
xdemitconf_t xecfg = {
3,/* ctxlen */
0,/* interhunkctxlen */
0,/* flags */
NULL, /* find_func */
NULL, /* find_func_priv */
NULL, /* hunk_consume_func */
};
xdemitcb_t ecb = {
0,   /* priv */
_outf, /* outf */
};

xdl_diff(, , , , );

free(a.ptr);
free(b.ptr);

return 0;
}
  
  
  
## contrib/xdiff/Makefile
xdiff: ../../mercurial/thirdparty/xdiff/*.c xdiff.c
gcc -O2 -g -std=c99 -I../.. -I../../mercurial/thirdparty/xdiff 
-I../../mercurial -o $@ $^

clean:
-rm -f xdiff

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2626

To: indygreg, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2626: xdiff: use xxhash in xdiff

[quark (Jun Wu)]

quark accepted this revision.
quark added a comment.


  I think the perf win comes from less hash collisions.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2626

To: indygreg, #hg-reviewers, quark
Cc: quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2627: xdiff: use memchr instead of character scanning

[quark (Jun Wu)]

quark added a comment.


  It seems `mdiff.textdiff` used by `perfbdiff` won't actually use xdiff. Maybe 
`perfunidiff --config experimental.xdiff=1` should be used instead?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2627

To: indygreg, #hg-reviewers, quark
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2625: xxhash: vendor external library

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  xxhash is a pretty fast non-cryptographically strong hashing
  function.
  
  We vendor the C library from Git commit
  921182349a87760bfc016dcd54fa3136d4c8150d from
  https://github.com/Cyan4973/xxHash.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2625

AFFECTED FILES
  mercurial/thirdparty/xxhash/LICENSE
  mercurial/thirdparty/xxhash/xxhash.c
  mercurial/thirdparty/xxhash/xxhash.h

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xxhash/xxhash.h 
b/mercurial/thirdparty/xxhash/xxhash.h
new file mode 100644
--- /dev/null
+++ b/mercurial/thirdparty/xxhash/xxhash.h
@@ -0,0 +1,294 @@
+/*
+   xxHash - Extremely Fast Hash algorithm
+   Header File
+   Copyright (C) 2012-2016, Yann Collet.
+
+   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions are
+   met:
+
+   * Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+   * Redistributions in binary form must reproduce the above
+   copyright notice, this list of conditions and the following disclaimer
+   in the documentation and/or other materials provided with the
+   distribution.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+   You can contact the author at :
+   - xxHash source repository : https://github.com/Cyan4973/xxHash
+*/
+
+/* Notice extracted from xxHash homepage :
+
+xxHash is an extremely fast Hash algorithm, running at RAM speed limits.
+It also successfully passes all tests from the SMHasher suite.
+
+Comparison (single thread, Windows Seven 32 bits, using SMHasher on a Core 2 
Duo @3GHz)
+
+NameSpeed   Q.Score   Author
+xxHash  5.4 GB/s 10
+CrapWow 3.2 GB/s  2   Andrew
+MumurHash 3a2.7 GB/s 10   Austin Appleby
+SpookyHash  2.0 GB/s 10   Bob Jenkins
+SBox1.4 GB/s  9   Bret Mulvey
+Lookup3 1.2 GB/s  9   Bob Jenkins
+SuperFastHash   1.2 GB/s  1   Paul Hsieh
+CityHash64  1.05 GB/s10   Pike & Alakuijala
+FNV 0.55 GB/s 5   Fowler, Noll, Vo
+CRC32   0.43 GB/s 9
+MD5-32  0.33 GB/s10   Ronald L. Rivest
+SHA1-32 0.28 GB/s10
+
+Q.Score is a measure of quality of the hash function.
+It depends on successfully passing SMHasher test set.
+10 is a perfect score.
+
+A 64-bit version, named XXH64, is available since r35.
+It offers much better speed, but for 64-bit applications only.
+Name Speed on 64 bitsSpeed on 32 bits
+XXH64   13.8 GB/s1.9 GB/s
+XXH326.8 GB/s6.0 GB/s
+*/
+
+#ifndef XXHASH_H_5627135585666179
+#define XXHASH_H_5627135585666179 1
+
+#if defined (__cplusplus)
+extern "C" {
+#endif
+
+
+/* 
+*  Definitions
+**/
+#include/* size_t */
+typedef enum { XXH_OK=0, XXH_ERROR } XXH_errorcode;
+
+
+/* 
+*  API modifier
+**/
+/** XXH_PRIVATE_API
+*   This is useful to include xxhash functions in `static` mode
+*   in order to inline them, and remove their symbol from the public list.
+*   Methodology :
+* #define XXH_PRIVATE_API
+* #include "xxhash.h"
+*   `xxhash.c` is automatically included.
+*   It's not useful to compile and link it as a separate module.
+*/
+#ifdef XXH_PRIVATE_API
+#  ifndef XXH_STATIC_LINKING_ONLY
+#define XXH_STATIC_LINKING_ONLY
+#  endif
+#  if defined(__GNUC__)
+#define XXH_PUBLIC_API static __inline __attribute__((unused))
+#  elif defined (__cplusplus) || (defined (__STDC_VERSION__) && 
(__STDC_VERSION__ >= 199901L) /* C99 */)
+#define XXH_PUBLIC_API static inline
+#  elif defined(_MSC_VER)
+#define XXH_PUBLIC_API static __inline
+#  else
+ /* this version may generate warnings for unused static functions */
+#define XXH_PUBLIC_API static
+#  endif
+#else
+#  define XXH_PUBLIC_API   /* do nothing */

D2627: xdiff: use memchr instead of character scanning

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Compilers (at least in the configuration used to build Mercurial)
  don't seem to be able to optimize a loop to look for a byte in a
  buffer very well.
  
  After removing hashing from the loop in our previous commit, we
  no longer have a good reason to use a loop at all: we can instead
  use memchr() to find a byte value in a memory range. memchr() will
  often be backed by platform-optimal, hand-written assembly that will
  perform better than anything a compiler can emit.
  
  Using memchr() to scan for newlines makes xdiff a bit faster. On
  the mozilla-central repository:
  
$ hg perfbdiff --alldata -c --count 10 --blocks --xdiff 40
! wall 0.796796 comb 0.79 user 0.79 sys 0.00 (best of 13)
! wall 0.589753 comb 0.59 user 0.59 sys 0.00 (best of 17)

$ hg perfbdiff -m --count 100 --blocks --xdiff 40
! wall 9.450092 comb 9.46 user 8.47 sys 0.99 (best of 3)
! wall 7.364899 comb 7.36 user 6.43 sys 0.93 (best of 3)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2627

AFFECTED FILES
  mercurial/thirdparty/xdiff/xutils.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xutils.c 
b/mercurial/thirdparty/xdiff/xutils.c
--- a/mercurial/thirdparty/xdiff/xutils.c
+++ b/mercurial/thirdparty/xdiff/xutils.c
@@ -303,12 +303,16 @@
 
 unsigned long xdl_hash_record(char const **data, char const *top, long flags) {
XXH32_hash_t h;
-   char const *ptr = *data;
+   char const *ptr;
 
if (flags & XDF_WHITESPACE_FLAGS)
return xdl_hash_record_with_whitespace(data, top, flags);
 
-   for (; ptr < top && *ptr != '\n'; ptr++) { }
+   ptr = memchr(*data, '\n', top - *data + 1);
+   if (!ptr) {
+   ptr = top;
+   }
+
h = XDIFF_XXH32(*data, ptr - *data + 1, 0);
*data = ptr < top ? ptr + 1: ptr;
return h;



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2626: xdiff: use xxhash in xdiff

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Diffing functions split inputs into blocks, typically lines. When
  inspecting each block, the diffing function needs to see if two blocks
  are identical. Because repeated memcmp() can be expensive, diff
  implementations will hash each block to an integer and then do a simple
  integer/register compare to check for block equivalence.
  
  It looks like xxhash was using djb2 for this hash. This hash was fine
  for its use. But the hash was operating on single bytes at a time
  and the output from a previous byte was needed before feeding in a
  new byte. So our CPU pipeline was very constricted.
  
  This commit changes the hash to xxhash. This hashing function takes
  a start address and a length. Internally, it operates on multiple
  bytes at a time. This requires fewer assembly instructions and makes
  the hash faster.
  
  It's worth noting we are using the 32-bit version of xxhash. The 64-bit
  version is faster on 64-bit hardware and we should ideally use it.
  But the 64-bit version returns an unsigned long long and xdiff wants
  the hash to be 32 bits. We could probably truncate the hash. Let's
  use the 32-bit hash for now.
  
  This change yields a minor perf win on the mozilla-central repo:
  
$ hg perfbdiff --alldata -c --count 10 --blocks --xdiff 40
! wall 0.845893 comb 0.84 user 0.84 sys 0.00 (best of 12)
! wall 0.796796 comb 0.79 user 0.79 sys 0.00 (best of 13)

$ hg perfbdiff -m --count 100 --blocks --xdiff 40
! wall 10.026769 comb 10.03 user 9.01 sys 1.02 (best of 3)
! wall  9.450092 comb  9.46 user 8.47 sys 0.99 (best of 3)
  
  Taken on its own, this isn't a significant improvement. But it does
  open the door to more efficient line searching...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2626

AFFECTED FILES
  mercurial/thirdparty/xdiff/xutils.c
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -871,9 +871,13 @@
   include_dirs=common_include_dirs,
   depends=common_depends),
 Extension('mercurial.cext.bdiff', ['mercurial/bdiff.c',
-   'mercurial/cext/bdiff.c'] + xdiff_srcs,
+   'mercurial/cext/bdiff.c',
+   'mercurial/thirdparty/xxhash/xxhash.c',
+   ] + xdiff_srcs,
   include_dirs=common_include_dirs,
-  depends=common_depends + ['mercurial/bdiff.h'] + xdiff_headers),
+  depends=common_depends + ['mercurial/bdiff.h',
+'mercurial/thirdparty/xxhash/xxhash.h',
+] + xdiff_headers),
 Extension('mercurial.cext.diffhelpers', ['mercurial/cext/diffhelpers.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
diff --git a/mercurial/thirdparty/xdiff/xutils.c 
b/mercurial/thirdparty/xdiff/xutils.c
--- a/mercurial/thirdparty/xdiff/xutils.c
+++ b/mercurial/thirdparty/xdiff/xutils.c
@@ -22,10 +22,13 @@
 
 #include 
 #include 
-#include "xinclude.h"
 
+/* Define XXHASH functions in a custom namespace */
+#define XXH_NAMESPACE XDIFF_
+#define XXH_PRIVATE_API
+#include "thirdparty/xxhash/xxhash.h"
 
-
+#include "xinclude.h"
 
 long xdl_bogosqrt(long n) {
long i;
@@ -299,19 +302,16 @@
 }
 
 unsigned long xdl_hash_record(char const **data, char const *top, long flags) {
-   unsigned long ha = 5381;
+   XXH32_hash_t h;
char const *ptr = *data;
 
if (flags & XDF_WHITESPACE_FLAGS)
return xdl_hash_record_with_whitespace(data, top, flags);
 
-   for (; ptr < top && *ptr != '\n'; ptr++) {
-   ha += (ha << 5);
-   ha ^= (unsigned long) *ptr;
-   }
+   for (; ptr < top && *ptr != '\n'; ptr++) { }
+   h = XDIFF_XXH32(*data, ptr - *data + 1, 0);
*data = ptr < top ? ptr + 1: ptr;
-
-   return ha;
+   return h;
 }
 
 unsigned int xdl_hashbits(unsigned int size) {



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@36658: 2 new changesets

[Mercurial Commits]

2 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/214f61abd865
changeset:   36657:214f61abd865
user:Yuya Nishihara 
date:Sat Mar 03 15:33:52 2018 -0500
summary: py3: back out c77c925987d7 to store bytes filename in IOError

https://www.mercurial-scm.org/repo/hg/rev/e1c6c1f9855d
changeset:   36658:e1c6c1f9855d
bookmark:@
tag: tip
user:Yuya Nishihara 
date:Sat Mar 03 15:41:12 2018 -0500
summary: py3: accept both unicode and byte strings as filename carried by 
IOError

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[yuja (Yuya Nishihara)]

yuja added inline comments.

INLINE COMMENTS

> templater.py:148
>  elif c == term:
>  yield ('end', None, pos + 1)
>  return

Perhaps this `pos + 1` is the reason why the error location is sometimes wrong.
I'll send a follow up.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

To: ryanmce, #hg-reviewers, durin42
Cc: yuja, pulkit, durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@36656: 49 new changesets

[Mercurial Commits]

49 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/1151c731686e
changeset:   36608:1151c731686e
user:Gregory Szorc 
date:Fri Mar 02 18:50:49 2018 -0500
summary: sshpeer: don't read from stderr when that behavior is disabled

https://www.mercurial-scm.org/repo/hg/rev/abc3b9801563
changeset:   36609:abc3b9801563
user:Gregory Szorc 
date:Fri Mar 02 09:47:37 2018 -0500
summary: wireproto: allow wire protocol commands to declare transport 
support

https://www.mercurial-scm.org/repo/hg/rev/af0d38f015bb
changeset:   36610:af0d38f015bb
user:Gregory Szorc 
date:Tue Feb 27 14:56:03 2018 -0800
summary: wireprotoserver: identify requests via version 2 of SSH protocol 
as such

https://www.mercurial-scm.org/repo/hg/rev/6906547c8476
changeset:   36611:6906547c8476
user:Gregory Szorc 
date:Tue Feb 27 15:06:10 2018 -0800
summary: wireproto: don't expose legacy commands to version 2 of wire 
protocol

https://www.mercurial-scm.org/repo/hg/rev/e89959970a08
changeset:   36612:e89959970a08
user:Gregory Szorc 
date:Tue Feb 27 15:23:04 2018 -0800
summary: wireproto: don't expose changegroupsubset capability if not 
available

https://www.mercurial-scm.org/repo/hg/rev/6e585bca962e
changeset:   36613:6e585bca962e
user:Gregory Szorc 
date:Tue Feb 27 16:24:02 2018 -0800
summary: wireproto: add transport specific capabilities in the transport

https://www.mercurial-scm.org/repo/hg/rev/1fa02265fae2
changeset:   36614:1fa02265fae2
user:Gregory Szorc 
date:Fri Mar 02 18:55:18 2018 -0500
summary: tests: add more tests around hook output and getbundle

https://www.mercurial-scm.org/repo/hg/rev/ed7705017749
changeset:   36615:ed7705017749
user:Gregory Szorc 
date:Tue Feb 27 16:31:44 2018 -0800
summary: wireproto: only expose "between" to version 1 of wire protocols

https://www.mercurial-scm.org/repo/hg/rev/a007db19dc4d
changeset:   36616:a007db19dc4d
user:Augie Fackler 
date:Fri Mar 02 18:47:27 2018 -0500
summary: tests: add missing b prefixes in test-atomictempfile.py

https://www.mercurial-scm.org/repo/hg/rev/5246f940a48e
changeset:   36617:5246f940a48e
user:Yuya Nishihara 
date:Sat Mar 03 05:50:45 2018 -0500
summary: py3: don't try to mangle C extension blob by code transformer

https://www.mercurial-scm.org/repo/hg/rev/9a639a33ad1f
changeset:   36618:9a639a33ad1f
user:Yuya Nishihara 
date:Sat Mar 03 05:58:41 2018 -0500
summary: py3: add PY23() macro to switch string literal depending on python 
version

https://www.mercurial-scm.org/repo/hg/rev/1f8c3fadbb8e
changeset:   36619:1f8c3fadbb8e
user:Yuya Nishihara 
date:Sat Mar 03 06:08:22 2018 -0500
summary: py3: bulk-replace bytes format specifier passed to Py_BuildValue()

https://www.mercurial-scm.org/repo/hg/rev/186c6df3a373
changeset:   36620:186c6df3a373
user:Yuya Nishihara 
date:Sat Mar 03 06:18:47 2018 -0500
summary: py3: bulk-replace 'const char*' format specifier passed to 
PyArg_ParseTuple*()

https://www.mercurial-scm.org/repo/hg/rev/4015b9248da0
changeset:   36621:4015b9248da0
user:Yuya Nishihara 
date:Sat Mar 03 06:41:52 2018 -0500
summary: cext: mark tuple_format as a constant

https://www.mercurial-scm.org/repo/hg/rev/0297f9c4caee
changeset:   36622:0297f9c4caee
user:Yuya Nishihara 
date:Sat Mar 03 06:44:47 2018 -0500
summary: py3: do not pass a float to dict_new_presized()

https://www.mercurial-scm.org/repo/hg/rev/a472a897c340
changeset:   36623:a472a897c340
user:Yuya Nishihara 
date:Sat Mar 03 06:57:02 2018 -0500
summary: cext: fix computephasesmapsets() not to return without setting an 
exception

https://www.mercurial-scm.org/repo/hg/rev/3118766266ae
changeset:   36624:3118766266ae
user:Yuya Nishihara 
date:Sat Mar 03 07:00:37 2018 -0500
summary: py3: do not pass a list of iterators to computephasesmapsets()

https://www.mercurial-scm.org/repo/hg/rev/edd3974bd500
changeset:   36625:edd3974bd500
user:Yuya Nishihara 
date:Sat Mar 03 07:24:25 2018 -0500
summary: py3: do not pass a memoryview to bdiff.bdiff()

https://www.mercurial-scm.org/repo/hg/rev/6754d0c5e1b5
changeset:   36626:6754d0c5e1b5
user:Yuya Nishihara 
date:Sat Mar 03 07:59:20 2018 -0500
summary: py3: make test-ancestors.py pass on Python 3 with C extensions

https://www.mercurial-scm.org/repo/hg/rev/418f9ea20c35
changeset:   36627:418f9ea20c35

[PATCH] py3: make gettext domain a system string

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1520122367 18000
#  Sat Mar 03 19:12:47 2018 -0500
# Node ID de317419d5babc9b88aa17664eed96e8d7f8dbe8
# Parent  1880fc2cbfc27c8b2fb0073f8aa9c0fa54937969
py3: make gettext domain a system string

diff --git a/mercurial/i18n.py b/mercurial/i18n.py
--- a/mercurial/i18n.py
+++ b/mercurial/i18n.py
@@ -51,7 +51,7 @@ if (pycompat.iswindows
 def setdatapath(datapath):
 datapath = pycompat.fsdecode(datapath)
 localedir = os.path.join(datapath, pycompat.sysstr('locale'))
-t = gettextmod.translation('hg', localedir, _languages, fallback=True)
+t = gettextmod.translation(r'hg', localedir, _languages, fallback=True)
 global _ugettext
 try:
 _ugettext = t.ugettext
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2623: dispatch: adding config items for overriding flag defaults

[rdamazio (Rodrigo Damazio Bovendorp)]

rdamazio added a comment.


  FYI this is a change I had previously sent to the list as 
60b3222e01f96f91ece7eda9681a89bf3bb930a6, and Yuya reviewed . I just had never 
followed up on it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2623

To: rdamazio, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] dispatch: don't clamp the range of the exit code twice

[Kevin Bullock]

# HG changeset patch
# User Kevin Bullock 
# Date 1520121770 18000
#  Sat Mar 03 19:02:50 2018 -0500
# Node ID 53daf5db6dada7eade4de6d4c328b78feb0425cf
# Parent  c1af0dc644d4c109efee03786aca819a8ac7775c
dispatch: don't clamp the range of the exit code twice

We already limit the range to (0, 255) in the call to sys.exit(). The
duplicated operation can't possibly be hurting us, but let's clean it up
to avoid confusion.

diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -85,7 +85,7 @@ def run():
 req = request(pycompat.sysargv[1:])
 err = None
 try:
-status = (dispatch(req) or 0) & 255
+status = (dispatch(req) or 0)
 except error.StdioError as e:
 err = e
 status = -1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5810] New: warning about overridden commands when both evolve and serveronly are enabled

[mercurial-bugs]

https://bz.mercurial-scm.org/show_bug.cgi?id=5810

Bug ID: 5810
   Summary: warning about overridden commands when both evolve and
serveronly are enabled
   Product: Mercurial
   Version: 4.4.2
  Hardware: PC
OS: Mac OS
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: evolution
  Assignee: bugzi...@mercurial-scm.org
  Reporter: kbullock+mercur...@ringworld.org
CC: lothiral...@gmail.com,
mercurial-devel@mercurial-scm.org,
pierre-yves.da...@ens-lyon.org

Since I enabled evolve in my $HOME/.hgrc on mercurial-scm.org, all my
interactions with hg-committed (which has the serveronly extension enabled in
the repo .hg/hgrc) show:

remote: extension 'evolve.serveronly' overrides commands: debugdepth
debugstablesort debugobsrelsethashtree debugobshashrange debugstablerange

It'd be nice if the extensions could work it out between themselves, so that
maybe serveronly doesn't get enabled if evolve is already loaded or something
similar.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2623: dispatch: adding config items for overriding flag defaults

[rdamazio (Rodrigo Damazio Bovendorp)]

rdamazio updated this revision to Diff 6523.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2623?vs=6521=6523

REVISION DETAIL
  https://phab.mercurial-scm.org/D2623

AFFECTED FILES
  mercurial/configitems.py
  mercurial/dispatch.py
  mercurial/ui.py
  tests/test-dispatch.t

CHANGE DETAILS

diff --git a/tests/test-dispatch.t b/tests/test-dispatch.t
--- a/tests/test-dispatch.t
+++ b/tests/test-dispatch.t
@@ -8,8 +8,10 @@
   $ hg -v log -v x
 
   $ echo a > a
+  $ echo b > b
   $ hg ci -Ama
   adding a
+  adding b
 
 Missing arg:
 
@@ -52,10 +54,10 @@
 Parsing of early options should stop at "--":
 
   $ hg cat -- --config=hooks.pre-cat=false
-  --config=hooks.pre-cat=false: no such file in rev cb9a9f314b8b
+  --config=hooks.pre-cat=false: no such file in rev 0cd96de13884
   [1]
   $ hg cat -- --debugger
-  --debugger: no such file in rev cb9a9f314b8b
+  --debugger: no such file in rev 0cd96de13884
   [1]
 
 Unparsable form of early options:
@@ -155,31 +157,75 @@
   abort: pre-log hook exited with status 1
   [255]
   $ HGPLAIN=+strictflags hg --cwd .. -q -Ra log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
   $ HGPLAIN=+strictflags hg --cwd .. -q --repository a log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
   $ HGPLAIN=+strictflags hg --cwd .. -q --repo a log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
 
 For compatibility reasons, HGPLAIN=+strictflags is not enabled by plain 
HGPLAIN:
 
   $ HGPLAIN= hg log --config='hooks.pre-log=false' -b default
   abort: pre-log hook exited with status 1
   [255]
   $ HGPLAINEXCEPT= hg log --cwd .. -q -Ra -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
 
 [defaults]
 
   $ hg cat a
   a
+  $ cp $HGRCPATH hgrc.bak
   $ cat >> $HGRCPATH < [defaults]
   > cat = -r null
   > EOF
   $ hg cat a
   a: no such file in rev 
   [1]
+  $ cp -f hgrc.bak $HGRCPATH
+
+new-style [commands] defaults and overrides
+
+  $ hg cat a
+  a
+  $ cat >> $HGRCPATH < [commands]
+  > cat.default.rev = null
+  > EOF
+  $ hg cat a
+  a: no such file in rev 
+  [1]
+
+  $ mv -f hgrc.bak $HGRCPATH
+  $ echo foo >> a
+  $ hg rm b
+  $ echo bar > c
+  $ hg add c
+  $ hg status
+  M a
+  A c
+  R b
+  ? bad.py
+  ? bad.pyc
+  $ cat >> $HGRCPATH < [commands]
+  > status.default.removed = 1
+  > EOF
+  $ hg status
+  R b
+  $ hg status --modified
+  M a
+  R b
+  $ hg status --modified --no-removed
+  M a
+  $ hg status --no-removed
+  M a
+  A c
+  R b
+  ? bad.py
+  ? bad.pyc
+  $ hg revert a b c
 
   $ cd "$TESTTMP"
 
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -379,10 +379,15 @@
 del cfg['defaults'][k]
 for k, v in cfg.items('commands'):
 del cfg['commands'][k]
-# Don't remove aliases from the configuration if in the exceptionlist
+# Don't remove specific sections from the configuration if in the
+# exception list.
 if self.plain('alias'):
 for k, v in cfg.items('alias'):
 del cfg['alias'][k]
+if self.plain('commanddefaults'):
+for k, v in cfg.items('commands'):
+if '.default.' in k:
+del cfg['commands'][k]
 if self.plain('revsetalias'):
 for k, v in cfg.items('revsetalias'):
 del cfg['revsetalias'][k]
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -611,6 +611,21 @@
 args = pycompat.maplist(
 util.expandpath, pycompat.shlexsplit(defaults)) + args
 c = list(entry[1])
+
+# Apply new-style defaults from config file by actually changing the
+# option defaults. We still let old-style defaults trump these (since
+# those are added to the command line).
+for idx, opt in enumerate(c):
+optname = opt[1]
+olddefault = opt[2]
+defaulttype = type(olddefault)
+cfgitem = "%s.default.%s" % (cmd, optname)
+# parse the new default as the same type as the original.
+newdefault = ui.configtyped("commands", cfgitem, defaulttype, 
olddefault)
+if olddefault != newdefault:
+# override the default in the flag declaration.
+c[idx] = (opt[0], opt[1], newdefault, opt[3])
+
 else:
 cmd = None
 c = []
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -184,6 +184,10 @@
 coreconfigitem('color', 'pagermode',
 default=dynamicdefault,
 )
+coreconfigitem('commands', '.*\.default\..*',
+generic=True,
+default=dynamicdefault,
+)
 coreconfigitem('commands', 'show.aliasprefix',
 default=list,
 )



To: rdamazio, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

D2624: perf: teach perfbdiff to call blocks() and to use xdiff

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2624

AFFECTED FILES
  contrib/perf.py

CHANGE DETAILS

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -939,11 +939,16 @@
 timer(d)
 fm.end()
 
-def _bdiffworker(q, ready, done):
+def _bdiffworker(q, blocks, xdiff, ready, done):
 while not done.is_set():
 pair = q.get()
 while pair is not None:
-mdiff.textdiff(*pair)
+if xdiff:
+mdiff.bdiff.xdiffblocks(*pair)
+elif blocks:
+mdiff.bdiff.blocks(*pair)
+else:
+mdiff.textdiff(*pair)
 q.task_done()
 pair = q.get()
 q.task_done() # for the None one
@@ -954,6 +959,8 @@
 ('', 'count', 1, 'number of revisions to test (when using --startrev)'),
 ('', 'alldata', False, 'test bdiffs for all associated revisions'),
 ('', 'threads', 0, 'number of thread to use (disable with 0)'),
+('', 'blocks', False, 'test computing diffs into blocks'),
+('', 'xdiff', False, 'use xdiff algorithm'),
 ],
 
 '-c|-m|FILE REV')
@@ -969,14 +976,21 @@
 measure bdiffs for all changes related to that changeset (manifest
 and filelogs).
 """
+opts = pycompat.byteskwargs(opts)
+
+if opts['xdiff'] and not opts['blocks']:
+raise error.CommandError('perfbdiff', '--xdiff requires --blocks')
+
 if opts['alldata']:
 opts['changelog'] = True
 
 if opts.get('changelog') or opts.get('manifest'):
 file_, rev = None, file_
 elif rev is None:
 raise error.CommandError('perfbdiff', 'invalid arguments')
 
+blocks = opts['blocks']
+xdiff = opts['xdiff']
 textpairs = []
 
 r = cmdutil.openrevlog(repo, 'perfbdiff', file_, opts)
@@ -1007,15 +1021,21 @@
 if not withthreads:
 def d():
 for pair in textpairs:
-mdiff.textdiff(*pair)
+if xdiff:
+mdiff.bdiff.xdiffblocks(*pair)
+elif blocks:
+mdiff.bdiff.blocks(*pair)
+else:
+mdiff.textdiff(*pair)
 else:
 q = util.queue()
 for i in xrange(threads):
 q.put(None)
 ready = threading.Condition()
 done = threading.Event()
 for i in xrange(threads):
-threading.Thread(target=_bdiffworker, args=(q, ready, 
done)).start()
+threading.Thread(target=_bdiffworker,
+ args=(q, blocks, xdiff, ready, done)).start()
 q.join()
 def d():
 for pair in textpairs:



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2623: dispatch: adding config items for overriding flag defaults

[rdamazio (Rodrigo Damazio Bovendorp)]

rdamazio created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This introduces the new defaults format "commandname.default.optionname" which
  directly overrides the default of the option, instead of prepending the
  command-line option. This is meant to replace the [defaults] section which is
  already deprecated in a manner that's easier and safer to use than creating
  aliases.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2623

AFFECTED FILES
  mercurial/configitems.py
  mercurial/dispatch.py
  mercurial/ui.py
  tests/test-dispatch.t

CHANGE DETAILS

diff --git a/tests/test-dispatch.t b/tests/test-dispatch.t
--- a/tests/test-dispatch.t
+++ b/tests/test-dispatch.t
@@ -8,8 +8,10 @@
   $ hg -v log -v x
 
   $ echo a > a
+  $ echo b > b
   $ hg ci -Ama
   adding a
+  adding b
 
 Missing arg:
 
@@ -52,10 +54,10 @@
 Parsing of early options should stop at "--":
 
   $ hg cat -- --config=hooks.pre-cat=false
-  --config=hooks.pre-cat=false: no such file in rev cb9a9f314b8b
+  --config=hooks.pre-cat=false: no such file in rev 0cd96de13884
   [1]
   $ hg cat -- --debugger
-  --debugger: no such file in rev cb9a9f314b8b
+  --debugger: no such file in rev 0cd96de13884
   [1]
 
 Unparsable form of early options:
@@ -155,31 +157,75 @@
   abort: pre-log hook exited with status 1
   [255]
   $ HGPLAIN=+strictflags hg --cwd .. -q -Ra log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
   $ HGPLAIN=+strictflags hg --cwd .. -q --repository a log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
   $ HGPLAIN=+strictflags hg --cwd .. -q --repo a log -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
 
 For compatibility reasons, HGPLAIN=+strictflags is not enabled by plain 
HGPLAIN:
 
   $ HGPLAIN= hg log --config='hooks.pre-log=false' -b default
   abort: pre-log hook exited with status 1
   [255]
   $ HGPLAINEXCEPT= hg log --cwd .. -q -Ra -b default
-  0:cb9a9f314b8b
+  0:0cd96de13884
 
 [defaults]
 
   $ hg cat a
   a
+  $ cp $HGRCPATH hgrc.bak
   $ cat >> $HGRCPATH < [defaults]
   > cat = -r null
   > EOF
   $ hg cat a
   a: no such file in rev 
   [1]
+  $ cp -f hgrc.bak $HGRCPATH
+
+new-style [commands] defaults and overrides
+
+  $ hg cat a
+  a
+  $ cat >> $HGRCPATH < [commands]
+  > cat.default.rev = null
+  > EOF
+  $ hg cat a
+  a: no such file in rev 
+  [1]
+
+  $ mv -f hgrc.bak $HGRCPATH
+  $ echo foo >> a
+  $ hg rm b
+  $ echo bar > c
+  $ hg add c
+  $ hg status
+  M a
+  A c
+  R b
+  ? bad.py
+  ? bad.pyc
+  $ cat >> $HGRCPATH < [commands]
+  > status.default.removed = 1
+  > EOF
+  $ hg status
+  R b
+  $ hg status --modified
+  M a
+  R b
+  $ hg status --modified --no-removed
+  M a
+  $ hg status --no-removed
+  M a
+  A c
+  R b
+  ? bad.py
+  ? bad.pyc
+  $ hg revert a b c
 
   $ cd "$TESTTMP"
 
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -379,10 +379,15 @@
 del cfg['defaults'][k]
 for k, v in cfg.items('commands'):
 del cfg['commands'][k]
-# Don't remove aliases from the configuration if in the exceptionlist
+# Don't remove specific sections from the configuration if in the
+# exception list.
 if self.plain('alias'):
 for k, v in cfg.items('alias'):
 del cfg['alias'][k]
+if self.plain('commanddefaults'):
+for k, v in cfg.items('commands'):
+if '.default.' in k:
+   del cfg['commands'][k]
 if self.plain('revsetalias'):
 for k, v in cfg.items('revsetalias'):
 del cfg['revsetalias'][k]
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -611,6 +611,21 @@
 args = pycompat.maplist(
 util.expandpath, pycompat.shlexsplit(defaults)) + args
 c = list(entry[1])
+
+# Apply new-style defaults from config file by actually changing the
+# option defaults. We still let old-style defaults trump these (since
+# those are added to the command line).
+for idx, opt in enumerate(c):
+optname = opt[1]
+olddefault = opt[2]
+defaulttype = type(olddefault)
+cfgitem = "%s.default.%s" % (cmd, optname)
+# parse the new default as the same type as the original.
+newdefault = ui.configtyped("commands", cfgitem, defaulttype, 
olddefault)
+if olddefault != newdefault:
+# override the default in the flag declaration.
+c[idx] = (opt[0], opt[1], newdefault, opt[3])
+
 else:
 cmd = None
 c = []
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -184,6 +184,10 @@
 coreconfigitem('color', 'pagermode',
 default=dynamicdefault,
 )

D2622: ui: adding a generic method to read config items with an arbitrary type

[rdamazio (Rodrigo Damazio Bovendorp)]

rdamazio created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This can be used for reading many config options of various types where the
  type of each is known in some other structure (e.g. options matching command
  flags).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2622

AFFECTED FILES
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -545,6 +545,16 @@
 
 return main, sub
 
+def configtyped(self, section, name, itemtype, default=_unset, 
untrusted=False):
+"""Get a config item as the given type."""
+if itemtype is type(False) or itemtype is type(None):
+return self.configbool(section, name, default, untrusted)
+if itemtype is type(1):
+return self.configint(section, name, default, untrusted)
+if itemtype is type([]):
+return self.configlist(section, name, default, untrusted)
+return self.config(section, name, default, untrusted)
+
 def configpath(self, section, name, default=_unset, untrusted=False):
 'get a path config item, expanded relative to repo root or config file'
 v = self.config(section, name, default, untrusted)



To: rdamazio, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2621: tests: port test-log to Python 3

[pulkit (Pulkit Goyal)]

pulkit requested changes to this revision.
pulkit added a comment.
This revision now requires changes to proceed.


  Sorry, but this does not passes for me on Python 3.5 Do we consider 
supporting that?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2621

To: durin42, pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2068: revlog: do not use delta for lfs revisions

[quark (Jun Wu)]

quark added inline comments.

INLINE COMMENTS

> ryanmce wrote in revlog.py:2110
> Then why is this `REVIDX_KNOWN_FLAGS` and not `REVIDX_RAWTEXT_CHANGING_FLAGS`?
> 
> Given the number of questions here from me and others, I think the comment 
> could use extension at least so future readers understand why this is the way 
> it is.

Should be `REVIDX_RAWTEXT_CHANGING_FLAGS`. I'll make the change.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2068

To: quark, indygreg, #hg-reviewers, ryanmce
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2603: mdiff: add a config option to use xdiff algorithm

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9b51ac874d90: mdiff: add a config option to use xdiff 
algorithm (authored by quark, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2603?vs=6460=6517

REVISION DETAIL
  https://phab.mercurial-scm.org/D2603

AFFECTED FILES
  mercurial/configitems.py
  mercurial/mdiff.py
  mercurial/patch.py

CHANGE DETAILS

diff --git a/mercurial/patch.py b/mercurial/patch.py
--- a/mercurial/patch.py
+++ b/mercurial/patch.py
@@ -2256,6 +2256,7 @@
 'context': get('unified', getter=ui.config),
 }
 buildopts['worddiff'] = ui.configbool('experimental', 'worddiff')
+buildopts['xdiff'] = ui.configbool('experimental', 'xdiff')
 
 if git:
 buildopts['git'] = get('git')
diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py
--- a/mercurial/mdiff.py
+++ b/mercurial/mdiff.py
@@ -63,6 +63,7 @@
 'upgrade': False,
 'showsimilarity': False,
 'worddiff': False,
+'xdiff': False,
 }
 
 def __init__(self, **opts):
@@ -188,6 +189,13 @@
 raise error.Abort(_('line range exceeds file size'))
 return filteredblocks, (lba, uba)
 
+def chooseblocksfunc(opts=None):
+if (opts is None or not opts.xdiff
+or not util.safehasattr(bdiff, 'xdiffblocks')):
+return bdiff.blocks
+else:
+return bdiff.xdiffblocks
+
 def allblocks(text1, text2, opts=None, lines1=None, lines2=None):
 """Return (block, type) tuples, where block is an mdiff.blocks
 line entry. type is '=' for blocks matching exactly one another
@@ -201,7 +209,7 @@
 if opts.ignorews or opts.ignorewsamount or opts.ignorewseol:
 text1 = wsclean(opts, text1, False)
 text2 = wsclean(opts, text2, False)
-diff = bdiff.blocks(text1, text2)
+diff = chooseblocksfunc(opts)(text1, text2)
 for i, s1 in enumerate(diff):
 # The first match is special.
 # we've either found a match starting at line 0 or a match later
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -577,6 +577,9 @@
 coreconfigitem('experimental', 'sshpeer.advertise-v2',
 default=False,
 )
+coreconfigitem('experimental', 'xdiff',
+default=False,
+)
 coreconfigitem('extensions', '.*',
 default=None,
 generic=True,



To: quark, #hg-reviewers, indygreg, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2604: tests: add tests about diff quality

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc1af0dc644d4: tests: add tests about diff quality (authored 
by quark, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D2604?vs=6481=6519#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2604?vs=6481=6519

REVISION DETAIL
  https://phab.mercurial-scm.org/D2604

AFFECTED FILES
  tests/hghave.py
  tests/test-diff-antipatience.t
  tests/test-diff-indent-heuristic.t

CHANGE DETAILS

diff --git a/tests/test-diff-indent-heuristic.t 
b/tests/test-diff-indent-heuristic.t
new file mode 100644
--- /dev/null
+++ b/tests/test-diff-indent-heuristic.t
@@ -0,0 +1,339 @@
+#testcases bdiff xdiff
+
+#if xdiff
+#require xdiff
+  $ cat >> $HGRCPATH < [experimental]
+  > xdiff = true
+  > EOF
+#endif
+
+  $ hg init
+
+  $ cat > a.c <<'EOF'
+  > /*
+  >  * This function returns 1.
+  >  */
+  > int f() {
+  >   return 1;
+  > }
+  > /*
+  >  * This function returns 2.
+  >  */
+  > int g() {
+  >   return 2;
+  > }
+  > /*
+  >  * This function returns 3.
+  >  */
+  > int h() {
+  >   return 3;
+  > }
+  > EOF
+
+  $ cat > b.c <<'EOF'
+  > if (x) {
+  >do_something();
+  > }
+  > 
+  > if (y) {
+  >do_something_else();
+  > }
+  > EOF
+
+  $ cat > c.rb <<'EOF'
+  > #!ruby
+  > ["foo", "bar", "baz"].map do |i|
+  >   i.upcase
+  > end
+  > EOF
+
+  $ cat > d.py <<'EOF'
+  > try:
+  > import foo
+  > except ImportError:
+  > pass
+  > try:
+  > import bar
+  > except ImportError:
+  > pass
+  > EOF
+
+The below two files are taken from git: t/t4061-diff-indent.sh
+
+  $ cat > spaces.txt <<'EOF'
+  > 1
+  > 2
+  > a
+  > 
+  > b
+  > 3
+  > 4
+  > EOF
+
+  $ cat > functions.c <<'EOF'
+  > 1
+  > 2
+  > /* function */
+  > foo() {
+  > foo
+  > }
+  > 
+  > 3
+  > 4
+  > EOF
+
+  $ hg commit -m 1 -A . -q
+
+  $ cat > a.c <<'EOF'
+  > /*
+  >  * This function returns 1.
+  >  */
+  > int f() {
+  >   return 1;
+  > }
+  > /*
+  >  * This function returns 3.
+  >  */
+  > int h() {
+  >   return 3;
+  > }
+  > EOF
+
+  $ cat > b.c <<'EOF'
+  > if (x) {
+  >do_something();
+  > }
+  > 
+  > if (y) {
+  >do_another_thing();
+  > }
+  > 
+  > if (y) {
+  >do_something_else();
+  > }
+  > EOF
+
+  $ cat > c.rb <<'EOF'
+  > #!ruby
+  > ["foo", "bar", "baz"].map do |i|
+  >   i
+  > end
+  > ["foo", "bar", "baz"].map do |i|
+  >   i.upcase
+  > end
+  > EOF
+
+  $ cat > d.py <<'EOF'
+  > try:
+  > import foo
+  > except ImportError:
+  > pass
+  > try:
+  > import baz
+  > except ImportError:
+  > pass
+  > try:
+  > import bar
+  > except ImportError:
+  > pass
+  > EOF
+
+  $ cat > spaces.txt <<'EOF'
+  > 1
+  > 2
+  > a
+  > 
+  > b
+  > a
+  > 
+  > b
+  > 3
+  > 4
+  > EOF
+
+  $ cat > functions.c <<'EOF'
+  > 1
+  > 2
+  > /* function */
+  > bar() {
+  > foo
+  > }
+  > 
+  > /* function */
+  > foo() {
+  > foo
+  > }
+  > 
+  > 3
+  > 4
+  > EOF
+
+#if xdiff
+  $ hg diff --git
+  diff --git a/a.c b/a.c
+  --- a/a.c
+  +++ b/a.c
+  @@ -4,12 +4,6 @@
+   int f() {
+ return 1;
+   }
+  -/*
+  - * This function returns 2.
+  - */
+  -int g() {
+  -  return 2;
+  -}
+   /*
+* This function returns 3.
+*/
+  diff --git a/b.c b/b.c
+  --- a/b.c
+  +++ b/b.c
+  @@ -2,6 +2,10 @@
+  do_something();
+   }
+   
+  +if (y) {
+  +   do_another_thing();
+  +}
+  +
+   if (y) {
+  do_something_else();
+   }
+  diff --git a/c.rb b/c.rb
+  --- a/c.rb
+  +++ b/c.rb
+  @@ -1,4 +1,7 @@
+   #!ruby
+  +["foo", "bar", "baz"].map do |i|
+  +  i
+  +end
+   ["foo", "bar", "baz"].map do |i|
+ i.upcase
+   end
+  diff --git a/d.py b/d.py
+  --- a/d.py
+  +++ b/d.py
+  @@ -2,6 +2,10 @@
+   import foo
+   except ImportError:
+   pass
+  +try:
+  +import baz
+  +except ImportError:
+  +pass
+   try:
+   import bar
+   except ImportError:
+  diff --git a/functions.c b/functions.c
+  --- a/functions.c
+  +++ b/functions.c
+  @@ -1,5 +1,10 @@
+   1
+   2
+  +/* function */
+  +bar() {
+  +foo
+  +}
+  +
+   /* function */
+   foo() {
+   foo
+  diff --git a/spaces.txt b/spaces.txt
+  --- a/spaces.txt
+  +++ b/spaces.txt
+  @@ -2,6 +2,9 @@
+   2
+   a
+   
+  +b
+  +a
+  +
+   b
+   3
+   4
+#else
+  $ hg diff --git
+  diff --git a/a.c b/a.c
+  --- a/a.c
+  +++ b/a.c
+  @@ -5,12 +5,6 @@
+ return 1;
+   }
+   /*
+  - * This function returns 2.
+  - */
+  -int g() {
+  -  return 2;
+  -}
+  -/*
+* This function returns 3.
+*/
+   int h() {
+  diff --git a/b.c b/b.c
+  --- a/b.c
+  +++ b/b.c
+  @@ -3,5 +3,9 @@
+   }
+   
+   if (y) {
+  +   do_another_thing();
+  +}
+  +
+  +if (y) {
+  do_something_else();
+   }
+  diff --git a/c.rb b/c.rb
+  --- a/c.rb
+  +++ b/c.rb
+  @@ -1,4 +1,7 @@
+   #!ruby
+   ["foo", "bar", "baz"].map do |i|
+  +  i
+  +end
+  +["foo", "bar", "baz"].map do |i|
+ i.upcase
+   end
+  diff --git a/d.py b/d.py
+  --- a/d.py
+  +++ b/d.py
+  @@ 

D2605: run-tests: allow #require inside #if

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG4e82b6f33e10: run-tests: allow #require inside #if 
(authored by quark, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2605?vs=6478=6518

REVISION DETAIL
  https://phab.mercurial-scm.org/D2605

AFFECTED FILES
  tests/run-tests.py
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -921,16 +921,24 @@
 
   $ cat > test-skip.t <   $ echo xyzzy
+  > #if true
   > #require false
+  > #end
+  > EOF
+  $ cat > test-noskip.t < #if false
+  > #require false
+  > #endif
   > EOF
   $ rt --nodiff
-  !.s
+  !.s.
   Skipped test-skip.t: missing feature: nail clipper
   Failed test-failure.t: output changed
-  # Ran 2 tests, 1 skipped, 1 failed.
+  # Ran 3 tests, 1 skipped, 1 failed.
   python hash seed: * (glob)
   [1]
 
+  $ rm test-noskip.t
   $ rt --keyword xyzzy
   .s
   Skipped test-skip.t: missing feature: nail clipper
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -1373,10 +1373,11 @@
 lsplit = l.split()
 if len(lsplit) < 2 or lsplit[0] != b'#require':
 after.setdefault(pos, []).append('  !!! invalid 
#require\n')
-haveresult, message = self._hghave(lsplit[1:])
-if not haveresult:
-script = [b'echo "%s"\nexit 80\n' % message]
-break
+if not skipping:
+haveresult, message = self._hghave(lsplit[1:])
+if not haveresult:
+script = [b'echo "%s"\nexit 80\n' % message]
+break
 after.setdefault(pos, []).append(l)
 elif l.startswith(b'#if'):
 lsplit = l.split()



To: quark, #hg-reviewers, indygreg, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2573: xdiff: remove patience and histogram diff algorithms

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3ee9ca23dad0: xdiff: remove patience and histogram diff 
algorithms (authored by quark, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2573?vs=6473=6512

REVISION DETAIL
  https://phab.mercurial-scm.org/D2573

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiff.h
  mercurial/thirdparty/xdiff/xdiffi.c
  mercurial/thirdparty/xdiff/xhistogram.c
  mercurial/thirdparty/xdiff/xpatience.c
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -27,7 +27,6 @@
 #define XDL_MAX_EQLIMIT 1024
 #define XDL_SIMSCAN_WINDOW 100
 #define XDL_GUESS_NLINES1 256
-#define XDL_GUESS_NLINES2 20
 
 
 typedef struct s_xdlclass {
@@ -181,9 +180,7 @@
if (!(recs = (xrecord_t **) xdl_malloc(narec * sizeof(xrecord_t *
goto abort;
 
-   if (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF)
-   hbits = hsize = 0;
-   else {
+   {
hbits = xdl_hashbits((unsigned int) narec);
hsize = 1 << hbits;
if (!(rhash = (xrecord_t **) xdl_malloc(hsize * 
sizeof(xrecord_t *
@@ -209,8 +206,7 @@
crec->ha = hav;
recs[nrec++] = crec;
 
-   if ((XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&
-   xdl_classify_record(pass, cf, rhash, hbits, crec) < 
0)
+   if (xdl_classify_record(pass, cf, rhash, hbits, crec) < 
0)
goto abort;
}
}
@@ -266,21 +262,12 @@
 
memset(, 0, sizeof(cf));
 
-   /*
-* For histogram diff, we can afford a smaller sample size and
-* thus a poorer estimate of the number of lines, as the hash
-* table (rhash) won't be filled up/grown. The number of lines
-* (nrecs) will be updated correctly anyway by
-* xdl_prepare_ctx().
-*/
-   sample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF
- ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);
+   sample = XDL_GUESS_NLINES1;
 
enl1 = xdl_guess_lines(mf1, sample) + 1;
enl2 = xdl_guess_lines(mf2, sample) + 1;
 
-   if (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF &&
-   xdl_init_classifier(, enl1 + enl2 + 1, xpp->flags) < 0)
+   if (xdl_init_classifier(, enl1 + enl2 + 1, xpp->flags) < 0)
return -1;
 
if (xdl_prepare_ctx(1, mf1, enl1, xpp, , >xdf1) < 0) {
@@ -295,18 +282,14 @@
return -1;
}
 
-   if ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&
-   (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&
-   xdl_optimize_ctxs(, >xdf1, >xdf2) < 0) {
-
+   if (xdl_optimize_ctxs(, >xdf1, >xdf2) < 0) {
xdl_free_ctx(>xdf2);
xdl_free_ctx(>xdf1);
xdl_free_classifier();
return -1;
}
 
-   if (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)
-   xdl_free_classifier();
+   xdl_free_classifier();
 
return 0;
 }
diff --git a/mercurial/thirdparty/xdiff/xpatience.c 
b/mercurial/thirdparty/xdiff/xpatience.c
deleted file mode 100644
--- a/mercurial/thirdparty/xdiff/xpatience.c
+++ /dev/null
@@ -1,390 +0,0 @@
-/*
- *  LibXDiff by Davide Libenzi ( File Differential Library )
- *  Copyright (C) 2003-2016 Davide Libenzi, Johannes E. Schindelin
- *
- *  This library is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU Lesser General Public
- *  License as published by the Free Software Foundation; either
- *  version 2.1 of the License, or (at your option) any later version.
- *
- *  This library is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- *  Lesser General Public License for more details.
- *
- *  You should have received a copy of the GNU Lesser General Public
- *  License along with this library; if not, see
- *  .
- *
- *  Davide Libenzi 
- *
- */
-#include "xinclude.h"
-#include "xtypes.h"
-#include "xdiff.h"
-
-/*
- * The basic idea of patience diff is to find lines that are unique in
- * both files.  These are intuitively the ones that we want to see as
- * common lines.
- *
- * The maximal ordered sequence of such line pairs (where ordered means
- * that the order in the sequence agrees with the order of the lines in
- * both files) naturally defines an initial set of common lines.
- *
- * Now, the algorithm tries to extend the set of common lines by growing
- * the line ranges where the files have 

D2601: xdiff: reduce indent heuristic overhead

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9f2597a6620e: xdiff: reduce indent heuristic overhead 
(authored by quark, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D2601?vs=6475=6515#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2601?vs=6475=6515

REVISION DETAIL
  https://phab.mercurial-scm.org/D2601

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiffi.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xdiffi.c 
b/mercurial/thirdparty/xdiff/xdiffi.c
--- a/mercurial/thirdparty/xdiff/xdiffi.c
+++ b/mercurial/thirdparty/xdiff/xdiffi.c
@@ -802,6 +802,14 @@
 }
 
 /*
+ * For indentation heuristic, skip searching for better slide position after
+ * checking MAX_BORING lines without finding an improvement. This defends the
+ * indentation heuristic logic against pathological cases. The value is not
+ * picked scientifically but should be good enough.
+ */
+#define MAX_BORING 100
+
+/*
  * Move back and forward change groups for a consistent and pretty diff output.
  * This also helps in finding joinable change groups and reducing the diff
  * size.
@@ -897,19 +905,43 @@
long shift, best_shift = -1;
struct split_score best_score;
 
-   for (shift = earliest_end; shift <= g.end; shift++) {
+   /*
+* This is O(N * MAX_BLANKS) (N = shift-able lines).
+* Even with MAX_BLANKS bounded to a small value, a
+* large N could still make this loop take several
+* times longer than the main diff algorithm. The
+* "boring" value is to help cut down N to something
+* like (MAX_BORING + groupsize).
+*
+* Scan from bottom to top. So we can exit the loop
+* without compromising the assumption "for a same best
+* score, pick the bottommost shift".
+*/
+   int boring = 0;
+   for (shift = g.end; shift >= earliest_end; shift--) {
struct split_measurement m;
struct split_score score = {0, 0};
+   int cmp;
 
measure_split(xdf, shift, );
score_add_split(, );
measure_split(xdf, shift - groupsize, );
score_add_split(, );
-   if (best_shift == -1 ||
-   score_cmp(, _score) <= 0) {
+
+   if (best_shift == -1) {
+   cmp = -1;
+   } else {
+   cmp = score_cmp(, _score);
+   }
+   if (cmp < 0) {
+   boring = 0;
best_score.effective_indent = 
score.effective_indent;
best_score.penalty = score.penalty;
best_shift = shift;
+   } else {
+   boring += 1;
+   if (boring >= MAX_BORING)
+   break;
}
}
 



To: quark, #hg-reviewers, indygreg, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2602: bdiff: add a xdiffblocks method

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd74589c85f5e: bdiff: add a xdiffblocks method (authored by 
quark, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2602?vs=6480=6516

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

AFFECTED FILES
  mercurial/cext/bdiff.c
  mercurial/policy.py
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -847,14 +847,33 @@
 if sys.platform == 'darwin':
 osutil_ldflags += ['-framework', 'ApplicationServices']
 
+xdiff_srcs = [
+'mercurial/thirdparty/xdiff/xdiffi.c',
+'mercurial/thirdparty/xdiff/xemit.c',
+'mercurial/thirdparty/xdiff/xmerge.c',
+'mercurial/thirdparty/xdiff/xprepare.c',
+'mercurial/thirdparty/xdiff/xutils.c',
+]
+
+xdiff_headers = [
+'mercurial/thirdparty/xdiff/xdiff.h',
+'mercurial/thirdparty/xdiff/xdiffi.h',
+'mercurial/thirdparty/xdiff/xemit.h',
+'mercurial/thirdparty/xdiff/xinclude.h',
+'mercurial/thirdparty/xdiff/xmacros.h',
+'mercurial/thirdparty/xdiff/xprepare.h',
+'mercurial/thirdparty/xdiff/xtypes.h',
+'mercurial/thirdparty/xdiff/xutils.h',
+]
+
 extmodules = [
 Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
 Extension('mercurial.cext.bdiff', ['mercurial/bdiff.c',
-   'mercurial/cext/bdiff.c'],
+   'mercurial/cext/bdiff.c'] + xdiff_srcs,
   include_dirs=common_include_dirs,
-  depends=common_depends + ['mercurial/bdiff.h']),
+  depends=common_depends + ['mercurial/bdiff.h'] + xdiff_headers),
 Extension('mercurial.cext.diffhelpers', ['mercurial/cext/diffhelpers.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
diff --git a/mercurial/policy.py b/mercurial/policy.py
--- a/mercurial/policy.py
+++ b/mercurial/policy.py
@@ -66,7 +66,7 @@
 # keep in sync with "version" in C modules
 _cextversions = {
 (r'cext', r'base85'): 1,
-(r'cext', r'bdiff'): 2,
+(r'cext', r'bdiff'): 3,
 (r'cext', r'diffhelpers'): 1,
 (r'cext', r'mpatch'): 1,
 (r'cext', r'osutil'): 3,
diff --git a/mercurial/cext/bdiff.c b/mercurial/cext/bdiff.c
--- a/mercurial/cext/bdiff.c
+++ b/mercurial/cext/bdiff.c
@@ -17,6 +17,7 @@
 
 #include "bdiff.h"
 #include "bitmanipulation.h"
+#include "thirdparty/xdiff/xdiff.h"
 #include "util.h"
 
 static PyObject *blocks(PyObject *self, PyObject *args)
@@ -256,18 +257,78 @@
return NULL;
 }
 
+static int hunk_consumer(long a1, long a2, long b1, long b2, void *priv)
+{
+   PyObject *rl = (PyObject *)priv;
+   PyObject *m = Py_BuildValue("", a1, a2, b1, b2);
+   if (!m)
+   return -1;
+   if (PyList_Append(rl, m) != 0) {
+   Py_DECREF(m);
+   return -1;
+   }
+   return 0;
+}
+
+static PyObject *xdiffblocks(PyObject *self, PyObject *args)
+{
+   Py_ssize_t la, lb;
+   mmfile_t a, b;
+   PyObject *rl;
+
+   xpparam_t xpp = {
+   XDF_INDENT_HEURISTIC, /* flags */
+   NULL, /* anchors */
+   0,/* anchors_nr */
+   };
+   xdemitconf_t xecfg = {
+   0,  /* ctxlen */
+   0,  /* interhunkctxlen */
+   XDL_EMIT_BDIFFHUNK, /* flags */
+   NULL,   /* find_func */
+   NULL,   /* find_func_priv */
+   hunk_consumer,  /* hunk_consume_func */
+   };
+   xdemitcb_t ecb = {
+   NULL, /* priv */
+   NULL, /* outf */
+   };
+
+   if (!PyArg_ParseTuple(args, PY23("s#s#", "y#y#"), , , ,
+ ))
+   return NULL;
+
+   a.size = la;
+   b.size = lb;
+
+   rl = PyList_New(0);
+   if (!rl)
+   return PyErr_NoMemory();
+
+   ecb.priv = rl;
+
+   if (xdl_diff(, , , , ) != 0) {
+   Py_DECREF(rl);
+   return PyErr_NoMemory();
+   }
+
+   return rl;
+}
+
 static char mdiff_doc[] = "Efficient binary diff.";
 
 static PyMethodDef methods[] = {
 {"bdiff", bdiff, METH_VARARGS, "calculate a binary diff\n"},
 {"blocks", blocks, METH_VARARGS, "find a list of matching lines\n"},
 {"fixws", fixws, METH_VARARGS, "normalize diff whitespaces\n"},
 {"splitnewlines", splitnewlines, METH_VARARGS,
  "like str.splitlines, but only split on newlines\n"},
+{"xdiffblocks", xdiffblocks, METH_VARARGS,
+ "find a list of matching lines using xdiff algorithm\n"},
 {NULL, NULL},
 };
 
-static const int version = 2;
+static const int version = 3;
 
 #ifdef IS_PY3K
 static struct PyModuleDef bdiff_module = {



To: quark, #hg-reviewers, indygreg, durin42
Cc: indygreg, mercurial-devel

D2621: tests: port test-log to Python 3

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a reviewer: pulkit.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Required some porting to >>> inline Python instead of using heredocs
  into $PYTHON.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2621

AFFECTED FILES
  contrib/python3-whitelist
  tests/test-log.t

CHANGE DETAILS

diff --git a/tests/test-log.t b/tests/test-log.t
--- a/tests/test-log.t
+++ b/tests/test-log.t
@@ -2016,33 +2016,31 @@
   $ hg init problematicencoding
   $ cd problematicencoding
 
-  $ $PYTHON > setup.sh < print(u'''
-  > echo a > text
-  > hg add text
-  > hg --encoding utf-8 commit -u '\u30A2' -m none
-  > echo b > text
-  > hg --encoding utf-8 commit -u '\u30C2' -m none
-  > echo c > text
-  > hg --encoding utf-8 commit -u none -m '\u30A2'
-  > echo d > text
-  > hg --encoding utf-8 commit -u none -m '\u30C2'
-  > '''.encode('utf-8'))
-  > EOF
+  >>> with open('setup.sh', 'wb') as f:
+  ... f.write(u'''
+  ... echo a > text
+  ... hg add text
+  ... hg --encoding utf-8 commit -u '\u30A2' -m none
+  ... echo b > text
+  ... hg --encoding utf-8 commit -u '\u30C2' -m none
+  ... echo c > text
+  ... hg --encoding utf-8 commit -u none -m '\u30A2'
+  ... echo d > text
+  ... hg --encoding utf-8 commit -u none -m '\u30C2'
+  ... '''.encode('utf-8')) and None
   $ sh < setup.sh
 
 test in problematic encoding
-  $ $PYTHON > test.sh < print(u'''
-  > hg --encoding cp932 log --template '{rev}\\n' -u '\u30A2'
-  > echo 
-  > hg --encoding cp932 log --template '{rev}\\n' -u '\u30C2'
-  > echo 
-  > hg --encoding cp932 log --template '{rev}\\n' -k '\u30A2'
-  > echo 
-  > hg --encoding cp932 log --template '{rev}\\n' -k '\u30C2'
-  > '''.encode('cp932'))
-  > EOF
+  >>> with open('test.sh', 'wb') as f:
+  ... f.write(u'''
+  ... hg --encoding cp932 log --template '{rev}\\n' -u '\u30A2'
+  ... echo 
+  ... hg --encoding cp932 log --template '{rev}\\n' -u '\u30C2'
+  ... echo 
+  ... hg --encoding cp932 log --template '{rev}\\n' -k '\u30A2'
+  ... echo 
+  ... hg --encoding cp932 log --template '{rev}\\n' -k '\u30C2'
+  ... '''.encode('cp932')) and None
   $ sh < test.sh
   0
   
@@ -2255,14 +2253,14 @@
   > from mercurial import namespaces
   > 
   > def reposetup(ui, repo):
-  > foo = {'foo': repo[0].node()}
+  > foo = {b'foo': repo[0].node()}
   > names = lambda r: foo.keys()
   > namemap = lambda r, name: foo.get(name)
   > nodemap = lambda r, node: [name for name, n in foo.items()
   >if n == node]
   > ns = namespaces.namespace(
-  > "bars", templatename="bar", logname="barlog",
-  > colorname="barcolor", listnames=names, namemap=namemap,
+  > b"bars", templatename=b"bar", logname=b"barlog",
+  > colorname=b"barcolor", listnames=names, namemap=namemap,
   > nodemap=nodemap)
   > 
   > repo.names.addnamespace(ns)
diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist
--- a/contrib/python3-whitelist
+++ b/contrib/python3-whitelist
@@ -172,6 +172,7 @@
 test-largefiles-small-disk.t
 test-locate.t
 test-lock-badness.t
+test-log.t
 test-logexchange.t
 test-lrucachedict.py
 test-mactext.t



To: durin42, pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2575: xdiff: add a bdiff hunk mode

[quark (Jun Wu)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3c56c4d26527: xdiff: add a bdiff hunk mode (authored by 
quark, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2575?vs=6474=6514

REVISION DETAIL
  https://phab.mercurial-scm.org/D2575

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiff.h
  mercurial/thirdparty/xdiff/xdiffi.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xdiffi.c 
b/mercurial/thirdparty/xdiff/xdiffi.c
--- a/mercurial/thirdparty/xdiff/xdiffi.c
+++ b/mercurial/thirdparty/xdiff/xdiffi.c
@@ -975,15 +975,32 @@
  xdemitconf_t const *xecfg)
 {
xdchange_t *xch, *xche;
-
-   for (xch = xscr; xch; xch = xche->next) {
-   xche = xdl_get_hunk(, xecfg);
-   if (!xch)
-   break;
-   if (xecfg->hunk_func(xch->i1, xche->i1 + xche->chg1 - xch->i1,
-xch->i2, xche->i2 + xche->chg2 - xch->i2,
-ecb->priv) < 0)
+   if ((xecfg->flags & XDL_EMIT_BDIFFHUNK) != 0) {
+   long i1 = 0, i2 = 0, n1 = xe->xdf1.nrec, n2 = xe->xdf2.nrec;
+   for (xch = xscr; xch; xch = xche->next) {
+   xche = xdl_get_hunk(, xecfg);
+   if (!xch)
+   break;
+   if (xch->i1 > i1 || xch->i2 > i2) {
+   if (xecfg->hunk_func(i1, xch->i1, i2, xch->i2, 
ecb->priv) < 0)
+   return -1;
+   }
+   i1 = xche->i1 + xche->chg1;
+   i2 = xche->i2 + xche->chg2;
+   }
+   if (xecfg->hunk_func(i1, n1, i2, n2, ecb->priv) < 0)
return -1;
+   } else {
+   for (xch = xscr; xch; xch = xche->next) {
+   xche = xdl_get_hunk(, xecfg);
+   if (!xch)
+   break;
+   if (xecfg->hunk_func(
+   xch->i1, xche->i1 + xche->chg1 - 
xch->i1,
+   xch->i2, xche->i2 + xche->chg2 - 
xch->i2,
+   ecb->priv) < 0)
+   return -1;
+   }
}
return 0;
 }
@@ -1026,18 +1043,15 @@
xdl_free_env();
return -1;
}
-   if (xscr) {
-   if (xpp->flags & XDF_IGNORE_BLANK_LINES)
-   xdl_mark_ignorable(xscr, , xpp->flags);
 
-   if (ef(, xscr, ecb, xecfg) < 0) {
-
-   xdl_free_script(xscr);
-   xdl_free_env();
-   return -1;
-   }
+   if (xpp->flags & XDF_IGNORE_BLANK_LINES)
+   xdl_mark_ignorable(xscr, , xpp->flags);
+   if (ef(, xscr, ecb, xecfg) < 0) {
xdl_free_script(xscr);
+   xdl_free_env();
+   return -1;
}
+   xdl_free_script(xscr);
xdl_free_env();
 
return 0;
diff --git a/mercurial/thirdparty/xdiff/xdiff.h 
b/mercurial/thirdparty/xdiff/xdiff.h
--- a/mercurial/thirdparty/xdiff/xdiff.h
+++ b/mercurial/thirdparty/xdiff/xdiff.h
@@ -48,6 +48,9 @@
 /* xdemitconf_t.flags */
 #define XDL_EMIT_FUNCNAMES (1 << 0)
 #define XDL_EMIT_FUNCCONTEXT (1 << 2)
+/* emit bdiff-style "matched" (a1, a2, b1, b2) hunks instead of "different"
+ * (a1, a2 - a1, b1, b2 - b1) hunks */
+#define XDL_EMIT_BDIFFHUNK (1 << 4)
 
 #define XDL_MMB_READONLY (1 << 0)
 



To: ryanmce, #hg-reviewers, indygreg, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[ryanmce (Ryan McElroy)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG8df20b7a39d2: templater: add hint to template parse errors 
to help locate issues (authored by ryanmce, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D2608?vs=6485=6509#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2608?vs=6485=6509

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

AFFECTED FILES
  mercurial/templater.py
  tests/test-command-template.t
  tests/test-export.t
  tests/test-log.t

CHANGE DETAILS

diff --git a/tests/test-log.t b/tests/test-log.t
--- a/tests/test-log.t
+++ b/tests/test-log.t
@@ -2289,6 +2289,23 @@
   $ hg --config extensions.names=../names.py log -r 0 --template '{bars}\n'
   foo
 
+Templater parse errors:
+
+simple error
+  $ hg log -r . -T '{shortest(node}'
+  hg: parse error at 15: unexpected token: end
+  ({shortest(node}
+ ^ here)
+  [255]
+
+multi-line template with error
+  $ hg log -r . -T 'line 1
+  > line2
+  > {shortest(node}
+  > line4\nline5'
+  hg: parse error at 28: unexpected token: end
+  [255]
+
   $ cd ..
 
 hg log -f dir across branches
diff --git a/tests/test-export.t b/tests/test-export.t
--- a/tests/test-export.t
+++ b/tests/test-export.t
@@ -218,6 +218,8 @@
   [255]
   $ hg export -o '%m{' tip
   hg: parse error at 3: unterminated template expansion
+  (%m{
+ ^ here)
   [255]
   $ hg export -o '%\' tip
   abort: invalid format spec '%\' in output filename
diff --git a/tests/test-command-template.t b/tests/test-command-template.t
--- a/tests/test-command-template.t
+++ b/tests/test-command-template.t
@@ -2766,19 +2766,29 @@
 
   $ hg log -T '{date'
   hg: parse error at 1: unterminated template expansion
+  ({date
+   ^ here)
   [255]
   $ hg log -T '{date(}'
   hg: parse error at 7: not a prefix: end
+  ({date(}
+ ^ here)
   [255]
   $ hg log -T '{date)}'
   hg: parse error at 5: invalid token
+  ({date)}
+   ^ here)
   [255]
   $ hg log -T '{date date}'
   hg: parse error at 6: invalid token
+  ({date date}
+^ here)
   [255]
 
   $ hg log -T '{}'
   hg: parse error at 2: not a prefix: end
+  ({}
+^ here)
   [255]
   $ hg debugtemplate -v '{()}'
   (template
@@ -2827,10 +2837,14 @@
 
   $ hg log -T '{"date'
   hg: parse error at 2: unterminated string
+  ({"date
+^ here)
   [255]
 
   $ hg log -T '{"foo{date|?}"}'
   hg: parse error at 11: syntax error
+  ({"foo{date|?}"}
+ ^ here)
   [255]
 
 Thrown an error if a template function doesn't exist
@@ -3362,6 +3376,8 @@
   -4
   $ hg debugtemplate '{(-)}\n'
   hg: parse error at 3: not a prefix: )
+  ({(-)}\n
+ ^ here)
   [255]
   $ hg debugtemplate '{(-a)}\n'
   hg: parse error: negation needs an integer argument
@@ -3527,6 +3543,8 @@
   foo
   $ hg log -r 2 -T '{if(rev, "{if(rev, \")}")}\n'
   hg: parse error at 21: unterminated string
+  ({if(rev, "{if(rev, \")}")}\n
+   ^ here)
   [255]
   $ hg log -r 2 -T '{if(rev, \"\\"")}\n'
   hg: parse error: trailing \ in string
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -213,35 +213,48 @@
 unescape = [parser.unescapestr, pycompat.identity][raw]
 pos = start
 p = parser.parser(elements)
-while pos < stop:
-n = min((tmpl.find(c, pos, stop) for c in sepchars),
-key=lambda n: (n < 0, n))
-if n < 0:
-yield ('string', unescape(tmpl[pos:stop]), pos)
-pos = stop
-break
-c = tmpl[n:n + 1]
-bs = 0  # count leading backslashes
-if not raw:
-bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
-if bs % 2 == 1:
-# escaped (e.g. '\{', '\\\{', but not '\\{')
-yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
-pos = n + 1
-continue
-if n > pos:
-yield ('string', unescape(tmpl[pos:n]), pos)
-if c == quote:
-yield ('end', None, n + 1)
-return
+try:
+while pos < stop:
+n = min((tmpl.find(c, pos, stop) for c in sepchars),
+key=lambda n: (n < 0, n))
+if n < 0:
+yield ('string', unescape(tmpl[pos:stop]), pos)
+pos = stop
+break
+c = tmpl[n:n + 1]
+bs = 0  # count leading backslashes
+if not raw:
+bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
+if bs % 2 == 1:
+# escaped (e.g. '\{', '\\\{', but not '\\{')
+yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
+pos = n + 1
+continue
+if n > pos:
+yield ('string', unescape(tmpl[pos:n]), pos)
+if c == quote:
+yield ('end', None, n + 1)
+return
 
-parseres, pos = 

D2609: templater: provide hint for multi-line templates with parse errors

[ryanmce (Ryan McElroy)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5d1bae328b7d: templater: provide hint for multi-line 
templates with parse errors (authored by ryanmce, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D2609?vs=6487=6510#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2609?vs=6487=6510

REVISION DETAIL
  https://phab.mercurial-scm.org/D2609

AFFECTED FILES
  mercurial/templater.py
  tests/test-log.t

CHANGE DETAILS

diff --git a/tests/test-log.t b/tests/test-log.t
--- a/tests/test-log.t
+++ b/tests/test-log.t
@@ -2304,6 +2304,8 @@
   > {shortest(node}
   > line4\nline5'
   hg: parse error at 28: unexpected token: end
+  (line 1\nline2\n{shortest(node}\nline4\nline5
+^ here)
   [255]
 
   $ cd ..
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -246,14 +246,16 @@
 except error.ParseError as inst:
 if len(inst.args) > 1:  # has location
 loc = inst.args[1]
-# TODO: Opportunity for improvement! If there is a newline in the
-# template, this hint does not point to the right place, so skip.
-if '\n' not in tmpl:
-# We want the caret to point to the place in the template that
-# failed to parse, but in a hint we get a open paren at the
-# start. Therefore, we print "loc" spaces (instead of "loc - 
1")
-# to line up the caret with the location of the error.
-inst.hint = tmpl + '\n' + ' ' * (loc) + '^ ' + _('here')
+# Offset the caret location by the number of newlines before the
+# location of the error, since we will replace one-char newlines
+# with the two-char literal r'\n'.
+offset = tmpl[:loc].count('\n')
+tmpl = tmpl.replace('\n', br'\n')
+# We want the caret to point to the place in the template that
+# failed to parse, but in a hint we get a open paren at the
+# start. Therefore, we print "loc" spaces (instead of "loc - 1")
+# to line up the caret with the location of the error.
+inst.hint = tmpl + '\n' + ' ' * (loc + offset) + '^ ' + _('here')
 raise
 yield ('end', None, pos)
 



To: ryanmce, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH v2] fancyopts: making config defaults actually override defaults

[Rodrigo Damazio via Mercurial-devel]

FYI I'm getting back to this old patch, but I'll send it through
phabricator this time.


On Sat, Apr 1, 2017 at 11:07 AM, Yuya Nishihara  wrote:

> On Fri, 24 Mar 2017 00:32:00 -0700, Rodrigo Damazio Bovendorp via
> Mercurial-devel wrote:
> > # HG changeset patch
> > # User Rodrigo Damazio 
> > # Date 1490340211 25200
> > #  Fri Mar 24 00:23:31 2017 -0700
> > # Node ID 60b3222e01f96f91ece7eda9681a89bf3bb930a6
> > # Parent  df82f375fa005b9c71b463182e6b54aa47fa5999
> > fancyopts: making config defaults actually override defaults
>
> Sorry for late review.
>
> This looks generally good to me. I have a few comments inline.
>
> >  c = list(entry[1])
> > +
> > +# override new-style defaults from config file by actually
> changing the
> > +# option defaults.
> > +for idx, opt in enumerate(c):
> > +optname = opt[1]
> > +shortname = opt[0]
> > +defaulttype = type(opt[2])
> > +rawdefault = (
> > +ui.config("commands", "%s.default.%s" % (cmd, optname))
> or
> > +ui.config("commands", "%s.default.%s" % (cmd,
> shortname)))
>
> I prefer not supporting "default." because shortname doesn't
> look
> like a config name. And if we do support it, an empty longname value should
> supersede a shortname:
>
>   ui.config(, default=ui.config())
>
> > +if rawdefault:
> > +# parse the new default using the type of the original
> default.
> > +default = fancyopts.parsevalue(optname, rawdefault,
> defaulttype,
> > +
>  util.parsebool(rawdefault))
>
> Can we use ui.configbool/list/int() instead of parsevalue() ? That will
> provide a slightly better error message.
>


smime.p7s
Description: S/MIME Cryptographic Signature
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2619: revsetlang: add a hint for more useful parse errors

[ryanmce (Ryan McElroy)]

ryanmce created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This logic is largely based on the similar logic added to template error
  messages in https://phab.mercurial-scm.org/D2608 and 
https://phab.mercurial-scm.org/D2609, but with a few tweaks based on how revsets
  actually work.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2619

AFFECTED FILES
  mercurial/revsetlang.py
  tests/test-annotate.t
  tests/test-default-push.t
  tests/test-fileset.t
  tests/test-revset.t
  tests/test-revset2.t

CHANGE DETAILS

diff --git a/tests/test-revset2.t b/tests/test-revset2.t
--- a/tests/test-revset2.t
+++ b/tests/test-revset2.t
@@ -690,6 +690,8 @@
 
   $ log '1 OR 2'
   hg: parse error at 2: invalid token
+  (1 OR 2
+ ^ here)
   [255]
 
 or operator should preserve ordering:
@@ -1562,6 +1564,8 @@
 test error message of bad revset
   $ hg log -r 'foo\\'
   hg: parse error at 3: syntax error in revset 'foo\\'
+  (foo\\
+  ^ here)
   [255]
 
   $ cd ..
diff --git a/tests/test-revset.t b/tests/test-revset.t
--- a/tests/test-revset.t
+++ b/tests/test-revset.t
@@ -399,6 +399,8 @@
   4
   $ log 'date(this is a test)'
   hg: parse error at 10: unexpected token: symbol
+  (date(this is a test)
+ ^ here)
   [255]
   $ log 'date()'
   hg: parse error: date requires a string
@@ -408,6 +410,8 @@
   [255]
   $ log 'date('
   hg: parse error at 5: not a prefix: end
+  (date(
+^ here)
   [255]
   $ log 'date("\xy")'
   hg: parse error: invalid \x escape
@@ -614,18 +618,28 @@
 
   $ hg debugrevspec '[0]'
   hg: parse error at 0: not a prefix: [
+  ([0]
+   ^ here)
   [255]
   $ hg debugrevspec '.#'
   hg: parse error at 2: not a prefix: end
+  (.#
+ ^ here)
   [255]
   $ hg debugrevspec '#rel'
   hg: parse error at 0: not a prefix: #
+  (#rel
+   ^ here)
   [255]
   $ hg debugrevspec '.#rel[0'
   hg: parse error at 7: unexpected token: end
+  (.#rel[0
+  ^ here)
   [255]
   $ hg debugrevspec '.]'
   hg: parse error at 1: invalid token
+  (.]
+^ here)
   [255]
 
   $ hg debugrevspec '.#generations[a]'
@@ -1330,6 +1344,8 @@
   6
   $ try 'grep(r"\")'
   hg: parse error at 7: unterminated string
+  (grep(r"\")
+  ^ here)
   [255]
   $ log 'head()'
   0
@@ -2774,3 +2790,13 @@
 
   $ cd ..
   $ cd repo
+
+test multiline revset with errors
+
+  $ hg log -r '
+  > . +
+  > .^ +'
+  hg: parse error at 9: not a prefix: end
+  ( . + .^ +
+^ here)
+  [255]
diff --git a/tests/test-fileset.t b/tests/test-fileset.t
--- a/tests/test-fileset.t
+++ b/tests/test-fileset.t
@@ -666,7 +666,11 @@
 
   $ fileset "status(' ', '4', added())"
   hg: parse error at 1: not a prefix: end
+  ( 
+^ here)
   [255]
   $ fileset "status('2', ' ', added())"
   hg: parse error at 1: not a prefix: end
+  ( 
+^ here)
   [255]
diff --git a/tests/test-default-push.t b/tests/test-default-push.t
--- a/tests/test-default-push.t
+++ b/tests/test-default-push.t
@@ -142,6 +142,8 @@
   $ hg --config 'paths.default:pushrev=(' push
   pushing to file:/*/$TESTTMP/pushurlsource/../pushurldest (glob)
   hg: parse error at 1: not a prefix: end
+  ((
+^ here)
   [255]
 
   $ cd ..
diff --git a/tests/test-annotate.t b/tests/test-annotate.t
--- a/tests/test-annotate.t
+++ b/tests/test-annotate.t
@@ -814,6 +814,8 @@
   [255]
   $ hg log -r 'followlines(baz, 2:4, startrev=20, descend=[1])'
   hg: parse error at 43: not a prefix: [
+  (followlines(baz, 2:4, startrev=20, descend=[1])
+  ^ here)
   [255]
   $ hg log -r 'followlines(baz, 2:4, startrev=20, descend=a)'
   hg: parse error: descend argument must be a boolean
diff --git a/mercurial/revsetlang.py b/mercurial/revsetlang.py
--- a/mercurial/revsetlang.py
+++ b/mercurial/revsetlang.py
@@ -539,7 +539,21 @@
 return tuple(foldconcat(t) for t in tree)
 
 def parse(spec, lookup=None):
-return _parsewith(spec, lookup=lookup)
+try:
+return _parsewith(spec, lookup=lookup)
+except error.ParseError as inst:
+if len(inst.args) > 1:  # has location
+# Add 1 to location because unlike templates, revset parse errors
+# point to the char where the error happened, not the char after.
+loc = inst.args[1] + 1
+# Remove newlines -- spaces are equivalent whitespace.
+spec = spec.replace('\n', ' ')
+# We want the caret to point to the place in the template that
+# failed to parse, but in a hint we get a open paren at the
+# start. Therefore, we print "loc + 1" spaces (instead of "loc")
+# to line up the caret with the location of the error.
+inst.hint = spec + '\n' + ' ' * loc + '^ ' + _('here')
+raise
 
 def _quote(s):
 r"""Quote a value in order to make it safe for the revset engine.



To: ryanmce, #hg-reviewers
Cc: mercurial-devel

D2618: py3: use b"%d" to covert integer to bytes instead of str

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe73e1f76b253: py3: use b%d to covert integer to 
bytes instead of str (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2618?vs=6502=6506

REVISION DETAIL
  https://phab.mercurial-scm.org/D2618

AFFECTED FILES
  hgext/journal.py
  hgext/mq.py
  hgext/patchbomb.py
  hgext/schemes.py

CHANGE DETAILS

diff --git a/hgext/schemes.py b/hgext/schemes.py
--- a/hgext/schemes.py
+++ b/hgext/schemes.py
@@ -94,7 +94,7 @@
 parts = parts[:-1]
 else:
 tail = ''
-context = dict((str(i + 1), v) for i, v in enumerate(parts))
+context = dict(('%d' % (i + 1), v) for i, v in enumerate(parts))
 return ''.join(self.templater.process(self.url, context)) + tail
 
 def hasdriveletter(orig, path):
diff --git a/hgext/patchbomb.py b/hgext/patchbomb.py
--- a/hgext/patchbomb.py
+++ b/hgext/patchbomb.py
@@ -212,7 +212,7 @@
 if not numbered:
 return '[PATCH%s]' % flag
 else:
-tlen = len(str(total))
+tlen = len("%d" % total)
 return '[PATCH %0*d of %d%s]' % (tlen, idx, total, flag)
 
 def makepatch(ui, repo, rev, patchlines, opts, _charsets, idx, total, numbered,
@@ -630,7 +630,7 @@
 if outgoing:
 revs = _getoutgoing(repo, dest, revs)
 if bundle:
-opts['revs'] = [str(r) for r in revs]
+opts['revs'] = ["%d" % r for r in revs]
 
 # check if revision exist on the public destination
 publicurl = repo.ui.config('patchbomb', 'publicurl')
diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -1932,7 +1932,7 @@
 length = len(self.series) - start
 if not missing:
 if self.ui.verbose:
-idxwidth = len(str(start + length - 1))
+idxwidth = len("%d" % (start + length - 1))
 for i in xrange(start, start + length):
 patch = self.series[i]
 if patch in applied:
@@ -2098,7 +2098,7 @@
 if not self.ui.verbose:
 p = pname
 else:
-p = str(self.series.index(pname)) + " " + pname
+p = ("%d" % self.series.index(pname)) + " " + pname
 return p
 
 def qimport(self, repo, files, patchname=None, rev=None, existing=None,
diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -351,7 +351,7 @@
 # Read just enough bytes to get a version number (up to 2
 # digits plus separator)
 version = f.read(3).partition('\0')[0]
-if version and version != str(storageversion):
+if version and version != "%d" % storageversion:
 # different version of the storage. Exit early (and not
 # write anything) if this is not a version we can handle or
 # the file is corrupt. In future, perhaps rotate the file
@@ -361,7 +361,7 @@
 return
 if not version:
 # empty file, write version first
-f.write(str(storageversion) + '\0')
+f.write(("%d" % storageversion) + '\0')
 f.seek(0, os.SEEK_END)
 f.write(bytes(entry) + '\0')
 
@@ -413,7 +413,7 @@
 
 lines = raw.split('\0')
 version = lines and lines[0]
-if version != str(storageversion):
+if version != "%d" % storageversion:
 version = version or _('not available')
 raise error.Abort(_("unknown journal file version '%s'") % version)
 



To: pulkit, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2617: py3: use bytes() instead of str()

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9d8ac10da5ce: py3: use bytes() instead of str() (authored 
by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2617?vs=6501=6505

REVISION DETAIL
  https://phab.mercurial-scm.org/D2617

AFFECTED FILES
  hgext/eol.py
  hgext/journal.py
  hgext/mq.py

CHANGE DETAILS

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -780,7 +780,7 @@
 
 diffopts = self.patchopts(diffopts, patch)
 patchf = self.opener(patch, "w")
-comments = str(ph)
+comments = bytes(ph)
 if comments:
 patchf.write(comments)
 self.printdiff(repo, diffopts, head, n, fp=patchf)
diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -363,7 +363,7 @@
 # empty file, write version first
 f.write(str(storageversion) + '\0')
 f.seek(0, os.SEEK_END)
-f.write(str(entry) + '\0')
+f.write(bytes(entry) + '\0')
 
 def filtered(self, namespace=None, name=None):
 """Yield all journal entries with the given namespace or name
diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -222,7 +222,7 @@
 data = ctx[f].data()
 if (target == "to-lf" and "\r\n" in data
 or target == "to-crlf" and singlelf.search(data)):
-failed.append((f, target, str(ctx)))
+failed.append((f, target, bytes(ctx)))
 break
 return failed
 



To: pulkit, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2616: py3: replace __str__ to __bytes__ in hgext/journal.py

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG148f64bcb6de: py3: replace __str__ to __bytes__ in 
hgext/journal.py (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2616?vs=6500=6504

REVISION DETAIL
  https://phab.mercurial-scm.org/D2616

AFFECTED FILES
  hgext/journal.py

CHANGE DETAILS

diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -24,6 +24,7 @@
 bookmarks,
 cmdutil,
 dispatch,
+encoding,
 error,
 extensions,
 hg,
@@ -219,15 +220,17 @@
 (timestamp, tz), user, command, namespace, name,
 oldhashes, newhashes)
 
-def __str__(self):
-"""String representation for storage"""
+def __bytes__(self):
+"""bytes representation for storage"""
 time = ' '.join(map(str, self.timestamp))
 oldhashes = ','.join([node.hex(hash) for hash in self.oldhashes])
 newhashes = ','.join([node.hex(hash) for hash in self.newhashes])
 return '\n'.join((
 time, self.user, self.command, self.namespace, self.name,
 oldhashes, newhashes))
 
+__str__ = encoding.strmethod(__bytes__)
+
 class journalstorage(object):
 """Storage for journal entries
 



To: pulkit, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2615: testrunner: add option to sort tests by previous run time

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3f502b22ba1d: testrunner: add option to sort tests by 
previous run time (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2615?vs=6499=6503

REVISION DETAIL
  https://phab.mercurial-scm.org/D2615

AFFECTED FILES
  tests/run-tests.py

CHANGE DETAILS

diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -345,6 +345,8 @@
 help="loop tests repeatedly")
 harness.add_argument('--random', action="store_true",
 help='run tests in random order')
+harness.add_argument('--order-by-runtime', action="store_true",
+help='run slowest tests first, according to .testtimes')
 harness.add_argument("-p", "--port", type=int,
 help="port on which servers should listen"
  " (default: $%s or %d)" % defaults['port'])
@@ -2307,47 +2309,57 @@
  separators=(',', ': '))
 outf.writelines(("testreport =", jsonout))
 
-def sorttests(testdescs, shuffle=False):
+def sorttests(testdescs, previoustimes, shuffle=False):
 """Do an in-place sort of tests."""
 if shuffle:
 random.shuffle(testdescs)
 return
 
-# keywords for slow tests
-slow = {b'svn': 10,
-b'cvs': 10,
-b'hghave': 10,
-b'largefiles-update': 10,
-b'run-tests': 10,
-b'corruption': 10,
-b'race': 10,
-b'i18n': 10,
-b'check': 100,
-b'gendoc': 100,
-b'contrib-perf': 200,
-}
-perf = {}
-
-def sortkey(f):
-# run largest tests first, as they tend to take the longest
-f = f['path']
-try:
-return perf[f]
-except KeyError:
+if previoustimes:
+def sortkey(f):
+  f = f['path']
+  if f in previoustimes:
+# Use most recent time as estimate
+return -previoustimes[f][-1]
+  else:
+# Default to a rather arbitrary value of 1 second for new tests
+return -1.0
+else:
+# keywords for slow tests
+slow = {b'svn': 10,
+b'cvs': 10,
+b'hghave': 10,
+b'largefiles-update': 10,
+b'run-tests': 10,
+b'corruption': 10,
+b'race': 10,
+b'i18n': 10,
+b'check': 100,
+b'gendoc': 100,
+b'contrib-perf': 200,
+}
+perf = {}
+
+def sortkey(f):
+# run largest tests first, as they tend to take the longest
+f = f['path']
 try:
-val = -os.stat(f).st_size
-except OSError as e:
-if e.errno != errno.ENOENT:
-raise
-perf[f] = -1e9  # file does not exist, tell early
-return -1e9
-for kw, mul in slow.items():
-if kw in f:
-val *= mul
-if f.endswith(b'.py'):
-val /= 10.0
-perf[f] = val / 1000.0
-return perf[f]
+return perf[f]
+except KeyError:
+try:
+val = -os.stat(f).st_size
+except OSError as e:
+if e.errno != errno.ENOENT:
+raise
+perf[f] = -1e9  # file does not exist, tell early
+return -1e9
+for kw, mul in slow.items():
+if kw in f:
+val *= mul
+if f.endswith(b'.py'):
+val /= 10.0
+perf[f] = val / 1000.0
+return perf[f]
 
 testdescs.sort(key=sortkey)
 
@@ -2418,8 +2430,6 @@
 os.umask(oldmask)
 
 def _run(self, testdescs):
-sorttests(testdescs, shuffle=self.options.random)
-
 self._testdir = osenvironb[b'TESTDIR'] = getattr(
 os, 'getcwdb', os.getcwd)()
 # assume all tests in same folder for now
@@ -2434,6 +2444,10 @@
 self._outputdir = self._testdir
 if testdescs and pathname:
 self._outputdir = os.path.join(self._outputdir, pathname)
+previoustimes = {}
+if self.options.order_by_runtime:
+  previoustimes = dict(loadtimes(self._outputdir))
+sorttests(testdescs, previoustimes, shuffle=self.options.random)
 
 if 'PYTHONHASHSEED' not in os.environ:
 # use a random python hash seed all the time



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2618: py3: use b"%d" to covert integer to bytes instead of str

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2618

AFFECTED FILES
  hgext/journal.py
  hgext/mq.py
  hgext/patchbomb.py
  hgext/schemes.py

CHANGE DETAILS

diff --git a/hgext/schemes.py b/hgext/schemes.py
--- a/hgext/schemes.py
+++ b/hgext/schemes.py
@@ -94,7 +94,7 @@
 parts = parts[:-1]
 else:
 tail = ''
-context = dict((str(i + 1), v) for i, v in enumerate(parts))
+context = dict(('%d' % (i + 1), v) for i, v in enumerate(parts))
 return ''.join(self.templater.process(self.url, context)) + tail
 
 def hasdriveletter(orig, path):
diff --git a/hgext/patchbomb.py b/hgext/patchbomb.py
--- a/hgext/patchbomb.py
+++ b/hgext/patchbomb.py
@@ -212,7 +212,7 @@
 if not numbered:
 return '[PATCH%s]' % flag
 else:
-tlen = len(str(total))
+tlen = len("%d" % total)
 return '[PATCH %0*d of %d%s]' % (tlen, idx, total, flag)
 
 def makepatch(ui, repo, rev, patchlines, opts, _charsets, idx, total, numbered,
@@ -630,7 +630,7 @@
 if outgoing:
 revs = _getoutgoing(repo, dest, revs)
 if bundle:
-opts['revs'] = [str(r) for r in revs]
+opts['revs'] = ["%d" % r for r in revs]
 
 # check if revision exist on the public destination
 publicurl = repo.ui.config('patchbomb', 'publicurl')
diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -1932,7 +1932,7 @@
 length = len(self.series) - start
 if not missing:
 if self.ui.verbose:
-idxwidth = len(str(start + length - 1))
+idxwidth = len("%d" % (start + length - 1))
 for i in xrange(start, start + length):
 patch = self.series[i]
 if patch in applied:
@@ -2098,7 +2098,7 @@
 if not self.ui.verbose:
 p = pname
 else:
-p = str(self.series.index(pname)) + " " + pname
+p = ("%d" % self.series.index(pname)) + " " + pname
 return p
 
 def qimport(self, repo, files, patchname=None, rev=None, existing=None,
diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -351,7 +351,7 @@
 # Read just enough bytes to get a version number (up to 2
 # digits plus separator)
 version = f.read(3).partition('\0')[0]
-if version and version != str(storageversion):
+if version and version != "%d" % storageversion:
 # different version of the storage. Exit early (and not
 # write anything) if this is not a version we can handle or
 # the file is corrupt. In future, perhaps rotate the file
@@ -361,7 +361,7 @@
 return
 if not version:
 # empty file, write version first
-f.write(str(storageversion) + '\0')
+f.write(("%d" % storageversion) + '\0')
 f.seek(0, os.SEEK_END)
 f.write(bytes(entry) + '\0')
 
@@ -413,7 +413,7 @@
 
 lines = raw.split('\0')
 version = lines and lines[0]
-if version != str(storageversion):
+if version != "%d" % storageversion:
 version = version or _('not available')
 raise error.Abort(_("unknown journal file version '%s'") % version)
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2616: py3: replace __str__ to __bytes__ in hgext/journal.py

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2616

AFFECTED FILES
  hgext/journal.py

CHANGE DETAILS

diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -24,6 +24,7 @@
 bookmarks,
 cmdutil,
 dispatch,
+encoding,
 error,
 extensions,
 hg,
@@ -219,15 +220,17 @@
 (timestamp, tz), user, command, namespace, name,
 oldhashes, newhashes)
 
-def __str__(self):
-"""String representation for storage"""
+def __bytes__(self):
+"""bytes representation for storage"""
 time = ' '.join(map(str, self.timestamp))
 oldhashes = ','.join([node.hex(hash) for hash in self.oldhashes])
 newhashes = ','.join([node.hex(hash) for hash in self.newhashes])
 return '\n'.join((
 time, self.user, self.command, self.namespace, self.name,
 oldhashes, newhashes))
 
+__str__ = encoding.strmethod(__bytes__)
+
 class journalstorage(object):
 """Storage for journal entries
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2617: py3: use bytes() instead of str()

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2617

AFFECTED FILES
  hgext/eol.py
  hgext/journal.py
  hgext/mq.py

CHANGE DETAILS

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -780,7 +780,7 @@
 
 diffopts = self.patchopts(diffopts, patch)
 patchf = self.opener(patch, "w")
-comments = str(ph)
+comments = bytes(ph)
 if comments:
 patchf.write(comments)
 self.printdiff(repo, diffopts, head, n, fp=patchf)
diff --git a/hgext/journal.py b/hgext/journal.py
--- a/hgext/journal.py
+++ b/hgext/journal.py
@@ -363,7 +363,7 @@
 # empty file, write version first
 f.write(str(storageversion) + '\0')
 f.seek(0, os.SEEK_END)
-f.write(str(entry) + '\0')
+f.write(bytes(entry) + '\0')
 
 def filtered(self, namespace=None, name=None):
 """Yield all journal entries with the given namespace or name
diff --git a/hgext/eol.py b/hgext/eol.py
--- a/hgext/eol.py
+++ b/hgext/eol.py
@@ -222,7 +222,7 @@
 data = ctx[f].data()
 if (target == "to-lf" and "\r\n" in data
 or target == "to-crlf" and singlelf.search(data)):
-failed.append((f, target, str(ctx)))
+failed.append((f, target, bytes(ctx)))
 break
 return failed
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2613: cmdutil: ensure PatchError messages are coerded to bytes, not str

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2dce0049176c: cmdutil: ensure PatchError messages are 
coerded to bytes, not str (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2613?vs=6491=6498

REVISION DETAIL
  https://phab.mercurial-scm.org/D2613

AFFECTED FILES
  mercurial/cmdutil.py

CHANGE DETAILS

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -1478,7 +1478,7 @@
 patch.patchrepo(ui, repo, p1, store, tmpname, strip, 
prefix,
 files, eolmode=None)
 except error.PatchError as e:
-raise error.Abort(str(e))
+raise error.Abort(util.forcebytestr(e))
 if opts.get('exact'):
 editor = None
 else:



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2612: tests: fix bytes literals in test-fncache.t

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3715a5ffcf92: tests: fix bytes literals in test-fncache.t 
(authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2612?vs=6490=6496

REVISION DETAIL
  https://phab.mercurial-scm.org/D2612

AFFECTED FILES
  tests/test-fncache.t

CHANGE DETAILS

diff --git a/tests/test-fncache.t b/tests/test-fncache.t
--- a/tests/test-fncache.t
+++ b/tests/test-fncache.t
@@ -259,14 +259,14 @@
   > def wrapper(orig, self, *args, **kwargs):
   > tr = orig(self, *args, **kwargs)
   > def fail(tr):
-  > raise error.Abort("forced transaction failure")
+  > raise error.Abort(b"forced transaction failure")
   > # zzz prefix to ensure it sorted after store.write
-  > tr.addfinalize('zzz-forcefails', fail)
+  > tr.addfinalize(b'zzz-forcefails', fail)
   > return tr
   > 
   > def uisetup(ui):
   > extensions.wrapfunction(
-  > localrepo.localrepository, 'transaction', wrapper)
+  > localrepo.localrepository, b'transaction', wrapper)
   > 
   > cmdtable = {}
   > 



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2614: py3: whitelist another 15 passing tests

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6f9442c6b404: py3: whitelist another 15 passing tests 
(authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2614?vs=6492=6497

REVISION DETAIL
  https://phab.mercurial-scm.org/D2614

AFFECTED FILES
  contrib/python3-whitelist

CHANGE DETAILS

diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist
--- a/contrib/python3-whitelist
+++ b/contrib/python3-whitelist
@@ -6,6 +6,7 @@
 test-ancestor.py
 test-annotate.py
 test-annotate.t
+test-atomictempfile.py
 test-audit-path.t
 test-audit-subrepo.t
 test-automv.t
@@ -167,6 +168,7 @@
 test-issue672.t
 test-issue842.t
 test-journal-exists.t
+test-largefiles-cache.t
 test-largefiles-small-disk.t
 test-locate.t
 test-lock-badness.t
@@ -200,12 +202,22 @@
 test-merge8.t
 test-merge9.t
 test-mq-git.t
+test-mq-header-date.t
+test-mq-header-from.t
 test-mq-pull-from-bundle.t
 test-mq-qdiff.t
+test-mq-qfold.t
+test-mq-qgoto.t
 test-mq-qimport-fail-cleanup.t
+test-mq-qpush-exact.t
 test-mq-qqueue.t
+test-mq-qrefresh-replace-log-message.t
 test-mq-qrefresh.t
+test-mq-qrename.t
 test-mq-qsave.t
+test-mq-safety.t
+test-mq-symlinks.t
+test-mv-cp-st-diff.t
 test-narrow-clone-no-ellipsis.t
 test-narrow-clone-nonlinear.t
 test-narrow-clone.t
@@ -222,6 +234,7 @@
 test-narrow-shallow-merges.t
 test-narrow-shallow.t
 test-narrow-update.t
+test-nested-repo.t
 test-newbranch.t
 test-obshistory.t
 test-obsmarker-template.t
@@ -288,6 +301,7 @@
 test-rebase-interruptions.t
 test-rebase-issue-noparam-single-rev.t
 test-rebase-legacy.t
+test-rebase-mq-skip.t
 test-rebase-named-branches.t
 test-rebase-newancestor.t
 test-rebase-obsolete.t
@@ -335,6 +349,7 @@
 test-status-terse.t
 test-strip-cross.t
 test-strip.t
+test-symlinks.t
 test-unamend.t
 test-uncommit.t
 test-unified-test.t



To: durin42, pulkit, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2611: scmutil: avoid using basestring and add explicit handling of unicodes

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb76248e51605: scmutil: avoid using basestring and add 
explicit handling of unicodes (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2611?vs=6489=6495

REVISION DETAIL
  https://phab.mercurial-scm.org/D2611

AFFECTED FILES
  mercurial/scmutil.py

CHANGE DETAILS

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -186,7 +186,10 @@
 ui.warn(_("(%s)\n") % inst.hint)
 except error.ResponseError as inst:
 ui.warn(_("abort: %s") % inst.args[0])
-if not isinstance(inst.args[1], basestring):
+msg = inst.args[1]
+if isinstance(msg, type(u'')):
+msg = pycompat.sysbytes(msg)
+elif not isinstance(inst.args[1], bytes):
 ui.warn(" %r\n" % (inst.args[1],))
 elif not inst.args[1]:
 ui.warn(_(" empty string\n"))



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2610: tests: fix inline extension bytes in test-ssh-proto-unbundle.t

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG11b279a75bf1: tests: fix inline extension bytes in 
test-ssh-proto-unbundle.t (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2610?vs=6488=6494

REVISION DETAIL
  https://phab.mercurial-scm.org/D2610

AFFECTED FILES
  tests/test-ssh-proto-unbundle.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto-unbundle.t b/tests/test-ssh-proto-unbundle.t
--- a/tests/test-ssh-proto-unbundle.t
+++ b/tests/test-ssh-proto-unbundle.t
@@ -158,36 +158,36 @@
   > from __future__ import print_function
   > import sys
   > def hook1line(ui, repo, **kwargs):
-  > ui.write('ui.write 1 line\n')
+  > ui.write(b'ui.write 1 line\n')
   > return 1
   > def hook2lines(ui, repo, **kwargs):
-  > ui.write('ui.write 2 lines 1\n')
-  > ui.write('ui.write 2 lines 2\n')
+  > ui.write(b'ui.write 2 lines 1\n')
+  > ui.write(b'ui.write 2 lines 2\n')
   > return 1
   > def hook1lineflush(ui, repo, **kwargs):
-  > ui.write('ui.write 1 line flush\n')
+  > ui.write(b'ui.write 1 line flush\n')
   > ui.flush()
   > return 1
   > def hookmultiflush(ui, repo, **kwargs):
-  > ui.write('ui.write 1st\n')
+  > ui.write(b'ui.write 1st\n')
   > ui.flush()
-  > ui.write('ui.write 2nd\n')
+  > ui.write(b'ui.write 2nd\n')
   > ui.flush()
   > return 1
   > def hookwriteandwriteerr(ui, repo, **kwargs):
-  > ui.write('ui.write 1\n')
-  > ui.write_err('ui.write_err 1\n')
-  > ui.write('ui.write 2\n')
-  > ui.write_err('ui.write_err 2\n')
+  > ui.write(b'ui.write 1\n')
+  > ui.write_err(b'ui.write_err 1\n')
+  > ui.write(b'ui.write 2\n')
+  > ui.write_err(b'ui.write_err 2\n')
   > return 1
   > def hookprintstdout(ui, repo, **kwargs):
   > print('printed line')
   > return 1
   > def hookprintandwrite(ui, repo, **kwargs):
   > print('print 1')
-  > ui.write('ui.write 1\n')
+  > ui.write(b'ui.write 1\n')
   > print('print 2')
-  > ui.write('ui.write 2\n')
+  > ui.write(b'ui.write 2\n')
   > return 1
   > def hookprintstderrandstdout(ui, repo, **kwargs):
   > print('stdout 1')
@@ -1901,10 +1901,10 @@
 
   $ cat > $TESTTMP/hook << EOF
   > def hookuiwrite(ui, repo, **kwargs):
-  > ui.write('ui.write 1\n')
-  > ui.write_err('ui.write_err 1\n')
-  > ui.write('ui.write 2\n')
-  > ui.write_err('ui.write_err 2\n')
+  > ui.write(b'ui.write 1\n')
+  > ui.write_err(b'ui.write_err 1\n')
+  > ui.write(b'ui.write 2\n')
+  > ui.write_err(b'ui.write_err 2\n')
   > EOF
 
   $ hg init uiwriterepo



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2607: hghave: fix up clang-libfuzzer regex to be bytes

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGdc11f257ad1d: hghave: fix up clang-libfuzzer regex to be 
bytes (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2607?vs=6482=6493

REVISION DETAIL
  https://phab.mercurial-scm.org/D2607

AFFECTED FILES
  tests/hghave.py

CHANGE DETAILS

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -703,7 +703,7 @@
 
 @check("clang-libfuzzer", "clang new enough to include libfuzzer")
 def has_clang_libfuzzer():
-mat = matchoutput('clang --version', 'clang version (\d)')
+mat = matchoutput('clang --version', b'clang version (\d)')
 if mat:
 # libfuzzer is new in clang 6
 return int(mat.group(1)) > 5



To: durin42, #hg-reviewers, indygreg
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2614: py3: whitelist another 15 passing tests

[pulkit (Pulkit Goyal)]

pulkit added a comment.


  yayay 

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2614

To: durin42, pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2611: scmutil: avoid using basestring and add explicit handling of unicodes

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This resolves some Python 3 defects, and I don't think it is a
  meaningful behavior change in Python 2.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2611

AFFECTED FILES
  mercurial/scmutil.py

CHANGE DETAILS

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -186,7 +186,10 @@
 ui.warn(_("(%s)\n") % inst.hint)
 except error.ResponseError as inst:
 ui.warn(_("abort: %s") % inst.args[0])
-if not isinstance(inst.args[1], basestring):
+msg = inst.args[1]
+if isinstance(msg, type(u'')):
+msg = pycompat.sysbytes(msg)
+elif not isinstance(inst.args[1], bytes):
 ui.warn(" %r\n" % (inst.args[1],))
 elif not inst.args[1]:
 ui.warn(_(" empty string\n"))



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2610: tests: fix inline extension bytes in test-ssh-proto-unbundle.t

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  1. skip-blame just b prefixes

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2610

AFFECTED FILES
  tests/test-ssh-proto-unbundle.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto-unbundle.t b/tests/test-ssh-proto-unbundle.t
--- a/tests/test-ssh-proto-unbundle.t
+++ b/tests/test-ssh-proto-unbundle.t
@@ -158,36 +158,36 @@
   > from __future__ import print_function
   > import sys
   > def hook1line(ui, repo, **kwargs):
-  > ui.write('ui.write 1 line\n')
+  > ui.write(b'ui.write 1 line\n')
   > return 1
   > def hook2lines(ui, repo, **kwargs):
-  > ui.write('ui.write 2 lines 1\n')
-  > ui.write('ui.write 2 lines 2\n')
+  > ui.write(b'ui.write 2 lines 1\n')
+  > ui.write(b'ui.write 2 lines 2\n')
   > return 1
   > def hook1lineflush(ui, repo, **kwargs):
-  > ui.write('ui.write 1 line flush\n')
+  > ui.write(b'ui.write 1 line flush\n')
   > ui.flush()
   > return 1
   > def hookmultiflush(ui, repo, **kwargs):
-  > ui.write('ui.write 1st\n')
+  > ui.write(b'ui.write 1st\n')
   > ui.flush()
-  > ui.write('ui.write 2nd\n')
+  > ui.write(b'ui.write 2nd\n')
   > ui.flush()
   > return 1
   > def hookwriteandwriteerr(ui, repo, **kwargs):
-  > ui.write('ui.write 1\n')
-  > ui.write_err('ui.write_err 1\n')
-  > ui.write('ui.write 2\n')
-  > ui.write_err('ui.write_err 2\n')
+  > ui.write(b'ui.write 1\n')
+  > ui.write_err(b'ui.write_err 1\n')
+  > ui.write(b'ui.write 2\n')
+  > ui.write_err(b'ui.write_err 2\n')
   > return 1
   > def hookprintstdout(ui, repo, **kwargs):
   > print('printed line')
   > return 1
   > def hookprintandwrite(ui, repo, **kwargs):
   > print('print 1')
-  > ui.write('ui.write 1\n')
+  > ui.write(b'ui.write 1\n')
   > print('print 2')
-  > ui.write('ui.write 2\n')
+  > ui.write(b'ui.write 2\n')
   > return 1
   > def hookprintstderrandstdout(ui, repo, **kwargs):
   > print('stdout 1')
@@ -1901,10 +1901,10 @@
 
   $ cat > $TESTTMP/hook << EOF
   > def hookuiwrite(ui, repo, **kwargs):
-  > ui.write('ui.write 1\n')
-  > ui.write_err('ui.write_err 1\n')
-  > ui.write('ui.write 2\n')
-  > ui.write_err('ui.write_err 2\n')
+  > ui.write(b'ui.write 1\n')
+  > ui.write_err(b'ui.write_err 1\n')
+  > ui.write(b'ui.write 2\n')
+  > ui.write_err(b'ui.write_err 2\n')
   > EOF
 
   $ hg init uiwriterepo



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2613: cmdutil: ensure PatchError messages are coerded to bytes, not str

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2613

AFFECTED FILES
  mercurial/cmdutil.py

CHANGE DETAILS

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -1478,7 +1478,7 @@
 patch.patchrepo(ui, repo, p1, store, tmpname, strip, 
prefix,
 files, eolmode=None)
 except error.PatchError as e:
-raise error.Abort(str(e))
+raise error.Abort(util.forcebytestr(e))
 if opts.get('exact'):
 editor = None
 else:



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2614: py3: whitelist another 15 passing tests

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a reviewer: pulkit.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2614

AFFECTED FILES
  contrib/python3-whitelist

CHANGE DETAILS

diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist
--- a/contrib/python3-whitelist
+++ b/contrib/python3-whitelist
@@ -6,6 +6,7 @@
 test-ancestor.py
 test-annotate.py
 test-annotate.t
+test-atomictempfile.py
 test-audit-path.t
 test-audit-subrepo.t
 test-automv.t
@@ -167,6 +168,7 @@
 test-issue672.t
 test-issue842.t
 test-journal-exists.t
+test-largefiles-cache.t
 test-largefiles-small-disk.t
 test-locate.t
 test-lock-badness.t
@@ -200,12 +202,22 @@
 test-merge8.t
 test-merge9.t
 test-mq-git.t
+test-mq-header-date.t
+test-mq-header-from.t
 test-mq-pull-from-bundle.t
 test-mq-qdiff.t
+test-mq-qfold.t
+test-mq-qgoto.t
 test-mq-qimport-fail-cleanup.t
+test-mq-qpush-exact.t
 test-mq-qqueue.t
+test-mq-qrefresh-replace-log-message.t
 test-mq-qrefresh.t
+test-mq-qrename.t
 test-mq-qsave.t
+test-mq-safety.t
+test-mq-symlinks.t
+test-mv-cp-st-diff.t
 test-narrow-clone-no-ellipsis.t
 test-narrow-clone-nonlinear.t
 test-narrow-clone.t
@@ -222,6 +234,7 @@
 test-narrow-shallow-merges.t
 test-narrow-shallow.t
 test-narrow-update.t
+test-nested-repo.t
 test-newbranch.t
 test-obshistory.t
 test-obsmarker-template.t
@@ -288,6 +301,7 @@
 test-rebase-interruptions.t
 test-rebase-issue-noparam-single-rev.t
 test-rebase-legacy.t
+test-rebase-mq-skip.t
 test-rebase-named-branches.t
 test-rebase-newancestor.t
 test-rebase-obsolete.t
@@ -335,6 +349,7 @@
 test-status-terse.t
 test-strip-cross.t
 test-strip.t
+test-symlinks.t
 test-unamend.t
 test-uncommit.t
 test-unified-test.t



To: durin42, pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2612: tests: fix bytes literals in test-fncache.t

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  1. skip-blame just b prefixes

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2612

AFFECTED FILES
  tests/test-fncache.t

CHANGE DETAILS

diff --git a/tests/test-fncache.t b/tests/test-fncache.t
--- a/tests/test-fncache.t
+++ b/tests/test-fncache.t
@@ -259,14 +259,14 @@
   > def wrapper(orig, self, *args, **kwargs):
   > tr = orig(self, *args, **kwargs)
   > def fail(tr):
-  > raise error.Abort("forced transaction failure")
+  > raise error.Abort(b"forced transaction failure")
   > # zzz prefix to ensure it sorted after store.write
-  > tr.addfinalize('zzz-forcefails', fail)
+  > tr.addfinalize(b'zzz-forcefails', fail)
   > return tr
   > 
   > def uisetup(ui):
   > extensions.wrapfunction(
-  > localrepo.localrepository, 'transaction', wrapper)
+  > localrepo.localrepository, b'transaction', wrapper)
   > 
   > cmdtable = {}
   > 



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2609: templater: provide hint for multi-line templates with parse errors

[ryanmce (Ryan McElroy)]

ryanmce updated this revision to Diff 6487.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2609?vs=6486=6487

REVISION DETAIL
  https://phab.mercurial-scm.org/D2609

AFFECTED FILES
  mercurial/templater.py
  tests/test-parse-errors.t

CHANGE DETAILS

diff --git a/tests/test-parse-errors.t b/tests/test-parse-errors.t
--- a/tests/test-parse-errors.t
+++ b/tests/test-parse-errors.t
@@ -16,4 +16,7 @@
   > {shortest(node}
   > line4\nline5'
   hg: parse error at 28: unexpected token: end
+  (line 1\nline2\n{shortest(node}\nline4
+^ here)
   [255]
+
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -245,14 +245,16 @@
 except error.ParseError as inst:
 if len(inst.args) > 1:  # has location
 loc = inst.args[1]
-# TODO: Opportunity for improvement! If there is a newline in the
-# template, this hint does not point to the right place, so skip.
-if '\n' not in tmpl:
-# We want the caret to point to the place in the template that
-# failed to parse, but in a hint we get a open paren at the
-# start. Therefore, we print "loc" spaces (instead of "loc - 
1")
-# to line up the caret with the location of the error.
-inst.hint = tmpl + '\n' + ' ' * (loc) + '^ ' + _('here')
+# Offset the caret location by the number of newlines before the
+# location of the error, since we will replace one-char newlines
+# with the two-char literal r'\n'.
+offset = tmpl[:loc].count('\n')
+tmpl = tmpl.replace('\n', r'\n')
+# We want the caret to point to the place in the template that
+# failed to parse, but in a hint we get a open paren at the
+# start. Therefore, we print "loc" spaces (instead of "loc - 1")
+# to line up the caret with the location of the error.
+inst.hint = tmpl + '\n' + ' ' * (loc + offset) + '^ ' + _('here')
 raise
 yield ('end', None, pos)
 



To: ryanmce, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[ryanmce (Ryan McElroy)]

ryanmce updated this revision to Diff 6485.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2608?vs=6484=6485

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

AFFECTED FILES
  mercurial/templater.py
  tests/test-command-template.t
  tests/test-export.t
  tests/test-parse-errors.t

CHANGE DETAILS

diff --git a/tests/test-parse-errors.t b/tests/test-parse-errors.t
new file mode 100644
--- /dev/null
+++ b/tests/test-parse-errors.t
@@ -0,0 +1,19 @@
+  $ hg init repo
+  $ cd repo
+  $ echo a > a
+  $ hg ci -qAm aa
+
+simple error
+  $ hg log -r . -T '{shortest(node}'
+  hg: parse error at 15: unexpected token: end
+  ({shortest(node}
+ ^ here)
+  [255]
+
+multi-line
+  $ hg log -r . -T 'line 1
+  > line2
+  > {shortest(node}
+  > line4\nline5'
+  hg: parse error at 28: unexpected token: end
+  [255]
diff --git a/tests/test-export.t b/tests/test-export.t
--- a/tests/test-export.t
+++ b/tests/test-export.t
@@ -218,6 +218,8 @@
   [255]
   $ hg export -o '%m{' tip
   hg: parse error at 3: unterminated template expansion
+  (%m{
+ ^ here)
   [255]
   $ hg export -o '%\' tip
   abort: invalid format spec '%\' in output filename
diff --git a/tests/test-command-template.t b/tests/test-command-template.t
--- a/tests/test-command-template.t
+++ b/tests/test-command-template.t
@@ -2766,19 +2766,29 @@
 
   $ hg log -T '{date'
   hg: parse error at 1: unterminated template expansion
+  ({date
+   ^ here)
   [255]
   $ hg log -T '{date(}'
   hg: parse error at 7: not a prefix: end
+  ({date(}
+ ^ here)
   [255]
   $ hg log -T '{date)}'
   hg: parse error at 5: invalid token
+  ({date)}
+   ^ here)
   [255]
   $ hg log -T '{date date}'
   hg: parse error at 6: invalid token
+  ({date date}
+^ here)
   [255]
 
   $ hg log -T '{}'
   hg: parse error at 2: not a prefix: end
+  ({}
+^ here)
   [255]
   $ hg debugtemplate -v '{()}'
   (template
@@ -2827,10 +2837,14 @@
 
   $ hg log -T '{"date'
   hg: parse error at 2: unterminated string
+  ({"date
+^ here)
   [255]
 
   $ hg log -T '{"foo{date|?}"}'
   hg: parse error at 11: syntax error
+  ({"foo{date|?}"}
+ ^ here)
   [255]
 
 Thrown an error if a template function doesn't exist
@@ -3362,6 +3376,8 @@
   -4
   $ hg debugtemplate '{(-)}\n'
   hg: parse error at 3: not a prefix: )
+  ({(-)}\n
+ ^ here)
   [255]
   $ hg debugtemplate '{(-a)}\n'
   hg: parse error: negation needs an integer argument
@@ -3527,6 +3543,8 @@
   foo
   $ hg log -r 2 -T '{if(rev, "{if(rev, \")}")}\n'
   hg: parse error at 21: unterminated string
+  ({if(rev, "{if(rev, \")}")}\n
+   ^ here)
   [255]
   $ hg log -r 2 -T '{if(rev, \"\\"")}\n'
   hg: parse error: trailing \ in string
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -212,35 +212,48 @@
 unescape = [parser.unescapestr, pycompat.identity][raw]
 pos = start
 p = parser.parser(elements)
-while pos < stop:
-n = min((tmpl.find(c, pos, stop) for c in sepchars),
-key=lambda n: (n < 0, n))
-if n < 0:
-yield ('string', unescape(tmpl[pos:stop]), pos)
-pos = stop
-break
-c = tmpl[n:n + 1]
-bs = 0  # count leading backslashes
-if not raw:
-bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
-if bs % 2 == 1:
-# escaped (e.g. '\{', '\\\{', but not '\\{')
-yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
-pos = n + 1
-continue
-if n > pos:
-yield ('string', unescape(tmpl[pos:n]), pos)
-if c == quote:
-yield ('end', None, n + 1)
-return
+try:
+while pos < stop:
+n = min((tmpl.find(c, pos, stop) for c in sepchars),
+key=lambda n: (n < 0, n))
+if n < 0:
+yield ('string', unescape(tmpl[pos:stop]), pos)
+pos = stop
+break
+c = tmpl[n:n + 1]
+bs = 0  # count leading backslashes
+if not raw:
+bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
+if bs % 2 == 1:
+# escaped (e.g. '\{', '\\\{', but not '\\{')
+yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
+pos = n + 1
+continue
+if n > pos:
+yield ('string', unescape(tmpl[pos:n]), pos)
+if c == quote:
+yield ('end', None, n + 1)
+return
 
-parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
-if not tmpl.endswith('}', n + 1, pos):
-raise error.ParseError(_("invalid token"), pos)
-yield ('template', parseres, n)
+parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
+if not tmpl.endswith('}', n + 1, pos):
+raise 

D2609: templater: provide hint for multi-line templates with parse errors

[ryanmce (Ryan McElroy)]

ryanmce created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously, we punted. Now we "rewrite" the template's newlines to r'\n' and
  offset the hint appropriately.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2609

AFFECTED FILES
  mercurial/templater.py
  tests/test-parse-errors.t

CHANGE DETAILS

diff --git a/tests/test-parse-errors.t b/tests/test-parse-errors.t
--- a/tests/test-parse-errors.t
+++ b/tests/test-parse-errors.t
@@ -16,4 +16,7 @@
   > {shortest(node}
   > line4\nline5'
   hg: parse error at 28: unexpected token: end
+  (line 1\nline2\n{shortest(node}\nline4
+^ here)
   [255]
+
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -245,14 +245,16 @@
 except error.ParseError as inst:
 if len(inst.args) > 1:  # has location
 loc = inst.args[1]
-# TODO: Opportunity for improvement! If there is a newline in the
-# template, this hint does not point to the right place, so skip.
-if '\n' not in tmpl:
-# We want the caret to point to the place in the template that
-# failed to parse, but in a hint we get a open paren at the
-# start. Therefore, we print "loc" spaces (instead of "loc - 
1")
-# to line up the caret with the location of the error.
-inst.hint = tmpl + '\n' + ' ' * (loc) + '^ ' + _('here')
+# if there are newlines in front of the location of the error,
+# offset the caret location by that number, since we will next
+# replace newlines with the two-char literal r'\n'.
+offset = tmpl[:loc].count('\n')
+tmpl = tmpl.replace('\n', r'\n')
+# We want the caret to point to the place in the template that
+# failed to parse, but in a hint we get a open paren at the
+# start. Therefore, we print "loc" spaces (instead of "loc - 1")
+# to line up the caret with the location of the error.
+inst.hint = tmpl + '\n' + ' ' * (loc + offset) + '^ ' + _('here')
 raise
 yield ('end', None, pos)
 



To: ryanmce, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[pulkit (Pulkit Goyal)]

pulkit added a comment.


  I like similar things in rust too. I am +1 on this.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

To: ryanmce, #hg-reviewers, durin42
Cc: pulkit, durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2607: hghave: fix up clang-libfuzzer regex to be bytes

[pulkit (Pulkit Goyal)]

pulkit added inline comments.

INLINE COMMENTS

> hghave.py:265
>  return True
>  return matchoutput('hg --version --quiet 2>&1', 'Mercurial Distributed 
> SCM')
>  

Since you are here, can you get this one and next call to matchoutput fixed too?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2607

To: durin42, #hg-reviewers
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[ryanmce (Ryan McElroy)]

ryanmce updated this revision to Diff 6484.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2608?vs=6483=6484

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

AFFECTED FILES
  mercurial/templater.py
  tests/test-command-template.t
  tests/test-export.t
  tests/test-parse-errors.t

CHANGE DETAILS

diff --git a/tests/test-parse-errors.t b/tests/test-parse-errors.t
new file mode 100644
--- /dev/null
+++ b/tests/test-parse-errors.t
@@ -0,0 +1,21 @@
+  $ hg init repo
+  $ cd repo
+  $ echo a > a
+  $ hg ci -qAm aa
+
+simple error
+
+  $ hg log -r . -T '{shortest(node}'
+  hg: parse error at 15: unexpected token: end
+  ({shortest(node}
+ ^ here)
+  [255]
+
+multi-line
+
+  $ hg log -r . -T 'line 1
+  > line2
+  > {shortest(node}
+  > line4'
+  hg: parse error at 28: unexpected token: end
+  [255]
diff --git a/tests/test-export.t b/tests/test-export.t
--- a/tests/test-export.t
+++ b/tests/test-export.t
@@ -218,6 +218,8 @@
   [255]
   $ hg export -o '%m{' tip
   hg: parse error at 3: unterminated template expansion
+  (%m{
+ ^ here)
   [255]
   $ hg export -o '%\' tip
   abort: invalid format spec '%\' in output filename
diff --git a/tests/test-command-template.t b/tests/test-command-template.t
--- a/tests/test-command-template.t
+++ b/tests/test-command-template.t
@@ -2766,19 +2766,29 @@
 
   $ hg log -T '{date'
   hg: parse error at 1: unterminated template expansion
+  ({date
+   ^ here)
   [255]
   $ hg log -T '{date(}'
   hg: parse error at 7: not a prefix: end
+  ({date(}
+ ^ here)
   [255]
   $ hg log -T '{date)}'
   hg: parse error at 5: invalid token
+  ({date)}
+   ^ here)
   [255]
   $ hg log -T '{date date}'
   hg: parse error at 6: invalid token
+  ({date date}
+^ here)
   [255]
 
   $ hg log -T '{}'
   hg: parse error at 2: not a prefix: end
+  ({}
+^ here)
   [255]
   $ hg debugtemplate -v '{()}'
   (template
@@ -2827,10 +2837,14 @@
 
   $ hg log -T '{"date'
   hg: parse error at 2: unterminated string
+  ({"date
+^ here)
   [255]
 
   $ hg log -T '{"foo{date|?}"}'
   hg: parse error at 11: syntax error
+  ({"foo{date|?}"}
+ ^ here)
   [255]
 
 Thrown an error if a template function doesn't exist
@@ -3362,6 +3376,8 @@
   -4
   $ hg debugtemplate '{(-)}\n'
   hg: parse error at 3: not a prefix: )
+  ({(-)}\n
+ ^ here)
   [255]
   $ hg debugtemplate '{(-a)}\n'
   hg: parse error: negation needs an integer argument
@@ -3527,6 +3543,8 @@
   foo
   $ hg log -r 2 -T '{if(rev, "{if(rev, \")}")}\n'
   hg: parse error at 21: unterminated string
+  ({if(rev, "{if(rev, \")}")}\n
+   ^ here)
   [255]
   $ hg log -r 2 -T '{if(rev, \"\\"")}\n'
   hg: parse error: trailing \ in string
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -212,35 +212,48 @@
 unescape = [parser.unescapestr, pycompat.identity][raw]
 pos = start
 p = parser.parser(elements)
-while pos < stop:
-n = min((tmpl.find(c, pos, stop) for c in sepchars),
-key=lambda n: (n < 0, n))
-if n < 0:
-yield ('string', unescape(tmpl[pos:stop]), pos)
-pos = stop
-break
-c = tmpl[n:n + 1]
-bs = 0  # count leading backslashes
-if not raw:
-bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
-if bs % 2 == 1:
-# escaped (e.g. '\{', '\\\{', but not '\\{')
-yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
-pos = n + 1
-continue
-if n > pos:
-yield ('string', unescape(tmpl[pos:n]), pos)
-if c == quote:
-yield ('end', None, n + 1)
-return
+try:
+while pos < stop:
+n = min((tmpl.find(c, pos, stop) for c in sepchars),
+key=lambda n: (n < 0, n))
+if n < 0:
+yield ('string', unescape(tmpl[pos:stop]), pos)
+pos = stop
+break
+c = tmpl[n:n + 1]
+bs = 0  # count leading backslashes
+if not raw:
+bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
+if bs % 2 == 1:
+# escaped (e.g. '\{', '\\\{', but not '\\{')
+yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
+pos = n + 1
+continue
+if n > pos:
+yield ('string', unescape(tmpl[pos:n]), pos)
+if c == quote:
+yield ('end', None, n + 1)
+return
 
-parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
-if not tmpl.endswith('}', n + 1, pos):
-raise error.ParseError(_("invalid token"), pos)
-yield ('template', parseres, n)
+parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
+if not tmpl.endswith('}', n + 1, pos):
+raise 

D2608: templater: add hint to template parse errors to help locate issues

[durin42 (Augie Fackler)]

durin42 requested changes to this revision.
durin42 added a comment.
This revision now requires changes to proceed.


  We should add a test about templates that contain newlines - it's probably 
Hard, so feel free to just not be helpful with newline-containing templates for 
now?

INLINE COMMENTS

> test-command-template.t:2770
> +  ({date
> +   ^)
>[255]

Nit: could we make this be "^ here" instead of just the caret?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

To: ryanmce, #hg-reviewers, durin42
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2608: templater: add hint to template parse errors to help locate issues

[ryanmce (Ryan McElroy)]

ryanmce created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously, we would print the error name and location, but this isn't as
  helpful as we can be. Let's add a hint that shows the location where we
  encountered the parse error.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2608

AFFECTED FILES
  mercurial/templater.py
  tests/test-command-template.t
  tests/test-export.t

CHANGE DETAILS

diff --git a/tests/test-export.t b/tests/test-export.t
--- a/tests/test-export.t
+++ b/tests/test-export.t
@@ -218,6 +218,8 @@
   [255]
   $ hg export -o '%m{' tip
   hg: parse error at 3: unterminated template expansion
+  (%m{
+ ^)
   [255]
   $ hg export -o '%\' tip
   abort: invalid format spec '%\' in output filename
diff --git a/tests/test-command-template.t b/tests/test-command-template.t
--- a/tests/test-command-template.t
+++ b/tests/test-command-template.t
@@ -2766,19 +2766,29 @@
 
   $ hg log -T '{date'
   hg: parse error at 1: unterminated template expansion
+  ({date
+   ^)
   [255]
   $ hg log -T '{date(}'
   hg: parse error at 7: not a prefix: end
+  ({date(}
+ ^)
   [255]
   $ hg log -T '{date)}'
   hg: parse error at 5: invalid token
+  ({date)}
+   ^)
   [255]
   $ hg log -T '{date date}'
   hg: parse error at 6: invalid token
+  ({date date}
+^)
   [255]
 
   $ hg log -T '{}'
   hg: parse error at 2: not a prefix: end
+  ({}
+^)
   [255]
   $ hg debugtemplate -v '{()}'
   (template
@@ -2827,10 +2837,14 @@
 
   $ hg log -T '{"date'
   hg: parse error at 2: unterminated string
+  ({"date
+^)
   [255]
 
   $ hg log -T '{"foo{date|?}"}'
   hg: parse error at 11: syntax error
+  ({"foo{date|?}"}
+ ^)
   [255]
 
 Thrown an error if a template function doesn't exist
@@ -3362,6 +3376,8 @@
   -4
   $ hg debugtemplate '{(-)}\n'
   hg: parse error at 3: not a prefix: )
+  ({(-)}\n
+ ^)
   [255]
   $ hg debugtemplate '{(-a)}\n'
   hg: parse error: negation needs an integer argument
@@ -3527,6 +3543,8 @@
   foo
   $ hg log -r 2 -T '{if(rev, "{if(rev, \")}")}\n'
   hg: parse error at 21: unterminated string
+  ({if(rev, "{if(rev, \")}")}\n
+   ^)
   [255]
   $ hg log -r 2 -T '{if(rev, \"\\"")}\n'
   hg: parse error: trailing \ in string
diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -212,35 +212,45 @@
 unescape = [parser.unescapestr, pycompat.identity][raw]
 pos = start
 p = parser.parser(elements)
-while pos < stop:
-n = min((tmpl.find(c, pos, stop) for c in sepchars),
-key=lambda n: (n < 0, n))
-if n < 0:
-yield ('string', unescape(tmpl[pos:stop]), pos)
-pos = stop
-break
-c = tmpl[n:n + 1]
-bs = 0  # count leading backslashes
-if not raw:
-bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
-if bs % 2 == 1:
-# escaped (e.g. '\{', '\\\{', but not '\\{')
-yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
-pos = n + 1
-continue
-if n > pos:
-yield ('string', unescape(tmpl[pos:n]), pos)
-if c == quote:
-yield ('end', None, n + 1)
-return
+try:
+while pos < stop:
+n = min((tmpl.find(c, pos, stop) for c in sepchars),
+key=lambda n: (n < 0, n))
+if n < 0:
+yield ('string', unescape(tmpl[pos:stop]), pos)
+pos = stop
+break
+c = tmpl[n:n + 1]
+bs = 0  # count leading backslashes
+if not raw:
+bs = (n - pos) - len(tmpl[pos:n].rstrip('\\'))
+if bs % 2 == 1:
+# escaped (e.g. '\{', '\\\{', but not '\\{')
+yield ('string', unescape(tmpl[pos:n - 1]) + c, pos)
+pos = n + 1
+continue
+if n > pos:
+yield ('string', unescape(tmpl[pos:n]), pos)
+if c == quote:
+yield ('end', None, n + 1)
+return
 
-parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
-if not tmpl.endswith('}', n + 1, pos):
-raise error.ParseError(_("invalid token"), pos)
-yield ('template', parseres, n)
+parseres, pos = p.parse(tokenize(tmpl, n + 1, stop, '}'))
+if not tmpl.endswith('}', n + 1, pos):
+raise error.ParseError(_("invalid token"), pos)
+yield ('template', parseres, n)
 
-if quote:
-raise error.ParseError(_("unterminated string"), start)
+if quote:
+raise error.ParseError(_("unterminated string"), start)
+except error.ParseError as inst:
+if len(inst.args) > 1:  # has location
+loc = inst.args[1]
+# We want the caret to 

D2607: hghave: fix up clang-libfuzzer regex to be bytes

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Fixes this check on Python 3.
  
  1. skip-blame just a b prefix

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2607

AFFECTED FILES
  tests/hghave.py

CHANGE DETAILS

diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -703,7 +703,7 @@
 
 @check("clang-libfuzzer", "clang new enough to include libfuzzer")
 def has_clang_libfuzzer():
-mat = matchoutput('clang --version', 'clang version (\d)')
+mat = matchoutput('clang --version', b'clang version (\d)')
 if mat:
 # libfuzzer is new in clang 6
 return int(mat.group(1)) > 5



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2604: tests: add tests about diff quality

[quark (Jun Wu)]

quark updated this revision to Diff 6481.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2604?vs=6468=6481

REVISION DETAIL
  https://phab.mercurial-scm.org/D2604

AFFECTED FILES
  tests/hghave.py
  tests/test-diff-antipatience.t
  tests/test-diff-indent-heuristic.t

CHANGE DETAILS

diff --git a/tests/test-diff-indent-heuristic.t 
b/tests/test-diff-indent-heuristic.t
new file mode 100644
--- /dev/null
+++ b/tests/test-diff-indent-heuristic.t
@@ -0,0 +1,339 @@
+#testcases bdiff xdiff
+
+#if xdiff
+#require xdiff
+  $ cat >> $HGRCPATH < [experimental]
+  > xdiff = true
+  > EOF
+#endif
+
+  $ hg init
+
+  $ cat > a.c <<'EOF'
+  > /*
+  >  * This function returns 1.
+  >  */
+  > int f() {
+  >   return 1;
+  > }
+  > /*
+  >  * This function returns 2.
+  >  */
+  > int g() {
+  >   return 2;
+  > }
+  > /*
+  >  * This function returns 3.
+  >  */
+  > int h() {
+  >   return 3;
+  > }
+  > EOF
+
+  $ cat > b.c <<'EOF'
+  > if (x) {
+  >do_something();
+  > }
+  > 
+  > if (y) {
+  >do_something_else();
+  > }
+  > EOF
+
+  $ cat > c.rb <<'EOF'
+  > #!ruby
+  > ["foo", "bar", "baz"].map do |i|
+  >   i.upcase
+  > end
+  > EOF
+
+  $ cat > d.py <<'EOF'
+  > try:
+  > import foo
+  > except ImportError:
+  > pass
+  > try:
+  > import bar
+  > except ImportError:
+  > pass
+  > EOF
+
+The below two files are taken from git: t/t4061-diff-indent.sh
+
+  $ cat > spaces.txt <<'EOF'
+  > 1
+  > 2
+  > a
+  > 
+  > b
+  > 3
+  > 4
+  > EOF
+
+  $ cat > functions.c <<'EOF'
+  > 1
+  > 2
+  > /* function */
+  > foo() {
+  > foo
+  > }
+  > 
+  > 3
+  > 4
+  > EOF
+
+  $ hg commit -m 1 -A . -q
+
+  $ cat > a.c <<'EOF'
+  > /*
+  >  * This function returns 1.
+  >  */
+  > int f() {
+  >   return 1;
+  > }
+  > /*
+  >  * This function returns 3.
+  >  */
+  > int h() {
+  >   return 3;
+  > }
+  > EOF
+
+  $ cat > b.c <<'EOF'
+  > if (x) {
+  >do_something();
+  > }
+  > 
+  > if (y) {
+  >do_another_thing();
+  > }
+  > 
+  > if (y) {
+  >do_something_else();
+  > }
+  > EOF
+
+  $ cat > c.rb <<'EOF'
+  > #!ruby
+  > ["foo", "bar", "baz"].map do |i|
+  >   i
+  > end
+  > ["foo", "bar", "baz"].map do |i|
+  >   i.upcase
+  > end
+  > EOF
+
+  $ cat > d.py <<'EOF'
+  > try:
+  > import foo
+  > except ImportError:
+  > pass
+  > try:
+  > import baz
+  > except ImportError:
+  > pass
+  > try:
+  > import bar
+  > except ImportError:
+  > pass
+  > EOF
+
+  $ cat > spaces.txt <<'EOF'
+  > 1
+  > 2
+  > a
+  > 
+  > b
+  > a
+  > 
+  > b
+  > 3
+  > 4
+  > EOF
+
+  $ cat > functions.c <<'EOF'
+  > 1
+  > 2
+  > /* function */
+  > bar() {
+  > foo
+  > }
+  > 
+  > /* function */
+  > foo() {
+  > foo
+  > }
+  > 
+  > 3
+  > 4
+  > EOF
+
+#if xdiff
+  $ hg diff --git
+  diff --git a/a.c b/a.c
+  --- a/a.c
+  +++ b/a.c
+  @@ -4,12 +4,6 @@
+   int f() {
+ return 1;
+   }
+  -/*
+  - * This function returns 2.
+  - */
+  -int g() {
+  -  return 2;
+  -}
+   /*
+* This function returns 3.
+*/
+  diff --git a/b.c b/b.c
+  --- a/b.c
+  +++ b/b.c
+  @@ -2,6 +2,10 @@
+  do_something();
+   }
+   
+  +if (y) {
+  +   do_another_thing();
+  +}
+  +
+   if (y) {
+  do_something_else();
+   }
+  diff --git a/c.rb b/c.rb
+  --- a/c.rb
+  +++ b/c.rb
+  @@ -1,4 +1,7 @@
+   #!ruby
+  +["foo", "bar", "baz"].map do |i|
+  +  i
+  +end
+   ["foo", "bar", "baz"].map do |i|
+ i.upcase
+   end
+  diff --git a/d.py b/d.py
+  --- a/d.py
+  +++ b/d.py
+  @@ -2,6 +2,10 @@
+   import foo
+   except ImportError:
+   pass
+  +try:
+  +import baz
+  +except ImportError:
+  +pass
+   try:
+   import bar
+   except ImportError:
+  diff --git a/functions.c b/functions.c
+  --- a/functions.c
+  +++ b/functions.c
+  @@ -1,5 +1,10 @@
+   1
+   2
+  +/* function */
+  +bar() {
+  +foo
+  +}
+  +
+   /* function */
+   foo() {
+   foo
+  diff --git a/spaces.txt b/spaces.txt
+  --- a/spaces.txt
+  +++ b/spaces.txt
+  @@ -2,6 +2,9 @@
+   2
+   a
+   
+  +b
+  +a
+  +
+   b
+   3
+   4
+#else
+  $ hg diff --git
+  diff --git a/a.c b/a.c
+  --- a/a.c
+  +++ b/a.c
+  @@ -5,12 +5,6 @@
+ return 1;
+   }
+   /*
+  - * This function returns 2.
+  - */
+  -int g() {
+  -  return 2;
+  -}
+  -/*
+* This function returns 3.
+*/
+   int h() {
+  diff --git a/b.c b/b.c
+  --- a/b.c
+  +++ b/b.c
+  @@ -3,5 +3,9 @@
+   }
+   
+   if (y) {
+  +   do_another_thing();
+  +}
+  +
+  +if (y) {
+  do_something_else();
+   }
+  diff --git a/c.rb b/c.rb
+  --- a/c.rb
+  +++ b/c.rb
+  @@ -1,4 +1,7 @@
+   #!ruby
+   ["foo", "bar", "baz"].map do |i|
+  +  i
+  +end
+  +["foo", "bar", "baz"].map do |i|
+ i.upcase
+   end
+  diff --git a/d.py b/d.py
+  --- a/d.py
+  +++ b/d.py
+  @@ -3,6 +3,10 @@
+   except ImportError:
+   pass
+   try:
+  +import baz
+  +except ImportError:
+  +pass
+  +try:
+   import bar
+   except ImportError:
+   pass
+  diff --git a/functions.c 

D2602: bdiff: add a xdiffblocks method

[quark (Jun Wu)]

quark updated this revision to Diff 6480.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2602?vs=6476=6480

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

AFFECTED FILES
  mercurial/cext/bdiff.c
  mercurial/policy.py
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -847,14 +847,33 @@
 if sys.platform == 'darwin':
 osutil_ldflags += ['-framework', 'ApplicationServices']
 
+xdiff_srcs = [
+'mercurial/thirdparty/xdiff/xdiffi.c',
+'mercurial/thirdparty/xdiff/xemit.c',
+'mercurial/thirdparty/xdiff/xmerge.c',
+'mercurial/thirdparty/xdiff/xprepare.c',
+'mercurial/thirdparty/xdiff/xutils.c',
+]
+
+xdiff_headers = [
+'mercurial/thirdparty/xdiff/xdiff.h',
+'mercurial/thirdparty/xdiff/xdiffi.h',
+'mercurial/thirdparty/xdiff/xemit.h',
+'mercurial/thirdparty/xdiff/xinclude.h',
+'mercurial/thirdparty/xdiff/xmacros.h',
+'mercurial/thirdparty/xdiff/xprepare.h',
+'mercurial/thirdparty/xdiff/xtypes.h',
+'mercurial/thirdparty/xdiff/xutils.h',
+]
+
 extmodules = [
 Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
 Extension('mercurial.cext.bdiff', ['mercurial/bdiff.c',
-   'mercurial/cext/bdiff.c'],
+   'mercurial/cext/bdiff.c'] + xdiff_srcs,
   include_dirs=common_include_dirs,
-  depends=common_depends + ['mercurial/bdiff.h']),
+  depends=common_depends + ['mercurial/bdiff.h'] + xdiff_headers),
 Extension('mercurial.cext.diffhelpers', ['mercurial/cext/diffhelpers.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
diff --git a/mercurial/policy.py b/mercurial/policy.py
--- a/mercurial/policy.py
+++ b/mercurial/policy.py
@@ -66,7 +66,7 @@
 # keep in sync with "version" in C modules
 _cextversions = {
 (r'cext', r'base85'): 1,
-(r'cext', r'bdiff'): 2,
+(r'cext', r'bdiff'): 3,
 (r'cext', r'diffhelpers'): 1,
 (r'cext', r'mpatch'): 1,
 (r'cext', r'osutil'): 3,
diff --git a/mercurial/cext/bdiff.c b/mercurial/cext/bdiff.c
--- a/mercurial/cext/bdiff.c
+++ b/mercurial/cext/bdiff.c
@@ -17,6 +17,7 @@
 
 #include "bdiff.h"
 #include "bitmanipulation.h"
+#include "thirdparty/xdiff/xdiff.h"
 #include "util.h"
 
 static PyObject *blocks(PyObject *self, PyObject *args)
@@ -229,18 +230,78 @@
return NULL;
 }
 
+static int hunk_consumer(long a1, long a2, long b1, long b2, void *priv)
+{
+   PyObject *rl = (PyObject *)priv;
+   PyObject *m = Py_BuildValue("", a1, a2, b1, b2);
+   if (!m)
+   return -1;
+   if (PyList_Append(rl, m) != 0) {
+   Py_DECREF(m);
+   return -1;
+   }
+   return 0;
+}
+
+static PyObject *xdiffblocks(PyObject *self, PyObject *args)
+{
+   Py_ssize_t la, lb;
+   mmfile_t a, b;
+   PyObject *rl;
+
+   xpparam_t xpp = {
+   XDF_INDENT_HEURISTIC, /* flags */
+   NULL, /* anchors */
+   0,/* anchors_nr */
+   };
+   xdemitconf_t xecfg = {
+   0,  /* ctxlen */
+   0,  /* interhunkctxlen */
+   XDL_EMIT_BDIFFHUNK, /* flags */
+   NULL,   /* find_func */
+   NULL,   /* find_func_priv */
+   hunk_consumer,  /* hunk_consume_func */
+   };
+   xdemitcb_t ecb = {
+   NULL, /* priv */
+   NULL, /* outf */
+   };
+
+   if (!PyArg_ParseTuple(args, PY23("s#s#", "y#y#"), , , ,
+ ))
+   return NULL;
+
+   a.size = la;
+   b.size = lb;
+
+   rl = PyList_New(0);
+   if (!rl)
+   return PyErr_NoMemory();
+
+   ecb.priv = rl;
+
+   if (xdl_diff(, , , , ) != 0) {
+   Py_DECREF(rl);
+   return PyErr_NoMemory();
+   }
+
+   return rl;
+}
+
 static char mdiff_doc[] = "Efficient binary diff.";
 
 static PyMethodDef methods[] = {
 {"bdiff", bdiff, METH_VARARGS, "calculate a binary diff\n"},
 {"blocks", blocks, METH_VARARGS, "find a list of matching lines\n"},
 {"fixws", fixws, METH_VARARGS, "normalize diff whitespaces\n"},
 {"splitnewlines", splitnewlines, METH_VARARGS,
  "like str.splitlines, but only split on newlines\n"},
+{"xdiffblocks", xdiffblocks, METH_VARARGS,
+ "find a list of matching lines using xdiff algorithm\n"},
 {NULL, NULL},
 };
 
-static const int version = 2;
+static const int version = 3;
 
 #ifdef IS_PY3K
 static struct PyModuleDef bdiff_module = {



To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

D2602: bdiff: add a xdiffblocks method

[quark (Jun Wu)]

quark added inline comments.

INLINE COMMENTS

> indygreg wrote in bdiff.c:239
> Calling `PyList_Append()` in tight loops can be a bit slow. It is faster to 
> allocate an array of `PyObject` and then allocate a `PyList` of final size 
> and call `PyList_SET_ITEM` to populate it. But we can optimize this later: 
> this is definitely the easiest first implementation.

Ideally, xdiff could return how many lines the input string has. So we can 
preallocate. But it's not in the current xdiff API.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2602: bdiff: add a xdiffblocks method

[quark (Jun Wu)]

quark added inline comments.

INLINE COMMENTS

> indygreg wrote in bdiff.c:263
> Does our C standard level allow to declare variables after non-declarations 
> in blocks?

Not sure. But the rest is C89. So I'll move this up.

> indygreg wrote in bdiff.c:283
> This exception type is nonsensical. But it is what `blocks()` uses. So not 
> worth worrying about.

It actually makes some sense - if the xdiff library is solid, then 
`hunk_consumer` returning -1 is the only reason things go wrong. And that's 
caused by NoMemory.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2604: tests: add tests about diff quality

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  Can you please fix the Python 3 compatibility issues? If you want to push 
back, we can do it as a follow-up.

INLINE COMMENTS

> test-diff-antipatience.t:14
> +  $ hg init
> +  >>> open('a', 'w').write('\n'.join(list('a' + 'x' * 10 + 'u' + 'x' * 30 + 
> 'a\n')))
> +  $ hg commit -m 1 -A a

This should be `'wb'` for Python 3 compatibility. And it should use `b''` 
literals.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2604

To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] py3: accept both unicode and byte strings as filename carried by IOError

[Augie Fackler]

On Sat, Mar 03, 2018 at 03:52:41PM -0500, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1520109672 18000
> #  Sat Mar 03 15:41:12 2018 -0500
> # Node ID 1880fc2cbfc27c8b2fb0073f8aa9c0fa54937969
> # Parent  8e3fd6529bad7f4d524f160761c766a7f5267a0f
> py3: accept both unicode and byte strings as filename carried by IOError

queued, thanks
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2573: xdiff: remove patience and histogram diff algorithms

[quark (Jun Wu)]

quark added a comment.


  Googling "patience diff", most results will say it is slower but has better 
diff quality sometimes.
  
  That is very misleading - the algorithm adds greediness (i.e. incorrectness) 
and is in theory faster in some cases. The "better" quality is also untrue 
comparing to indent heuristic.
  
  Technically, I don't think it's worthwhile to keep - definitely worse quality 
and unpredictable performance changes comparing to default diff + indent 
heuristic setup.
  
  From a non-technical point of view, I think educating people that they cannot 
be misled is very important. This one alone is an enough reason to do the 
cleanup, imo.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2573

To: ryanmce, #hg-reviewers, indygreg
Cc: quark, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2602: bdiff: add a xdiffblocks method

[indygreg (Gregory Szorc)]

indygreg accepted this revision.
indygreg added inline comments.
This revision is now accepted and ready to land.

INLINE COMMENTS

> bdiff.c:239
> + return -1;
> + if (PyList_Append(rl, m) != 0) {
> + Py_DECREF(m);

Calling `PyList_Append()` in tight loops can be a bit slow. It is faster to 
allocate an array of `PyObject` and then allocate a `PyList` of final size and 
call `PyList_SET_ITEM` to populate it. But we can optimize this later: this is 
definitely the easiest first implementation.

> bdiff.c:263
> +
> + xpparam_t xpp = {
> + XDF_INDENT_HEURISTIC, /* flags */

Does our C standard level allow to declare variables after non-declarations in 
blocks?

> bdiff.c:283
> + Py_DECREF(rl);
> + return PyErr_NoMemory();
> + }

This exception type is nonsensical. But it is what `blocks()` uses. So not 
worth worrying about.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@36607: 28 new changesets

[Mercurial Commits]

28 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/04e50037d957
changeset:   36580:04e50037d957
user:Augie Fackler 
date:Fri Mar 02 11:37:21 2018 -0500
summary: revset: use {force,}bytestr to fix some %r formatting issues

https://www.mercurial-scm.org/repo/hg/rev/b755eab7e677
changeset:   36581:b755eab7e677
user:Augie Fackler 
date:Fri Mar 02 11:37:41 2018 -0500
summary: revset: use %d to turn an int into a bytestr

https://www.mercurial-scm.org/repo/hg/rev/27911e0fb50e
changeset:   36582:27911e0fb50e
user:Augie Fackler 
date:Fri Mar 02 11:38:16 2018 -0500
summary: tests: add missing b prefixes and fix a %s to %d in test-revset.t

https://www.mercurial-scm.org/repo/hg/rev/edf736c3c064
changeset:   36583:edf736c3c064
user:Augie Fackler 
date:Fri Mar 02 13:47:49 2018 -0500
summary: tests: add some re and globs for test-revset on python3

https://www.mercurial-scm.org/repo/hg/rev/f5427483eebe
changeset:   36584:f5427483eebe
user:Augie Fackler 
date:Fri Mar 02 14:09:20 2018 -0500
summary: util: add missing r prefix on some __setattr__ calls

https://www.mercurial-scm.org/repo/hg/rev/26a6b62919e2
changeset:   36585:26a6b62919e2
user:Augie Fackler 
date:Fri Mar 02 14:09:50 2018 -0500
summary: util: work around Python 3 returning None at EOF instead of ''

https://www.mercurial-scm.org/repo/hg/rev/ff6b8484400b
changeset:   36586:ff6b8484400b
user:Augie Fackler 
date:Fri Mar 02 14:10:34 2018 -0500
summary: debugcommands: add an r prefix to make file mode for fdopen a 
sysstr

https://www.mercurial-scm.org/repo/hg/rev/dfcf589a4031
changeset:   36587:dfcf589a4031
user:Augie Fackler 
date:Fri Mar 02 14:12:17 2018 -0500
summary: debugcommands: add some strkwargs love to some **args calls

https://www.mercurial-scm.org/repo/hg/rev/281f66777ff0
changeset:   36588:281f66777ff0
user:Yuya Nishihara 
date:Thu Mar 01 18:22:36 2018 -0500
summary: py3: silence "bad escape" warning emitted by re.sub()

https://www.mercurial-scm.org/repo/hg/rev/3790610c2793
changeset:   36589:3790610c2793
user:Kevin Bullock 
date:Fri Mar 02 15:37:57 2018 -0500
summary: py3: bail on ratcheting tests forward on 3.6.0 and 3.6.1

https://www.mercurial-scm.org/repo/hg/rev/0083e373e5f5
changeset:   36590:0083e373e5f5
user:Yuya Nishihara 
date:Sun Feb 25 16:22:55 2018 +0900
summary: templatekw: switch showdict template keywords to new API

https://www.mercurial-scm.org/repo/hg/rev/121a20e5da56
changeset:   36591:121a20e5da56
user:Yuya Nishihara 
date:Sun Feb 25 16:45:44 2018 +0900
summary: templatekw: switch most of showlist template keywords to new API 
(issue5779)

https://www.mercurial-scm.org/repo/hg/rev/b0054f3c055a
changeset:   36592:b0054f3c055a
user:Yuya Nishihara 
date:Sun Feb 25 18:56:06 2018 +0900
summary: namespace: use registrar to add template keyword

https://www.mercurial-scm.org/repo/hg/rev/900e5ee44307
changeset:   36593:900e5ee44307
user:Yuya Nishihara 
date:Sun Feb 25 18:52:51 2018 +0900
summary: templatekw: switch namespace template keywords to new API

https://www.mercurial-scm.org/repo/hg/rev/59ee648870a7
changeset:   36594:59ee648870a7
user:Yuya Nishihara 
date:Sun Feb 25 19:05:57 2018 +0900
summary: templatekw: switch obsfate-related template keywords to new API

https://www.mercurial-scm.org/repo/hg/rev/2da414105809
changeset:   36595:2da414105809
user:Yuya Nishihara 
date:Sun Feb 25 19:08:02 2018 +0900
summary: templatekw: switch revset() to new API

https://www.mercurial-scm.org/repo/hg/rev/b5d39a09656a
changeset:   36596:b5d39a09656a
user:Yuya Nishihara 
date:Sun Feb 25 19:23:06 2018 +0900
summary: templatekw: switch latesttags template keywords to new API

https://www.mercurial-scm.org/repo/hg/rev/d57f383516f6
changeset:   36597:d57f383516f6
user:Yuya Nishihara 
date:Sun Feb 25 20:55:53 2018 +0900
summary: templatekw: switch manifest template keyword to new API

https://www.mercurial-scm.org/repo/hg/rev/c3f9d0c303e8
changeset:   36598:c3f9d0c303e8
user:Yuya Nishihara 
date:Sun Feb 25 19:25:14 2018 +0900
summary: templatekw: switch remainder of _showlist template keywords to new 
API

https://www.mercurial-scm.org/repo/hg/rev/c9dcc379e2dc
changeset:   36599:c9dcc379e2dc
user:Yuya Nishihara 
date:Sun Feb 25 19:34:35 2018 +0900
summary: templatekw: deprecate showdict() 

D2606: largefiles: headers and values need to be sysstrs, add r prefixes

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5c4c9eb1feb6: largefiles: headers and values need to be 
sysstrs, add r prefixes (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2606?vs=6477=6479

REVISION DETAIL
  https://phab.mercurial-scm.org/D2606

AFFECTED FILES
  hgext/largefiles/proto.py

CHANGE DETAILS

diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
--- a/hgext/largefiles/proto.py
+++ b/hgext/largefiles/proto.py
@@ -97,7 +97,7 @@
 # it ...
 if issubclass(self.__class__, httppeer.httppeer):
 res = self._call('putlfile', data=fd, sha=sha,
-headers={'content-type':'application/mercurial-0.1'})
+headers={r'content-type': r'application/mercurial-0.1'})
 try:
 d, output = res.split('\n', 1)
 for l in output.splitlines(True):



To: durin42, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] py3: back out c77c925987d7 to store bytes filename in IOError

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1520109232 18000
#  Sat Mar 03 15:33:52 2018 -0500
# Node ID 8e3fd6529bad7f4d524f160761c766a7f5267a0f
# Parent  68026dd7c4f9bf1462c15a2a307d4bb253a0959f
py3: back out c77c925987d7 to store bytes filename in IOError

Appears that this is how Python 3 works.

diff --git a/mercurial/lock.py b/mercurial/lock.py
--- a/mercurial/lock.py
+++ b/mercurial/lock.py
@@ -200,10 +200,9 @@ class lock(object):
 return
 locker = self._testlock(locker)
 if locker is not None:
-raise error.LockHeld(
-errno.EAGAIN,
-encoding.strfromlocal(self.vfs.join(self.f)),
-self.desc, locker)
+raise error.LockHeld(errno.EAGAIN,
+ self.vfs.join(self.f), self.desc,
+ locker)
 else:
 raise error.LockUnavailable(why.errno, why.strerror,
 why.filename, self.desc)
@@ -212,8 +211,7 @@ class lock(object):
 # use empty locker to mean "busy for frequent lock/unlock
 # by many processes"
 raise error.LockHeld(errno.EAGAIN,
- encoding.strfromlocal(self.vfs.join(self.f)),
- self.desc, "")
+ self.vfs.join(self.f), self.desc, "")
 
 def _readlock(self):
 """read lock and return its value
diff --git a/mercurial/statichttprepo.py b/mercurial/statichttprepo.py
--- a/mercurial/statichttprepo.py
+++ b/mercurial/statichttprepo.py
@@ -206,7 +206,7 @@ class statichttprepository(localrepo.loc
 return statichttppeer(self)
 
 def wlock(self, wait=True):
-raise error.LockUnavailable(0, _('lock not available'), r'lock',
+raise error.LockUnavailable(0, _('lock not available'), 'lock',
 _('cannot lock static-http repository'))
 
 def lock(self, wait=True):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] py3: accept both unicode and byte strings as filename carried by IOError

[Yuya Nishihara]

# HG changeset patch
# User Yuya Nishihara 
# Date 1520109672 18000
#  Sat Mar 03 15:41:12 2018 -0500
# Node ID 1880fc2cbfc27c8b2fb0073f8aa9c0fa54937969
# Parent  8e3fd6529bad7f4d524f160761c766a7f5267a0f
py3: accept both unicode and byte strings as filename carried by IOError

Follows up 77f98867538f. We could assume there's no bytes filename in our
codebase, but it's probably better to not raise UnicodeError because of
a unicode filename.

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -163,12 +163,12 @@ def callcatch(ui, func):
 else:
 reason = _('lock held by %r') % inst.locker
 ui.warn(_("abort: %s: %s\n")
-% (inst.desc or encoding.strtolocal(inst.filename), reason))
+% (inst.desc or util.forcebytestr(inst.filename), reason))
 if not inst.locker:
 ui.warn(_("(lock might be very busy)\n"))
 except error.LockUnavailable as inst:
 ui.warn(_("abort: could not lock %s: %s\n") %
-(inst.desc or encoding.strtolocal(inst.filename),
+(inst.desc or util.forcebytestr(inst.filename),
  encoding.strtolocal(inst.strerror)))
 except error.OutOfBandError as inst:
 if inst.args:
@@ -234,7 +234,7 @@ def callcatch(ui, func):
 if getattr(inst, "filename", None):
 ui.warn(_("abort: %s: %s\n") % (
 encoding.strtolocal(inst.strerror),
-encoding.strtolocal(inst.filename)))
+util.forcebytestr(inst.filename)))
 else:
 ui.warn(_("abort: %s\n") % encoding.strtolocal(inst.strerror))
 else:
@@ -243,7 +243,7 @@ def callcatch(ui, func):
 if getattr(inst, "filename", None) is not None:
 ui.warn(_("abort: %s: '%s'\n") % (
 encoding.strtolocal(inst.strerror),
-encoding.strtolocal(inst.filename)))
+util.forcebytestr(inst.filename)))
 else:
 ui.warn(_("abort: %s\n") % encoding.strtolocal(inst.strerror))
 except MemoryError:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2573: xdiff: remove patience and histogram diff algorithms

[indygreg (Gregory Szorc)]

indygreg accepted this revision as: indygreg.
indygreg added a comment.


  I'm leaning towards keeping these algorithms so we can expose them as 
alternate implementations in the future. It will also making syncing code from 
upstream easier. But removing the unused-by-us code is fine.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2573

To: ryanmce, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2572: xdiff: vendor xdiff library from git

[indygreg (Gregory Szorc)]

indygreg accepted this revision as: indygreg.
indygreg added a comment.


  I'm OK taking this provided we set up fuzzers in contrib/fuzz. We can do that 
post landing though.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2572

To: ryanmce, #hg-reviewers, indygreg
Cc: indygreg, quark, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2605: run-tests: allow #require inside #if

[quark (Jun Wu)]

quark updated this revision to Diff 6478.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2605?vs=6467=6478

REVISION DETAIL
  https://phab.mercurial-scm.org/D2605

AFFECTED FILES
  tests/run-tests.py
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -921,16 +921,24 @@
 
   $ cat > test-skip.t <   $ echo xyzzy
+  > #if true
   > #require false
+  > #end
+  > EOF
+  $ cat > test-noskip.t < #if false
+  > #require false
+  > #endif
   > EOF
   $ rt --nodiff
-  !.s
+  !.s.
   Skipped test-skip.t: missing feature: nail clipper
   Failed test-failure.t: output changed
-  # Ran 2 tests, 1 skipped, 1 failed.
+  # Ran 3 tests, 1 skipped, 1 failed.
   python hash seed: * (glob)
   [1]
 
+  $ rm test-noskip.t
   $ rt --keyword xyzzy
   .s
   Skipped test-skip.t: missing feature: nail clipper
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -1371,10 +1371,11 @@
 lsplit = l.split()
 if len(lsplit) < 2 or lsplit[0] != b'#require':
 after.setdefault(pos, []).append('  !!! invalid 
#require\n')
-haveresult, message = self._hghave(lsplit[1:])
-if not haveresult:
-script = [b'echo "%s"\nexit 80\n' % message]
-break
+if not skipping:
+haveresult, message = self._hghave(lsplit[1:])
+if not haveresult:
+script = [b'echo "%s"\nexit 80\n' % message]
+break
 after.setdefault(pos, []).append(l)
 elif l.startswith(b'#if'):
 lsplit = l.split()



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2606: largefiles: headers and values need to be sysstrs, add r prefixes

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  1. skip-blame just some r prefixes

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2606

AFFECTED FILES
  hgext/largefiles/proto.py

CHANGE DETAILS

diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
--- a/hgext/largefiles/proto.py
+++ b/hgext/largefiles/proto.py
@@ -97,7 +97,7 @@
 # it ...
 if issubclass(self.__class__, httppeer.httppeer):
 res = self._call('putlfile', data=fd, sha=sha,
-headers={'content-type':'application/mercurial-0.1'})
+headers={r'content-type': r'application/mercurial-0.1'})
 try:
 d, output = res.split('\n', 1)
 for l in output.splitlines(True):



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2601: xdiff: reduce indent heuristic overhead

[quark (Jun Wu)]

quark updated this revision to Diff 6475.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2601?vs=6458=6475

REVISION DETAIL
  https://phab.mercurial-scm.org/D2601

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiffi.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xdiffi.c 
b/mercurial/thirdparty/xdiff/xdiffi.c
--- a/mercurial/thirdparty/xdiff/xdiffi.c
+++ b/mercurial/thirdparty/xdiff/xdiffi.c
@@ -801,6 +801,14 @@
exit(1);
 }
 
+/*
+ * For indentation heuristic, skip searching for better slide position after
+ * checking MAX_BORING lines without finding an improvement. This defends the
+ * indentation heuristic logic against pathological cases. The value is not
+ * picked scientifically but should be good enough.
+ */
+#define MAX_BORING 100
+
 /*
  * Move back and forward change groups for a consistent and pretty diff output.
  * This also helps in finding joinable change groups and reducing the diff
@@ -897,19 +905,43 @@
long shift, best_shift = -1;
struct split_score best_score;
 
-   for (shift = earliest_end; shift <= g.end; shift++) {
+   /*
+* This is O(N * MAX_BLANKS) (N = shift-able lines).
+* Even with MAX_BLANKS bounded to a small value, a
+* large N could still make this loop take several
+* times longer than the main diff algorithm. The
+* "boring" value is to help cut down N to something
+* like (MAX_BORING + groupsize).
+*
+* Scan from bottom to top. So we can exit the loop
+* without compromising the assumption "for a same best
+* score, pick the bottommost shift".
+*/
+   int boring = 0;
+   for (shift = g.end; shift >= earliest_end; shift--) {
struct split_measurement m;
struct split_score score = {0, 0};
+   int cmp;
 
measure_split(xdf, shift, );
score_add_split(, );
measure_split(xdf, shift - groupsize, );
score_add_split(, );
-   if (best_shift == -1 ||
-   score_cmp(, _score) <= 0) {
+
+   if (best_shift == -1) {
+   cmp = -1;
+   } else {
+   cmp = score_cmp(, _score);
+   }
+   if (cmp < 0) {
+   boring = 0;
best_score.effective_indent = 
score.effective_indent;
best_score.penalty = score.penalty;
best_shift = shift;
+   } else {
+   boring += 1;
+   if (boring >= MAX_BORING)
+   break;
}
}
 



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2602: bdiff: add a xdiffblocks method

[quark (Jun Wu)]

quark updated this revision to Diff 6476.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2602?vs=6459=6476

REVISION DETAIL
  https://phab.mercurial-scm.org/D2602

AFFECTED FILES
  mercurial/cext/bdiff.c
  mercurial/policy.py
  setup.py

CHANGE DETAILS

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -847,14 +847,33 @@
 if sys.platform == 'darwin':
 osutil_ldflags += ['-framework', 'ApplicationServices']
 
+xdiff_srcs = [
+'mercurial/thirdparty/xdiff/xdiffi.c',
+'mercurial/thirdparty/xdiff/xemit.c',
+'mercurial/thirdparty/xdiff/xmerge.c',
+'mercurial/thirdparty/xdiff/xprepare.c',
+'mercurial/thirdparty/xdiff/xutils.c',
+]
+
+xdiff_headers = [
+'mercurial/thirdparty/xdiff/xdiff.h',
+'mercurial/thirdparty/xdiff/xdiffi.h',
+'mercurial/thirdparty/xdiff/xemit.h',
+'mercurial/thirdparty/xdiff/xinclude.h',
+'mercurial/thirdparty/xdiff/xmacros.h',
+'mercurial/thirdparty/xdiff/xprepare.h',
+'mercurial/thirdparty/xdiff/xtypes.h',
+'mercurial/thirdparty/xdiff/xutils.h',
+]
+
 extmodules = [
 Extension('mercurial.cext.base85', ['mercurial/cext/base85.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
 Extension('mercurial.cext.bdiff', ['mercurial/bdiff.c',
-   'mercurial/cext/bdiff.c'],
+   'mercurial/cext/bdiff.c'] + xdiff_srcs,
   include_dirs=common_include_dirs,
-  depends=common_depends + ['mercurial/bdiff.h']),
+  depends=common_depends + ['mercurial/bdiff.h'] + xdiff_headers),
 Extension('mercurial.cext.diffhelpers', ['mercurial/cext/diffhelpers.c'],
   include_dirs=common_include_dirs,
   depends=common_depends),
diff --git a/mercurial/policy.py b/mercurial/policy.py
--- a/mercurial/policy.py
+++ b/mercurial/policy.py
@@ -66,7 +66,7 @@
 # keep in sync with "version" in C modules
 _cextversions = {
 (r'cext', r'base85'): 1,
-(r'cext', r'bdiff'): 2,
+(r'cext', r'bdiff'): 3,
 (r'cext', r'diffhelpers'): 1,
 (r'cext', r'mpatch'): 1,
 (r'cext', r'osutil'): 3,
diff --git a/mercurial/cext/bdiff.c b/mercurial/cext/bdiff.c
--- a/mercurial/cext/bdiff.c
+++ b/mercurial/cext/bdiff.c
@@ -17,6 +17,7 @@
 
 #include "bdiff.h"
 #include "bitmanipulation.h"
+#include "thirdparty/xdiff/xdiff.h"
 #include "util.h"
 
 static PyObject *blocks(PyObject *self, PyObject *args)
@@ -229,18 +230,76 @@
return NULL;
 }
 
+static int hunk_consumer(long a1, long a2, long b1, long b2, void *priv)
+{
+   PyObject *rl = (PyObject *)priv;
+   PyObject *m = Py_BuildValue("", a1, a2, b1, b2);
+   if (!m)
+   return -1;
+   if (PyList_Append(rl, m) != 0) {
+   Py_DECREF(m);
+   return -1;
+   }
+   return 0;
+}
+
+static PyObject *xdiffblocks(PyObject *self, PyObject *args)
+{
+   Py_ssize_t la, lb;
+   mmfile_t a, b;
+   PyObject *rl;
+
+   if (!PyArg_ParseTuple(args, PY23("s#s#", "y#y#"), , , ,
+ ))
+   return NULL;
+
+   a.size = la;
+   b.size = lb;
+
+   rl = PyList_New(0);
+   if (!rl)
+   return PyErr_NoMemory();
+
+   xpparam_t xpp = {
+   XDF_INDENT_HEURISTIC, /* flags */
+   NULL, /* anchors */
+   0,/* anchors_nr */
+   };
+   xdemitconf_t xecfg = {
+   0,  /* ctxlen */
+   0,  /* interhunkctxlen */
+   XDL_EMIT_BDIFFHUNK, /* flags */
+   NULL,   /* find_func */
+   NULL,   /* find_func_priv */
+   hunk_consumer,  /* hunk_consume_func */
+   };
+   xdemitcb_t ecb = {
+   rl,   /* priv */
+   NULL, /* outf */
+   };
+
+   if (xdl_diff(, , , , ) != 0) {
+   Py_DECREF(rl);
+   return PyErr_NoMemory();
+   }
+
+   return rl;
+}
+
 static char mdiff_doc[] = "Efficient binary diff.";
 
 static PyMethodDef methods[] = {
 {"bdiff", bdiff, METH_VARARGS, "calculate a binary diff\n"},
 {"blocks", blocks, METH_VARARGS, "find a list of matching lines\n"},
 {"fixws", fixws, METH_VARARGS, "normalize diff whitespaces\n"},
 {"splitnewlines", splitnewlines, METH_VARARGS,
  "like str.splitlines, but only split on newlines\n"},
+{"xdiffblocks", xdiffblocks, METH_VARARGS,
+ "find a list of matching lines using xdiff algorithm\n"},
 {NULL, NULL},
 };
 
-static const int version = 2;
+static const int version = 3;
 
 #ifdef IS_PY3K
 static struct PyModuleDef bdiff_module = {



To: quark, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2573: xdiff: remove patience and histogram diff algorithms

[quark (Jun Wu)]

quark updated this revision to Diff 6473.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2573?vs=6456=6473

REVISION DETAIL
  https://phab.mercurial-scm.org/D2573

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiff.h
  mercurial/thirdparty/xdiff/xdiffi.c
  mercurial/thirdparty/xdiff/xhistogram.c
  mercurial/thirdparty/xdiff/xpatience.c
  mercurial/thirdparty/xdiff/xprepare.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -27,7 +27,6 @@
 #define XDL_MAX_EQLIMIT 1024
 #define XDL_SIMSCAN_WINDOW 100
 #define XDL_GUESS_NLINES1 256
-#define XDL_GUESS_NLINES2 20
 
 
 typedef struct s_xdlclass {
@@ -181,9 +180,7 @@
if (!(recs = (xrecord_t **) xdl_malloc(narec * sizeof(xrecord_t *
goto abort;
 
-   if (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF)
-   hbits = hsize = 0;
-   else {
+   {
hbits = xdl_hashbits((unsigned int) narec);
hsize = 1 << hbits;
if (!(rhash = (xrecord_t **) xdl_malloc(hsize * 
sizeof(xrecord_t *
@@ -209,8 +206,7 @@
crec->ha = hav;
recs[nrec++] = crec;
 
-   if ((XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&
-   xdl_classify_record(pass, cf, rhash, hbits, crec) < 
0)
+   if (xdl_classify_record(pass, cf, rhash, hbits, crec) < 
0)
goto abort;
}
}
@@ -266,21 +262,12 @@
 
memset(, 0, sizeof(cf));
 
-   /*
-* For histogram diff, we can afford a smaller sample size and
-* thus a poorer estimate of the number of lines, as the hash
-* table (rhash) won't be filled up/grown. The number of lines
-* (nrecs) will be updated correctly anyway by
-* xdl_prepare_ctx().
-*/
-   sample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF
- ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);
+   sample = XDL_GUESS_NLINES1;
 
enl1 = xdl_guess_lines(mf1, sample) + 1;
enl2 = xdl_guess_lines(mf2, sample) + 1;
 
-   if (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF &&
-   xdl_init_classifier(, enl1 + enl2 + 1, xpp->flags) < 0)
+   if (xdl_init_classifier(, enl1 + enl2 + 1, xpp->flags) < 0)
return -1;
 
if (xdl_prepare_ctx(1, mf1, enl1, xpp, , >xdf1) < 0) {
@@ -295,18 +282,14 @@
return -1;
}
 
-   if ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&
-   (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&
-   xdl_optimize_ctxs(, >xdf1, >xdf2) < 0) {
-
+   if (xdl_optimize_ctxs(, >xdf1, >xdf2) < 0) {
xdl_free_ctx(>xdf2);
xdl_free_ctx(>xdf1);
xdl_free_classifier();
return -1;
}
 
-   if (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)
-   xdl_free_classifier();
+   xdl_free_classifier();
 
return 0;
 }
diff --git a/mercurial/thirdparty/xdiff/xpatience.c 
b/mercurial/thirdparty/xdiff/xpatience.c
deleted file mode 100644
--- a/mercurial/thirdparty/xdiff/xpatience.c
+++ /dev/null
@@ -1,390 +0,0 @@
-/*
- *  LibXDiff by Davide Libenzi ( File Differential Library )
- *  Copyright (C) 2003-2016 Davide Libenzi, Johannes E. Schindelin
- *
- *  This library is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU Lesser General Public
- *  License as published by the Free Software Foundation; either
- *  version 2.1 of the License, or (at your option) any later version.
- *
- *  This library is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- *  Lesser General Public License for more details.
- *
- *  You should have received a copy of the GNU Lesser General Public
- *  License along with this library; if not, see
- *  .
- *
- *  Davide Libenzi 
- *
- */
-#include "xinclude.h"
-#include "xtypes.h"
-#include "xdiff.h"
-
-/*
- * The basic idea of patience diff is to find lines that are unique in
- * both files.  These are intuitively the ones that we want to see as
- * common lines.
- *
- * The maximal ordered sequence of such line pairs (where ordered means
- * that the order in the sequence agrees with the order of the lines in
- * both files) naturally defines an initial set of common lines.
- *
- * Now, the algorithm tries to extend the set of common lines by growing
- * the line ranges where the files have identical lines.
- *
- * Between those common lines, the patience diff algorithm is applied
- * recursively, until no unique line pairs can be found; these line 

D2575: xdiff: add a bdiff hunk mode

[quark (Jun Wu)]

quark updated this revision to Diff 6474.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2575?vs=6457=6474

REVISION DETAIL
  https://phab.mercurial-scm.org/D2575

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiff.h
  mercurial/thirdparty/xdiff/xdiffi.c

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xdiffi.c 
b/mercurial/thirdparty/xdiff/xdiffi.c
--- a/mercurial/thirdparty/xdiff/xdiffi.c
+++ b/mercurial/thirdparty/xdiff/xdiffi.c
@@ -975,15 +975,32 @@
  xdemitconf_t const *xecfg)
 {
xdchange_t *xch, *xche;
-
-   for (xch = xscr; xch; xch = xche->next) {
-   xche = xdl_get_hunk(, xecfg);
-   if (!xch)
-   break;
-   if (xecfg->hunk_func(xch->i1, xche->i1 + xche->chg1 - xch->i1,
-xch->i2, xche->i2 + xche->chg2 - xch->i2,
-ecb->priv) < 0)
+   if ((xecfg->flags & XDL_EMIT_BDIFFHUNK) != 0) {
+   long i1 = 0, i2 = 0, n1 = xe->xdf1.nrec, n2 = xe->xdf2.nrec;
+   for (xch = xscr; xch; xch = xche->next) {
+   xche = xdl_get_hunk(, xecfg);
+   if (!xch)
+   break;
+   if (xch->i1 > i1 || xch->i2 > i2) {
+   if (xecfg->hunk_func(i1, xch->i1, i2, xch->i2, 
ecb->priv) < 0)
+   return -1;
+   }
+   i1 = xche->i1 + xche->chg1;
+   i2 = xche->i2 + xche->chg2;
+   }
+   if (xecfg->hunk_func(i1, n1, i2, n2, ecb->priv) < 0)
return -1;
+   } else {
+   for (xch = xscr; xch; xch = xche->next) {
+   xche = xdl_get_hunk(, xecfg);
+   if (!xch)
+   break;
+   if (xecfg->hunk_func(
+   xch->i1, xche->i1 + xche->chg1 - 
xch->i1,
+   xch->i2, xche->i2 + xche->chg2 - 
xch->i2,
+   ecb->priv) < 0)
+   return -1;
+   }
}
return 0;
 }
@@ -1026,18 +1043,15 @@
xdl_free_env();
return -1;
}
-   if (xscr) {
-   if (xpp->flags & XDF_IGNORE_BLANK_LINES)
-   xdl_mark_ignorable(xscr, , xpp->flags);
-
-   if (ef(, xscr, ecb, xecfg) < 0) {
 
-   xdl_free_script(xscr);
-   xdl_free_env();
-   return -1;
-   }
+   if (xpp->flags & XDF_IGNORE_BLANK_LINES)
+   xdl_mark_ignorable(xscr, , xpp->flags);
+   if (ef(, xscr, ecb, xecfg) < 0) {
xdl_free_script(xscr);
+   xdl_free_env();
+   return -1;
}
+   xdl_free_script(xscr);
xdl_free_env();
 
return 0;
diff --git a/mercurial/thirdparty/xdiff/xdiff.h 
b/mercurial/thirdparty/xdiff/xdiff.h
--- a/mercurial/thirdparty/xdiff/xdiff.h
+++ b/mercurial/thirdparty/xdiff/xdiff.h
@@ -48,6 +48,9 @@
 /* xdemitconf_t.flags */
 #define XDL_EMIT_FUNCNAMES (1 << 0)
 #define XDL_EMIT_FUNCCONTEXT (1 << 2)
+/* emit bdiff-style "matched" (a1, a2, b1, b2) hunks instead of "different"
+ * (a1, a2 - a1, b1, b2 - b1) hunks */
+#define XDL_EMIT_BDIFFHUNK (1 << 4)
 
 #define XDL_MMB_READONLY (1 << 0)
 



To: ryanmce, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2068: revlog: do not use delta for lfs revisions

[quark (Jun Wu)]

quark updated this revision to Diff 6470.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2068?vs=5651=6470

REVISION DETAIL
  https://phab.mercurial-scm.org/D2068

AFFECTED FILES
  mercurial/revlog.py
  tests/test-lfs-bundle.t
  tests/test-revlog-raw.py

CHANGE DETAILS

diff --git a/tests/test-revlog-raw.py b/tests/test-revlog-raw.py
--- a/tests/test-revlog-raw.py
+++ b/tests/test-revlog-raw.py
@@ -114,6 +114,8 @@
 else:
 # suboptimal deltaparent
 deltaparent = min(0, parentrev)
+if not rlog.candelta(deltaparent, r):
+deltaparent = -1
 return {'node': rlog.node(r), 'p1': pnode, 'p2': node.nullid,
 'cs': rlog.node(rlog.linkrev(r)), 'flags': rlog.flags(r),
 'deltabase': rlog.node(deltaparent),
@@ -151,12 +153,12 @@
 for r in rlog:
 p1 = rlog.node(r - 1)
 p2 = node.nullid
-if r == 0:
+if r == 0 or rlog.flags(r):
 text = rlog.revision(r, raw=True)
 cachedelta = None
 else:
-# deltaparent is more interesting if it has the EXTSTORED flag.
-deltaparent = max([0] + [p for p in range(r - 2) if rlog.flags(p)])
+# deltaparent cannot have EXTSTORED flag.
+deltaparent = max([-1] + [p for p in range(r) if not 
rlog.flags(p)])
 text = None
 cachedelta = (deltaparent, rlog.revdiff(deltaparent, r))
 flags = rlog.flags(r)
@@ -262,8 +264,9 @@
 result.append((text, rawtext))
 
 # Verify flags like isdelta, isext work as expected
-if bool(rlog.deltaparent(rev) > -1) != isdelta:
-abort('rev %d: isdelta is ineffective' % rev)
+# isdelta can be overridden to False if this or p1 has isext set
+if bool(rlog.deltaparent(rev) > -1) and not isdelta:
+abort('rev %d: isdelta is unexpected' % rev)
 if bool(rlog.flags(rev)) != isext:
 abort('rev %d: isext is ineffective' % rev)
 return result
diff --git a/tests/test-lfs-bundle.t b/tests/test-lfs-bundle.t
--- a/tests/test-lfs-bundle.t
+++ b/tests/test-lfs-bundle.t
@@ -90,10 +90,7 @@
    Applying src-normal.bundle to dst-normal 
   OK
    Applying src-normal.bundle to dst-lfs 
-   X@2: unpacking bcc7d23fa6b7: integrity check failed on data/X.i:2
-   Y@2: unpacking 46017a6640e7: integrity check failed on data/Y.i:2
-  2 integrity errors encountered!
-  (first damaged changeset appears to be 2)
+  CRASHED
    Applying src-lfs.bundle to dst-normal 
   OK
    Applying src-lfs.bundle to dst-lfs 
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -407,6 +407,9 @@
 for candidaterevs in self._getcandidaterevs(p1, p2, cachedelta):
 nominateddeltas = []
 for candidaterev in candidaterevs:
+# no delta for rawtext-changing revs (see "candelta" for why)
+if revlog.flags(candidaterev) & REVIDX_RAWTEXT_CHANGING_FLAGS:
+continue
 candidatedelta = self._builddeltainfo(revinfo, candidaterev, 
fh)
 if revlog._isgooddeltainfo(candidatedelta, revinfo.textlen):
 nominateddeltas.append(candidatedelta)
@@ -2101,7 +2104,14 @@
 deltacomputer = _deltacomputer(self)
 
 revinfo = _revisioninfo(node, p1, p2, btext, textlen, cachedelta, 
flags)
-deltainfo = deltacomputer.finddeltainfo(revinfo, fh)
+
+# no delta for flag processor revision (see "candelta" for why)
+# not calling candelta since only one revision needs test, also to
+# avoid overhead fetching flags again.
+if flags & REVIDX_KNOWN_FLAGS:
+deltainfo = None
+else:
+deltainfo = deltacomputer.finddeltainfo(revinfo, fh)
 
 if deltainfo is not None:
 base = deltainfo.base



To: quark, indygreg, #hg-reviewers, ryanmce
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2067: changegroup: do not delta lfs revisions

[quark (Jun Wu)]

quark updated this revision to Diff 6469.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2067?vs=5650=6469

REVISION DETAIL
  https://phab.mercurial-scm.org/D2067

AFFECTED FILES
  mercurial/changegroup.py
  mercurial/revlog.py
  tests/test-lfs-bundle.t
  tests/test-lfs.t

CHANGE DETAILS

diff --git a/tests/test-lfs.t b/tests/test-lfs.t
--- a/tests/test-lfs.t
+++ b/tests/test-lfs.t
@@ -371,7 +371,7 @@
   uncompressed size of bundle content:
* (changelog) (glob)
* (manifests) (glob)
-   *  a (glob)
+  * a (glob)
   $ hg --config extensions.strip= strip -r 2 --no-backup --force -q
   $ hg -R bundle.hg log -p -T '{rev} {desc}\n' a
   5 branching
diff --git a/tests/test-lfs-bundle.t b/tests/test-lfs-bundle.t
--- a/tests/test-lfs-bundle.t
+++ b/tests/test-lfs-bundle.t
@@ -95,6 +95,6 @@
   2 integrity errors encountered!
   (first damaged changeset appears to be 2)
    Applying src-lfs.bundle to dst-normal 
-  CRASHED
+  OK
    Applying src-lfs.bundle to dst-lfs 
   OK
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -77,6 +77,8 @@
 REVIDX_EXTSTORED,
 ]
 REVIDX_KNOWN_FLAGS = util.bitsfrom(REVIDX_FLAGS_ORDER)
+# bitmark for flags that could cause rawdata content change
+REVIDX_RAWTEXT_CHANGING_FLAGS = REVIDX_ISCENSORED | REVIDX_EXTSTORED
 
 # max size of revlog with inline data
 _maxinline = 131072
@@ -96,7 +98,8 @@
 """Register a flag processor on a revision data flag.
 
 Invariant:
-- Flags need to be defined in REVIDX_KNOWN_FLAGS and REVIDX_FLAGS_ORDER.
+- Flags need to be defined in REVIDX_KNOWN_FLAGS and REVIDX_FLAGS_ORDER,
+  and REVIDX_RAWTEXT_CHANGING_FLAGS if they can alter rawtext.
 - Only one flag processor can be registered on a specific flag.
 - flagprocessors must be 3-tuples of functions (read, write, raw) with the
   following signatures:
@@ -738,6 +741,18 @@
 except KeyError:
 return False
 
+def candelta(self, baserev, rev):
+"""whether two revisions (baserev, rev) can be delta-ed or not"""
+# Disable delta if either rev requires a content-changing flag
+# processor (ex. LFS). This is because such flag processor can alter
+# the rawtext content that the delta will be based on, and two clients
+# could have a same revlog node with different flags (i.e. different
+# rawtext contents) and the delta could be incompatible.
+if ((self.flags(baserev) & REVIDX_RAWTEXT_CHANGING_FLAGS)
+or (self.flags(rev) & REVIDX_RAWTEXT_CHANGING_FLAGS)):
+return False
+return True
+
 def clearcaches(self):
 self._cache = None
 self._chainbasecache.clear()
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -774,6 +774,8 @@
 progress(msgbundling, None)
 
 def deltaparent(self, revlog, rev, p1, p2, prev):
+if not revlog.candelta(prev, rev):
+raise error.ProgrammingError('cg1 should not be used in this case')
 return prev
 
 def revchunk(self, revlog, rev, prev, linknode):
@@ -833,16 +835,19 @@
 # expensive. The revlog caches should have prev cached, meaning
 # less CPU for changegroup generation. There is likely room to add
 # a flag and/or config option to control this behavior.
-return prev
+base = prev
 elif dp == nullrev:
 # revlog is configured to use full snapshot for a reason,
 # stick to full snapshot.
-return nullrev
+base = nullrev
 elif dp not in (p1, p2, prev):
 # Pick prev when we can't be sure remote has the base revision.
 return prev
 else:
-return dp
+base = dp
+if base != nullrev and not revlog.candelta(base, rev):
+base = nullrev
+return base
 
 def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags):
 # Do nothing with flags, it is implicitly 0 in cg1 and cg2



To: quark, indygreg, #hg-reviewers, ryanmce
Cc: martinvonz, durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel